{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# OPTaaS - Advanced Options\n",
    "\n",
    "### <span style=\"color:red\">Note:</span> To run this notebook, you need an API Key. You can get one <a href=\"mailto:info@mindfoundry.ai\">here</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to the OPTaaS server using your API Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindfoundry.optaas.client.client import OPTaaSClient\n",
    "\n",
    "client = OPTaaSClient('https://optaas.mindfoundry.ai', '<Your OPTaaS API key>')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Store additional data in a Task\n",
    "\n",
    "This can be a JSON, array, string, number or boolean:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'description': 'Lorem ipsum...', 'tags': ['abc', 'defg']}\n"
     ]
    }
   ],
   "source": [
    "from mindfoundry.optaas.client.parameter import FloatParameter\n",
    "\n",
    "task = client.create_task(\n",
    "    title='My Task with User-defined Data', \n",
    "    parameters=[\n",
    "        FloatParameter('x', minimum=0, maximum=5),\n",
    "        FloatParameter('y', minimum=1, maximum=5),\n",
    "    ],\n",
    "    user_defined_data={\n",
    "        'description': 'Lorem ipsum...',\n",
    "        'tags': ['abc', 'defg']\n",
    "    }\n",
    ")\n",
    "\n",
    "print(task.user_defined_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Warm start\n",
    "\n",
    "If you've already tried some configurations, you can record the scores upfront, thus giving the optimizer a \"warm start\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{ 'configuration': {'type': 'user-defined', 'values': {'x': 0, 'y': 1}},\n",
       "   'score': 0.0,\n",
       "   'user_defined_data': None},\n",
       " { 'configuration': {'type': 'user-defined', 'values': {'x': 1, 'y': 2}},\n",
       "   'score': 1.5,\n",
       "   'user_defined_data': None},\n",
       " { 'configuration': {'type': 'user-defined', 'values': {'x': 0.5, 'y': 1.5}},\n",
       "   'score': 0.4166666666666667,\n",
       "   'user_defined_data': None}]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def scoring_function(x, y):\n",
    "    return (x * y) - (x / y)\n",
    "\n",
    "warm_start_values = [\n",
    "    {'x': 0, 'y': 1},\n",
    "    {'x': 1, 'y': 2},\n",
    "    {'x': 0.5, 'y': 1.5},\n",
    "]\n",
    "for values in warm_start_values:\n",
    "    task.add_user_defined_configuration(values, scoring_function(**values))\n",
    "\n",
    "task.get_results()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Store additional data in a Result\n",
    "\n",
    "This can be a JSON, array, string, number or boolean:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{ 'configuration': {'type': 'default', 'values': {'x': 2.5, 'y': 3.0}},\n",
       "   'score': 6.666666666666667,\n",
       "   'user_defined_data': {'Any data': ['you', 'like']}}]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "configuration = task.generate_configurations()[0]\n",
    "score = scoring_function(**configuration.values)\n",
    "task.record_result(configuration, score, user_defined_data={'Any data': ['you', 'like']})\n",
    "\n",
    "task.get_results()[-1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Report an error\n",
    "\n",
    "If you encountered an error while calculating the score for a configuration, you can report it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score: None  Unexpected error: code 12345\n"
     ]
    }
   ],
   "source": [
    "error_configuration = task.generate_configurations()[0]\n",
    "next_configuration = task.record_result(error_configuration, \n",
    "                                        error='Unexpected error: code 12345')\n",
    "error_result = task.get_results()[-1]\n",
    "print(f'Score: {error_result.score}  {error_result.error}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get best result and configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{ 'configuration': {'type': 'default', 'values': {'x': 2.5, 'y': 3.0}},\n",
       "  'score': 6.666666666666667,\n",
       "  'user_defined_data': {'Any data': ['you', 'like']}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "task.get_best_result_and_configuration()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get top N results and configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{ 'configuration': {'type': 'default', 'values': {'x': 2.5, 'y': 3.0}},\n",
       "   'score': 6.666666666666667,\n",
       "   'user_defined_data': {'Any data': ['you', 'like']}},\n",
       " { 'configuration': {'type': 'user-defined', 'values': {'x': 1, 'y': 2}},\n",
       "   'score': 1.5,\n",
       "   'user_defined_data': None},\n",
       " { 'configuration': {'type': 'user-defined', 'values': {'x': 0.5, 'y': 1.5}},\n",
       "   'score': 0.4166666666666667,\n",
       "   'user_defined_data': None}]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "task.get_results(limit=3, best_first=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get results and configurations as a Pandas DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>config.x</th>\n",
       "      <th>config.y</th>\n",
       "      <th>error</th>\n",
       "      <th>score</th>\n",
       "      <th>variance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00</td>\n",
       "      <td>1.0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.00</td>\n",
       "      <td>2.0</td>\n",
       "      <td>None</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.5</td>\n",
       "      <td>None</td>\n",
       "      <td>0.416667</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2.50</td>\n",
       "      <td>3.0</td>\n",
       "      <td>None</td>\n",
       "      <td>6.666667</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3.75</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Unexpected error: code 12345</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   config.x  config.y                         error     score  variance\n",
       "0      0.00       1.0                          None  0.000000       0.0\n",
       "1      1.00       2.0                          None  1.500000       0.0\n",
       "2      0.50       1.5                          None  0.416667       0.0\n",
       "3      2.50       3.0                          None  6.666667       0.0\n",
       "4      3.75       2.0  Unexpected error: code 12345       NaN       NaN"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "task.get_results(as_dataframe=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resume a completed task\n",
    "Completing a task means that no further configurations can be generated and no further results can be recorded for it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Status: 400  Message: Cannot add configurations to a completed task\n"
     ]
    }
   ],
   "source": [
    "task.complete()\n",
    "try:\n",
    "    task.generate_configurations()\n",
    "except Exception as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, you can resume a completed task if necessary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'type': 'exploitation', 'values': {'x': 4.375, 'y': 4.5}}]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "task.resume()\n",
    "task.generate_configurations()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## New client version available\n",
    "You will be notified with a UserWarning if a new OPTaaS Client version is available. It will look like this:\n",
    "```\n",
    "A new version of the OPTaaS client is available. Please run:\n",
    "    pip install mindfoundry-optaas-client==1.2.8\n",
    "To stop these messages, use OPTaaSClient(url, api_key, disable_version_check=True)\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "nav_menu": {},
  "toc": {
   "navigate_menu": true,
   "number_sections": false,
   "sideBar": true,
   "threshold": 6,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}