{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPyOpt: external objective function evaluation\n",
    "\n",
    "### Written by Andrei Paleyes, Amazon Research Cambridge\n",
    "\n",
    "\n",
    "*Last updated 12 October 2017*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GPyOpt allows to evaluate objective function externally. If users opt for this approach, they can use GPyOpt to suggest next point for evaluation, and then evaluate the objective function themselves as well as decide on the stopping criteria of the evaluation loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/gojav/anaconda/lib/python3.5/site-packages/matplotlib/__init__.py:1350: UserWarning:  This call to matplotlib.use() has no effect\n",
      "because the backend has already been chosen;\n",
      "matplotlib.use() must be called *before* pylab, matplotlib.pyplot,\n",
      "or matplotlib.backends is imported for the first time.\n",
      "\n",
      "  warnings.warn(_use_error_msg)\n",
      "/Users/gojav/anaconda/lib/python3.5/site-packages/matplotlib/__init__.py:1350: UserWarning:  This call to matplotlib.use() has no effect\n",
      "because the backend has already been chosen;\n",
      "matplotlib.use() must be called *before* pylab, matplotlib.pyplot,\n",
      "or matplotlib.backends is imported for the first time.\n",
      "\n",
      "  warnings.warn(_use_error_msg)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "warning in stationary: failed to import cython module: falling back to numpy\n",
      "warning in stationary: failed to import cython module: falling back to numpy\n"
     ]
    }
   ],
   "source": [
    "%pylab inline\n",
    "import GPyOpt\n",
    "from numpy.random import seed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the purposes of this notebook we are going to use one of the predefined objective functions that come with GPyOpt. However the key thing to realize is that the function could be anything (e.g., the results of a physical experiment). As long as users are able to externally evaluate the suggested points somehow and provide GPyOpt with results, the library has opinions about the objective function's origin."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "func = GPyOpt.objective_examples.experiments1d.forrester() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define the domain of the function to optimize as usual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "domain =[{'name': 'var1', 'type': 'continuous', 'domain': (0,1)}]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we are going to run the optimization loop outside of GPyOpt, and only use GPyOpt to get the next point to evaluate our function.\n",
    "\n",
    "There are two thing to pay attention to when creating the main optimization object:\n",
    "* Objective function f is explicitly set to None\n",
    "* Since we recreate the object anew for each iteration, we need to pass data about all previous iterations to it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_init = np.array([[0.0],[0.5],[1.0]])\n",
    "Y_init = func.f(X_init)\n",
    "\n",
    "iter_count = 10\n",
    "current_iter = 0\n",
    "X_step = X_init\n",
    "Y_step = Y_init\n",
    "\n",
    "while current_iter < iter_count:\n",
    "    bo_step = GPyOpt.methods.BayesianOptimization(f = None, domain = domain, X = X_step, Y = Y_step)\n",
    "    x_next = bo_step.suggest_next_locations()\n",
    "    y_next = func.f(x_next)\n",
    "    \n",
    "    X_step = np.vstack((X_step, x_next))\n",
    "    Y_step = np.vstack((Y_step, y_next))\n",
    "    \n",
    "    current_iter += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's visualize the results. The size of the marker denotes the order in which the point was evaluated - the bigger the marker the later was the evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhIAAAFkCAYAAAB1rtL+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XmYXFWd//H3IQFk0cii7FriBuqIJqAsCZAfDgwwbCpL\nCwHBn6ioj0bFXdl+7uOgM+IgjkFFaYkkCAiCCkGC4pa4MQgqptQQCLIFZQlJ+vv743aNnaY76aq+\nt+691e/X89QDXXXr1rdOqvt+6pxzz00RgSRJUic2KLsASZJUXwYJSZLUMYOEJEnqmEFCkiR1zCAh\nSZI6ZpCQJEkdM0hIkqSOGSQkSVLHDBKSJKljBglJktSxQoNESul9KaWfppQeSiktTyldllJ63gjb\nnZ1SWpZSeiSl9L2U0nOKrEuSJOWj6B6JGcB/Ai8HXgFsCHw3pbRJa4OU0nuAtwCnAi8DHgauTSlt\nVHBtkiRpnFI3L9qVUtoauAfYNyJuGrxvGfCpiDh38OenAMuBkyJibteKkyRJbev2HImnAgHcD5BS\nehawLXBda4OIeAj4CbBXl2uTJEltmtytF0opJeAzwE0Rcevg3duSBYvlwzZfPvjYSPvZCjgIaAKP\nFVKsJEm96UlAA7g2Iu7LY4ddCxLA54EXAPuMcz8HAV8ffzmSJE1YxwMX57GjrgSJlNLngEOAGRFx\n15CH7gYSsA1r90psA/xilN01Ab72ta+x66675l+sRjR79mzOPffcssuYUGzz7rPNu882767f/va3\nnHDCCTB4LM1D4UFiMEQcAewXEX8e+lhELEkp3Q0cAPx6cPunkJ3lcd4ou3wMYNddd2Xq1KmF1a21\nTZkyxfbuMtu8+2zz7rPNS5Pb1IBCg0RK6fNAH3A48HBKaZvBh1ZEROtNfAb4YErpD2QJ6RxgKXB5\nkbVJkqTxK7pH4o1kkylvGHb/ycBXASLikymlTYEvkJ3VsRA4OCIeL7g2SZI0ToUGiYgY0+mlEXEm\ncGaRtUiSpPx5rQ2NSV9fX9klTDi2effZ5t1nm9dfV1e2zENKaSqwaNGiRU7QkSSpDYsXL2batGkA\n0yJicR77tEdCkiR1zCAhSZI6ZpCQJEkdM0hIkqSOGSQkSVLHDBKSJKljBglJktQxg4QkSeqYQUKS\nJHXMICFJkjpmkJAkSR0zSEiSpI4ZJCRJUscMEpIkqWMGCUmS1DGDhCRJ6phBQpIkdcwgIUmSOmaQ\nkCRJHTNISJKkjhkkJElSxwwSkiSpYwYJSZImiGXLluW+T4OEJEkTQLPZ5KxTT819vwYJSZJ6XLPZ\n5JSZMznjrrty3/fk3PcoSZIqoxUi5jSb3F/A/u2RkCSpRw0NEY2CXsMgIUlSD+pGiACDhCRJPadb\nIQIMEpIk9ZRuhggwSEiS1DO6HSLAICFJUk8oI0SAQUKSpNorK0SAQUKSpForM0SAQUKSpNoqO0SA\nQUKSpFqqQogAg4QkSbU0e9Yszig5RIBBQpKkWjr3oos4q9GgWXIdBglJkmqo0WgwZ8ECTik5TBQa\nJFJKM1JKV6SU7kwpDaSUDh/2+IWD9w+9XV1kTZIk9YoqhImieyQ2A34JnAbEKNt8B9gG2Hbw1ldw\nTZIk9Yyyw0ShQSIiromID0fE5UAaZbOVEfHXiLhn8LaiyJokSeo1rTBx0k7dDxNVmCOxf0ppeUrp\ntpTS51NKW5ZdkCRJdbPNNg0e2moBr9igu2Gi7CDxHeBE4P8A7wb2A65OKY3WeyFJkoaJgJNPhttv\nb/Dp+d0d5pjcpdcZUUTMHfLj/6SUfgPcAewPLFjXc2fPns2UKVPWuq+vr4++PqdYSJImlv5+uOQS\nmDsXjjiiwW67LeDQPfZgu3vvZdMh2xUxdyBFjDYHMucXSmkAODIirljPdvcAH4iIL47y+FRg0aJF\ni5g6dWoBlUqSVB8PPAC77AL77ZcFiZaRVr5cDEzL/ndaRCzO4/XLHtpYS0ppR2Ar4K6ya5EkqQ4+\n8AF49FE499y17+/W2RxFryOxWUppt5TSSwbv2nnw550GH/tkSunlKaVnppQOAL4F/A64tsi6JEnq\nBT/5CZx/Pvy//wc77PDEx7sRJorukdgd+AWwiGwdiU+T9aycBawBXgxcDtwOfBH4GbBvRKwquC5J\nkmpt9Wp44xvhpS+F004bfbuhYWJZAXUUOtkyIn7AusPKvxT5+pIk9arPfQ5+9ausV2Lyeo7mrTBx\n9N57w135zh6o1BwJSZK0fkuXwoc+lPVE7LHH2J7TaDQ444ILcq/FICFJUs28612w+ebwkY+097zt\nt98+91pKXUdCkiS155ZbsjUjLrgAhi2nVAp7JCRJqpGzz4ZGA046qexKMvZISJJUE7fcApdeCl/4\nAmy0UdnVZOyRkCSpJs45B57xjOr0RoA9EpIk1cL//A9885vV6o0AeyQkSaqFc86BnXaqVm8E2CMh\nSVLl3XprdkGu88+vVm8E2CMhSVLltXojXvvasit5InskJEmqsN/+Nls34r/+q3q9EWCPhCRJlXbu\nubDddnDyyWVXMjKDhCRJFXX//fC1r8Gb3lTN3ggwSEiSVFlf+hKsWQOnnlp2JaMzSEiSVEFr1sB5\n58Fxx8HTn152NaMzSEiSVEFXXgl/+hO89a1lV7JuBglJkiroP/8T9toLdt+97ErWzdM/JUmqmFtu\ngeuvh4svLruS9bNHQpKkivnc52DbbeFVryq7kvUzSEiSVCEPPAAXXVTtUz6HMkhIklQhc+bAqlXV\nPuVzKIOEJEkVMTCQnfJ5zDHZ0EYdONlSkqSKuP56WLIkW82yLuyRkCSpIubMgV12yU77rAuDhCRJ\nFfDAAzB/PpxyCqRUdjVjZ5CQJKkC+vth9WqYNavsStpjkJAkqQLmzIFDDqnPJMsWg4QkSSX79a9h\n0aJsWKNuDBKSJJXswguzK3weemjZlbTPICFJUokefzxbyXLWLNhww7KraZ9BQpKkEl15Jdx3H5x8\nctmVdMYgIUlSiebMgZe/HF74wrIr6YxBQpKkktx5J1xzTT0nWbYYJCRJKslXvwobbwzHHlt2JZ0z\nSEiSVIKILEgcdRRMmVJ2NZ0zSEiSVIJf/hJuuw1OOKHsSsbHICFJUgkuvhi23hpe8YqyKxkfg4Qk\nSV02MJBdW+OYY+q5dsRQBglJkrrsxhuzMzaOP77sSsbPICFJUpddfDE0GrDXXmVXMn6FBomU0oyU\n0hUppTtTSgMppcNH2ObslNKylNIjKaXvpZSeU2RNkiSVaeVK+OY34TWvgZTKrmb8iu6R2Az4JXAa\nEMMfTCm9B3gLcCrwMuBh4NqU0kYF1yVJUimuuQYefDALEr1gcpE7j4hrgGsAUhoxd70NOCcivj24\nzYnAcuBIYG6RtUmSVIaLL4YXv7i+S2IPV9ociZTSs4Btgeta90XEQ8BPgB4YNZIkaW0PPQRXXNEb\nkyxbypxsuS3ZcMfyYfcvH3xMkqSe8q1vwWOPwXHHlV1JfjxrQ5KkLvn612HffeEZzyi7kvwUOkdi\nPe4GErANa/dKbAP8Yn1Pnj17NlOGLU7e19dHX19fnjVKkpSL5cvh+9+Hz3++O6/X399Pf3//Wvet\nWLEi99dJEU84maIQKaUB4MiIuGLIfcuAT0XEuYM/P4UsVJwYEd8cZT9TgUWLFi1i6tSpXahckqTx\nO+88ePvb4e67Yautyqlh8eLFTJs2DWBaRCzOY5+F9kiklDYDnkPW8wCwc0ppN+D+iPgL8Bnggyml\nPwBN4BxgKXB5kXVJktRtc+dm19UoK0QUpeihjd2BBWSTKgP49OD9XwFOiYhPppQ2Bb4APBVYCBwc\nEY8XXJckSV1z112wcCF86UtlV5K/oteR+AHrmdAZEWcCZxZZhyRJZZo3DyZPhiOPLLuS/HnWhiRJ\nBWsNa2yxRdmV5M8gIUlSgZYtg5tuyi4Z3osMEpIkFag1rHHEEWVXUgyDhCRJBZo7Fw48sDeHNcAg\nIUlSYe68MxvWOProsispjkFCkqSCzJsHG27Yu8MaYJCQJKkwc+fCQQfBU59adiXFMUhIklSApUvh\nhz/s7WENMEhIklSISy+FjTaCww8vu5JiGSQkSSrAN7/Z+8MaYJCQJCl3S5fCj34Er3512ZUUzyAh\nSVLOLrssO1uj14c1wCAhSVLu5s2DAw7o/WENMEhIkpSre+7JLhn+qleVXUl3GCQkScrRt76V/beX\nF6EayiAhSVKO5s2D/faDpz2t7Eq6wyAhSVJOHngArr9+4gxrgEFCkqTcXHEFrF4NRx1VdiXdY5CQ\nJCkn8+bB3nvD9tuXXUn3GCQkScrB3/4G3/0uvPKVZVfSXQYJSZJycNVVsHKlQUKSJHVg3jyYOhWe\n9ayyK+kug4QkSeP0yCNw9dUT62yNFoOEJEnjdO21WZgwSEiSpLbNnw8vfCE8//llV9J9BglJksbh\n8cfhyisnZm8EGCQkSRqXBQtgxYqJd7ZGi0FCkqRxmD8fdt4ZXvzisisph0FCkqQOrVkDl1+eLYmd\nUtnVlMMgIUlSh26+GZYvn7jDGmCQkCSpY/Pnw7bbwp57ll1JeQwSkiR1ICILEkcdBRtM4KPpBH7r\nkiR17he/gD/9aWIPa4BBQpKkjsyfD1tsAfvtV3Yl5TJISJLUgcsug8MOgw03LLuSchkkJElq0223\nwa23OqwBBglJktp22WWw6aZw4IFlV1I+g4QkSW2aPx8OOQQ22aTsSspnkJAkqQ1//jP8/OcOa7QY\nJCRJasNll2UTLA85pOxKqsEgIUlSG+bPh3/+Z5gypexKqqH0IJFSOiOlNDDsdmvZdUmSNNzy5bBw\nocMaQ00uu4BBtwAHAK1rp60usRZJkkZ0+eXZVT6POKLsSqqjKkFidUT8tewiJElal/nzs5Ust966\n7Eqqo/ShjUHPTSndmVK6I6X0tZTSTmUXJEnSUA8+CNdd57DGcFUIEj8GXgscBLwReBZwY0ppszKL\nkiRpqCuvhNWrs6t96h9KH9qIiGuH/HhLSumnwJ+AY4ALR3ve7NmzmTJsymxfXx99fX2F1ClJmtjm\nz4c994Qddii7krHp7++nv79/rftWrFiR++ukiMh9p+M1GCa+FxEfGOGxqcCiRYsWMXXq1O4XJ0ma\ncB5+OJsXcfbZcPrpZVfTucWLFzNt2jSAaRGxOI99VmFoYy0ppc2B5wB3lV2LJEkA11wDjz3m/IiR\nlB4kUkqfSintm1J6Zkppb+AyYBXQv56nSpLUFfPnw267wbOfXXYl1VN6kAB2BC4GbgO+AfwV2DMi\n7iu1KkmSgJUr4dvftjdiNFWYbOnsSElSZV13HTz0kEFiNFXokZAkqbLmz4fnPQ9e+MKyK6kmg4Qk\nSaNYvTpbFvuoo7KlsfVEBglJkkbxgx/AvffC0UeXXUl1GSQkSRrFpZdCowEuWzS62gaJZcuWlV2C\nJKmHrVmTzY949asd1liX2gaJs049lWazWXYZkqQeddNNcM89WZDQ6GobJM646y5OmTnTMCFJKsSl\nl8JOO8HLXlZ2JdVW2yCxPTCn2TRMSJJyNzAA8+bBq17lsMb61DZIADQwTEiS8nfzzXDXXQ5rjEWt\ngwQYJiRJ+bv0UthuO9hrr7Irqb7aBwkwTEiS8jMwkAWJV70KNuiJo2SxeqaJGhgmJEnj97OfwdKl\nDmuMVc8ECTBMSJLG79JL4elPh+nTy66kHnoqSIBhQpLUuYgsSLzylTBpUtnV1EPPBQkwTEiSOrN4\nMTSbDmu0oyeDBGRh4oxmk9mzZpVdiiSpJi69FLbaCvbbr+xK6mNy2QUUpQmc1Wgw56KLyi5FklQD\nEXDJJdmwxuSePTrmryd7JJrAKY0GcxYsoNFolFyNJKkOfvYzWLIEjj227ErqpeeCRBNDhCSpfZdc\nkp2t4bBGe3oqSDQxREiS2jcwAHPnwtFHO6zRrp5priYwa4cGFxkiJEltuvnmbBEqhzXa1xM9Ek3g\noA0bLGEBkyc3Sq5GmriazSZHzZjhadeqnW98A3bYAfbZp+xK6qf2QaJJNpzx5R8sYNKkBgceCPfe\nW3ZV0sTTHFy75e033eQaLqqVNWuy0z6PPtpra3Si1k3W5B9zIvbaq8H3vgf33QcHHwx/+1vZ1UkT\nRytEzGk22Q8XhFO93Hgj3H23wxqdqm2QWMYTJ1Y+73lw7bXwu9/B4YfDo4+WWqI0IQwNEY3B+xoY\nJlQfl1wCz3wmvPzlZVdST7UNEmdtt92IZ2e85CVw1VXw059mi4qsXFlOfdJEMFKIaGlgmFD1rVqV\nDWsceyykVHY19VTbIHHGBReMenbG9OlwxRWwYEH24Vi1qru1qdqcEJiPdYWIlgaGCVXb9ddnQ+IO\na3SutkFi++23X+fjBxwAl10GV18NJ5wAq1d3qTBVWq9NCCwrFI0lRLQ0MEyoui65BJ7zHHjpS8uu\npL5qGyTG4uCDsw/JvHlwyinZgiOauHptQmBZoaidENHSoP7trd6zcmX2hdNhjfHp6SABcNRR8PWv\nZ7fXvz47zUfVl/c37V6bEFhWKOokRLQ0qG97qzddey08+KDDGuMWEbW6AVOBWLRoUbTjoosiNtgg\nYtasiFWr2nqqumzJkiUxs9GIGyBmNhqxZMmSXPa3JLu43xNuS3J6nW4Z7f0U/T7W145jvdWtvdW7\njj464sUvLruK7lq0aFEAAUyNvI7Lee2oW7dOg0RExDe+ETFpUsRxx0U8/njbTy/VkiVL4sjp00v5\n49vN1x5+sBrvQWesB7+6HNzKCkV5hYi6tbd614MPRmy8ccSnPlV2Jd1lkBhnkIiImDcvYvLkiFe+\nMmLlyo520XV5f0Ov6mvn/U273YNf1Q9uZYWivENEXdpbve1LX4pIKWLp0rIr6S6DRA5BIiLiiisi\nNtoo4rDDIh59tOPddEWn39Dz6EXIu3egndca70Gn04NfVQ9uZYWiokJE1dtbvW/mzIgDDii7iu4z\nSOQUJCIivvOdiCc9KWK//bIurirq9Bt6Hr0I3RyHz/ub9ngPflU7uJUViooOEVVtb/W+P/856424\n8MKyK+k+g0SOQSIiYuHCiKc+NZtsc+ed495drjr9hp5HL0I3x+Hz/qbdaxMCywpF3QoRQ+t8wSaN\nOOqoJTF7dsRHPxrxxS9GXHNNxJ/+FDEwUEz7amL6xCeyL5IrVpRdSfcZJCLfIBERccstETvuGPHM\nZ0bcdlsuuxy3Tr+h59GL0M1x+Ly/affahMCyQlG3Q8TQOnfZuBE777wkttoq+8bYenjzzSP22CPi\npJMizj8/4vbbDRfq3D/9U8Sxx5ZdRTkMEpF/kIjIurl23TViq60ifvzj3HbbkU6/oS9cuHDcvQjd\nHIfP+5t2r00ILDMUHTl9etzQ5RDRut0AceT06RGRnab9xz9GfPvb2cz6k0+O2H337MwriNhhh4jj\nj8+6p++/v9h/D/WOX/0q+/xceWXZlZTDIBHFBImIiPvui9h774hNNono789112M2nm/ou06aFAvH\ncTDp5jh83t+01xei6hYmygxFAwMRCxcuiZc9rZweibG080MPRVx1VcS73hUxbVrWc7HhhhFHHBFx\nySURDz+c77+Hesvpp2dfGuu2BEBeDBJRXJCIiHjkkewbDkS8730Ra9bk/hKjyuUb+uB/2/1j3c1x\n+LwPkgvJQlTRB7123uN4FD2sMPx9/PWvEVdfHXHmmRGHHhrx9Ke3Nl0Sz5nU3TkSnbbvnXdGnHtu\nxMtelu1u880j/u//jbj11nz/bVR/q1dnPVlvfnPZlZTHIBFRaJCIyL6RffKT2becf/3X7kzGyfUb\nOu2FiW6OwxfSXT+G91uFg103PwdjeR8v3KwRO+645H/v3nLLiIMOivjQh7LTo++6q55nbfz+9xFn\nnRWx3XbZ7g89NGLBAudTKHPdddnn4uaby66kPD0bJIA3A0uAR4EfA3usY9tCg0TL1VdHTJmSzZ24\n/fbiXqesg+sSiBk77hh77rBDV8bh6x4iijjoFdk+Y3kfuz2lEf/xH0vijjtGP9B2u4ckLytXRnz5\ny9mkOsiGQK66ykAx0Z18csSznz2xPwc9GSSAY4HHgBOBXYAvAPcDW4+yfVeCRER2Fsfznx+x2WbZ\nhK68P3yFjoWP4SC7BGJGzgfjkQ4MvRIiijr4lXmWRDfW5ehWO45kYCDi2muz9WIgW4DoF78o7OVU\nYX//e8STnxxxxhllV1KuXg0SPwY+O+TnBCwF3j3K9l0LEhERf/tblmIhu0ZHXotXdeWb3hgOtkUc\nlIceIHotROR9ECwrRLT7Pso8iyQPAwPZkM0uu2TDliedFPGXv3TlpVURF16Y/dtP9HXPei5IABsC\nq4DDh93/ZeCyUZ7T1SDR0t8f8ZSnRDQaET/60fj21dWx5zEcdIs4OC+hoKGTnOss42C4alU2yfF3\nv4s4YLfyTrVs3YaectmNz22n7ZaHVasiPv/5iKc9LWLTTSM++9nuTqpWeaZPj3jFK8quony9GCS2\nAwaAlw+7/xPAzaM8p5QgEZGd077XXtl57O98Z9Zb0a4yxsJLDRM57bOI+vJ6jy/arBGHHbYkjjkm\nuxjcoYdmf7CmT88WUXrRi7IAuvXW2Wp6a+9iSTSofo9EXp/fdl+vKCtWZDP3IWKffaqzGJ2Kcdtt\n2b91Waf2V0kRQSJFBGVJKW0H3AnsFRE/GXL/J4B9I2KvEZ4zFVi07777MmXKlLUe6+vro6+vr9Ca\nV6+Gf/s3OPts2HprOO88OOywsT232WxyysyZzGk2aRRa5bDXBU4B5sA6X3es2xXx2utzFPB2YL/x\nl5S7HwCve8p0GnssZPJk2HjjtW+bbAJPfjJsvvk/bltsAVtumd0efrjJ+46byYV/KuFz0WgwZ8EC\nGo2xv3Knn+NOX69IN94Ir3sd/OUv2e/0O94BkyeXXZXy9p73wBe/CMuWwZOeVHY13dPf309/f/9a\n961YsYIbb7wRYFpELM7lhfJKJJ3cqNHQxnB33BHxL/+Spdyjjlr/eGslxsLH8I1+rNsV8drdriu3\n91bDuRLjrbvdevNqpyI8/HDWw7jBBtmidH/+c9kVKU+PPx6xzTYRb31r2ZVUQ88NbUQEjDzZ8i/A\n6aNsX4kgEZFN4Lrkkohtt81WxDz99Ih7733idmWHiHYPxkUctPPYZxF1jfs91fDsjbzqHmu9ebdT\nUX74w4iddspWPbz66rKrUV4uvzz7KHq2TqZXg8QxwCOsffrnfcDTRtm+MkGi5cEHs4V8Nt88O73o\nzDP/sZBVVUJEuwfjIg7aeeyziLrKPBgPV7d1G9ZXb1HtVJR774045JCs/Pe+N5ucqXo7/PCIqVPL\nrqI6ejJIRATAaWRDqI8CNwO7r2PbygWJlnvuiXjHOyI23jj7VvO2ty2JfXeqToho92BcxEE7j30W\nUVeZB+Ph6rZuw2j1Ft1ORVmzJuLjH88mVc+Yka3yqXpatiz7dzzvvLIrqY6eDRJtFVzhINGydGnE\nrFlLYudUvRDR7sG4iIN2Hvssoq4yD8bD1W3dhuH1dqudirRwYbbU9jOekV0xUvXz8Y9nZ0o98EDZ\nlVSHQSKiFkGiasMZ4z0YF3HQzmOfRdRV5sG4qM9Rt+pu1XtDl9upSH/5S8RLXpINW151VdnVqB0D\nAxHPe152IUb9g0EiovJBoi4hot2DcREH7RsgjuxS/bm0UwkHx7qt27BkyZI4cvr0nggRLX/7WzbO\nvsEG2QJWAxP4Og11snBh9mtw3XVlV1ItBomISgeJuoWIdg/GeR60q7qvKhyM8/pclV13L1m9OuJd\n78qa9rTTsp9VbSecELHzzq5cOlwRQWKDzleg0FBlLTaVhwbZglGnkM14He9269Mk34WvGsDZwCGT\nJo2rrpE0KX8Rpcbg65/SaIz5/TUpv+5eMmkSfOpTcMEFcP750NcHjz9edlUazfLlMHcuvOlNsIFH\nucLZxDmZPWsWZ9QwRLQ0gDOA2TltN5om2Xm+ea+e+eFGgwtuuKGtg+1Y9luVg3E7YaJJderuNa9/\nPcybB5dfnq1o+/DDZVekkfz3f2cB4pRTyq5kgsira6NbNyo6tFHXYY12hwfGM4ywhIIu5DWk+75u\nExTb1WvrNtTVdddlEzD32ivi/vvLrkZDrVoVscMOEa97XdmVVJNzJKK6QSKivmGiWyEi70uLj3bQ\nrNsExXb12roNdfXTn2brxbzoRdl6BaqGSy/NfiVcyXJkBomodpCIqF+Y6HaIyKud1nfQ7PUJir24\nbkMd3Xpr9u33uc+NuPPOsqtRRMT++2dX3tXIDBJR/SARUZ8wUVaIGG87jfWg2e7+63Yw7sV1G+ro\nD3/IrtHx3Odmi9GpPL/5Tfbr/I1vlF1JdRkkoh5BIqL6YaLsENFpO7V7sB/r/usWIlp6cd2GOrrj\nDsNEFbzxjdlqpCtXll1JdRkkoj5BIqK6YaIqIaLddur0YO8ERXXDHXdky2kbJsrxwAMRm26aXTRR\no3MdiZrp5Pz/ojUZ2xoOY91u1Oe2cfrhWNqp3X2Odf/j2a801M47ww03wMqVsP/+sGxZMa/TbDY5\nasYMms1mMS9QU1/5Sra2x6mnll3JBJRXIunWjRr1SLRUpWeiaj0RY22nvHoMnKCobvjjH7Nhjl13\nza4InCfnxYxszZqsJ+i448qupPoc2qhpkIgoP0xUPUSM1k55H+z9Q6xu+N3vIrbZJrvgV15XnjQI\nj+7KK7M/QzfdVHYl1WeQqHGQiCgvTNQlRAxvp6IO9k5QVDf8+tcRW24Zseee2YW/xsO1Q0Y3MBCx\n997ZbcALqq2XQaLmQSKi+2GinRAxowIhYmg7ebBX3f3sZxFPfnLEzJkRjzzS2T6cLLxuN96YNcWV\nV5ZdST042bIHdHMCZpOxT6w8cccdWbXDDp29RgGTFRuNBpctXOgESNXa7rvDVVfBj38MRx8Nq1a1\n9/yxXAywAcwZ3G4iTsD82MfgRS+CQw4pu5KJyyBRgm6EiSZtnJ3RaPDVhQvpv+kmrzAp5WzGDPjW\nt+C734WTT4aBgbE9r50rCjeYmGHiV7+C73wH3vter/JZJpu+JEWGiSbthYhWEPAKk1IxDjwQvvY1\nuPhimD07S/rNAAAPQ0lEQVQ7G5NYl3ZCREuDiRcmPv5xaDTg2GPLrmRiM0iUqIgw0aSzENFOTaM9\nV9LojjkGzjsP/uM/4CMfGX27TkJES4OJEybuuAPmzoXTT4fJk8uuZoLLa7JFt27UfLLlSPKagNnW\n2RkdLl090Sd2SeN1zjnZr9PnP//Ex3L9W9Djv6dveEPE05/e+STWicqzNno0SETkdDXMnELEaDVN\nhD9OUtEGBiLe9raIlNa+uFTeZ3T18u/rsmURG20U8dGPll1J/XjWRg8bzzBHEzhk0iTOprPhjLHU\n9IM2nytpZCnBv/87vOY1MGsWfP/74xvOGE2D3h3m+Mxn4ElPgtNOK7sSgXMkKqWTMNEkO8BfcMMN\nfLiAeQ2tmj4zfbohQsrJBhvAhRfCAQfAEUc0OX7vfENES4PeCxPLl2dzTU47DaZMKbsagUGicjo9\nc2L64IG+iAtTuaaDlL8NN4RPf7rJTqtn8vW78g8RLQ16K0ycc07Wdu9+d9mVqMUgUUGdnjkx0vNG\n2k5S+ZrNJm85dCbXPF5ciGhpMHKYqNuVRP/wB/jCF+D974cttii7GrUYJCqq00tfO69Bqr4i5kSs\nT4O1w0SrhrffdFNteis++EHYdlt4y1vKrkRDGSQqrNMeBuc1SNVVRohoaZCFib7p0zlxxgzmNJvs\nR/tDH2X0ZPz853DJJXDWWbDJJl17WY2BQaLiOu1hcF6DVD1lhoihNrzzTr66dOn/1tBg7GGijJ6M\nCHjPe+AFL4ATTyz85dQmg0QN2MMg1V8VQkSTbOXbr/LEU8UbrD9MDH0PnfRkdOp734Prr88u0OUq\nltVjkKgJexik+qpSiFjX8vkNRg8HI72HdW2fl4GBrDdin33gsMMKeQmNk0FCkgpUlxDR0uCJ4WBd\n72Gk7fP09a/DL3+ZXaArpdx3rxwYJCSpIHULES0N/hEObhqcC7Gu9zB0+zzDxP33wzvfCa9+NUyf\nnttulTODhCQVoK4hoqUBnN1scur++4/pPTTIP0ycfjo8/nh2xVRVl0FCknJW9xDRev6HgavXrOmo\nJ2O8YWLBApgzBz7xCdhuu3HtSgUzSEhSjnolRIynJ2O8YeKxx+ANb8iGM17/+o52oS4ySEhSTiZ6\niGhpML4w8ZGPQLMJF1yQXeBM1eY/kSTlZPasWZwxwUNES4POwsQtt2RnaLz//bDrruMsQl1hkJCk\nnJx70UWcNYYr9xahSXVCREuD9sLEmjVw6qnw7GfD+96XUxEqnEFCknIyliv3FqFJ9UJES4Oxh4mz\nz4af/AS++EXYeOOcC1FhSg0SKaVmSmlgyG1NSsmrzEuqrW6HiSbVDREtDdYfJq6+OgsS55wDM2YU\nVIgKUXaPRAAfBLYBtgW2A/6z1IokaZy6FSaaVD9EtDRY1/LbcMIJ8K//Cu99b8GFKHdlBwmAv0fE\nXyPinsHbo2UXJEnjVXSYaFKfENHS4Ilh4rHHspUrp0yBr37VszTqqAr/ZO9NKd2bUlqcUnpXSmlS\n2QVJUh6KChNN6hciWhqsHSbe9rbsTI1582CLLbpcjHJRdpD4LHAcsD9wPvB+4BNlFiRJeco7TDSp\nb4hoaZCFiWP2mMkFFzQ57zyYOrWkYjRuuQeJlNLHhk2gHH5bk1J6HkBEfCYiboyIWyLiAuAdwFtT\nShvmXZcklSWvMNGk/iGipQHMvbfJP20+kwMOaJZcjcYjRUS+O0xpK2Cr9Wz2x4hYPcJzXwD8Btgl\nIn4/yv6nAov23XdfpkyZstZjfX199PX1dVa4JBVsPCtfNumdEDFUEzhlMGg1Go2Sq+kt/f399Pf3\nr3XfihUruPHGGwGmRcTiPF4n9yAxHiml44EvA1tHxIpRtpkKLFq0aBFT7QuTVDOdhIkmvRkiWpoY\nJrpl8eLFTJs2DXIMEqXNkUgp7ZlSeltK6cUppWcNhoh/By4aLURIUt21O8zRpLdDBBRzCXJ1T5mT\nLVeSTbS8AbgFeB/waeANJdYkSYUba5ho0vshoqWBYaKuSgsSEfGLiNgrIraMiM0i4kUR8cmIWFVW\nTZLULesLE00mTohoaWCYqKOyT/+UpAlrtDDRZOKFiJYGhom6MUhIUomGh4kmEzdEtDQwTNSJQUKS\nSjY0TLwJOIOJGyJaGhgm6sIgIUkV0AoTD+6wAx+CtheuatI7IaKlgWGiDgwSklQRjUaD/ptuYtUO\nO3AiYw8TTXovRLQ0MExUnUFCkiqkkzAxm86HQ+qgAZzRbDJ71qyyS9EIDBKSVDHtholzgbPGsF1d\nNYGzGg3OveiiskvRCAwSklRB7YSJBtmwxinr2a6Omrh8dtUZJCSpoiZ6mGhiiKgDg4QkVdhEDRNN\nDBF1YZCQpIqbaGGiiSGiTgwSklQDQ8PEa+jdMNHEEFE3BglJqolWmNh89905cccdey5MNDFE1JFB\nQpJqpNFo8N2f/YyvLly43suQN6hPmGhiiKgrg4Qk1dD6LkP+v9tR/TDRxBBRZwYJSaqpXggTTQwR\ndWeQkKQaq3OYaGKI6AUGCUmquTqGiSaGiF5hkJCkHlCnMNHEENFLDBKS1CPqECaaGCJ6jUFCknpI\nlcNEE0NELzJISFKPqWKYaGKI6FUGCUnqQVUKE00MEb3MICFJPaoKYaKJIaLXGSQkqYeVGSaaGCIm\nAoOEJPW4dsLE2cAhkyaNO0w0MURMFAYJSZoAxhImmsCHGw0uuOGG9YaOdWliiJhIDBKSNEGsK0w0\n+cfBf/r06WPqwRjJ0P0YIiYGg4QkTSAjhYkmTzz4j3U4ZKiR9qPeZ5CQpAlmaEj4AaMf/NsJE811\n7Ee9zSAhSRNQKyR8ZnAYY7SD/1jnVhgiJi6DhCRNUI1Gg8sWLlzvwX+scysMEROTQUKStF5jnVuh\niccgIUkak7HOrdDEYpCQJI3ZWOdWaOKYXHYBkqR6ac2tkMAeCUmSNA4GCUmS1DGDhCRJ6phBQpIk\ndcwgoTHp7+8vu4QJxzbvPtu8+2zz+issSKSU3p9S+mFK6eGU0v2jbLNTSumqwW3uTil9MqVkuKkg\nf9m7zzbvPtu8+2zz+ivyoL0hMBf4r5EeHAwMV5OdgroncBLwWuDsAmuSJEk5KixIRMRZEfFZ4Dej\nbHIQsAtwfET8JiKuBT4EvDml5PoWkiTVQJnDCHsCv4mIe4fcdy0wBXhhOSVJkqR2lPnNf1tg+bD7\nlg957FejPO9JAL/97W8LKksjWbFiBYsXLy67jAnFNu8+27z7bPPuGnLsfFJe+0wRMfaNU/oY8J51\nbBLArhHxuyHPOQk4NyK2HLavLwDPiIiDh9y3CfAwcPDgUMdINbwG+PqYi5YkScMdHxEX57Gjdnsk\n/g24cD3b/HGM+7ob2GPYfdsMeWw01wLHk13B9rExvpYkScp6Ihpkx9JctBUkIuI+4L6cXvtm4P0p\npa2HzJM4EFgB3LqeGnJJUZIkTUA/ynNnhc2RSCntBGwJPBOYlFLabfChP0TEw8B3yQLDRSml9wDb\nAecAn4uIVUXVJUmS8tPWHIm2dpzShcCJIzw0MyJuHNxmJ7J1JvYnmxvxZeB9ETFQSFGSJClXhQUJ\nSZLU+1yOWpIkdcwgIUmSOlbJIJFSenNKaUlK6dGU0o9TSsNPEx2+/f4ppUUppcdSSr8bXLtCbWin\nzVNKR6WUvptSuieltCKl9KOU0oHdrLcXtPs5H/K8fVJKq1JKruLTpg7+tmyUUvpISqk5+Pfljyml\n13ap3J7QQZsfn1L65eDFHJellL6UUtpyXc9RJqU0I6V0RUrpzpTSQErp8DE8Z9zHz8oFiZTSscCn\ngTOAl5KtcHltSmnrUbZvAN8GrgN2Az4L/HdK6Z+7UW8vaLfNgX3Jzro5GJgKLACuHHJmjtajgzZv\nPW8K8BXg+4UX2WM6bPNvAjOBk4HnAX3A7QWX2jM6+Hu+D9nn+4vAC4BXAy8DLuhKwfW3GfBL4DSy\nBSLXKbfjZ0RU6gb8GPjskJ8TsBR49yjbfwL49bD7+oGry34vdbm12+aj7OMW4INlv5e63Dpt88HP\n9llkf5gXl/0+6nTr4G/LvwD3A08tu/a63jpo83cCvx9231uAP5f9Xup2AwaAw9ezTS7Hz0r1SKSU\nNgSmkaUjACJ7Z98H9hrlaXvyxG9n165jew3RYZsP30cCnkz2R1fr0Wmbp5ROBp5FFiTUhg7b/DDg\n58B7UkpLU0q3p5Q+lVLK7RoFvazDNr8Z2CmldPDgPrYBjgauKrbaCSuX42elggSwNTCJkS/mte0o\nzxnt4l9PSSltnG95PamTNh/udLIutbk51tXL2m7zlNJzgY+SrY/vOivt6+RzvjMwg+xqxEcCbyPr\naj+voBp7TdttHhE/Ak4ALkkpPQ7cBTxA1iuh/OVy/KxakFDNDF5E7UPA0bH2JeGVk5TSBmQXqjsj\nIu5o3V1iSRPFBmTdw6+JiJ9HxDXAO4CT/JJSjJTSC8jG6c8km391EFkv3BdKLEvrUeZlxEdyL7CG\nf1y8q2UbRr+Q192jbP9QRKzMt7ye1EmbA5BSOo5sEtSrI2JBMeX1pHbb/MnA7sBLUkqtb8MbkI0q\nPQ4cGBE3FFRrr+jkc34XcGdE/H3Ifb8lC3E7AneM+Cy1dNLm7wV+GBH/PvjzLSml04CFKaUPRMTw\nb88an1yOn5XqkYjsGhuLgANa9w2Ovx/A6BcZuXno9oMOHLxf69Fhm5NS6gO+BBw3+E1NY9RBmz8E\nvAh4CdnM6t2A84HbBv//JwWXXHsdfs5/CGyfUtp0yH3PJ+ulWFpQqT2jwzbfFFg97L4BsjMQ7IXL\nXz7Hz7Jnlo4wi/QY4BGy63TsQtaldR/wtMHHPwZ8Zcj2DeBvZLNPn0922svjwCvKfi91uXXQ5q8Z\nbOM3kqXX1u0pZb+XutzabfMRnu9ZGwW3Odm8nz8BlwC7kp32fDtwftnvpS63Dtr8JGDl4N+WZwH7\nAD8FflT2e6nDbfAzuxvZl44B4O2DP+80Snvncvws/Y2P0hinAU3gUbJktPuQxy4Erh+2/b5kyfdR\n4PfArLLfQ91u7bQ52boRa0a4zSn7fdTp1u7nfNhzDRJdaHOytSOuBf4+GCo+CWxc9vuo062DNn8z\n8JvBNl9Ktq7EdmW/jzrcgP0GA8SIf5uLOn560S5JktSxSs2RkCRJ9WKQkCRJHTNISJKkjhkkJElS\nxwwSkiSpYwYJSZLUMYOEJEnqmEFCkiR1zCAhSZI6ZpCQJEkdM0hIkqSO/X8SUZoLEnbPzQAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d426fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.arange(0.0, 1.0, 0.01)\n",
    "y = func.f(x)\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(x, y)\n",
    "for i, (xs, ys) in enumerate(zip(X_step, Y_step)):\n",
    "    plt.plot(xs, ys, 'rD', markersize=10 + 20 * (i+1)/len(X_step))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compare the results, let's now execute the whole loop with GPyOpt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "bo_loop = GPyOpt.methods.BayesianOptimization(f = func.f, domain = domain, X = X_init, Y = Y_init)\n",
    "bo_loop.run_optimization(max_iter=iter_count)\n",
    "X_loop = bo_loop.X\n",
    "Y_loop = bo_loop.Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's print the results of this optimization and compare to the previous external evaluation run. As before, size of the marker corresponds to its evaluation order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhIAAAFkCAYAAAB1rtL+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XmYHFXd9vHvgYQtSESiJIRgAwpEFHACSCALeYJB4kMg\nQoABAiY+oqJeEhURlTcCLgiPBhUU8CUg25iQRUAiASFAIqAwATSyClNAFsISEpAl2/zeP2r6ZTJM\nz0xXV9Wpqr4/19WXTnd11a9Phjl3n3OqypkZIiIiIlFs5rsAERERyS8FCREREYlMQUJEREQiU5AQ\nERGRyBQkREREJDIFCREREYlMQUJEREQiU5AQERGRyBQkREREJDIFCREREYks0SDhnDvbOfd359zr\nzrmVzrm5zrk9OtnuPOfccufcW865O5xzH0myLhEREYlH0iMSw4FfA58CDgN6A7c757Yub+CcOwv4\nGnAacCDwJjDfObdFwrWJiIhIjVyaN+1yzvUDXgJGmNmitueWAxeZ2bS2n7cDVgKnmtnM1IoTERGR\nqqW9RuL9gAGrAJxzuwL9gTvLG5jZ68DfgKEp1yYiIiJV6pXWgZxzDrgYWGRmj7U93Z8wWKzssPnK\nttc6288OwOFAALyTSLEiIiLFtBVQAuab2atx7DC1IAH8BvgYcEiN+zkcuL72ckREROrWScANcewo\nlSDhnLsEGAsMN7MV7V56EXDAjmw6KrEj8HCF3QUA1113HYMHD46/WOnUlClTmDZtmu8y6oraPH1q\n8/SpzdP1+OOPc/LJJ0NbXxqHxINEW4g4ChhpZs+3f83MWpxzLwKjgX+0bb8d4Vkel1bY5TsAgwcP\npqGhIbG6ZVN9+/ZVe6dMbZ4+tXn61ObexLY0INEg4Zz7DdAIjAPedM7t2PbSGjMrf4iLgR845/5N\nmJDOB5YCNyVZm4iIiNQu6RGJLxMupry7w/OTgGsAzOxC59w2wOWEZ3UsBI4ws3UJ1yYiIiI1SjRI\nmFmPTi81sx8CP0yyFhEREYmf7rUhPdLY2Oi7hLqjNk+f2jx9avP8S/XKlnFwzjUAzc3NzVqgIyIi\nUoXFixczZMgQgCFmtjiOfWpEQkRERCJTkBAREZHIFCREREQkMgUJERERiUxBQkRERCJTkBAREZHI\nFCREREQkMgUJERERiUxBQkRERCJTkBAREZHIFCREREQkMgUJERERiUxBQkRERCJTkBAREZHIFCRE\nREQkMgUJERERiUxBQkRERCJTkBAREZHIFCREREQkMgUJERERiUxBQkRERCJTkBAREZHIFCRERETq\nxPLly2Pfp4KEiIhIHQiCgHNPOy32/SpIiIiIFFwQBEweNYqpK1bEvu9ese9RREREMqMcIqYHAasS\n2L9GJERERAqqfYgoJXQMBQkREZECSiNEgIKEiIhI4aQVIkBBQkREpFDSDBGgICEiIlIYaYcIUJAQ\nEREpBB8hAhQkREREcs9XiAAFCRERkVzzGSJAQUJERCS3fIcIUJAQERHJpSyECFCQEBERyaUpEycy\n1XOIAAUJERGRXJp27bWcWyoReK5DQUJERCSHSqUS0xcsYLLnMJFokHDODXfO3eycW+aca3XOjevw\n+lVtz7d/zEuyJhERkaLIQphIekSiD/AIcDpgFbb5M7Aj0L/t0ZhwTSIiIoXhO0wkGiTM7DYz+z9m\ndhPgKmy21sxeNrOX2h5rkqxJRESkaMph4tRB6YeJLKyRONQ5t9I594Rz7jfOuQ/4LkhERCRvdtyx\nxOs7LOCwzdINE76DxJ+BU4D/Ar4DjATmOecqjV6IiIhIB2YwaRI8+WSJn89Jd5qjV0rH6ZSZzWz3\n47+cc/8EngEOBRZ09d4pU6bQt2/fTZ5rbGyksVFLLEREpL40NcGMGTBzJhx1VIl9913AZw84gAGv\nvMI27bZLYu2AM6u0BjLmAznXChxtZjd3s91LwPfN7HcVXm8Ampubm2loaEigUhERkfx47TXYay8Y\nOTIMEmWdXflyMTAk/L9DzGxxHMf3PbWxCefczsAOwArftYiIiOTB978Pb78N06Zt+nxaZ3MkfR2J\nPs65fZ1z+7U9tVvbz4PaXrvQOfcp59yHnXOjgT8CTwHzk6xLRESkCP72N7jsMvjRj2DgwPe+nkaY\nSHpEYn/gYaCZ8DoSPyccWTkX2AjsA9wEPAn8DngQGGFm6xOuS0REJNc2bIAvfxk++Uk4/fTK27UP\nE8sTqCPRxZZmdg9dh5XPJHl8ERGRorrkEnj00XBUolc3vXk5TEw4+GBYEe/qgUytkRAREZHuLV0K\n55wTjkQccEDP3lMqlZh6xRWx16IgISIikjPf/jZsuy38+MfVvW+nnXaKvRav15EQERGR6ixZEl4z\n4ooroMPllLzQiISIiEiOnHcelEpw6qm+KwlpREJERCQnliyBWbPg8sthiy18VxPSiISIiEhOnH8+\n7LJLdkYjQCMSIiIiufCvf8GNN2ZrNAI0IiEiIpIL558PgwZlazQCNCIhIiKSeY89Ft6Q67LLsjUa\nARqREBERybzyaMTnP++7kvfSiISIiEiGPf54eN2I3/42e6MRoBEJERGRTJs2DQYMgEmTfFfSOQUJ\nERGRjFq1Cq67Dr7ylWyORoCChIiISGZdeSVs3Ainnea7ksoUJERERDJo40a49FI44QT40Id8V1OZ\ngoSIiEgG3XILPPccfP3rvivpmoKEiIhIBv361zB0KOy/v+9KuqbTP0VERDJmyRK46y644QbflXRP\nIxIiIiIZc8kl0L8/HHOM70q6pyAhIiKSIa+9Btdem+1TPttTkBAREcmQ6dNh/fpsn/LZnoKEiIhI\nRrS2hqd8HndcOLWRB1psKSIikhF33QUtLeHVLPNCIxIiIiIZMX067LVXeNpnXihIiIiIZMBrr8Gc\nOTB5Mjjnu5qeU5AQERHJgKYm2LABJk70XUl1FCREREQyYPp0GDs2P4ssyxQkREREPPvHP6C5OZzW\nyBsFCREREc+uuiq8w+dnP+u7kuopSIiIiHi0bl14JcuJE6F3b9/VVE9BQkRExKNbboFXX4VJk3xX\nEo2ChIiIiEfTp8OnPgV77+27kmgUJERERDxZtgxuuy2fiyzLFCREREQ8ueYa2HJLOP5435VEpyAh\nIiLigVkYJMaPh759fVcTnYKEiIiIB488Ak88ASef7LuS2ihIiIiIeHDDDdCvHxx2mO9KaqMgISIi\nkrLW1vDeGscdl89rR7SnICEiIpKye+8Nz9g46STfldROQUJERCRlN9wApRIMHeq7ktolGiScc8Od\nczc755Y551qdc+M62eY859xy59xbzrk7nHMfSbImERERn9auhRtvhBNPBOd8V1O7pEck+gCPAKcD\n1vFF59xZwNeA04ADgTeB+c65LRKuS0RExIvbboPVq8MgUQS9kty5md0G3AbgXKe56xvA+Wb2p7Zt\nTgFWAkcDM5OsTURExIcbboB99snvJbE78rZGwjm3K9AfuLP8nJm9DvwNKMCskYiIyKZefx1uvrkY\niyzLfC627E843bGyw/Mr214TEREplD/+Ed55B044wXcl8dFZGyIiIim5/noYMQJ22cV3JfFJdI1E\nN14EHLAjm45K7Ag83N2bp0yZQt8OFydvbGyksbExzhpFRERisXIl/OUv8JvfpHO8pqYmmpqaNnlu\nzZo1sR/Hmb3nZIpEOOdagaPN7OZ2zy0HLjKzaW0/b0cYKk4xsxsr7KcBaG5ubqahoSGFykVERGp3\n6aVwxhnw4ouwww5+ali8eDFDhgwBGGJmi+PYZ6IjEs65PsBHCEceAHZzzu0LrDKzF4CLgR845/4N\nBMD5wFLgpiTrEhERSdvMmeF9NXyFiKQkPbWxP7CAcFGlAT9ve/73wGQzu9A5tw1wOfB+YCFwhJmt\nS7guERGR1KxYAQsXwpVX+q4kfklfR+IeulnQaWY/BH6YZB0iIiI+zZ4NvXrB0Uf7riR+OmtDREQk\nYeVpje23911J/BQkREREErR8OSxaFN4yvIgUJERERBJUntY46ijflSRDQUJERCRBM2fCmDHFnNYA\nBQkREZHELFsWTmtMmOC7kuQoSIiIiCRk9mzo3bu40xqgICEiIpKYmTPh8MPh/e/3XUlyFCREREQS\nsHQp/PWvxZ7WAAUJERGRRMyaBVtsAePG+a4kWQoSIiIiCbjxxuJPa4CChIiISOyWLoX77oNjj/Vd\nSfIUJERERGI2d254tkbRpzVAQUJERCR2s2fD6NHFn9YABQkREZFYvfRSeMvwY47xXUk6FCRERERi\n9Mc/hv9b5ItQtacgISIiEqPZs2HkSPjgB31Xkg4FCRERkZi89hrcdVf9TGuAgoSIiEhsbr4ZNmyA\n8eN9V5IeBQkREZGYzJ4NBx8MO+3ku5L0KEiIiIjE4I034Pbb4XOf811JuhQkREREYnDrrbB2rYKE\niIiIRDB7NjQ0wK67+q4kXQoSIiIiNXrrLZg3r77O1ihTkBAREanR/PlhmFCQEBERkarNmQN77w17\n7um7kvQpSIiIiNRg3Tq45Zb6HI0ABQkREZGaLFgAa9bU39kaZQoSIiIiNZgzB3bbDfbZx3clfihI\niIiIRLRxI9x0U3hJbOd8V+OHgoSIiEhE998PK1fW77QGKEiIiIhENmcO9O8PBx3kuxJ/FCREREQi\nMAuDxPjxsFkd96Z1/NFFRESie/hheO65+p7WAAUJERGRSObMge23h5EjfVfil4KEiIhIBHPnwpFH\nQu/evivxS0FCRESkSk88AY89pmkNUJAQERGp2ty5sM02MGaM70r8U5AQERGp0pw5MHYsbL2170r8\nU5AQERGpwvPPw0MPaVqjTEFCRESkCnPnhgssx471XUk2KEiIiIhUYc4c+PSnoW9f35Vkg/cg4Zyb\n6pxr7fB4zHddIiIiHa1cCQsXalqjvV6+C2izBBgNlO+dtsFjLSIiIp266abwLp9HHeW7kuzISpDY\nYGYv+y5CRESkK3PmhFey7NfPdyXZ4X1qo81HnXPLnHPPOOeuc84N8l2QiIhIe6tXw513alqjoywE\niQeAzwOHA18GdgXudc718VmUiIhIe7fcAhs2hHf7lHd5n9ows/ntflzinPs78BxwHHBVpfdNmTKF\nvh2WzDY2NtLY2JhInSIiUt/mzIGDDoKBA31X0jNNTU00NTVt8tyaNWtiP44zs9h3Wqu2MHGHmX2/\nk9cagObm5mYaGhrSL05EROrOm2+G6yLOOw/OPNN3NdEtXryYIUOGAAwxs8Vx7DMLUxubcM5tC3wE\nWOG7FhEREYDbboN33tH6iM54DxLOuYuccyOccx92zh0MzAXWA03dvFVERCQVc+bAvvvC7rv7riR7\nvAcJYGfgBuAJ4A/Ay8BBZvaq16pERESAtWvhT3/SaEQlWVhsqdWRIiKSWXfeCa+/riBRSRZGJERE\nRDJrzhzYYw/Ye2/flWSTgoSIiEgFGzaEl8UePz68NLa8l4KEiIhIBffcA6+8AhMm+K4kuxQkRERE\nKpg1C0ol0GWLKsttkFi+fLnvEkREpMA2bgzXRxx7rKY1upLbIHHuaacRBIHvMkREpKAWLYKXXgqD\nhFSW2yAxdcUKJo8apTAhIiKJmDULBg2CAw/0XUm25TZI7ARMDwKFCRERiV1rK8yeDccco2mN7uQ2\nSACUUJgQEZH43X8/rFihaY2eyHWQAIUJERGJ36xZMGAADB3qu5Lsy32QAIUJERGJT2trGCSOOQY2\nK0QvmazCNFEJhQkREandgw/C0qWa1uipwgQJUJgQEZHazZoFH/oQDBvmu5J8KFSQAIUJERGJziwM\nEp/7HGy+ue9q8qFwQQIUJkREJJrFiyEINK1RjUIGCQjDxNQgYMrEib5LERGRnJg1C3bYAUaO9F1J\nfvTyXUBSAuDcUonp117ruxQREckBM5gxI5zW6FXY3jF+hRyRCIDJpRLTFyygVCp5rkZERPLgwQeh\npQWOP953JflSuCARoBAhIiLVmzEjPFtD0xrVKVSQCFCIEBGR6rW2wsyZMGGCpjWqVZjmCoCJA0tc\nqxAhIiJVuv/+8CJUmtaoXiGCRAAc3rvEmyygV6+S52pERCRv/vAHGDgQDjnEdyX5k/upjYBwOuPq\nexaw+eYlxoyBV17xXZWIiOTFxo3haZ8TJujeGlHkuskC3l0TMXRoiTvugFdfhSOOgDfe8F2dSL4F\nQcD44cN1UTcpvHvvhRdf1LRGVLkNEst578LKPfaA+fPhqadg3Dh4+22vJYrkVtB2ZdgzFi2q6gqx\nCh+SRzNmwIc/DJ/6lO9K8im3QeLcAQM6PTtjv/3g1lvh738PLyqydq2f+kTyqhwipgcBI+n55eaj\nhg8Rn9avD6c1jj8enPNdTT7lNkhMveKKimdnDBsGN98MCxaEvxzr16dbm0hetQ8RpbbnSnQfJqKG\nDxHf7rornBLXtEZ0uQ0SO+20U5evjx4Nc+fCvHlw8smwYUNKhUlu1fuwfGchoqxE5XAQNXyIZMGM\nGfCRj8AnP+m7kvzKbZDoiSOOCH9JZs+GyZPDC46IdKbeh+W7ChFlJd4bDqKGD5EsWLs2/MKpaY3a\nFDpIAIwfD9dfHz6++MXwNB+R9up9WL4nIaKsxLvts6gtdFUbPkSyYv58WL1a0xq1KnyQgPCX5Pe/\nh6uvhkmTNM2RF2lMNdT7sHw1IaKsRNg+px16KOdVGT6K3p6SL9ddB/vsA5/4hO9K8q0uggSE6yRu\nuCF8TJyoBZhZl8ZUQ70Py0cJEWUlYN7Gjfwfwuu59GT7oren5MuaNeGi/IkTfVeSf3UTJCAcmZg5\nMzzV54QTYN063xXlT9qjBElNNURdE1AUtYSIshIwHZiMwoTkz+zZYR/Q2Oi7kgIws1w9gAbAmpub\nLaqbbzbbYguzI480e/vtyLvxpqWlxY4eNsxaWloSfU9n+xhVKtndYKNKpZr21d0xWsCs3aMlxmNW\nOkalR5zHzoJqP3+P2qftf+uxPSWfRo0yGz3adxXpa25uNsCABourX45rR2k94ggSZmZ//rPZVluZ\njRxptnp1TbtKVZTOPI4A0LHzSaIz6K6Di+OYUTvRonR+cYcIhQnJo+efN3PO7KqrfFeSPgWJGIOE\nmdnChWbvf7/ZPvuYLVtW8+4SF6UzjyMAZGmUoJZj1tqJ5r3zSypE1BImBm9VsoMPbrFPf9ps7Fiz\niRPNpkwx+8lPzH73O7PbbjN77jmz1lbfrSdF8rOfhV8k16zxXUn6FCRiDhJmZkuWmO28s9mHP2z2\nxBOx7DIRUTrzOAJAFkcJ4gxD9RImkg4RtYSJvfuUbOzYFhs3zmzYMLM99zTbYYfwG2N50223NTvg\nALNTTzW77DKzJ59UuJDoPvEJs+OP912FHwoSFn+QMAuHuQYPDv94PfBAbLuNTZTOPI4AkOVRgjjD\nUNHDRFohImqYuBvs6GHD3lP3+vVmzz5r9qc/mV10kdmkSWb772+2+ebhWwcONDvppHB4etWq9NtV\n8unRR8Pfn1tu8V2JHwoSlkyQMDN79VWzgw8223prs6amWHddkyideRwBIA+jBHGGoaKGibRDRLVh\nIko7vv662a23mn3722ZDhoQjF717mx11lNmMGWZvvplce0r+nXlm+KVx3TrflfihIGHJBQkzs7fe\nCr/hgNnZZ5tt3Bj7IaoSpTMfvvPOdtDAgTUFgDyNEnR1zFTWBGQ8TBw9bJjdnXKIKD/uBjs6hfZb\ntsxs2jSzAw8Md73ttmb/8z9mjz0WTxtKcWzYEI5kffWrvivxR0HCkg0SZuG864UXht9y/vu//S3G\nqaUzH07Ph5U7/kHP4yhBZ8dMdU1AhsNEVkckkmq3p582O/dcswEDwkN99rNmCxZoPYWE7rwz/L24\n/37flfhT2CABfBVoAd4GHgAO6GLbRINE2bx5Zn37hmsnnnwy0UO9RyydeRd/xCu9p9rRjCidaxpT\nDV7WBGQsTKxcaTZ/frg6/cgjW+yjvbKzRiKN9lq71uzqq8NFdRBOgdx6qwJFvZs0yWz33ev796CQ\nQQI4HngHOAXYC7gcWAX0q7B9KkHCLDyLY889zfr0CRd0pfHLF+uQfxd/zCu9p9rRjGo61zSmGg4a\nONCG77yzn2/gHsLEmjXht6srrzT75jfNDj/83W/jEA7zH3yw2amnttgB/fyftZF2O7W2hoFq5Miw\nhNGjzR5+OJVDS8b85z9m73uf2dSpvivxq6hB4gHgl+1+dsBS4DsVtk8tSJiZvfFGmGLB7IQTkr14\nVSJD/l38UY/rPT3pXNMYJWghviBU6+eNS2ur2UsvmT34oNmsWWYXXGD2hS+YjRixaWBwLvymNW6c\n2fe/b3bjjeEwf/t1PqmsGemi/ZNon2ra8eabzfbaK2yrU081e+GF1MsQj666Kvy3z9DAoReFCxJA\nb2A9MK7D81cDcyu8J9UgUdbUZLbddmalktl998W//0SH/KvsXKO8p8vOPaYpk7RqrrmObjrL9evN\nXn7Z7KmnwtON580zu+aa8BTHb30rXPA7enQ4Grb11pseYrvtwmH6E04wO+ccs2uvNVu8uOdnKvj6\nPetJu6Rh/Xqz3/zG7IMfNNtmG7Nf/tL/ompJx7BhZocd5rsK/4oYJAYArcCnOjz/M+D+Cu/xEiTM\nwnPahw4Nz2P/1rfC0Yo4+P6mmHTH3EKyIwVx1hpXPYO3Ktn++7fYAQeYffzjYQDt1y+8ml6lt263\nndkee4SjDRMmmJ1xRng2wpw5Zg89FIaPOKbX0h75aiEbIaK9NWvClftgdsgh2b4YndTuiSfCf+ss\nndrvSxJBwplZhdt5Jc85NwBYBgw1s7+1e/5nwAgzG9rJexqA5hEjRtC3b99NXmtsbKQx4Vu5bdgA\n//u/cN550K8fXHopHHlk9P3FcRfGHh2H8C6N06HHx4nynjT21dF44AxgZMz7rcU9wFc+OIyDxy1k\n663hfe+Dbbd997H99vCBD2z62Hrr9OqL6/cuAMZuvjlXbNzIsAqvTy6VmL5gAaVSLUdKxr33whe+\nAC+8EP43/c1vQq9evquSuJ11Fvzud7B8OWy1le9q0tPU1ERTU9Mmz61Zs4Z7770XYIiZLY7lQHEl\nkigPcjS10dEzz5h95jNhyh0/Ptp8a9avOBj3t/0495XGfmuqJ2PfwDsT16m+CxcuTPxeLEl6881w\nhHGzzcKFqc8/77siidO6dWY77mj29a/7riQbCje1YWbQ+WLLF4AzK2yfiSBhFg4zz5hh1r9/OJd9\n5plmr7zSs/dm9fz+pDvqpDr9pPYbtXPNeudZFtfFxzruJ2/tYGb217+aDRoUXvVw3jzf1Uhcbrop\n/LXV2TqhogaJ44C32PT0z1eBD1bYPjNBomz16nDh27bbhqcX/fCHXV/IyleIqKXTjbOjTqrTT2q/\nUTvXvIhyBdXOPmd5P3fntB3Mwi8CY8eGH/W73w0XZ0q+jRtn1tDgu4rsKGSQMDOA0wmnU98G7gf2\n72LbzAWJspdeCs/l33LL8FvN1KnvvT257xBRS6cbZ0edVKffgp9TQCt1rnnR09/L7j5nS0uLHT1s\nWG7bwSw8i+OCC8JF1cOHm61Y4bsiiWr58vDf8dJLfVeSHYUNElUVnOEgUbZ0abgivE+f8Jf42GPD\ny/Q++2w2QkSRw0QL6V+UKu8hoqy7MFGUz9lTCxeG1+rYZZfwjpGSPxdcEJ4p9dprvivJDgUJy0eQ\nKFu92uxXvwovggMttkfv7ISIrISJu+n6xk5RO3TdayOaSu1WtM/ZUy+8YLbffuG05a23+q5GqtHa\nGp5OfdJJvivJFgUJy1eQKHv22RYb2j97IcJ3mIhjH111dKlcn6OAnWsRFk7G6Y03wnn2zTYLL2BV\nz/dpyJOFC8P/VO+803cl2aIgYfkLEllZE5G1MNFCsvf1SLr9i965FmHhZJw2bDD79rfDf/7TTw9/\nlmw7+WSz3XbTlUs7UpCwfAWJvISIWoJBlKmJFtK502hS/w5FDxFlRVg4GbcrrghHJiZMCO8wKtn0\n4otmW2wRXnZeNqUgYfkKEkcPG2Z3ZyAgVPOoJhhECR7tO+G4LojUk44urjBRLyFCKps7N+ykxowJ\n7ygp2fOjH4WLLF991Xcl2aMgYfkKEkUekag1RNTaRlE69DSDixTbnXeGCzCHDjVbtcp3NdLe+vVm\nAweGd8mV90oiSGzWwytpSwSltnsMTC6VCHwX042Ant8Lo5ptN3lPJ/dciNJGlfbVnVr+PaIeU4rp\nv/4L7roLnnoKRoyAFSt8VyRlN90Ey5bB177mu5L6oSCRsDyEiQA/IaKsmjbqbl/dSTO4SLEdcAAs\nXAivvQYjR4Y3hBL/LrkEhg2D/fbzXUn9UJBIQZbDRIDfEFHWkzbq6b66k2ZwkWIbPBjuuQfeeQcO\nPTT8Jiz+LFkCd9+t0Yi0KUikJIthIiAbIaKsqzaqdl+1HCupY0ox7b572Hm98w6MGqUw4dOll8KA\nATB+vO9K6ouCRIqyFCYCshUiyjpro6j7inKssqSOKcW0225hmFi7VmHCl9Wr4Zpr4Etfgi228F1N\nfVGQSFkWwkRANkNEWfs2uqfGfVVzrKDtuSDhY0oxtQ8Thx6qNRNp+/3vYd06OO0035XUHwUJD3yG\niYBsh4iychtdPGxY4h16msFFim3XXd8NE4cdBi+/7Lui+tDaGk5rHHtsOLUh6VKQ8MRHmAioLkSc\n0sNtN9l/zOsY5i5cmEqHnmZwkWLbdVe4805YtQrGjAmH3CVZ8+bB009rkaUvChIepRkmAqoMETvv\nzPqBA6vbf86/yacZXKTYPvpRuOMOeP55OOII+M9/fFdUXGbw05/CwQeHD0mfgoRnaYSJgCqnM0ol\nrlm4kKZFi3SKpEhEn/gEzJ8P//oXjBsHb78d/zGCIGD88OEEQRD/znNi0SK47z44+2xwznc19UlB\nIgOSDBMB1YeIciDQKZIitdl/f7j1VnjgAZgwAdavj2/fQRAwedQozli0iMmjRtVtmPjpT+HjH4ex\nY31XUr8UJDIiiTARED1E9KSuSu8RkXcNHw5//CPcfjtMmhQuDKxVOURMDwJGAtPbfq63MPHoo/Dn\nP8N3vwtup5kHAAAPYklEQVSbqTfzRk2fIXGGiYDaQ0RXdXX3HhF515gxcN11cMMNMGVKOK8fVfsQ\nUWp7rkR9hokLLoBSCY4/3ncl9U1BImPiCBMBPT/jIqD6S1jrFEmR6h13XHiK4q9+BT/+cbR9dBYi\nykrUV5h45hmYORPOPBN69fJdTX1TkMigWsJEQM/PuAiIdglrnSIpEs1XvgLnnw/nnAO//W117+0q\nRJSVqJ8wcdFF0K9fOF0knsV1P/K0HkADYM3NzXHcmj3TWlpabFSpZC3hSGi3jxYIt29p6fa97bcV\nkfS0tpp94xtmzpn94Q89e08tfwuKaPlysy22MPvJT3xXkj/Nzc0GGNBgMfXLGpHIsFruUqlFkiLZ\n5Bz84hdw4okwcSL85S9db9+TkYiOShR7ZOLii2GrreD0031XIqCpjcyr5RRMLZIUyabNNoOrroLR\no8M7VT70UOfbRQkRZSWKGSZWrgzXmpx+OvTt67saAQWJXKhldEGLJEWyqXdvmDUL9t47vAbC009v\n+notIaKsRPHCxPnnh233ne/4rkTKFCRyopbRBS2SFMmmPn3CC1btsEN4iuiKFeHzcYSIshLphokk\nr7b573/D5ZfD974H228f++4lIgWJHKlldEH3kRDJph12CC+lvWEDfOYz8I9/xBciykpsGiaS6uyT\nvtrmD34A/fvr5lxZoyCRMxpdECmeXXaB224LO+LjD4w3RJSVCMNE47BhnDJ8eOydfdJX23zoIZgx\nA849F7beOpZdSkwUJHJIowsixdOnT8DH+4ziz2vjDxHt9V62jGuWLo3c2Xc2mpH01TbN4Kyz4GMf\ng1NOqWlXkgAFCRERz8od8fUrkgsRAeFl86+ByJ19Z1MXaVxt84474K67wht06SqW2aMgISLiUZwL\nKyseg8r33inRs86+s6mL8jRJklfbbG0NRyMOOQSOPLLqt0sKFCRERDzxHSLKSnTd2VeqszxNUmm/\nPd1/V66/Hh55JLxBl3NVvVVSoiAhIuJBVkJEWYnOO/vO6izvt/00SXcq7b8rq1bBt74Fxx4Lw4b1\n8ECSOgUJEZGUZS1ElJV472milUJENfuttP/unHkmrFsX3jFVskvLVkREUpTVEFFW4t31D72d22Tq\nopb9dtz/5FGjujyFfcECmD4dLrsMBgyIeDBJhUYkRERSkvUQ0V7H9Q9x7Re6H5l45x340pfC6Ywv\nfrHGg0niFCRERFKQlxBR3kf79Q9x7LejEpXDxI9/DEEAV1wR3uBMsk3/RCIiCctbiGi/jzj2W0mJ\n94aJJUvCMzS+9z0YPDjmA0oiFCRERBKkENG1Eu+GiWeeCTjtNNh9dzj77IQOKLHTYksRkYQoRPRM\niTBMjB8yikdfX8A995TYcsuEDyqx8Toi4ZwLnHOt7R4bnXO6y7yIFMKUiROZqhDRIyVg7pqA/fqO\nYtCgIKWjShx8T20Y8ANgR6A/MAD4tdeKRERiMu3aazm3VCJIYN8BxQkRZSVgzup47xoqyfMdJAD+\nY2Yvm9lLbY+3fRckIhKHUqnE9AULmBxzmAgoXogoKxH/LcglWVkIEt91zr3inFvsnPu2c25z3wWJ\niMQl7jARUNwQUVZCYSJPfAeJXwInAIcClwHfA37msyARkbjFFSYCih8iykooTORF7EHCOffTDgso\nOz42Ouf2ADCzi83sXjNbYmZXAN8Evu6c6x13XSIiPtUaJgLqJ0SUlVCYyANnZvHu0LkdgB262exZ\nM9vQyXs/BvwT2MvMnq6w/wagecSIEfTt23eT1xobG2lsbIxWuIhICqKcEhpQfyGivQCY3BbEKt2b\nQ96rqamJpqamTZ5bs2YN9957L8AQM1scx3FiDxK1cM6dBFwN9DOzNRW2aQCam5ubaWhoSLM8EZFY\nVBMmAuo7RJQFKEzEYfHixQwZMgRiDBLe1kg45w5yzn3DObePc27XthDxC+DaSiFCRKQIejrNEaAQ\nUVZC0xxZ5XOx5VrChZZ3A0uAs4GfA1/yWJOISCq6CxMBChEdlVCYyCJvQcLMHjazoWb2ATPrY2Yf\nN7MLzWy9r5pERNJUKUwEKERUUkJhImt8n/4pIlLXOoaJAIWI7pRQmMgSBQkREc/ah4mvAFNRiOhO\nCYWJrFCQEBHJgHKYWD1wIOdAbNea6Oy5oiihMJEFChIiIhlRKpVoWrSI9QMHcgrVhYmAzgPDFGob\n4ci6EjA1CJgycaLvUuqWgoSISIZECRMBlUcdpgHn9nA/eRQA55ZKTLv2Wt+l1C0FCRGRjKk2THQ1\n6lAiDBiTe7CfvAnQRaqyQEFCRCSDqgkT3Y06lChemAhQiMgKBQkRkYzqaZgo0X1Q6Mk2eRGgEJEl\nChIiIhmmMLGpAIWIrFGQEBHJOIWJUIBCRBYpSIiI5EC9h4kAhYisUpAQEcmJeg0TAQoRWaYgISKS\nI/UWJgIUIrJOQUJEJGfah4kTKW6YCFCIyAMFCRGRHCqHiW33359Tdt65cGEiQCEiLxQkRERyqlQq\ncfuDD3LNwoX//zbknW5HvsJEgEJEnihIiIjkXPvbkAeVtiEfYSJAISJvFCRERAqgCGEiQCEijxQk\nREQKIs9hIkAhIq8UJERECiSPYSJAISLPFCRERAomT2EiQCEi7xQkREQKKA9hIkAhoggUJERECqqa\nMFHrVTKrFaAQURQKEiIiBdaTMAGEl9yu8cJWPRWgEFEkChIiIgXXVZgICDv1pkWLYrmwVXfKx1OI\nKA4FCRGROtBZmAjYtFOPa11FJR2PJ8WgICEiUifaB4V76LxTTypMBBWOJ/mnICEiUkfKQeHiYcMq\ndupxh4kAhYgiU5AQEakzpVKJuQsXdtmpx3XGR4BCRNEpSIiISKdqPeMjQCGiHihIiIhIRVHP+Ci/\nphBRfAoSIiLSpWrP+Ki0kFOKSUFCRES6Vc0ZH10t5JTiUZAQEZEe6ekZH90t5JRi6eW7ABERyY9y\nUBAp04iEiIiIRKYgISIiIpEpSIiIiEhkChIiIiISmYKEiIiIRKYgIT3S1NTku4S6ozZPn9o8fWrz\n/EssSDjnvuec+6tz7k3n3KoK2wxyzt3ats2LzrkLnXMKNxmk/9jTpzZPn9o8fWrz/Euy0+4NzAR+\n29mLbYFhHuG1LA4CTgU+D5yXYE0iIiISo8SChJmda2a/BP5ZYZPDgb2Ak8zsn2Y2HzgH+KpzThfK\nEhERyQGf0wgHAf80s1faPTcf6Avs7ackERERqYbPb/79gZUdnlvZ7rVHK7xvK4DHH388obKkM2vW\nrGHx4sW+y6gravP0qc3TpzZPV7u+c6u49unMrOcbO/dT4KwuNjFgsJk91e49pwLTzOwDHfZ1ObCL\nmR3R7rmtgTeBI9qmOjqr4UTg+h4XLSIiIh2dZGY3xLGjakck/he4qpttnu3hvl4EDujw3I7tXqtk\nPnASEADv9PBYIiIiEo5ElAj70lhUFSTM7FXg1ZiOfT/wPedcv3brJMYAa4DHuqkhlhQlIiJSh+6L\nc2eJrZFwzg0CPgB8GNjcObdv20v/NrM3gdsJA8O1zrmzgAHA+cAlZrY+qbpEREQkPlWtkahqx85d\nBZzSyUujzOzetm0GEV5n4lDCtRFXA2ebWWsiRYmIiEisEgsSIiIiUny6HLWIiIhEpiAhIiIikWUy\nSDjnvuqca3HOve2ce8A51/E00Y7bH+qca3bOveOce6rt2hVShWra3Dk33jl3u3PuJefcGufcfc65\nMWnWWwTV/p63e98hzrn1zjldxadKEf62bOGc+7FzLmj7+/Ksc+7zKZVbCBHa/CTn3CNtN3Nc7py7\n0jn3ga7eIyHn3HDn3M3OuWXOuVbn3LgevKfm/jNzQcI5dzzwc2Aq8EnCK1zOd871q7B9CfgTcCew\nL/BL4P865z6dRr1FUG2bAyMIz7o5AmgAFgC3tDszR7oRoc3L7+sL/B74S+JFFkzENr8RGAVMAvYA\nGoEnEy61MCL8PT+E8Pf7d8DHgGOBA4ErUik4//oAjwCnE14gskux9Z9mlqkH8ADwy3Y/O2Ap8J0K\n2/8M+EeH55qAeb4/S14e1bZ5hX0sAX7g+7Pk5RG1zdt+t88l/MO82PfnyNMjwt+WzwCrgPf7rj2v\njwht/i3g6Q7PfQ143vdnydsDaAXGdbNNLP1npkYknHO9gSGE6QgACz/ZX4ChFd52EO/9dja/i+2l\nnYht3nEfDngf4R9d6UbUNnfOTQJ2JQwSUoWIbX4k8BBwlnNuqXPuSefcRc652O5RUGQR2/x+YJBz\n7oi2fewITABuTbbauhVL/5mpIAH0Azan85t59a/wnko3/9rOObdlvOUVUpQ27+hMwiG1mTHWVWRV\nt7lz7qPATwivj6/rrFQvyu/5bsBwwrsRHw18g3Co/dKEaiyaqtvczO4DTgZmOOfWASuA1whHJSR+\nsfSfWQsSkjNtN1E7B5hgm94SXmLinNuM8EZ1U83smfLTHkuqF5sRDg+faGYPmdltwDeBU/UlJRnO\nuY8RztP/kHD91eGEo3CXeyxLuuHzNuKdeQXYyLs37yrbkco38nqxwvavm9naeMsrpChtDoBz7gTC\nRVDHmtmCZMorpGrb/H3A/sB+zrnyt+HNCGeV1gFjzOzuhGotiii/5yuAZWb2n3bPPU4Y4nYGnun0\nXVIWpc2/C/zVzH7R9vMS59zpwELn3PfNrOO3Z6lNLP1npkYkLLzHRjMwuvxc2/z7aCrfZOT+9tu3\nGdP2vHQjYpvjnGsErgROaPumJj0Uoc1fBz4O7Ee4snpf4DLgibb//7eES869iL/nfwV2cs5t0+65\nPQlHKZYmVGphRGzzbYANHZ5rJTwDQaNw8Yun//S9srSTVaTHAW8R3qdjL8IhrVeBD7a9/lPg9+22\nLwFvEK4+3ZPwtJd1wGG+P0teHhHa/MS2Nv4yYXotP7bz/Vny8qi2zTt5v87aSLjNCdf9PAfMAAYT\nnvb8JHCZ78+Sl0eENj8VWNv2t2VX4BDg78B9vj9LHh5tv7P7En7paAXOaPt5UIX2jqX/9P7BKzTG\n6UAAvE2YjPZv99pVwF0dth9BmHzfBp4GJvr+DHl7VNPmhNeN2NjJY7rvz5GnR7W/5x3eqyCRQpsT\nXjtiPvCftlBxIbCl78+Rp0eENv8q8M+2Nl9KeF2JAb4/Rx4ewMi2ANHp3+ak+k/dtEtEREQiy9Qa\nCREREckXBQkRERGJTEFCREREIlOQEBERkcgUJERERCQyBQkRERGJTEFCREREIlOQEBERkcgUJERE\nRCQyBQkRERGJTEFCREREIvt/VY7urrugNaMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1106a9978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "plt.plot(x, y)\n",
    "for i, (xl, yl) in enumerate(zip(X_loop, Y_loop)):\n",
    "    plt.plot(xl, yl, 'rD', markersize=10 + 20 * (i+1)/len(X_step))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To allow even more control over the execution, this API allows to specify points that should be ignored (say the objetive is known to fail in certain locations), as well as points that are already pending evaluation (say in case the user is running several candidates in parallel). Here is how one can provide this information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.78428913]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pending_X = np.array([[0.75]])\n",
    "ignored_X = np.array([[0.15], [0.85]])\n",
    "bo = GPyOpt.methods.BayesianOptimization(f = None, domain = domain, X = X_step, Y = Y_step, de_duplication = True)\n",
    "bo.suggest_next_locations(pending_X = pending_X, ignored_X = ignored_X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Few things to note here:\n",
    "* It is required to set `de_duplication` flag of the optimization object to `True` to enable treatment of pending and ignored locations.\n",
    "* At the moment library treats ignored and pending the same way - does not suggest them as candidates. There are plans to leverage their semantic difference to further optimize the library's behavior."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}