{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPyOpt: mixing different types of variables\n",
    "\n",
    "### Javier Gonzalez, Amazon Research Cambridge.\n",
    "\n",
    "\n",
    "*Last updated Monday, July 2017*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we are going to see how to used GPyOpt to solve optimizaiton problems in which the domain of the fucntion is defined in terms of a variety of continous and discrete variables. To this end we start by loading GPyOpt. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%pylab inline  \n",
    "import GPyOpt\n",
    "from numpy.random import seed\n",
    "import numpy as np\n",
    "seed(123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use the **Alpine1** function, that it is available in the benchmark of functions of the package. This function is defined for arbitrary dimension. In this example we will work in dimension 9. The functional form of the Alpine1 function is:\n",
    "\n",
    "$$f(x_1,x_2,...,x_9)=\\sum_{i=1}^{9} \\lvert {x_i \\sin \\left( x_i \\right) + 0.1 x_i} \\rvert$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the function from GPyOpt, assuming that noisy free values will be sampled."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "func  = GPyOpt.objective_examples.experimentsNd.alpine1(input_dim=9) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will consider that variables $x_1$, $x_2$ and $x_3$ are continuous and defined in the interval $[-5,5]$, variable $x_9$ is takes continuous values in the interval $[-1,2]$, variable $x_3$ is discrete and takes values $\\{3,8,10\\}$, and finally $x_4$ is categorical with three different categories. Next we define this domain to use it in GPyOpt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "mixed_domain =[{'name': 'var1', 'type': 'continuous', 'domain': (-5,5),'dimensionality': 3},\n",
    "               {'name': 'var3', 'type': 'discrete', 'domain': (3,8,10),'dimensionality': 2},\n",
    "               {'name': 'var4', 'type': 'categorical', 'domain': (0,1,2),'dimensionality': 1},\n",
    "               {'name': 'var5', 'type': 'continuous', 'domain': (-1,2)}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "myBopt = GPyOpt.methods.BayesianOptimization(f=func.f,                     # Objective function       \n",
    "                                             domain=mixed_domain,          # Box-constraints of the problem\n",
    "                                             initial_design_numdata = 5,   # Number data initial design\n",
    "                                             acquisition_type='EI',        # Expected Improvement\n",
    "                                             exact_feval = True)           # True evaluations, no sample noise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we run the optimization for 20 iterations or a maximum of 60 seconds and we show the convergence plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "max_iter = 20       ## maximum number of iterations\n",
    "max_time = 60       ## maximum allowed time\n",
    "eps      = 1e-6     ## tolerance, max distance between consicutive evaluations.\n",
    "\n",
    "myBopt.run_optimization(max_iter,eps=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "np.round(myBopt.X,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "myBopt.plot_convergence()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}