{ "cells": [ { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import GPy\n", "from GPy.inference.optimization import Optimizer\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import climin" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In Paramz/GPy we can implement an own optimizer in a really simple way. We need to supply GPy with an implementation of the Optimizer class.\n", "The Optimizer has a name, which is the most important. \n", "\n", "It also provides an opt() method, which opitmizes the result given an optimization function and a starting point." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Get the parameters for Rprop of climin:\n", "climin.Rprop?" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class RProp(Optimizer):\n", " # We want the optimizer to know some things in the Optimizer implementation:\n", " def __init__(self, step_shrink=0.5, step_grow=1.2, min_step=1e-06, max_step=1, changes_max=0.1, *args, **kwargs):\n", " super(RProp, self).__init__(*args, **kwargs)\n", " self.opt_name = 'RProp (climin)'\n", " self.step_shrink = step_shrink\n", " self.step_grow = step_grow\n", " self.min_step = min_step\n", " self.max_step = max_step\n", " self.changes_max = changes_max\n", " \n", " def opt(self, x_init, f_fp=None, f=None, fp=None):\n", " # We only need the gradient of the \n", " assert not fp is None\n", "\n", " # Do the optimization, giving previously stored parameters\n", " opt = climin.rprop.Rprop(x_init, fp, \n", " step_shrink=self.step_shrink, step_grow=self.step_grow, \n", " min_step=self.min_step, max_step=self.max_step, \n", " changes_max=self.changes_max)\n", "\n", " # Get the optimized state and transform it into Paramz readable format by setting\n", " # values on this object:\n", " # Important ones are x_opt and status:\n", " for info in opt:\n", " if info['n_iter']>=self.max_iters:\n", " self.x_opt = opt.wrt\n", " self.status = 'maximum number of function evaluations exceeded'\n", " break" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is all we need, GPy/Paramz will handle the rest for you : )" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": true }, "outputs": [], "source": [ "m = GPy.examples.regression.toy_rbf_1d_50(optimize=False, plot=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is the model plot before optimization:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n",
"Model: GP regression
\n",
"Objective: 71.7157240818
\n",
"Number of Parameters: 3
\n",
"Number of Optimization Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | value | constraints | priors |
---|---|---|---|
rbf.variance | 1.0 | +ve | |
rbf.lengthscale | 1.0 | +ve | |
Gaussian_noise.variance | 1.0 | +ve |
\n",
"Model: GP regression
\n",
"Objective: -15.5366629083
\n",
"Number of Parameters: 3
\n",
"Number of Optimization Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | value | constraints | priors |
---|---|---|---|
rbf.variance | 2.09912029816 | +ve | |
rbf.lengthscale | 0.275423851372 | +ve | |
Gaussian_noise.variance | 0.0106782996505 | +ve |