{ "metadata": { "name": "", "signature": "sha256:9776069c5f71fd8190f4aaa3e0428598ee879422ea3dafa8ba88297db9dfa1ae" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction to GPy: Gaussian Process Regression in GPy\n", "\n", "## Machine Learning Summer School, Sydney, Australia\n", "\n", "### 23rd February 2015\n", "\n", "### Neil D. Lawrence and Nicolas Durrande\n" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%matplotlib inline\n", "import numpy as np\n", "import pods\n", "import pylab as plt\n", "import GPy\n", "from IPython.display import display" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stderr", "text": [ "/Users/neil/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/pytz/__init__.py:29: UserWarning: Module pods was already imported from /Users/neil/sods/ods/pods/__init__.pyc, but /Users/neil/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages is being added to sys.path\n", " from pkg_resources import resource_stream\n" ] } ], "prompt_number": 1 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Covariance Function Parameter Estimation\n", "\n", "In this session we are going to optimize the parameters of the Gaussian process using gradient based optimization approaches. These maximize the likelihood function: which is defined as the probability of the model given the parameters, $p(\\mathbf{y}|\\mathbf{X}, \\boldsymbol{\\theta})$. \n", "\n", "First we'll load in the olympic marathon data." ] }, { "cell_type": "code", "collapsed": false, "input": [ "data = pods.datasets.olympic_marathon_men()\n", "x = data['X']\n", "y = data['Y']" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 2 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we'll construct a Gaussian process model with an exponentiated quadratic covariance function." ] }, { "cell_type": "code", "collapsed": false, "input": [ "k = GPy.kern.RBF(1)\n", "model = GPy.models.GPRegression(x, y, k)\n", "display(model)\n", "model.plot()" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "\n", "\n", "
\n",
"Model: GP regression
\n",
"Log-likelihood: -118.821194703
\n",
"Number of Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
rbf.variance | 1.0 | +ve | ||
rbf.lengthscale | 1.0 | +ve | ||
Gaussian_noise.variance | 1.0 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -6.94713791215
\n",
"Number of Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
rbf.variance | 25.3995048249 | +ve | ||
rbf.lengthscale | 152.045313 | +ve | ||
Gaussian_noise.variance | 0.048506481396 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -5.99078279431
\n",
"Number of Parameters: 6
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
add.rbf.variance | 2.58425103654 | +ve | ||
add.rbf.lengthscale | 102.447254296 | +ve | ||
add.Mat52.variance | 0.0204066240251 | +ve | ||
add.Mat52.lengthscale | 6.52778209539 | +ve | ||
add.bias.variance | 17.5408872485 | +ve | ||
Gaussian_noise.variance | 0.0368133074589 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -2.10275061954
\n",
"Number of Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
rbf.variance | 0.00025506380619 | +ve | ||
rbf.lengthscale | 10.0 | +ve | ||
Gaussian_noise.variance | 0.0806582576232 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -2.12547857371
\n",
"Number of Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
rbf.variance | 0.000255972082804 | +ve | ||
rbf.lengthscale | 3162.27766017 | +ve | ||
Gaussian_noise.variance | 0.0809454799079 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -1.41450534969
\n",
"Number of Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
rbf.variance | 0.0671183959588 | +ve | ||
rbf.lengthscale | 10.0 | +ve | ||
Gaussian_noise.variance | 0.00671183959588 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -4.33641210151
\n",
"Number of Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
rbf.variance | 0.779949847367 | +ve | ||
rbf.lengthscale | 3162.27766017 | +ve | ||
Gaussian_noise.variance | 0.0779949847367 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -16.7147336689
\n",
"Number of Parameters: 3
\n",
"Updates: True
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
rbf.variance | 1.0 | +ve | ||
rbf.lengthscale | 1.0 | +ve | ||
Gaussian_noise.variance | 1.0 | +ve |
\n",
"Model: GP regression
\n",
"Log-likelihood: -74285.8774977
\n",
"Number of Parameters: 5
\n",
"
GP_regression. | \n", "Value | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
add.rbf.variance | 1.0 | +ve | ||
add.rbf.lengthscale | (2,) | +ve | ||
add.bias.variance | 1.0 | +ve | ||
Gaussian_noise.variance | 1e-05 | fixed |
Index | \n", "GP_regression.add.rbf.lengthscale | \n", "Constraint | \n", "Prior | \n", "Tied to | \n", "
---|---|---|---|---|
[0] | 28.9811652911 | +ve | N/A | |
[1] | 303.000201018 | +ve | N/A |