{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"#configure plotting\n",
"%matplotlib inline\n",
"%config InlineBackend.figure_format = 'svg'\n",
"import matplotlib;matplotlib.rcParams['figure.figsize'] = (8,5)\n",
"from matplotlib import pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial : A kernel overview\n",
"\n",
"### Nicolas Durrande and James Hensman, 2013, 2014\n",
"The aim of this tutorial is to give a better understanding of the kernel objects in GPy and to list the ones that are already implemented.\n",
"\n",
"First we import the libraries we will need"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import GPy\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## basics: construction, printing and plotting\n",
"For most kernels, the input dimension (domain) is the only mandatory parameter to define a kernel object. However, it is also possible to specify the values of the parameters. For example, the three following commands are valid for defining a squared exponential kernel (ie rbf or Gaussian)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"ker1 = GPy.kern.RBF(1) # Equivalent to ker1 = GPy.kern.rbf(input_dim=1, variance=1., lengthscale=1.)\n",
"ker2 = GPy.kern.RBF(input_dim=1, variance = .75, lengthscale=2.)\n",
"ker3 = GPy.kern.RBF(1, .5, .5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A `print` and a `plot` function are implemented to represent kernel objects."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" \u001b[1mrbf. \u001b[0;0m | value | constraints | priors\n",
" \u001b[1mvariance \u001b[0;0m | 0.75 | +ve | \n",
" \u001b[1mlengthscale\u001b[0;0m | 2.0 | +ve | \n"
]
}
],
"source": [
"print ker2"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"_ = ker1.plot(ax=plt.gca())\n",
"_ = ker2.plot(ax=plt.gca())\n",
"_ = ker3.plot(ax=plt.gca())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Implemented kernels\n",
"\n",
"Many kernels are already implemented in GPy. The following figure gives a summary of some of them (a comprehensive list can be list can be found but typing `GPy.kern.`):"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
" /home/maxz/anaconda/lib/python2.7/site-packages/matplotlib/figure.py:1653: UserWarning:This figure includes Axes that are not compatible with tight_layout, so its results might be incorrect.\n"
]
},
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"figure, axes = plt.subplots(3,3, figsize=(10,10), tight_layout=True)\n",
"kerns = [GPy.kern.RBF(1), GPy.kern.Exponential(1), GPy.kern.Matern32(1), GPy.kern.Matern52(1), GPy.kern.Brownian(1), GPy.kern.Bias(1), GPy.kern.Linear(1), GPy.kern.PeriodicExponential(1), GPy.kern.White(1)]\n",
"for k,a in zip(kerns, axes.flatten()):\n",
" k.plot(ax=a, x=1)\n",
" a.set_title(k.name.replace('_', ' '))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Operations to combine kernels\n",
"\n",
"In GPy, kernel objects can be added or multiplied to create a mutlitude of kernel objects. Parameters and their gradients are handled automatically, and so appear in the combined objects. When kernels are used inside GP objects all the necessary graidents are automagically computed using the chain-rule. "
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" \u001b[1mmul. \u001b[0;0m | value | constraints | priors\n",
" \u001b[1mrbf.variance \u001b[0;0m | 1.0 | +ve | \n",
" \u001b[1mrbf.lengthscale \u001b[0;0m | 2.0 | +ve | \n",
" \u001b[1mMat32.variance \u001b[0;0m | 0.5 | +ve | \n",
" \u001b[1mMat32.lengthscale\u001b[0;0m | 0.2 | +ve | \n",
" \u001b[1msum. \u001b[0;0m | value | constraints | priors\n",
" \u001b[1mrbf.variance \u001b[0;0m | 1.0 | +ve | \n",
" \u001b[1mrbf.lengthscale \u001b[0;0m | 2.0 | +ve | \n",
" \u001b[1mMat32.variance \u001b[0;0m | 0.5 | +ve | \n",
" \u001b[1mMat32.lengthscale\u001b[0;0m | 0.2 | +ve | \n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Product of kernels\n",
"k1 = GPy.kern.RBF(1,1.,2.)\n",
"k2 = GPy.kern.Matern32(1, 0.5, 0.2)\n",
"k_prod = k1 *k2\n",
"print k_prod\n",
"k_prod.plot()\n",
"\n",
"\n",
"\n",
"# Sum of kernels\n",
"k1 = GPy.kern.RBF(1,1.,2.)\n",
"k2 = GPy.kern.Matern32(1, 0.5, 0.2)\n",
"k_add = k1 + k2\n",
"print k_add\n",
"k_add.plot()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the kernels that have been added are pythonic in that the objects remain linked: changing parameters of an add kernel changes those of the constituent parts, and vice versa"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" \u001b[1mrbf. \u001b[0;0m | value | constraints | priors\n",
" \u001b[1mvariance \u001b[0;0m | 1.0 | +ve | \n",
" \u001b[1mlengthscale\u001b[0;0m | 2.0 | +ve | \n",
"\n",
" \u001b[1mrbf. \u001b[0;0m | value | constraints | priors\n",
" \u001b[1mvariance \u001b[0;0m | 12.0 | +ve | \n",
" \u001b[1mlengthscale\u001b[0;0m | 2.0 | +ve | \n"
]
}
],
"source": [
"print k1, '\\n'\n",
"k_add.rbf.variance = 12.\n",
"print k1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Operating on different domains\n",
"\n",
"When multiplying and adding kernels, there are two general possibilites, one can assume that the kernels to add/multiply are defined on the same space or on different spaces:\n",
"\n",
" * a kernel over $\\mathbb{R} \\times \\mathbb{R}: k(x,x') = k_1(x,x') \\times k_2(x,x')$\n",
" * a kernel over $\\mathbb{R}^2 \\times \\mathbb{R}^2: k(\\mathbf{x},\\mathbf{x}') = k_1(x_1,x'_1) \\times k_2(x_2,x'_2)$\n",
" \n",
"To keep things as general as possible, in GPy kernels are assigned `active_dims` which tell the kernel what to work on. To create a kernel which is a product of krnels on different spaces, we can do"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"k1 = GPy.kern.Linear(input_dim=1, active_dims=[0]) # works on the first column of X, index=0\n",
"k2 = GPy.kern.ExpQuad(input_dim=1, lengthscale=3, active_dims=[1]) # works on the second column of X, index=1\n",
"k = k1 * k2\n",
"k.plot(x=np.ones((1,2)))"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
" /home/maxz/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:5: RuntimeWarning:covariance is not positive-semidefinite.\n"
]
},
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"