{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Probabilistic metrics for GP-LVM and Bayesian GP-LVMs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook introduces the use of a probabilistic geometries approach in the context of Gaussian processes latent variable models, presenting examples of learning a GPLVM and Bayesian GPLVMs on data and show the latent embedding. Especially the magnification factor and what it says about the latent space.\n", "\n", "Authors: *Alessandra Tosi, Max Zwiessele*\n", "\n", "Note: Requires GPy Version >= 0.8" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/maxz/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.\n", " warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')\n" ] } ], "source": [ "import numpy as np\n", "%matplotlib inline\n", "from matplotlib import pyplot as plt\n", "import GPy # import GPy package\n", "np.random.seed(12345)\n", "GPy.plotting.change_plotting_library('plotly')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction to the visualization tools" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We interpret the input space of the GP as a low dimensional $\\textit{manifold}$ embedded into a higher dimensional space and we are interested in investigating its geometrical properties. An important aspect of our analysis is the fact that the geometry of the input space is treated in a probabilistic way, consistently with the considered models.\n", "\n", "We investigate the form of the $\\textit{expected metric}$ of the latent space of GPLVM with exponentiated quadratic kernel. In this notation the output space is denoted with $\\mathbf{Y}$ and the input (latent) space with $\\mathbf{X}$." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Define dataset \n", "N = 100\n", "k1 = GPy.kern.RBF(5, variance=1, lengthscale=1./np.random.dirichlet(np.r_[10,10,10,0.1,0.1]), ARD=True)\n", "k2 = GPy.kern.RBF(5, variance=1, lengthscale=1./np.random.dirichlet(np.r_[10,0.1,10,0.1,10]), ARD=True)\n", "k3 = GPy.kern.RBF(5, variance=1, lengthscale=1./np.random.dirichlet(np.r_[0.1,0.1,10,10,10]), ARD=True)\n", "X = np.random.normal(0, 1, (N, 5))\n", "A = np.random.multivariate_normal(np.zeros(N), k1.K(X), 10).T\n", "B = np.random.multivariate_normal(np.zeros(N), k2.K(X), 10).T\n", "C = np.random.multivariate_normal(np.zeros(N), k3.K(X), 10).T\n", "\n", "Y = np.vstack((A,B,C))\n", "labels = np.hstack((np.zeros(A.shape[0]), np.ones(B.shape[0]), np.ones(C.shape[0])*2))\n", "\n", "input_dim = 2 # How many latent dimensions to use\n", "kernel = GPy.kern.RBF(input_dim, 1, ARD=True) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Following from the properties of Gaussian processes (and given that we are using a differentiable kernel) for a test point $\\textbf{x}_*$ we can compute the predictive Jacobian $\\textbf{J}$ as follows:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$ \\mathbf{J} \\sim \\prod_p \\mathcal{N}( \\partial\\mathbf{K}_{*,f}\\mathbf{K}_{f,f}^{-1}\\mathbf{Y}_{:,p}, \\partial^2\\mathbf{K}_{*,*} - \\partial\\mathbf{K}_{*,f}\\mathbf{K}^{-1}_{f,f}\\partial\\mathbf{K}_{f,*} ) $$\n", "\n", "Given that the distribution of the Jacobian is Gaussian and considering the fact that its $p$ rows are independent, we compute the $\\textit{expected metric tensor}$:\n", "\n", "$$ \\mathbf{G} = E(\\mathbf{J}^T \\mathbf{J}) = E(\\mathbf{J}^T)E(\\mathbf{J}) + d*Cov(\\mathbf{J}) $$\n", "\n", "For the visualization of the metric tensor we use the square root of its determinant, called the $\\textit{magnification factor}$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Predictive metric (including covariance)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The computation of the expected metric tensor $\\mathbf{G}$ contains two terms, one related to the mean of the Jacobian $\\mathbf{J}$ and one related to its covariance. The plotting tool below allows to separately visualize the effect of each term. If we use covariance= False, we visualize only the mean and, vice versa, if we use mean = False we plot the effect of the covariance term into the metric." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n",
"Model: GPLVM
\n",
"Objective: 3569.70617678
\n",
"Number of Parameters: 603
\n",
"Number of Optimization Parameters: 603
\n",
"Updates: True
\n",
"
GPLVM. | value | constraints | priors |
---|---|---|---|
latent_mean | (300, 2) | ||
rbf.variance | 1.0 | +ve | |
rbf.lengthscale | 0.2 | +ve | |
Gaussian_noise.variance | 1.0 | +ve |