{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Modelling Motion Capture Data with the GP-LVM\n", "\n", "### 2021-05-19" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "plt.rcParams.update({'font.size': 22})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## pods\n", "\n", "\\[edit\\]\n", "\n", "In Sheffield we created a suite of software tools for ‘Open Data\n", "Science.’ Open data science is an approach to sharing code, models and\n", "data that should make it easier for companies, health professionals and\n", "scientists to gain access to data science techniques.\n", "\n", "You can also check this blog post on [Open Data\n", "Science](http://inverseprobability.com/2014/07/01/open-data-science).\n", "\n", "The software can be installed using" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install --upgrade git+https://github.com/lawrennd/ods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "from the command prompt where you can access your python installation.\n", "\n", "The code is also available on github: \n", "\n", "Once `pods` is installed, it can be imported in the usual manner." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## mlai\n", "\n", "\\[edit\\]\n", "\n", "The `mlai` software is a suite of helper functions for teaching and\n", "demonstrating machine learning algorithms. It was first used in the\n", "Machine Learning and Adaptive Intelligence course in Sheffield in 2013.\n", "\n", "The software can be installed using" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install --upgrade git+https://github.com/lawrennd/mlai.git" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "from the command prompt where you can access your python installation.\n", "\n", "The code is also available on github: \n", "\n", "Once `mlai` is installed, it can be imported in the usual manner." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import mlai" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install gpy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GPy: A Gaussian Process Framework in Python\n", "\n", "\\[edit\\]\n", "\n", "Gaussian processes are a flexible tool for non-parametric analysis with\n", "uncertainty. The GPy software was started in Sheffield to provide a easy\n", "to use interface to GPs. One which allowed the user to focus on the\n", "modelling rather than the mathematics.\n", "\n", "\n", "\n", "Figure: GPy is a BSD licensed software code base for implementing\n", "Gaussian process models in Python. It is designed for teaching and\n", "modelling. We welcome contributions which can be made through the Github\n", "repository \n", "\n", "GPy is a BSD licensed software code base for implementing Gaussian\n", "process models in python. This allows GPs to be combined with a wide\n", "variety of software libraries.\n", "\n", "The software itself is available on\n", "[GitHub](https://github.com/SheffieldML/GPy) and the team welcomes\n", "contributions.\n", "\n", "The aim for GPy is to be a probabilistic-style programming language,\n", "i.e. you specify the model rather than the algorithm. As well as a large\n", "range of covariance functions the software allows for non-Gaussian\n", "likelihoods, multivariate outputs, dimensionality reduction and\n", "approximations for larger data sets.\n", "\n", "The documentation for GPy can be found\n", "[here](https://gpy.readthedocs.io/en/latest/)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## CMU Mocap Database\n", "\n", "\\[edit\\]\n", "\n", "Motion capture data from the CMU motion capture data base (CMU Motion\n", "Capture Lab, 2003)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can download any subject and motion from the data set. Here we will\n", "download motion `01` from subject `35`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "subject='35' \n", "motion=['01']" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "data = pods.datasets.cmu_mocap(subject, motion)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data dictionary contains the keys ‘Y’ and ‘skel,’ which represent\n", "the data and the skeleton.." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "data['Y'].shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data was used in the hierarchical GP-LVM paper (Lawrence and Moore,\n", "2007) in an experiment that was also recreated in the Deep Gaussian\n", "process paper (Damianou and Lawrence, 2013)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(data['citation'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And extra information about the data is included, as standard, under the\n", "keys `info` and `details`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(data['info'])\n", "print()\n", "print(data['details'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The original data has the figure moving across the floor during the\n", "motion capture sequence. We can make the figure walk ‘in place,’ by\n", "setting the x, y, z positions of the root node to zero. This makes it\n", "easier to visualize the result." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Make figure move in place.\n", "data['Y'][:, 0:3] = 0.0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also remove the mean of the data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Y = data['Y']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we create the GP-LVM model." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import GPy" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model = GPy.models.GPLVM(Y, 2, normalizer=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we optimize the model." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model.optimize(messages=True, max_f_eval=10000)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import mlai.plot as plot\n", "import mlai.mlai as ma" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%matplotlib notebook" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig, _ = plt.subplots(figsize=plot.big_wide_figsize)\n", "latent_axes = fig.add_suplot(131)\n", "sense_axes = fig.add_subplot(132)\n", "viz_axes = fig.add_subplot(133, projection='3d')\n", "\n", "model.plot_latent(ax=latent_axes)\n", "latent_axes.set_aspect('equal')\n", "\n", "y = model.Y[0, :]\n", "\n", "data_show = GPy.plotting.matplot_dep.visualize.skeleton_show(y[None, :], data['skel'], viz_axes)\n", "\n", "lvm_visualizer = GPy.plotting.matplot_dep.visualize.lvm(model.X[0].copy(), model, data_show, latent_axes=latent_axes, sense_axes=sense_axes)\n", "ma.write_figure(figure=fig,\n", " filename='cmu-mocap-gplvm.svg', \n", " directory = './gplvm')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "input(‘Press enter to finish’) lvm_visualizer.close() data_show.close()}\n", "\n", "\n", "\n", "Figure: Gaussian process latent variable model visualisation of CMU\n", "motion capture data set." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## References\n", "\n", "\\[edit\\]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "CMU Motion Capture Lab, 2003. The CMU mocap database.\n", "\n", "Damianou, A., Lawrence, N.D., 2013. Deep Gaussian processes. pp.\n", "207–215.\n", "\n", "Lawrence, N.D., Moore, A.J., 2007. Hierarchical Gaussian process latent\n", "variable models. pp. 481–488." ] } ], "nbformat": 4, "nbformat_minor": 5, "metadata": {} }