{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Meta-Modelling and Deploying ML Software\n", "### [Neil D. Lawrence](http://inverseprobability.com), Amazon Cambridge and University of Sheffield\n", "### 2019-05-23\n", "\n", "**Abstract**: Data is not so much the new oil, it is the new software. Data driven\n", "algorithms are increasingly present in continuously deployed production\n", "software. What challenges does this present and how can the mathematical\n", "sciences help?\n", "\n", "$$\n", "\\newcommand{\\Amatrix}{\\mathbf{A}}\n", "\\newcommand{\\KL}[2]{\\text{KL}\\left( #1\\,\\|\\,#2 \\right)}\n", "\\newcommand{\\Kaast}{\\kernelMatrix_{\\mathbf{ \\ast}\\mathbf{ \\ast}}}\n", "\\newcommand{\\Kastu}{\\kernelMatrix_{\\mathbf{ \\ast} \\inducingVector}}\n", "\\newcommand{\\Kff}{\\kernelMatrix_{\\mappingFunctionVector \\mappingFunctionVector}}\n", "\\newcommand{\\Kfu}{\\kernelMatrix_{\\mappingFunctionVector \\inducingVector}}\n", "\\newcommand{\\Kuast}{\\kernelMatrix_{\\inducingVector \\bf\\ast}}\n", "\\newcommand{\\Kuf}{\\kernelMatrix_{\\inducingVector \\mappingFunctionVector}}\n", "\\newcommand{\\Kuu}{\\kernelMatrix_{\\inducingVector \\inducingVector}}\n", "\\newcommand{\\Kuui}{\\Kuu^{-1}}\n", "\\newcommand{\\Qaast}{\\mathbf{Q}_{\\bf \\ast \\ast}}\n", "\\newcommand{\\Qastf}{\\mathbf{Q}_{\\ast \\mappingFunction}}\n", "\\newcommand{\\Qfast}{\\mathbf{Q}_{\\mappingFunctionVector \\bf \\ast}}\n", "\\newcommand{\\Qff}{\\mathbf{Q}_{\\mappingFunctionVector \\mappingFunctionVector}}\n", "\\newcommand{\\aMatrix}{\\mathbf{A}}\n", "\\newcommand{\\aScalar}{a}\n", "\\newcommand{\\aVector}{\\mathbf{a}}\n", "\\newcommand{\\acceleration}{a}\n", "\\newcommand{\\bMatrix}{\\mathbf{B}}\n", "\\newcommand{\\bScalar}{b}\n", "\\newcommand{\\bVector}{\\mathbf{b}}\n", "\\newcommand{\\basisFunc}{\\phi}\n", "\\newcommand{\\basisFuncVector}{\\boldsymbol{ \\basisFunc}}\n", "\\newcommand{\\basisFunction}{\\phi}\n", "\\newcommand{\\basisLocation}{\\mu}\n", "\\newcommand{\\basisMatrix}{\\boldsymbol{ \\Phi}}\n", "\\newcommand{\\basisScalar}{\\basisFunction}\n", "\\newcommand{\\basisVector}{\\boldsymbol{ \\basisFunction}}\n", "\\newcommand{\\activationFunction}{\\phi}\n", "\\newcommand{\\activationMatrix}{\\boldsymbol{ \\Phi}}\n", "\\newcommand{\\activationScalar}{\\basisFunction}\n", "\\newcommand{\\activationVector}{\\boldsymbol{ \\basisFunction}}\n", "\\newcommand{\\bigO}{\\mathcal{O}}\n", "\\newcommand{\\binomProb}{\\pi}\n", "\\newcommand{\\cMatrix}{\\mathbf{C}}\n", "\\newcommand{\\cbasisMatrix}{\\hat{\\boldsymbol{ \\Phi}}}\n", "\\newcommand{\\cdataMatrix}{\\hat{\\dataMatrix}}\n", "\\newcommand{\\cdataScalar}{\\hat{\\dataScalar}}\n", "\\newcommand{\\cdataVector}{\\hat{\\dataVector}}\n", "\\newcommand{\\centeredKernelMatrix}{\\mathbf{ \\MakeUppercase{\\centeredKernelScalar}}}\n", "\\newcommand{\\centeredKernelScalar}{b}\n", "\\newcommand{\\centeredKernelVector}{\\centeredKernelScalar}\n", "\\newcommand{\\centeringMatrix}{\\mathbf{H}}\n", "\\newcommand{\\chiSquaredDist}[2]{\\chi_{#1}^{2}\\left(#2\\right)}\n", "\\newcommand{\\chiSquaredSamp}[1]{\\chi_{#1}^{2}}\n", "\\newcommand{\\conditionalCovariance}{\\boldsymbol{ \\Sigma}}\n", "\\newcommand{\\coregionalizationMatrix}{\\mathbf{B}}\n", "\\newcommand{\\coregionalizationScalar}{b}\n", "\\newcommand{\\coregionalizationVector}{\\mathbf{ \\coregionalizationScalar}}\n", "\\newcommand{\\covDist}[2]{\\text{cov}_{#2}\\left(#1\\right)}\n", "\\newcommand{\\covSamp}[1]{\\text{cov}\\left(#1\\right)}\n", "\\newcommand{\\covarianceScalar}{c}\n", "\\newcommand{\\covarianceVector}{\\mathbf{ \\covarianceScalar}}\n", "\\newcommand{\\covarianceMatrix}{\\mathbf{C}}\n", "\\newcommand{\\covarianceMatrixTwo}{\\boldsymbol{ \\Sigma}}\n", "\\newcommand{\\croupierScalar}{s}\n", "\\newcommand{\\croupierVector}{\\mathbf{ \\croupierScalar}}\n", "\\newcommand{\\croupierMatrix}{\\mathbf{ \\MakeUppercase{\\croupierScalar}}}\n", "\\newcommand{\\dataDim}{p}\n", "\\newcommand{\\dataIndex}{i}\n", "\\newcommand{\\dataIndexTwo}{j}\n", "\\newcommand{\\dataMatrix}{\\mathbf{Y}}\n", "\\newcommand{\\dataScalar}{y}\n", "\\newcommand{\\dataSet}{\\mathcal{D}}\n", "\\newcommand{\\dataStd}{\\sigma}\n", "\\newcommand{\\dataVector}{\\mathbf{ \\dataScalar}}\n", "\\newcommand{\\decayRate}{d}\n", "\\newcommand{\\degreeMatrix}{\\mathbf{ \\MakeUppercase{\\degreeScalar}}}\n", "\\newcommand{\\degreeScalar}{d}\n", "\\newcommand{\\degreeVector}{\\mathbf{ \\degreeScalar}}\n", "% Already defined by latex\n", "%\\newcommand{\\det}[1]{\\left|#1\\right|}\n", "\\newcommand{\\diag}[1]{\\text{diag}\\left(#1\\right)}\n", "\\newcommand{\\diagonalMatrix}{\\mathbf{D}}\n", "\\newcommand{\\diff}[2]{\\frac{\\text{d}#1}{\\text{d}#2}}\n", "\\newcommand{\\diffTwo}[2]{\\frac{\\text{d}^2#1}{\\text{d}#2^2}}\n", "\\newcommand{\\displacement}{x}\n", "\\newcommand{\\displacementVector}{\\textbf{\\displacement}}\n", "\\newcommand{\\distanceMatrix}{\\mathbf{ \\MakeUppercase{\\distanceScalar}}}\n", "\\newcommand{\\distanceScalar}{d}\n", "\\newcommand{\\distanceVector}{\\mathbf{ \\distanceScalar}}\n", "\\newcommand{\\eigenvaltwo}{\\ell}\n", "\\newcommand{\\eigenvaltwoMatrix}{\\mathbf{L}}\n", "\\newcommand{\\eigenvaltwoVector}{\\mathbf{l}}\n", "\\newcommand{\\eigenvalue}{\\lambda}\n", "\\newcommand{\\eigenvalueMatrix}{\\boldsymbol{ \\Lambda}}\n", "\\newcommand{\\eigenvalueVector}{\\boldsymbol{ \\lambda}}\n", "\\newcommand{\\eigenvector}{\\mathbf{ \\eigenvectorScalar}}\n", "\\newcommand{\\eigenvectorMatrix}{\\mathbf{U}}\n", "\\newcommand{\\eigenvectorScalar}{u}\n", "\\newcommand{\\eigenvectwo}{\\mathbf{v}}\n", "\\newcommand{\\eigenvectwoMatrix}{\\mathbf{V}}\n", "\\newcommand{\\eigenvectwoScalar}{v}\n", "\\newcommand{\\entropy}[1]{\\mathcal{H}\\left(#1\\right)}\n", "\\newcommand{\\errorFunction}{E}\n", "\\newcommand{\\expDist}[2]{\\left<#1\\right>_{#2}}\n", "\\newcommand{\\expSamp}[1]{\\left<#1\\right>}\n", "\\newcommand{\\expectation}[1]{\\left\\langle #1 \\right\\rangle }\n", "\\newcommand{\\expectationDist}[2]{\\left\\langle #1 \\right\\rangle _{#2}}\n", "\\newcommand{\\expectedDistanceMatrix}{\\mathcal{D}}\n", "\\newcommand{\\eye}{\\mathbf{I}}\n", "\\newcommand{\\fantasyDim}{r}\n", "\\newcommand{\\fantasyMatrix}{\\mathbf{ \\MakeUppercase{\\fantasyScalar}}}\n", "\\newcommand{\\fantasyScalar}{z}\n", "\\newcommand{\\fantasyVector}{\\mathbf{ \\fantasyScalar}}\n", "\\newcommand{\\featureStd}{\\varsigma}\n", "\\newcommand{\\gammaCdf}[3]{\\mathcal{GAMMA CDF}\\left(#1|#2,#3\\right)}\n", "\\newcommand{\\gammaDist}[3]{\\mathcal{G}\\left(#1|#2,#3\\right)}\n", "\\newcommand{\\gammaSamp}[2]{\\mathcal{G}\\left(#1,#2\\right)}\n", "\\newcommand{\\gaussianDist}[3]{\\mathcal{N}\\left(#1|#2,#3\\right)}\n", "\\newcommand{\\gaussianSamp}[2]{\\mathcal{N}\\left(#1,#2\\right)}\n", "\\newcommand{\\given}{|}\n", "\\newcommand{\\half}{\\frac{1}{2}}\n", "\\newcommand{\\heaviside}{H}\n", "\\newcommand{\\hiddenMatrix}{\\mathbf{ \\MakeUppercase{\\hiddenScalar}}}\n", "\\newcommand{\\hiddenScalar}{h}\n", "\\newcommand{\\hiddenVector}{\\mathbf{ \\hiddenScalar}}\n", "\\newcommand{\\identityMatrix}{\\eye}\n", "\\newcommand{\\inducingInputScalar}{z}\n", "\\newcommand{\\inducingInputVector}{\\mathbf{ \\inducingInputScalar}}\n", "\\newcommand{\\inducingInputMatrix}{\\mathbf{Z}}\n", "\\newcommand{\\inducingScalar}{u}\n", "\\newcommand{\\inducingVector}{\\mathbf{ \\inducingScalar}}\n", "\\newcommand{\\inducingMatrix}{\\mathbf{U}}\n", "\\newcommand{\\inlineDiff}[2]{\\text{d}#1/\\text{d}#2}\n", "\\newcommand{\\inputDim}{q}\n", "\\newcommand{\\inputMatrix}{\\mathbf{X}}\n", "\\newcommand{\\inputScalar}{x}\n", "\\newcommand{\\inputSpace}{\\mathcal{X}}\n", "\\newcommand{\\inputVals}{\\inputVector}\n", "\\newcommand{\\inputVector}{\\mathbf{ \\inputScalar}}\n", "\\newcommand{\\iterNum}{k}\n", "\\newcommand{\\kernel}{\\kernelScalar}\n", "\\newcommand{\\kernelMatrix}{\\mathbf{K}}\n", "\\newcommand{\\kernelScalar}{k}\n", "\\newcommand{\\kernelVector}{\\mathbf{ \\kernelScalar}}\n", "\\newcommand{\\kff}{\\kernelScalar_{\\mappingFunction \\mappingFunction}}\n", "\\newcommand{\\kfu}{\\kernelVector_{\\mappingFunction \\inducingScalar}}\n", "\\newcommand{\\kuf}{\\kernelVector_{\\inducingScalar \\mappingFunction}}\n", "\\newcommand{\\kuu}{\\kernelVector_{\\inducingScalar \\inducingScalar}}\n", "\\newcommand{\\lagrangeMultiplier}{\\lambda}\n", "\\newcommand{\\lagrangeMultiplierMatrix}{\\boldsymbol{ \\Lambda}}\n", "\\newcommand{\\lagrangian}{L}\n", "\\newcommand{\\laplacianFactor}{\\mathbf{ \\MakeUppercase{\\laplacianFactorScalar}}}\n", "\\newcommand{\\laplacianFactorScalar}{m}\n", "\\newcommand{\\laplacianFactorVector}{\\mathbf{ \\laplacianFactorScalar}}\n", "\\newcommand{\\laplacianMatrix}{\\mathbf{L}}\n", "\\newcommand{\\laplacianScalar}{\\ell}\n", "\\newcommand{\\laplacianVector}{\\mathbf{ \\ell}}\n", "\\newcommand{\\latentDim}{q}\n", "\\newcommand{\\latentDistanceMatrix}{\\boldsymbol{ \\Delta}}\n", "\\newcommand{\\latentDistanceScalar}{\\delta}\n", "\\newcommand{\\latentDistanceVector}{\\boldsymbol{ \\delta}}\n", "\\newcommand{\\latentForce}{f}\n", "\\newcommand{\\latentFunction}{u}\n", "\\newcommand{\\latentFunctionVector}{\\mathbf{ \\latentFunction}}\n", "\\newcommand{\\latentFunctionMatrix}{\\mathbf{ \\MakeUppercase{\\latentFunction}}}\n", "\\newcommand{\\latentIndex}{j}\n", "\\newcommand{\\latentScalar}{z}\n", "\\newcommand{\\latentVector}{\\mathbf{ \\latentScalar}}\n", "\\newcommand{\\latentMatrix}{\\mathbf{Z}}\n", "\\newcommand{\\learnRate}{\\eta}\n", "\\newcommand{\\lengthScale}{\\ell}\n", "\\newcommand{\\rbfWidth}{\\ell}\n", "\\newcommand{\\likelihoodBound}{\\mathcal{L}}\n", "\\newcommand{\\likelihoodFunction}{L}\n", "\\newcommand{\\locationScalar}{\\mu}\n", "\\newcommand{\\locationVector}{\\boldsymbol{ \\locationScalar}}\n", "\\newcommand{\\locationMatrix}{\\mathbf{M}}\n", "\\newcommand{\\variance}[1]{\\text{var}\\left( #1 \\right)}\n", "\\newcommand{\\mappingFunction}{f}\n", "\\newcommand{\\mappingFunctionMatrix}{\\mathbf{F}}\n", "\\newcommand{\\mappingFunctionTwo}{g}\n", "\\newcommand{\\mappingFunctionTwoMatrix}{\\mathbf{G}}\n", "\\newcommand{\\mappingFunctionTwoVector}{\\mathbf{ \\mappingFunctionTwo}}\n", "\\newcommand{\\mappingFunctionVector}{\\mathbf{ \\mappingFunction}}\n", "\\newcommand{\\scaleScalar}{s}\n", "\\newcommand{\\mappingScalar}{w}\n", "\\newcommand{\\mappingVector}{\\mathbf{ \\mappingScalar}}\n", "\\newcommand{\\mappingMatrix}{\\mathbf{W}}\n", "\\newcommand{\\mappingScalarTwo}{v}\n", "\\newcommand{\\mappingVectorTwo}{\\mathbf{ \\mappingScalarTwo}}\n", "\\newcommand{\\mappingMatrixTwo}{\\mathbf{V}}\n", "\\newcommand{\\maxIters}{K}\n", "\\newcommand{\\meanMatrix}{\\mathbf{M}}\n", "\\newcommand{\\meanScalar}{\\mu}\n", "\\newcommand{\\meanTwoMatrix}{\\mathbf{M}}\n", "\\newcommand{\\meanTwoScalar}{m}\n", "\\newcommand{\\meanTwoVector}{\\mathbf{ \\meanTwoScalar}}\n", "\\newcommand{\\meanVector}{\\boldsymbol{ \\meanScalar}}\n", "\\newcommand{\\mrnaConcentration}{m}\n", "\\newcommand{\\naturalFrequency}{\\omega}\n", "\\newcommand{\\neighborhood}[1]{\\mathcal{N}\\left( #1 \\right)}\n", "\\newcommand{\\neilurl}{http://inverseprobability.com/}\n", "\\newcommand{\\noiseMatrix}{\\boldsymbol{ E}}\n", "\\newcommand{\\noiseScalar}{\\epsilon}\n", "\\newcommand{\\noiseVector}{\\boldsymbol{ \\epsilon}}\n", "\\newcommand{\\norm}[1]{\\left\\Vert #1 \\right\\Vert}\n", "\\newcommand{\\normalizedLaplacianMatrix}{\\hat{\\mathbf{L}}}\n", "\\newcommand{\\normalizedLaplacianScalar}{\\hat{\\ell}}\n", "\\newcommand{\\normalizedLaplacianVector}{\\hat{\\mathbf{ \\ell}}}\n", "\\newcommand{\\numActive}{m}\n", "\\newcommand{\\numBasisFunc}{m}\n", "\\newcommand{\\numComponents}{m}\n", "\\newcommand{\\numComps}{K}\n", "\\newcommand{\\numData}{n}\n", "\\newcommand{\\numFeatures}{K}\n", "\\newcommand{\\numHidden}{h}\n", "\\newcommand{\\numInducing}{m}\n", "\\newcommand{\\numLayers}{\\ell}\n", "\\newcommand{\\numNeighbors}{K}\n", "\\newcommand{\\numSequences}{s}\n", "\\newcommand{\\numSuccess}{s}\n", "\\newcommand{\\numTasks}{m}\n", "\\newcommand{\\numTime}{T}\n", "\\newcommand{\\numTrials}{S}\n", "\\newcommand{\\outputIndex}{j}\n", "\\newcommand{\\paramVector}{\\boldsymbol{ \\theta}}\n", "\\newcommand{\\parameterMatrix}{\\boldsymbol{ \\Theta}}\n", "\\newcommand{\\parameterScalar}{\\theta}\n", "\\newcommand{\\parameterVector}{\\boldsymbol{ \\parameterScalar}}\n", "\\newcommand{\\partDiff}[2]{\\frac{\\partial#1}{\\partial#2}}\n", "\\newcommand{\\precisionScalar}{j}\n", "\\newcommand{\\precisionVector}{\\mathbf{ \\precisionScalar}}\n", "\\newcommand{\\precisionMatrix}{\\mathbf{J}}\n", "\\newcommand{\\pseudotargetScalar}{\\widetilde{y}}\n", "\\newcommand{\\pseudotargetVector}{\\mathbf{ \\pseudotargetScalar}}\n", "\\newcommand{\\pseudotargetMatrix}{\\mathbf{ \\widetilde{Y}}}\n", "\\newcommand{\\rank}[1]{\\text{rank}\\left(#1\\right)}\n", "\\newcommand{\\rayleighDist}[2]{\\mathcal{R}\\left(#1|#2\\right)}\n", "\\newcommand{\\rayleighSamp}[1]{\\mathcal{R}\\left(#1\\right)}\n", "\\newcommand{\\responsibility}{r}\n", "\\newcommand{\\rotationScalar}{r}\n", "\\newcommand{\\rotationVector}{\\mathbf{ \\rotationScalar}}\n", "\\newcommand{\\rotationMatrix}{\\mathbf{R}}\n", "\\newcommand{\\sampleCovScalar}{s}\n", "\\newcommand{\\sampleCovVector}{\\mathbf{ \\sampleCovScalar}}\n", "\\newcommand{\\sampleCovMatrix}{\\mathbf{s}}\n", "\\newcommand{\\scalarProduct}[2]{\\left\\langle{#1},{#2}\\right\\rangle}\n", "\\newcommand{\\sign}[1]{\\text{sign}\\left(#1\\right)}\n", "\\newcommand{\\sigmoid}[1]{\\sigma\\left(#1\\right)}\n", "\\newcommand{\\singularvalue}{\\ell}\n", "\\newcommand{\\singularvalueMatrix}{\\mathbf{L}}\n", "\\newcommand{\\singularvalueVector}{\\mathbf{l}}\n", "\\newcommand{\\sorth}{\\mathbf{u}}\n", "\\newcommand{\\spar}{\\lambda}\n", "\\newcommand{\\trace}[1]{\\text{tr}\\left(#1\\right)}\n", "\\newcommand{\\BasalRate}{B}\n", "\\newcommand{\\DampingCoefficient}{C}\n", "\\newcommand{\\DecayRate}{D}\n", "\\newcommand{\\Displacement}{X}\n", "\\newcommand{\\LatentForce}{F}\n", "\\newcommand{\\Mass}{M}\n", "\\newcommand{\\Sensitivity}{S}\n", "\\newcommand{\\basalRate}{b}\n", "\\newcommand{\\dampingCoefficient}{c}\n", "\\newcommand{\\mass}{m}\n", "\\newcommand{\\sensitivity}{s}\n", "\\newcommand{\\springScalar}{\\kappa}\n", "\\newcommand{\\springVector}{\\boldsymbol{ \\kappa}}\n", "\\newcommand{\\springMatrix}{\\boldsymbol{ \\mathcal{K}}}\n", "\\newcommand{\\tfConcentration}{p}\n", "\\newcommand{\\tfDecayRate}{\\delta}\n", "\\newcommand{\\tfMrnaConcentration}{f}\n", "\\newcommand{\\tfVector}{\\mathbf{ \\tfConcentration}}\n", "\\newcommand{\\velocity}{v}\n", "\\newcommand{\\sufficientStatsScalar}{g}\n", "\\newcommand{\\sufficientStatsVector}{\\mathbf{ \\sufficientStatsScalar}}\n", "\\newcommand{\\sufficientStatsMatrix}{\\mathbf{G}}\n", "\\newcommand{\\switchScalar}{s}\n", "\\newcommand{\\switchVector}{\\mathbf{ \\switchScalar}}\n", "\\newcommand{\\switchMatrix}{\\mathbf{S}}\n", "\\newcommand{\\tr}[1]{\\text{tr}\\left(#1\\right)}\n", "\\newcommand{\\loneNorm}[1]{\\left\\Vert #1 \\right\\Vert_1}\n", "\\newcommand{\\ltwoNorm}[1]{\\left\\Vert #1 \\right\\Vert_2}\n", "\\newcommand{\\onenorm}[1]{\\left\\vert#1\\right\\vert_1}\n", "\\newcommand{\\twonorm}[1]{\\left\\Vert #1 \\right\\Vert}\n", "\\newcommand{\\vScalar}{v}\n", "\\newcommand{\\vVector}{\\mathbf{v}}\n", "\\newcommand{\\vMatrix}{\\mathbf{V}}\n", "\\newcommand{\\varianceDist}[2]{\\text{var}_{#2}\\left( #1 \\right)}\n", "% Already defined by latex\n", "%\\newcommand{\\vec}{#1:}\n", "\\newcommand{\\vecb}[1]{\\left(#1\\right):}\n", "\\newcommand{\\weightScalar}{w}\n", "\\newcommand{\\weightVector}{\\mathbf{ \\weightScalar}}\n", "\\newcommand{\\weightMatrix}{\\mathbf{W}}\n", "\\newcommand{\\weightedAdjacencyMatrix}{\\mathbf{A}}\n", "\\newcommand{\\weightedAdjacencyScalar}{a}\n", "\\newcommand{\\weightedAdjacencyVector}{\\mathbf{ \\weightedAdjacencyScalar}}\n", "\\newcommand{\\onesVector}{\\mathbf{1}}\n", "\\newcommand{\\zerosVector}{\\mathbf{0}}\n", "$$\n", "\n", "\n", ".\n", "\n", "\n", "\n", "\n", "# Introduction\n", "\n", "## Peppercorns [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "::: {.figure}\n", "::: {#peppercorn-siri-figure .figure-frame}" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.lib.display import YouTubeVideo\n", "YouTubeVideo('1y2UKz47gew')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ ":::\n", "\n", "::: {#peppercorn-siri-magnify .magnify onclick=\"magnifyFigure('peppercorn-siri')\"}\n", "\n", ":::\n", "\n", "::: {#peppercorn-siri-caption .caption-frame}\n", "Figure: A peppercorn is a system design failure which is not a bug, but\n", "a conformance to design specification that causes problems when the\n", "system is deployed in the real world with mischevious and adversarial\n", "actors.\n", ":::\n", ":::\n", "\n", "Asking Siri \"What is a trillion to the power of a thousand minus one?\"\n", "leads to a 30 minute response[^1] consisting of only 9s. I found this\n", "out because my nine year old grabbed my phone and did it. The only way\n", "to stop Siri was to force closure. This is an interesting example of a\n", "system feature that's *not* a bug, in fact it requires clever processing\n", "from Wolfram Alpha. But it's an unexpected result from the system\n", "performing correctly.\n", "\n", "This challenge of facing a circumstance that was unenvisaged in design\n", "but has consequences in deployment becomes far larger when the\n", "environment is uncontrolled. Or in the extreme case, where actions of\n", "the intelligent system effect the wider environment and change it.\n", "\n", "These unforseen circumstances are likely to lead to need for much more\n", "efficient turn-around and update for our intelligent systems. Whether we\n", "are correcting for security flaws (which *are* bugs) or unenvisaged\n", "circumstantial challenges: an issue I'm referring to as *peppercorns*.\n", "Rapid deployment of system updates is required. For example, Apple have\n", "\"fixed\" the problem of Siri returning long numbers.\n", "\n", "The challenge is particularly acute because of the *scale* at which we\n", "can deploy AI solutions. This means when something does go wrong, it may\n", "be going wrong in billions of households simultaneously.\n", "\n", "You can also check my blog post on [\"Decision Making and\n", "Diversity\"](http://inverseprobability.com/2017/11/15/decision-making)\n", "You can also check my blog post on [\"Natural vs Artifical\n", "Intelligence\"](http://inverseprobability.com/2018/02/06/natural-and-artificial-intelligence)\n", "\n", "# Deep Learning [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "\n", "### DeepFace [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "::: {.figure}\n", "::: {#deep-face-figure .figure-frame}\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#deep-face-magnify .magnify onclick=\"magnifyFigure('deep-face')\"}\n", "\n", ":::\n", "\n", "::: {#deep-face-caption .caption-frame}\n", "Figure: The DeepFace architecture [@Taigman:deepface14], visualized\n", "through colors to represent the functional mappings at each layer. There\n", "are 120 million parameters in the model.\n", ":::\n", ":::\n", "\n", "The DeepFace architecture [@Taigman:deepface14] consists of layers that\n", "deal with *translation* and *rotational* invariances. These layers are\n", "followed by three locally-connected layers and two fully-connected\n", "layers. Color illustrates feature maps produced at each layer. The\n", "neural network includes more than 120 million parameters, where more\n", "than 95% come from the local and fully connected layers.\n", "\n", "### Deep Learning as Pinball [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "::: {.figure}\n", "::: {#early-pinball-figure .figure-frame}\n", "::: {.centered .centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#early-pinball-magnify .magnify onclick=\"magnifyFigure('early-pinball')\"}\n", "\n", ":::\n", "\n", "::: {#early-pinball-caption .caption-frame}\n", "Figure: Deep learning models are composition of simple functions. We can\n", "think of a pinball machine as an analogy. Each layer of pins corresponds\n", "to one of the layers of functions in the model. Input data is\n", "represented by the location of the ball from left to right when it is\n", "dropped in from the top. Output class comes from the position of the\n", "ball as it leaves the pins at the bottom.\n", ":::\n", ":::\n", "\n", "Sometimes deep learning models are described as being like the brain, or\n", "too complex to understand, but one analogy I find useful to help the\n", "gist of these models is to think of them as being similar to early pin\n", "ball machines.\n", "\n", "In a deep neural network, we input a number (or numbers), whereas in\n", "pinball, we input a ball.\n", "\n", "Think of the location of the ball on the left-right axis as a single\n", "number. Our simple pinball machine can only take one number at a time.\n", "As the ball falls through the machine, each layer of pins can be thought\n", "of as a different layer of 'neurons'. Each layer acts to move the ball\n", "from left to right.\n", "\n", "In a pinball machine, when the ball gets to the bottom it might fall\n", "into a hole defining a score, in a neural network, that is equivalent to\n", "the decision: a classification of the input object.\n", "\n", "An image has more than one number associated with it, so it is like\n", "playing pinball in a *hyper-space*." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pods\n", "from ipywidgets import IntSlider" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pods.notebook.display_plots('pinball{sample:0>3}.svg', \n", " '../slides/diagrams',\n", " sample=IntSlider(1, 1, 2, 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#pinball-initialization-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#pinball-initialization-magnify .magnify onclick=\"magnifyFigure('pinball-initialization')\"}\n", "\n", ":::\n", "\n", "::: {#pinball-initialization-caption .caption-frame}\n", "Figure: At initialization, the pins, which represent the parameters of\n", "the function, aren't in the right place to bring the balls to the\n", "correct decisions.\n", ":::\n", ":::\n", "\n", "::: {.figure}\n", "::: {#pinball-trained-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#pinball-trained-magnify .magnify onclick=\"magnifyFigure('pinball-trained')\"}\n", "\n", ":::\n", "\n", "::: {#pinball-trained-caption .caption-frame}\n", "Figure: After learning the pins are now in the right place to bring the\n", "balls to the correct decisions.\n", ":::\n", ":::\n", "\n", "Learning involves moving all the pins to be in the correct position, so\n", "that the ball ends up in the right place when it's fallen through the\n", "machine. But moving all these pins in hyperspace can be difficult.\n", "\n", "In a hyper-space you have to put a lot of data through the machine for\n", "to explore the positions of all the pins. Even when you feed many\n", "millions of data points through the machine, there are likely to be\n", "regions in the hyper-space where no ball has passed. When future test\n", "data passes through the machine in a new route unusual things can\n", "happen.\n", "\n", "*Adversarial examples* exploit this high dimensional space. If you have\n", "access to the pinball machine, you can use gradient methods to find a\n", "position for the ball in the hyper space where the image looks like one\n", "thing, but will be classified as another.\n", "\n", "Probabilistic methods explore more of the space by considering a range\n", "of possible paths for the ball through the machine. This helps to make\n", "them more data efficient and gives some robustness to adversarial\n", "examples.\n", "\n", "## Containerization [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "::: {.figure}\n", "::: {#container-2539942_1920-figure .figure-frame}\n", "::: {.centered .centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#container-2539942_1920-magnify .magnify onclick=\"magnifyFigure('container-2539942_1920')\"}\n", "\n", ":::\n", "\n", "::: {#container-2539942_1920-caption .caption-frame}\n", "Figure: The container is one of the major drivers of globalization, and\n", "arguably the largest agent of social change in the last 100 years. It\n", "reduces the cost of transportation, significantly changing the\n", "appropriate topology of distribution networks. The container makes it\n", "possible to ship goods half way around the world for cheaper than it\n", "costs to process those goods, leading to an extended distribution\n", "topology.\n", ":::\n", ":::\n", "\n", "Containerization has had a dramatic effect on global economics, placing\n", "many people in the developing world at the end of the supply chain.\n", "\n", "::: {.figure}\n", "::: {#wild-alaskan-cod-figure .figure-frame}\n", "\n", "\n", "\n", "\n", "\n", "
\n", "::: {.centered .centered style=\"\"}\n", "\n", ":::\n", "\n", "\n", "::: {.centered .centered style=\"\"}\n", "\n", ":::\n", "\n", "
\n", ":::\n", "\n", "::: {#wild-alaskan-cod-magnify .magnify onclick=\"magnifyFigure('wild-alaskan-cod')\"}\n", "\n", ":::\n", "\n", "::: {#wild-alaskan-cod-caption .caption-frame}\n", "Figure: Wild Alaskan Cod, being solid in the Pacific Northwest, that is\n", "a product of China. It is cheaper to ship the deep frozen fish thousands\n", "of kilometers for processing than to process locally.\n", ":::\n", ":::\n", "\n", "For example, you can buy Wild Alaskan Cod fished from Alaska, processed\n", "in China, sold in North America. This is driven by the low cost of\n", "transport for frozen cod vs the higher relative cost of cod processing\n", "in the US versus China. Similarly,\n", "Scottish\n", "prawns are also processed in China for sale in the UK.\n", "\n", "This effect on cost of transport vs cost of processing is the main\n", "driver of the topology of the modern supply chain and the associated\n", "effect of globalization. If transport is much cheaper than processing,\n", "then processing will tend to agglomerate in places where processing\n", "costs can be minimized.\n", "\n", "> Solve Supply Chain, then solve everything else.\n", "\n", "## Emulation [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "::: {.figure}\n", "::: {#statistical-emulation-1-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#statistical-emulation-1-magnify .magnify onclick=\"magnifyFigure('statistical-emulation-1')\"}\n", "\n", ":::\n", "\n", "::: {#statistical-emulation-1-caption .caption-frame}\n", "Figure: Real world systems consiste of simulators, that capture our\n", "domain knowledge about how our systems operate. Different simulators run\n", "at different speeds and granularities.\n", ":::\n", ":::\n", "\n", "In many real world systems, decisions are made through simulating the\n", "environment. Simulations may operate at different granularities. For\n", "example, simulations are used in weather forecasts and climate\n", "forecasts. The UK Met office uses the same code for both, but operates\n", "climate simulations one at greater spatial and temporal resolutions.\n", "\n", "::: {.figure}\n", "::: {#statistical-emulation-2-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#statistical-emulation-2-magnify .magnify onclick=\"magnifyFigure('statistical-emulation-2')\"}\n", "\n", ":::\n", "\n", "::: {#statistical-emulation-2-caption .caption-frame}\n", "Figure: A statistical emulator is a system that reconstructs the\n", "simulation with a statistical model.\n", ":::\n", ":::\n", "\n", "A statistical emulator is a data-driven model that learns about the\n", "underlying simulation. Importantly, learns with uncertainty, so it\n", "'knows what it doesn't know'. In practice, we can call the emulator in\n", "place of the simulator. If the emulator 'doesn't know', it can call the\n", "simulator for the answer.\n", "\n", "::: {.figure}\n", "::: {#statistical-emulation-5-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#statistical-emulation-5-magnify .magnify onclick=\"magnifyFigure('statistical-emulation-5')\"}\n", "\n", ":::\n", "\n", "::: {#statistical-emulation-5-caption .caption-frame}\n", "Figure: A statistical emulator is a system that reconstructs the\n", "simulation with a statistical model. As well as reconstructing the\n", "simulation, a statistical emulator can be used to correlate with the\n", "real world.\n", ":::\n", ":::\n", "\n", "As well as reconstructing an individual simulator, the emulator can\n", "calibrate the simulation to the real world, by monitoring differences\n", "between the simulator and real data. This allows the emulator to\n", "characterise where the simulation can be relied on, i.e. we can validate\n", "the simulator.\n", "\n", "Similarly, the emulator can adjudicate between simulations. This is\n", "known as *multi-fidelity emulation*. The emulator characterizes which\n", "emulations perform well where.\n", "\n", "If all this modelling is done with judiscious handling of the\n", "uncertainty, the *computational doubt*, then the emulator can assist in\n", "desciding what experiment should be run next to aid a decision: should\n", "we run a simulator, in which case which one, or should we attempt to\n", "acquire data from a real world intervention.\n", "\n", "## Uncertainty Quantification [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "> Uncertainty quantification (UQ) is the science of quantitative\n", "> characterization and reduction of uncertainties in both computational\n", "> and real world applications. It tries to determine how likely certain\n", "> outcomes are if some aspects of the system are not exactly known.\n", "\n", "We will to illustrate different concepts of [Uncertainty\n", "Quantification](https://en.wikipedia.org/wiki/Uncertainty_quantification)\n", "(UQ) and the role that Gaussian processes play in this field. Based on a\n", "simple simulator of a car moving between a valley and a mountain, we are\n", "going to illustrate the following concepts:\n", "\n", "- **Systems emulation**. Many real world decisions are based on\n", " simulations that can be computationally very demanding. We will show\n", " how simulators can be replaced by *emulators*: Gaussian process\n", " models fitted on a few simulations that can be used to replace the\n", " *simulator*. Emulators are cheap to compute, fast to run, and always\n", " provide ways to quantify the uncertainty of how precise they are\n", " compared the original simulator.\n", "\n", "- **Emulators in optimization problems**. We will show how emulators\n", " can be used to optimize black-box functions that are expensive to\n", " evaluate. This field is also called Bayesian Optimization and has\n", " gained an increasing relevance in machine learning as emulators can\n", " be used to optimize computer simulations (and machine learning\n", " algorithms) quite efficiently.\n", "\n", "- **Multi-fidelity emulation methods**. In many scenarios we have\n", " simulators of different quality about the same measure of interest.\n", " In these cases the goal is to merge all sources of information under\n", " the same model so the final emulator is cheaper and more accurate\n", " than an emulator fitted only using data from the most accurate and\n", " expensive simulator.\n", "\n", "## Mountain Car Simulator [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "To illustrate the above mentioned concepts we we use the [mountain car\n", "simulator](https://github.com/openai/gym/wiki/MountainCarContinuous-v0).\n", "This simulator is widely used in machine learning to test reinforcement\n", "learning algorithms. The goal is to define a control policy on a car\n", "whose objective is to climb a mountain. Graphically, the problem looks\n", "as follows:\n", "\n", "::: {.figure}\n", "::: {#mountain-car-figure .figure-frame}\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#mountain-car-magnify .magnify onclick=\"magnifyFigure('mountain-car')\"}\n", "\n", ":::\n", "\n", "::: {#mountain-car-caption .caption-frame}\n", "Figure: The mountain car simulation from the Open AI gym.\n", ":::\n", ":::\n", "\n", "The goal is to define a sequence of actions (push the car right or left\n", "with certain intensity) to make the car reach the flag after a number\n", "$T$ of time steps.\n", "\n", "At each time step $t$, the car is characterized by a vector\n", "$\\inputVector_{t} = (p_t,v_t)$ of states which are respectively the the\n", "position and velocity of the car at time $t$. For a sequence of states\n", "(an episode), the dynamics of the car is given by\n", "\n", "$$\\inputVector_{t+1} = \\mappingFunction(\\inputVector_{t},\\textbf{u}_{t})$$\n", "\n", "where $\\textbf{u}_{t}$ is the value of an action force, which in this\n", "example corresponds to push car to the left (negative value) or to the\n", "right (positive value). The actions across a full episode are\n", "represented in a policy $\\textbf{u}_{t} = \\pi(\\inputVector_{t},\\theta)$\n", "that acts according to the current state of the car and some parameters\n", "$\\theta$. In the following examples we will assume that the policy is\n", "linear which allows us to write $\\pi(\\inputVector_{t},\\theta)$ as\n", "\n", "$$\\pi(\\inputVector,\\theta)= \\theta_0 + \\theta_p p + \\theta_vv.$$\n", "\n", "For $t=1,\\dots,T$ now given some initial state $\\inputVector_{0}$ and\n", "some some values of each $\\textbf{u}_{t}$, we can **simulate** the full\n", "dynamics of the car for a full episode using\n", "[Gym](https://gym.openai.com/envs/). The values of $\\textbf{u}_{t}$ are\n", "fully determined by the parameters of the linear controller.\n", "\n", "After each episode of length $T$ is complete, a reward function\n", "$R_{T}(\\theta)$ is computed. In the mountain car example the reward is\n", "computed as 100 for reaching the target of the hill on the right hand\n", "side, minus the squared sum of actions (a real negative to push to the\n", "left and a real positive to push to the right) from start to goal. Note\n", "that our reward depend on $\\theta$ as we make it dependent on the\n", "parameters of the linear controller.\n", "\n", "## Emulate the Mountain Car" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import gym" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "env = gym.make('MountainCarContinuous-v0')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our goal in this section is to find the parameters $\\theta$ of the\n", "linear controller such that\n", "\n", "$$\\theta^* = arg \\max_{\\theta} R_T(\\theta).$$\n", "\n", "In this section, we directly use Bayesian optimization to solve this\n", "problem. We will use [GPyOpt](https://sheffieldml.github.io/GPyOpt/) so\n", "we first define the objective function:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import mountain_car as mc\n", "import GPyOpt" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "obj_func = lambda x: mc.run_simulation(env, x)[0]\n", "objective = GPyOpt.core.task.SingleObjective(obj_func)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For each set of parameter values of the linear controller we can run an\n", "episode of the simulator (that we fix to have a horizon of $T=500$) to\n", "generate the reward. Using as input the parameters of the controller and\n", "as outputs the rewards we can build a Gaussian process emulator of the\n", "reward.\n", "\n", "We start defining the input space, which is three-dimensional:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## --- We define the input space of the emulator\n", "\n", "space= [{'name':'postion_parameter', 'type':'continuous', 'domain':(-1.2, +1)},\n", " {'name':'velocity_parameter', 'type':'continuous', 'domain':(-1/0.07, +1/0.07)},\n", " {'name':'constant', 'type':'continuous', 'domain':(-1, +1)}]\n", "\n", "design_space = GPyOpt.Design_space(space=space)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we initizialize a Gaussian process emulator." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model = GPyOpt.models.GPModel(optimize_restarts=5, verbose=False, exact_feval=True, ARD=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In Bayesian optimization an acquisition function is used to balance\n", "exploration and exploitation to evaluate new locations close to the\n", "optimum of the objective. In this notebook we select the expected\n", "improvement (EI). For further details have a look to the review paper of\n", "[Shahriari et al\n", "(2015)](http://www.cs.ox.ac.uk/people/nando.defreitas/publications/BayesOptLoop.pdf)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "aquisition_optimizer = GPyOpt.optimization.AcquisitionOptimizer(design_space)\n", "acquisition = GPyOpt.acquisitions.AcquisitionEI(model, design_space, optimizer=aquisition_optimizer)\n", "evaluator = GPyOpt.core.evaluators.Sequential(acquisition) # Collect points sequentially, no parallelization." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To initalize the model we start sampling some initial points (25) for\n", "the linear controler randomly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from GPyOpt.experiment_design.random_design import RandomDesign" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "n_initial_points = 25\n", "random_design = RandomDesign(design_space)\n", "initial_design = random_design.get_samples(n_initial_points)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before we start any optimization, lets have a look to the behavior of\n", "the car with the first of these initial points that we have selected\n", "randomly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "random_controller = initial_design[0,:]\n", "_, _, _, frames = mc.run_simulation(env, np.atleast_2d(random_controller), render=True)\n", "anim=mc.animate_frames(frames, 'Random linear controller')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.core.display import HTML" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "HTML(anim.to_jshtml())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#mountain-car-random-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#mountain-car-random-magnify .magnify onclick=\"magnifyFigure('mountain-car-random')\"}\n", "\n", ":::\n", "\n", "::: {#mountain-car-random-caption .caption-frame}\n", "Figure: Random linear controller for the Mountain car. It fails to move\n", "the car to the top of the mountain.\n", ":::\n", ":::\n", "\n", "As we can see the random linear controller does not manage to push the\n", "car to the top of the mountain. Now, let's optimize the regret using\n", "Bayesian optimization and the emulator for the reward. We try 50 new\n", "parameters chosen by the EI." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "max_iter = 50\n", "bo = GPyOpt.methods.ModularBayesianOptimization(model, design_space, objective, acquisition, evaluator, initial_design)\n", "bo.run_optimization(max_iter = max_iter )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we visualize the result for the best controller that we have found\n", "with Bayesian optimization." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "_, _, _, frames = mc.run_simulation(env, np.atleast_2d(bo.x_opt), render=True)\n", "anim=mc.animate_frames(frames, 'Best controller after 50 iterations of Bayesian optimization')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "HTML(anim.to_jshtml())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#mountain-car-similated-bayes-opt-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#mountain-car-similated-bayes-opt-magnify .magnify onclick=\"magnifyFigure('mountain-car-similated-bayes-opt')\"}\n", "\n", ":::\n", "\n", "::: {#mountain-car-similated-bayes-opt-caption .caption-frame}\n", "Figure: Mountain car simulator trained using Bayesian optimization and\n", "the simulator of the dynamics. Fifty iterations of Bayesian optimization\n", "are used to optimize the controler.\n", ":::\n", ":::\n", "\n", "he car can now make it to the top of the mountain! Emulating the reward\n", "function and using the EI helped as to find a linear controller that\n", "solves the problem.\n", "\n", "## Data Efficient Emulation [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "In the previous section we solved the mountain car problem by directly\n", "emulating the reward but no considerations about the dynamics\n", "$\\inputVector_{t+1} = \\mappingFunction(\\inputVector_{t},\\textbf{u}_{t})$\n", "of the system were made. Note that we had to run 75 episodes of 500\n", "steps each to solve the problem, which required to call the simulator\n", "$500\\times 75 =37500$ times. In this section we will show how it is\n", "possible to reduce this number by building an emulator for $f$ that can\n", "later be used to directly optimize the control.\n", "\n", "The inputs of the model for the dynamics are the velocity, the position\n", "and the value of the control so create this space accordingly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import gym" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "env = gym.make('MountainCarContinuous-v0')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import GPyOpt" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "space_dynamics = [{'name':'position', 'type':'continuous', 'domain':[-1.2, +0.6]},\n", " {'name':'velocity', 'type':'continuous', 'domain':[-0.07, +0.07]},\n", " {'name':'action', 'type':'continuous', 'domain':[-1, +1]}]\n", "design_space_dynamics = GPyOpt.Design_space(space=space_dynamics)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The outputs are the velocity and the position. Indeed our model will\n", "capture the change in position and velocity on time. That is, we will\n", "model\n", "\n", "$$\\Delta v_{t+1} = v_{t+1} - v_{t}$$\n", "\n", "$$\\Delta x_{t+1} = p_{t+1} - p_{t}$$\n", "\n", "with Gaussian processes with prior mean $v_{t}$ and $p_{t}$\n", "respectively. As a covariance function, we use a Matern52. We need\n", "therefore two models to capture the full dynamics of the system." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "position_model = GPyOpt.models.GPModel(optimize_restarts=5, verbose=False, exact_feval=True, ARD=True)\n", "velocity_model = GPyOpt.models.GPModel(optimize_restarts=5, verbose=False, exact_feval=True, ARD=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we sample some input parameters and use the simulator to compute\n", "the outputs. Note that in this case we are not running the full\n", "episodes, we are just using the simulator to compute\n", "$\\inputVector_{t+1}$ given $\\inputVector_{t}$ and $\\textbf{u}_{t}$." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from GPyOpt.experiment_design.random_design import RandomDesign\n", "import mountain_car as mc" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### --- Random locations of the inputs\n", "n_initial_points = 500\n", "random_design_dynamics = RandomDesign(design_space_dynamics)\n", "initial_design_dynamics = random_design_dynamics.get_samples(n_initial_points)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### --- Simulation of the (normalized) outputs\n", "y = np.zeros((initial_design_dynamics.shape[0], 2))\n", "for i in range(initial_design_dynamics.shape[0]):\n", " y[i, :] = mc.simulation(initial_design_dynamics[i, :])\n", "\n", "# Normalize the data from the simulation\n", "y_normalisation = np.std(y, axis=0)\n", "y_normalised = y/y_normalisation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In general we might use much smarter strategies to design our emulation\n", "of the simulator. For example, we could use the variance of the\n", "predictive distributions of the models to collect points using\n", "uncertainty sampling, which will give us a better coverage of the space.\n", "For simplicity, we move ahead with the 500 randomly selected points.\n", "\n", "Now that we have a data set, we can update the emulators for the\n", "location and the velocity." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "position_model.updateModel(initial_design_dynamics, y[:, [0]], None, None)\n", "velocity_model.updateModel(initial_design_dynamics, y[:, [1]], None, None)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now have a look to how the emulator and the simulator match.\n", "First, we show a contour plot of the car aceleration for each pair of\n", "can position and velocity. You can use the bar bellow to play with the\n", "values of the controler to compare the emulator and the simulator." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.html.widgets import interact" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see how the emulator is doing a fairly good job approximating the\n", "simulator. On the edges, however, it struggles to captures the dynamics\n", "of the simulator.\n", "\n", "Given some input parameters of the linear controlling, how do the\n", "dynamics of the emulator and simulator match? In the following figure we\n", "show the position and velocity of the car for the 500 time steps of an\n", "episode in which the parameters of the linear controller have been fixed\n", "beforehand. The value of the input control is also shown." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "controller_gains = np.atleast_2d([0, .6, 1]) # change the valus of the linear controller to observe the trayectories." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#emu-sim-comparison-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#emu-sim-comparison-magnify .magnify onclick=\"magnifyFigure('emu-sim-comparison')\"}\n", "\n", ":::\n", "\n", "::: {#emu-sim-comparison-caption .caption-frame}\n", "Figure: Comparison between the mountain car simulator and the emulator.\n", ":::\n", ":::\n", "\n", "We now make explicit use of the emulator, using it to replace the\n", "simulator and optimize the linear controller. Note that in this\n", "optimization, we don't need to query the simulator anymore as we can\n", "reproduce the full dynamics of an episode using the emulator. For\n", "illustrative purposes, in this example we fix the initial location of\n", "the car.\n", "\n", "We define the objective reward function in terms of the simulator." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### --- Optimize control parameters with emulator\n", "car_initial_location = np.asarray([-0.58912799, 0]) \n", "\n", "### --- Reward objective function using the emulator\n", "obj_func_emulator = lambda x: mc.run_emulation([position_model, velocity_model], x, car_initial_location)[0]\n", "objective_emulator = GPyOpt.core.task.SingleObjective(obj_func_emulator)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And as before, we use Bayesian optimization to find the best possible\n", "linear controller." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### --- Elements of the optimization that will use the multi-fidelity emulator\n", "model = GPyOpt.models.GPModel(optimize_restarts=5, verbose=False, exact_feval=True, ARD=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The design space is the three continuous variables that make up the\n", "linear controller." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "space= [{'name':'linear_1', 'type':'continuous', 'domain':(-1/1.2, +1)},\n", " {'name':'linear_2', 'type':'continuous', 'domain':(-1/0.07, +1/0.07)},\n", " {'name':'constant', 'type':'continuous', 'domain':(-1, +1)}]\n", "\n", "design_space = GPyOpt.Design_space(space=space)\n", "aquisition_optimizer = GPyOpt.optimization.AcquisitionOptimizer(design_space)\n", "\n", "random_design = RandomDesign(design_space)\n", "initial_design = random_design.get_samples(25)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We set the acquisition function to be expected improvement using\n", "`GPyOpt`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "acquisition = GPyOpt.acquisitions.AcquisitionEI(model, design_space, optimizer=aquisition_optimizer)\n", "evaluator = GPyOpt.core.evaluators.Sequential(acquisition)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bo_emulator = GPyOpt.methods.ModularBayesianOptimization(model, design_space, objective_emulator, acquisition, evaluator, initial_design)\n", "bo_emulator.run_optimization(max_iter=50)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "_, _, _, frames = mc.run_simulation(env, np.atleast_2d(bo_emulator.x_opt), render=True)\n", "anim=mc.animate_frames(frames, 'Best controller using the emulator of the dynamics')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.core.display import HTML" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "HTML(anim.to_jshtml())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#mountain-car-emulated-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#mountain-car-emulated-magnify .magnify onclick=\"magnifyFigure('mountain-car-emulated')\"}\n", "\n", ":::\n", "\n", "::: {#mountain-car-emulated-caption .caption-frame}\n", "Figure: Mountain car controller learnt through emulation. Here 500 calls\n", "to the simulator are used to fit the controller rather than 37,500 calls\n", "to the simulator required in the standard learning.\n", ":::\n", ":::\n", "\n", "And the problem is again solved, but in this case we have replaced the\n", "simulator of the car dynamics by a Gaussian process emulator that we\n", "learned by calling the simulator only 500 times. Compared to the 37500\n", "calls that we needed when applying Bayesian optimization directly on the\n", "simulator this is a great gain.\n", "\n", "## Multi-Fidelity Emulation [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "In some scenarios we have simulators of the same environment that have\n", "different fidelities, that is that reflect with different level of\n", "accuracy the dynamics of the real world. Running simulations of the\n", "different fidelities also have a different cost: hight fidelity\n", "simulations are more expensive the cheaper ones. If we have access to\n", "these simulators we can combine high and low fidelity simulations under\n", "the same model.\n", "\n", "So let's assume that we have two simulators of the mountain car\n", "dynamics, one of high fidelity (the one we have used) and another one of\n", "low fidelity. The traditional approach to this form of multi-fidelity\n", "emulation is to assume that\n", "\n", "$$\\mappingFunction_i\\left(\\inputVector\\right) = \\rho\\mappingFunction_{i-1}\\left(\\inputVector\\right) + \\delta_i\\left(\\inputVector \\right)$$\n", "\n", "where $\\mappingFunction_{i-1}\\left(\\inputVector\\right)$ is a low\n", "fidelity simulation of the problem of interest and\n", "$\\mappingFunction_i\\left(\\inputVector\\right)$ is a higher fidelity\n", "simulation. The function $\\delta_i\\left(\\inputVector \\right)$ represents\n", "the difference between the lower and higher fidelity simulation, which\n", "is considered additive. The additive form of this covariance means that\n", "if $\\mappingFunction_{0}\\left(\\inputVector\\right)$ and\n", "$\\left\\{\\delta_i\\left(\\inputVector \\right)\\right\\}_{i=1}^m$ are all\n", "Gaussian processes, then the process over all fidelities of simuation\n", "will be a joint Gaussian process.\n", "\n", "But with Deep Gaussian processes we can consider the form\n", "\n", "$$\\mappingFunction_i\\left(\\inputVector\\right) = \\mappingFunctionTwo_{i}\\left(\\mappingFunction_{i-1}\\left(\\inputVector\\right)\\right) + \\delta_i\\left(\\inputVector \\right),$$\n", "\n", "where the low fidelity representation is non linearly transformed by\n", "$\\mappingFunctionTwo(\\cdot)$ before use in the process. This is the\n", "approach taken in @Perdikaris:multifidelity17. But once we accept that\n", "these models can be composed, a highly flexible framework can emerge. A\n", "key point is that the data enters the model at different levels, and\n", "represents different aspects. For example these correspond to the two\n", "fidelities of the mountain car simulator.\n", "\n", "We start by sampling both of them at 250 random input locations." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import gym" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "env = gym.make('MountainCarContinuous-v0')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import GPyOpt" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### --- Collect points from low and high fidelity simulator --- ###\n", "\n", "space = GPyOpt.Design_space([\n", " {'name':'position', 'type':'continuous', 'domain':(-1.2, +1)},\n", " {'name':'velocity', 'type':'continuous', 'domain':(-0.07, +0.07)},\n", " {'name':'action', 'type':'continuous', 'domain':(-1, +1)}])\n", "\n", "n_points = 250\n", "random_design = GPyOpt.experiment_design.RandomDesign(space)\n", "x_random = random_design.get_samples(n_points)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we evaluate the high and low fidelity simualtors at those\n", "locations." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import mountain_car as mc" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "d_position_hf = np.zeros((n_points, 1))\n", "d_velocity_hf = np.zeros((n_points, 1))\n", "d_position_lf = np.zeros((n_points, 1))\n", "d_velocity_lf = np.zeros((n_points, 1))\n", "\n", "# --- Collect high fidelity points\n", "for i in range(0, n_points):\n", " d_position_hf[i], d_velocity_hf[i] = mc.simulation(x_random[i, :])\n", "\n", "# --- Collect low fidelity points \n", "for i in range(0, n_points):\n", " d_position_lf[i], d_velocity_lf[i] = mc.low_cost_simulation(x_random[i, :])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is time to build the multi-fidelity model for both the position and\n", "the velocity.\n", "\n", "As we did in the previous section we use the emulator to optimize the\n", "simulator. In this case we use the high fidelity output of the emulator." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### --- Optimize controller parameters \n", "obj_func = lambda x: mc.run_simulation(env, x)[0]\n", "obj_func_emulator = lambda x: mc.run_emulation([position_model, velocity_model], x, car_initial_location)[0]\n", "objective_multifidelity = GPyOpt.core.task.SingleObjective(obj_func)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And we optimize using Bayesian optimzation." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from GPyOpt.experiment_design.random_design import RandomDesign" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model = GPyOpt.models.GPModel(optimize_restarts=5, verbose=False, exact_feval=True, ARD=True)\n", "space= [{'name':'linear_1', 'type':'continuous', 'domain':(-1/1.2, +1)},\n", " {'name':'linear_2', 'type':'continuous', 'domain':(-1/0.07, +1/0.07)},\n", " {'name':'constant', 'type':'continuous', 'domain':(-1, +1)}]\n", "\n", "design_space = GPyOpt.Design_space(space=space)\n", "aquisition_optimizer = GPyOpt.optimization.AcquisitionOptimizer(design_space)\n", "\n", "n_initial_points = 25\n", "random_design = RandomDesign(design_space)\n", "initial_design = random_design.get_samples(n_initial_points)\n", "acquisition = GPyOpt.acquisitions.AcquisitionEI(model, design_space, optimizer=aquisition_optimizer)\n", "evaluator = GPyOpt.core.evaluators.Sequential(acquisition)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bo_multifidelity = GPyOpt.methods.ModularBayesianOptimization(model, design_space, objective_multifidelity, acquisition, evaluator, initial_design)\n", "bo_multifidelity.run_optimization(max_iter=50)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "_, _, _, frames = mc.run_simulation(env, np.atleast_2d(bo_multifidelity.x_opt), render=True)\n", "anim=mc.animate_frames(frames, 'Best controller with multi-fidelity emulator')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.core.display import HTML" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "HTML(anim.to_jshtml())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Best Controller with Multi-Fidelity Emulator\n", "\n", "::: {.figure}\n", "::: {#mountain-car-multi-fidelity-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#mountain-car-multi-fidelity-magnify .magnify onclick=\"magnifyFigure('mountain-car-multi-fidelity')\"}\n", "\n", ":::\n", "\n", "::: {#mountain-car-multi-fidelity-caption .caption-frame}\n", "Figure: Mountain car learnt with multi-fidelity model. Here 250\n", "observations of the high fidelity simulator and 250 observations of the\n", "low fidelity simulator are used to learn the controller.\n", ":::\n", ":::\n", "\n", "And problem solved! We see how the problem is also solved with 250\n", "observations of the high fidelity simulator and 250 of the low fidelity\n", "simulator.\n", "\n", "## Emukit Playground [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "Emukit playground is a software toolkit for exploring the use of\n", "statistical emulation as a tool. It was built by [Adam\n", "Hirst](https://twitter.com/_AdamHirst), during his software engineering\n", "internship at Amazon and supervised by Cliff McCollum.\n", "\n", "::: {.figure}\n", "::: {#emukit-playground-figure .figure-frame}\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#emukit-playground-magnify .magnify onclick=\"magnifyFigure('emukit-playground')\"}\n", "\n", ":::\n", "\n", "::: {#emukit-playground-caption .caption-frame}\n", "Figure: Emukit playground is a tutorial for understanding the\n", "simulation/emulation relationship.\n", "\n", ":::\n", ":::\n", "\n", "::: {.figure}\n", "::: {#emukit-playground-bayes-opt-figure .figure-frame}\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#emukit-playground-bayes-opt-magnify .magnify onclick=\"magnifyFigure('emukit-playground-bayes-opt')\"}\n", "\n", ":::\n", "\n", "::: {#emukit-playground-bayes-opt-caption .caption-frame}\n", "Figure: Tutorial on Bayesian optimization of the number of taxis\n", "deployed from Emukit playground.\n", "\n", ":::\n", ":::\n", "\n", "You can explore Bayesian optimization of a taxi simulation.\n", "\n", "## Emukit [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "::: {.figure}\n", "::: {#emukit-software-page-figure .figure-frame}\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#emukit-software-page-magnify .magnify onclick=\"magnifyFigure('emukit-software-page')\"}\n", "\n", ":::\n", "\n", "::: {#emukit-software-page-caption .caption-frame}\n", "Figure: The Emukit software is a set of software tools for emulation and\n", "surrogate modeling. \n", ":::\n", ":::\n", "\n", "The aim is to provide a suite where different approaches to emulation\n", "are assimilated under one roof. The current version of Emukit includes\n", "*multi-fidelity emulation* for build surrogate models when data is\n", "obtained from multiple information sources that have different fidelity\n", "and/or cost; *Bayesian optimisation* for optimising physical experiments\n", "and tune parameters of machine learning algorithms or other\n", "computational simulations; *experimental design and active learning*:\n", "design the most informative experiments and perform active learning with\n", "machine learning models; *sensitivity analysis*: analyse the influence\n", "of inputs on the outputs of a given system; and *Bayesian quadrature*:\n", "efficiently compute the integrals of functions that are expensive to\n", "evaluate.\n", "\n", "{For monitoring systems in production, emulation needn't just be about\n", "simulator models. What we envisage, is that even data driven models\n", "could be emulated. This is important for understanding system behaviour,\n", "how the different components are interconnected. This drives the notion\n", "of the *information dynamics* of the machine learning system. What is\n", "the effect of one particular intervention in the wider system? One way\n", "of answering this is through emulation. But it requires that our machine\n", "learning models (and our simulators) are deployed in an environment\n", "where emulation can be automatically deployed. The resulting system\n", "would allow us to monitor the downstream effects of indivdiual decision\n", "making on the wider system.\n", "\n", "You can also check my blog post on [\"New Directions in Kernels and\n", "Gaussian\n", "Processes\"](http://inverseprobability.com/2016/11/29/new-directions-in-kernels-and-gaussian-processes)\n", "\n", "# Deep Gaussian Processes\n", "\n", "One challenge is developing flexible enough models to perform the\n", "emulation that also propagate uncertainty through the model. One\n", "candidate set of models for this challenge is *deep Gaussian processes*\n", "(DGPs). For the remainder of these notes we introduce the theory behind\n", "DGPs.\n", "\n", "While there are some difficulties in algorithmically implementing these\n", "algorithms at scale, they are mathematically far simpler than the\n", "equivalent neural network models, and perhaps as a result offer greater\n", "promise for theoretical understanding of deep learning [see e.g.\n", "@Dunlop:deep2017]." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import teaching_plots as plot" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#deep-nn-bottleneck-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#deep-nn-bottleneck-magnify .magnify onclick=\"magnifyFigure('deep-nn-bottleneck')\"}\n", "\n", ":::\n", "\n", "::: {#deep-nn-bottleneck-caption .caption-frame}\n", "Figure: Inserting the bottleneck layers introduces a new set of\n", "variables.\n", ":::\n", ":::\n", "\n", "Including the low rank decomposition of $\\mappingMatrix$ in the neural\n", "network, we obtain a new mathematical form. Effectively, we are adding\n", "additional *latent* layers, $\\latentVector$, in between each of the\n", "existing hidden layers. In a neural network these are sometimes known as\n", "*bottleneck* layers. The network can now be written mathematically as $$\n", "\\begin{align}\n", " \\latentVector_{1} &= \\eigenvectwoMatrix^\\top_1 \\inputVector\\\\\n", " \\hiddenVector_{1} &= \\basisFunction\\left(\\eigenvectorMatrix_1 \\latentVector_{1}\\right)\\\\\n", " \\latentVector_{2} &= \\eigenvectwoMatrix^\\top_2 \\hiddenVector_{1}\\\\\n", " \\hiddenVector_{2} &= \\basisFunction\\left(\\eigenvectorMatrix_2 \\latentVector_{2}\\right)\\\\\n", " \\latentVector_{3} &= \\eigenvectwoMatrix^\\top_3 \\hiddenVector_{2}\\\\\n", " \\hiddenVector_{3} &= \\basisFunction\\left(\\eigenvectorMatrix_3 \\latentVector_{3}\\right)\\\\\n", " \\dataVector &= \\mappingVector_4^\\top\\hiddenVector_{3}.\n", "\\end{align}\n", "$$\n", "\n", "$$\n", "\\begin{align}\n", " \\latentVector_{1} &= \\eigenvectwoMatrix^\\top_1 \\inputVector\\\\\n", " \\latentVector_{2} &= \\eigenvectwoMatrix^\\top_2 \\basisFunction\\left(\\eigenvectorMatrix_1 \\latentVector_{1}\\right)\\\\\n", " \\latentVector_{3} &= \\eigenvectwoMatrix^\\top_3 \\basisFunction\\left(\\eigenvectorMatrix_2 \\latentVector_{2}\\right)\\\\\n", " \\dataVector &= \\mappingVector_4 ^\\top \\latentVector_{3}\n", "\\end{align}\n", "$$\n", "\n", "Now if we replace each of these neural networks with a Gaussian process.\n", "This is equivalent to taking the limit as the width of each layer goes\n", "to infinity, while appropriately scaling down the outputs.\n", "\n", "$$\n", "\\begin{align}\n", " \\latentVector_{1} &= \\mappingFunctionVector_1\\left(\\inputVector\\right)\\\\\n", " \\latentVector_{2} &= \\mappingFunctionVector_2\\left(\\latentVector_{1}\\right)\\\\\n", " \\latentVector_{3} &= \\mappingFunctionVector_3\\left(\\latentVector_{2}\\right)\\\\\n", " \\dataVector &= \\mappingFunctionVector_4\\left(\\latentVector_{3}\\right)\n", "\\end{align}\n", "$$\n", "\n", "## Olympic Marathon Data [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n", "- Gold medal times for Olympic Marathon since 1896.\n", "- Marathons before 1924 didn't have a standardised distance.\n", "- Present results using pace per km.\n", "- In 1904 Marathon was badly organised leading to very slow times.\n", "\n", "\n", "::: {.centered .centered style=\"\"}\n", "\n", ":::\n", "\n", "Image from Wikimedia Commons \n", "
\n", "The first thing we will do is load a standard data set for regression\n", "modelling. The data consists of the pace of Olympic Gold Medal Marathon\n", "winners for the Olympics from 1896 to present. First we load in the data\n", "and plot." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pods" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "data = pods.datasets.olympic_marathon_men()\n", "x = data['X']\n", "y = data['Y']\n", "\n", "offset = y.mean()\n", "scale = np.sqrt(y.var())" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import teaching_plots as plot\n", "import mlai" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\n", "xlim = (1875,2030)\n", "ylim = (2.5, 6.5)\n", "yhat = (y-offset)/scale\n", "\n", "fig, ax = plt.subplots(figsize=plot.big_wide_figsize)\n", "_ = ax.plot(x, y, 'r.',markersize=10)\n", "ax.set_xlabel('year', fontsize=20)\n", "ax.set_ylabel('pace min/km', fontsize=20)\n", "ax.set_xlim(xlim)\n", "ax.set_ylim(ylim)\n", "\n", "mlai.write_figure(figure=fig, \n", " filename='../slides/diagrams/datasets/olympic-marathon.svg', \n", " transparent=True, \n", " frameon=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#olympic-marathon-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-magnify .magnify onclick=\"magnifyFigure('olympic-marathon')\"}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-caption .caption-frame}\n", "Figure: Olympic marathon pace times since 1892.\n", ":::\n", ":::\n", "\n", "Things to notice about the data include the outlier in 1904, in this\n", "year, the olympics was in St Louis, USA. Organizational problems and\n", "challenges with dust kicked up by the cars following the race meant that\n", "participants got lost, and only very few participants completed.\n", "\n", "More recent years see more consistently quick marathons.\n", "\n", "## Alan Turing [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "::: {.figure}\n", "::: {#turing-run-times-figure .figure-frame}\n", "\n", "\n", "\n", "\n", "\n", "
\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", "\n", "\n", "::: {.centered .centered style=\"\"}\n", "\n", ":::\n", "\n", "
\n", ":::\n", "\n", "::: {#turing-run-times-magnify .magnify onclick=\"magnifyFigure('turing-run-times')\"}\n", "\n", ":::\n", "\n", "::: {#turing-run-times-caption .caption-frame}\n", "Figure: Alan Turing, in 1946 he was only 11 minutes slower than the\n", "winner of the 1948 games. Would he have won a hypothetical games held in\n", "1946? Source: [Alan Turing Internet\n", "Scrapbook](http://www.turing.org.uk/scrapbook/run.html).\n", ":::\n", ":::\n", "\n", "If we had to summarise the objectives of machine learning in one word, a\n", "very good candidate for that word would be *generalization*. What is\n", "generalization? From a human perspective it might be summarised as the\n", "ability to take lessons learned in one domain and apply them to another\n", "domain. If we accept the definition given in the first session for\n", "machine learning, $$\n", "\\text{data} + \\text{model} \\xrightarrow{\\text{compute}} \\text{prediction}\n", "$$ then we see that without a model we can't generalise: we only have\n", "data. Data is fine for answering very specific questions, like \"Who won\n", "the Olympic Marathon in 2012?\", because we have that answer stored,\n", "however, we are not given the answer to many other questions. For\n", "example, Alan Turing was a formidable marathon runner, in 1946 he ran a\n", "time 2 hours 46 minutes (just under four minutes per kilometer, faster\n", "than I and most of the other [Endcliffe Park\n", "Run](http://www.parkrun.org.uk/sheffieldhallam/) runners can do 5 km).\n", "What is the probability he would have won an Olympics if one had been\n", "held in 1946?\n", "\n", "To answer this question we need to generalize, but before we formalize\n", "the concept of generalization let's introduce some formal representation\n", "of what it means to generalize in machine learning.\n", "\n", "Our first objective will be to perform a Gaussian process fit to the\n", "data, we'll do this using the [GPy\n", "software](https://github.com/SheffieldML/GPy)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import GPy" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "m_full = GPy.models.GPRegression(x,yhat)\n", "_ = m_full.optimize() # Optimize parameters of covariance function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first command sets up the model, then `m_full.optimize()` optimizes\n", "the parameters of the covariance function and the noise level of the\n", "model. Once the fit is complete, we'll try creating some test points,\n", "and computing the output of the GP model in terms of the mean and\n", "standard deviation of the posterior functions between 1870 and 2030. We\n", "plot the mean function and the standard deviation at 200 locations. We\n", "can obtain the predictions using `y_mean, y_var = m_full.predict(xt)`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "xt = np.linspace(1870,2030,200)[:,np.newaxis]\n", "yt_mean, yt_var = m_full.predict(xt)\n", "yt_sd=np.sqrt(yt_var)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we plot the results using the helper function in `teaching_plots`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import teaching_plots as plot" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig, ax = plt.subplots(figsize=plot.big_wide_figsize)\n", "plot.model_output(m_full, scale=scale, offset=offset, ax=ax, xlabel='year', ylabel='pace min/km', fontsize=20, portion=0.2)\n", "ax.set_xlim(xlim)\n", "ax.set_ylim(ylim)\n", "mlai.write_figure(figure=fig,\n", " filename='../slides/diagrams/gp/olympic-marathon-gp.svg', \n", " transparent=True, frameon=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#olympic-marathon-gp-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-gp-magnify .magnify onclick=\"magnifyFigure('olympic-marathon-gp')\"}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-gp-caption .caption-frame}\n", "Figure: Gaussian process fit to the Olympic Marathon data. The error\n", "bars are too large, perhaps due to the outlier from 1904.\n", ":::\n", ":::\n", "\n", "## Fit Quality\n", "\n", "In the fit we see that the error bars (coming mainly from the noise\n", "variance) are quite large. This is likely due to the outlier point in\n", "1904, ignoring that point we can see that a tighter fit is obtained. To\n", "see this making a version of the model, `m_clean`, where that point is\n", "removed." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "x_clean=np.vstack((x[0:2, :], x[3:, :]))\n", "y_clean=np.vstack((y[0:2, :], y[3:, :]))\n", "\n", "m_clean = GPy.models.GPRegression(x_clean,y_clean)\n", "_ = m_clean.optimize()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Deep GP Fit [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "Let's see if a deep Gaussian process can help here. We will construct a\n", "deep Gaussian process with one hidden layer (i.e. one Gaussian process\n", "feeding into another).\n", "\n", "Build a Deep GP with an additional hidden layer (one dimensional) to fit\n", "the model." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import GPy\n", "import deepgp" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "hidden = 1\n", "m = deepgp.DeepGP([y.shape[1],hidden,x.shape[1]],Y=yhat, X=x, inits=['PCA','PCA'], \n", " kernels=[GPy.kern.RBF(hidden,ARD=True),\n", " GPy.kern.RBF(x.shape[1],ARD=True)], # the kernels for each layer\n", " num_inducing=50, back_constraint=False)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import deepgp" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Call the initalization\n", "m.initialize()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now optimize the model." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "for layer in m.layers:\n", " layer.likelihood.variance.constrain_positive(warning=False)\n", "m.optimize(messages=True,max_iters=10000)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "m.staged_optimize(messages=(True,True,True))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Olympic Marathon Data Deep GP\n", "\n", "::: {.figure}\n", "::: {#olympic-marathon-deep-gp-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-magnify .magnify onclick=\"magnifyFigure('olympic-marathon-deep-gp')\"}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-caption .caption-frame}\n", "Figure: Deep GP fit to the Olympic marathon data. Error bars now change\n", "as the prediction evolves.\n", ":::\n", ":::" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig, ax = plt.subplots(figsize=plot.big_wide_figsize)\n", "plot.model_sample(m, scale=scale, offset=offset, samps=10, ax=ax, \n", " xlabel='year', ylabel='pace min/km', portion = 0.225)\n", "ax.set_xlim(xlim)\n", "ax.set_ylim(ylim)\n", "mlai.write_figure(figure=fig, filename='../slides/diagrams/deepgp/olympic-marathon-deep-gp-samples.svg', \n", " transparent=True, frameon=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Olympic Marathon Data Deep GP\n", "\n", "::: {.figure}\n", "::: {#olympic-marathon-deep-gp-samples-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-samples-magnify .magnify onclick=\"magnifyFigure('olympic-marathon-deep-gp-samples')\"}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-samples-caption .caption-frame}\n", "Figure: Point samples run through the deep Gaussian process show the\n", "distribution of output locations.\n", ":::\n", ":::\n", "\n", "## Fitted GP for each layer\n", "\n", "Now we explore the GPs the model has used to fit each layer. First of\n", "all, we look at the hidden layer." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "m.visualize(scale=scale, offset=offset, xlabel='year',\n", " ylabel='pace min/km',xlim=xlim, ylim=ylim,\n", " dataset='olympic-marathon',\n", " diagrams='../slides/diagrams/deepgp')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pods" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pods.notebook.display_plots('olympic-marathon-deep-gp-layer-{sample:0>1}.svg', \n", " '../slides/diagrams/deepgp', sample=(0,1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "::: {.figure}\n", "::: {#olympic-marathon-deep-gp-layer-0-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-layer-0-magnify .magnify onclick=\"magnifyFigure('olympic-marathon-deep-gp-layer-0')\"}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-layer-0-caption .caption-frame}\n", "Figure: The mapping from input to the latent layer is broadly, with some\n", "flattening as time goes on. Variance is high across the input range.\n", ":::\n", ":::\n", "\n", "::: {.figure}\n", "::: {#olympic-marathon-deep-gp-layer-1-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-layer-1-magnify .magnify onclick=\"magnifyFigure('olympic-marathon-deep-gp-layer-1')\"}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-layer-1-caption .caption-frame}\n", "Figure: The mapping from the latent layer to the output layer.\n", ":::\n", ":::" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig, ax = plt.subplots(figsize=plot.big_wide_figsize)\n", "m.visualize_pinball(ax=ax, scale=scale, offset=offset, points=30, portion=0.1,\n", " xlabel='year', ylabel='pace km/min', vertical=True)\n", "mlai.write_figure(figure=fig, filename='../slides/diagrams/deepgp/olympic-marathon-deep-gp-pinball.svg', \n", " transparent=True, frameon=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Olympic Marathon Pinball Plot\n", "\n", "::: {.figure}\n", "::: {#olympic-marathon-deep-gp-pinball-figure .figure-frame}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-pinball-magnify .magnify onclick=\"magnifyFigure('olympic-marathon-deep-gp-pinball')\"}\n", "\n", ":::\n", "\n", "::: {#olympic-marathon-deep-gp-pinball-caption .caption-frame}\n", "Figure: A pinball plot shows the movement of the 'ball' as it passes\n", "through each layer of the Gaussian processes. Mean directions of\n", "movement are shown by lines. Shading gives one standard deviation of\n", "movement position. At each layer, the uncertainty is reset. The overal\n", "uncertainty is the cumulative uncertainty from all the layers. There is\n", "some grouping of later points towards the right in the first layer,\n", "which also injects a large amount of uncertainty. Due to flattening of\n", "the curve in the second layer towards the right the uncertainty is\n", "reduced in final output.\n", ":::\n", ":::\n", "\n", "The pinball plot shows the flow of any input ball through the deep\n", "Gaussian process. In a pinball plot a series of vertical parallel lines\n", "would indicate a purely linear function. For the olypmic marathon data\n", "we can see the first layer begins to shift from input towards the right.\n", "Note it also does so with some uncertainty (indicated by the shaded\n", "backgrounds). The second layer has less uncertainty, but bunches the\n", "inputs more strongly to the right. This input layer of uncertainty,\n", "followed by a layer that pushes inputs to the right is what gives the\n", "heteroschedastic noise.\n", "\n", "## MXFusion: Modular Probabilistic Programming on MXNet [\\[[edit]{.editsection style=\"\"}\\]]{.editsection-bracket style=\"\"}\n", "\n", "One challenge for practitioners in Gaussian processes, is flexible\n", "software that allows the construction of the relevant GP modle. With\n", "this in mind, the Amazon Cambridge team has developed MXFusion. It is a\n", "modular probabilistic programming language focussed on efficient\n", "implementation of hybrid GP-neural network models, but with additional\n", "probabilistic programming capabilities.\n", "\n", "::: {.figure}\n", "::: {#mxfusion-software-figure .figure-frame}\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", ":::\n", "\n", "::: {#mxfusion-software-magnify .magnify onclick=\"magnifyFigure('mxfusion-software')\"}\n", "\n", ":::\n", "\n", "::: {#mxfusion-software-caption .caption-frame}\n", "Figure: MXFusion is a probabilistic programming language targeted\n", "specifically at Gaussian process models and combining them with\n", "probaiblistic neural network. It is available through the MIT license\n", "and we welcome contributions throguh the Github repository\n", ".\n", ":::\n", ":::\n", "\n", "We developed the framework for greater ease of transitioning models from\n", "'science' to 'production', our aim was to have code that could be\n", "created by scientists, but deployed in our systems through solutions\n", "such as AWS SageMaker.\n", "\n", "::: {.figure}\n", "::: {#mxfusion-software-logo-figure .figure-frame}\n", "\n", "\n", "\n", "\n", "\n", "
\n", "- Work by Eric Meissner and Zhenwen Dai.\n", "- Probabilistic programming.\n", "- Available on [Github](https://github.com/amzn/mxfusion)\n", "\n", "\n", "::: {.centered style=\"\"}\n", "\n", ":::\n", "\n", "
\n", ":::\n", "\n", "::: {#mxfusion-software-logo-magnify .magnify onclick=\"magnifyFigure('mxfusion-software-logo')\"}\n", "\n", ":::\n", "\n", "::: {#mxfusion-software-logo-caption .caption-frame}\n", "Figure: The MXFusion software.\n", ":::\n", ":::\n", "\n", "## Conclusion\n", "\n", "Machine learning models are deployed as components in an interacting\n", "system to achieve modern AI. Some of those components are inspired by a\n", "mechanistic understanding of the world around us (e.g. economic or\n", "physical understanding).\n", "\n", "Meta modelling involves fiting machine learning models to existing\n", "systems to improve speed and interpretability.\n", "\n", "Deep Gaussian processes are a flexible approach to meta modelling, which\n", "provide the necessary uncertainty estimates and the potential for being\n", "more mathematically tractable.\n", "\n", "# References {#references .unnumbered}\n", "\n", "[^1]: Apple has fixed this issue so that Siri no longer does this." ] } ], "metadata": {}, "nbformat": 4, "nbformat_minor": 2 }