{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Basis Functions\n", "\n", "## Data Science School, Nyeri, Kenya\n", "\n", "### 15th June 2015 Neil Lawrence\n", "\n", "We've now seen how we may perform linear regression. Now, we are going to consider how we can perform *non-linear* regression. However, before we get into the details of how to do that we first need to consider in what ways the regression can be non-linear. \n", "\n", "Multivariate linear regression allows us to build models that take many features into account when making our prediction. In this session we are going to introduce *basis functions*. The term seems complicted, but they are actually based on rather a simple idea. If we are doing a multivariate linear regression, we get extra features that *might* help us predict our required response varible (or target value), $y$. But what if we only have one input value? We can actually artificially generate more input values with basis functions.\n", "\n", "## Non-linear in the Inputs\n", "\n", "When we refer to non-linear regression, we are normally referring to whether the regression is non-linear in the input space, or non-linear in the *covariates*. The covariates are the observations that move with the target (or *response*) variable. In our notation we have been using $\\mathbf{x}_i$ to represent a vector of the covariates associated with the $i$th observation. The coresponding response variable is $y_i$. If a model is non-linear in the inputs, it means that there is a non-linear function between the inputs and the response variable. Linear functions are functions that only involve multiplication and addition, in other words they can be represented through *linear algebra*. Linear regression involves assuming that a function takes the form\n", "$$\n", "f(\\mathbf{x}) = \\mathbf{w}^\\top \\mathbf{x}\n", "$$\n", "where $\\mathbf{w}$ are our regression weights. A very easy way to make the linear regression non-linear is to introduce non-linear functions. When we are introducing non-linear regression these functions are known as *basis functions*.\n", "\n", "### Basis Functions\n", "\n", "Here's the idea, instead of working directly on the original input space, $\\mathbf{x}$, we build models in a new space, $\\boldsymbol{\\phi}(\\mathbf{x})$ where $\\boldsymbol{\\phi}(\\cdot)$ is a *vector valued* function that is defined on the space $\\mathbf{x}$. \n", "\n", "Remember, that a vector valued function is just a vector that contains functions instead of values. Here's an example for a one dimensional input space, $x$, being projected to a *quadratic* basis. First we consider each basis function in turn, we can think of the elements of our vector as being indexed so that we have\n", "\\begin{align*}\n", "\\phi_1(x) = 1, \\\\\n", "\\phi_2(x) = x, \\\\\n", "\\phi_3(x) = x^2.\n", "\\end{align*}\n", "Now we can consider them together by placing them in a vector,\n", "$$\n", "\\boldsymbol{\\phi}(x) = \\begin{bmatrix} 1\\\\ x \\\\ x^2\\end{bmatrix}.\n", "$$\n", "This is the idea of the vector valued function, we have simply collected the different functions together in the same vector making them notationally easier to deal with in our mathematics. \n", "\n", "When we consider the vector valued function for each data point, then we place all the data into a matrix. The result is a matrix valued function,\n", "$$\n", "\\boldsymbol{\\Phi}(\\mathbf{x}) = \n", "\\begin{bmatrix} 1 & x_1 & x_1^2 \\\\\n", "1 & x_2 & x_2^2\\\\\n", "\\vdots & \\vdots & \\vdots \\\\\n", "1 & x_n & x_n^2\n", "\\end{bmatrix}\n", "$$\n", "where we are still in the one dimensional input setting so $\\mathbf{x}$ here represents a vector of our inputs with $n$ elements. \n", "\n", "Let's try constructing such a matrix for a set of inputs. First of all, we create a function that returns the matrix valued function" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import numpy as np # import numpy for the arrays.\n", "\n", "def quadratic(x):\n", " \"\"\"Take in a vector of input values and return the design matrix associated \n", " with the basis functions.\"\"\"\n", " return np.hstack([np.ones((n, 1)), x, x**2])\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This function takes in an $n\\times 1$ dimensional vector and returns an $n\\times 3$ dimensional *design matrix* containing the basis functions. We can plot those basis functions against there input as follows." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAEKCAYAAADpSmgQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYXFWd//H3lySICYR9CQaIDEHEkX2JiBAQJCyKjIjg\nKMI4I+qoM4qDC/MbaEeN4o64xnVQBFxwIpKEbSLgAihBZQmCMbIHkC0QlxC+vz/Obbq6091VvVZ1\n1/v1PPV0V9ete091V9/61KnzPScyE0mSJEl9W6fZDZAkSZJanaFZkiRJqsPQLEmSJNVhaJYkSZLq\nMDRLkiRJdRiaJUmSpDoMzZIkSVIdhmZJbSsiZkTE0xExbOfCiHhfRMwbrv0NVUR8ISL+s9ntGKiI\n+MeIWNTsdkhSp3BxE0mtJCJOAk4FtgceBy4C3peZj43AsWYAy4CJmfn0IO4/Gzg3M7cZhrYsBvYF\nngLWAL8G/jUzbxrqvgfZnqeBVUDni8TqzNxkhI41gyH8HSRpNNjTLKllRMSpwEcooXkqMAvYDrgs\nIiaNclsiImIUD5mUkLwBsAmwGDh3FI/fm10yc4PqMiKBuYfR/H1L0oAYmiW1hIiYCpwJvC0zL83M\nNZn5R+A4YAbwumq7b0TEf9fcb3ZE3FVz/b0RcUdEPB4RN0fEK2tuWyciPh4RD0bE74Eje7RhcUR8\nMCJ+CjwJbB8RJ0fELdX+fh8Rb6q2nQIsALaOiJXV7dMi4syIOLdmn/tHxM8i4pGIuDMi3lDvd1H1\ntl4A7Fyzn30i4ufVfu6NiM/WvpGIiE9FxIqIeCwifhMRO/f8fUXEZhFxcbWPP0XEVQN9Y1ANZ9m+\n5nrt/mdHxN0R8a6qLfdWnxx0bvvsiPhERCyPiEer468HXFVt8mj1e5wVESdFxNU1990vIq6v7ndd\nRLyo5rbFEfGBiLimuv+iiNi0um29iPhWRDxUPe7rImKLgTxmSQJDs6TWsR+wHvCD2h9m5pPAJcAh\nnT+ia8hAb+4A9s/MqUAH8K2I2LK67U2UoLwbsBdwbC/7eh3wz8D6wB+BFcCR1f5OBj4VEbtX7ZoD\n3Fv1xE7NzPtq9xcR21Vt/wywWXXcG/tpe1T3Wxf4R+DnNbc9BfwbsCnwIuClwFur7Q8DXgLMzMwN\ngVcDD/fy+zoVuKtqyxaUYS/9/S4bCdQ9/x5bUj4l2Bp4I/C5iNiwuu3jwO5V+zcBTgOertoOsGH1\ne/xFt0ZEbAL8GPh0db9PAj+OiI1rNjsBOKl6XOsC765+/oaqPdOr+54C/LmBxyVJ3RiaJbWKzYCH\n+hjTej8lLHbqM8xl5vcy8/7q+wuB24F9qpuPAz6Vmfdk5iPAh3vsK4FvZOatmfl0Zj6VmZdk5h+q\n/V0FXEpXyOutHbU/ey1wWWZeUPWcP5yZv+6j6QGcHRGPUMZyvxX4QM3juiEzr6va9Ufgy8CB1c2r\ngQ2A50fEOpl5W+fvoIe/AdOAGVV7ftpHWzrdUPXOPhIRn+5nu9rHvBr4QLX/BcATwPOqYsuTgX/L\nzPuqx/GLzPwb9cP5kcBtmfnt6n7nA0uBV1S3J/D1zLwjM/8CXEh5g9L5mDelvKHIzFySmSvrHE+S\n1mJoltQqHgI262Mmi2mUHt+6IuLEiFjSGfaAv6cE8s793FWz+Z297KL2diLi8Ij4RTWc4RHgCLoH\n+P5sQylwa0QCb8/MjTNzPeDlwPci4oVVO3ashlbcFxGPAR/qbEdmXgmcA3wOWBERX4qIDWofRvX1\nY5Se+EuroSbvqdOm3av2bJyZ/97g4/hTjzc+qyi99ptRPkn4fYP7qbU1a/+t/lj9vFPtm4Q/V8eE\nMi58EXB+RNwTER+NiImDaIOkNmdoltQqfg78FXhV7Q8jYn3KMIhLqx89CUyu2WSrmm23o/TA/iuw\nSWZuDNxEV2i8D9i25r6133eqHV7xLOD7wFnAFtX+LqnZX73ph+4E/q7ONr3KzGsoAffQ6kdfAG4B\ndqiGYJxOzTk8Mz+bmXtRxkHvCPxHL/t8IjPfnZl/R+mlfVdEHDzApq2i++9/GvV/D1DeFP0F2KGX\n2+rd/x5KQWit7aqf96v6tOADmfkCyhCgo4AT6zdXkrozNEtqCdWUch3AZyPisIiYFGUqsgspvZMX\nVJveCBwRERtHxFZAbQ/oFEoAewhYJyJOpvQ0d7oQeEdEPKcaD/veXppSO1Rg3eryEPB0RBwOvKzm\n9hXAplGKGHtzHnBIRLw6IiZGxKYRsWs/v4Znjl0Vuu0M3Fz9aH1gJbAqInYC3lI9ViJir4jYtyoM\nXEUJp2t62edREbFDVfz3eLVN53aNuhH4x4iYEBFzgAMauVPV+/w14JNRCiYnRMSLqvHbD1LGNvf1\nBmMBsGNEnFD9Hl8D7ARcXLNNr0M8IuKgiHhhREyg/P5WM/DHLEmGZkmtIzM/BryfUjD2OGVoQwJz\nMvOparNzKXMYLwcWAudX25CZtwCfoPRa308JzNfUHGIe5aP6XwO/pPQi9+zlfOZ6Nfb1HZSw/TCl\n2Ox/a25fCnwHWBYRD0dEZ69rZ3vupAznOBX4E7AE2KWfX8E5UWbiWAn8D3B6ZnYu8PFuyhjpxym9\n6efX3G9q9bOHq9/LQ5ShGJ2Pp/Mx7QBcRgmPPwM+l5k/6aMtffX+/htl6MgjVXsuavB+nY/ht8D1\nlN/HXMp6Aasow01+Wv0e96X77/FPlB7iU6vH9m7gqMx8uGbf2eP7zutbAt8FHqP01C+m+VP5SRqD\nXNxEUsuqpiv7KPCizGx0bLAkScPO0CyppUXE64A1mfmdZrdFktS+DM2SJElSHY5pliRJkupombkq\nI8Iub0mSJI24zGxkxdNuWiY0w+AegDTSIuLMzDyz2e2QeuPzU63K56Za1WA7ah2eIUmSJNVhaJYk\nSZLqMDRL9S1udgOkfixudgOkPixudgOk4dQyU85FRDqmWZIkSSNpsJnTnmZJkiSpDkOzJEmSVIeh\nWZIkSarD0CxJkiTVYWiWJEmS6jA0S5IkSXUYmiVJkqQ6DM2SJElSHUMOzRHxtYhYERG/7WebsyPi\n9oj4dUTsPtRjSpIkSaNpOHqavw7M6evGiDgC2CEzZwJvAr4wDMeUJEmSRs2QQ3NmXg080s8mrwC+\nWW17LbBRRGw51ONKkiRJo2U0xjQ/B7ir5vrdwPRROK4kSZL0jDjxZc8d7H0nDmdD+hE9rmdvG708\n9sjduZF1SGYDs0e6VZIkSRq3/joBztkcvv9suOnpXeG+g4DLBrWv0QjN9wDb1FyfXv1sLRdzw1UX\nw6bAezrgkszew7UkSZLUU3READsCh1WXl3DzsXez4DOb8NTkB2Cdt8Knrx7MvkcjNM8H3gacHxGz\ngEczc0Uf284GXg58HDg1gv/I5Fej0EZJkiSNQdERGwIHU0LyHGACsIhfvun/WHD2s1nzrOnAW4GL\nMsnoOf6h0eNkDq0zNyK+AxwIbAasAM4AJgFk5peqbc6pHsSTwMmZeUMv+8nMjPI9E4E3AmcCVwL/\nmckfhtRQSZIkjXnREesAe9LVm7wb8DNgEbCQ//7zE6xZ7wOU7PnfwJczWf3M/Wsy54COO9TQPFx6\newARbACcCrwdOBf4UCYPNqN9kiRJao7oiGl0heRDgAcoIXkRcFWekX+OYFPgfcDJwBeBj2by+Fr7\nGo+hues2tgT+H3A88GngU5k8OZrtkyRJ0uiIjngWsD9dQXkb4HJKSL40z8hnZmaLYDLwDuDdwHeB\nD2RyX5/7Hs+huWsbdqB0sx8IfAD4am13uyRJksaeXgv44Ga6epOvzzPyqW73KcN5T6IM5/05cHom\nv6t7rHYIzV3bsifwEWA74L+ACzN5eiTbJ0mSpOHTo4DvMMoEFZ0h+Yo8Ix/u9X5BAK+idKTeD7w3\nk2sbPm47heau+3AI8GFK4eH7gYVOUydJktR66hTwLQJuyTP6DqZVWD6Ukv3WoWS/RQPNfm0Zmsv9\nCOAY4IPAQ8D7MvnpcLdPkiRJA1MV8L2MMpNFrwV8De0nmAXMBbYG/hP4/mBHGbRtaO66PxOA11PG\ntdxMGddy4zA1T5IkSXX0UcB3BbCQHgV8De0v+HvgQ8AeQAfwjUye6v9e9fbZ5qG5az88CziFMuXI\nVcCZmdw61P1KkiSpu8EU8DW032BHSkfoS4GPAp/P5C/D0mZDc8/9MYWyEuGpwAKgI5Nlw7V/SZKk\ndtTnCnx1Cvga2ncwgzLN8CuATwGfzWTlUNvc/RiG5j72y4bAv1MWSPk+8MFMBvTRgCRJUruquwIf\n3NpfAV9Dxwi2Bk6nrMnxeeATmTw6lH32fSxDc539syll0us3Ad8GPpLJvSN1PEmSpLGqpoDvMMqM\nFYMq4Kt7nGAr4D3AicDXgLNGevVnQ3PDx2FL4DTKEov/QwnP94/0cSVJklrVcBfw1T1esAUlj/0T\ncC4lj/W5it/wHtvQPMDjPfPO5g3A1ynvbFaM1vElSZKapSrgm0kZkzxsBXx1jxtsDvwH8EbgPEpY\nvme4j9N/GwzNgzwuWwPvBV4HfBX4WCYPjHY7JEmSRlIvK/BNomtc8pAK+OoeO9iMMjnDm4ALgLnN\nqjEzNA/5+EynhOfXUnqeP+awDUmSNFaNRgFf3TaUYRjvBv4ZuBD4cCZ3juQx67fJ0DxM7eA5lDE2\nr6eMeT7LgkFJkjQWjFYBX912lGGw/0GpIfsOZRhGS8xeZmgeZhFMo/yxT6LMtvHRTO5uaqMkSZJq\njHYBX932lGGvp1Fmw/gWJT+N6pjlegzNI6SabePdlAHr36X88V0kRZIkjbpmFfDVbVdZlOQ0yjzL\n36R8Uj8qs2EMlKF5hFUD2P8deDNwCWUAu8tzS5KkEdVLAd9EhmkFviG3rSx3/T7KCn5fBj7V6hMq\nGJpHSQQbAf8KvAO4CvhQJjc2t1WSJGm8iI6YQFcB38voXsC3CLhlpAv46olgF+D9wEuBz1KWu36k\nmW1qlKF5lEUwBTiFMn3KjZSe52ua2ypJkjQWRUdsTfcCvhU0oYCvnghmUXqW9wE+CXwxk5XNbdXA\nGJqbJIL1KAuknAbcC8wFFmTSGr9YSZLUcqIj1qN7Ad90SgHfImDRaBfw9SeCoAT69wHbAR8Dvp5J\nSwT5gTI0N1kEE4FjKU8ogI8A381k1AfjS5Kk1lIV8O1IV0huiQK+/kQwAXgVZR2LdSnZ5oJMVje1\nYUNkaG4R1buxwynheWvg48A3xuq7MUmSNDhVAd9L6V7At5AWKODrT/Up+uspU+8+RPkU/ceZPN3U\nhg0TQ3MLimB/yhNuFnAO8PlM/tTcVkmSpJFQFfDtQdd0cLvSYgV8/YlgY+AtwNuBG4CzgKvG25BT\nQ3MLi+D5lLmej6FM9P3JTJY3tVGSJGnIqhX4OnuSW7aArz8RbAu8k1KjNR/4eCY3NbdVI8fQPAZU\nq+S8A/gX4FLKk/JXzW2VJElq1Fgq4Ksngt0os4AdAXwN+Ew7rH5saB5DIpgKvIkSoJcBn2AcjRWS\nJGm86FHAN4cSmFu6gK8/EaxDeRynAs+jzLH8pUwebWrDRpGheQyKYBLwasoTdwrwKeB/LBqUJKl5\n+ijg6wzJl7dqAV9/quK+1wHvAv5G6bC7IJO/NbVhTWBoHsOqGTcOpITnfYAvAl/I5P6mNkySpDZQ\nU8DX2Zs8pgr4+hPBFsCbgbdSivs+AVw53or7BsLQPE5E8DzKYPzXAP9LGV+0pLmtkiRpfBkrK/AN\nVrXM9b8B/wB8D/hUJrc0t1WtwdA8zkSwCaVg8G3A74FPAz/KZE1TGyZJ0hgUHfEsyoIiY76Ary/V\neOUjgX8HdgI+B3w5k4ea2rAWY2gep6pxz/9A+QfYkjJg/+vtNGBfkqSBGosr8A1WNcHASZT5lR+l\n1Eh9rx3HKzfC0NwGIphF+Yc4Ajgf+KwftUiSVPRTwLeQFl6Bb7Ai2InyifRrgcuAs4GftfN45UYY\nmttIBNOAU6rLLZR/kosduiFJaidjfQW+wYhgAnA4ZdraXYEvA1/M5J6mNmwMMTS3oQjWpUxZ93Zg\nK+DzwNccuyRJGq96FPAdAjzAOCrg60tV63QyZRaMhynDNS/M5C9NbdgYZGhucxHsDfwr8ErKEpif\nA67zIxpJ0lhWp4Dv0jwj72xi80ZcBHtRXt+PAX5EeX2/1tf3wTM0C4AINqW8E30LpRjgc8D5maxq\nasMkSWpAOxXw9SWCZwPHUcLyFsAXKJ8kP9jUho0TTQvNETGHMh3aBOArmfnRHrfPpsw3vKz60fcz\n84O97MfQPIyqaWcOo/zDzQK+RVkm89amNkySpB7G4wp8gxHBjpR6pROBX1I6vhZYszS8mhKaI2IC\ncBtlTNE9wPXACZl5a802s4F3ZeYr6uzL0DxCIphBmfP5jZS/1xeBH2Ty12a2S5LUntqxgK8vVX3S\nKymr9r0A+DowL5PfN7Vh49hgM+fEIR53H+COzFxeNeJ84GhYqzfTMNxEmSwHTo+gA3gF5R/zMxF8\ngzLp+R1NbJ4kqQ1ER0yjqye5dgW+/2YcF/D1JYLtKR1aJ1NmwvoS8EM7tFrXUEPzc4DaFXTuBvbt\nsU0C+0XErym90e/OTOcWboJqkvPvAd+LYCbln/VnEdwEzAMusgpXkjQc6hTwnTYeVuAbqAieRelV\n/mdgN+B/gAMzua2pDVNDhhqaG/no5AZgm8xcFRGHAz+kDPBXE2VyO3BaBP+P0vv8L8DZEXwb+Eom\nNzW1gZKkMaVOAd8ptEEBX18ieD7ldfZ1wG8pHVU/tKNqbBlqaL4H2Kbm+jaU3uZnZObKmu8XRMTn\nI2KTzLUH9UfEmTVXF2fm4iG2T3VUHwN9F/huBM8F/glYGMFdwFcoc0Cu7G8fkqT21E8B3zeB17dL\nAV9vIlgfOJbSq7w98A3gRY5VHn1Vfd3sIe9niIWAEymFZS8F7gWuY+1CwC2BBzIzI2If4MLMnNHL\nviwEbBERTKQUZ7yR8iT7X+BrwNXOCylJ7Ss6Yh1gT7pC8m60aQFfbyIIYD9KB9Q/AFcDXwV+nElb\n9rK3oqYUAmbmUxHxNso/ygTgq5l5a0ScUt3+Jcq7rLdExFPAKuD4oRxTI6/6x74YuDiCLSkfJ30B\nWLcqHvxmZvdPFCRJ41MvK/A9SHnd/yBtWMDXmwi2pkwTdzLwNKWj6fmZ3N/UhmlYubiJGlK9e96b\n8u75OMr0gt+kjMly4RRJGifqFPAtascCvt5EsB5lxrA3AC+iDHX8OvALP5Vtba4IqFETwWS6ThT7\nAD+gBOhrPFFI0tjSRwHfLZSQvJA2LuDrqWb4xRson6T/ivL6d1EmTzazbWqcoVlNUX0k9Y+UE8hk\nyvQ551roIEmtqyrgO5iuxUXacgW+RlWF8q+jDMF4ihKUv+VQxbHJ0Kymqt5970E5oRxPWTb9W5TZ\nNx5sZtskqd1ZwDdwEWwKvJoSlp8HXEDpGLreT1XHNkOzWkYEkyjFIq8HjgSuogToHzn+WZJGRz8F\nfIsoBXyej3uI4NnAUZRPUA8CFlBevxZlsrqZbdPwMTSrJUWwAXAM5Z363sCPgO8Al3sCkqThUxXw\n7U/XkIvaAr5L84y8s4nNa1nVNKsvBU6g1Ov8ihKUf5DJ481sm0aGoVktL4JplI+6TgB2oCzp/R1K\nAeHTzWybJI01FvANXgTrUGa8OIHyuvRHyuvRhZnc08y2aeQZmjWmVEUVx1NOWBtTxopdiGPFJKlP\n/azAZwFfHTW1N8dRXn+eoATl8zO5o5lt0+gyNGvMiuAFlBPYccC6lPB8IXCDAVpSO7OAb2iqoLwr\n8BrKa0xSXl/OB37ra0x7MjRrzKtObrtQTmyvqX58IWXC+Bs9uUlqB9ER0+gKyRbwDVDNa8mrKa8n\nEymvJRfga4kwNGucqU56u1PC87HVj79fXa7zpCdpvKgp4OsMytvgCnwDUr1m7AW8qrpMpNTNXAD8\nytcM1TI0a9yq+XjtVZQAvT5lFcLvAz/NZE0TmydJA2IB3/CoKebrDMp/obwufA9YYlBWXwzNahsR\n7ExXgN4KmA/8ELgik780s22S1BsL+IZHBM+izJ98DPAK4CG6gvLNBmU1wtCsthTB9pR5NY+hjGG7\nFLgIuCSTx5rZNkntywK+4RPBVOBw4JWUOahvpnSU/NBZLzQYhma1vQi2AF5OObEeCFxL6YX+USbL\nm9g0SW2glwK+B+hewPfnJjZvTIlgW8r5/BWUIRhXU4Ly/ExWNLNtGvsMzVKNCNYHDqWcdI8C7qes\nRjifMhe0i6lIGpI+Cvgup2sFPgv4GlSNT96TrqD8HODHlPP2pZmsbGLzNM4YmqU+RDAB2IdyIn45\nsBmwgHJCvsxhHJIa0UcB38109SZbwDcA1bCLQ4AjgSOARykdG/OBX1jkrZFiaJYaVI2DPoJyon4x\n8CtKgP4xsNRCEkmdqgK+g+m9gO8KC/gaV82EtCPl3HskpTPjZ5Rz7yWOT9ZoMTRLgxDBZMoLYudJ\n/GnKlE8LgSszebyJzZM0yuoU8C0EbrWAr3ERbECZ7WIO5ff5LLo6Ka7I5MkmNk9tytAsDVHVC/J8\nysl9DqX45Fd0vVj+2rHQ0vhjAd/wqVmNrzMk700pyu7sjHBaODWdoVkaZhFMoczCcRhluqONKEU+\nl1HGQt/dxOZJGiQL+IZXBM+hvNk4tPr6BKVuZBGwOJMnmtg8aS2GZmmERbAd5UXhUMoiBQ9RBWjg\nJxYUSq3JAr7hVRXwHUDX+XBL4Eq6OhT+0MTmSXUZmqVRVE2PtDtdLxr7UF6Er6wuP8tkVfNaKLW3\nHgV8c4AJWMA3KBE8mzJc7eDqsgtwPV2dBjc404XGEkOz1EQRrEf3F5VdgV9SAvRPgGtd4lsaORbw\nDZ9qqep9KMPTDq6+/y1wBeWc9vNMHOetMcvQLLWQqmJ8f0rV+IHACygh+ifV5Rf2REtDUxXwvYzS\nk2wB3yBVPcn7ArMp56u9gVsp56r/A652JiGNJ4ZmqYVVIfrFlBekAykfb/6asjTsNcBPM3mkeS2U\nWp8FfMMjgo2A/Si/y5dQhprdRNeb+p9ao6HxzNAsjSHVzByzKC9Y+1N6eZZTAvTVwE+BO52aSe3M\nAr6hq6aA24YSkjvPN9tTxiR3vmn/hctUq50YmqUxLIJJlDGYnT0/+1EWWvk5ZVzmzynFNo6L1rhm\nAd/QVOORd6ecQ15UXdalnEc6Q/INmaxuWiOlJjM0S+NI1Ts0g64Xvv2A5wG/oSwUcC1wHbDM3miN\nZRbwDV7NeWJfSrHevpTf3+8ob7Q733R7npBqGJqlcS6C9SkFOp0vjvtSepCuqy7XAr/K5MGmNVJq\ngAV8gxPBpsBedJ0D9gGeoutN9LXA9Q61kPpnaJbaULUSV+0L6B7Ao5Tlv39ZXX6ViR9pq2ks4Bu4\nCDam9MDvVfN1E2AJXQH5OuBue5GlgTE0S+pcdOXvKC+wnS+2ewB/orzY1l7u9cVWI6Eq4JtJ6Um2\ngK8f1RCLaZRxyLtThlfsDmxB+T995s0vcHsmTzepqdK4YWiW1KsqSM9k7RdlgBury2+qy9JM/taM\ndmps61HAdxgwEQv4uolgXUptwgsp0052/i9OYO03tbe7yp40MgzNkhpW9W5tTXnB3pXyAr4Lpajo\ndrpC9E2UHkKnv1M3FvD1rWaatxcAf09XSH4e8Ee6/r9+QwnIDrGQRpGhWdKQVSuDPZ/yAr8r5UX/\nBcBU4BZKgO683Arc5cfF7aMq4OsMyW1fwFd9ijOd8j/zgprLzsATdP2vdAbkW1wJVGo+Q7OkEVMV\nJb2gx+V5wMaU6a2WUkL0UuA24I5MnmxOazVcoiPWo3sB33TgCqqg3C4FfBFMBnagPOd3qi7Ppyy8\n8hjlOV/7hvJmi2+l1mVoljTqIphK9yDRedmeUnx4OyVU315d7qDMGdtWPZJjRVXA9zy6QvL+tEkB\nX/Upy3Mp4Xhmddmx+roZ8AfKm8Lay20uNy2NPYZmSS0jggmUXsnO0NEZQP6OMm76YeD3wLKar3+g\njPe8zwKo0VMV8L2UrqA8Llfgq4ZSTAO2o4Tj7SnPx86vm1Kef7+n+5u931GGIfmclMYJQ7OkMaEK\n1FvTFVg6Q8uM6rIJcBclwCyvvt5Vc7nboR+DFx0xge4FfLsyDgr4qiEU0ykFeJ2X7SjPqe2q649S\nnlPL6f6G7ffAPQZjqT00LTRHxBzg05Teia9k5kd72eZs4HBgFXBSZi7pZRtDsyQiWA/Ylq7AM4MS\neKbXfP0zVYAG7u3j8oAhqIiO2JqyAt9hwKHACsZIAV/1JmsLSi/x1r1cOp8XUyjPh2feXNEVkP9I\nmQGmZR+npNHTlNAcERMoBRCHAPcA1wMnZOatNdscAbwtM4+IiH2Bz2TmrOF6AJLaSzWd12Z0Beie\nYWoa8BxKj/UjlIBYe3mgujzU4/LoeJkJpJ8CvoU0eQW+apjEhpS/Ye1lC2DLHpct6Po73kN5M3Qf\nXW+M7qMrID/otG2SGjHYzDlxiMfdB7gjM5dXjTgfOJpSRd/pFcA3ATLz2ojYKCK2zMwVQzy2pDZU\nBaMHq8sNfW0XwURKGOsZwrakFLv1DG3rR/AIZbx17dfay2PA4zVfO79fCaxqVs92VcC3IyUgz6F7\nAd8pDGMBXxV6pwAbUKYi3LD6Wvv9xtVlk16+3wR4kvJG5UG6v3G5lzJvce0bnAczGZfFh5LGlqGG\n5udQ3uV3uhvYt4FtplNOiJI0IqqgdX91qSuCSZRisI3pPfRtx9rhsPP7DYDJEfyVEgifpMzT+yRl\nWNpfKENKOi+d1/9Wc1ld83U1sKaPy9NAMvXOyTzvR7uw2W27M+W4sqrcE1vdyIM7/5o75nyfx7b7\nC2XY3E7AC+JMJgLrVpdJNd+vC6wHPLvm0nl9MiUgTwHWr76uVz2mx+n+xqH2+iOUccLX0/WGo/NN\nyJ8yWd3I30SSWslQQ3OjH4X17ALv9X4RcWbN1cWZuXgQbZKkAauCXMMhu6dq2Miz6R4w16cEz84Q\n2vNrZ2jdgO4hdhIl8NZcnp7AlAc3ZcqKrZj8py1Y98mpPPD3j7B89gOsnHYzf9l4JcTTlHA/nbWD\ndm8BfRVs3kvSAAARt0lEQVQl8PYM87XfP0HXG4AngD87DELSWBIRs4HZQ93PUEPzPZRxhZ22ofQk\n97fN9Opna8nMM4fYHklqiipIrqouDw7HPnsU8L2UMlxhYXVp6QI+SWoVVSfs4s7rEXHGYPYz1ND8\nS2BmRMygjEV7DXBCj23mA28Dzo+IWcCjjmeWpLVFRzwLeAm9r8B3WruswCdJrWhIoTkzn4qIt1FO\n6BOAr2bmrRFxSnX7lzLzkog4IiLuoHy8d/KQWy1J40CPAr7DKIF5RAr4JElD4+ImkjSKelmBbyJd\ncyZfPl5W4JOkVuWKgJLUgqoV+PagTAXXcwW+RcAtY3EFPkkaqwzNktQixvIKfJI03hmaJalJ6hTw\nLbKAT5Jah6FZkkZJnQK+RVjAJ0kty9AsSSPIAj5JGh8MzZI0jGoK+DpD8m5YwCdJY56hWZKGKDpi\nGl0h+VDKCnyLcAU+SRo3DM2SNEAW8ElS+zE0S1IdFvBJkgzNktQLC/gkSbUMzZIEREesA+yJBXyS\npF4YmiW1rR4r8B0CPIgFfJKkXhiaJbWNqoBvf0pInoMFfJKkBhmaJY1bfRTw3UJXb7IFfJKkhhia\nJY0rFvBJkkaCoVnSmGYBnyRpNBiaJY05PVbgqy3gW0Qp4FvVxOZJksYhQ7OkltejgO8wYBss4JMk\njSJDs6SWYwGfJKnVGJoltQQL+CRJrczQLKkpLOCTJI0lhmZJo8YCPknSWGVoljRiLOCTJI0XhmZJ\nw8YCPknSeGVoljQkVQHfwcAcLOCTJI1ThmZJA2IBnySpHRmaJdVlAZ8kqd0ZmiWtxQI+SZK6MzRL\n6quA72a6epMt4JMktTVDs9Smagr4DqMU8U2gKyRfYQGfJEldDM1Sm6hTwLcQuNUCPkmSemdolsax\nXgr4HqB7Ad+fm9g8SZLGDEOzNI70UcB3OSUkX2oBnyRJg2NolsYwC/gkSY2IiNYIbmNEb9nS0CyN\nMRbwSZIGqspLzW7GmBARhmZpLLKAT5I0VIbmxhmapTGkKuB7GaUn2QI+SdKQGJob1zKhOSI2AS4A\ntgOWA8dl5qO9bLcceBxYA6zOzH362J+hWWNePyvwLcQCPknSEBmaG9dKofks4KHMPCsi3gNsnJnv\n7WW7PwB7ZvY/PtPQrLGoKuCbSelJtoBPkjSiDM2Na6XQvBQ4MDNXRMRWwOLM3KmX7f4A7JWZf6qz\nP0OzxoQeBXyHAROxgE+SNAoMzY1rpdD8SGZuXH0fwMOd13tstwx4jDI840uZOa+P/Rma1ZIs4JMk\ntYp2C8033ngj3/rWt/j4xz8+4PsOd2ieWOdglwFb9XLT6bVXMjP7mTfwxZl5X0RsDlwWEUsz8+qB\nNlQaTf0U8H0QC/gkSapryZIlrL/++sycOXNQ9//kJz/JNddcw4YbbjjMLRucfkNzZh7a120RsSIi\ntsrM+yNiGiVU9LaP+6qvD0bERcA+QK+hOSLOrLm6ODMX9998aXjUWYHvNAv4JEkamNtuu43jjz9+\n0Pd/17vexaabbsrixYuH1I6ImA3MHtJOqBOa65gPvAH4aPX1hz03iIjJwITMXBkRUyg9dx197TAz\nzxxCe6SG1SngezMW8EmSNCRl9G53y5YtY968XkfqAjBr1iyOPvroZ64Px1CUqhN2cU27zhjMfoYS\nmj8CXBgRb6Sacq5qyNbAvMw8kjK04wfVL20i8O3MvHQIx5QGrZ8Cvm8Ar7eAT5I05vUSVAdlkGH1\n8ccf57zzzmP69OksWbKE2267jeOOO46ddipzRWy//fbMnTu34f31FrybZdChuZpC7pBefn4vcGT1\n/TJK0ZQ06uoU8J2NBXySpPGmiS9rq1at4sQTT2TevHksX76cN7/5zUydOpVjjz2WK6+8clD7bKWi\nx6H0NEstp6aA7zDgUCzgkyRpVJx77rnstttubL755lx++eXsvffeLFu2jNWrVz+zzUCHZ4yLnmap\nFfSzAt8i4D0W8EmSNDomTZrE1KlTu/2so6ODs84665nrAx2e0Uo9zYOep3m4OU+zGlEV8O1IV0h2\nBT5JUtto5Xma16xZw9y5c9l2221ZsmQJM2bM4KCDDmKXXXYZ1P7OOeccLrzwQu666y5OOukk3vnO\nd64VyvvTMoubDDdDs/pSFfC9lN5X4LvcAj5JUrto5dDcacGCBRxwwAFMmTKlqe0Y1cVNpGaIjphA\n9wK+XelewHeLBXySJLWmlStXNj0wjwR7mtUSoiO2pnsB3wq6epMt4JMkibHR09wqHJ6hcSE6Yj26\nF/BNpxTwLQQutYBPkqS1GZob5/AMjUk9CvjmUAJzZwHfKVjAJ0mSWpg9zRoxfRTwLaQE5Sss4JMk\naWDsaW6cwzPUsuoU8C3CAj5JkobE0Nw4Q7NaSi8FfA/Q1ZtsAZ8kScPI0Nw4Q7OaqlqB7yWsXcC3\nCFhkAZ8kSSPH0Nw4CwE1quqswGcBnyRJagv2NGstFvBJktSa7GlunMMzNOyqAr49KFPBWcAnSVKL\nMjQ3ztCsYeEKfJIkjT1jKTQvWbKE9ddfn5kzZzbl+I5p1qDUKeA7zQI+SZI0nG677TaOP/74Zjdj\n2BiaxykL+CRJUjNFrN2Zu2zZMubNm9fnfWbNmsXRRx89ks0aNIdnjCN9FPB1Drm43AI+SZLGtnrD\nM6JjeKJUnjG4fPj4449z3nnnMX36dK655homT57Mcccdx0477TTgfc2fP58JEyZw9dVX88IXvpCF\nCxdy+umnN7wvxzTrGTUFfJ0heTcs4JMkadxq5THNq1at4rWvfS3z5s1j+fLlbL755kydOpVjjz2W\nK6+8ckD7uvPOO/nb3/7GDjvswJ577skVV1zBNddcw8EHH8zkyZMb2odjmttcPwV8HwSuzjNyVROb\nJ0mS2tS5557Lbrvtxuabb87ll1/O3nvvzbJly1i9evUz2zQ6PGPbbbcFYMWKFWywwQZstNFGHHXU\nUSP+GPpjaG5xVQHf/nRNB2cBnyRJajmTJk1i6tSp3X7W0dHBWWed9cz17bffnrlz59bd19KlS/nr\nX//KDTfcwAEHHADAJZdcwhFHHDG8jR4Ah2e0mKqAbyZdIbm2gG8RFvBJktS2Wnl4xpo1a5g7dy7b\nbrstS5YsYcaMGRx00EHssssuA97X2WefzcqVK5k2bRpLly5lv/32Y/r06ey1114N78MxzeNQVcB3\nMF1jkyfhCnySJKmHVg7NnRYsWMABBxzAlClTmtoOxzSPA9ER6wB70nsB39nArRbwSZKksWjlypVN\nD8wjwZ7mURIdMY2ukHwI8ACuwCdJkgZgLPQ0twqHZ4wRNQV8nUF5G0oB30LgUgv4JEnSQBmaG2do\nblF1VuCzgE+SJA2ZoblxhuYW4gp8kiRpNBmaG2dobqI6BXyuwCdJkkaUoblxhuZR1mMFvkOABykB\neSEW8EmSpFFkaG6coXmk21EK+F5CV29y7Qp8iyzgkyRJzWJobpzzNA+zOgV8p2ABnyRJUttry55m\nC/gkSdJYZE9z4xyeMZh9W8AnSZLGAUNz4wzNje6v7wK+zhX4Vg3XsSRJkkaDoRluv/12brrpJn7z\nm9/w8pe/nD322KPX7RzT3IceK/DNoXsB32kW8EmSJI19F198MS9+8Ys55JBDOOWUUzjvvPNG5bhj\nNjT3UcB3CxbwSZIkjVvvfOc7Abjlllt47nOfO2rHHXRojohXA2cCOwF7Z+YNfWw3B/g0MAH4SmZ+\ndNDH7LuA75vA6y3gkyRJag1Llixh/fXXZ+bMmSOy/4suuojTTz99RPbdm6H0NP8WOAb4Ul8bRMQE\n4BzKmOJ7gOsjYn5m3trIAeoU8J2NBXySJEkt6bbbbuP4448fkX3Pnz+fd7zjHdxzzz0jFsp7GnRo\nzsylUAZZ92Mf4I7MXF5tez5wNNBnaO6ngO9DWMAnSZI0JvSWEZctW8a8efP6vM+sWbM4+uij+93v\nRRddxIc//GE++9nPMnv27FHrbR7y7BkR8X/Aqb0Nz4iIY4HDMvNfquuvA/bNzLf3sm1yJr/FFfgk\nSZJ61eqzZzz++OOcd955TJ8+nWuuuYbJkydz3HHHsdNOOw14X/Pnz2fChAlcffXVvPCFL2ThwoWc\nfvrpDe9rVGfPiIjLgK16uen9mfmjBvY/sL/qF7iOB/ghydOUHmoDsyRJUoP6HwDQuMHk8lWrVnHi\niScyb948li9fzpvf/GamTp3Ksccey5VXXjmgfd15553svPPO7LDDDvzXf/0X733ve9lwww3Zdttt\nB9yuiJgNzB7wHXvoNzRn5qFD3P89wDY117cB7u7zePfnPw/xeJIkSW2rmZ3Q5557Lrvtthubb745\nl19+OXvvvTfLli1j9erVz2zT6PCMznC8YsUKNthgAzbaaCOOOuqoQbUrMxcDizuvR8QZg9nPcE05\n19f7ml8CMyNiBnAv8BrghGE6piRJklrEpEmTmDp1arefdXR0cNZZZz1zffvtt2fu3Ll197V06VL+\n+te/csMNN3DAAQcAcMkll3DEEUcMb6MHYNBjmiPiGMoMFpsBjwFLMvPwiNgamJeZR1bbHU7XlHNf\nzcxef1MjuYy2JEnSeNDKY5rXrFnD3Llz2XbbbVmyZAkzZszgoIMOYpdddhnwvs4++2xWrlzJtGnT\nWLp0Kfvttx/Tp09nr732angfLqMtSZLUplo5NHdasGABBxxwAFOmTGlqO4Y7NK8zPM2SJEmSYOXK\nlU0PzCPBnmZJkqQxYiz0NLcKe5olSZKkUWZoliRJkuowNEuSJEl1GJolSZKkOgzNkiRJUh2GZkmS\nJKmO4VpGW5IkSaMgwhl6m8HQLEmSNEa4pkXzODxDqiMiZje7DVJffH6qVfnc1HhjaJbqm93sBkj9\nmN3sBkh9mN3sBkjDydAsSZIk1WFoliRJkuqIzGx2GwCIiNZoiCRJksa1wRRUtkxoliRJklqVwzMk\nSZKkOgzNkiRJUh1NCc0R8eqIuDki1kTEHv1sNycilkbE7RHxntFso9pXRGwSEZdFxO8i4tKI2KiP\n7ZZHxG8iYklEXDfa7VT7aORcGBFnV7f/OiJ2H+02qn3Ve35GxOyIeKw6Vy6JiP9sRjvVXiLiaxGx\nIiJ+2882AzpvNqun+bfAMcBVfW0QEROAc4A5wM7ACRHx/NFpntrce4HLMnNH4Irqem8SmJ2Zu2fm\nPqPWOrWVRs6FEXEEsENmzgTeBHxh1BuqtjSA1+qfVOfK3TPzg6PaSLWrr1Oel70azHmzKaE5M5dm\n5u/qbLYPcEdmLs/M1cD5wNEj3zqJVwDfrL7/JvDKfrZ1OVONtEbOhc88ZzPzWmCjiNhydJupNtXo\na7XnSo2qzLwaeKSfTQZ83mzlMc3PAe6quX539TNppG2ZmSuq71cAff0TJXB5RPwyIv5ldJqmNtTI\nubC3baaPcLskaOz5mcB+1Ufgl0TEzqPWOqlvAz5vThyplkTEZcBWvdz0/sz8UQO7cC48jZh+np+n\n117JzOxnDvEXZ+Z9EbE5cFlELK3e2UrDqdFzYc+ePM+hGg2NPM9uALbJzFURcTjwQ2DHkW2W1JAB\nnTdHLDRn5qFD3MU9wDY117ehvAuQhqy/52dVOLBVZt4fEdOAB/rYx33V1wcj4iLKx5SGZg23Rs6F\nPbeZXv1MGml1n5+ZubLm+wUR8fmI2CQzHx6lNkq9GfB5sxWGZ/Q1zumXwMyImBER6wKvAeaPXrPU\nxuYDb6i+fwOlV6SbiJgcERtU308BXkYpcJWGWyPnwvnAiQARMQt4tGaIkTSS6j4/I2LLiIjq+30o\nC6sZmNVsAz5vjlhPc38i4hjgbGAz4McRsSQzD4+IrYF5mXlkZj4VEW8DFgETgK9m5q3NaK/azkeA\nCyPijcBy4DiA2ucnZWjHD6rXgYnAtzPz0uY0V+NZX+fCiDiluv1LmXlJRBwREXcATwInN7HJaiON\nPD+BY4G3RMRTwCrg+KY1WG0jIr4DHAhsFhF3AWcAk2Dw502X0ZYkSZLqaIXhGZIkSVJLMzRLkiRJ\ndRiaJUmSpDoMzZIkSVIdhmZJkiSpDkOzJEmSVIehWZIkSarD0CxJkiTV8f8BlXSRkNHfdcAAAAAA\nSUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# ensure plots appear in the notebook.\n", "%matplotlib inline \n", "import pylab as plt\n", "\n", "# first let's generate some inputs\n", "n = 100\n", "x = np.zeros((n, 1)) # create a data set of zeros\n", "x[:, 0] = np.linspace(-1, 1, n) # fill it with values between -1 and 1\n", "\n", "Phi = quadratic(x)\n", "\n", "fig, ax = plt.subplots(figsize=(12,4))\n", "ax.set_ylim([-1.2, 1.2]) # set y limits to ensure basis functions show.\n", "ax.plot(x[:,0], Phi[:, 0], 'r-', label = '$\\phi=1$')\n", "ax.plot(x[:,0], Phi[:, 1], 'g-', label = '$\\phi = x$')\n", "ax.plot(x[:,0], Phi[:, 2], 'b-', label = '$\\phi = x^2$')\n", "ax.legend(loc='lower right')\n", "ax.set_title('Quadratic Basis Functions')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The actual function we observe is then made up of a sum of these functions. This is the reason for the name basis. The term *basis* means 'the underlying support or foundation for an idea, argument, or process', and in this context they form the underlying support for our prediction function. Our prediction function can only be composed of a weighted linear sum of our basis functions. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Different Basis\n", "\n", "Our choice of basis can be made based on what our beliefs about what is appropriate for the data. For example, the polynomial basis extends the quadratic basis to aribrary degree, so we might define the $j$th basis function associated with the model as\n", "$$\n", "\\phi_j(x_i) = x_i^j\n", "$$\n", "which can be implemented as a function in code as follows" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def polynomial(x, num_basis=4, data_limits=[-1., 1.]):\n", " Phi = np.zeros((x.shape[0], num_basis))\n", " for i in xrange(num_basis):\n", " Phi[:, i:i+1] = x**i\n", " return Phi" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To aid in understanding how a basis works, we've provided you with a small interactive tool for exploring this polynomial basis. The tool can be summoned with the following command." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAEACAYAAABI0Ut2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYZHV97/H3lxlQQRRwcFgHVDCIC6BkGDBXxriEJaLm\nJoqaSDRXjQkxNzFRMfFKjImiubleEvWqUR9coqhxIS5RQEbJExZHFkGGTVmGbWRHZlhm+d4/fqfp\n6p7qPtVd1XVqeb+e5zzn1KlTVd85c3rm07/6nd8vMhNJkiRJM9um6QIkSZKkQWdoliRJkmoYmiVJ\nkqQahmZJkiSphqFZkiRJqmFoliRJkmp0HZoj4lMRsS4iLpvlmFMj4pqIuDQiDun2MyVJkqR+6kVL\n86eBo2Z6MiKOAfbLzP2BNwIf7cFnSpIkSX3TdWjOzHOBu2c55DjgtOrYC4CdImJpt58rSZIk9Us/\n+jTvCaxteXwTsFcfPleSJEnqiX7dCBjTHjt3tyRJkobG4j58xs3A3i2P96r2TRERBmlJkiQtuMyc\n3qBbqx+h+QzgROCLEbECuCcz17U/NAE+D/xepq3R8xURJ2fmyU3XMSo8n73l+ewtz2fveC57y/PZ\nW57P3plvQ23XoTkivgAcCSyJiLXAu4FtATLzY5n57Yg4JiKuBdYDr5vl7dYDrwGuBt7TbW2SJElS\nL3QdmjPzVR0cc2KHb3c88A3gbyK4JpMvdFWcJEmS1AMDNSNgJt8E3lo9/HQERzRZzxBb1XQBI2ZV\n0wWMmFVNFzBiVjVdwAhZ1XQBI2ZV0wWMmFVNFzDuInMwug5HRGZmRBDAh4E3A7cDh2VyXbPVSZIk\naRRMZM45v27QQnPZZjHwLeDFwBXAEZnc22R9kiRJGn7zDc0D1T1jQiabgFdQAvOBwJcjys2FkiRJ\nUr8NZGgGqFqWjwV+AbwI+HjVdUOSJEnqq4ENzQCZXA+8BNgA/D5wcoPlSJIkaUwNdGgGyORC4JXA\nFuB/RfCGhkuSJEnSmBn40AyPDEX35urhRyM4tsl6JEmSNF6GIjQDZPJx4L3AIuBLERzacEmSJEka\nEwM55NzMxxDAp4ETKDcIHp7Jz/tRnyRJkobfSI3TPPtxbEsZw/lFwDWUMZzvWOj6JEmSNPxGapzm\n2WSyEfht4BJgf+DfI9i+2aokSZI0yoYuNANkch9lDOcbgRXA6U5+IkmSpIUylKEZIJNbgKOAO4Hf\nBD4VMbx/HkmSJA2uoQ6ZmawBjgHWA78L/KOzBkqSJKnXhjo0wyOTn7wM2Aj8KfDOZiuSJEnSqBn6\n0AyQyVnAq4EE3hvBHzZckiRJkkbISIRmgEy+Ao+E5Y9E8Iom65EkSdLoGJnQDI/MGvhOIIDPRfDi\nhkuSJEnSCBip0Fx5P/CPwLbA1yJY0XA9kiRJGnJDNyNgZ+/FNpTptl8L3AM8P5NLevHekiRJGl5j\nMyNgJzLZAvwP4OvATsCZERzYbFWSJEkaViMZmuGR6baPB/4DWAKcFcF+zVYlSZKkYTSyoRkgk4eA\n3wLOAXYHzo5gn2arkiRJ0rAZ6dAMkMkDwHHAecAy4PsR7NlsVZIkSRomIx+aATK5Hzga+DHwZEpX\njSc2W5UkSZKGxViEZoBM7gV+A7gcOIByc+AuzVYlSZKkYTA2oRkgkzuBFwJXA88CvhvBTs1WJUmS\npEE3VqEZIJN1wAuA64BDge8ZnCVJkjSbsQvNAJncBDwfuB74VQzOkiRJmsVYhmaATG4AVlJanA3O\nkiRJmtHYhmZoG5zPjGDnRouSJEnSwBnr0AyQyY2U4PxzSh9ng7MkSZKmGPvQDFsF5+dgcJYkSVIL\nQ3Mlk7WU4PwzSnA+y3GcJUmSBIbmKaYF52dTptx25kBJkqQxZ2iephqObiVlApSDgB9EsGejRUmS\nJKlRhuY2quD8POAyypTb50bwpGarkiRJUlMMzTOoZg5cCfwIeBIlOB/QaFGSJElqhKF5FpncBbwQ\n+CGwJ/DDCJ7VbFWSJEnqN0NzjUzuA44GvgfsCqyKYHmzVUmSJKmfDM0dyGQDcBzwdWBn4OwIjmy2\nKkmSJPWLoblDmTwEvAL4V+CxwH9EcFyzVUmSJKkfDM1zkMlG4LXAx4BHA1+N4PXNViVJkqSFZmie\no0w2A28G3gMsAj4ZwTsiiGYrkyRJ0kKJzGy6BgAiIjNzqIJnBCcCpwIBfAh4ayZbmq1KkiRJM5lv\n5jQ0dymCVwKfBbYFPg+8PpOHm61KkiRJ7cw3c3bdPSMijoqIKyPimoh4e5vnV0bEvRFxcbX8dbef\nOUgyOR04FlgPvAY4I4Idmq1KkiRJvdRVS3NELAKuokwAcjNl9rxXZeaalmNWAn+embOONDGsLc0T\nIjgU+A6wBLgAeEkmtzdblSRJklo11dK8HLg2M6/PzI3AF4GXtquvy88ZeJmsBp4L3AAcBvxXBPs3\nW5UkSZJ6odvQvCewtuXxTdW+VgkcERGXRsS3I+LALj9zYGVyNXAEcDGwH3BeBEc0W5UkSZK6tbjL\n13fSt+MiYO/M3BARR1Nm1XtquwMj4uSWh6syc1WX9fVdJrdUswV+ETgG+H4Ev5fJlxsuTZIkaexU\nXYVXdv0+XfZpXgGcnJlHVY9PArZk5imzvOY64DmZede0/UPdp3m6CBYD/wy8qdr1NuAfMjv6RUOS\nJEkLoKk+zauB/SNi34jYDnglcMa0wpZGRFTbyylB/a6t32q0ZLKJMgnKxIgiHwA+XIVpSZIkDZGu\nAlxmboqIE4HvUs2Ol5lrIuJN1fMfA34beHNEbAI2AMd3WfPQqFqVPxDB9cBnKCF6WQTHZ3J/o8VJ\nkiSpY05u0icR/BrwDWAX4BLgpZnc2GxVkiRJ46WxyU3UmUz+EzgcuBY4GLgwgsObrUqSJEmdMDT3\nUTUk3WHA2cBSYFUEr222KkmSJNUxNPdZJncBR1NG1tgOOC2CUyJY1GxlkiRJmol9mhsUwR8C/0S5\nIfObwKsz+WWzVUmSJI2u+WZOQ3PDIng+8BXKDYKXA8dlcl2zVUmSJI0mbwQcUpmcAywH1gDPAH4U\nwQubrUqSJEmtDM0DIJOfUUbW+DbwBOC7Ebw9grFreZckSRpEhuYBkcm9wEuA91D+Xt4PfDmCHRst\nTJIkSfZpHkQRvAT4HPA4SreN38rkymarkiRJGn72aR4hmfw7cCjwU+BplIlQXt5sVZIkSePL0Dyg\nMrkGWAF8CdgR+GoEf+94zpIkSf1naB5gmdwPHA/8BbAFOAk4M4LdGi1MkiRpzNineUhE8OvAF4An\nAuuA12RydrNVSZIkDRf7NI+4TL4PHAycAyyltDifbHcNSZKkhWdoHiKZ3Aq8CPibate7sbuGJEnS\ngrN7xpCK4AXA5ymtznbXkCRJ6oDdM8ZMFZAPBr7PZHeN90SwuNnKJEmSRo+heYhlchvwYuDkate7\ngHMjeEpjRUmSJI0gu2eMiAiOBD4L7A3cD/wJcFomg/EXLEmSNADsnjHmMvkBcBBlMpTHAp8GTo9g\n50YLkyRJGgGG5hGSyd2UyVBOoLQ2/w7wkwhWNlmXJEnSsDM0j5hMMpPPUG4SPB/YC/h+BKdE8Khm\nq5MkSRpOhuYRlcnPgOcB7wESeBuwOoJnN1qYJEnSEPJGwDEQwRHAacB+wCbg74C/z+ThRguTJEnq\ns/lmTkPzmIhgB+DvgbdUuy4BTsjkJ81VJUmSBk0Ei4DFwCJKr4RFM2wDbK6WTdPWm4FNgziKl6FZ\nHaluCvw0sC+wkTIl9ymZbGqwLEmSNA8RPAZYAjweeFyb9fTt7YFHA4+ZZd3LidK2AA8AG6Yt0/f9\nErgbuKdluXva9l2ZbO62IEOzOhbBY4EPAG+udq0GXpfJ5c1VJUnSeIsggB2B3atlV0ognm3ZfgFK\nSaa2GG+ZYTuYbHlePMN2L20B7gDWAb+o1utaHt8KrAXWZnLfTG9iaNacRfBC4JPAMsoPxynAezN5\nsNHCJEkaMVWL8DLKqFa7A3swGY5bt3eY41s/TAmS9wD3Vcu9M2zfx2Qr74PVunV7Yr2xF90qql8C\nFlNasLevlse02d6B8svCzsBO1TJ9e5dq3al7gRspIbp1fQPEDw3NmrMIHkcJy39Y7boaeGM1WYok\nSapRhcMllFC8z7T1xPauHb7dA8AtlFbTdZRAPNuyfhD7DS+ECLalnMelwBOrdev27pSZkZdRAvlM\n74ShWfMWwa8BnwAOqHb9C/C2asIUSZLGWjXXwb7AU6plv5btfZg1pAHlPqK1wE2UUDwRjCeWicf3\njUsIXijVLzFPYDJAT6yXAftCHGZoVleqfxBOAt4JbEv5DfdPgK/4AyxJGnURbA/sTwnEraH4KZTA\nNVtOuQe4gUe6AHDjtO3bMtmyYMWrY/ZpVs9EcCCl1fmIatc3gbdkcl1zVUmS1L2qFXIp5ZvV6cs+\ns7x0CyUA/6xarm3Zvn62G880WAzN6qkItgHeBLyfMkTNg8D7gA94o6AkadBVYw3vBzyNrcPx42d4\n2SZKCL6arcPxDU4KNhoMzVoQEewOfBB4TbXr55RW5281V5UkSUXVcrwX8IxqeWa1PhB41Awvuwe4\nss3y80w2LnTNapahWQsqgiOBDwNPr3adAfxPu2xIkvolgiewdTh+BjO3HN8IXAGsYWo4vt17dcaX\noVkLrhrq5UTKLII7Mtll44OZPNBkbZKk0VF1rdgfOBg4pFo/kzKkWDt3ApdVy+XV8tNM7l34ajVs\nDM3qmzZdNtYC7wC+4G/ukqS5qCb9eCZTA/KzaD/T3QYmQ3FrQF7n/z/qlKFZfVd12fgQ5R84gAuA\nP8vkvOaqkiQNqgiWUP7PaA3IBwDbtDl8LXAxcEm1/AS4zmHb1C1DsxpRfYV2AvB3wG7V7tOBd2Ry\nfVN1SZKaFcGuwHOAQ6vlOZQb9qbbTOlzPBGOLwYuzeTOPpWqMWNoVqMi2BF4O/BWyhzzDwH/B3if\nY1dK0miLYGemBuRDaT/m8XrgUqYG5J96X4z6ydCsgRDBMsrNga+udt1JaYX+qOM7S9Lwi+BxwLOZ\nGpCf0ubQ9cBFwOpq+TFwbSab+1Sq1JahWQMlgsOAfwB+rdq1Fng38NlMNjVWmCSpY9W00ocwNSD/\nCltPJ/0gpeV4NfCjan2VAVmDyNCsgVMNOH8MpeX5mdXuNcBfA1/zTmdJGhzVTLC/AhwGrKjWzwQW\nTTt0I6WLxeqW5QonBdGwMDRrYFX/EL8K+FvgSdXuC4F3At83PEtS/1U36rUG5F9l60lCtlCGdpto\nPV4NXJ7JQ30sVeopQ7MGXgTbAW8A3gUsrXb/J2WylLMNz5K0MCJ4NKWbxWEty5PaHHoTZfjQieXH\nmazvV51SPxiaNTQieCzwp5SRNnaudv8XcDJwluFZkuav6hq3H1MD8sHAttMOXU9pOX4kJGdycx9L\nlRphaNbQqe7APpESnnepdp9HaXn+nuFZkupFsAuwnMmAvBx4wrTDEriCyYB8PqUfsjdma+w0Fpoj\n4ijKrHCLgH/JzFPaHHMqcDRl+svfz8yL2xxjaB5T1RjPfwz8BZP/0J9PuYHwm87+JElFBIuBZ1D6\nIR9erZ/a5tB1TA3Iqx0zXyoaCc0RsQi4CnghcDPlRoFXZeaalmOOAU7MzGMi4jDg/2bmil79ATQ6\nqm4bfwT8JbCk2r0G+CDw+Uwebqo2SWpCdbPeRDg+nHKz3g7TDnuQMh7y+UwG5Rv9tk5qr6nQfDjw\n7sw8qnr8DoDMfH/LMf8POCczT68eXwkcmZnrevEH0OiJYAfKDYN/Duxd7b6ZMsPgJ2wtkTSKItiW\nMsTb4UwG5XaThvycEpDPq9aXOtyb1Ln5Zs7FXX7unpRJKybcROlPVXfMXpSvjqStVHdqfyiCDwPH\nA2+jfB35D8C7IvgIcGomtzVYpiR1JYKlTA3Ivwo8ZtphGyjf4p5XLRdk+v+n1IRuQ3OnzdTT03zb\n10XEyS0PV2XmqnnUpBFRtZx8NoLPUSZJeRvwPOAk4C8iOJ0Snn/UYJmSVKsacvMgpna12LfNodcy\nGZDPBy7zZj2pOxGxEljZ9ft02T1jBXByS/eMk4AtrTcDVt0zVmXmF6vHds/QvEWwghKeXwpsU+0+\nDzgV+De/opQ0CCLYg6kB+TnAo6cddj9loqeJgHx+Jnf0s05pHDXVp3kx5UbAFwC3UH74Z7sRcAXw\nIW8EVLci2Jdy0+AbgJ2q3bcAHwE+nsntDZUmacxE8CjKxCGtI1osa3PoVUz2RT4P+Gkmm/tVp6Si\nySHnjmZyyLlPZub7IuJNAJn5seqYfwaOogyk/rrMvKjN+xiaNWfVTYO/C7wFOLDa/TDwFeDjwA+9\ng1xSL0WwN1MD8nOA7aYd9kvKKBYTAfnCTO7sZ52S2nNyE421agasF1BmGjyWyX70V1HC82f82lPS\nXFXTTz+bqV0t9mxz6Bqm9kVeYyuyNJgMzVIlgn2AP6iWPardDwP/BnwMW58ltdEy/XTr7HqHsPX0\n0/dS9UFmckSLe/pYqqQuGJqlaaqZs44F3kiZkXLi+vo58BlK6/N1DZUnqWERLGEyIC+vll2mHZbA\nT5naF/kqZyqVhpehWZpFS+vz65n61eq5wGnAl500RRpdVTeLg5lsQV5O+4lDJqafvrBa/yiTe/tV\np6SFZ2iWOhDBIuDXgdcC/53JiQQeBL4GfA44yym7peEVwTaUbhaHtSwHsXU3iweAHzM59fSFOP20\nNPIMzdIcRbAjJTifwNRBz+8GvgqcDpzjxALSYItgN+BQpna12GnaYQlcwWQL8gXA5f58S+PH0Cx1\noeq+8XvAKylTdk+4nXID4enAud4NLzUrgidShng7tGXdbjSLW5nagrzaLliSwNAs9UwETwdeQQnQ\nv9Ly1DrgDOAbwNmZPNhAedLYiOAJTAbjiZDcbtKQ+4CLgB8xGZJvspuFpHYMzVKPVcNPPYsSnl8J\nPLnl6fXAf1AC9Lcyuav/FUqjI4KdKeMhtwbkJ7U5dD0lIK9uWa51NAtJnTI0SwuoCtAHAS+tlkNa\nnt5MGYXjG5QgfZUtXFJ71c/SnpSRLA6m/CwdzNRfSic8AFzM1IB8td2kJHXD0Cz1UQTLgOMoAXol\nsLjl6RuB71IC9NkOV6VxVY2V/lSmhuODgSVtDn8IuITJcPxjyqx63qgnqacMzVJDItiJMnnKbwIv\nZmog2EyZFOG71XKRIUCjqBqN5ulMDcfPAh7d5vC7KQH5EkpL8iXAlZls7E+1ksaZoVkaANX4sIcA\nRwG/ARzO1FboX1K6cvwAWIUhWkMmgu0oN8g+A3hmy3rfGV5yA5PBeCIkr7ULk6SmGJqlARTB4yiT\nqfwG8CK2noHsfuA/KSH6B5QQ/VBfi5TaqH4B3JepwfgZlMC8uM1LNgJrmBqOL83k7n7UK0mdMjRL\nQyCCvYAjq2UlsP+0Qx6mjAxwPnBetbZVTgumajneDzigZXlatezQ5iUJ/Ay4vFouq9bX2L1C0jAw\nNEtDKII9mAzQ/40SVKa7lRKeL6DcHHVxJnf2q0aNhmpItwPaLE8BFs3wsluZDMUTAXlNJusXvGBJ\nWiCGZmkEVDcVLgdWtCw7tzl0LZP9RC+ulhttkR5vVTB+CqXleGK9H+UbjaUzvCyB64ArKd0rrpzY\n9pczSaPI0CyNoGpM2/0p4Xk55SbDg2j/tfndlNbAKyjh54pqucUwPRqqfsa7USb9aA3GE+tdZnn5\nA0wG4keCMaVbhbNbShobhmZpTESwiBKQDpm2tBv7FsoUwxNB+hrgWkqf1J85hvRgieBRwN7APtWy\nrGV7n+q5bWd5iw1M/v22rq+l9I131jxJY8/QLI2xqkV6D0qf6ANblqcze+vjHVQBulpuAG6qlrWZ\n3LeAZY+NqoX4CZS/o92r9fTtZZRW5Dq3U/6eJsJwa0C+zW8VJGl2hmZJW6nC9K5MhujWr/OfDDym\n5i1+Sek/fVO1vhn4Rctye7W+a5xaMavzuiPl3C6pll2nrZdQ+hFPhON2w7RNt5lyrm+kBOOJZeLx\njZls6OWfRZLGjaFZ0pxUwW93JoP0kylf/+8N7FWt60L1hM2UVuvbgTuBe4B7q6V1e+Lxekof262W\nhZjspfqzPory55m+PLpa7wA8HnhctX78DI93ogTi2bpJtHM3cAtlRIpbpm3fSvml5BYnu5GkhWVo\nltRTVdDcmckQvRewJ/BESmvqE1uWnXr40ZsoAXpzB0tQhktrXRZPe7wtJRj3+t+X9ZRfEu5g8heG\nO6bt+wVVKPZmO0kaDIZmSY2pJshYQgnQu1BCdGvL7PTt7dm6xXdi30L9O7CRyRbtDWzdyr2BctNk\na6v4fTNs35HJAwtUpyRpARmaJQ29qnV7O0rL8PQW5HZLsnXr86Y2jx/IZHM//yySpME038zZyY0p\nktQX1cgPD1WLJEkDY5umC5AkSZIGnaFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFo\nliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJ\nkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqrF4vi+MiF2A\n04F9gOuBV2TmPW2Oux64D9gMbMzM5fP9TEmSJKkJ3bQ0vwM4MzOfCpxdPW4ngZWZeYiBWZIkScOo\nm9B8HHBatX0a8LJZjo0uPkeSJElqVDeheWlmrqu21wFLZzgugbMiYnVEvKGLz5MkSZIaMWuf5og4\nE9itzVN/1fogMzMicoa3eW5m3hoRuwJnRsSVmXnuDJ93csvDVZm5arb6JEmSpNlExEpgZdfvkzlT\n1q0t4EpKX+XbImJ34JzMPKDmNe8G7s/M/93mucxMu3FIkiRpwcw3c3bTPeMM4IRq+wTg622K2j4i\ndqy2dwBeDFzWxWdKkiRJfddNS/MuwJeAZbQMORcRewCfyMxjI+LJwFerlywGPp+Z75vh/WxpliRJ\n0oKab+acd2juNUOzJEmSFloT3TMkSZKksWBoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJ\nkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoY\nmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJ\nkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQa\nhmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJ\nkiSphqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZkiRJqmFoliRJkmrMOzRHxO9ExE8jYnNE\nPHuW446KiCsj4pqIePt8P0+SJElqSjctzZcBLwd+ONMBEbEI+GfgKOBA4FUR8bQuPlMdiIiVTdcw\nSjyfveX57C3PZ+94LnvL89lbns/mzTs0Z+aVmXl1zWHLgWsz8/rM3Ah8EXjpfD9THVvZdAEjZmXT\nBYyYlU0XMGJWNl3ACFnZdAEjZmXTBYyYlU0XMO4Wuk/znsDalsc3VfskSZKkobF4ticj4kxgtzZP\nvTMz/72D9895VSVJkiQNkMjsLtdGxDnAWzPzojbPrQBOzsyjqscnAVsy85Q2xxqwJUmStOAyM+b6\nmllbmudgpg9eDewfEfsCtwCvBF7V7sD5FC9JkiT1QzdDzr08ItYCK4BvRcR3qv17RMS3ADJzE3Ai\n8F3gCuD0zFzTfdmSJElS/3TdPUOSJEkadY3MCBgRH4yINRFxaUR8NSIeP8NxTozSgTlMNHN9RPwk\nIi6OiAv7WeMwceKe3oqIXSLizIi4OiK+FxE7zXCc1+cMOrnWIuLU6vlLI+KQftc4TOrOZ0SsjIh7\nq2vx4oj46ybqHAYR8amIWBcRl81yjNdmh+rOp9dm5yJi74g4p/r//PKIeMsMx3V8fTY1jfb3gKdn\n5kHA1cBJ0w9wYpQ5qZ1oppLAysw8JDOXL3xZQ8uJe3rrHcCZmflU4OzqcTten210cq1FxDHAfpm5\nP/BG4KN9L3RIzOFn9wfVtXhIZr63r0UOl09TzmVbXptzNuv5rHhtdmYj8GeZ+XRKV+I/7vbfzkZC\nc2aemZlbqocXAHu1OcyJUTrU4UQzE7zhsoYT9/TcccBp1fZpwMtmOdbrc2udXGuPnOPMvADYKSKW\n9rfModHpz67XYgcy81zg7lkO8dqcgw7OJ3htdiQzb8vMS6rt+4E1wB7TDpvT9dlUS3Or1wPfbrPf\niVF6L4GzImJ1RLyh6WKGnNdn55Zm5rpqex0w0z9IXp/tdXKttTumXWOEOjufCRxRfV377Yg4sG/V\njR6vzd7y2pyHahS3QygNta3mdH32asi5rXQyMUpE/BXwcGb+a5vjvEOxRQ8mmgF4bmbeGhG7AmdG\nxJXVb7Vjx4l7emuW8/lXrQ8yM2cZk93rs71Or7XprU9eo+11cl4uAvbOzA0RcTTwdeCpC1vWSPPa\n7B2vzTmKiMcCXwH+tGpx3uqQaY9nvD4XLDRn5otmez4ifh84BnjBDIfcDOzd8nhvym8AY6nufHb4\nHrdW69sj4muUrynHMpT04Hx6fbaY7XxWN7Xslpm3RcTuwC9meA+vz/Y6udamH7NXtU9bqz2fmfnL\nlu3vRMRHImKXzLyrTzWOEq/NHvLanJuI2Bb4N+Bzmfn1NofM6fpsavSMo4C/BF6amQ/OcNgjE6NE\nxHaUiVHO6FeNQ6xtX6eI2D4idqy2dwBeTLnhTbOrnbjH67PWGcAJ1fYJlJaRKbw+Z9XJtXYG8Fp4\nZCbWe1q6xGiq2vMZEUsjIqrt5ZThWQ0l8+O12UNem52rztMngSsy80MzHDan63PBWppr/BOwHeUr\nWIDzMvOPImIP4BOZeWxmboqIiYlRFgGfdGKU9iLi5cCpwBLKRDMXZ+bRreeT8tX5V6vzvRj4fGZ+\nr7GiB1j20/NIAAAAsklEQVQn59Prc07eD3wpIv4AuB54BZSJkPD6rDXTtRYRb6qe/1hmfjsijomI\na4H1wOsaLHmgdXI+gd8G3hwRm4ANwPGNFTzgIuILwJHAkigTnr0b2Ba8Nuej7nzitTkXzwV+F/hJ\nRFxc7XsnsAzmd306uYkkSZJUYxBGz5AkSZIGmqFZkiRJqmFoliRJkmoYmiVJkqQahmZJkiSphqFZ\nkiRJqmFoliRJkmoYmiVJkqQa/x+RPNicdItAHAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import pods\n", "pods.notebook.display_prediction(basis=polynomial, num_basis=4)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAEACAYAAABbBguCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADvBJREFUeJzt3V+IpXd9x/HPt7sqCA1RFnKxWdFqsNqSVMVotdSRBFwD\nNSClYf1T/AOGwkrvjPXCLBQFLwoilpCGGLxyL1RoCiFBqIMiMRowieJu2FUDuxsJxn+I5GIXv72Y\niZ5OZuec3Zw/k/m9Xld55vnlme+SHzPvPHvOeaq7AwAAI/uzVQ8AAACrJooBABieKAYAYHiiGACA\n4YliAACGJ4oBABje1Ciuqi9V1VNV9cMd1nyhqk5V1aNV9Yb5jggAAIs1y53ie5IcvtjJqropyWu6\n+5okH0tyx5xmAwCApZgaxd397SS/3mHJe5J8eXPtQ0murKqr5jMeAAAs3jxeU3wwyZmJ47NJrp7D\ndQEAYCnm9Ua72nLs2dEAALxg7J/DNc4lOTRxfPXm1/6fqhLKAAAsRXdvvWm7o3lE8b1JjiY5XlVv\nTfKb7n5qHsOx91XVse4+tuo52F3sC7ZjX7Ad+4LtXM7N2KlRXFVfSfKOJAeq6kyS25O8KEm6+87u\nvq+qbqqq00l+n+TDlzoEAACs0tQo7u4jM6w5Op9xAABg+TzRjlVbX/UA7Errqx6AXWl91QOwK62v\negD2hupezvvfqqq9phgAgEW7nO50pxgAgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAY\nnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAA\nhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgA\ngOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhjc1iqvqcFWdrKpTVXXbNucPVNX9VfVIVf2oqj60\nkEkBAGBBqrsvfrJqX5LHk9yY5FyS7yc50t0nJtYcS/KS7v63qjqwuf6q7r6w5Vrd3TX/PwIAAPzJ\n5XTntDvF1yc53d1PdPf5JMeT3Lxlzc+TXLH5z1ck+eXWIAYAgN1s/5TzB5OcmTg+m+QtW9bcleR/\nq+rJJH+e5J/mNx4AACzetCi++Gsr/uRTSR7p7rWqenWSb1TVdd39u60LN19q8az17l6feVIAANhG\nVa0lWXs+15gWxeeSHJo4PpSNu8WT3pbkM0nS3T+pqp8leW2Sh7derLuPXfakAACwjc0brevPHlfV\n7Zd6jWmvKX44yTVV9cqqenGSW5Lcu2XNyWy8ES9VdVU2gvinlzoIAACsyo53irv7QlUdTfJAkn1J\n7u7uE1V16+b5O5N8Nsk9VfVoNiL7E939qwXPDQAAc7PjR7LN9Rv5SDYAAJZgER/JBgAAe54oBgBg\neKIYAIDhiWIAAIYnigEAGJ4oBgBgeKIYAIDhiWIAAIYnigEAGJ4oBgBgeKIYAIDhiWIAAIYnigEA\nGJ4oBgBgeKIYAIDhiWIAAIYnigEAGJ4oBgBgeKIYAIDhiWIAAIYnigEAGJ4oBgBgeKIYAIDhiWIA\nAIYnigEAGJ4oBgBgeKIYAIDhiWIAAIYnigEAGJ4oBgBgeKIYAIDhiWIAAIYnigEAGJ4oBgBgeKIY\nAIDhTY3iqjpcVSer6lRV3XaRNWtV9YOq+lFVrc99SgAAWKDq7oufrNqX5PEkNyY5l+T7SY5094mJ\nNVcm+U6Sd3X32ao60N1Pb3Ot7u6a9x8AAAAmXU53TrtTfH2S0939RHefT3I8yc1b1rwvyde6+2yS\nbBfEAACwm02L4oNJzkwcn9382qRrkry8qr5ZVQ9X1QfnOSAAACza/innL/7aij95UZI3JrkhyUuT\nPFhV3+3uU893OAAAWIZpUXwuyaGJ40PZuFs86UySp7v7mSTPVNW3klyX5DlRXFXHJg7Xu3v9UgcG\nAIBJVbWWZO15XWPKG+32Z+ONdjckeTLJ9/LcN9r9ZZIvJnlXkpckeSjJLd394y3X8kY7AAAW7nK6\nc8c7xd19oaqOJnkgyb4kd3f3iaq6dfP8nd19sqruT/JYkj8kuWtrEAMAwG62453iuX4jd4oBAFiC\nRXwkGwAA7HmiGACA4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA4Yli\nAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHii\nGACA4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABie\nKAYAYHiiGACA4YliAACGJ4oBABje1CiuqsNVdbKqTlXVbTuse3NVXaiq9853RAAAWKwdo7iq9iX5\nYpLDSV6f5EhVve4i6z6X5P4ktYA5AQBgYabdKb4+yenufqK7zyc5nuTmbdZ9PMlXk/xizvMBAMDC\nTYvig0nOTByf3fzaH1XVwWyE8h2bX+q5TQcAAEswLYpnCdzPJ/lkd3c2Xjrh5RMAALyg7J9y/lyS\nQxPHh7Jxt3jSm5Icr6okOZDk3VV1vrvv3Xqxqjo2cbje3euXOjAAAEyqqrUka8/rGhs3eC/6DfYn\neTzJDUmeTPK9JEe6+8RF1t+T5H+6++vbnOvudhcZAICFupzu3PFOcXdfqKqjSR5Isi/J3d19oqpu\n3Tx/52VPCwAAu8SOd4rn+o3cKQYAYAkupzs90Q4AgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAA\nhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgA\ngOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigG\nAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4ohgAgOGJYgAAhieKAQAYnigGAGB4M0VxVR2uqpNVdaqq\nbtvm/Pur6tGqeqyqvlNV185/VAAAWIzq7p0XVO1L8niSG5OcS/L9JEe6+8TEmr9N8uPu/m1VHU5y\nrLvfuuU63d017z8AAABMupzunOVO8fVJTnf3E919PsnxJDdPLujuB7v7t5uHDyW5+lKGAACAVZol\nig8mOTNxfHbzaxfz0ST3PZ+hAABgmfbPsGbn11dMqKp3JvlIkrdf5PyxicP17l6f9doAALCdqlpL\nsvZ8rjFLFJ9Lcmji+FA27hZvHebaJHclOdzdv97uQt197DJmBACAi9q80br+7HFV3X6p15jl5RMP\nJ7mmql5ZVS9OckuSeycXVNUrknw9yQe6+/SlDgEAAKs09U5xd1+oqqNJHkiyL8nd3X2iqm7dPH9n\nkk8neVmSO6oqSc539/WLGxsAAOZn6keyze0b+Ug2AACWYFEfyQYAAHuaKAYAYHiiGACA4YliAACG\nJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA\n4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYA\nYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA4YliAACGJ4oBABieKAYAYHiiGACA4U2N4qo6XFUn\nq+pUVd12kTVf2Dz/aFW9Yf5jAgDA4uwYxVW1L8kXkxxO8vokR6rqdVvW3JTkNd19TZKPJbljQbOy\nB1XV2qpnYPexL9iOfcF27AvmZdqd4uuTnO7uJ7r7fJLjSW7esuY9Sb6cJN39UJIrq+qquU/KXrW2\n6gHYldZWPQC70tqqB2BXWlv1AOwN06L4YJIzE8dnN782bc3Vz380AABYjmlR3DNepy7z3wMAgJXb\nP+X8uSSHJo4PZeNO8E5rrt782nNUlVjmOarq9lXPwO5jX7Ad+4Lt2BfMw7QofjjJNVX1yiRPJrkl\nyZEta+5NcjTJ8ap6a5LfdPdTWy/U3VvvJgMAwK6wYxR394WqOprkgST7ktzd3Seq6tbN83d2931V\ndVNVnU7y+yQfXvjUAAAwR9XtFQ0AAIxt7k+087APtjNtX1TV+zf3w2NV9Z2qunYVc7I8s/ys2Fz3\n5qq6UFXvXeZ8rMaMv0PWquoHVfWjqlpf8oiswAy/Qw5U1f1V9cjmvvjQCsZkiarqS1X1VFX9cIc1\nl9Sbc41iD/tgO7PsiyQ/TfL33X1tkn9P8l/LnZJlmnFPPLvuc0nuz3M/5YY9ZsbfIVcm+c8k/9Dd\nf53kH5c+KEs148+Lo0l+0N1/k43PLf6Pqpr2vile2O7Jxp7Y1uX05rzvFHvYB9uZui+6+8Hu/u3m\n4UPxWdd73Sw/K5Lk40m+muQXyxyOlZllX7wvyde6+2ySdPfTS56R5ZtlX/w8yRWb/3xFkl9294Ul\nzsiSdfe3k/x6hyWX3JvzjmIP+2A7s+yLSR9Nct9CJ2LVpu6JqjqYjV98z/7fvTdA7H2z/Ky4JsnL\nq+qbVfVwVX1wadOxKrPsi7uS/FVVPZnk0ST/uqTZ2L0uuTfn/VcLHvbBdmb+71tV70zykSRvX9w4\n7AKz7InPJ/lkd3dVVbx8YgSz7IsXJXljkhuSvDTJg1X13e4+tdDJWKVZ9sWnkjzS3WtV9eok36iq\n67r7dwuejd3tknpz3lE814d9sGfMsi+y+ea6u5Ic7u6d/kqEF75Z9sSbsvH550lyIMm7q+p8d9+7\nnBFZgVn2xZkkT3f3M0meqapvJbkuiSjeu2bZF29L8pkk6e6fVNXPkrw2G89bYEyX3JvzfvnEHx/2\nUVUvzsbDPrb+Ars3yT8nyU4P+2BPmbovquoVSb6e5APdfXoFM7JcU/dEd/9Fd7+qu1+VjdcV/4sg\n3vNm+R3y30n+rqr2VdVLk7wlyY+XPCfLNcu+OJnkxiTZfN3oa7PxBm7Gdcm9Odc7xR72wXZm2RdJ\nPp3kZUnu2LwzeL67r1/VzCzWjHuCwcz4O+RkVd2f5LEkf0hyV3eL4j1sxp8Xn01yT1U9mo0bfp/o\n7l+tbGgWrqq+kuQdSQ5U1Zkkt2fj5VWX3Zse3gEAwPDm/vAOAAB4oRHFAAAMTxQDADA8UQwAwPBE\nMQAAwxPFAAAMTxQDADA8UQwAwPD+Dz1kClZY1ABKAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots(figsize=(12,4))\n", "plt.close(fig)\n", "from IPython.display import display\n", "display(fig)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Try moving the sliders around to change the weight of each basis function. Click the control box `display_basis` to show the underlying basis functions (in red). The prediction function is shown in a thick blue line. *Warning* the sliders aren't presented quite in the correct order. `w_0` is associated with the bias, `w_1` is the linear term, `w_2` the quadratic and here (because we have four basis functions) we have `w_3` for the *cubic* term. So the subscript of the weight parameter is always associated with the corresponding polynomial's degree." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question \n", "\n", "Try increasing the number of basis functions (thereby increasing the *degree* of the resulting polynomial). What do you see as you increase number of basis up to 10? Is it easy to change the function in intiutive ways?\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Radial Basis Functions\n", "\n", "Another type of basis is sometimes known as a 'radial basis' because the effect basis functions are constructed on 'centres' and the effect of each basis function decreases as the radial distance from each centre increases. " ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def rbf(x, num_basis=4, data_limits=[-1., 1.]):\n", " centres=np.linspace(data_limits[0], data_limits[1], num_basis)\n", " width = (centres[1]-centres[0])/2.\n", " Phi = np.zeros((x.shape[0], num_basis))\n", " for i in xrange(num_basis):\n", " Phi[:, i:i+1] = np.exp(-0.5*((x-centres[i])/width)**2)\n", " return Phi" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAEACAYAAABI0Ut2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm87XO9x/HXh0OZpSNzuYpCkwYU6tySDEVuiOqi7jVU\n0nWL0ug2S5OhEd1UrqEMV6FIjlBmiRwhzs1MhjgOOZzP/eP7Pc527H1+e1h7/9Ze6/V8PNbjrPVb\nv73XZ//Ob+/1Xt/fd4jMRJIkSdLQFmm7AEmSJKnbGZolSZKkBoZmSZIkqYGhWZIkSWpgaJYkSZIa\nGJolSZKkBmMOzRHxg4i4KyKuXsg+h0XEDRFxVURsMNbXlCRJkiZSJ1qa/xvYcqgnI2Jr4AWZuTaw\nJ/CdDrymJEmSNGHGHJoz83zg/oXssi1wTN33YmD5iFhprK8rSZIkTZSJ6NO8GnDLgMe3AqtPwOtK\nkiRJHTFRAwFjgceu3S1JkqRJY8oEvMZtwBoDHq9etz1FRBikJUmSNO4yc8EG3UYTEZpPA/YBjo+I\njYEHMvOuwXYczQ+gp4uIgzLzoLbr6BUez87yeHaWx7NzPJad5fHsLI9n54y2oXbMoTkijgNeD0yN\niFuAzwCLAWTm9zLzjIjYOiJuBB4G3jPW15QkSZIm0phDc2buMox99hnr60iSJEltcUXA3jS97QJ6\nzPS2C+gx09suoMdMb7uAHjK97QJ6zPS2C+gx09suoN9FZneMv4uItE+zJEmSxtNoM6ctzZIkSVID\nQ7MkSZLUwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7Mk\nSZLUwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLU\nwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAs\nSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAsSZIk\nNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAsSZIkNTA0S5IkSQ0MzZIkSVIDQ7MkSZLUwNAsSZIkNTA0\nS5IkSQ0MzZIkSVKDMYfmiNgyIq6LiBsi4qODPD8tIv4eEVfW2yfH+pqSJEnSRJoyli+OiEWBI4DN\ngduASyPitMycscCu52XmtmN5LUmSJKktY21p3hC4MTNnZuYc4Hhgu0H2izG+jiRJktSasYbm1YBb\nBjy+tW4bKIHXRsRVEXFGRKw3xteUJEmSJtSYumdQAnGTK4A1MnN2RGwFnAqsM9iOEXHQgIfTM3P6\nGOuTJElSH4uIacC0MX+fzOHk3iGL2Bg4KDO3rI8PBOZm5sEL+ZqbgVdm5n0LbM/MtBuHJEmSxs1o\nM+dYu2dcBqwdEWtGxOLAO4DTFihspYiIen9DSlC/7+nfSpIkSepOY+qekZmPR8Q+wK+ARYGjM3NG\nROxVn/8esAPwvoh4HJgN7DzGmiVJkqQJNabuGZ1k9wxJkiSNt9FmzrEOBJQkSWoUwVLAGgNuywFL\nAktQrlbPBh4GZgG3U2bnugW4P3NYEw9I48rQLEmSOiqCKcBGwCbAK+vt+aP8dvdGcBXwR8qMXOdn\nMrMTdUojYfcMSZI0ZhEsB2wPbA28CVh+gV3mAH+tt1uA+ymty48AcyktzksCywKrU1qjnwssPcjL\n/RWYDpwBnJnJg539adTLRps5Dc2SJGlUIlgU2ALYFXgb8MwBT18PnEOZaety4NpM5ozw+wclQL+s\n3jYENgOeNWC3OcC5lHUgfpbJPaP6YdQ3DM2SJGlCRLA08B7gP4C1Bjx1LnASpfX3pnF67UWAlwBv\nBLYDNmX+FLqPA6cDxwCnZ/LYeNSgyc3QLEmSxlXtgvFhYB/mt/beBBwNHJvJ/7VQ04rANsCOwJsp\ngwoB/gb8N/CtNupS9zI0S5KkcRHBM4D3A58Anl03/x74KvC/mTzRVm0DRbAy8C5gd+DFdfNc4BTg\nUOACZ+KQoVmSJHVcBNtRAufz6qYLgI9ncn57VS1c7Qu9IbAvsBPzZwu7EvgScHK3BH1NPEOzJEnq\nmAieBxwOvLVuugY4kNJXuDvCwzBEsCqwd72tWDf/GfgicNxIBydq8jM0S5KkMasD7T5ICZVLAg9R\numV8ezK3zkbwTGA34GPAmnXzzZSW5x8anvuHoVmSJI1JBKsDP6TMTAFwIrBfJre3VlSHRbAY8E7g\n48A6dfMNwKeBEzOZ21ZtmhiGZkmSNGoR7AR8j7Ioyd+APTI5td2qxk+dY3on4L+AtevmP1Ba1c+c\nTF1QNDKjzZyLNO8iSZJ6VQSLR3A4cAIlMJ8BvKSXAzNAJk9kchywPrAncBvwcso8z7+NYJM261P3\nsaVZkqQ+Vbtj/BTYmLKy3n9S5jXujnAwgSJYAvgAZbDjCnXzz4CPjtdCLWqH3TMkSdKwRbAZZfW+\nFYFbgB0yuaTdqtpXF3D5CGURlyWAx4DDgC9k8kCbtakzDM2SJGlYIng3ZRW/xYGzgXdm8rd2q+ou\ntRX+C8CuddO9wGeA7zvTxuRmn2ZJkrRQEUQEnwV+TAnMhwFbGZifLpNbM9kNeDXwW8pKiEcAf4xg\nm7qAivqILc2SJPWBOk/xD4BdKEtLfyiTI9qtanKoAfltwCHA8+vms4EPZ3J1a4VpVOyeIUmSBhXB\nMsCpwBuAWcA7Mjmj3aomnwgWpwwW/DRlppG5wFHApzK5u83aNHyGZkmS9DQRrACcCWwI3AW8OZOr\n2q1qcovg2cBBwPuARSmrJn4BODSTR1ssTcNgaJYkSU8RwarAWZS5iGcCb8rkxlaL6iERrAt8Fdi6\nbroZOAA4qR+n7ZssHAgoSZKeFMGawPmUwHwtsKmBubMymZHJNsCbgT8B/0SZ9/q8CF7VanHqOEOz\nJEk9pgbm6cBawKXA6zK5rc2aelkmZ1FWE3wfZQnyzYBLIzgmgtVaLU4dY/cMSZJ6SATPowTmNYGL\nKH2YH2yzpn5SF0f5BPAhyrR+s4GvAIdkMrvN2lTYp1mSpD4XwRrAeZRuAhdTAvPf262qP0XwfOBg\n4O11022UJbqPzWRua4XJ0CxJUj+rK9idR+mScQmwhYG5fRG8DvgG8Iq66VJgv0wubK+q/uZAQEmS\n+lQEKwPnUgLzZdjC3DUy+S1lVcHdgdvr/QsiODGCf2qzNo2MLc2SJE1iESxPaWF+KXAFsHkm97db\nlQYTwVKUKen2B5YA5gDfB76QyR1t1tZP7J4hSVKfiWBJyjzMmwB/BjbL5J52q1KT2vf888C/AgE8\nAhxKGSx4X5u19QNDsyRJfSSCxShLY28N3Apskslf261KIxHB+sDngO3rpr8Dh1BWFpzVWmE9ztAs\nSVKfiGAR4MfAO4F7KS3MM9qtSqMVwauBLwKb1013U8Lzdw3PnWdoliSpD0QQwGHAPsAs4A2ZXNpu\nVeqECN5ACc8b1U33Al8HvuXAzs4xNEuS1AciOJASrB4Dts7knJZLUgfVD0VbAZ8CNq6bH6B8UDrU\nPs9jZ2iWJKnHRbAjcCKQwI6ZnNRySRonNTy/gRKeX183zwKOBA7LZGZLpU16hmZJknpYBBtSppZ7\nJnBAJoe0XJImSASbAZ8Etqib5gInAd/I5PetFTZJGZolSepRETyXssrfSsBRwJ6ZdMcbuCZMBBsA\n+wG7AFPq5osoKw6enMnjbdU2mRiaJUnqQREsA1wIvAT4DbBlJnParUptimA14APA3sCz6uY7gKOB\nozL5v7ZqmwwMzZIk9ZgIpgD/S5mL+c/Aa1ztT/PUFQZ3BfYFXlQ3J/BLykqDv7D1+ekMzZIk9ZgI\nDqUEovuAjTK5seWS1IXqoMHNgD2BHYBn1KfuAo4HfgJcbpeewtAsSVIPieD9wLeAOcDmmfy25ZI0\nCUTwbGA3SoB+4YCn/kwJz8dmcnMbtXULQ7MkST0igjcDpwOLArtl8qOWS9IkU1ufXwW8mzJwcMUB\nT18BnAycAszotxZoQ7OkvhHB0sBawGr1tjKwFLAksASlT98/6m0WcCdlkMztwA2urKVuFsH6wO+A\nZYEvZPLJlkvSJBfBYpQlut8NbEf5eznPnynh+RfAxf3QB9rQLKknRfAs4DXAJsBLgRcDa47x294G\nXAv8gTIrwe8yuWeM31MaswieA1xMOcd/CuycydxWi1JPiWAJSoDenhKgVxjw9N+Bc4BfAb/q1Vk4\nDM2SekIEiwObAttQJvJ/8SC7zQH+AtxKCcB3AA8Bs4FH6j7PpAyGWRZYhdIavTqwTn1uQdcBZ1Ba\nWy5wSi9NtBpmfkNZOvkSYFrmk+ez1HF1dpbNgLcBW1L+Pg50A3D+gNtNvdCVw9AsadKqlw63AN4F\nvAVYZsDTjwGXAhcAlwPXADeONtRGsCilFW99YCNKC/aGlG4d8zxIuVz5E+DcTJ4YzWtJw1X7n/4P\nsDPwV8pMGXe2W5X6TQRrAm+ut8156t9iKA0UF1Cu0F0O/CGTWRNZYycYmiVNOhG8Gtgd2AmYOuCp\nP1EGQZ1B6WP36DjXsRilC8g2lNC+3oCn76CE5+/0+4hzjZ8I/gv4NKUP/iaZ/LHlktTn6t/Fl1Na\nojejXAGcusBuSekTfXm9XUnp+nZPN7dIG5olTQoRPIMSkvehtPDOM4MSTo/P5KY2apsngnWAd1IG\nzTy/bk5KkD8COKub3xA0uUTwLsq5Pxd4ayZntFyS9DT1asgLKQF6Q+CVlFUqpwyy+32U8DxjwL83\nALd0Q9e31kJzRGwJfJMyLc5RmXnwIPscBmxF6W+4e2ZeOcg+hmaph0UwlbJIw97Mn/rofuCHwI8p\nl/m6KojWN4mNKTXvDCxen/oD8HngFAdpaSwi2ITSj3lxYN9MDm+5JGnYaiPISygBel6IXo8ylmQw\ncyljUW4ecJvJ/LEpdwAPjPd7QSuhOSIWpTTLb075gS8FdsnMGQP22RrYJzO3joiNgEMzc+NO/QCS\nulsEqwAfpgTPedMcXQUcDhyXyey2ahuJOqvBvwMfpAwqhNJ68kXKz2G/Z41IBGtRZsqYCnwrk31a\nLkkas9rYsAolPK9b/12P+dOENmW9R5kfoG8H7gHupbRez/t34P37RzpNXluh+TXAZzJzy/r4YwCZ\n+eUB+3wXODczT6iPrwNen5l3deIHkNSdIlgJ+CSwB/OXdD0T+DJwfre1Kg9XneHgvcBHgTXq5quB\nAyhTNE3Kn0sTK4Llgd8DLwJ+SemW0fPz46q/1Zbp51IGY/9Tva1JCdmr1n+XHsW3nk0ZDzCLMpPS\nrCEezy63+OpoMudg/VBGYjXglgGPb6WMRm/aZ3XKeuhPEcHrKNNFLXibDTzmm5HU/SJYBvgIpXV5\nXsvyycAXM7m8tcI6pE4B9q0IjgT+lTJ46yWUDwS/juCATJ7WBU2apw6w+iklMP+JMhezgVk9L5N/\nUPo23zDUPvU9ZBXmB+mplLmkVwCePcj9Z1EWtloSeM44lj/m0DzcELtgmh/i6w46b/79afU2/2si\nnhKkH6JMwv1g/XfB+/Me30tp2r8buM/+h9L4qPMr7wV8ivl9lk8DPpHJNa0VNk4yeQw4OoJjKV02\nPkHpqnZ5DdQHZnJfmzWq+9RL14dTzpW7gbe4QqU0XyYPUTLe9cPZP4JFKIF56Xpb5un3v/lyOH99\nmLIYTJlCmS1pxMbaPWNj4KAB3TMOBOYOHAxYu2dMz8zj6+Mhu2dAXkCZK3Ww2+KM3Vzgb5QQPS9I\n303pj30rpUX8FuC2+mlI0jBEsAUlCMybGP93wEczuaC9qiZWBM+mBOcPUhok/kbpsnGMH9Y1TwT7\nAV+nLPE+LZOLWi5J6jtt9WmeQhkI+EZKZ+1LWPhAwI2Bb45mIGBdkGBgiF4GWK7elh1wf8HHUymt\nXitSmvCH625KgJ5JuYxwI/MvKdxpVxEJIngeJQD8S910PSUontavvyMRrAd8i/mXyi4E9szk2taK\nUleI4K3A/1Kuvu6cyQktlyT1pTannNuK+VPOHZ2ZX4qIvQAy83t1nyMoyzM+DLwnM68Y5PuM+0DA\n2o9sKqXPy7wgvRKlj/XqlEE9a1D60Cy6kG/1MCVEX08ZAPTH+u9MW5TUDyJ4JrA/cCDlQ+zDwGeB\nb9ZuC32tXoLfhfKBYiXKqoYHAYfYd7U/RfByykpqSwGfzuRzLZck9S0XN+mg2qq9MiVArwW8AFi7\n3l5A6Xw+mFmUJX7/SFkV5xLg6m6YyFvqlAj+Gfg+5XcB4Hhg/0xuba+q7hTBcsAhlBlEAC4D3tOL\nfbw1tAhWpbwfrEZZxGTXfr0SI3UDQ/MEiuBZlAD9Isqo+ZfW28qD7P4ocAVlLs5LgN9l8tcJKlXq\nmDpF1iGUuYqhjPrfJ5PprRU1SdQ+30dSplqaQ2mV/7Ktzr0vgqWA8ygLP1wIvNExM1K7DM1dIIIV\nKSH6ZZQ/kBtSwvWCZgLT6+28TGZOSIHSKEXwL5R+uitTuhp8HjjYrhjDF8GywMGURV6gBKh3+/vf\nu+pVy5OA7YCbgI0zuafdqiQZmrtUBCsAr6bMX70R8Fpg+QV2+z/KMqpnAmdn8sCEFikNoa7mdwTz\nB/pdCOyRyYyhv0oLE8HmwI8oc5A+COyVyfHtVqXxEMHXgf2AB4DXZHJdyyVJwtA8adSWh5cCr6eM\nrn8dT53V4wnKdF1n1ttV9n1TGyLYGfg25fycBXwM+I6DXccugqnAUZQWSCghep86P6l6QAQfoHzg\nnANsYTcmqXsYmiepOin3S4EtgK2ATXnqojO3AadQLvGdn8kTE16k+kqdb/jbwE5105nA3vbF76w6\nw8aewDcoM5DcBLwjk8taLUxjFsHWwM+BRYDdMzmm5ZIkDWBo7hG13+PmlAC9NWX6u3nuAU6lBOjf\nOCuHOq2+2R9N6bs8i3Jp+WivdoyfCNYFjqOMhXiMcsy/4zGfnOrUcudTViP7XCafbrkkSQswNPeg\n2hL1KuDtwA7A8wc8fR9wIvBj4Pe+wWosIlga+Bql5RPKm/7umdzUXlX9I4JnUI7/B+qm4ykLothd\nYxKJYHXgIsrUcv9DGejp32apyxiae1wN0C9lfoBed8DTN1Hm/vxJJje0UJ4msQg2BY6hzEn+GGUp\n6G/YFWji1X7kR1JaKf8M7JjJ1e1WpeGIYBngt/DkIiabO7Wc1J0MzX0mgpcB7wbeyVO7cFxEedM9\nIZOH26hNk0Nt3fwc8BHKsr5/AP7VhTfaFcELgZ8BLwYeAd5nn9juVlebPZXSpe4GykwZ97ZblaSh\nGJr7VJ2N458pAfrtlBYqgIcorc/fz+QPLZWnLlX7Xf6YEszmAl8CPuu8y90hgiUpgzF3q5uOBj6Y\nySPtVaXB1KuAPwB2B+6lBGav+EldzNCseStP7UDpl/raAU9dSml9Pi6TWW3Upu4QwRTgAOAgYDFK\nq9iumVzUZl0aXATvpSwq80zKlYC328+8u0TwJcp0jLOBN2RyccslSWpgaNZTRPBiYA9gV+YvpvIg\npcXqCN94+08Ea1PmA964bvoW8FG78XS32hXrJMpA4AcoH3J+3m5VAojgPyhTBj4BvDWTM1suSdIw\nGJo1qAiWoLQ+78381ucETgMOA851dHdvq5eP3wccAixJmfv7vZmc1WphGrYIlqcM1ty2bvoi8GkH\na7Yngl0oM2QA7JbJj9qsR9LwGZrVKIJXAvsCOwOL183XUMLzsZnMbqs2jY86BdbRlMVzoPRz3zeT\n+9urSqNRF0LanxKYFwHOAd6Zyd2tFtaHItgC+AWli9P+mXy15ZIkjYChWcMWwUrAXpTWx5Xr5vso\nS74ekck9bdWmzqity7tQumAsTxmgtHcmP2u1MI1ZBP9Mmcf5OZSrBjtm8vt2q+ofEWxE+cCyFPC1\nTD7SckmSRsjQrBGLYHFgR+BDwKvr5kcpI8G/nslf2qpNo1eXwf4O5f8W4HRgj0zuaK8qdVIEq1EW\nN3ot8Djwn5QPvN3xB71HRfAK4DfAcpSrNrtlMrfdqiSNlKFZo1ZbJTelzKrwlrp5LmXw0Vcyuayt\n2jQyEWwDHIXLYPe8OjfwV4D/qJuOp3w4coaccRDBS4DpwAqUv407Z/J4q0VJGhVDszoigvUpi128\ni9JfD+Bcypvzrwxf3SmC5SjLMP9b3eQy2H0igp0oV4eWAq6lTEt3XbtV9ZYIXkRZ7W9FSl/mtzun\nuTR5GZrVUXUA2YcofZ+XqZuvpoTnEzKZ01ZteqoI3gp8l7IypMtg96EI1qW0fq5LucLw3kx+2m5V\nvSGCF1AC8yrAWcB2mTzablWSxsLQrHFRWzD3olwCXqVu/iulVfNo5/htTwRTgUMpS6kDXEwJS9e2\nV5XaEsHSlK4576ibvkGZh9sPuKNU5zY/B1iD0jVjG2cZkiY/Q7PGVQTPoCzVvT/wwrp53owbh2fy\nt7Zq6ze1D/qOlGO/IvAIpXX5MFuX+1s9N/YBvg5MAS4Edsrk9lYLm4RqV7VfU8YHXAhsaX9xqTcY\nmjUh6lyx2wEfBTaqmx+hzAX8tUxmtlRaX4jgeZR5tectcnEuZfCXM53oSRG8FvgppcvO3cA7Mpne\nalGTSJ0l4yzg2ZTZMrb1qprUO0abORcZj2LUuzKZm8kpwGuA1wNnAEtQWrdujODYuuyvOiiCxSM4\nEJhBCcwPUbrNbG5g1oIy+R2wAeVD1XOAcyI4oLZEayEieA0lKD+bMl3jWwzMksCWZnVAnYrpAMpi\nGovWzb8CDgamO+PG2ETwBsoiJS+qm44HPuwldzWJYArwWeDAuulUyqwqf2+vqu4VwVuAEyjLzf8M\neJezZEi9x+4Zal3tOrAfsAflTQfgUsqMG6fY33ZkIngu5YPHznXT9cAHMvl1e1VpMopgW+BHlEU5\nbqT0c76y3aq6SwR7Ad+mXIE9Bvh352GWepOhWV2jrkj3AeCDwNS6+Qbgq8CPnK5p4SJYFvgY5QPI\nMymrNH4e+Gom/2izNk1eETyfMi3dy4A5wKco51Rff5itXVa+wPzW+M8Bn/EKmdS7DM3qOhEsCbyH\nsljKmnXzXZSBbEdmck9LpXWlusLbHsBBlFkxoFwqPjCTm9uqS70jgiUoH17fXzedD+zarwN46/E4\nkrKY0xPA3pkc1W5VksaboVldq/ar3JEy48a8QYKPUQLhEZlc0lZt3SCCRSn9wT8NrF03/47Sb/mi\n1gpTz4pgK8oqgisDD1IG8v6kn1pXa/enk4FXAg8DO2ZyZrtVSZoIhmZ1vXoZ9E2UbhvbwJMj+S+l\nzDl8Yj913ahh+R2UsDxv7uu/UD5cnNxPAUYTry6O831g+7rpJGCfTO5sr6qJEcE0ypR8U4Gbge0z\nuarVoiRNGEOzJpUI1gL2Bv4deFbd/DfKAJwfZnJNW7WNt9ptZTdKn+V5Lcs3U/pS/sQV3DRR6gfZ\n3SldppYGHqAsYHR0L35oqx9UD6D8ri0KnA3sksm9rRYmaUIZmjUp1QC5M+Xy8AYDnroM+CFwXCb3\ntVBax0WwMmWA5Psoc8ACzKQM8vuRYVltqTPffBvYum46D9grkz+3V1VnRbA68GNgWt30FeDj/T4Q\nUupHhmZNarXFa0NKq9culKmxoPR9/gXlUurpmTzUSoGjVPtzbwn8G/BW5s9jfQnwNUo3DKe1Uuvq\n7+DOwKGUgaj/oATLgyfz4h7159oB+C6wAmWFxN0y+WWrhUlqjaFZPaOOaN+OEqC3YH7f538Av6Qs\nOnB6Jve3UmCD+ib9Ksob9bspSxlDGZ1/GvB14MJevPytya9OGflVyu8fwO2U6dh+ksnctuoajQjW\noIyXmLfs/C8pi7vc1V5VktpmaFZPimA1SvjcAdiE+QF6LqW19izK6oOXtNliG8EzKEuLbwu8HXju\ngKdvAI6idMHo+UFW6g0RbAp8g/IBEEqXqQMyObe9qoanTt+4N/BFSl/tBylzn39vsgV/SZ1naFbP\ni2BV4F8ooXRTYMqApx8ELgJ+X/+9ZDz7QkewFPByYDPgDbWeJQbscjtlOqsTgQtsVdZkFMEilKsl\nX2L+FZPzgIMymd5WXUOpV3neBnwZWKduPgX4YCa3tVaYpK5iaFZfiWAZ4PXAmyldONYZZLfbgGuB\nGcB1wF8pYfZ24J6mFqcajFekhIXn19s6lLD8Iua3es9zDaXl+2fAxbZoqVfU34X9gA8Dy9fN51Na\nok9rezBdDcubA5+hXJGCslz4/pmc2lphkrqSoVl9rY6M33jA7ZWUJaiH8gQwu94eoSxVvSiweL0t\nz1Nbjhf0OPAnSheRc4Dp9pNUr4tgOWBf4D+ZH55nAodT+jzfPcH1LE6ZZ/oA4BV18z3AfwHfd0Ya\nSYMxNEsD1PlY1wTWA9alLB6yGqXVeFXmT/m2MI9S3oDvAm6iLDzyF+Aq4Jp+WohFGiiCZSkDBfel\nXIGB8kH0l5Rp3X4xXjNu1FblFwO7UuY7n7fk/N2UmT+OyOTB8XhtSb3B0CyNQB0otGS9LUFplX6c\nMsXdY5Q+0g/bF1kaWv1wug2wJ2VqxXlTKj4G/BY4k7KAyIyxDNSt3bFeS+mCsT3zgzrA1ZQ5po/J\n5JHRvoak/mFoliS1JoLnUOZ5fidlzvWBf88fAa4ErqBcrZlJGVvwd+AhIIHFKB9gnwOsDKwFrE9p\nVX4p8wM5lCtApwJHUwb9dscbmaRJwdAsSeoKEUylDNDditJCvNYYv+UTwOWU1uufU+Y5dyU/SaNi\naJYkdaW6YMorgJdRxhqsSWlNXgZYtu42h7KA0d3AncCtlMG2fwKuymTWhBYtqWcZmiVJkqQGo82c\ni4xHMZIkSVIvMTRLkiRJDQzNkiRJUgNDsyRJktRgymi/MCJWAE4AnkeZc3OnzHxgkP1mUhaKeAKY\nk5kbjvY1JUmSpDaMpaX5Y8DZmbkOcE59PJgEpmXmBgZmSZIkTUZjCc3bAsfU+8cAb1vIvk4lJ0mS\npElrLKF5pcy8q96/C1hpiP0S+HVEXBYRe4zh9SRJkqRWLLRPc0ScTVm1aUGfGPggMzMihlolZZPM\nvCMiVgSJzzziAAAHTklEQVTOjojrMvP8IV7voAEPp2fm9IXVJ0mSJC1MREwDpo35+4x2RcCIuI7S\nV/nOiFgFODczX9TwNZ8BZmXm1wZ5zhUBJUmSNK7aWBHwNGC3en834NRBiloyIpap95cCtgCuHsNr\nSpIkSRNuLC3NKwAnAs9lwJRzEbEqcGRmbhMRawEn1y+ZAhybmV8a4vvZ0ixJkqRxNdrMOerQ3GmG\nZkmSJI23NrpnSJIkSX3B0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJktTA0CxJkiQ1MDRLkiRJDQzN\nkiRJUgNDsyRJktTA0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJktTA0CxJkiQ1MDRLkiRJDQzNkiRJ\nUgNDsyRJktTA0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJktTA0CxJkiQ1MDRLkiRJDQzNkiRJUgND\nsyRJktTA0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJktTA0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJ\nktTA0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJktTA0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJktTA\n0CxJkiQ1MDRLkiRJDQzNkiRJUgNDsyRJktTA0CxJkiQ1GHVojogdI+JPEfFERLxiIfttGRHXRcQN\nEfHR0b6eJEmS1JaxtDRfDWwP/HaoHSJiUeAIYEtgPWCXiFh3DK+pYYiIaW3X0Es8np3l8ewsj2fn\neCw7y+PZWR7P9o06NGfmdZl5fcNuGwI3ZubMzJwDHA9sN9rX1LBNa7uAHjOt7QJ6zLS2C+gx09ou\noIdMa7uAHjOt7QJ6zLS2C+h3492neTXglgGPb63bJEmSpEljysKejIizgZUHeerjmfnzYXz/HFVV\nkiRJUheJzLHl2og4F/hwZl4xyHMbAwdl5pb18YHA3Mw8eJB9DdiSJEkad5kZI/2ahbY0j8BQL3wZ\nsHZErAncDrwD2GWwHUdTvCRJkjQRxjLl3PYRcQuwMXB6RJxZt68aEacDZObjwD7Ar4BrgRMyc8bY\ny5YkSZImzpi7Z0iSJEm9rpUVASPikIiYERFXRcTJEbHcEPu5MMowjGChmZkR8ceIuDIiLpnIGicT\nF+7prIhYISLOjojrI+KsiFh+iP08P4cwnHMtIg6rz18VERtMdI2TSdPxjIhpEfH3ei5eGRGfbKPO\nySAifhARd0XE1QvZx3NzmJqOp+fm8EXEGhFxbn0/vyYi9h1iv2Gfn20to30WsH5mvgy4HjhwwR1c\nGGVEGheaqRKYlpkbZOaG41/WpOXCPZ31MeDszFwHOKc+Hozn5yCGc65FxNbACzJzbWBP4DsTXugk\nMYLf3fPqubhBZn5+QoucXP6bciwH5bk5Ygs9npXn5vDMAfbLzPUpXYk/MNa/na2E5sw8OzPn1ocX\nA6sPspsLowzTMBeamccBlw1cuKfjtgWOqfePAd62kH09P59uOOfak8c4My8Glo+IlSa2zEljuL+7\nnovDkJnnA/cvZBfPzREYxvEEz81hycw7M/MP9f4sYAaw6gK7jej8bKuleaD3AmcMst2FUTovgV9H\nxGURsUfbxUxynp/Dt1Jm3lXv3wUM9QfJ83NwwznXBttnsMYIDe94JvDaern2jIhYb8Kq6z2em53l\nuTkKdRa3DSgNtQON6Pzs1JRzTzOchVEi4hPAY5n5P4Ps5wjFATqw0AzAJpl5R0SsCJwdEdfVT7V9\nx4V7Omshx/MTAx9kZi5kTnbPz8EN91xbsPXJc3RwwzkuVwBrZObsiNgKOBVYZ3zL6mmem53juTlC\nEbE08DPgQ7XF+Wm7LPB4yPNz3EJzZr5pYc9HxO7A1sAbh9jlNmCNAY/XoHwC6EtNx3OY3+OO+u89\nEXEK5TJlX4aSDhxPz88BFnY866CWlTPzzohYBbh7iO/h+Tm44ZxrC+6zet2mp2s8npn50ID7Z0bE\ntyNihcy8b4Jq7CWemx3kuTkyEbEYcBLwk8w8dZBdRnR+tjV7xpbA/sB2mfnoELs9uTBKRCxOWRjl\ntImqcRIbtK9TRCwZEcvU+0sBW1AGvGnhGhfu8fxsdBqwW72/G6Vl5Ck8PxdqOOfaacCu8ORKrA8M\n6BKjp2o8nhGxUkREvb8hZXpWQ8noeG52kOfm8NXjdDRwbWZ+c4jdRnR+jltLc4PDgcUpl2ABfp+Z\n74+IVYEjM3ObzHw8IuYtjLIocLQLowwuIrYHDgOmUhaauTIztxp4PCmXzk+ux3sKcGxmntVa0V1s\nOMfT83NEvgycGBH/BswEdoKyEBKen42GOtciYq/6/Pcy84yI2DoibgQeBt7TYsldbTjHE9gBeF9E\nPA7MBnZureAuFxHHAa8HpkZZ8OwzwGLguTkaTccTz82R2AR4N/DHiLiybvs48FwY3fnp4iaSJElS\ng26YPUOSJEnqaoZmSZIkqYGhWZIkSWpgaJYkSZIaGJolSZKkBoZmSZIkqYGhWZIkSWpgaJYkSZIa\n/D+iVUdKsOd7HwAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "pods.notebook.display_prediction(basis=rbf, num_basis=4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fourier Basis\n", "\n", "Fourier noticed that any *stationary* function could be converted to a sum of sines and cosines. A Fourier basis is a linear weighted sum of these functions." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def fourier(x, num_basis=4, data_limits=[-1., 1.]):\n", " tau = 2*np.pi\n", " span = float(data_limits[1]-data_limits[0])\n", " Phi = np.zeros((x.shape[0], num_basis))\n", " for i in xrange(num_basis):\n", " count = float((i+1)/2)\n", " frequency = count/span\n", " if i % 2:\n", " Phi[:, i:i+1] = np.sin(tau*frequency*x)\n", " else:\n", " Phi[:, i:i+1] = np.cos(tau*frequency*x)\n", " return Phi" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this code, basis functions with an *odd* index are sine and basis functions with an *even* index are cosine. The first basis function (index 0, so cosine) has a frequency of 0 and then frequencies increase to 1, 2, 3, 4 etc every time a sine and cosine are included." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAEACAYAAABI0Ut2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYXVXVx/HvIgmh96JAREA6AqEEpEiohiBEuoiIkY5U\nQUjoKFWkhd6lCAhICRKEwGvoBELoIVTpEJAWOiSs94+1kxnCzJyZuWXf8vs8D485Z87cs3I9OXfd\nfdZe29wdERERERFp33S5AxARERERqXVKmkVERERECihpFhEREREpoKRZRERERKSAkmYRERERkQJK\nmkVERERECpScNJvZxWY2wcye7OCYYWb2vJk9bmZ9Sz2niIiIiEg1lWOk+RJgQHs/NLOBwI/cfXFg\nV+CcMpxTRERERKRqSk6a3f0e4IMODtkMuDQdOxqYw8zmL/W8IiIiIiLVUo2a5gWB11ptvw4sVIXz\nioiIiIiURbUmAto021q7W0RERETqRs8qnOMNoE+r7YXSvm8xMyXSIiIiIlJx7j7tgG6haiTNw4G9\ngKvNbHXgQ3ef0NaB3fkLyHeZ2VHuflTuOBqF3s/y0vtZXno/y0fvZXnp/SwvvZ/l092B2pKTZjO7\nClgHmMfMXgOOBHoBuPt57j7CzAaa2QvAp8DgUs8pIiIiIlJNJSfN7r5dJ47Zq9TziIiIiIjkohUB\nG9Oo3AE0mFG5A2gwo3IH0GBG5Q6ggYzKHUCDGZU7gAYzKncAzc7ca2P+nZm5appFREREpJK6m3Nq\npFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQK\nKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGR\nAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlERERE\npICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERER\nESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRURE\nREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhER\nERGRAkqaRUREREQKKGkWERERESmgpFlEREREpEDJSbOZDTCz8Wb2vJkd3MbP+5vZR2b2aPrvsFLP\nKSIiIiJSTT1L+WUz6wGcCWwAvAE8bGbD3f2ZaQ69y903K+VcIiIiIiK5lDrS3A94wd1fdvevgauB\nQW0cZyWeR0REREQkm1KT5gWB11ptv572tebAGmb2uJmNMLNlSjyniIiIiEhVlVSeQSTERcYCfdz9\nMzPbGLgRWKKtA83sqFabo9x9VInxiYiIiEgTM7P+QP+SX8e9M3lvu0GsDhzl7gPS9lDgG3c/sYPf\n+S+wsru/P81+d3eVcYiIiIhIxXQ35yy1PGMMsLiZ/dDMpge2BYZPE9j8Zmbpz/2IRP39776UiIiI\niEhtKqk8w90nmdlewG1AD+Aid3/GzHZLPz8P2ArYw8wmAZ8BvywxZhERERGRqiqpPKOcVJ4hIiIi\nIpWWqzxDRERERKThKWkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlE\nREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkW\nERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqa\nRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICS\nZhERERGRAkqaRUREREQKKGkWERERESmgpFlEREREpICSZhERERGRAkqaRUREREQKKGkWERERESmg\npFlEREREpICSZhERERGRAj1zB9DIzDBgdmAe4r3+Bvgf8KE73+SMTUSkVpkxHXHfnA2YHviMuG9+\nmDUwEWlqSprLyIxZgY2A9YBVgGWBmds49CszxgNPAKOAke68Wq04RURqiRkLAD8D+gMrAYsDvds4\n7iPgeeBh4D7gVnfer16kItLMzN1zxwCAmbm7W+44uiqNiGwA7ApsSoyKtPYp8A7wNdCDGD2ZvY2X\negq4FLjCnbcrFrCISA0wY3pgW2AwkSxPe/9/H/iQuHfOQNw7px2EmAzcDVwNXOnOJxUMWUQaRHdz\nTiXN3ZSS5S2BI4kRZYjyiweAEcCDwGPAB+74NL87K7AMsCqwIbAuMGv68WTgeuB4dx6t8F9DRKSq\nzOgN7AkcCCyQdn8F3A7cCdwPjHdn4jS/Z8DcwHLAasS9cx1anphOJAYeTnXnvxX+a4hIHVPSXEVm\nrAKcTSS9AG8A5wIXu/NmN15vemAg8FtgE1o+BP4NHOrO2FJjFhHJzYwtgZOARdKup4HTgWu7U69s\nxpzEE75dgTXT7knAxcCxKnsTkbYoaa6CNEJyPLAf8SjxTeAYIln+skznWBD4A7Ab8SjSgcuI5PmN\ncpxDRKSazJgXOAvYOu16GhgK/GvaJ3ElnGMF4ABge6Iz1FdEgn68O5+W4xwi0hiUNFeYGUsDVwEr\nECUUpwJ/cufjCp1vbuJDZR+gF1EbPQQ4W503RKRemLExUTYxL3EfOwg4351JFTrfUkTZ3C/TrteA\nfd25oRLnE5H6o6S5gszYBvgbMCPwIrCdOw9X6dyLEaMlm6dd9wE7ufNsNc4vItIdqQZ5CHAs8WTu\nP8S9qyr1xmasAZwJ9E27rgL2due9apxfRGpXd3NOLW7SATPMjEOAfxAJ8+VA32olzADuvOjOFsAW\nwNtE3d7jZuybPpRERGqKGTMR983j0q7DgQ2qOUHPnfuJeSd7E32etwOeMmPTasUgIo1FI83tMKMn\ncD7RDsmJmd6nlqv+rpsxzQn8Ffhd2nULMNidd3PFJCLSmhmzATcDPyU6WvzanZszx7QYcAmwdtp1\nJnBgueaiiEh9UXlGGZnRC7gC2IYYodjenRvzRtXCjM2Bi4A5gbeAX7kzKmtQItL0zJiH6PqzMjFR\nekN3xuWNKqQ2ofsBJxDzRMYA27rzUtbARKTqspVnmNkAMxtvZs+b2cHtHDMs/fxxM+vb1jG1IiXM\nVxEJ88fEI8WaSZgB0oSWFYF7ge8Dd5ixn8o1RCSXlDCPIhLmF4G1aiVhBnDnG3dOAdYCXiZWbR2r\ncg0R6aySkmYz60E85hpALNaxnZktPc0xA4EfufviRC/Nc0o5ZyWZ0QO4kli05CNilOSBvFG1LfUf\nXZcYNelBdPO4zIwZswYmIk3HjNmJEeZlgXHA2rW6wIg7DxFLdd9IrM56kxlDNeggIkVKHWnuB7zg\n7i+7+9fEUqaDpjlmM6LdEO4+GpjDzOYv8bxll26Yw4CtiDq8DdwZnTeqjrkzyZ2htJSR/Bq414yF\n80YmIs0iTfq7mRhhfom4d76VN6qOufMBMbn6kLTrOODK9HcREWlTqUnzgkQPzCleT/uKjlmoxPNW\nwiHE0q5fAZu5MyZzPJ3mzrXA6sQH1krAQ2asljcqEWl0rZ7OrU2sjFrzCfMU7rg7xxMDPZ8QfZ3v\nMaNP3shEpFb1LD6kQ52dRTjtY682f8/Mjmq1OcrdR3Ujpi4zYwdiZT8nJv3dVY3zlpM7T5qxKtHm\naQNglBk7uHNd5tCkHenpxozEpKRewMeazS91ZkrS+SGwUa2WZHTEnZvNWB0YTgw6PGzGptVsLSpd\n0+reOQvwBfCpO5PzRiW1zMz6A/1Lfp1SumeY2erAUe4+IG0PBb5x9xNbHXMukQBfnbbHA+u4+4Rp\nXitL9wwz+gF3A72JxvdnVjuGckoTGc8Gdk67DgZOytkqT8CM7wE/AVYjJnH+EFgYmGGaQz8k+nE/\nBTxKTKx6qFKrp4l0lxm/I7r4TAIGuHNn5pBKklZhvQZYD/gc+KU7w/NGJanccB2i5/bSwFLAfMRA\nQ2sfERNQxwMPAvcAj+uzT9qSpeWcmfUEngXWJ9oLPQRs5+7PtDpmILCXuw9MSfZp7r56uf4CpTDj\n+0TboQWAc93Zo5rnr5T0LfyPwJQvLxcCe7rzdb6omo8ZyxKTSjcj6j3b8mX6bxIxKalHG8d8BFxP\ntEEcpWXUJTcz1iRW+OsF7OrOBZlDKos06HAe0Z//G2L57boeSKk3qTXgasRcnUHAIu0c+iVRVtOb\nGHFuy6vEvfMid54qc6hSx7L1aTazjYHTiA/7i9z9eDPbDcDdz0vHTOmw8Skw2N3HtvE6VU2azZge\nuIuoBb6HqMX7qlrnrwYztiQSrRmAO4Ct3fkwb1SNzYxZgG2Jkf7WXw4/B+4nRkDGAC8Ar7jzcavf\nnY7ovd0HWJ6YaLshsESr13kGOAW4rNGuV6kPZsxLPAVZEDjdnf0yh1RWadDhMOBPadcpwB/1ZbWy\n0nW1M7AbfGsy+0fEZ/V9wNPESPLrrUvZ0r1zbmAxYAVi5dwNiJasU9xDlGGO1OizaHGTLp+Pk4E/\nEJMUV3HnnWqdu5rShMDhxOOsp4FN3Hklb1SNJ5Vf7A/sAcyadk8ErgVuAP7Pnc+7+dpLAtsTK0FO\nmWj7IlF6c70+AKRa0sS/EcBGRBKzbqM+wTLjN0T5SU/gn8AO3f03LO0zY3niyeg2wPRp92vAdcT9\n86Hu1CunRLof8BtgB1pGo+8FDnHnnhJDlzqmpLlL52IT4F/AZKIB/4PVOG8uZvyQ+KBbmqiX3bSe\nuoPUsjTTfgiwE/GYECKZuAC4zp1Py3iuXsQHy+HAkmn3vcDO7jxbrvOItMeMw4kR2P8Bfd15PXNI\nFWXGesTj/dmBB4BB7rybN6rGYMYKwBFE6z+Iifi3EGs/jCznyL4ZsxLdsQ4C5kq7LyGeILxXrvNI\n/VDS3OnzsCDwOPEoZ4j71LrfhmbGHMRoyXpET+ftNMml+9L7ORTYl5Zk+Qbg+ErPuk/J887A0cC8\nxOzxocAwPUKWSkkJ5B1pc4A7t+eMp1rS3IQRwA+IsqqB7jyfN6r6ZcaixKJcW6ddXwLnA6dWuvuK\nGbMBBxADHdMTX/52c+f6Sp5Xao+S5k6dgx7AncRM3NuBjZspyUh13OcDOxLf6vdzZ1jeqOpLeg/3\nJEZ7p4xY/AP4sztPVzmWOYj5BDumXXcSX4Y0EiZlZcZcwJPEpOk/uXNk5pCqKk0avwXoC7wH/MKd\ne/NGVV/SqpGHEgMN0xNf9s8D/uLOm1WOZUngXFpakJ0JHKiWn81DSXOnzsEhwLFEicKK7kwo+JWG\n08Ykl9OBA9TjslgaaTubltKIu4jHe1n7uZqxKVEOMj8xW3wLdx7JGZM0FjOuBLYjJrP+tBnvF2mS\n79XAJsQiWL9156q8UdW+VFu8E/HZO2/afRlwaM7ynvRZuA9wEtEF5hFgc/dvLcYmDUpJc+Hr82Pi\nH0Uv4GfN8mixPWZsD1xMfOMfDvyqnPW3jSRN8jsZ+FXa9RxwIPCvWpmEl8qOriM6dnwJ7OjOP/JG\nJY3AjG2JZPFTYAV3XswcUjZm9AROBfZKu44AjqmV+0CtMWM5YjR5jbTrXmD/WppTkxYFu4bonf8m\nMVn+saxBScV1N+csdRntupBqQC8lEuZzmz1hBnDn78QM+A+IPsJ3peRQEjN6mPF7osXRr4jHiYcB\ny7tzcy19ULrzBvGo8XyixvoqM/bOGpTUvfRl7Jy0+YdmTpgB3Jnkzt5EiYETT+z+lsq2JDFjJjOO\nJ1oTrkE83f0l8ZSiZhJmgPSkcGXiyeECxFLqG+eNSmpVU4w0m3EEMWnqZSLh+bjj32geqbZrBLAo\n8Wh/YLVrc2tRmtl9IbBK2jWCWDHypXxRFUuPHA8iJtpAPBI9vJYSfKkP6VoaQfTYHwH8XNdRCzM2\nA64CZiJW7tzSnfezBlUDzPgZ8UVrEeKLxTlEi7ePsgZWwIzeRIvB7YnFprZz57q8UUmlqDyj3ddl\nReBhotfmeu78p9znqHepqfxw4tH+ROLmf0fHv9WYzJiRmOR3ELFgz+vEqNIN9ZQwmDGYqHPuAfyF\n6BRTN/FLfqlP8aXA+8By7ryVOaSaY8bKwM3EIhrPEo/2m3I03oz5iInJ26VdTxCrRY7OF1XXpC+K\nJxJ9oycD26vMrTGpPKMNqf7sEiJhPlMJc9tSt4X1iJZ0swG3mrFb3qiqz4x1iHaEQ4l/G2cAy7jX\n3wIi7lwCbEWMmBxEPGkR6ZSUAJ2aNvdXwty2NOF2NaKzyJLAaDPWzxtVdZlh6QvWM0TC/Dlxz1ml\nnhJmgHSfP5h4QtcDuNJs6pcAkcZOmomZsSsSZRlD8oZS29JKV9sQM4l7AueacYEZM+SNrPLMmN2M\n84hHrIsTN/+13Nmnnkt53LmRqCOcDBxuxqGZQ5L6cSrRUnEkcHnmWGpa6rawFnAr0f//djMOSKOW\nDc2MRYDbiCcScxGtXJd156R6XSkyJc6HA0cROdJlqnGWKRq2PMOMHwDjgJmJR2YjyvXajc6MHYgJ\nZTMQpS1bNmobHjN+QbSR+z7wNTHCcEIj9es045fA34kPgJ3duShzSFLDUoIwghgxXK7W6/hrRVoH\n4GiY+uX0auLfW8N1JUp/132BPxM13e8D+wOX19tTuY6Y8ReiVONzYH13HsgckpSJapq/83rcCAwC\n/unOVuV63WZhRl9i+dgfAu8StV0jswZVRqlTyBkw9dp4kPiAa8hJkKnc5lxi1HlTd27NHJLUIDNm\nBp4GFgYOcuekzCHVHTM2J/oQzwI8BWzrzri8UZWPGcsTk6RXTbuuIhbKeidfVJWRnhZcBAwmOk2t\n6c4zeaOSclBNcytmDCIS5o+Jb8PSRe48SnSOuJ1oSH+7GX9NM4zrVmojtztRgrEV0Xt2H6IcoyET\nZgB3zgOOI+r0rjVjpcwhSW06nEiYH6Wlplm6wJ0bgH7ExMDlgDFm7Fbv5RpmzGzGccR6B6sSk6Q3\ndedXjZgww9RSjV2Bm4A5geFpdUxpUg030pxWbRoH9AH21TLRpUmP4YYS9V09iA/TX7kzPmdc3WFG\nP6IUY+W061Zgd3dezRdV9aQP7UuBHYgPvJUb9cNOui61n3ySmNOwujsPZQ6prqXPojOA36ZdNxBP\ns+qqLV26b2xBfInqQ7SRO4toI1e3cz66Ij2BuYdYRv1OYON6rdmWoJHmFkcS/7AfIf5hSwncmezO\nMcREl/8SN42xZuyTEuqaZ8Y8ZpxPlGCsTCSMWxO17k2RMMPUUZNdiKWQFyJGnHvljUpqQUqMhhEL\nQF2khLl07nzizmCio8REYHPgyfQktC6kL1K3EauN9iEGTdZ0Z+9mSZgBUl36IGACsD56CtO0Gmqk\n2YyliJGSHkC/Wlt5qN6ZMRtwJjFSCfAQsIs7T+SLqn2p88dewCHEo7VJxHLYx7jzSc7YcjLj+8AY\nYvWrM9MKZ9LEUh3u9cCHwBKpDaWUiRk/JCbjTllO+jpisaS3swXVATPmIe6bexFfpD4gJjie787k\nnLHlZMbqRJel3sAe7pybNyLpLo00h1OIR4sXKGEuP3cmuvMbYsTkDaJub2yqdZ4zb3QtUt3yb4Hn\niBZ6cxKP1JZ3Z0gzJ8wAqefulsBXwF5pIRRpUmbMRMvI2aFKmMvPnZeBnxJzbD4l5lM8Y8Z+tbQE\ntxmzmHE48BLRDaMXMelvSXfOaeaEGcCdB4mndQCnp8VtpIk0zEizGQOBW4CPgMV146+sNOp8LPB7\nwIiRiGOBs9z5IlNMvYi+xEOAZdLuJ4hm9bc1UiukcjBjJ+ID8QviycyTmUOSDMw4GjgCeIxYkKKp\nE6NKM2NhYm7FwLTrJeKedV2ue1S6n+9CLEoyX9r9b6Ju+dEcMdUyM84C9iT+v1up1pcIl+9q6pZz\n6Zv6k8ASwAHunFLW4KRdZqxCjOb2T7teBf4KXFyt/qTphv9b4ADgB2n3K8BhwJXufFONOOqRGRcC\nOxHdRFZtxJ6y0j4zFiUmTvcmOsjclzmkppBqyDch7p1Lpd2PAicQbVKr8sUllWrtA+wBzJ52jwaG\nuDOqGjHUo1T6dz9Mbc26lQZl6kuzJ837E6UZzwE/duersgYnHUofABsDJxItliBGns8HLnHn2Qqd\n8yfAzsC2RIN9iDZPJwJ/13VQLM0KfxhYmpgAtnPmkKSKzLiOKNW5wn3qXAWpEjN6El9ajwbmT7tf\nJHqqX+7OhAqcsxeRsA9O/ztlQvfdwF+AEUoAi5mxGDAWmA116qo7TZs0mzEfkSzPjlb+yyp10xhE\nrKC0eqsfPQhcS7R4G9/dG3J6ovCTdI5BwKKtfnw3UZc5XCPLXWPGj4nEuTfRTvCqzCFJFZixFtFG\n6zNi8t8bmUNqWmbMCOxI3Dun3NcmEyUSNwG3uPNmCa8/C7ABsGn6b95W57gJOCnV60oXmLEV8dn2\nNVHi9ljmkKSTmjlpPo9oPv5vd60PXyvMWIMYQdmGWBlrileJJPoRYrWs14hJhR8T3S0AZiUm7y0I\nLEk8vlydmHg4Q6vXmgD8jSgFea5Cf5WmkBZ8OYf4/2EFd/6bOSSpIDOmAx4g/k39yZ0jM4ckTB14\nmDIK/HNiYvsU44iORWOA54kStLeAz9yZlH53yr1zUeBHwPLEvXMF+FaL0HHAJcQThprs4FEvzDgH\n2J1YSXOVXHN6pGuaMmk2Y0Xi8chkojOClresMenx/6bEB8EAYJ6CX3HocOWsZ4gJnzcBD2jSUnmk\ncpdriUf1dwPrasS+cZmxHXAl8DYxcbqpO8rUIjPmJ56o/ZwYJZ6xg8MnQ4d98ycTyfbN6b8nVYJR\nHukz7lFgceBUd/6QOSTphGZNmm8HNgROd2e/ykQm5ZJGQpYlFhhZmbjJ9CEW2piJlpv+Z0RN9ASi\nRvlZ4svR/e68V+Wwm0bqzfoUUVu5vzunZQ5JKiCVAownJs3u7M5FmUOSAmb0JkaL+wErAYsQy53P\nRyTT0xEDDh8THaReBl4g7p2jgYc1ybdy0mqz9xOfYRu4c2fmkKRA0yXNZqwP3EGstLSokqn6lx4Z\nT+c+tUxDqsyMzYhR/C+AvvW4XLp0zIwhwPFEO8aV9LSm/qXJfZP1dCgfM44EjiJWnF3enQ/yRiQd\naarFTdKj5BPS5olKmBuDO98oYc7LneFEnfgMwKVpdr80iDRx+pC0eYAS5sbgztdKmLM7lhjVX4hY\neVYaUF0mzcRqSqsQ9XinZ45FpNHsR0zQ7EcsdiCN42histgId+7IHYxIo0gDPjsCXwKDzdgoc0hS\nAXVXnpEeQz1N1MNq7XeRCjBjA2Ak0UppZa0WWP/MWIqoWYd4fDwuZzwijahV+dMrwHKaZFubmqk8\nYzCRMD8PmsAiUglpFPJcoBdwQZrEKfXtGGKi0kVKmEUq5q/ExPWFieRZGkhdjTSbMRMxI/j7wLbu\nXFOV4ESaUFqefBzRL3sfd87IHJJ0U5rdP5qY4LlYKQtliEjHzFiBaPHXE1jbnXszhyTTaJaR5n2I\nhHkscF3mWEQamjsTgb3S5nFm9MkZj3TPNBOnT1fCLFJZ7jxOy7+5i1KbR2kAdTPSbMZcwEvEctkb\nuTOyasGJNDEz/glsAfwL2EyLItSXNCHpNuBDoj2nWmGJVFjqrT0WWAY4zp1DM4ckrTTDSPPBRMJ8\npxJmkaram1gw4efA1pljkS5Ivc+n1FWeoIRZpDrc+RLYJW3+0Yylc8Yj5VEXSbMZCxGlGQBDcsYi\n0mzS4/yD0+YwM+bMGY90ydbECnJvgmrSRarJnfuBC4gJ1WenUimpY3WRNANHEostXOvOmNzBiDSh\nC4B7iCW2j80ci3RCas95TNo82p3PcsYj0qSGAP8D+gO/zhuKlKrma5pTb9GnAQeWcee5qgcnIpix\nHPAY8WV7VXceyRySdMCM3YFzgOeAZbXapkgeZuxIrLT6LrCUO+/njUgauab5GCLOi5Qwi+TjzlPA\naYARjxrr4f7RlFJ7ziPT5mFKmEWyugy4C5gX9W6uazU90tyqt+jnwI/UKkkkLzNmBcYDCwC7uXN+\n5pCkDWYMBY4DHiGeCtTGjV6kSaWJgI8T9c1ruPNA5pCaWsONNKu3qEjtcedjYP+0ebwZ8+SMR74r\nteecMnFziBJmkfzceYZYLRDgXDN65oxHuqdmk2ZgI2Bd4APgxMyxiEiLa4E7gLnQo8ZaNIRoz3lH\nWg5dRGrDMcDLwPLAHnlDke6oyfKMVCv5CLAicJA7J2UNTkS+xYwlgSeJR40/cefBzCEJU9tzPk90\nG1pV3YZEaosZg4AbicWGlnDn3cwhNaVGK8/YhkiY3wDOzByLiEzDnWdpedR4thk9csYjU6k9p0ht\nG06s0DkHMe9A6kjNjTSbMT0wDlgM2MWdCzOHJiJtMGNm4t/qD4C93fUFN6c0+v902lR7TpEalVrp\nPgn0APrpC271NdJI885Ewjye6GsoIjXInU+B/dLmn82YL2c8wjHEh/DFSphFapc742lp33mm2nfW\nj5oaaQafFXiBWHVsS3euzxyWiHQgdbm5FfgZ0Ut958whNSUzVgEeBr4g2nO+kTkkEemAGbMBzwLf\nAwa7a5CwmhplpHlfImF+CLghcywiUiC1M9sH+BrYKfVWl+qbUht5hhJmkdrnzkTgoLR5ohmz54xH\nOqfWkuYpF5B6i4rUiVQKcHLaPEuPGqvLjPWBDYGPaOltLyK17wrgfmA+WlbwlBrW7fIMM5sL+Aew\nMNF3cBt3/7CN414GJgKTga/dvc2RqFSeAXCbOwO6FZSIZGHGLMQ8hAXRBN6qSeUxDwL9gEPdNRtf\npJ6YsRIwhsiRVnBnXOaQmkKO8owhwEh3XwK4M223xYH+7t63vYR5GkNLiElEMnDnE+CAtHlCWpVO\nKu8XRMI8ATg9cywi0kXujAXOB3oCw9IXYalRpSTNmwGXpj9fSty829PZi+Aqdx4tISYRyecaYBQw\nN/DnvKGaRf+oAAATCUlEQVQ0vrQM77Fp88+pm4mI1J9DidWP1we2yBxLwzNjzu7+bilJ8/zuPiH9\neQIxga8tDtxhZmPMbJeC1zy8hHhEJKM0D2Fv4jHj7mb0zRxSo9sBWBr4L3BB5lhEpJvceQ84LG2e\nYsZMOeNpAt3ONXt29EMzG0m0Q5nWoa030qok7RVHr+nub5nZvMBIMxvv7ve0c8YdrGVMepS7j+oo\nPhGpLe48ZcYwYH+i/+hamtRbfmbMABydNg9356uc8YhIyc4DdgVWIMpdj8gbTmMxs/5Af1hwdvjd\n3t1+nRImAo4napXfNrPvA/9x96UKfudI4BN3P7mNn3WrKFtEaktqnfQs8fTpN+5cnjmkhmPGfsCp\nwBNAX3e+yRySiJTIjLWAe4AviVU9X8ocUsMx42/AjmBUeyLg8DgxpP+98bvB2UxmNmv688zARsTS\nkSLSoNz5iJb2kSelJv5SJun9nPK071AlzCKNwZ17iTZ0vYFTMofTcMxYDvgNsa5At5SSNJ8AbGhm\nzwHrpW3MbAEzuyUd8z3gHjN7DBgN/Mvdby/hnCJSH6b0H50fOCpvKA3nD8A8wH3ALQXHikh9OQj4\nBBhkxsa5g2kwxxGNKc7r7gvU1DLaKs8QaRxpIuAjwDdE/9GnM4dU98yYF3gJmAVYO41MiUgDMeNA\n4CTgeeDH7nyZOaS616r05VNgMbC3G2EZbRFpEKl95LlAD+AM9R8ti8OIhPkWJcwiDWsYsVjU4sSk\nailB+uyZslrqKe5M6Oj4Dl9LI80iUilpkZPniN7N27pzTeaQ6pYZiwPjiC8hK7rzROaQRKRCzNgQ\nuJ0YGV3Kndczh1S3zNgcuB74H7CYOxNzrAgoItIhd96nZZXPk9Ny29I9xxNtQi9RwizS2NwZSSR6\nMxOlGtINZvQCTkybR7szsaTX00iziFSSGT2AB4FVgOPdOSRzSHXHjDWIiX+fA4u780bmkESkwsxY\nmCjTmAFY151ReSOqP2bsBZxB1Icv6x6dMzTSLCI1yZ3JwF5p80AzlsgZT71J9Xh/TZsnK2EWaQ7u\nvEJ0fICYF9LhgnTybWnNgKPS5kFTEuZSKGkWkYpzZzRwMdALOF2TArtkS+AnwDvAXzLHIiLVdRLR\nMWc5YM/MsdSbocR8mnuAm8rxgirPEJGqMGM+YlLg7MAv3MtzE2tkZkxPTP5bDNjDnXMzhyQiVWbG\nZkTSNxFYopTuD83CjB8Qnze9gdXceejbP1d5hojUMHfeAQ5Pm6eZMWPOeOrE7kTCPB64MHMsIpLH\nzcCtwGzEhGApdiyRMF89bcJcCo00i0jVpJq8R4DlgaPcOTpzSDXLjDmAF4jHi5u5c3PmkEQkk9Ry\n8ilgemD1VPImbTBjZWAM8BXRru+/3z1GI80iUuPcmUTLpMAhZiySM54adySRMN8F/CtzLCKSkTvP\nAyenzbNSVyKZRpovc0raHNZWwlzS62ukWUSqzYwrgO2BG93ZPHc8tcaMZYAnAANWcufxzCGJSGZm\nzEyUai0E/N6dszOHVHPM2Ba4mljIZAl3Pmj7OI00i0j9OAj4BPiFGQNyB1NL0kjJacTKf+crYRYR\nAHc+BfZLm8ebsUDOeGpN+lIxpT3nIe0lzKVQ0iwiVefOmzC1nnmYGb1zxlNjNgM2BD6kZeKkiAjE\nKoE3E5MCh2WOpdYMIUbhxxItTstO5RkikkVqp/Y4sBQxKtD0s8LNmIFoMbcIsLc7Z2YOSURqTGqn\nNo5YYluThAEzFiXek97AWu7c1/HxKs8QkTrizlfA3mnziDQ7vNn9gUiYnwL1ZBaR73LnVeCwtHmm\nGbPkjKdGnEwkzFcUJcyl0EiziGRlxmXADkSXiPXc+SZzSFmY0Qd4hhg9Wt+d/8sckojUqNS+czSw\nEnCqO3/IHFI2ZmwE3AZ8Skz+e7P4dzTSLCL1aX/gXWAdYOfMseQ0jEiY/6mEWUQ6ktp37gJ8A+yb\nehM3nVTSdkba/HNnEuZSKGkWkazceY+WMo2TzFgwZzw5mDEI+AXRUWS/gsNFRHBnLHA6kctdYEav\nzCHlMBRYgmjFd1qlT6akWURqwTXAcGJG+Dmp7VpTMGNWmDrh71B3Xs8Zj4jUlSOAV4C+RALZNMxY\nipa/867ufFnpcyppFpHs3HFgT2AisCmwTd6Iqupook3SI8BZmWMRkTrizifA79Lm4WasmDOeajFj\nOuA8oBdwoTv3VOW8mggoIrXCjF2JG+F7wHLuvJ05pIoyoy8wJm2umh63ioh0iRlnAr8nVhJdNXUn\nalhm/A64CHgHWNqd97v2+5oIKCL170JgJDA3cFEjl2mkPtWXEPfhM5Qwi0gJDgZeBJYnSjYalhnf\no2Xlv/27mjCXQkmziNSM1G5uMLEa3kBidnijOgxYAXiJlp6rIiJdlpbYHgw4MMSMVTOHVBFpIOUC\nYE6izdxV1Ty/kmYRqSnuvAHskTZPMWOxnPFUQmoPdUjaHJzqEkVEui3V9Z4G9AAuM2PmzCFVwo7A\nz4GPgJ3TfJiqUdIsIjXHnauBq4m+xZenRv4NwYzewKXEB9vp7tydOSQRaRyHEoskLUUVWrBVU1o+\n/PS0uXeOTkNKmkWkVv0eeAP4CfCnzLGU01HAssDztIw2i4iUzJ3PgW2BL4Gdzdg2c0hlkbplXEy0\nJb0RuCJLHOqeISK1yox1gP8jvuAPdOfWzCGVxIx1gTvT5lru3J8zHhFpTGbsAZxNtPHs685LmUMq\niRn7EiPn/wOWdeed0l5P3TNEpMG4cxdweNq83Iw+OeMphRnzAX8HDDhGCbOIVNC5wA3EyOxVqVtP\nXTJjFeCktLlrqQlzSbFopFlEall6LHcLMAC4H+jvztd5o+qa9Hf4F7AxcA+wnjuT8kYlIo3MjLmA\nx4A+wDnu7Jk5pC4zY3ZgLLAocKY7e5fndTXSLCINKLWh24Gob16DlhGHevJHImF+H9heCbOIVFrq\nX7w18BWwh1l9tfBs1V5uUeBR4j6alZJmEal57vyPWFr7a2BfM3bOHFKnmfEz4Li0Odid13LGIyLN\nw53RwG5p8ywz1swZTxcdSCT9HwPbuPNF5niUNItIfUg1wLunzbPNWDtnPJ1hxo+I1nnTAX9yZ3jm\nkESkybjzN2AY0Av4pxkL5Y2omBkDgRPT5m/deSFnPFOopllE6ooZpwD7E7Oo13Dn+cwhtcmM2Yga\n7GWBm4AtUqmJiEhVmdGLWEFvXeBxYB13PsobVdvMWAoYTUxiPMqdo8t/ju7lnEqaRaSupIVOhhM1\nwi8Da7rzZtagppEWMBkBrEcsNLC6OxPzRiUizcyMeYD7gCWAUcDGtVDy0JoZ3yNiXBT4J1GWUfbB\nBk0EFJGmkCbRbUOMRPwQuM2MObMG1UrqlHEZkTC/DWyihFlEcktzQ34GvAX0J9p49sgaVCtmzAH8\nm0iYHyHKMmrq6ZySZhGpO+58AmxCjOIuB9yabrhZpdnew4ikfiIwwJ3/5o1KRCS48zLRvnMisBXw\nt/T0LiszZiTK2FYAniNGwT/JG9V3KWkWkbrkznvEqMkrwGrAnWbMnSueNMJ8FrH891fAIHcezxWP\niEhb3HmCGHT4FPg1MeKcLXE2YxbgZuCnwJvARu68myuejihpFpG6ldq3/RR4EVgJGJVq4qoqPeK8\nENgD+BLY3J1R1Y5DRKQz3LkX2Iho5/ZL4B9ptLeq0hPC24H1gQlEwvxKtePoLCXNIlLX3HmVSJzH\nE6UaD5mxYrXOb8bMwDXAYOBz4OfujKjW+UVEuiO18dwQ+AjYAvhPNQcdzFgQ+A/wE+BVYG13nq7W\n+btDSbOI1L3UPWMd4AFiydj7zNii0udN/U7vJj5wptQw31Hp84qIlENa/GQtWsrcRldj0MGM1YAx\nwIrA80TCXJPtQ1tT0iwiDcGdd4iOFZcDMxFN/E+v1CNHMzYFHiPKQl4i2srdXYlziYhUijtPEQnz\ng8APgAfN2DtNbC4rM6YzY39isOF7ROu7NdITw5qnpFlEGkbqOboj8EdgErAP8IgZ65TrHGbMbcaF\nRK/ouYkFA1Zz55lynUNEpJrcmUAMOpwP9Ca6AI00Y4lynSO91u3AKcD0xMTpjVIrvLqgxU1EpCGZ\nsTLwd2DJtOtq4Gh3xnfz9WYEdgGOIJLlr4EhwGm11ktURKS7zNiSSJ7nIjoBnQGclBLr7rzevMCB\nxEquvYjVXHdyZ3h5Iu5OTFoRUETkW1Ki+0dgKDAD4MCNwMXAbe583YnXWBTYAdgdpk6S+T/g991N\nwEVEallaPfBE4Hdp1+dE6dvFwEPudJg8ptKOlYknfzvB1DK5S4Ch3U3Ay0VJs4hIO8xYmEicBxOP\nBQE+IOrqHiRqkicA3xD10AsDPyYmF/641UuNBf4M3FT0oSEiUu/MWIl4ujao1e43iIGDR4l2nx8R\n5XBzEzXRqxAdjRZp9TsjgD+liYfZKWkWESlgxgLEyMcOwNKd/LXPgOuBS4E7lSyLSLMxYxli0OHX\n0Om2dBOAfwAX19pCT1VPms1sa+AoYClgVXcf285xA4DTIJr/u/uJ7RynpFlEqiI9OlwEWBvoS4ws\nz5t+/CXRfukF4F7iUeQXOeIUEakl6d65HNCfmC+yGDAzUav8ATEK/QTR/vMxdyblibRjOZLmpYhH\nmecBB7SVNJtZD+BZYAPijXwY2M7dvzPLXElz+ZhZf3cflTuORqH3s7z0fpaX3s/y0XtZXno/y0vv\nZ/l0N+fsdss5dx/v7s8VHNYPeMHdX3b3r4nZ64MKfkdK1z93AA2mf+4AGkz/3AE0mP65A2gg/XMH\n0GD65w6gwfTPHUCzq3Sf5gWB11ptv572iYiIiIjUjZ4d/dDMRtJ2wfch7n5zJ15fE2ZEREREpO6V\n3D3DzP5D+zXNqwNHufuAtD0U+KatyYBmpgRbRERERCquOzXNHY40d0F7Jx4DLG5mPwTeBLYFtmvr\nQE0CFBEREZFa1e2aZjPb3MxeA1YHbjGzW9P+BczsFgB3nwTsBdwGjAP+0VbnDBERERGRWlYzi5uI\niIiIiNSqSnfPaJOZnWRmz5jZ42Z2vZnN3s5xA8xsvJk9b2YHVzvOemFmW5vZ02Y22cxW6uC4l83s\nCTN71MweqmaM9aQL76euz04ws7nMbKSZPWdmt5vZHO0cp+uzHZ251sxsWPr542bWt9ox1pOi99PM\n+pvZR+lafNTMDssRZz0ws4vNbIKZPdnBMbo2O6no/dS12Xlm1sfM/pM+z58ys33aOa7T12eWpBm4\nHVjW3VcAngOGTntAWhjlTGAAsAywnZl1dtnbZvMksDlwd8FxDvR3977u3q/yYdWtwvdT12eXDAFG\nuvsSwJ1puy26PtvQmWvNzAYCP3L3xYFdgXOqHmid6MK/3bvStdjX3Y+papD15RLivWyTrs0u6/D9\nTHRtds7XwP7uvixRSvz7Uu+dWZJmdx/p7t+kzdHAQm0cpoVROqmTC81MoQmXBbRwT9ltBlya/nwp\n8IsOjtX1+V2dudamvsfuPhqYw8zmr26YdaOz/3Z1LXaCu99DLJ/cHl2bXdCJ9xN0bXaKu7/t7o+l\nP38CPAMsMM1hXbo+c400t/Y7YEQb+7UwSvk5cIeZjTGzXXIHU+d0fXbe/O4+If15AtDeDUnXZ9s6\nc621dUxbgxHSuffTgTXS49oRZrZM1aJrPLo2y0vXZjekLm59iYHa1rp0fZar5dx3dGZhFDM7FPjK\n3a9s4zjNUGylDAvNAKzp7m+Z2bzASDMbn77VNh0t3FNeHbyfh7becHfvoCe7rs+2dfZam3b0Sddo\n2zrzvowF+rj7Z2a2MXAjsERlw2poujbLR9dmF5nZLMB1wL5pxPk7h0yz3e71WbGk2d037OjnZvZb\nYCCwfjuHvAH0abXdh/gG0JSK3s9OvsZb6X/fNbMbiMeUTZmUlOH91PXZSkfvZ5rU8j13f9vMvg+8\n085r6PpsW2eutWmPWSjtk+8qfD/d/eNWf77VzM42s7nc/f0qxdhIdG2Wka7NrjGzXsA/gSvc/cY2\nDunS9Zmre8YA4I/AIHf/op3Dpi6MYmbTEwujDK9WjHWszVonM5vJzGZNf54Z2IiY8CYdK1y4R9dn\noeHAjunPOxIjI9+i67NDnbnWhgO/gakrsX7YqiRGvq3w/TSz+c3M0p/7Ee1ZlZR0j67NMtK12Xnp\nfboIGOfup7VzWJeuz4qNNBc4A5ieeAQL8IC772lmCwAXuPsm7j7JzKYsjNIDuEgLo7TNzDYHhgHz\nEAvNPOruG7d+P4lH59en97sn8Hd3vz1b0DWsM++nrs8uOQG4xsx2Al4GtoFYCAldn4Xau9bMbLf0\n8/PcfYSZDTSzF4BPgcEZQ65pnXk/ga2APcxsEvAZ8MtsAdc4M7sKWAeYx2LBsyOBXqBrszuK3k90\nbXbFmsCvgSfM7NG07xDgB9C961OLm4iIiIiIFKiF7hkiIiIiIjVNSbOIiIiISAElzSIiIiIiBZQ0\ni4iIiIgUUNIsIiIiIlJASbOIiIiISAElzSIiIiIiBZQ0i4iIiIgU+H/SM7SgTvfOigAAAABJRU5E\nrkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "pods.notebook.display_prediction(basis=fourier, num_basis=4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fitting to Data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are going to consider how these basis functions can be adjusted to fit the Olympic Marathon data. First we will scale the output of the data to be zero mean and variance 1." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [], "source": [ "data = pods.datasets.olympic_marathon_men()\n", "y = data['Y']\n", "x = data['X']\n", "y -= y.mean()\n", "y /= y.std()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Redefining the Basis\n", "\n", "Now we are going to redefine our polynomial basis. Have a careful look at the operations we perform on `x` to create `z`. We use `z` in the polynomial computation. What are we doing to the inputs? Why do you think we are changing `x` in this manner?\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtIAAAEACAYAAACedJETAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGr5JREFUeJzt3X+sX3d93/HnK3EyGKB6iMpA4syeSERTUQjZnAjT5tJR\nGmItgKArbG0ZlbaICoFo5/LDVLiTMmjdain9AVEHVcomoGJdlDZhaYpiflgoEBIHk8QQq7GWpMRM\nCoYGU5GQ9/74Hud+fX2v43vO+d7vud/v8yFd3fPjc87nk8/3G5+3P36fzydVhSRJkqTVOWPaDZAk\nSZLWIwNpSZIkqQUDaUmSJKkFA2lJkiSpBQNpSZIkqQUDaUmSJKmFToF0kqcluS3J/iT3JPnAMmUW\nknw3yZ3Nz/u61ClJkiQNwYYuF1fVPyZ5RVUdS7IB+GKSl1fVF5cU/VxVXdmlLkmSJGlIOqd2VNWx\nZvNs4EzgkWWKpWs9kiRJ0pB0DqSTnJFkP3AEuLWq7llSpICXJbkryU1JLuxapyRJkjRtfYxIP1FV\nLwHOBX4mycKSIncAm6vqxcAfAtd3rVOSJEmatlRVfzdLfgv4QVX93inK3A9cXFWPLDneX0MkSZKk\nFVRVL2nHnV42TPIc4PGqOprk6cDPAb+9pMwm4NtVVUm2MQrel8uj7u0/SquTZHdV7Z52O+aV/T9d\n9v/02PfTZf9Pl/0/PX0O3nYKpIHnAdclOYNRmsjHq+qzSa4CqKprgTcAb03yOHAMeGPHOiVJkqSp\n6zr93QHgpcscv3Zs+4+BP+5SjyRJkjQ0rmwogL3TbsCc2zvtBsy5vdNuwBzbO+0GzLm9027AnNs7\n7Qaou15fNuwiSZkjLUmSpEnqM+Z0RFqSJElqwUBakiRJasFAWpIkSWrBQFqSJElqwUBakiRJasFA\nWv1IdpBsXHJsI8mOKbVIkiRpogyk1Zd9wNVPBtOj31c3xyVJkmaO80irP4vB8x5gJ7CLqqPTbZQk\nSdKiPmNOA2n1K9kC3A9sperwVNsiSZK0hAuyaJhGI9I7ga3AzpNypiVJkmaIgbT6sZjWsasZid7F\neM60JEnSjDG1Q/0Yzc6x74Sc6FEQvZ2qG6fWLkmSpDHmSEuSJEktmCMtSZIkTVmnQDrJ05LclmR/\nknuSfGCFch9Kcl+Su5Jc1KVOSZIkaQg2dLm4qv4xySuq6liSDcAXk7y8qr54vEySK4AXVNX5SS4B\nPgxc2q3ZkiRJ0nR1Tu2oqmPN5tnAmcAjS4pcCVzXlL0N2JhkU9d6JUmSpGnqHEgnOSPJfuAIcGtV\n3bOkyDnAA2P7DwLndq1XkiRJmqZOqR0AVfUE8JIkPwbcnGShqvYuKbb0zchlpwpJsntsd+8y95Ek\nSZJOW5IFYGEi9+5z+rskvwX8oKp+b+zYRxgFxZ9s9g8Cl1XVkSXXOv2dJEmSJmow098leU6aleuS\nPB34OeDOJcVuAH6lKXMpcHRpEC1JkiStN11TO54HXJfkDEZB+cer6rNJrgKoqmur6qYkVyQ5BHwf\neEvHOiVJkqSpc2VDSZIkzY3BpHZIkiRJ88pAWpIkSWrBQFqSJElqwUBakiRJasFAWpIkSWrBQFqS\nJElqwUBakiRJasFAWpIkSWrBQFqSJElqwUBakiRJasFAWpIkSWrBQFqSJElqwUBakiRJasFAWpIk\nSWrBQFqSJElqwUBakiRJaqFTIJ1kc5Jbk9yd5OtJ3r5MmYUk301yZ/Pzvi51SpIkSUOwoeP1jwHv\nrKr9SZ4JfDXJLVV175Jyn6uqKzvWJUmSJA1GpxHpqnq4qvY3248C9wLPX6ZoutQjSZIkDU1vOdJJ\ntgAXAbctOVXAy5LcleSmJBf2VackSZI0LV1TOwBo0jo+DbyjGZkedwewuaqOJXk1cD1wwQr32T22\nu7eq9vbRPkmSJM2nJAvAwkTuXVXdbpCcBfw18JmquuY0yt8PXFxVjyw5XlVlCogkSZImps+Ys+us\nHQE+CtyzUhCdZFNTjiTbGAXvjyxXVpIkSVovuqZ2bAd+CfhakjubY+8FzgOoqmuBNwBvTfI4cAx4\nY8c6JUmSpKnrnNrRF1M7JEmSNGmDSe2QJEmS5pWBtCRJktSCgbQkSZLUgoG0JEmS1IKBtCRJktSC\ngbQkSZLUgoG0JEmS1IKBtCRJktSCgbQkSZLUgoG0JEmS1IKBtCRJktSCgbQkSZLUgoG0JEmS1IKB\ntCRJktSCgbQkSZLUgoG0JEmS1EKnQDrJ5iS3Jrk7ydeTvH2Fch9Kcl+Su5Jc1KVOSZIkaQg2dLz+\nMeCdVbU/yTOBrya5paruPV4gyRXAC6rq/CSXAB8GLu1YryRJkjRVnUakq+rhqtrfbD8K3As8f0mx\nK4HrmjK3ARuTbOpSryRJkjRtveVIJ9kCXATctuTUOcADY/sPAuf2Va8kSZI0DV1TOwBo0jo+Dbyj\nGZk+qciS/VrhPrvHdvdW1d4+2idJkqT5lGQBWJjIvauWjWlP/wbJWcBfA5+pqmuWOf8RRkHxJ5v9\ng8BlVXVkSbmqqqUBtyRJktSbPmPOrrN2BPgocM9yQXTjBuBXmvKXAkeXBtGSJEnSetNpRDrJy4HP\nA19jMV3jvcB5AFV1bVPuj4DLge8Db6mqO5a5lyPSkiRJmqg+Y87OqR19MZCWJEnSpA0mtUOSJEma\nVwbS60myg2TjkmMbSXZMqUWSJElzy0B6fdkHXP1kMD36fXVzfHUMyiVJkjoxkF5Pqo4CuxgF01sY\nBdG7muOr1V9QLkmSNId82XA9GgXR9wNbqTrc4T7Hg+c9wE7aB+WSJEnrgi8bzrNR8LsT2ArsPCk9\nYzVGQfMeRkH5HoNoSZKk02cgvZ4sjiDvakaij6d5tAum+wzKJUmS5oypHevJ6EXAfSeMHI+C3+1U\n3bjKe40H5UdP2pckSZpBLsii7voMyiVJktYJA2lJkiSpBV82lCRJkqbMQFqSJElqwUBakiRJasFA\nWpIkSWrBQFqSJElqoXMgneRjSY4kObDC+YUk301yZ/Pzvq51SpIkSdO2oYd7/Bnwh8Cfn6LM56rq\nyh7qkiRJkgah84h0VX0B+M5TFHN+aEmSJM2UtciRLuBlSe5KclOSC9egTkmSJGmi+kjteCp3AJur\n6liSVwPXAxesQb2SJEnSxEw8kK6qfxjb/kySP0ny7Kp6ZGnZJLvHdvdW1d5Jt0+SJEmzK8kCsDCR\ne1dV95skW4C/qqoXLXNuE/Dtqqok24C/qKoty5Trbd1zSZIkaTl9xpydR6STfAK4DHhOkgeA9wNn\nAVTVtcAbgLcmeRw4Bryxa52SJEnStPUyIt0HR6QlSZI0aX3GnK5sKEmSJLVgIC1JkiS1YCAtSZIk\ntWAgLUmSJLVgIC1JkiS1YCAtSZIktWAgLUmSJLVgIK3hSXaQbFxybCPJjim1SJIk6SQG0hqifcDV\nTwbTo99XN8clSZIGwZUNNUyLwfMeYCewi6qj022UJEla7/qMOQ2kNVzJFuB+YCtVh6faFkmSNBNc\nIlyzbzQivRPYCuw8KWdakiRpygykNTyLaR27mpHoXYznTEuSJA2AqR0antHsHPtOyIkeBdHbqbpx\nau2SJEnrnjnSkiRJUgvmSEuSJElT1jmQTvKxJEeSHDhFmQ8luS/JXUku6lqnJEmSNG19jEj/GXD5\nSieTXAG8oKrOB/4T8OEe6pQkSZKmqnMgXVVfAL5ziiJXAtc1ZW8DNibZ1LVeSZIkaZrWIkf6HOCB\nsf0HgXPXoF5JkiRpYtbqZcOlb0YOY6oQSZIkqaUNa1DHQ8Dmsf1zm2MnSbJ7bHdvVe2dXLMkSZI0\n65IsAAsTuXcf80gn2QL8VVW9aJlzVwBvq6orklwKXFNVly5TznmkJUmSNFF9xpydR6STfAK4DHhO\nkgeA9wNnAVTVtVV1U5IrkhwCvg+8pWudkiRJ0rS5sqEkSZLmhisbSpIkSVNmIC1JkiS1YCAtSZIk\ntWAgLUmSJLVgIC1JkiS1YCCt2ZbsINm45NhGkh1TapEkSZoRBtKadfuAq58Mpke/r26OS5IkteY8\n0pp9i8HzHmAnsIuqo9NtlCRJmoY+Y04Dac2H0TL29wNbqTo81bZIkqSpcUEWaTVGI9I7ga3AzpNy\npiVJklowkNZsW0zr2NWMRO9iPGdakiSpJVM7NNtGs3PsOyEnehREb6fqxqm1S5IkTYU50pIW+ZcF\nSZJOmznSksY5xZ8kSVPgiLQ0C5ziT5Kk02Jqh6STOcWfJElPaVCpHUkuT3IwyX1J3rXM+YUk301y\nZ/Pzvq51SlrCKf4kSVpznUakk5wJfAN4JfAQ8BXgTVV171iZBeDXq+rKp7iXI9JSGydO8Xf0pH1J\nkvSkIY1IbwMOVdXhqnoM+CTwmmXKGSBLk7Od8aB59HtXc1ySJE1I10D6HOCBsf0Hm2PjCnhZkruS\n3JTkwo51ShpXdeNJI89VR536TpKkydrQ8frTyQu5A9hcVceSvBq4HrhguYJJdo/t7q2qvR3bJ0mS\npDnWpBkvTOTeHXOkLwV2V9Xlzf57gCeq6ndOcc39wMVV9ciS4+ZIS5IkaaKGlCN9O3B+ki1JzgZ+\nEbhhvECSTUnSbG9jFLw/cvKtJEmSpPWjU2pHVT2e5G3AzcCZwEer6t4kVzXnrwXeALw1yePAMeCN\nHdssSZIkTZ0LskiSJGluDCm1Q5IkSZpLBtKSJElSCwbSkjQUyY6TlndPNpLsmFKLJEmnYCAtScOx\nD7j6yWB6cbn3fdNslCRpeb5sKElDshg87wF2Mr78uySpsz5jTgNpSRqaZAtwP7CVqsNTbYskzRhn\n7ZA0GeboTt+o/3cCW4GdJ30ekqTBMJCWNM4c3Wla7O9dzUj0LsY/D0nSoJjaIelE5uhOz2jkf98J\n/T36PLZTdePU2iVJM8QcaUmT1UeOrkGhJGmAzJGWNDn95eiaJiJJmmkG0tI0DPWlvj5zdEcj0cev\n3zJ2X9NEJEkzwdQOaRpODFiPnrQ/vXb1n47hVG6SpAExR1qaBfPwUt88/DdKktYVA2lpVszyaG3f\no+6+vChJ6oEvG0qzYPYX3tjOeNC8mDO9veX9fHlRkjQonQPpJJcnOZjkviTvWqHMh5rzdyW5qGud\n0ro3DwtvVN140shz1dHWo8e+vChJGphOqR1JzgS+AbwSeAj4CvCmqrp3rMwVwNuq6ooklwB/UFWX\nLnMvUzs0P0xTaG+W02EkSRM3pNSObcChqjpcVY8BnwRes6TMlcB1AFV1G7AxyaaO9UrrW9+jtfNi\n9tNhJEnryIaO158DPDC2/yBwyWmUORc4svRmCftXWf9qh9PbDL9Puo4htmkt6hhim9aiDtvU8pqz\n+OGGF3LXCw7ywkOPcfafjvYPHjiYHx56jLMfn0ab1mkdQ2zTWtRhm4ZTxxDbtBZ1DLFNa1HHENvU\nm66B9Ok2fOnw+QrX7X7x4vZC8yNJ8Bhnc4CfgtFfxE/alyRpeXubn/51DaQfAjaP7W9mNOJ8qjLn\nNseWsXs1LyKuNrelTS7MpOsYYpvWoo4htmkt6rBNw6ljiG1aizqG2Ka1qMM2DaeOIbZpLeoYYpvW\noo6BtGmBscHZwG9fu8o6Vq6848uGGxi9bPivgb8HvsypXza8FLjGlw0lSZI0DYN52bCqHgfeBtwM\n3AN8qqruTXJVkquaMjcBf5fkEHAt8Gsd2yxJWkvJjpNe7Ew2NrPPSNLccmVDSdKp9b1KpSRNkUuE\nS5LW1mLwvIfRFIQG0ZLWJQNpSdLaczEcSTNgMDnSkqSB6juv2cVwJOkkBtKSNJv2AVc/GfAupmbs\nW/WdTsyJPgzsOuHekjSnTO2QpFnVV17zaBR73wnXju693WXtJa035khLkk6Pec2SdAJzpCVJT828\nZkmaKANpSZpF5jVL0sQZSEvSbNrOeE706Peu5riW4wqOklbJHGlJksAVHKU54cuGkiRNgis4SjPP\nQFqSpElxphNppjlrhyRJk+BMJ5JWwUBakiRwphNJq2ZqhyRJ4AqO0pwwR1qSJElqoc+Yc0OHRjwb\n+BTwz4HDwL+tZd5sTnIY+B7wI+CxqtrWtk5JkiRpKLrkSL8buKWqLgA+2+wvp4CFqrrIIFqSJEmz\noksgfSVwXbN9HfDaU5Q1ZUOSJEkzpUsgvamqjjTbR4BNK5Qr4G+T3J7kP3aoT5IkSRqMU+ZIJ7kF\neO4yp3aN71RVJVnprcXtVfWtJD8O3JLkYFV9YYX6do/t7q2qvadqnyRpnXFmDElrLMkCsDCRe7ed\ntSPJQUa5zw8neR5wa1W98CmueT/waFX9/jLnnLVDkmbdiXM1Hz1pX5ImbCgrG94AvLnZfjNw/dIC\nSf5pkmc1288AXgUc6FCnJGk9GwXLxxc62YJBtKR1rMuI9LOBvwDOY2z6uyTPB/60qnYk+RfAXzaX\nbAD+Z1V9YIX7OSItSfNiFETfD2xtVhGUpDXhgiySpPVrMZ1jD7ATR6TVJ/Pw9RSGktohSdLqnJgT\nfZjFNI+NU22XZsk+xr9Ti9+5fa3uluw46fuZbGwCds05A2lJ0lrazvgI9GLO9PZV32nIAc6Q2zbr\n+s/D7zcw10wxtUOStD4NeQaQIbdtXvSZh2860kwxR1qSJBh2gDPkts26SfS9L8jODHOkJUmC4/+M\nv4dRgLNnUIHqENs2Dyknk8jDH127E9gK7BxETv88fJbrgIG0JGn9GmKAc9ww2zbMfN9+g8L+8vCP\nt2OYL8gO87OcMwbSkqT1abgBTr9t6zPIHO6COP0FhVU3nvTfU3W0w9R3w3xBdrif5VwxR1qStD4N\neb7gPts2iRcXh5jvOw855fPyWQ6cLxtKkjRP+gwy+7rXJP4iMw9B4RA/yznjy4aSJM2Tvl5c7Dcd\npu+FT4aYU96/YX6WaslAWpKkoesvyOwv37fPHN15CgqH+FmqNVM7JEkasqEv7tJHOsaQ8937NPTP\nsg/r4LM0tUOSpPkx3JHHvkZX+59pY6iG+1n2Z66m5XNEWpIkrd48jK6qnYG/BOmsHZIkabrWwT/h\n6zTN2Qwsg0jtSPILSe5O8qMkLz1FucuTHExyX5J3ta1PkiQNyPykY8wDZ2BpqUuO9AHgdcDnVyqQ\n5Ezgj4DLgQuBNyX5iQ51agKSLEy7DfPM/p8u+3967Pvpsv+na1D97wwsrbUOpKvqYFV98ymKbQMO\nVdXhqnoM+CTwmrZ1amIWpt2AObcw7QbMuYVpN2COLUy7AXNuYdoNmHML027ACfqa33o+Xqh80qRn\n7TgHeGBs/8HmmCRJkobCGVhaOWUgneSWJAeW+fk3p3n/YbzJKEmSpOXNWTpGnzrP2pHkVuA3quqO\nZc5dCuyuqsub/fcAT1TV7yxT1qBbkiRJE9fXrB0b+rgJsFJjbgfOzyhx/e+BXwTetFxBp76TJEnS\netJl+rvXJXkAuBS4MclnmuPPT3IjQFU9DrwNuBm4B/hUVd3bvdmSJEnSdA1mQRZJkiRpPZnYrB1J\nPpbkSJIDY8e2JflykjuTfCXJv2qOPy3JJ5J8Lck9Sd49ds3FzQuO9yX5g0m1d9as0P8vTvKlpp9v\nSPKssXPvafr4YJJXjR23/1dpNX2f5OeS3N4cvz3JK8ause9bWO13vzl/XpJHk/zG2DH7v4UWf/b8\nVHPu6835s5vj9v8qrfLPHp+7PUuyOcmtGS1W9/Ukb2+OP7uZvOGbSf4mYy/w+eztx2r7vtdnb1VN\n5Af4aeAi4MDYsb3AzzfbrwZubbb/A/CJZvvpjOYwPK/Z/zKwrdm+Cbh8Um2epZ8V+v8rwE83228B\n/kuzfSGwHzgL2AIcYvFfK+z/yfb9S4DnNts/CTw4do19P+H+Hzv/aeBTjF6ctv/XqP8ZvadzF/Ci\nZv+fAWfY/2vS9z53++//5wIvabafCXwD+Angd4HfbI6/C/hgs+2zd3p939uzd2Ij0lX1BeA7Sw5/\nC/ixZnsj8NDY8WdktBLiM4AfAt9L8jzgWVX15abcnwOvnVSbZ8kK/X9+cxzgb4HXN9uvYfQH6mM1\nmvbmEHCJ/d/Oavq+qvZX1cPN8XuApyc5y75vb5XffZK8Fvg7Rv1//Jj939Iq+/9VwNeq6kBz7Xeq\n6gn7v51V9r3P3Z5V1cNVtb/ZfhS4l9HaGVcC1zXFrmOxP3329mS1fd/ns3fSC7Is9W7g95P8X0ar\n57wXoKpuBr7H6H/sw8CeGk3mfQ6jRVyOewgXdOni7iTHV5b8BWBzs/18Tuzn4wvnLD1u/7e3Ut+P\nez3w1RqtAup3v1/L9n+SZwK/CexeUt7+79dK3/8LgEryf5J8NcnO5rj9359l+97n7mRlNFvZRcBt\nwKaqOtKcOgJsarZ99k7Aafb9uE7P3rUOpD8KvL2qzgPe2eyT5JcY/dPS8xitqPOfk2xd47bNg18F\nfi3J7Yz+6eOHU27PPDll3yf5SeCDwFVTaNs8WKn/dwP/raqOsfI0nupupf7fALwc+HfN79cl+Vlc\nzKtPy/a9z93Jaf6C/r+Ad1TVP4yfq1G+gN/vCVlt3/fx7O1rHunTta2qXtlsfxr47832y4D/XVU/\nAv5fkn3AxcAXgXPHrj+XxXQQrVJVfQP4eYAkFwA7mlMPceII6bmM/kb2EPZ/L07R9yQ5F/hL4Jer\n6v7msH3fo2X6/4rm1Dbg9Ul+l1G62RNJfsDo87D/e3KK7/8DwOer6pHm3E3AS4H/gf3fi1N8933u\nTkCSsxgFch+vquubw0eSPLeqHm5SB77dHPfZ26NV9n1vz961HpE+lOSyZvtngW822webfZI8g9Hc\n1Aeb/JXvJbkkSYBfBq5HrST58eb3GcD7gA83p24A3pjk7GZE4nzgy/Z/f1bq++YN4huBd1XVl46X\nr6pvYd/3Zpn+/whAVf1MVW2tqq3ANcDVVfUnfvf7dYo/e24GXpTk6Uk2AJcBd9v//Vnpu4/P3d41\n/fVR4J6qumbs1A3Am5vtN7PYnz57e7Lavu/12TvBNyg/wWg1wx8yGnV4C/AvGeWs7Ae+BFzUlP0n\njEYgDgB3c+Kb8xc3xw8BH5pUe2ftZ5n+/1Xg7YzeZP0G8F+XlH9v08cHaWZWsf8n3/eMHmyPAneO\n/TzHvl+b/l9y3fuBXx/bt//XoP+Bfw98venrD9r/a9P3Pncn0v8vB55oYpzjf55fDjyb0Yue3wT+\nBtg4do3P3in0fZ/PXhdkkSRJklpY69QOSZIkaSYYSEuSJEktGEhLkiRJLRhIS5IkSS0YSEuSJEkt\nGEhLkiRJLRhIS5IkSS0YSEuSJEkt/H/U15KSUZFeJgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def polynomial(x, num_basis=4, data_limits=[-1., 1.]):\n", " centre = data_limits[0]/2. + data_limits[1]/2.\n", " span = data_limits[1] - data_limits[0]\n", " z = x - centre\n", " z = 2*z/span\n", " Phi = np.zeros((x.shape[0], num_basis))\n", " for i in xrange(num_basis):\n", " Phi[:, i:i+1] = z**i\n", " return Phi\n", "\n", "import pods\n", "#x[:, 0] = np.linspace(1888, 2020, 1000)\n", "fig, ax = plt.subplots(figsize=(12,4))\n", "\n", "ax.plot(x, y, 'rx')\n", "pods.notebook.display_prediction(basis=dict(rbf=rbf, polynomial=polynomial, fourier=fourier), \n", " data_limits=(1888, 2020),\n", " fig=fig, ax=ax,\n", " offset=0.,\n", " wlim = (-4., 4., 0.001),\n", " num_basis=4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task\n", "\n", "Use the tool provided above to try and find the best fit you can to the data. Explore the parameter space and give the weight values you used for the \n", "\n", "(a) polynomial basis\n", "(b) RBF basis\n", "(c) Fourier basis\n", "\n", "Which is the easiest to intuitively fit to the te data?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "np.asarray([[1, 2, 3, 4]]).shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We like to make use of *design* matrices for our data. Design matrices, as you will recall, involve placing the data points into rows of the matrix and data features into the columns of the matrix. By convention, we are referincing a vector with a bold lower case letter, and a matrix with a bold upper case letter. The design matrix is therefore given by\n", "$$\n", "\\boldsymbol{\\Phi} = \\begin{bmatrix} 1 & \\mathbf{x} & \\mathbf{x}^2\\end{bmatrix}\n", "$$\n", "\n", "\n", "### Non-linear but Linear in the Parameters\n", "\n", "One rather nice aspect of our model is that whilst it is non-linear in the inputs, it is still linear in the parameters $\\mathbf{w}$. This means that our derivations from before continue to operate to allow us to work with this model. In fact, although this is a non-linear regression it is still known as a *linear model* because it is linear in the parameters, \n", "$$\n", "f(\\mathbf{x}) = \\mathbf{w}^\\top \\boldsymbol{\\phi}(\\mathbf{x})\n", "$$\n", "where the vector $\\mathbf{x}$ appears inside the basis functions, making our result, $f(\\mathbf{x})$ non-linear in the inputs, but $\\mathbf{w}$ appears outside our basis function, making our result *linear* in the parameters. In practice, our basis function itself may contain its own set of parameters,\n", "$$\n", "f(\\mathbf{x}) = \\mathbf{w}^\\top \\boldsymbol{\\phi}(\\mathbf{x}; \\boldsymbol{\\theta}),\n", "$$\n", "that we've denoted here as $\\boldsymbol{\\theta}$. If these parameters appear inside the basis function then our model is *non-linear* in these parameters." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question\n", "For the following prediction functions see whether the model is linear in the inputs, the parameters or both.\"\"\"\n", "\n", "(a) $f(x) = w_1x_1 + w_2$\n", "\n", "(b) $f(x) = w_1\\exp(x_1) + w_2x_2 + w_3$\n", "\n", "(c) $f(x) = \\log(x_1^{w_1}) + w_2x_2^2 + w_3$\n", "\n", "(d) $f(x) = \\exp(-\\sum_i(x_i - w_i)^2)$\n", "\n", "(e) $f(x) = \\exp(-\\mathbf{w}^\\top \\mathbf{x})$\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fitting the Model Yourself\n", "\n", "You now have everything you need to fit a non-linear (in the inputs) basis function model to the marathon data. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task\n", "\n", "Choose one of the basis functions you have explored above. Compute the design matrix on the covariates (or input data), `x`. Use the design matrix and the response variable `y` to solve the following linear system for the model parameters `w`.\n", "$$\n", "\\boldsymbol{\\Phi}^\\top\\boldsymbol{\\Phi}\\mathbf{w} = \\boldsymbol{\\Phi}^\\top \\mathbf{y}\n", "$$\n", "Compute the corresponding error on the training data. How does it compare to the error you were able to achieve fitting the basis above? Plot the form of your prediction function from the least squares estimate alongside the form of you prediction function you fitted by hand. You can make use of [the notebook on regression](./regression.ipynb) for assistance in fitting the model.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Task Code\n", "# " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Lecture on Basis Functions from GPRS Uganda" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from IPython.display import YouTubeVideo\n", "YouTubeVideo('PoNbOnUnOao')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use of QR Decomposition for Numerical Stability\n", "\n", "In the last session we showed how rather than computing $\\mathbf{X}^\\top\\mathbf{X}$ as an intermediate step to our solution, we could compute the solution to the regressiond directly through [QR-decomposition](http://en.wikipedia.org/wiki/QR_decomposition). With polynomials doing the example with QR decomposition can be critical in obtaining the right answer. Can you rewrite the linear regression using QR decomposition?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 0 }