{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# EART 70013 \n", " \n", "# Geophysical Inversion \n", " \n", "## Lecture 1 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Today's learning objectives \n", " \n", " \n", "1. To introduce some of the key concepts underlying inversion (and the closely related topic of optimisation)\n", "\n", " \n", "2. To introduce and review some key linear algebra results\n", " \n", " \n", "3. To practice some Python coding\n" ] }, { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import scipy.linalg as sl\n", "import scipy.optimize as sop\n", "from pprint import pprint" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction - Inversion & Optimisation\n", "\n", "Many numerical problems in science, engineering, economics and other quantitative subjects can be posed as inversion or optimisation problems. \n", "\n", "We will start by illustrating these problems in a single real variable where the concepts will already be somewhat familiar. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Inversion\n", "\n", "Inversion can be thought of as seeking to use a given relationship such as the following \n", "\n", "$$y = f(x)$$\n", "\n", "to find $x$, given:\n", "\n", "1. a value for $y$, and \n", "\n", "\n", "\n", "2. the form of the relationship embodied here in the function $f$ or a means to evaluate $f$. \n", "\n", "
\n", "\n", "At this point we can think of $f$ in an abstract way as some mapping - it could be a mathematical operator, it could be the output of a complex numerical model (which could be very expensive to execute for each $x$), it could be a [neural network](https://en.wikipedia.org/wiki/Neural_network), ..." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can learn a lot about solution methodologies, pitfalls to be aware of etc, by considering $f$ to be a linear mapping, in which case we can think of $f(x)$ as the matrix-vector multiplication $A\\boldsymbol{x}$. Note that in this case I am thinking of $\\boldsymbol{x}$ as well as $\\boldsymbol{y}$ to be vectors, and will use bold font (or underlined) symbols to represent these vectors.\n", "\n", "This linear case is what my lectures will focus on.\n", "\n", "
\n", "\n", "Rhodri's lectures will focus more on practical examples where $f$ represents a complex numerical model." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Forward vs inverse problems\n", "\n", "
\n", "\n", "The **forward problem** can be thought of as: given $x$ what is $y$?\n", "\n", "\n", "The **inverse problem** is the opposite: given $y$ what is $x$?\n", "\n", "
\n", "\n", "[When posing an inverse problem we will often swap the sides of the equation, i.e. $f(x)=y$, so that *known* information (e.g. given data) is on the right and all of the *unknowns* are on the left - cf. the linear case we will see later $A\\boldsymbol{x}=\\boldsymbol{b}$].\n", "\n", "
\n", "\n", "The first is in principle an easy task to achieve as we have access to $f$ (although $f$ may be very expensive and time consuming to evaluate), the second is hard as we do not have ready access to its inverse.\n", "\n", "This inversion problem has no general solution, and solving the inverse problem may be trivial, easy, difficult or impossible depending upon the form of $f$ and the value of $y$. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A simple scalar linear example\n", "\n", "For example, the solution to a linear version of this problem, that is for the problem\n", "\n", "$$y = ax$$\n", "\n", "given an $x$ what is $y$, is \n", "\n", "$$ x = \\frac{1}{a}y$$\n", "\n", "where the operation of multiplying by $1/a$ is clearly the inverse of multiplying by $a$.\n", "\n", "
\n", "\n", "Here we can call \"*multiplying by a*\" our **forward model** and \"*multiplying by* $1/a$\" is then the corresponding **inverse model**. \n", "\n", "
\n", "\n", "This solution to the iversion problem is of course trivially simple at first glance. \n", "\n", "However, note that some more thinking reveals some subtle features that we will see are important in the more general, complex case.\n", "\n", "In particular, our naive solution will not work, or is not defined, if $a = 0$; in that case, there is no solution, unless $y=0$ in which case there are infinitely many $x$ solutions.\n", "\n", "
\n", "\n", "We could chose to write the inverse equation as\n", "\n", "$$ x = (a)^{-1} y$$\n", "\n", "which makes the inverse relationship explicit, that is the use of the inverse operator. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A simple scalar nonlinear example\n", "\n", "A nonlinear example is\n", "\n", "$$ y = \\cos(x) $$\n", "\n", "which of course has the solution \n", "\n", "$$x = \\cos^{-1}(y) $$\n", "\n", "but suppose we didn't know or have a way to compute $\\cos^{-1}\\equiv \\arccos$?\n", "\n", "
\n", "\n", "For this case, assuming that $x$ and $y$ are real numbers, notice that there is no solution if $|y|>1$, and there are infinitely many solutions otherwise. \n", "\n", "Note that infinitely many solutions does not imply that any real number will be a valid solution. \n", "\n", "Despite the lack of a unique solution, the solutions that we obtain are still meaningful and are likely to be useful in practical problems. \n", "\n", "In many real problems, we are likely to have additional *a priori* information that will allow us to select one or more solutions from the infinite collection. \n", "\n", "For example, we may know that possible solutions are constrained such that $0\\leq x\\leq \\pi$ , which together with our forward model ($y=\\cos(x)$), is sufficient to provide a unique solution (provided $|y|\\le 1$). \n", "\n", "\n", "
\n", "\n", "\n", "In general, the solution to our original equation ($y = f(x)$) can be written as\n", "\n", "$$ x = f^{-1}(y)$$\n", "\n", "where\n", "\n", "$$f\\left[f^{-1}(y)\\right] = y$$\n", "\n", "for all values of $y$ for which the inverse is defined.\n", "\n", "There is no general solution to this problem. It is easy to solve (in the scalar case) if $f$ is a linear function, and a much wider range of problems will have solutions we can find robustly (but maybe not cheaply) as long as we are able to begin from a reasonably good guess. \n", "\n", "
\n", "\n", "Newton's method we saw in the Numerical Methods module iteratively solves the problem $\\hat{f}(x)=0$ (I'm introducing a hat here to emphasise that this is a different function to the $f$ above, unless $y=0$ of course) from a starting guess, and can be used for multi-dimensional (i.e. vector valued) problems as well as scalar problems." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A reminder on Newton's method\n", "\n", "#### For finding roots of equations\n", "\n", "Recall that given a problem written in the form\n", "\n", "\\begin{equation}\n", "\\hat{f}(x)=0\n", "\\end{equation}\n", "\n", "The hat here is introduced as we're using $\\hat{f}$ in our definition of a problem written as a root-finding task.\n", "\n", "[What's the link between this $\\hat{f}$ and the $f$ we introduced to define our forward problem at the start?]\n", "\n", "
\n", "\n", "Newton's method is a powerful root-finding algorithm (or \"nonlinear solver\") which finds a solution $x$ (termed a root as the RHS of this equation is zero) via the iteration\n", "\n", "\\begin{equation}\n", "x_{i+1} = x_i - \\frac{\\hat{f}(x_i)}{\\hat{f}'(x_i)}\n", "\\end{equation}\n", "\n", "\n", "We wrote our own code to solve this in your Numerical Methods module from year 2, but let's remind ourselves how to use the in built SciPy function:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.567143290409784\n", "0.5671432904097835\n" ] } ], "source": [ "# An example f hat function we want to find a/the roots of\n", "def fh(x):\n", " return x - np.exp(-x)\n", "\n", "# its derivative\n", "def dfhdx(x):\n", " return 1 + np.exp(-x)\n", "\n", "x0 = -1. # initial guess\n", "print(sop.newton(fh, x0, dfhdx))\n", "\n", "\n", "# and if you don't have the derivative function it will use a quasi-Newton approach:\n", "print(sop.newton(fh, x0))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### For inversion\n", "\n", "How do we turn our inversion problem into something Newton's method is appropriate to solve?\n", "\n", "Suppose we want to solve the inversion problem from above where our forward model is described by the equation\n", "\n", "$$ y = \\cos(x) $$\n", "\n", "and we have a given output of the model $y=y_d$ (the subscript $d$ is added to emphasise given data).\n", "\n", "Given these two pieces of information, what is $x$?\n", "\n", "Let's first plot our situation for a particular value of $y_d$." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiYAAAGKCAYAAAA4+IpOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABdb0lEQVR4nO3deXxU5bkH8N+ThbCFLYQk7CiLoAhiRAHbYhVFa6Xu0NpqvbfYulSt3o+02tattZvVeutS9eJatRYX1IJ71bogAoKobGEPWQh7gEC29/7xzJHJZJJMkplz3nPO7/v55DNkzpk57zwZzvucdztijAERERGRDdK8LgARERGRg4kJERERWYOJCREREVmDiQkRERFZg4kJERERWYOJCREREVmDiUmKiMglImJEZJeI9IzZlhHZdrMLxx8c9dwGEXk0VcdMFRG5WUR8Pa893t8jzFL9/Q8Dv/5/JmoJE5PU6w7gBq8LEXE2gNu8LkQbPAxggteFaKd/QT9DqdcFISKyGROT1HsdwFUiku91QYwxnxpj1npdjnhEJKupbcaYYmPMAjfLk6jmyh3NGFNhjFlgjDmY6jKFTaJ/AyLyByYmqXd75PHGlnYUkfEi8qaI7BWRfSLyloiMj9nnUREpFpFjROQ/IrJfRNaIyI8TeP8GTb9R3QsniMjfRWSPiJSIyD0i0jHmtZ1F5Pcisl5EqiOPN4pIWtQ+HUXkLhH5PPIZykTkZRE5Iua9nON+XUT+KSK7AHzcTLkbdeVEXn+7iPw0UpZKEXlXRI6M2uc+ESkXkYyY12aJyE4RuTvqud4icr+IbBGRgyKyUkRmJlpuETlORN4Qke2Rv8k6EbkvzmsHRz2XGfkMGyIx3RD5PTNqn8GR110mIreKSGmke/BlEenfVMzaQkS+KyKfRv52u0VkuYhcFrPPNyKfc3fkO7pMRP4ravt0EXlbRCoi7/OpiFycwLGHisgTkb9lVSR+90vjblDn+z9BRD4UkSoAf2jhvc8RkQWRv8uuyN9uYNT22yPxPy7quS4iskpEPnK+P5G/8ZzI8asi238rIp1ijveOiLwvIlNFZGlk309F5HjRbtzfRv6OOyKfp0vUa52/9+Ui8mcR2Rop9yuSQDegiAwR/b9cEfkeLxWRs1t6HZFNmJikXimAvwKYKSKDmtpJRI4G8C6AngAuAfADAN0AvCsiY2J27wbgKQBPApgG4BMA94vISW0s4xMA1gI4B8D9AK4A8POosmUAeA3AfwP4C4DTod0rvwTwx6j3yQKQDU3GvgXgJwA6Algg8VuM/g5gPYDzAMxqQ7kvihznagA/BDAQwNyoRORxAH0AnBrzujMB9IB+bohINwAfRN7r5sjjy9CYXtVSuUWkKzQ+ddC/3RkAbgWQEee10R6Dfu7HI2V6BNrt91icfX8OYCiASyOfd0KkHEkhIidCv0/vAvgOgPMBPASNk7PPNABvAegA4DLod282gOjv9WEA5gD4XuR9XgbwsLScOPcFUAzgGgCnQeN3MoB5cfbtDuAZAE9Dv4tPNfO5fgzgOQBfQv9elwE4Cvr/Kjuy280AFgF4KvK3BIB7AeQD+K4xpjby3EAASwH8GMBU6P+FS6F/t1hDof83fgeNZRaAl6D/vwqg35NboXH6dZzX/xzAMOj3+goAxwJ4PTppjfNZB0AT5TEArgVwFoAlAJ4TkbOaeh2RdYwx/EnBD/TEY6AnqF4AdgGYHdmWEdl2c9T+cyL79Ih6rhuAHQCej3ru0chrT4p6LgvANgAPxjn+4KjnNgB4NM4+t8SU/RUAq6N+/35kv6/H7HcjgGoAfZqIQTqAzgAqAVwb57h3JRjLm/Wr2uA5A2ANgMyo586LPD8x6rnVAJ6Oee2LAL6M+v2XAA4AGBaz30ORuGY0V24AhZHnj07g+zA48vtRsd+ByPM3Rb8XgMGR39+N2e/6yPN9k/R9vR7Ajma2S+T7swhAWoLvmRb5rj8EYFmcv9/Nzbw2A8CJkf2OifP9n5bA8bsC2I3I/7uo5wdHvrfXxDy3C5oUzogc47stxCMDmhzXA8iJ2vYOgBoAh0U9d1bkPd+MeZ/nAayPKYeBJlJpUc9Pijz/X1HPbUDD/8//B6AiuiyR598AsDQZ3xP+8MeNH7aYuMAYswPAnQB+ICIjmtjt6wBeMcbsinrdHuhV1jdi9t1vjPl31H4HoZX0QLTNv2J+Xx7zXlMBbATwYaQpOiPSKvE6gEwAJzg7isgFIvKxaDdHLYB90Aoi3ud+oY3ldbxhjKmJKTdiyv4kgGnO1bGI9IJeZT8etc9U6JXm+pjP9xqAHACjWij3Gmil9jcRuShy5dqSr0eVL5rze+zfPN7fCGjmby4iadGfR0TSmynPJwB6isiTInKmiPSI2T4C2jLysDGmvpljDhORp0VkC7RyroG2tDX1vXde10FEfiHahVYVed1/oo4drRaaPLdkAjS5/3vM37UYwEoc+hvAGLMB2hLyA2gLyOPGmAYtMSLSTbQ7cy2Ag5EyPgFNUobFHHu1MWZd1O8rI4+vxey3EkB/EZGY5+dEx9kY80Gk3M0NAp8KbWHaHed7PCbSMkhkPSYm7rkL2vpxaxPbeyH+jI0yaPdOtJ1x9jsI7TZpix1x3it6QGEfaKVUE/OzMLI9BwBE5NsA/gFgBYDvAjgewHHQq7h4ZWvvDJV45UbMsZ6I/H5e5Pfp0GQquhukD7SSiv18/4xsz4k5ToNyG2N2AzgJQAmA+wBsEh1nc24zZe8V772gf+/o7Y5EPmusX6Hh53mrqR2NMe9CuxwGQBOvCtHxTkdHdnFiUNzUe0S6Qd6AdiXMAvA16N9/Nhp+n+K5A9oy9iS0K208tGsRaPwZtxpj6lp4P0D/rgDwJhr/bUej8d/1XwC2R8p6V5z3ewSavNwDYAr0s13RRBlj/49WN/N8BrR1MVp5nOOXA+gX53lHH2hiFftZne7W2M9LZKWW+sApSYwxe0XkDmjLyR/j7LID2qcdKx+NKyW3bYeOqbigie0bIo/TARQZYy5xNkT6xGMrWUfK1yYxxqwXkQ+gTe6PRB7fMcZsjtptO4Ct0LEb8ayKfds4x1kK4NzIFWohdIzAsyIyxhjzeZz3dP6m+dDxPYj63SlTez2Ihi0Llc3tbIyZA2BOJMGYDOD3AF4VHWS7LbJbcxXjBGgC+zVjzPvOkxIz+LgJ06GtFM5gcSfRiVvUBN4POBTDSwB8EWd7bDzuhSYIawE8KCKTnBY50cHg06DdT3+JKuPoBMvSWnlNPLe0mddsh7Yy/b6J7SXtLBORK5iYuOs+AD/DoZk60d4F8C0RyTbGVAJApPvh29A+ay+9CuBcAHuNMSub2a8ztJk92vfR+GrQbU9AB7JOhlaeP4zZ/iqAqwBsMsZsbc+BjA6UXCAiv4SOKxgJIF5i8m7kcTqA30Q9/73I43vtKUekLCVoQ2VkjNkL4BUROQw6wDMHOlZnA4D/FpEHjTHxkoPOkcevutdEZ9VMS+CwnaNfFxH7d2qtD6HJx1BjTLwBxV8Rke9Cv6sXAFgH4CNo66YzCDwL+j2OLeMl7SxjU84TkZud7hwRmQSgf6RcTXkV+v3+whhTlaJyEaUcExMXGWMOisit0CvZWLdBZ2a8JSK/h14V3gA9YTfV/eOWv0MribdE5E4Ay6AzMw6HVr7fMcbsh54YvyMid0Gv1I8F8FPo+AsvPQttfn8SQBV0lka0uwBcCOA/kbKvAtAFwBHQq/9mK1YRORPATOig2vWR1/4UWinGrUiMMV+IyNMAbo60KHwIrVR+CR2s+1nrP2bbRb6XeQD+DU1m+kM/w1JjTEVkn2uggzXfFpEHoF10I6GDn38d+Qx7ANwrIr+GxuEmaGtL9xaK8CqAi0VkOYAiaDfOxPZ8JmPMHhH5n0h5cgHMhw6G7Qcdw/OOMeYpERkCnS3zf8aYf0Y+640Aficirxtj/m2M2S0iCwBcJyKlkc90KZpvQWqPbAAvisjfAORCu7rWoOHYqFi/gnavvicif4Umkj2hA60PM8ZcmqKyEiUVExP3PQLgfxAzWM4Y81nkiv430JkBAmABgG8YY5a5XMYGjDE1InIadNzATABDoINa10L75Z3+84egYxQuhU7L/ATa4tPeQa7tYozZJSIvQ8eZPO20SEVt3y0iE6En9huglc0uaIISm8TEswaa8PwSOhW0EvrZpxhjmhyTAeBi6NX5pdAKvATaDH9Lwh8ueT6GJiJ3QbvetkIHN//S2cEYM1dEpkSe+7/I02sB3B3ZXhFZM+NO6CyzEmiLSy/EnxIb7Srod95pPZoHnR2zsMlXJMAY8zcR2Qz9P/dd6PiiLdAWqaWRpPAp6Nie6K68P0HHkTwR6Y7bHinP/dAunypowns1EhuI21p3QGf0PQpN8P4N4MqYwd4NGGM2iUghdKzOb6EJzXZoi12zLUZENpH4LbJEROQ20UXU1gP4kTHmYY+LQ+QJzsohIiIiazAxISIiImuwK4eIiIiswRYTIiIisgYTEyIiIrIGpwsD6N27txk8eLDXxSAi8pXFixdvM8bkel0OChYmJgAGDx6MRYsWeV0MIiJfEZGNXpeBgoddOURERGQNJiZERERkDSYmREREZA0mJkRERGQNJiZERERkDc7KISIi6yxevLhDWlraT9LT039ojOkOvfs0+ZsRkd11dXWP1NfX33/sscdWx9uJiQkREVknIyPjoW7duk3q27fvvg4dOmwXYV7id8YYVFdXZ5aUlFy1Z8+ecQAujrcfu3KIiMhGJw4aNGh3VlZWDZOSYBARZGVl1QwaNGg3gBOb2o+JCRER2Sg9LS2Nd5kNoMjfNb3J7S6WJWEiMltEtorI501sFxG5R0SKROQzERkXtW2qiKyKbJvlXqmJiIiovaxMTAA8CmBqM9tPBzAs8jMTwP0AICLpAO6NbB8FYIaIjEppSYmIiChprExMjDHvAdjRzC7TADxu1AIAPUSkAMB4AEXGmHXGmGoAz0T2pRSpqADeew/Yu9frkgRPaanGtqrK65IEz+bNwPvvA9Vx5wQQkZesTEwS0A/A5qjfiyPPNfV8IyIyU0QWiciiioqKlBU0yObOBYYMAb7xDWDoUID3QUyep58GBg/W2B5xBLBihdclCo4HHtDv7de+BowZA2zkbeiIrOLXxCTeEG3TzPONnzTmQWNMoTGmMDeXd+1uraIiYMYM4MgjgX/+E+jUCTjrLGDPHq9L5n+LFwM/+AFw/PHAM88ABw8C06YBBw54XTL/e/tt4Cc/AU47DXjiCW2VOuccoK7O65IR2aeioiI9JydnzBdffJGV6GumTp162M0335zXnuP6NTEpBjAg6vf+AEqaeZ6S7OqrgcxM4PnngfPO0+SkrAy45RavS+ZvxgBXXQXk5AAvvQRceKFWoGvWAHfd5XXp/K22FrjiCuDww4E5c4CLLgL+9jdgyRLg//7P69IR2eemm24qOOmkk3YfeeSRBxN9za233lp61113FWzfvr3JWTct8Wti8hKAH0Rm55wAYLcxphTAJwCGicgQEekAYHpkX0qiZcuAefOAG24A+kU6ygoL9Sr/gQeAnTu9LZ+f/fvfwEcfAbfeCvTooc9NmQJ861vAnXdyvEl7vPgisHIl8PvfawsfAFxwATBhAnDHHWw1IYpWWVmZ9vTTT/f+0Y9+tK01rxs/fnzVgAEDDj744IO92npsKxMTEXkawEcARohIsYj8l4j8WER+HNllHoB1AIoAPATgcgAwxtQCuBLAawBWAHjWGPOF6x8g4O69F+jSRZvEo/3sZ8D+/bz6bI+//lVbS77//YbP/+xnwPbt2rVDbfO//6tjS77znUPPiWhsN2wA/vUvr0pGQZOXl3d0bHfGwoULO2VlZY1bvHhxx2Qco76+Hr/+9a/zBg0adFSHDh3G5eXlHX3FFVd8NaayqqpKLr300gE5OTljsrKyxo0ZM+aI1157rWv0e8yfP7/rmDFjjujcufMx2dnZY48++ugjPvnkk44AMGfOnO5paWmYMmXKV1MbZs+e3bNDhw7jVq9e3cF57oc//OGAAQMGHLV58+avVpKfOnXqrjlz5rQ5MbFySXpjzIwWthsAVzSxbR40caEUqK7WZvBzzgF69my47eij9erziSeA66/3pnx+tnMn8MorwE9/euiK3nHSScDw4RrbH/7Qm/L52aZNOsPpN78B0mMamKdNA/Lzgccf13FSZK9LL8WAzz9HZzePedRR2D97doNJFS0aN27c3sWLFzco59VXXz3gwgsv3Hbsscc2GC02a9as/Hvuuaegufd7/vnn10ydOrXB3Merrrqq3+OPP5572223bT7llFP2lpWVZSxatOirY15++eX9X3nllZ733nvvhuHDhx/8wx/+kHf22WcPW7FixeeDBg2qqampwYUXXjh0+vTp25566qn11dXV8vHHH3dOj/wHee+997oeeeSR+9LSDrVfXHLJJTv//Oc/5//qV78qeOaZZzb+6le/yps7d26vd999d+WAAQNqnf1OOOGEfX/5y18K9u7dK127dm31InlWJiZkr9df1wp0+vT426dP1/EnK1fqbBJK3MsvAzU1wPnnN94morG9/XYdy5Of7375/Oy55/QxXmwzM4Fzz9WWvr17ga5dG+9D1BrHH3/8vtmzZ381q+KJJ57o8eWXX3Z+4YUX1sbue+2111ZcdNFFzXaADx48uMHE9t27d6c9/PDDebfddtvma665ZjsAHHXUUQdPOeWUfQCwZ8+etCeffDL3rrvu2jh9+vTdAPDkk09uHDJkSPadd96Ze88995Ts2LEjvbKyMn3atGm7nDEkxxxzzFdJ0+bNmzvk5eXVRB83LS0Nt95665YLLrhg6C9+8YuDf/nLXwrmzZu3avTo0Q3GoAwYMKCmtrZWNm7c2KE141McTEyoVZ59FujVCzjllPjbzztPE5Pnnwd+8Qt3y+Z3zz0HDBgAjB8ff/sFF+jYk7lzgcsuc7dsfjdnjk4NHjYs/vYLLtAuyvnz4ycvZIfWtlx4ZdKkSXtvueWW/uXl5endunWrv/HGG/tfd911Jfn5+Y1GMuXl5dXl5eW1aoTTkiVLOlZXV8vpp58edx7kihUrsmpra+Wkk076qpUlIyMD48aN27dy5cpOznHPPffc7eecc87wCRMm7Jk8eXLlRRddtGPo0KE1AHDgwIG0Pn361Ma+9znnnLNn9OjR+//whz/0e+qpp9Z84xvf2B+7T+fOnesBYP/+/W26yZGVY0zITsZoi8nUqUCHDvH36dsXOOYY3Y8SV1UFvPYacPbZ2joSz6hRwMCBjG1rVVQAH36o3Y9NmTgR6N5d/wZE7XXiiSfuz8zMNB988EGX22+/PS89Pd3ccMMNcRfMmjVrVn7nzp2Pae7n1VdfbdCOZ4xptsKvr68HAIhIo26U6BsizpkzZ8M777yzYtKkSXvnzZvX46ijjhr93HPPdQOAXr161e7atavRzJqXXnope+XKlZ2MMejbt2+jxAUAKioqMgAgPz8/7vaWsMWEErZ8OVBerrNEmnPqqcCf/wxUVgLZ2e6Uze8+/FDXKznttKb3EdHY/vOfOvU1g/97E/LOO/p46qlN75ORAXzzm8Abb2gCzpvZUnt06tTJjBw5cv/cuXO7P/fcc70feuihdVlZWXHHWrSlK+eYY46p6tChg5k/f3630aNHN0p4jjzyyIOZmZnm7bffzh41atR2AKitrcWSJUu6nHPOOQ1WVZ8wYULVhAkTqn7zm9+Uff3rXx/26KOP5px77rl7xo4du//pp5/uHb3vRx991Ol73/ve4Xfcccem+fPn95g1a1a/999/f03s8ZctW9YpNze3JnrcSWvw1EYJc67UE0lMfv97rRC+/e2UFysQ3n5bB2V+7WvN73fqqcDDDwOffKIDjallb72lCXJhYfP7TZkCvPCCrhkzfLg7ZaPgKiws3PfII4/0mThx4p4ZM2bsbmq/tnTl9OzZs/7SSy8tv/322/tlZWXVn3LKKXu3bt2avnDhwi433HBDRbdu3eovuuiiiltvvbVfbm5u7bBhww7+8Y9/zNu+fXvmz372swoAWLlyZYd77rkn9+yzz941aNCgmlWrVmWtXLmy0yWXXFIBAGeeeeae3/zmN/3LysrS8/Pz61avXt1h2rRpwy677LLya665ZvukSZP2HX/88Ue+8sor2WeeeWZldPnef//9rpMnT27yM7eEiQkl7M03dUBrv7iL/B8yaZLOKnnrLSYmiXr7bR1b0lIL08kn69X8m28yMUnU22/r0v4ttTA5CfebbzIxofYbO3bs/rS0NNx9990pGRfz17/+dUvPnj3r/vjHP/a9/vrrM3NycmrPP//87c72e++9txgALr/88sGVlZXpI0eO3P/CCy+sGTRoUA0AdOnSpb6oqKjj9773vcN37dqVkZOTU3POOefsuO2228oAXY9k9OjR+x555JFel1566Y6pU6cOO+WUU3b/6U9/KgWA44477sDpp5++86abbup35plnrnSOu3//fnn99dd7vPjii41aUhIlOvM23AoLC80i3uilWfX1Oj14xgxdRK0lkyfrmiYLF6a8aL5XWamxveEGnc7aktGjgf79daAmNW/LFo3Vn/8MXHtt8/sao/tOngz8/e+uFM/3RGSxMaaFtqi2WbZs2YYxY8a0anEvm0yaNGnYYYcddvCJJ57Y5HVZ2mrOnDndrr/++oFFRUWfZyTYd3zHHXfkvvLKKz0++OCDZhOTZcuW9R4zZszgeNvYYkIJWbVK74Nz/PGJ7T9hAvCnP+mgztg1OaihxYt11dETT0xs/4kTgX/8Q5PFNA5fb9bHH+vjxIkt7yui39uPPkptmSi46urqUFJSkvHAAw/0Xr16dacXX3xxnddlao/zzjtvz8qVK7euW7euw/DhwxO6F3dmZqa577772pWM8bRGCVmwQB9POCGx/SdO1AGabIhq2Sef6ONxxyW2/4QJwO7dulYMNW/hQl2nZMyYxPafMAFYv14HeRO11vz587MHDRo05umnn8556qmn1ubm5vr+Rgc33XTT1kSTEgC4/vrrt40ZM6bVa5dEY2JCCVmwQKdTjhiR2P7O+IcPP0xdmYJi4ULgsMOA3r1b3hc4FFte2bds4UJNSjomuAg4Y0vtceaZZ1bW19cvXrdu3RdTpkzZ53V5/IqJCSXk44+1GyfRroPevYGhQw81pVPTFi5MvLUE0IGZvXodasWi+OrqtMUu0e5HABg3TltYGFsi7zAxoRbt3atrmLTmBA/oSX7p0pQUKTDKyvQ+Lk2t9hqPCGObiFWrdGBxa2LbsSNw1FHAp5+mrlxE1DwmJtSiZct0oGVrruoBYOxY7a/ftSsVpQoGZ3xJaypPQGO7fLneW4fic2aEtSW2n36qs3SIyH1MTKhFn32mj2PHtu51zv7O66mxxYu1BeSYY1r3urFjdaXYVatSUqxAWLJEb8jX2jVJxo7VZezLylJSLCJqARMTatGyZUCPHrrGQ2s4lS27HJq2fLmOxenSpXWvc5I+xrZpn32m3TKtnVLN2BJ5i4kJteizz4Cjj279/UPy84G8PPbXN2f5cl0wrbVGjACyslh5NsWYtsf26KP1kbEl8gYTE2pWfb2e4J2TdWuNHcsTfFP27weKitpWeWZk6OsY2/hKS4EdO9oW2x49gMGDtaWQiNzHxISatWGDzspJdIGqWGPHAl98wUGa8Xz5pV7Zt6XyBDhIsznLl+tje2LLpI/IG0xMqFnOVWNbW0yOOkqTkqKi5JUpKNpbeR51lLYKcJXSxtob26OPBlav1lsqEJG7mJhQsz77TMeWHHlk214/cqQ+rliRvDIFxfLleh+hww9v2+tHjdJHxrax5cuBggIgJ6dtrx85Ului1rT5/qhE1FZMTKhZK1YAQ4a0ftaI44gj9PHLL5NXpqBYvlyTi/T0tr2eSV/T2jrw1cHYEnmHiQk1a9WqxO+PE0+XLsCgQTzBx/PFF21viQKAfv2A7GzGNlZ9vd7gsD2xHT5cWwoZW2qLvLy8o2+++ea86OcWLlzYKSsra9zixYsTvHOT98fwSobXBSB71ddrP/vkye17n5Ej2WISq7JSZ444LUptIaKvZ+XZUHGxjg1pT0LdqZO2FDK29hk/fnyjv+w555yzY9asWRWVlZVpJ5988rDY7RdddNG2n/70p9tLS0szpk2b1qjz9Ec/+tHWH/3oRzuLiooyv/vd7x4WvW3hwoWtXsZw3LhxexcvXtw5+rmrr756wIUXXrjt2GOPPRD9/KxZs/Lvueeegube7/nnn18zderUvW09ht8wMaEmFRfrlNb2nOABTUzefVcTndYudhVUq1frY2tXJY01ciTw5pvtL0+QOKvhJiO2TEyoLY4//vh9s2fPznV+f+KJJ3p8+eWXnV944YW1sftee+21FRdddNHO5t5v8ODB1e05huPEE08cdtddd20+7rjjrE5cmJhQk5wTfHuu6gEdR1FVBWzcqFehlNzE5PHHgd27ge7d21+uIHBim4yE+s039S7FbR0HRMnXXAtGdnZ2fXPbCwoKapvbPnTo0Jq2tJDEmjRp0t5bbrmlf3l5eXq3bt3qb7zxxv7XXXddSX5+fl3svnl5eXV5eXmNnk/mMRzr1q3rOHbsWKuTEoCJCTXDSUyScYIHtDuHiYlatUq7YoYObd/7OLFdubL1d38OqlWr9B45Bc02jrds5Ei9H9H69e3/O1G4nHjiifszMzPNBx980GXx4sWd09PTzQ033FARb9+2duUkcozFixd3nDlz5qC9e/emf/e7393WvXv3uszMzPZ/wBRjYkJNWrlSB1fm57fvfZwWl5UrgW99q/3lCoLVq4GBA3UsQ3tEzx5hYqJWrTo0eLU9omPLxIRao1OnTmbkyJH7586d2/25557r/dBDD63LysqKuxRiW7tyWjpGTU0NLr300sEPP/zwhuOOO+7AGWeccdioUaP2t//TpR4TE2rSqlWaVLT3BJ+TA/TsCaxtsuczfNo728kxZIh2M3ABu0NWrwYmTGj/+zjdbIwttUVhYeG+Rx55pM/EiRP3zJgxY3dT+7W1K6elYzz22GM9x4wZs98ZTzJs2LADPXv2bNNx3GblUEQRmSoiq0SkSERmxdn+PyKyNPLzuYjUiUivyLYNIrI8sm2R+6UPjmRVnoBecfIEr4zRyrO940sAIDNTp2Mz6VPOWKZkfG979dL75vB7S20xduzY/Wlpabj77rs3e3GMzz77rNOYMWO+aiFZunRp52OOOcYXLSbWJSYikg7gXgCnAxgFYIaIjIrexxjzR2PMWGPMWAA/B/CuMWZH1C4nRbYXulXuoNm3D9i8mYlJKpSV6f2HGNvkKyrSxC8ZSZ8zBoixpbZ45plnes2YMaOisLAwZYNNmztGTk5O7eeff94JAObOnZv9n//8p/txxx3ni5ss2NiVMx5AkTFmHQCIyDMApgFoaiWMGQCedqlsoeEsxZ3MyvMf/wCqq4EOHZLznn6VrOmsjqFDgb//XSvk9na7+V2yZuQ4Dj8c+OST5LwXBV9dXR1KSkoyHnjggd6rV6/u9OKLL67z6hgzZ87cMWXKlGEjR44cNWLEiKq8vLzqvn371ia7PKlgY2LSD0B0s1QxgLjD+kSkM4CpAK6MetoAeF1EDIC/GWMeTFVBg8zpGkjWoL+hQ3Udkw0bklch+1WyK8+hQ3W68I4dbb83TFA4Sd+wRktstc3QocCcOXojSh9MZiCPzZ8/P/uss84aPnjw4ANPPfXU2tzc3KSP6Uj0GAUFBbWff/65L1fisTExiXfN19SN3b8N4IOYbpxJxpgSEekD4A0RWWmMea/RQURmApgJAAMHDmxvmQNnXSQHP+yw5vdLlJPgFBUxMVmzBsjKAgYMSM77Rcc27IlJUZHOIsvOTs77DR2q65hs3MiZOdSyM888s7K+vn6x34/hNevGmEBbSKJP2f0BlDSx73TEdOMYY0oij1sBvADtGmrEGPOgMabQGFOYm5sbb5dQW7dOK7lkLdoVXXmG3bp1wODByVsF17k7MWOra44kK5kGGFsiL9iYmHwCYJiIDBGRDtDk46XYnUSkO4BvAJgb9VwXEcl2/g3gVACfu1LqgFm3Lrkn+NxcvYrlCV4rz2QuNHfYYTq2hDNzkh9bJ6FmbIncY11XjjGmVkSuBPAagHQAs40xX4jIjyPbH4jsejaA140x+6JengfgBdERgBkAnjLGvOpe6YNj3Trg2GOT934ievXJxEQrz2Sss+Ho2BHo35+xranRmWTJTEzy84HOnRlbIjdZl5gAgDFmHoB5Mc89EPP7owAejXluHYAxKS5e4NXW6iDV889P7vsOHQp89lly39Nvdu4Edu1K/tL8nNYKbNqkA6yT2dLHhNpT9fX19ZKWltbUGEPyqfr6egFQ39R2G7tyyGPFxZqcJPMED2jluX69vndYrV+vj0xMki+VsWVXjvtEpKyqqqqj1+Wg5KuqquooImVNbWdiQo0ke0aO4/DDtbm9uDi57+snTuWZithWVAB79iT3ff0kVd9bJzGpb/L6jlKhtrb2lg0bNnTYt29fp8gVNvlcfX297Nu3r9OGDRs61NbW3tLUflZ25ZC3UnWCHzxYHzduPPTvsEnVVX10bEePTu57+8X69brWSL9+yX3fIUN0YcCyMqBv3+S+NzVt3Lhxry1ZsuTKtWvX/toYkw9eSAdBvYiU1dbW3jJu3LjXmtqJiQk1sm4dkJGRvHU2HIMG6eOGDcA3vpHc9/aL9ev1/is9eiT3fZmYaGwHDtSbGiZT9PeWiYm7IpVXkxUYBRMzUGrEWWcj2Sd4Zx27jRuT+75+kuxp2I7oyjOsUhXb6KSPiFKPiQk1kqoTfFYWUFAQ7soz2etsOPLydNpwmCvPVMWWSR+Ru5iYUCPr16duDMjgweGtPJ17BaWi8hTRFqmwVp6VlcC2balJqLt00VWQw/q9JXIbExNqYP9+PcE7V4nJNnhweCvPsjLg4MHUJCZAuJO+VA0qdoQ5tkRuY2JCDWyO3Nc5Vfc1HDRIF8KqS/o9N+2XqtlOjkGDwpv0pToxCXNsidzGxIQa2LRJH1OVmAwerAuslZam5v1t5sZVfUWFtnqFjZP0pbrFxHANUqKUY2JCDTjN1anqygnzQELnM6c6tmHscti06dBYkFQYNAioqtLEj4hSi4kJNbBpE5CWlrr1GsI89XLzZqBPH509kwphju2mTdrKJylaHzTMsSVyGxMTamDTJk1KMjNT8/5hbjHZtCn5i9ZFC3NsN292J7ZMTIhSj4kJNeBceaZKp07aahDGE/zmzamNbUGBJpRhjS2TPqJgYGJCDaQ6MQHCOWXYmNS3mKSn6/uHLbYHD+pU7FR+b3v0ALp3D2fSR+Q2Jib0lfp6vfJM1eBMx6BB4TvB794N7N3rTtIXtthu2aKPqUz6AE4ZJnILExP6Snm53kXVrcozTLeRd9aHSXXlGcbEJNVT3B1hjC2RF5iY0FfcOsEPHKjN79u2pfY4NnFim+rEZOBAoKREE8ywcCvpC2NLH5EXmJjQV9xKTPr318fi4tQexyapXlHX4VTOJSWpPY5N3Er6BgwA9uzRHyJKHSYm9BXnatCtxMSprMNg82YgIwPIz0/tccKa9PXurTO+UimMsSXyAhMT+sqmTUC3bjoDIZWcK9swneA3bQL69dOZM6kUxsrTjZlkQDi/t0ReYGJCX3HrBJ+bq+tthOkEn+p1NhxhbY1yM7Zh+t4SeYGJCX3FrcQkLU1bD8JUeaZ6DRNHt276E6bK063YOrdpCFNsibzAxIS+4lZiAujVZ1hO8PX1+lkZ2+TbvVsHo7oR2w4dgLy88MSWyCtMTAgAsG8fsH27e5XngAHhOcFv3QrU1LhzVQ+EKzFxa6qwI0yxJfIKExMCcOhk6/YJ3hh3juclt6azOvr3D083mVvTsB1hii2RV5iYEIBDy3r36+fO8fr3D88ia15UnmVl2koTdG4nfWFq6SPyChMTAuB+YhKmqZdeVJ7GAKWl7hzPS5s36xTsggJ3jte/P7Brl973iIhSw8rERESmisgqESkSkVlxtk8Wkd0isjTy86tEX0vxedFiAoQjMdm8GejcGejVy53jhSm2mzbpbJmMDHeO58TW+f9CRMnn0n/nxIlIOoB7AUwBUAzgExF5yRjzZcyu/zHGnNnG11KMLVv0tu5durhzvDCtt7F5s35eEXeOF6bYFhcf+rxuiE76Roxw77hEYWJji8l4AEXGmHXGmGoAzwCY5sJrQ23LFvdaSwCgTx+9yg3DVf2WLd5VnkFXUuLu9zZMSR+RV6xrMQHQD0D0f/tiAMfH2W+CiCwDUALgemPMF614bdJMnjy50XMXXHABLr/8cuzfvx9nnHFGo+2XXHIJLrnkEmzbtg3nnXdeo+0/+clPcOGFF2Lz5s34/ve/32j7ddddh29/+9tYtWoVLrvsskbbb7rpJpxyyilYunQprrnmmkbbf/vb32LixIn48MMP8Ytf/AIAsGSJJgqTJwN33303xo4dizfffBO33357o9f/7W9/w4gRI/Dyyy/jzjvvbLT9iSeewIABA/CPf/wD999/f6Ptc+bMQe/evdG9+6OYPftRfPhhw+3z5s1D586dcd999+HZZ59t9Pp33nkHAPCnP/0Jr7zySoNtnTp1wvz58wEAt912G956660G23NycvDcc88BAH7+85/jo48+arC9f//+ePLJJwEA11xzDZYuXdpg+/Dhw/Hggw8CAGbOnInVq1c32D527FjcfffdAICLLroIxcXFWLJEW6MmTwYmTJiAO+64AwBw7rnnYvv27Q1ef/LJJ+OXv/wlAOD0009HVVVVg+1nnnkmrr/+egBNf/d+8pPL0aXLftx11xl4+eWG22387kVr7XdvzRod7+GEItHv3qOPPopHH3200faWvnuvvvoOAOCZZ/6ERx+1/7sXranvnvP/icgWNraYxGvwjp1UugTAIGPMGAD/C+DFVrxWdxSZKSKLRGRRRUVFW8saGAcP6gJSburZEzhwwN1jeqG6GsjKcu94ItqKcPCge8f0Ql2dLl7nZmw7dtRbKuze7d4xicJGjGULSYjIBAA3G2NOi/z+cwAwxtzRzGs2ACgEMKy1rwWAwsJCs2jRoqSU34/q6vTkPmsWEOciNWWmTwcWLwbWrHHvmG6rqNBuq3vuAa66yr3jTpkCVFYCCxa4d0y3rVwJjBwJPPkk8L3vuXfcceN0wG1MY10oichiY0yh1+WgYLGxxeQTAMNEZIiIdAAwHcBL0TuISL6IDiUUkfHQz7E9kddSY+Xlmpy42VcPHFoTwrLcOKmc2RvOfVbcEoYVSt2eSeYIQ2yJvGTdGBNjTK2IXAngNQDpAGYbY74QkR9Htj8A4DwAPxGRWgBVAKYbbfqJ+1pPPoiPeHmCP3BAl8Lv3dvdY7vFy9iWlgK1te5NpXVbSYk+epH0ffCBu8ckChMrT1nGmHkA5sU890DUv/8K4K+Jvpaa52XlCegMh6AmJl5VngMG6PiLsjJ3ZwS5ycvv7Y4dwP79uj4NESWXjV055DKnWdrtE7xzvCCvUOpUnm6tTOpwYhvkaa0lJe6uveMI03RsIi8wMSFs2aLN/X36uHtcpxXBaVUIopISjWtmprvHdWIb9KTP7ZYoIBwJNZGXmJgQtmzRK/o0l78N+fn6GOTExO2F6xxhSUy8iK3T+hXk7y2Rl5iYkGcn+A4ddE2IIJ/g3V6Z1JGbqze3C3psvWgxCUNLH5GXmJiQ60umRysoCP5VvReVZ1qaxjaolWd9vX5vvEj6uncHOnUK9veWyEtMTMizFhNAK+2gVp7V1brAmlexDXJiUlGhU6G9iK1IsL+3RF5jYhJye/bovUaYmCSfc0XtRYuJc9ygXtV7tXCdI8hJH5HXmJiEnFdrQTj69tW1NurqvDl+KjkVF5O+5LPhexvUpI/Ia0xMQs6GE3x9PbB1qzfHTyWvr+r79tVVdYN4Mz+vFq5zBDnpI/IaE5OQ8zoxcaZeBvHq0+vYBnnK8JYtOsDXmXLutr59tQu0stKb4xMFGROTkLOl8gzi1WdJiU6Jzsnx5vhBTvpKSoC8PO/uA8S1TIhSh4lJyJWUAD166PRHLwQ5MXGmCut9sN0Xhth6JcitUUReY2IScl4tUuXIy9OKO4iVp1eLqzmCnJgwtkTBxcQk5MrK3L/BXLTMTL2XTBBP8F5f1efkaHwZ2+RjYkKUOkxMQq601NvEBAju6q9eX9U7g0ODFtsDB3S2kZexzc4GOncOXmyJbMDEJMSM0ROrVzMbHEGceuksXOflVT0QzNg6yYCXiQlXfyVKHSYmIbZrl65x4XWLSRBP8F6vs+EIcmz5vSUKJiYmIeZcedpwgi8v13ufBIXXy9E7glh5lpXpow3f26DFlsgGTExCzKbExBhNToLCqTy97iYrKAB27tRxGUFhy/fWGRtljLflIAoaJiYhZsuVZxAXArOl8gziehtlZUB6OtC7t7fl6NsX2LePq78SJRsTkxCzrfIMUrN4WRmQlQV07+5tOYIY29JSXf8mzeOzVxBjS2QDJiYhVlqqUx6zs70tRxBP8M5sJ69WfXUEMbZlZd53kQFclp4oVZiYhJgtlWefPnr1G6QTvNcL1zmCWHnaMMUdCGY3GZENmJiEmA2LqwF6I7a8vGCd4G25qndWfw1abG343gaxNYrIBkxMQsyWxATQcgTpBG9LbIO2EFhdnc7esiHpy84GunYNTmyJbMHEJMRsufIEglV5Vlfrkuk2VJ5AsGK7bRtQX2/P9zaot1Mg8hITk5CqqgJ277bnBB+kytNZj4WxTT5b1odxBCm2RLZgYhJStkwVdhQUAFu3AjU1Xpek/WyrPIPUTWbb95aJCVHyWZmYiMhUEVklIkUiMivO9u+JyGeRnw9FZEzUtg0islxElorIIndL7h/OCd6myhPQ5MTvbIzt7t3aSuZ3tiZ9XP2VKHmsS0xEJB3AvQBOBzAKwAwRGRWz23oA3zDGHA3gNgAPxmw/yRgz1hhTmPIC+5RtV55OReNUPH5my4q6Die2QVjy38akr6pK7yRNRMlhXWICYDyAImPMOmNMNYBnAEyL3sEY86ExZmfk1wUA+rtcRt9jYpI6Tmz79PG2HI4gxbasDOjWTRcGtEGQYktkCxsTk34ANkf9Xhx5rin/BWB+1O8GwOsislhEZqagfIFQVqbrh3h9vxFHkE7wZWUa1w4dvC6JcpLPIMTWlsXVHE5ZODOHKHkyvC5AHPHWIY3bgysiJ0ETkxOjnp5kjCkRkT4A3hCRlcaY9+K8diaAmQAwcODA9pfaZ2y534gjSIkJK8/UsWmKOxCs7y2RLSyplhooBjAg6vf+ABqNexeRowE8DGCaMWa787wxpiTyuBXAC9CuoUaMMQ8aYwqNMYW5ublJLL4/2LIAmCMrC+jZk5VnKuTm6kJrQag8bU36ghBbIlvYmJh8AmCYiAwRkQ4ApgN4KXoHERkI4HkA3zfGrI56vouIZDv/BnAqgM9dK7mP2HaCB7Q8QTjB27IcvSMjQ5OToMTWpqSvVy+NbxBiS2QL67pyjDG1InIlgNcApAOYbYz5QkR+HNn+AIBfAcgBcJ/oHehqIzNw8gC8EHkuA8BTxphXPfgY1istBcbHbUvyThASE2Psa40CghHbvXv1x6akLy1Nu0T9Hlsim1iXmACAMWYegHkxzz0Q9e//BvDfcV63DsCY2OepodpaoKLCzspz4UKvS9E+u3bpkvQ2VZ5AMBIT26ZhOwoK/B9bIpvY2JVDKVZerlf2tp3gg1B52rbOhiMI93SxNbb5+f6PLZFNmJiEkM1Xnvv2AZWVXpek7WyNrZP0+XmFUttjS0TJwcQkhGxbXM0RhBkONl/V19QAO3e2vK+tbI7t1q1AXZ3XJSEKBiYmIcTEJHVsvqoH/B/bjAwgJ8frkjSUnw/U1wPbtnldEqJgYGISQk5ikpfnbTliBaHyLC0FOnbUZdNtEpTY2rQooCMIsSWyiWX/xckNpaV61WnLkumOIJzgnXU2JN76xR4KSmxt68YBghFbIpswMQkhG9fZADRZysjw9wwHGxeuAw79vf0cW9sWV3MEIbZENmFiEkK2nuCDsFiVrVf13bppF5OfY2tr0ud0ifo5tkQ2YWISQra2mAD+n3ppa9In4u/Y1tXZuSggAHTpAmRn+ze2RLZhYhIyxthbeQL+rjwPHgR27LDzqh7wd2y3btWZL4wtUfAxMQmZHTvsXDLd4ecTfHm5PjLpSz5bp2E7/BxbItswMQkZP5zgy8v9uViVrQuAOfy8dLofYsvEhCg5mJiEjO2JSUGBfxer8kNst2/XFjO/sT22fk76iGzDxCRknBO8zVeegD+vPv1wVQ/oeA2/sT22BQXAnj3A/v1el4TI/5iYhIztJ3g/JyZlZTr7pU8fr0sSn99j26OHTnm2kRNbZ5wREbUdE5OQKSsDOnXS6Y028nPlWVoK9O4NZGZ6XZL4/B5bW5NpwN+xJbINE5OQcRYAs23JdIezWJUf++ttXVzN4ZSNsU0+JiZEycPEJGRsP8F37ao/fjzB27w+DODvFUptXhQQYGJClExMTELG9sQE0ArIjyd427sbsrKAXr38F1tnUUCbY5ubq62QfmyNIrINE5OQsf0ED/hzTQjbV9R1+DG2lZU628Xm2GZkaHLit9gS2YiJSYhUV+s6FkxMkm/HDqCmhrFNBdunuDv82tJHZBsmJiHirF9h+wnez5WnzVf1gD8XAvNTbP32vSWyEROTEPHTCX73bqCqyuuSJM729WEcTuVpjNclSZzfYktE7cPEJET81CQO+Osk76fYVlXpuA2/8FNC7bekj8hGTExCxC+Vpx+nXjpX9X6oPAH/xTYzE+jZ0+uSNC8/X8cZ7djhdUmI/I2JSYg4lZGtS6Y7/Fh52r6irsOvsc3LA9IsP1v5MbZENrL8vzolU1mZrmORleV1SZrnxxO8swCYrSvqOvwcW9v5MbZENmJiEiJ+WMME8OdiVX6JrR+XpfdLbP04NorIRi0mJiIy3Y2CxBxzqoisEpEiEZkVZ7uIyD2R7Z+JyLhEXxtmfjnB+3GxKr/Etlcvja+fYmv7iroOtpgQJUciLSaPi8jbIjIq5aUBICLpAO4FcDqAUQBmxDn26QCGRX5mAri/Fa8NLb+c4AH/LVbll+6GtDR/TWutqQG2bfNHbLt1Azp29E9siWyVSGJyLIBMAJ+KyJ9EpGuKyzQeQJExZp0xphrAMwCmxewzDcDjRi0A0ENEChJ8bSj54X4j0fxUeR44AOzaxdimQkWFfnf9EFsRfy5gR2SbFhMTY8xyY8zXoC0TFwFYJSIzUlimfgA2R/1eHHkukX0SeW0o7d2r9xvxwwke8FflWV6uj4xt8vllDROHn2JLZKuEB78aYx4DMALAiwCeEJF/i8iRKShTvHkNsUsWNbVPIq/VNxCZKSKLRGRRRUVFK4voP35Zw8Thp8Wq/Fh5+uWq3q/fWyJqu1bNyjHG7DbGXAHgOAC9od07d4pIMldvKAYwIOr3/gBKEtwnkdcCAIwxDxpjCo0xhbm5ue0utO38eIL3y2JVfoxtRQVQV+d1SVrmt9j6bWwUkY0SSkxEJFNExovIT0XkKQDPATgSQAaAKwCsFJGzklSmTwAME5EhItIBwHQAL8Xs8xKAH0Rm55wAYLcxpjTB14aS307wfprW6sfY1tcfuqmjzZy/f16et+VIVH6+3sG7utrrkhD5VyLThT8EsAfARwDuBDAcwMsALoS2SPSBDjKdIyI/bm+BjDG1AK4E8BqAFQCeNcZ8ISI/jnr/eQDWASgC8BCAy5t7bXvLFAR+qzz9tCaEU3navqKuw0+xLSsDunfXVXX9wPn/5Yekj8hWGQnssxfAHQA+ALDAGLMvzj7XiUg5gF8AeKC9hTLGzIMmH9HPPRD1bwNtqUnotaQn+PR0ICfH65Ikxk9rQpSVAb176/1c/MBvsfVLMg00bOnr39/bshD5VYuJiTHm1ATf6z0Av2tfcShV/HK/EYdzVe+Xrhy/DHwF/JeYMLZE4ZLMamoZuGaItfx2gs/O1uZ7P5zg/XxVbzvGlih8kpaYGGOqjDEvJ+v9KLn8doIX8c8MB7/FtnNnXaWUsU0+Z5CuH2JLZCufNOxTe/ntBA/4Y70Nv62o6ygosD+2e/fqj59im5Wl9yNiYkLUdkxMQqC+Xlcn9dMJHvBHi8muXcDBg/6LrR8WAvPbTDKHH5I+IpsxMQmB7dt1MS2/neD90GLit1VfHX6oPP0aWz8kfUQ2Y2ISAn698szP1xaJAwe8LknT/Bxb2ytPv8bWD0kfkc2YmISAc5L04wkesLsC9XPluW8fUFnpdUma5tfY+uk+T0Q2YmISAn4+wQNMTFLBL7H106KAjoICHXe0a5fXJSHyJyYmIeDXytMPLSalpToTo0cPr0vSOn6IbVmZLvOfnu51SVrHD0kfkc2YmIRAWRnQpQvQtavXJWkdPyxW5UwVFvG6JK3jh9iWlvovmQb8tWoxkY2YmISAH9fZAPRqWcTuK0+/xtYvLSZ+m5EDsMWEqL2YmISAXyvPjAwgN9fuK0+/xrZXL40vY5t8bDEhah8mJiHg1xM8YP8ia36NbVqaLp9ua2z9uiggAHTvruOObI0tke2YmISAXytPwO5F1mpqgG3b/Btbm9fb8OuigMCh+zzZGlsi2zExCbiDB4GdO/15ggfsXghs61Zdq4KxTT6/ziRz2BxbItsxMQm48nJ99OMgQkDLXV6uTfu28euS6Q6bu8mc1gY/x5YtJkRtw8Qk4IJw5VlTA+zY4XVJGgtCbLduBWprvS5JY0GIra1JH5HtmJgEnN9P8DZPaw1CbI0BKiq8LkljQYjtjh3alUpErcPEJOD8foK3eSEwJ7Z5ed6Wo61sj60fFwV0OLF1ulKJKHFMTALOqTz79PG2HG1le4tJjx5Ax45el6RtbI+tX5NpwO7YEtmOiUnAlZUBvXsDmZlel6RtbL6qLy317+BMwO7Y+j0xsTm2RLZjYhJwfj/BZ2drk76NV55+j63NS6f7PeljiwlR2zExCTi/3ggtmq0zHPyemHTsqF1RNl7V+z22zn2ebIwtke2YmASc30/wgL2rvwYhtjauZXLgALBrl79jm5mpXai2xZbID5iYBJgxrDxTZe9eYN8+/8fWxtYoZyZLEGJrY0JNZDsmJgG2Z49effIEn3x+X5nUYWNs/T7F3WFjQk3kB0xMAixIJ/jdu4GqKq9LckiQYltWpq1rtvD7Uv8OG5M+Ij+wKjERkV4i8oaIrIk89oyzzwAR+beIrBCRL0Tk6qhtN4vIFhFZGvk5w91PYJegVJ42zh4JUmz37wcqK70uySFOZe732NqY9BH5gVWJCYBZAN4yxgwD8Fbk91i1AK4zxowEcAKAK0RkVNT2u4wxYyM/81JfZHsFpfK0ceolY5s6ZWU6oyU31+uStI/N93kispltick0AI9F/v0YgO/E7mCMKTXGLIn8uxLACgD93CqgnwSl8rS1xSQjA8jJ8bok7WPjQmB+XxTQYWPSR+QHtiUmecaYUkATEADNLqQuIoMBHAPg46inrxSRz0RkdryuoDApKQE6dAB69fK6JO3jnOBtqzzz8oA02/4HtZKNlWcQZpIBdiZ9RH7g+mlVRN4Ukc/j/Exr5ft0BfAcgGuMMXsiT98P4HAAYwGUArizmdfPFJFFIrKowsbbqyaBs3qmiNclaZ/cXE0AbKo8g7BwHWBn5VlW5v+Br4CdSR+RH2S4fUBjzClNbRORchEpMMaUikgBgK1N7JcJTUr+box5Puq9y6P2eQjAK82U40EADwJAYWFhIIenlZQE4wSfnq7JiW2VZ9++Xpei/Xr10i4TmyrPsjJgxAivS9F+NiZ9RH5gW0P0SwAujvz7YgBzY3cQEQHwfwBWGGP+HLMtuho+G8DnKSqnL5SWBqPyBOxbEyIo3Q0idi2yZkxwWqOys4HOne2JLZFf2JaY/A7AFBFZA2BK5HeISF8RcWbYTALwfQDfjDMt+A8islxEPgNwEoBrXS6/Vfx+I7RoNq0JUV8PbN0ajMoT0O+ILbHdtQuorg5GbJ2kz5bYEvmF6105zTHGbAdwcpznSwCcEfn3+wDijpowxnw/pQX0kaoqYOfOYLWYLF/udSlURQVQVxespG/DBq9LoUpK9DFI31u2mBC1jm0tJpQkQVk905Gfr/dQqa/3uiTBrDxtuaoPWmzZYkLUekxMAso5wQclMSkoAGprge3bvS5JMCvPbdt0MTCvBeUeRA62mBC1HhOTgHJO8EGqPAE7TvJBTEyM0XEzXgtaQp2fr+NmbLrPE5HtmJgEVNBO8DYtsubENggDNAG71tsoKQG6dwe6dPG6JMnhxLa8vPn9iOgQJiYBVVqq61P4fcl0h00tJqWluq6K35dMd9i03kZJSXBaogC7Emoiv2BiElDOWhB+XzLdwcozdWxqMQnSFHfAroSayC8CUm1RrKCs+uro2lV/bDjBBy0xycvTR8Y2+dhiQtR6TEwCKkirvjpsWaE0aJVnVpYuTe915WlM8GJr432eiGzHxCSggtZiAtix3kZdnQ5kDFpsbUj6duzQVV+DFNv0dKBPH++/t0R+wsQkgA4e1JN8kK48ATsWq9q6VRd5C1psbUj6gjbF3WFD0kfkJ0xMAihoq746+vY9NFXXK0Fbw8RRUMDYpooNsSXyEyYmARS0NUwc/foBe/cClZXelSGolWffvtpiYYx3ZQh6bIkoMUxMAiioTeLO59myxbsyBLXy7NdPx3d4ueR/kBPq8nK9pQIRtYyJSQAF+QQPeNssXlqqt7N3ptgGhQ1JX2kp0KMH0KmTd2VIhb59dVwSx5kQJYaJSQCVlupsgNxcr0uSXDZUniUlOssiI8O7MqSCk/R5HdugtUQBdiTURH7CxCSASkqCteqrw6m0vDzBs/JMnaDG1oaEmshPAlZ1ERDMxdUAXfm1Wzde1aeCs3S617ENWvcjYEdrFJGfMDEJoKDdbyRav36sPFOhQwftovKqxcSY4CbUubna9ceuHKLEMDEJoKBWnoC3a5nU1uoCa0GsPAH9XF4lfdu3AzU1wYxtWpr+f2SLCVFimJgETHU1sG1bME/wgLctJuXlemXP2CZfUKdhO/r1Y4sJUaKYmARMUFd9dfTrp03+9fXuHzvolaeXrVFBneLu8LI1ishvmJgETBhO8LW1QEWF+8cOemz79dOuqupq948d1EUBHV6PjSLyEyYmAeOc/Pr397YcqeLltNagt5g4sfViIbAwJH179ugtFYioeUxMAqa4WB+Dmph4uSZEaakOZOzTx/1ju8HL2JaUAL16AR07un9sN9iwBg+RXzAxCZgtW4CsLCAnx+uSpIbXLSZ5ecFb9dXhdWyD2loCcC0TotZgYhIwxcV6EhTxuiSpkZenn82rq/qgduMA3rdGhSG2bDEhahkTk4DZsiW43TgAkJmpyQmv6pOvd2+NrxeJSXFxsBMTtpgQJY6JScAUFwc7MQG8m3pZXAwMGOD+cd0i4s2U4dpabTEJcmyzs/WHiQlRy6xKTESkl4i8ISJrIo89m9hvg4gsF5GlIrKota8PKmP0xOdcnQWVF1Mv9+/X1UmDnvR5EduyMl2XJuix9XKdGCI/sSoxATALwFvGmGEA3or83pSTjDFjjTGFbXx94GzfDhw8yBN8KgR9GrbDixVKnZlkQW4xAbiWCVGibEtMpgF4LPLvxwB8x+XX+1rQpwo7+vXTZfcPHnTvmGGpPL3oJgvL95YtJkSJsS0xyTPGlAJA5LGpFSMMgNdFZLGIzGzD6wPJOcEHvSvHGSTprBbqhrBUnv366SJge/a4d8zNm/UxDLEtKfHmdgpEfuL6igwi8iaA/DibbmzF20wyxpSISB8Ab4jISmPMe60sx0wAMwFg4MCBrXmptcLU3QDo5x082J1jOpVnWJK+khKgWzd3jllcDHTqBPQM+Iiwfv30DsrbtgV3kT6iZHC9xcQYc4ox5qg4P3MBlItIAQBEHrc28R4lkcetAF4AMD6yKaHXR177oDGm0BhTmJubm7wP6KHiYiA9HciPl/YFiBfrbRQX68qknTu7d0wveDGt1ZntFNS1dxxcy4QoMbZ15bwE4OLIvy8GMDd2BxHpIiLZzr8BnArg80RfH2RbtmhSkp7udUlSyxnn4XSvuCHoU4UdTmLidmyD3soHeBNbIj+yLTH5HYApIrIGwJTI7xCRviIyL7JPHoD3RWQZgIUA/mWMebW514dFWE7wPXoAXboAmza5d8ywxNb5jE7XlRvCElsnsXUztkR+ZNVdP4wx2wGcHOf5EgBnRP69DsCY1rw+LIqLgZEjvS5F6okAAwe6e4LfvBkYP77l/fyuUycgN9e9pK+uLvirFTvy8/U+S0xMiJpnW4sJtUNYTvCAXn26dYI/cEAHLIahKwdwN+krL9fkJAzf2/R07c5hYkLUPCYmAbFnj/6E4QQPaJLg1lV9WGY7OdxM+sKyPoxj4EB3uyCJ/IiJSUA4lWfQp7M6Bg7Uq203FlkLyxomDjeTvjDGli0mRM1jYhIQYbyqB9yZ1hqWBcAcAwcClZXA7t2pP1bYEpOBA/Uzc5E1oqYxMQmIsJ3gncTEjSt7xjZ1Nm8GOnYEcnJSfywbDBigi6yVl3tdEiJ7MTEJCOeq3lnEKeicxXrdaBYvLtZVSbt0Sf2xbOB2bPv3D/7iag4nthxnQtQ0JiYBsXGjTkfs2NHrkrjDzfU2wrLOhsPN9TYYWyKKxcQkIDZuBAYN8roU7uncGejd273uhjBVngUFOrXVrW6yMMWWLSZELWNiEhCbNh066YWFWzMcwrIcvcOt9Tbq68O19g5waNVitpgQNY2JSQAYo4lJmFpMAHcSk4MHga1bw1V5Au6st1FergNBw5T0OasWs8WEqGlMTAKgokJXJw1bi4kbJ3jn/Zn0Jd+GDfo4eHBqj2MbrmVC1DwmJgGwcaM+hrHy3L1b19xIlTBXnqlebyOs31u2mBA1j4lJAIT1BO/GDIewxnbgQKC6WruxUsVJ+sIW2wED3Fu1mMiPmJgEgHP1FcauHCC1V58bNhwaDBombiV9OTlA166pO4aNnO+ts3AfETXExCQANm4EsrN1xH+YuFF5btigx8nISN0xbORW0he2LjKAa5kQtYSJSQA4U4XDsnqmo29fIC0ttZVn2NaHcbjVYhLG2HItE6LmMTEJgLCe4DMytIvFGQeSCmG9qu/VSxexS1VsjQlvbJ2p50xMiOJjYhIAGzeGb3yJY8gQYP361Lx3dTVQUhLOpE8ktbHdtg2oqgpnbDt1AvLyDg3+JaKGmJj43N69wI4d4TzBA8BhhwHr1qXmvZ3psmG8qgdSm5iEdUaO47DDUhdbIr9jYuJzYV0AzDFkiLZqHDiQ/PcO61Rhh5OYGJP893ZiG+akL1UJNZHfMTHxOecEH+auHCA1YyHCuria47DDdPG67duT/95sMdGBxTU1XpeEyD5MTHwu7Cd4JzFJRbP4hg061iJs98lxpDK2GzcC3buHb4q7Y8gQoK6OU4aJ4mFi4nPr1gFZWTp1NoycyjMVzeIbN+qsnw4dkv/efpDK2G7YEN5kGtAWE4DjTIjiYWLic2vX6kkuLaR/yYICTcxS1WIS5soz1S0mYe0iA1Kb9BH5XUirs+BYt+7Q1VcYpaVpBcfKM/mys4HevZMfW2cNkzAnff376zo8bDEhaoyJiY8Zoy0mhx/udUm8lYpprTU12v8f5sQESM3ske3bdVBtmBPq9HRNzNhiQtQYExMf27ZN1zEJ8wkeSE3luWGDDk4cOjS57+s3qVhvo6hIHxlbtpgQxcPExMfWrtVHtpgAu3bpT7I4sQ175TlkiHZp1dUl7z2ZmCiuZUIUn1WJiYj0EpE3RGRN5LFnnH1GiMjSqJ89InJNZNvNIrIlatsZrn8IFzkntbC3mKRihgMrTzVkCFBbq6vgJktR0aEl78PssMO01bOy0uuSENnFqsQEwCwAbxljhgF4K/J7A8aYVcaYscaYsQCOBbAfwAtRu9zlbDfGzHOj0F5xEpOwn+BTMXukqAjo0kXvaRJmqUj61q7VuxdnZSXvPf0olbOeiPzMtsRkGoDHIv9+DMB3Wtj/ZABrjTEpvL+svdau1fVLOnXyuiTeSsXUy6IibS0RSd57+lGqkr6wt0QBTEyImmJbYpJnjCkFgMhjnxb2nw7g6ZjnrhSRz0RkdryuoCBZt47jSwCgZ0+gV69D3S/JwMpTDRyoM0gY2+RzWqOc8UxEpFxPTETkTRH5PM7PtFa+TwcAZwH4Z9TT9wM4HMBYAKUA7mzm9TNFZJGILKqoqGj9B7GAs7gaAcOHA6tXJ+e96uo06WPlCWRm6pV9smK7e7eOq2BCDeTkaEK9Zo3XJSGyS4bbBzTGnNLUNhEpF5ECY0ypiBQA2NrMW50OYIkxpjzqvb/6t4g8BOCVZsrxIIAHAaCwsDAF909NraoqYMsWnuAdw4cDb76ZnPdybq7GxESNGAGsWpWc9+Jsp4aGD09ebImCwraunJcAXBz598UA5jaz7wzEdONEkhnH2QA+T2rpLOJcZQ0f7m05bDFiBFBSouu6tBdn5DQ0YoR+3+rr2/9ejG1DyUz6iILCtsTkdwCmiMgaAFMiv0NE+orIVzNsRKRzZPvzMa//g4gsF5HPAJwE4Fp3iu0+52Q2YoS35bCFk6Alo8vBqTzZGqWGDwcOHEjOnXAZ24achJpThokOcb0rpznGmO3QmTaxz5cAOCPq9/0AcuLs9/2UFtAiK1fq47Bh3pbDFk6Ctno1MG5c+95r7VqdytqvX/vLFQTRsW3v/W2KioD8fJ2KTYcS6jVr2v+9JQoK21pMKEGrVulaEDzBK6drIBktJitXasIX1js2x3ISk2R0OaxYwVa+aMmMLVFQ8NTrU6tWAUcc4XUp7NGpk05tTVblOXJk+98nKPLzga5d25/0GcPYxnLWymFiQnQIExMfMkZPZLzybGjEiPZXngcO6IJXrDwPEUnOIM2yMp0uzNge0rGjdo8lazo2URAwMfGhsjIdLMfEpCFn6qVpx+Tv1at19gkrz4aSsU7MihX6yNg2xJk5RA0xMfEhzsiJb8QITdjKytr+Hl9+qY+sPBsaMULvMlxV1fb3YGISn9PS156EmihImJj4kDMjh4lJQ86YG6cCbIsVK3TQK9eHaeiII7TibE+ryYoVQHY2ZzvFGjFC19/ZssXrkhDZgYmJDy1fDnTrprNy6JDRo/Vx+fK2v8eKFboEe9hvjBjrqKP0sb2xPeII3hgx1pFH6uPngV0Okqh1mJh44I47gN/+tu2vX75cK2Ge4BvKywN6925/5cmuhsaGD9f75jC2ydfehLq+HrjwQmDevJb3JfIDJiYeWLAAePLJtr3WGOCzzw6dzOgQEb2yb+sJvqZGuypYeTaWmalxaWtsd+4ESkuBUaOSW64g6NVLu7faGtsNG4Bnn9UVZImCgImJB8aO1QGsbRlIWFysUy6PPjrpxQqE0aOBL75o231dVq0CqquBMWOSX64gGD267ZXnsmX6OHZs0ooTKKNH6wVHWzh/E16sUFAwMfHAmDFacbalT5knoeaNHg3s26dXka316af6yMozvtGjNTHeubP1r126VB8Z2/hGj9aurpqa1r/20091wLYzDojI75iYeMA5OTsn69Zwrqp4EoqvPf31S5fqglec7RSfE9u2JNRLl+oKsnl5SS1SYIwera11zl3DW+PTT/U7y9tTUFAwMfHA4ME6q8Zp3m6N5ct16fUePZJdqmBwZji0NTEZPRrIsOrWlvZob9LH1pKmOV2zbYntkiXAMccktzxEXmJi4oG0ND0RtaXFZOlSji9pTna23oBvyZLWvc4YVp4t6d9fB2o6XV6Jqq7WhesY26YdcYQmxK09J1RUaPca70xMQcLExCNjx2qLSWsGaVZWaj90YWHKihUIxx0HLFzYutcUFwM7drDybI6IxvaTT1r3OmfsBGPbtKwsHXvW2u+tkySyxYSChImJR8aO1dUei4oSf82SJXplf9xxKStWIIwfr6totmb6JAdnJmb8eB1jsm9f4q9xKk/Odmre+PGa9LXmYoUDtimImJh45IQT9HHBgsRf41ypMjFpnhOf1lzZf/wxkJ7OyrMl48cDdXWt685ZsEDHVHGZ/+aNH6+toq25od+SJTpmrVevlBWLyHVMTDwyciTQvTvw4YeJv+aTT/QW6bm5qStXEBxzjCYZrWkW/+gjTUo4s6F5TtLX2tgef7yOraKmHX+8Pn78ceKvWbCAFyoUPDxVeCQtDZgwoXWJycKFPAklolMnHSCcaItJba1WBhMnprZcQZCXp8lxoolJZaV2/TC2LRsxQgdvJxrbzZuBTZuAE09MbbmI3MbExEMTJ+pJe/fulvfdvFkXDeNJKDHjx+sJvq6u5X2dMRMTJqS+XEEwfnziXZALF+qYCca2ZWlpeuGRaIvJBx/o46RJqSsTkReYmHho4kQdzJrIiejdd/Vx8uSUFikwvv51TfgSmX7ptFqx8kzMiScCGzcmtrquE1unm4KaN2mSfmd37Wp53w8+0K5HjouioGFi4qHx4/Uq6T//aXnfd94BevbkUvSJOukkfXz77Zb3festYMAAHURILTv5ZH1MJLZvvqljfrggYGJOPllbmJwLkea8954OoueCgBQ0TEw8lJ2tV5Kvvdbyvu+8o60AHECYmIICHWD87383v19trSYmp56q63RQy0aNAvr0aTkx2btXB76eeqo75QqCE07QMVJvvdX8fqWlenuKU05xp1xEbmI157HTTwcWLdIVHJuydq3+fPOb7pUrCE46Sa8qq6ub3mfRIu3yYeWZOBH9Lr79tnZFNuXdd3VhtSlT3Cub32VlAV/7WsuJiXMxM3Vq6stE5DYmJh6bOlVP7q+/3vQ+c+fq41lnuVOmoDj9dB3U2tyV/WuvaUXLK8/WOe00vWpfvLjpfV57Ta/+OTizdU47TZfwX7eu6X3mz9ebInJ8CQURExOPHXusrkvy0ktN7/Piizr9lWMgWmfKFO0ue+65pvd5/nkd9MoFqlrnrLN0bMOcOfG319cDL7ygf4OOHd0tm9+dc44+NhXb6mq9kJk6ld2PFExMTDyWlgZccIEmJvGmDZeX6+j7adPcL5vfZWUBZ56pFWRtbePtK1dqP/2FF7pfNr/r1UsHas6ZE787Z8ECvf/QBRe4Xza/GzxY74f1z3/G3z5/vs7aYWwpqJiYWODii4EDB+KfiB59VK8+Z8xwvViBcMEFwPbtwLx5jbc99pgmhued5365guD883XsU7xFAh95RLtxvv1t98sVBBdcoOOfVq5svO3xx7WVlWN3KKiYmFigsFBnkDzwQMOrz9pa4MEHdTbOyJHelc/PvvUtoF8/4K9/bfh8VZXG9jvfAfr29aRovnfhhXpbhf/934bP79gB/P3vwEUX6T1yqPUuvhjo0KFxbNev167dSy7hNGEKLqsSExE5X0S+EJF6ESlsZr+pIrJKRIpEZFbU871E5A0RWRN57OlOydtHBPif/9GBhNFjTR55RAfA/exn3pXN7zIzgSuvBN54o+HaEH/5i1ag11zjWdF8r2tXYOZMbelbtuzQ83fcoS2AP/2pd2Xzuz59NLGbPVuTEcdvfqP3gbr6au/KRpRqYpqb7+cyERkJoB7A3wBcb4xZFGefdACrAUwBUAzgEwAzjDFfisgfAOwwxvwukrD0NMbc0NJxCwsLzaJFjQ7lqtpaHeC6a5cmKFVVugDbiBHA++9zkFt77N+vLU4dO+p4ndJSHfA6ZYqOP6G227FD7xo8aJCutbN8ua5O/IMfAA8/7HXp/K24GDjiCF3bZN48XazuW98Crr8e+OMfvS6dEpHFxpgmLyKJ2sKqxMQhIu+g6cRkAoCbjTGnRX7/OQAYY+4QkVUAJhtjSkWkAMA7xpgRLR3PhsQE0KvOiRP1Kr+uTh8XLODt4pPhvfd0rZIuXTRR6dlT7+PSv7/XJfO/l18Gzj5bB8Tu2gUMHKi3WcjJ8bpk/vfII8Cll+rU4IoK4KijdExP585el0wxMaFUsKorJ0H9AGyO+r048hwA5BljSgEg8tinqTcRkZkiskhEFlU0t7qZi8aM0ZPOWWfp4DcmJcnz9a9rV85pp2n//ccfMylJlm9/W7vKTjoJ+PGP9TvMpCQ5fvhD7d6dNEm7dP/9b3uSEqJUcb3FRETeBJAfZ9ONxpi5kX3eQdMtJucDOM0Y89+R378PYLwx5ioR2WWM6RG1705jTIvjTGxpMSEi8hO2mFAquD6u2xjT3jU2iwEMiPq9P4CSyL/LRaQgqitnazuPRURERC7yY1fOJwCGicgQEekAYDoAZy7LSwAujvz7YgBzPSgfERERtZFViYmInC0ixQAmAPiXiLwWeb6viMwDAGNMLYArAbwGYAWAZ40xX0Te4ncApojIGuisnd+5/RmIiIio7aycleM2jjEhImo9jjGhVLCqxYSIiIjCjYkJERERWYOJCREREVmDiQkRERFZg4kJERERWYOJCREREVmDiQkRERFZg4kJERERWYOJCREREVmDK78CEJEKABtdPmxvANtcPqbtGJPGGJP4GJfGvIjJIGNMrsvHpIBjYuIREVnEpZwbYkwaY0ziY1waY0woKNiVQ0RERNZgYkJERETWYGLinQe9LoCFGJPGGJP4GJfGGBMKBI4xISIiImuwxYSIiIiswcTEZSJyvoh8ISL1IlIYs+3nIlIkIqtE5DSvyuglEblZRLaIyNLIzxlel8krIjI18l0oEpFZXpfHBiKyQUSWR74bi7wujxdEZLaIbBWRz6Oe6yUib4jImshjTy/LSNQeTEzc9zmAcwC8F/2kiIwCMB3AkQCmArhPRNLdL54V7jLGjI38zPO6MF6I/O3vBXA6gFEAZkS+IwScFPluhHVq7KPQc0S0WQDeMsYMA/BW5HciX2Ji4jJjzApjzKo4m6YBeMYYc9AYsx5AEYDx7paOLDIeQJExZp0xphrAM9DvCIWcMeY9ADtinp4G4LHIvx8D8B03y0SUTExM7NEPwOao34sjz4XRlSLyWaTJOqxN0vw+xGcAvC4ii0VkpteFsUieMaYUACKPfTwuD1GbZXhdgCASkTcB5MfZdKMxZm5TL4vzXCCnTDUXHwD3A7gN+tlvA3AngEvdK501QvN9aKVJxpgSEekD4A0RWRlpQSCigGBikgLGmFPa8LJiAAOifu8PoCQ5JbJLovERkYcAvJLi4tgqNN+H1jDGlEQet4rIC9AuLyYmQLmIFBhjSkWkAMBWrwtE1FbsyrHHSwCmi0iWiAwBMAzAQo/L5LrISdVxNnSwcBh9AmCYiAwRkQ7QgdEveVwmT4lIFxHJdv4N4FSE9/sR6yUAF0f+fTGAplpmiazHFhOXicjZAP4XQC6Af4nIUmPMacaYL0TkWQBfAqgFcIUxps7LsnrkDyIyFtptsQHAZZ6WxiPGmFoRuRLAawDSAcw2xnzhcbG8lgfgBREB9Nz1lDHmVW+L5D4ReRrAZAC9RaQYwK8B/A7AsyLyXwA2ATjfuxIStQ9XfiUiIiJrsCuHiIiIrMHEhIiIiKzBxISIiIiswcSEiIiIrMHEhIiIiKzBxISIiIiswcSEiIiIrMHEhIiIiKzBxITIIiIyVERqROSWmOfvF5FKESn0qmxERG5gYkJkEWNMEYCHAVwrIr0BQER+Bb3D8tnGmEVelo+IKNW4JD2RZUQkH8BaAPcBWAngQQAzjDHPelowIiIX8CZ+RJYxxpSJyN0AroP+H/0pkxIiCgt25RDZaQ2ALAAfGWPu9bowRERuYWJCZBkR+SaAvwH4CMAkERnjcZGIiFzDxITIIiIyDsCL0AGwkwFsAvBbD4tEROQqJiZElhCRoQDmA3gdwFXGmGoAtwA4Q0S+7mnhiIhcwlk5RBaIzMT5ENpCcpox5mDk+XQAnwPYaYyZ6GERiYhcwcSEiIiIrMGuHCIiIrIGExMiIiKyBhMTIiIisgYTEyIiIrIGExMiIiKyBhMTIiIisgYTEyIiIrIGExMiIiKyBhMTIiIissb/A5w+hG4mxn7cAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(6, 6))\n", "ax1 = plt.subplot(111)\n", "\n", "x = np.linspace(-10,10,1000)\n", "y = np.cos(x)\n", "# plot the cos(x) part\n", "ax1.plot(x, y, 'b', label=r'$y=\\cos(x)$')\n", "# plot the y value we have (our data)\n", "yd = 0.5\n", "xlim = ax1.get_xlim()\n", "ax1.plot([xlim[0], xlim[1]], [yd, yd], 'k--', label='$y=y_d$')\n", "ax1.set_ylim([-1.1,1.1])\n", "ax1.set_xlabel('$x$', fontsize=16)\n", "ax1.set_ylabel('$y$', fontsize=16)\n", "ax1.set_title('Nonlinear inversion - scalar example', fontsize=16)\n", "ax1.legend(bbox_to_anchor=(1.05, 1), loc='upper left', fontsize=14)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you think about this plot you should be able to see that we can specify inversion problems very easily that have no solutions (non-existence), or infinitely many (non-uniqueness) solutions - these are very possible scenarios to encounter, not just a mathematical peculiarity.\n", "\n", "To solve this problem using Newton's method we just need to formulate it in a way that this method works with, i.e. as a root-finding problem:\n", "\n", "let's define the function we want to find the root of as the one whose solution is also a solution to our inversion problem, e.g.\n", "\n", "$$\\hat{f}:=y_d - \\cos(x)$$" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.0471975511965979\n", "1.0471975511966034\n", "0.4999999999999951\n" ] } ], "source": [ "yd = 0.5\n", " \n", "def fh(x):\n", " return yd - np.cos(x)\n", "\n", "def dfhdx(x):\n", " return np.sin(x)\n", "\n", "# initial guess\n", "x0 = 1.0\n", "print(sop.newton(fh, x0, dfhdx))\n", "\n", "\n", "# and if you don't have the derivative function it will use a quasi-Newton approach:\n", "print(sop.newton(fh, x0))\n", "\n", "# check our solution\n", "print(np.cos(sop.newton(fh, x0)) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Can you remember what Newton's iterative updates do?\n", "\n", "Why would an initial guess of 0 be bad, what about a very small value for the initial guess?\n", "\n", "Test in the code above.\n", "\n", "Can you come up with some data (i.e. a $y_d$ value) such that this problem has no solution, what does the Newton function do in this case?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### In multi-dimensions\n", "\n", "Note that we will be interested in multi-dimensional problems in this module. Newton's method does generalise to multi-dimensional problems - the division by the derivative at each iteration is replaced by the need to solve a linear system comprising the Jacobian matrix - but this is expensive and so a number of more efficient nonlinear optimisation algorithms have been developed (cf. later for the equivalence between inversion and optimisation problems)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Multi-dimensional and nonlinear problems\n", "\n", "In this module, we will look at methods that can be used to solve the multi-dimensional analogue of our original problem, that is find a solution for the vector $\\boldsymbol{x}$ given the equation \n", "\n", "$$\\boldsymbol{y} = F(\\boldsymbol{x})$$\n", "\n", "and in this module it will be vital that we consider scenarios where $\\boldsymbol{x}$ and $\\boldsymbol{y}$ may be of different lengths, i.e. live in spaces with different numbers of dimensions, either of which may also be very large, and where $F$ is an operator that maps a vector with the dimensions of $\\boldsymbol{x}$ into a vector with the dimensions of $\\boldsymbol{y}$. \n", "\n", "
\n", "\n", "In practice, we will also seldom have an analytical form for $F$.\n", "Also in general $F$ will not be a matrix; rather it is a (typically non-linear) function that turns one vector into another. You can think of it also as a large complicated numerical model/computer code that calculates $\\boldsymbol{y}$ for us given a particular value for $\\boldsymbol{x}$.\n", "\n", "
\n", "\n", "However, also note that numerical approaches to solve completely general problems such as this often approximate it as a series of linear/matrix problems (i.e. solve a nonlinear problem by considering a series of linear problems - note this is also what Newton's method does), and that is why we focus on situations where our inversion problem can be written as a linear system. \n", "\n", "Also, the concepts we learn and issues we appreciate can be more easily understood in the linear/matrix system, but these concepts are still highly applicable to/representative of the nonlinear case. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Terminology\n", "\n", "#### Parameter estimation and inversion\n", "\n", "Note that in the above abstract discussion, if we consider $\\boldsymbol{y}$ to be a vector of observations (data), and $\\boldsymbol{x}$ to be a vector of model parameters that control a model ($F$), \n", "\n", "then the problem: given $\\boldsymbol{y}$ find $\\boldsymbol{x}$ is often called a *parameter estimation problem*.\n", "\n", "
\n", "\n", "Note that the $\\boldsymbol{x}$ we want to invert for, or estimate, doesn't have to include ALL parameters that govern the model, i.e. all inputs to the model. \n", "\n", "In this case you may see notation something like\n", "\n", "$$F(\\boldsymbol{x}_1;\\boldsymbol{x}_2) = \\boldsymbol{y}$$\n", "\n", "The semi-colon helps differentiate different types of variables/parameters that are input into the model, e.g. those that we wish to invert for and those that will remain fixed during the inversion. \n", "\n", "For example, a known physical parameter may be fixed (gravity or fluid viscosity), whereas there are other parameters such as those describing the geology, the initial condition, etc, that either aren't known, or there is uncertainty over their precise values.\n", "\n", "
\n", "\n", "Sometimes when we have a relatively small number of \"parameters\" to invert for people use the term \"parameter estimation\", but when there are a large number, e.g. representing an entire unknown field, people tend to use the language \"inverse problem\", but it's really the same thing.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Model and parameters\n", "\n", "Note also that different communities use the term *model* in different contexts.\n", "\n", "Mathematicians/computational scientists would probably refer to $F$ (or $\\boldsymbol{y} = F(\\boldsymbol{x})$) as the \"model\", e.g. the underlying equations (or their discretisation in a computer code), and $\\boldsymbol{x}$ as \"parameters\".\n", "\n", "Others would refer to $F$ as the operator (and think of it as a given, and in a sense not a \"model\"; they may also refer to $F(\\boldsymbol{x})=\\boldsymbol{y}$ as the \"mathematical model\"), and would reserve the word \"model\" for the parameters $\\boldsymbol{x}$ being inverted for. \n", "\n", "
\n", "\n", "In seismic inversion problems for example this is because it represents a \"model\" for the real geological make-up of the subsurface. \n", "\n", "In fluid dynamics we would probably call this a \"parametrisation\" of some unknown, uncertain or unresolved physical process - again \"parameter\"!\n", "\n", "
\n", "\n", "In what follows I will generally try to stick to the language that $\\boldsymbol{y}=F(\\boldsymbol{x})$ is the model (e.g. a wave equation or the Navier-Stokes equations and specifically their implementation in code), with $ \\boldsymbol{y}$ the data (or variables that can be post-processed into a form that can be compared to observational data) and *some of* the $\\boldsymbol{x}$ the parameters. \n", "\n", "
\n", "\n", "
\n", "\n", "However, moving forward I will use common seismic inversion notation for the linear(ised) case we will mostly consider:\n", "\n", "$$G\\boldsymbol{m}=\\boldsymbol{d}$$\n", "\n", "where $G$ is a matrix representing the linear *model*, $\\boldsymbol{d}$ is *data* and $\\boldsymbol{m}$ are *parameters* we are inverting for." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Linear algebra review\n", "\n", "It turns out that to understand inversion problems linear algebra is key (even when we have nonlinear problems). So let's quickly review some notation and key results.\n", "\n", "## Notation & terminology\n", "\n", "### Vectors and matrices\n", "\n", "\n", "- We will use bold fonts to indicate vectors, e.g. $\\boldsymbol{x } $ :\n", "\n", "$$ \\boldsymbol{x} = (x_1,x_2,\\ldots, x_m)^T$$\n", "\n", "(the transpose is here since we generally assume a vector is a *column* vector (i.e. has dimension, or shape, $m\\times 1$), but it's clearly easier and more compact to display it as the transpose of a *row* vector in writing - note that when writing on paper it's common to indicate a bold symbol with an underline: $\\boldsymbol{x} \\equiv \\underline{x} $. Sometimes people will use $\\vec{x}$.)\n", "\n", "- and capital letters to indicate matrices, e.g. $A$:\n", "\n", "$$A = \n", "\\begin{pmatrix}\n", "a_{11} & a_{12} \\\\\n", "a_{21} & a_{22}\n", "\\end{pmatrix}$$\n", "\n", "\n", "- Subscript letters or numbers will be used to indicate components of arrays or matrices, e.g. $x_i$ is the $i$-th component of the vector $\\boldsymbol{x}$, and $a_{ij}$ is the entry from the $i$-th row and the $j$-th column of the matrix $A$, with of course us needing to be careful over our numbering which starts from zero in our Python implementations.\n", "\n", "\n", "- We will identify column vectors of the matrix $A$ as\n", "\n", "$$A = \\begin{pmatrix}\n", " & & & \\\\\n", " \\boldsymbol{a}_{\\,:1} & \\boldsymbol{a}_{\\,:2} & \\ldots & \\boldsymbol{a}_{\\,:n} \\\\\n", " & & & \n", "\\end{pmatrix}$$\n", "\n", "i.e. $\\boldsymbol{a}_{\\,:j}$ indicates the $j$-th column of matrix $A$.\n", "\n", "\n", "\n", "- Square or rounded (i.e. paretheses) brackets are both fine for vectors and matrices (not for intervals where they mean different things!). \n", "\n", "
\n", "\n", "\n", "See also \n", "\n", "Note that I've made the choice to start my indices at '1' for the purposes of writing things down mathematically. \n", "\n", "You could choose to start with '0' to better match Python indexing if you want, you just need to be consistent." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Matrix dimension\n", "\n", "A general [matrix](https://en.wikipedia.org/wiki/Matrix_(mathematics)) of dimension (shape in NumPy) $m \\times n$ is\n", "\n", "\\begin{pmatrix}\n", " a_{11} & a_{12} & \\dots & a_{1n} \\\\\n", " a_{21} & a_{22} & \\dots & a_{2n} \\\\\n", " \\vdots & \\vdots & \\ddots & \\vdots \\\\\n", " a_{m1} & a_{m2} & \\dots & a_{mn} \n", "\\end{pmatrix}\n", "\n", "where for a real matrix all *entries* $a_{ij}\\in\\mathbb{R}$, i.e. are real numbers, and for a complex matrix $a_{ij}\\in\\mathbb{C}$. \n", "\n", "Mathematically we could also write $A\\in\\mathbb{R}^{m\\times n}$ or $A\\in\\mathbb{C}^{m\\times n}$.\n", "\n", "We would say out loud that the matrix is of dimension \"m rows by n columns\" or just \"m by n\".\n", "\n", "The convention is to state the number of rows first, and columns second - we will see why this is sensible when we talk about matrix multiplication - indeed thinking about matrix multiplication is exactly how I remember that incrementing the first index ($i$) means we move down, and incrementing the second index ($j$) means we move to the right.\n", "\n", "A *square* matrix is one where $m=n$.\n", "\n", "An $m\\times 1$ matrix is also a *column vector*.\n", "\n", "A $1\\times n$ matrix is also a *row vector*." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[ True True]\n", " [ True True]]\n", "True\n", "\n", " [[ True True]\n", " [ True False]]\n", "False\n" ] } ], "source": [ "A = np.array([[1, 2], [3, 4]])\n", "B = np.array([[1, 2], [3, 4]])\n", "\n", "# the following compares element-wise\n", "print(A==B)\n", "# above works on pairs of entries element-wise, \n", "\n", "# while array_equal looks at the whole array.\n", "print(np.array_equal(A,B))\n", "\n", "# change one of the entries\n", "B = np.array([[1, 2], [3, 5]])\n", "\n", "print('\\n',A==B)\n", "print(np.array_equal(A,B))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Be careful about assignments vs copies\n", "\n", "See the two examples below, and discussions at \n", "\n", "\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "True\n", "True\n", "True\n", "False\n" ] } ], "source": [ "A = np.array([[1, 2], [3, 4]])\n", "B = A\n", "\n", "# the following should of course be true\n", "print(np.array_equal(A,B))\n", "\n", "# is it still true if we update one of the entries of ONE of the matrices but not the other?\n", "B[0]=2\n", "print(np.array_equal(A,B))\n", "\n", "# we may or may not want the above behaviour\n", "\n", "# alternatively we can make a copy before we start making changes:\n", "A = np.array([[1, 2], [3, 4]])\n", "B = np.copy(A)\n", "print(np.array_equal(A,B))\n", "B[0]=2\n", "print(np.array_equal(A,B))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Transpose of a matrix\n", "\n", "- If $A=A^T$ we say that $A$ is symmetric.\n", "\n", "\n", "- If $A=-A^T$ we say that $A$ is skew-symmetric (what does this imply the diagonal must be?)\n", "\n", "\n", "\n", "### Properties of the transpose operator\n", "\n", "The following properties are easy to establish:\n", "\n", "For matrices $A$ and $B$ of appropriate sizes (i.e. [*conformable*](https://en.wikipedia.org/wiki/Conformable_matrix) for the given operation - addition or multiplication - see later) and a scalar $\\alpha$:\n", "\n", "\n", "1. $(A+B)^T = A^T + B^T$\n", "\n", "\n", "2. $(\\alpha A)^T = \\alpha (A^T)$\n", "\n", "\n", "3. $(AB)^T = B^T A^T$\n", "\n", "\n", "4. $(A^T)^T = A$\n", "\n", "\n", "As an exercise you can try and prove these from the mathematical definitions, and also demonstrate for some simple matrices using appropriate NumPy commands. The third is particularly interesting and perhaps non-obvious so worth trying to prove/verify - see the homework exercises.\n", "\n", "NB. We haven't actually defined matrix multiplication ($AB$) yet, but we will do so shortly.\n", "\n", "How does the multiplicative property expand to more matrices, i.e. what is $(ABC)^T$?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Matrix-vector multiplication\n", "\n", "### Mathematical definition\n", "\n", "This is a good point to review how we can pre-multiply a vector by a matrix, and code up this operation ourselves.\n", "\n", "[Note that matrix-vector multiplication is just a special case of the more general matrix-matrix multiplication].\n", "\n", "Given an $m\\times n$ matrix $A$ and a vector $\\boldsymbol{x}\\;$ the product is a new vector we will call $\\boldsymbol{b}\\;$:\n", "\n", "$$ A\\boldsymbol{x}=\\boldsymbol{b}$$\n", "\n", "The size of $A$ dictates the size of $\\boldsymbol{x}\\;$ where this product makes sense (is defined), and tells us the resulting size of $\\boldsymbol{b}\\;$: \n", "\n", "if $A$ is $m\\times n$, then \n", "$\\boldsymbol{x}\\;$ has to be $n\\times 1$ and $\\boldsymbol{b}\\;$ will then be $m\\times 1$.\n", "\n", "\n", "$$b_i = \\sum_{j=1}^n a_{ij} x_j \\quad\\text{for}\\quad i=1,\\ldots, m $$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Code\n", "\n", "Let's implement our own matrix vector product function" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "def mat_vec_product(A, x):\n", " \"\"\"Function to compute the matrix vector product Ax.\n", "\n", " Parameters\n", " ----------\n", " A : ndarray\n", " Some matrix A with shape (m,n)\n", " x : array_like\n", " Some vector x with shape (n,)\n", "\n", " Returns\n", " -------\n", " b : ndarray\n", " RHS vector b with shape (m,)\n", " \"\"\"\n", " m, n = np.shape(A)\n", " assert x.ndim == 1 # restrict to the case where x is 1D\n", " assert n == len(x) # as 1D we can check the length of x is consistent with A\n", " b = np.zeros(m) # and then initialise to zero the appropriate length array for b\n", " for i in range(m):\n", " for j in range(n):\n", " b[i] += A[i, j] * x[j]\n", " return b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll test this shortly, but first well think about some properties and see how to do this in NumPy." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Properties\n", "\n", "Note that the action of multiplying by a matrix is what is called a [linear operator](https://en.wikipedia.org/wiki/Linear_map). This means that matrix multiplication satisfies the properties (for a matrix $A$, vectors $\\boldsymbol{x}$ and $\\boldsymbol{y}$, and scalar $\\alpha$)\n", "\n", "\n", "1. $A (x + y) = Ax + Ay$\n", "\n", "\n", "2. $A (\\alpha x) = \\alpha A x$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrix multiplication using NumPy\n", "\n", "We can easily work out that for any $2 \\times 2$ matrix $A$ and\n", "\n", "$$\\boldsymbol{x} = \\begin{pmatrix}\n", " 1\\\\\n", " 1\n", "\\end{pmatrix}$$ \n", "\n", "Notice that in this case $A\\boldsymbol{x}$ will just be the vector which is the summation of the columns of $A$. \n", "\n", "[We'll come back to this idea of interpreting matrix vector multiplication as a weighted sums of the columns of the matrix shortly.]\n", "\n", "Let's test this using NumPy." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[ 2 3]\n", " [ 1 -4]]\n", "\n", " [[ 2 3]\n", " [ 1 -4]]\n" ] } ], "source": [ "A = np.array([[2, 3], [1, -4]])\n", "x = np.array([1, 1])\n", "\n", "print(A)\n", "print('\\n',A*x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Not** what we were expecting. \n", "\n", "The `*` operator is equivalent to `np.multiply` , \n", "\n", "this is the same as the elementwise product in Matlab: `.*`. \n", "\n", "We want the `@` operator:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[ 2 3]\n", " [ 1 -4]]\n", "\n", " [ 5 -3]\n" ] } ], "source": [ "A = np.array([[2, 3], [1, -4]])\n", "x = np.array([1, 1])\n", "\n", "print(A)\n", "print('\\n',A@x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the `@` operator was introduced relatively recently. If you look at older code you may see the use of \n", "`numpy.dot` or `numpy.matmul` " ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[ 5 -3]\n", "\n", " [ 5 -3]\n" ] } ], "source": [ "A = np.array([[2, 3], [1, -4]])\n", "x = np.array([1, 1])\n", "\n", "print(np.dot(A,x)) # or print('\\n',A.dot(x))\n", "print('\\n',np.matmul(A,x)) # or print('\\n',A.matmul(x))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Testing our code against Numpy\n", "\n", "OK, so now we know how to do matrix-vector multiplication properly with NumPy, let's use it to check our code." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[ 5. -3.]\n", "mat_vec_product(A, x) == A@x: True\n", "[ 5. -3. 2.]\n", "mat_vec_product(A, x) == A@x: True\n" ] } ], "source": [ "A = np.array([[2, 3], [1, -4]])\n", "x = np.array([1, 1])\n", "\n", "print(mat_vec_product(A,x))\n", "print('mat_vec_product(A, x) == A@x: ', np.allclose(mat_vec_product(A, x), A@x))\n", "# numpy.allclose is a convenient function to check that two arrays are essentially \n", "# (i.e. to round off) equivalent\n", "\n", "# Check a non-square matrix\n", "A = np.array([[2, 3], [1, -4], [1,1]])\n", "# based on the above weighted sum of columns, what should Ax be for the following x?\n", "x = np.array([1, 1])\n", "\n", "print(mat_vec_product(A,x))\n", "print('mat_vec_product(A, x) == A@x: ', np.allclose(mat_vec_product(A, x), A@x))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## A valuable geometrical interpretation of matrix-vector multiplication\n", "\n", "If you think about it, the resultant vector $\\boldsymbol{b}$ in \n", "\n", "$$ \\boldsymbol{b} = A\\boldsymbol{x}$$\n", "\n", "\n", "is the weighted sum (i.e. a linear combination) of the columns of $A$, where the weighting of each column comes from the entries of $\\boldsymbol{x}$:\n", "\n", "\\begin{align}\n", "\\begin{pmatrix}\n", " b_1\\\\\n", " b_2\\\\\n", " \\vdots\\\\\n", " b_m\n", "\\end{pmatrix}\n", "&=\n", "\\begin{pmatrix}\n", " & & & \\\\ \n", " & & & \\\\ \n", " \\boldsymbol{a}_{\\,:1} & \\boldsymbol{a}_{\\,:2} & \\ldots & \\boldsymbol{a}_{\\,:n} \\\\\n", " & & & \\\\\n", " & & & \\\\\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", " x_1\\\\\n", " x_2\\\\\n", " \\vdots\\\\\n", " x_n\n", "\\end{pmatrix}\\\\\n", "&=\n", "x_1 \n", "\\begin{pmatrix}\n", " \\\\\n", " \\\\\n", " \\boldsymbol{a}_{\\,:1} \\\\\n", " \\\\\n", " ~ \n", "\\end{pmatrix}\n", "+\n", "x_2 \n", "\\begin{pmatrix}\n", " \\\\\n", " \\\\\n", " \\boldsymbol{a}_{\\,:2} \\\\\n", " \\\\\n", " ~ \n", "\\end{pmatrix}\n", "+ \\cdots +\n", "x_n\n", "\\begin{pmatrix}\n", " \\\\\n", " \\\\\n", " \\boldsymbol{a}_{\\,:n} \\\\\n", " \\\\\n", " ~ \n", "\\end{pmatrix}\n", "\\end{align}\n", "\n", "where we have used the notation $\\boldsymbol{a}_{\\,:j}$ to indicate the $j$-th column of matrix $A$.\n", "\n", "If you find it easier to think about things geometrically, then you will see later that this is a very useful way to think about what's going on when you multiply a matrix by a vector.\n", "\n", "
\n", "\n", "A homework exercise asks you to code up a version of matrix-vector multiplication based on this mathematical interpretation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll come back to this interpretation in later lectures, but for now consider briefly a $2\\times 2$ case (and so $A$ maps vectors or points in $\\mathbb{R}^2$ to $\\mathbb{R}^2$).\n", "\n", "What properties of the columns of $A$ do we require so that we know in advance that for *any* $\\boldsymbol{b}$ we choose or are given we can always find a corresponding solution $\\boldsymbol{x}$?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Some inversion (parameter estimation) examples\n", "\n", "## A linear model (polynomial interpolation)\n", "\n", "
\n", "\n", "We have seen a useful example in previous modules.\n", "\n", "
\n", "\n", "Suppose that our \"model\" is given by the polynomial (of degree $N$) function\n", "\n", "$$f(x; \\boldsymbol{a} ) = a_0 + a_1 x + a_2 x^2 + a_3 x^3 + \\ldots a_N x^N$$\n", "\n", "
\n", "\n", "For a given, fixed vector $\\boldsymbol{a}$, $f$ can be thought of as taking in a scalar $x$ and returning a scalar $y$.\n", "\n", "
\n", "\n", "The vector of $N+1$ parameters $\\boldsymbol{a}$ fully describes our model, and in this case our model is linear in these parameters (and hence the process of finding the parameters is sometimes termed [linear regression](https://en.wikipedia.org/wiki/Linear_regression)). \n", "\n", "We will see that we end up with a linear problem to solve, even though our model is nonlinear in the parameter(s) $x$ we do not invert for. \n", "\n", "\n", "
\n", "\n", "You can think of $x$ as locations we evaluate our model at, while $\\boldsymbol{a}$ are unknown input parameters to our model that describe in some sense the governing \"physics\".\n", "\n", "\n", "For the inversion problem we are assuming we know the $x$'s (e.g. maybe these are the locations our data is collected at) and the $y$'s (the observations of the output of our problem/model at these locations), and we want to find $\\boldsymbol{a}$ (i.e. the parameters that govern our model).\n", "\n", "\n", "We described this problem as polynomial interpolation in *Numerical Methods*, and as long as we are given $N+1$ data points $(x_i, y_i)$ (with distinct $x_i$'s) we can solve it for $\\boldsymbol{a}$ and thus fix/fully define our \"model\"." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Problems leading to a square linear/matrix system\n", "\n", "First of all let's consider the case where we match the number of model parameters to the number of data points we have.\n", "\n", "
\n", "\n", "### Fitting a single data point\n", "\n", "The polynomial that fits the single data point $(x_0,y_0)$ is clearly simply the *constant* function given by\n", "\n", "$$ y = f(x; \\boldsymbol{a} ) \\equiv a_0 $$\n", "\n", "where through subsitution of the data we have\n", "\n", "$$ a_0 = f(x_0; \\boldsymbol{a} ) = y_0 $$\n", "\n", "So this case is completely trivial.\n", "\n", "
\n", "\n", "### Fitting two data points\n", "\n", "The polynomial that fits the two data points $\\{(x_0,y_0),(x_1,y_1)\\}$ is clearly the *linear* function given by\n", "\n", "$$ y = f(x; \\boldsymbol{a} ) \\equiv a_0 + a_1\\,x$$\n", "\n", "where through substitution of our data into the functional form we arrive at two simultaneous equations for two unknown parameters (or a $2\\times 2$ matrix system)\n", "\n", "\\begin{align*}\n", "(1) & \\;\\;\\;\\; y_0 = a_0 + a_1\\,x_0, \\\\[5pt]\n", "(2) & \\;\\;\\;\\; y_1 = a_0 + a_1\\,x_1. \n", "\\end{align*}\n", "\n", "
\n", "\n", "[A homework exercise asks you to solve this problem via substitution and show that the solution is\n", "\n", "$$ a_0 = y_0 - \\frac{y_1-y_0}{x_1-x_0}x_0, \\;\\;\\;\\;\\;\\;\\;\\; a_1 = \\frac{y_1-y_0}{x_1-x_0}. $$\n", "\n", "]\n", "\n", "\n", "\n", "However, the substitution approach isn't easy to do with large amounts of data/large dimensional $\\boldsymbol{a}$.\n", "\n", "\n", "Instead we can note that inverting for our $\\boldsymbol{a}$ parameters in this case is equivalent to forming and solving the linear system\n", "\n", "$$\n", "\\begin{pmatrix}\n", "1 & x_0 \\\\\n", "1 & x_1 \n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "a_0\\\\\n", "a_1\n", "\\end{pmatrix}\n", "=\n", "\\begin{pmatrix}\n", "y_0\\\\\n", "y_1\n", "\\end{pmatrix}.\n", "$$\n", "\n", "
\n", "\n", "Let's implement and plot some examples to check that this is correct visually in these cases on one and two parameters." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtwAAAE5CAYAAACjyPfFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABG80lEQVR4nO3dd5gUVdbH8e8BJAoKklZQUAmKu4Oyiu4qMphWXF0JBmAMmDBhWMMaMKCAuAZWXSOioq+oiAIqKophEMyoiAsIggQJiohKjnPfP27N2NP0zPQM010dfp/nmQe6uqr63Aqnb9+6t8qcc4iIiIiISGJUCTsAEREREZFMpgq3iIiIiEgCqcItIiIiIpJAqnCLiIiIiCSQKtwiIiIiIgmkCreIiIiISAKlVYXbzPqamYv4W2NmX5lZfzOrVs515ZtZfoJCTQozaxlsh74VWDbty58IZtbNzK5M8mdWeD+m0mcEn7PQzEZWYLmkb/fyMrPcYBvmVmDZlC9fqlG+L075vvIl4rwM8sRAM0vJ+lXEedWynMvtGpSrQ4JCqxQVPdaTUb6UPCDicArwF6An8CnwX+DmUCMKx3L8dngt7EAySDcg2RUj7cdwtnt5fYHfT19UYNlupH75UpXyvac8Ufm6UfnnZS5wC+lbvyrJrvhypXSFG7g4+CuvXUlw+crVSpBCpjvn5gX/f8vMWgFXkGVJ2Dm3Cfg47DiSxcxqBGXOKNm2H9OVc2412k9hUL4n+/JEpuZ7SSzn3KywYyhJpvwC+wyoa2aNAczsODP7yMw2mNlvZjbezNqWtLCZNTWzzWZ2eYz3BprZejOrH7zON7OpZna0mX0RvPc/M+sWY9ky44hY33FmNj2Y90szO8TMqpnZ7Wa23MxWmdlIM6sTsex2lxjN7GAze9HMlgTrmhOso1YFtitmdmtQzt/MbKWZvWtmh0bNU3ipvWcQ4y9mttrMRpnZblHzOjMbYmYDImJ838wOKGG7nBhsj00Ev1rNrKOZvW1ma81snZm9Y2YdI5YdHOzPgyOm1Qm2xUclXY423xXiLKBZxGXshRHvtzWzcWb2axD3x2Z2XBzbsKmZPWVmy8xsU7A/J0Qcr7H248hg+xxoZlOC4+xbM7swxvqPDrbRRjObZ2bnBcsvjJ43xrKdg+23JtiWb5rZH8taLlj2cvNdSDaa2TQz6xRjnkZm9qiZzQ3K8L2ZPWtmzSLLSgnb3cxqmtl/zJ9ja83sBzN71cz2jSO+8hyX9czsgYh9NMfM/mlmFmN9uRHTyswHZR1XUm7K979PU75PoXxvZgPxraQAWwrXG7z3PzMbETHvLma2zcyWRK3jAzN7IeJ1mbmplHj2NrPXguP2JzO7D6gRY75ewb7+KdjOX5rZWRHvtwQWBC8fi9hefYP3jzWz14Njt/AcucrMqsYRY+G+PylYbpOZfWNmp8aYN95zLD/ideHx+o9gO64MyvmMme0aZ/n+ZmYfBp+5NtgH5f/B75xLmz+gL+CAVlHTxwBbgdrAccA2YBLwD6APMA/4CWgWsUw+kB/x+gVgVtR6qwLfAyOjllsOzARODz5vUvD5rSLmK08cPwBfA72AE4BZwI/AY8CTwN/wLTpbgDsjlm0ZbI++EdN6AjcG6+mMT1o/AM9Hla1Y+UvZ5iOAM4AuwTqfBzYDORHz5AZxfB/EexxwKbAGeC9qfYXzfYC/nHcaMAf4GWgQFd8K/ElwTvAZOcHfBuBz4OSgvJ8F09oHy1YDPgS+BXYOpo0EfgP2KqWs++Av164ADg3+Dgze2z3Yd98F+/1EYGKwj7uWsQ0nAXOBPOAI/CXyR4CWpezHkcBqYDZwAXAM8GwwX5eI+doBm4ApwfY8FX8sLQYWlnGs/B1/3L4MnBT8fQj8AuxRRpnODdZXuL/7A0uCbRx5vrQF7gv20xH4Y/wzYCFQM47tvgv+GOyFP567B9vzV6BpGTHmEsdxiW94mAKsA64Cjg1idsDtMdaXW558UFr59Kd8j/J9YXxpn++B5sE2dMBhhesN3rsf+C5i3pOA9cG8bYJpdYLtfVHwOq7cVEIs1YH5wDLgbHy+fyXYH47g+yeY94bg2DkWOBq4DX/8XRi8XwOfex1we8T2ahS8f2EQX9fg2LkmOB7uiOOYy8cfs4si4pwAFFD8u66i53puEPcCfHe0Y/HH6wbgqbLKB+yN/44dFcRwJP47+d/lzmlhJdOK/PF7Am6LP8nqBwXfBowP5pmGP/GqRSy3V3DwDItjp3SKmPaPYNqhUcttAVpHTGscxHBDxLTyxLEF2DvG574dVf6xwIKI1y2JSsBR81uwnU4PDt7dSip/nNu/arC+OcB9MbbdxKj584LpR0VMc8BKoE5UObYAg6LiKwAOiFrni/jK1q4R0+oBq4CxUev8FXgK6B18bp84yjgSWBJj+t1s/yVbNdgWX5SxzrXAZaW8v91+DOKIrlzXCLbd8Ihpz+ITTu2IaX8ANlJ2hXse8E5ULPWCz7i3lHir4JN29P4+LfiMkWUcQ3sE83Uva7uXsHxtfDL/ZxnzxnVc4isW251H+C/OTUDDqPXlxjh/y8oHcZVPf8W2f1+U7xdEvG4Z6ziNeF/5PjXy/cDg86tFTS+s0LUIXt+LrwB/C1wQTDsumGff4HVcuamEOM5n++O5Cv7HoyOiwh21XJVgvz8GfBXj+DuvjPIXHocD8I03VcqYPz9GnFWBb4ApFTzHYp3rT0V97gP470krrXz4H3oOqFee8yfWX7p2KfkGv5FXAQ/hf3mcY/7yWwdgtHNua+HMzrkF+F/YnUtaoXMuH9/ScEHE5AuAGc656H5z3zrnvo1YdgX+V/Ke4C9nlTOOuc6576LKB/BmjHI3L+1SUnD56d9mNh9/Qm4B/g9/ErQuablS1ne0mb1nZj/jE9AWoA3+SzDaC1Gvx+CT6F+ipr/unFtX+MI5txDfNzF6voXOuelR044AJjjnfo1YfjU+cXWOmLYQ/6v7THwrzNPOuWdLKmccjgA+dr/3JcU5tw14DjjAzOqVsuxnwDXmu2D8KZ5LgYH1zrn3Ij5vEz7h7Bkxz6H47bk+Yr7l+BafEplZa3wLzyjzl7KrBZde1wMfBeUtSfPgL3p/v4Q/RqI/6yLzd5dYG7y/OHirxMv+UcufamafmNmvwfLrgJ3jXT5GnNHH5RHB6+ei5nsG30oUfVxGKzUfyA5Tvi+B8n2xdaZKvi/JZPz2OTJ4fSTwbvAXOW25c67wmNiR3PQX4PvI49k5V8D2+w0za21mz5nZUvw+3wKcR/w5+g/muw4uwrfQbwEG4wciNo5jFdFxbsMfTx3NrMqOnOsRogcbf41vxGpSxnLT8eV53sxOtqArW0Wka4W7O3AwsC/+l/OZzrlV+BYQw18CjPYD0KCM9T4MnGxmu5lZC/yvzUdizLcqxrRNQM3g/+WN45eo15tLmV4N/+uvJE/iE8/9+G4IBwOXBO/VLGmhWMzfHud1fAvtufjK3cHAVyWs68fIF865zUEZmpU2X8S06Plibb8GJUz/Ab/dI72Gv3RZA/hPjGXKo7TPtRifHek0/BfEv4AZwFIzu9nKvm1U9P6H4scZ+NbsFTHmi7WNIxUmjcf5PcEW/p0A7FbCcoWfud1nBInw58hpZnYpvpL0NtAD6Ig/jiCO49HMTgRG47vW9AEOwR+DP8WzfAlxRh+XDYBVbvsBWj9EvF+asvKB7Bjl+5Ip3/8uVfJ9TMEx+xXQxcwaAn8E3gv+coPZugSvI+OoaG76AyVv+yJmtjO+m0Z74DqgE36/P0GM/t7Rgu+xV/DfG4PxPxoOBoYEs8RzHJYUZ3V8t44dPddh+/O4cJuWGl/wo+tv+Pry/wE/BA1A8VTyi0nXu5T8L/KXZ4Rf8E3/TWO815SoykAMTwND8Zcy6+P7+IyqQHw7GkeFmFlNfL+wgc65+yKm/6mCq+yJb+Xo4ZzbErG++vjLd9GK/VI0s+r47bi0tPkipkXP52LMt4qSt2v0CfUg/stqPjDczA6LLEc5lfa5LsZnFwlaxC4BLgkGeJwF3IqvND5cwXgKLSd2C0JZv9oLj8Hr8ZXhaJtjTIv8zO0+I2ghj66o98J3W7kqYr69yogtevl5zrm+EcvvRHwJtlBZx+UqoIGZVQ8qDYUK93dCzleJm/J9DMr3qZnvy/AevgGmC/64mEGQw83sMOBA4NGoOCqam5YD+8eYHr0//gK0wHevmlo40eK/1/0+wEHAGc65ZyKWPzHO5WPFVDhtM/57shYhnGOFgivN75lZDXzf/NuA18yspXNuZbzrSdcW7piCy1afA6dEjo4NWi/+ir+kU9ryq/EJ9wL8wI1ng2lJjWMH1MAnnOgk07eC66uN76tYlAjN7EhKvlQePar4FPwx9lHU9OMtavQ9vjUler5YJgN/N7O6EcvXxQ9qmRwxrQ9+8E8/fII7AH+SlGUT/uSO9bmHWsTDAoJ9exrwpXNuTRzrxjk3xzl3A/5LOq67gZThY/z2rB0R1x/wSaE0c/ADF/d3zk2L8TejlGWX4PtwR+/vnmz/I7422x+PZ8dYZ0nbvTbbd1M5g9Jb/aKVdVxODl6fEjVfHj7hR3cxqIiSyicVpHyvfB8xLZXyfWHLaaz1vodv2b8A38/YBQ0yM/GNMFXxXUwi46hobvoI2MMi7jITtEZH77fC747oH1knxVmuWMvvFMQYr+g4q+LL/KlzriBJ51hp+w3w3Tqdc+8Cd+IHuJan8ShtW7hLcxP+0tIEM3sI39fzVvyI5XviWP4hfu/XF+vyYrLiKDfn3G9m9jFwlZktxw9WOYftL93FayJ+tPxIM3sS35fvJrZvmSi0fzDf88G8Q4DJzrl3oubbgL+f7l34L41b8XfkiOcy4CD8pat3zOzf+C+Ha/En/W1Q1IL6MPC4c25MMG0AcIeZvRXZLzqGWfgWhYvwgzQ2Oue+DmLrC0wys1uCeC8Oyvn3klZmZrvgW5BH8Xtf1JPwLUFvxVHesgzGD+p408zuxm/Pm/CX4wpKWsg558zsEuDloGXqBfzx0gSfwBY754aVsGyBmd0KjIjY363wreXRFZaJwLVmdgP+oSVHBvFGK2m7TwS6mdl/8CPX/wxcRuwWt5KUdVy+AUwFHjGzRvgvv+PxfRiHlqcFoxQllU92jPK98n3K5PuIdYLfL28A25xz04Jp7+N/1BzF711/wFfE++PzbmT//h3JTU/hu4iMDfLvCnz3o+j+5x8G5XswKGsd/J1vVuLvElXoR3xLci8zm4EfS7MA391vETDEzLbhv+P+WUpcsfwIjA4+/yfgIvy2vihinkSfYyWV7xR8X/rX8Q1NDfHfdcuA/5XrE9wOjrpM5h8l3CYqxnzH4X/dbcDvjJeBtlHz5FPCqG18699nJbyXD0yNMX0hUXdnKEccU6OmtST2aNmBRIx+JvadJ1riT9I1+BPsAXyCcGx/d4WY5Y/6zEuDg24DfvDf0dHL8vso4B74Ud+/Bp//LFGjqIP5huBvQ7QEP0p4CtuPTo+5nYP3DsFXYtfiT4p3gI7Be9WCbT6H4iPjDV/BXULE6P0Y666DH6BSeJl4YcR7bYHxwb7ciG9dOK6M7VcDf4lwZhDv6mA79omYJ9Z+HEns0fPb7Td8383p+F/o3+ErEOPwLTElfkYw/S/4iuwvQZkW4r9A/xLHsXE5PtFuxH9ZHU7UeYBvLXgYn0TXBJ+1VxDLwLK2O751ZzA+ua3Ht2QcGP05JcSXS/zHZT38ubIc33I0F/+lYTHWF30elZkPSjuu9Ffi/uuL8r3yfRrl+2C5qvjuLSvwjR4u6v1PiLgTSTCt8A4mI2Osr8zcVEose+MriuvxOfg+/PeDo/htAY8Evgz2+3x8o8bAGLF3w/+g2BJ5LOKvKEwNPmcJ/sfQedGfU9o5hr9bz//w32NzgNMq41zn9+P16BLyS8vSyof/jnwZX9neFOyHMdGfG89f4e1QJGBmbfAtkec75x4PO550YGa5+F/oxzjnYvUHjpzXAUOcczcmIbSsFAyCmQe85pw7N+x4wlKe41Kyk/J9+SnfS2Uy/5Caas65w8OOJdEysUtJhZhZc/xl8Vvxv2B25JZCIkljZv/FXxZchn9gw+X4Liv3lbacSLZSvheRZEv6oEkze8LMVphZzL4v5t1v/hHVM8zfqigZzsMPVmiCv9y/IUmfK7KjagL/xl9CHY6/7Hq0K33go0ilSuHcHovyvYgkVdK7lJjZEfi+WE8757a7S4OZHY/vR3Y8vu/Wfc65Q5IapIiIlItyu4hIyZLewu2ce5/S72F5Ej5hO+efPLRrcJszERFJUcrtIiIlS8U+3M3wo0ELLQmmbfeEITPrh7/vJjVr1vzznntmx5OUCwoKqFIlo26hHlO2lBNU1kw0d+7clc65RmHHkULiyu3K65kvW8qaLeWEssu6dm01li2rRb16W2jadGMSI6t8Fc3tqVjhthjTYvZ7cc4Nx/dZpW3btm7OnDmJjCtl5Ofnk5ubG3YYCZct5QSVNROZ2aKwY0gxceV25fXMly1lzZZyQullnT0bOnaEgw+G99+HmvE87D2FVTS3p+JPryXAHhGvm+PvviAiIulLuV0ky/z2G3TrBrVrw0svpX9le0ekYoX7FeDMYET7ocBvzrntupOIiEhaUW4XySIFBXDGGfDddzBmDOyxR9nLZLKkdykxs+fwT/5paGZLgFuAnQCcc4/gn4p0PP7BHeuBs5Mdo4iIlI9yu4hEGjwYXn0V7r8fjjgi7GjCl/QKt3OudxnvO+CSJIUjIiKVQLldRApNmAC33AJnngn9+4cdTWpIxS4lIiIiIpKG5s6FvDzo0AEeeQQs1nDpLJSKdykRyRqrV69mxYoV7LLLLsyePTvscJIiU8pap04dmjdvnjW3/UpFhefPli1bwg5lh2XKeRGPevXqsXr1aurVqxd2KFLJVq/2gySrV4exY6FWrbAjSh2qcIuEZPXq1fz44480a9aMrVu3Zs2Xz5o1a6hbt27YYeyQgoICli5dysqVK2ncuHHY4WSlyPOnVq1aWJo3o2XCeREP5xw///wzP/74I0DW5L1sUFAAffv6Fu5Jk6BFi7AjSi1qmhEJyYoVK2jWrBm1a9dO+8pCtqlSpQpNmjTht99+CzuUrKXzJz2ZGTVq1KBZs2asWLEi7HCkEg0dCuPGwV13QZcuYUeTelThFgnJli1bqKXrbWlrp512YuvWrWGHkbV0/qS3WrVqZURXIPE++aQBN90EvXvDFVeEHU1qUoVbJERqmUtf2nfh0z5IX9p3mWPePBg8eD9ycmDECA2SLIkq3CIiIiJSbmvXQvfuUKWK705Su3bYEaUuVbhFREREpFycg3POgVmz4KabZrHXXmFHlNpU4RaRtPXiiy/q0rRIBen8kR1x113+ke1Dh8JBB/0SdjgpTxVuEckqLVu25O677w47DJG0pPNHAN56C66/Hk49Fa65Juxo0oMq3CJpbtSoUbRs2ZIqVarQsmVLRo0aldTP37x5c1I/T6Qy6fwRKZ8FC/zdSNq1g8cf1yDJeKnCLZLGRo0aRb9+/Vi0aBHOORYtWkS/fv0SWmnIzc3loosu4uqrr6ZRo0YcdthhAAwbNoycnBzq1KlDs2bNOO+88/j111+LlmvatCmjR48uen3YYYdRt27dolvrffvtt5gZS5cuLfGzn376aVq0aEHt2rU54YQTih6eUWj+/PmcdNJJNG3alDp16tChQwcmTJhQLPZFixZxzTXXYGZFl9N//vlnevfuTfPmzalVqxb7778/Tz755A5vK0ltqXT+PPDAAzp/JOWtX+8HSRYU+EGSO+8cdkTpQxVukTQ2YMAA1q9fX2za+vXrGTBgQEI/95lnnsE5x5QpU3j66acB/zCYe++9l5kzZ/Lss8/y6aefcumllxYt07lzZ957772iGKdNm0aNGjWYNm0aAPn5+bRq1YpmzZrF/MxPPvmEvn370q9fP6ZPn86JJ57IzTffXGyetWvX0rVrVyZNmsRXX31Fz5496dGjB9988w0AY8eOpXnz5tx8880sX76c5cuXA7Bx48aiysXMmTO5/PLLueCCC3jnnXcqd8NJSkml88fMdP5ISnMOzj8fZsyAZ5+FVq3CjijNOOcy4q9NmzYuW7z33nthh5AUmV7OWbNmFf1/9erVFVqHmTlguz8zq6wwt9O5c2f3pz/9qcz53njjDVe9enW3bds255xzDz30kGvTpo1bvXq1e+utt9y+++7rzjzzTHf77bc755zr06ePO++880pcX+/evd3RRx9dbNq5557rfBor2SGHHOIGDRpU9LpFixburrvuKjP+0047zZ177rmlzhO5D6MB01wK5MZ0/istr5e27eOVSudPdA4o6fxxzqX9+VNY1srYh6ks077Dhg1zDpwbMmT79zKtrKWpaG5XC7dIGttzzz3LNb2y/PnPf95u2rvvvssxxxxD8+bNqVu3Lj169GDz5s388MMPgL8cPXfuXJYvX05+fj5dunQhNzeX/Px8ACZPnkxubm6Jnzl79mz+8pe/FJsW/XrdunX861//ol27dtSvX5+dd96ZadOmsXjx4lLLs23bNoYMGUJOTg677bYbO++8M2PHji1zOUlvqXT+TJ48Oa7zZ9myZTp/JOnee88PjuzRww+WlPJThVskjQ0ZMoTaUU8aqF27NkOGDEno59apU6fY60WLFvH3v/+d/fbbjzFjxvD555/zxBNPAL8PCttvv/1o0qQJU6ZMKaowdOnShQ8++IBZs2axdOnSUisMvmGhdFdffTVjxoxh0KBBTJ48menTp9OxY8cyB6bdfffd3HPPPVxzzTW88847TJ8+nW7dumlAW4ZLpfPnlFNOiev8yc/P1/kjSbV4sb8bSZs2MHKkBklWVLWwAxCRisvLywN8X9TFixez5557MmTIkKLpyTJt2jQ2b97Mf/7zH6pWrQpQbLBVoc6dO/Pmm28ybdo0OnfuTOPGjWnYsCF33nlnqf1PAdq1a8fHH39cbFr066lTp3LmmWfSs2dPwPctnT9/Pm3atCmap3r16mzbtm275U488UTOOOMMwFdO5s6dy6677hr/RpC0k47nz2uvvabzR5Jmwwbfqr15M4wfD3Xrhh1R+lILt0iay8vLY+HChRQUFLBw4cKkVxYAWrduTUFBAffeey8LFizgueee4957791uvtzcXMaOHUvr1q1p3Lgx4CsRzzzzTKmtcwCXXXYZb7/9NkOHDuXbb7/lscceY9y4ccXmadOmDePGjeOLL77g66+/5vTTT2fjxo3F5mnZsiVTpkxh6dKlrFy5smi5d955h6lTp/LNN9/Qv39/FixYUPENImkj3c6f0aNH6/yRpHAOLroIPv8cRo3yLdxScapwi8gOy8nJ4b777mPYsGG0a9eOESNGxHw4RpcuXdi2bVuxykGsabEceuihPP744zz88MPk5OQwduxYBg4cWGyeYcOG0bhxYzp16kTXrl059NBD6dSpU7F5brvtNr7//nv22WcfGjVqBMCNN95Ix44d6dq1K0cccQR16tQJpeIl2SknJ4d///vfOn8kpTz4IDz1FNx6K5xwQtjRpD+Lp19XOmjbtq2bM2dO2GEkRX5+fpnJNRNkejlnz57NfvvtB8CaNWuomyXX6jKprJH7MJqZfe6cOyjJIWWU0vJ6ads+HWXSeVGWwrJm2j6Mls7fYe+/D0cdBccf7++3XaWM5tl0Lmt5VTS3q4VbRERERABYsgROOQX23huefrrsyrbER4MmRURERISNG6FnT/9Eyfx82GWXsCPKHKpwi4iIiGQ55+CSS+DTT2HsWMjg3j6h0IUCERERkSz36KPwxBMwYAB07x52NJlHFW4RERGRLPbBB3DZZX6Q5K23hh1NZlKFW0RERCRLLVsGJ58MLVr4+20Hz16SSqY+3CIiIiJZaPNmX9leswYmTQI9HDRxVOEWERERyUKXXw4ffQRjxsAf/xh2NJlNXUpEREREssyIEfDII3Dddb6VWxJLFW4REZEU0bJly5iPdRepTJ984m8BeOyxMHhw2NFkB1W4RaRCfvzxRy6//HL22WcfatSoQbNmzejatSuvv/56Uj4/Pz8fM2PlypVptW6Rvn37csIJJ8R877PPPuPiiy9OckSSTX74wT/cplkzeO45DZJMFvXhFpFyW7hwIYcddhh169Zl6NChtG/fnoKCAt555x0uvPBCFi9eHHaIImmpUaNGYYcAwObNm6levXrYYUgl27zZP7Z91Srfd7tBg7Ajyh5q4RaRcrv44otxzjFt2jROPfVU2rZty3777Uf//v356quvAFi8eDHdu3enbt261K1blx49erBkyZKidQwcOJA//vGPPP/88+yzzz7UrVuXbt26FWtV/vrrrznqqKOoV68edevWpX379rz33nssXLiQLl26AL6CYmb07dsXgIkTJ9KpUyfq169PgwYN+Nvf/sbs2bOL1rlw4ULMjJdeeoljjjmG2rVr065dOyZNmlT0fknrFkm06C4lZsbw4cM55ZRTqFOnDnvvvTfPPPNMsWWWLl1Kr169qF+/PvXr1+fvf/873377bdH78+fP56STTqJp06bUqVOHDh06MGHChO0+d+DAgZxzzjnsuuuu5OXlJbagEoorr4SpU+Hxx6F9+7CjyS5q4RZJIVdcAdOnJ/czDzgA7r03/vlXrVrFxIkTGTx4MDvvvPN279evXx/nHN26daNmzZq8++67mBn9+/enW7duvPvuu0XzLly4kNGjRzNu3DjWrVtHr169GDBgAI8++igAffr0oX379nz66adUq1aNr7/+mpo1a7LHHnvw0ksv0bNnT2bOnEmDBg2oVasWAOvWreOKK64gJyeHDRs2MHjwYE488URmzZpVrMVuwIAB3HXXXTz00EMMHjyYXr16sWjRolLXLaktHc6firjtttu44447GDp0KI8//jjnnHMOnTp1okWLFqxfv54uXbrw17/+lcmTJ1O9enXuvvtujj76aGbPnk3t2rVZu3YtXbt2ZfDgwdSqVYvRo0fTo0cPZsyYwb777lv0OcOGDePGG29k2rRpOOcSWyhJupEj4cEH4aqroHfvsKPJPqpwi0i5zJs3D+cc++23X4nzvP3223z11VfMnz+fli1bAvDss8/SqlUr8vPz+cc//gHA1q1bGTlyJLvssgsA/fr148knnyxaz6JFi7j66quLKgWtWrUqeq9BcC20cePGNGzYsGh6z549i8Xy5JNPUq9ePT799FMOP/zwoun//Oc/OfHEEwG4/fbbefrpp5k+fTqHH354iesWCcMZZ5zB6aefDsCgQYO47777mDJlCi1atOD555/HOceTTz6JmQHw6KOP0rhxYyZMmMCpp55K+/btaR/RnDlgwABeffVVXnzxRW688cai6Z07d+Zf//pXcgsnSTFtGlx4IRx5JNxxR9jRZCdVuEVSSKJbyipDPC1fs2fPZvfddy+qbAPsvffe7L777nzzzTdFFe4WLVoUVbYBdt99d1asWFH0+sorr+S8887jqaee4qijjqJnz57FWuRimT9/PjfddBOffPIJP/30EwUFBRQUFGzXrzwnJ6fY5wLFPlvSTzqcPxUReaxWq1aNRo0aFR2rn3/+OQsWLKBu3brFllm/fj3z588H/FWfW2+9lQkTJrB8+XK2bNnCxo0bi60X4KCDDkpwSSQMK1ZAjx7QtCmMHg3VVPMLhfpwi0i5tG7dGjMr1i86mnOuqLUtWuT0nXbaabv3CgoKil4PHDiQWbNm0a1bNz788ENycnJ44oknSo3vxBNP5KeffuLRRx/lk08+4csvv6RatWps3ry52HyRn10YU+Rni6SK0s6TgoICDjjgAKZPn17sb+7cuVxwwQUAXH311YwZM4ZBgwYxefJkpk+fTseOHbc7J+rUqZOcAknSbN0Kp50GP/0EY8eCLtiFRxVuESmXwoGIDzzwAGvXrt3u/V9//ZV27dqxdOlSFi5cWDT9u+++Y9myZWW2UEdr3bo1l112Ga+99hrnnnsuI0aMACjqj71t27aieX/++Wdmz57NDTfcwNFHH81+++3HmjVr2Lp1a7k+M9a6RVJRhw4dmDdvHg0bNqRVq1bF/gq7Rk2dOpUzzzyTnj17kpOTQ/PmzYtavyWzXXMN5OfD8OHQoUPY0WQ3VbhFpNweeughnHMcdNBBjBkzhjlz5vDNN9/w8MMPk5OTw9FHH0379u3Jy8vj888/Z9q0aeTl5dGhQwc6d+4c12ds2LCBSy65hPz8fBYuXMgnn3zC1KlTadeuHeC7o5gZr732Gj/99BNr166lfv36NGzYkMcee4x58+YxefJkLrzwQqqV8xpqrHWLVKbVq1cXa5GeMWNGsR+o8crLy6NJkyacdNJJTJ48mQULFvD+++9z1VVXFd2ppE2bNowbN44vvviCr7/+mtNPP52NGzdWcokk1TzzjO9mdfnlcMYZYUcjqnCLSLnttddefPHFFxxzzDFce+215OTkcOSRR/LKK6/w6KOPYmaMHz+eRo0akZubS5cuXWjatCnjx48vsatJtKpVq/LLL79w1lln0bZtW7p3785f/vIXhg0bBkCzZs249dZbGTBgAE2aNKF///5UqVKF0aNHM2PGDP74xz9yySWXMGjQIGrUqFGu8sVat0hlmjJlCgceeGDR3+GHH87VV19d7vXUrl2b999/n7333ptTTjmFfffdl7POOotffvmF+vXrA/7uI40bN6ZTp0507dqVQw89lE6dOlV2kSSFfPklnH8+dO4Md90VdjQCYGHc+sfMjgPuA6oCI5xzd0S9vwvwDLAnfmDn3c65J7dbUYS2bdu6OXPmJCji1JKfn09ubm7YYSRcppdz9uzZRXf6WLNmzXaDnjJVJpU1ch9GM7PPnXNZMwot2Xm9tG2fjjLpvChLYVkzbR9GS/Z32KhRoxgwYACLFq2jatUv2WWX+syeXYfGjRP/2Zn+fR2pork96S3cZlYVeBDoCrQDeptZu6jZLgFmOefaA7nAPWamR16JiKQg5XWRcI0aNYp+/fqxaNES4Dm2bWvIunXHMWnSqLBDk0AYXUo6AvOcc9855zYDzwMnRc3jgLrmrz3vDKwCyjfqSUREkkV5XSREAwYMYP369cBQ4GjgIjZtmsqAAQNCjkwKhXE3xmbA9xGvlwCHRM3zAPAKsAyoC5zmnNvufl1m1g/oB/4RzPn5+YmIN+WsXbs2K8qa6eXcZZddWLNmDeDvhlH4/0yXSWXduHFjRh+j5ZD0vB55/mSCTDovylJY1kw/f5L5HeafM3AqcA3+YtPIounJiCHTv68rQxgV7lgjpqI7kv8NmA4cCewDTDKzKc651cUWcm44MBx8X79s6T+ULX2lMr2cs2fPLuqzmY39NzNBzZo1OfDAA8MOIxUkPa9Hnj+ZIJPOi7IUljXTz59kfoc1bXosy5c/AUwF/lk0fc8990xKDJn+fV0ZwuhSsgTYI+J1c3yLR6SzgbHOmwcsAMp3816RNBDGoGWpHNp3xYSS17UP0pf2XeVZtQq2bRuD2W/AKcAWwN/BZsiQIaHGJr8Lo8L9GdDazPYKBsz0wl9mjLQYOArAzJoAbYHvkhqlSILttNNObNiwIewwpIK2bNlS7vt7Z7Ck53WdP+ltw4YN2z1BU8pv2zbo0wd++aUut9wygxYtamBmtGjRguHDh5OXlxd2iBJI+reFc26rmfUH3sTfPuoJ59xMM7sweP8RYBAw0sy+xl+qvNY5tzLZsYokUuPGjVm6dCnNmjVTa0+aKSgo4Mcff2SXXXYJO5SUEEZejzx/atWqFff93SVczjk2bdrEL7/8QpMmTcIOJ+3ddBO8+aZ/kuT55x/HLbcsDDskKUEozTPOudeB16OmPRLx/2XAscmOSySZ6tWrB8CyZctYs2YNNWvWDDmi5Ni4cWNGlLVOnTo0bNgw7DBSRrLzeuT5s2XLlspabWgy5byIx4YNG2jVqlXRPpSKeeklGDrUP+Dm/PPDjkbKouuhIiGqV68e9erVIz8/P6MHD0XKprJKYhWeP5kgm86L/Pz8jNlvYZk5E846Cw49FP7737CjkXjo0e4iIiIiaeLXX6F7d6hb17dy16gRdkQSD7Vwi4iIiKSBggI44wxYsADy82H33cOOSOKlCreIiIhIGrj1VpgwAR58EA47LOxopDzUpUREREQkxb38Mtx2G5x9Nlx0UdjRSHmpwi0iIiKSwr75xnclOeggeOgh0F0w048q3CIiIiIpavVq6NYNataEsWP9v5J+1IdbREREJAUVFMCZZ8K8efD227DHHmFHJBWlCreIiIhIChoyxPfdvu8+yM0NOxrZEepSIiIiIpJiXnsNbrnF992+9NKwo5EdpQq3iIiISAr59lvIy4MDDoBHH9UgyUygCreIiIhIilizxg+SrFYNxo2DWrXCjkgqg/pwi4iIiKQA56BvX38bwEmToEWLsCOSyqIKt4iIiEgKuOMOf+u/e+6BI48MOxqpTOpSIiIiIhKyiRNhwADo1Qv++c+wo5HKpgq3iIiISIjmz4feveFPf4IRIzRIMhOpwi0iIiISknXroHt3X8keNw7q1Ak7IkkE9eEWERERCYFzcO65MHMmvPEG7L132BFJoqjCLSIiIhKCe+6B0aNh6FA49tiwo5FEUpcSERERkSR7+2249lo4+WT/r2Q2VbhFREREkmjBAjjtNNhvP3jySQ2SzAaqcIuIiIgkyfr10KMHFBTA+PGw885hRyTJoD7cIiIiIkngHPTrB199BRMmQKtWYUckyaIKt4iIiEgS3H8/jBoFgwfD8ceHHY0kk7qUiIiIiCRYfj5cdZW/5/b114cdjSSbKtwiIiIiCbR4MZx6KrRuDSNHQhXVvrKOdrmIiIhIgmzY4AdJbtrkB0nWqxd2RBIG9eEWERERSQDn4OKL4fPP4eWXoW3bsCOSsKiFW0RERCQBHn7YdyG55Rb4xz/CjkbCpAq3iIiISCWbMWMXLr8cTjgBbr457GgkbKpwi4iIiFSipUth4MD92XtveOYZDZIUVbhFREREKs2mTX6Q5MaNVRg3DnbZJeyIJBWowi0iIiJSCZyDSy6BTz+F6677hnbtwo5IUoUq3CIiIiKVYPhwePxxuOEGOOKIlWGHIylEFW4RERGRHfTRR3DppXDccXDbbWFHI6lGFW4RERGRHbB8OfTsCXvsAc8+C1Wrhh2RpBo9+EZERESkgjZvhpNPht9+gzffhPr1w45IUpEq3CIiIiIVdMUV8OGHMHo0/OlPYUcjqUpdSkREREQq4PHH/dMk//UvOPXUsKORVKYKt4iIiEg5ffopXHwxHHMM3H572NFIqgulwm1mx5nZHDObZ2bXlTBPrplNN7OZZjY52TGKiEj8lNclm/z4o3+4ze67w3PPaZCklC3pfbjNrCrwIHAMsAT4zMxecc7NiphnV+Ah4Djn3GIza5zsOEVEJD7K65JNtmyBU06BVat83+3ddgs7IkkHYbRwdwTmOee+c85tBp4HToqapw8w1jm3GMA5tyLJMYqISPyU1yVrXHUVTJkCI0bAAQeEHY2kizAq3M2A7yNeLwmmRWoD1DezfDP73MzOTFp0IiJSXsrrkhWefhr++1+48kro0yfsaCSdhHFbQIsxzUW9rgb8GTgKqAV8ZGYfO+fmFluRWT+gH0CjRo3Iz8+v/GhT0Nq1a7OirNlSTlBZJe0pr++gbDov0rWsc+bszKWXduDAA3/j+ONnkJ8ffYgXl67lrIhsKmtFhVHhXgLsEfG6ObAsxjwrnXPrgHVm9j7QHiiWmJ1zw4HhAG3btnW5ubmJijml5Ofnkw1lzZZygsoqaU95fQdl03mRjmX96Sc46yz4wx/gzTfr06hR5zKXScdyVlQ2lbWiwuhS8hnQ2sz2MrPqQC/glah5XgY6mVk1M6sNHALMTnKcIiISH+V1yVhbt8Jpp8GKFTBuHDRqFHZEko6S3sLtnNtqZv2BN4GqwBPOuZlmdmHw/iPOudlmNhGYARQAI5xz/0t2rCIiUjbldclk114L773n+2936BB2NJKuQnm0u3PudeD1qGmPRL2+C7grmXGJiEjFKK9LJnr2WRg2DC69FM44I+xoJJ3pSZMiIiIiUaZPh/POg06d4J57wo5G0p0q3CIiIiIRfv4ZuneHBg1gzBjYaaewI5J0F0qXEhEREZFUtHUr9O4Ny5b5B9w0aRJ2RJIJVOEWERERCQwYAJMmweOPQ8eOYUcjmUJdSkRERESAF16AO++Eiy6Cc84JOxrJJKpwi4iISNb7+ms4+2z461/h3nvDjkYyjSrcIiIiktVWrYJu3aBePXjxRahePeyIJNOoD7eIiIhkrW3bIC8Pvv8e8vP949tFKpsq3CIiIpK1br4ZJk6ERx7x3UlEEkFdSkRERCQrjR0Lt9/uH3DTr1/Y0UgmU4VbREREss6sWXDWWf7Wfw88AGZhRySZTBVuERERySq//eYHSdap41u5a9QIOyLJdOrDLSIiIlmjoABOPx0WLIB334VmzcKOSLKBKtwiIiKSNW67DSZM8N1IOnUKOxrJFupSIiIiIlnhlVfg1lt93+2LLw47GskmqnCLiIhIxvvmG9+V5KCD/C0ANUhSkqnMCreZ9UpGICJhGzVqFC1btqRKlSq0bNmSUaNGhR2SSEIor0u2Wb0auneHmjX9IMmaNcOOSLJNPC3cT5vZu2bWLuHRiIRk1KhR9OvXj0WLFuGcY9GiRfTr10+VbslUyuuSNQoKfBeSb7+FF16APfYIOyLJRvEMmvwz8BDwpZn9FxjonFub2LDK7/vva5ObG3YUyfHrrwew665hR5F4ySznxx/vw6ZNrxWbtn49nHtuTR57LPGfny37FLKrrCksLfK6SGW4/XYYPx7+8x+ypp4gqafMFm7n3NfOuU5AP+B0YI6Z9U54ZCJJtGnTxnJNF0lnyuuSLV5/3T+6PS8PLr887Ggkm8V9W0Dn3FNmNh64Hfg/M+sH9HfOzUxUcOWxxx7ryc8PO4rkyM+fTm4W/ExPZjlbtuzLokWLtpveokUL8vMXJvzzs2WfQvaUNR0GZKV6XhfZEfPmQZ8+0L49DB+eHuekZK5y3aXEOfebc+4S4GCgIf5y5D1mVjch0YkkyZAhQ6hdu3axabVr12bIkCEhRSSSHMrrkonWrvVPkqxaFcaNg6j0LpJ0cVW4zWwnM+toZpeZ2bPAS8D++BbyS4BvzOwfCYxTJKHy8vIYPnw4LVq0wMxo0aIFw4cPJy8vL+zQRBJCeV0ylXNw9tkwezaMHg0tW4YdkUgcXUrM7EPgQKA6UAB8BbwKTAU+ANYCtwAvmtllzrlHEheuSOLk5eWpgi1ZQXldMtmdd8KLL8Jdd8HRR4cdjYgXTx/utcBQfBL+2Dm3LsY8V5nZj8ANgBKziEhqU16XjPTWW3DDDXDaaXDVVWFHI/K7Mivczrlj41zX+8AdOxaOiIgkmvK6ZKLvvoNevWD//eHxxzVIUlJLZT7a/SvgpEpcn4iIhEt5XdLCunX+SZLgB0nWqRNuPCLR4r4tYFmccxvwfQBFRCQDKK9LOnAOzjsPvv4a3ngD9tkn7IhEtldpFW4RERGRZBs2DJ5/3j9R8m9/Czsakdgqs0uJiIiISNK8/Tb861/Qsydcd13Y0YiUTBVuERERSTuLFvlBkvvuC08+qUGSktpU4RYREZG0smGDHyS5dSuMHw919VxUSXHqwy0iIiJpwzno1w+mT4dXX4XWrcOOSKRsqnCLiIhI2vjvf+GZZ+DWW+Hvfw87GpH4qEuJiIiIpIXJk+HKK+Gkk+DGG8OORiR+qnCLiIhIyvv+ezjlFGjVCp5+GqqoBiNpRIeriIiIpLSNG6FHD//v+PFQr17YEYmUj/pwi4iISMpyDi66CKZN849t33ffsCMSKT+1cIuIiEjKevhhGDkSbroJunULOxqRilGFW0RERFLS1Klw+eVw/PEwcGDY0YhUXCgVbjM7zszmmNk8MyvxYaxmdrCZbTOzk5MZn4iIlI/yulS2pUvh5JOhZUsYNUqDJCW9Jf3wNbOqwINAV6Ad0NvM2pUw37+BN5MboYiIlIfyulS2TZugZ09Yu9YPktx117AjEtkxYfxe7AjMc85955zbDDwPnBRjvkuBl4AVyQxORETKTXldKtVll8Enn8BTT8H++4cdjciOC6PC3Qz4PuL1kmBaETNrBnQHHkliXCIiUjHK61Jphg/3f9df71u5RTJBGLcFtBjTXNTre4FrnXPbzGLNHqzIrB/QD6BRo0bk5+dXUoipbe3atVlR1mwpJ6iskvaU13dQNp0XpZV15sx6XHHFARx88K8cddQM0nmTaJ9KpDAq3EuAPSJeNweWRc1zEPB8kJQbAseb2Vbn3PjImZxzw4HhAG3btnW5ubkJCjm15Ofnkw1lzZZygsoqaU95fQdl03lRUlmXL4c+fWDPPWHixAY0aLD9POlE+1QihVHh/gxobWZ7AUuBXkCfyBmcc3sV/t/MRgITopOyiIikDOV12SGbN/vHtv/2G0ycCA0ahB2RSOVKeoXbObfVzPrjR6lXBZ5wzs00swuD99W/T0QkjSivy4668kr44AN4/nnIyQk7GpHKF8qj3Z1zrwOvR02LmZCdc32TEZOIiFSc8rpU1JNPwoMPwjXXwGmnhR2NSGLoNvIiIiISis8+g4sugqOPhttvDzsakcRRhVtERESSbsUK6NED/vAH35WkWijX3EWSQ4e3iIiIJNWWLX6Q5MqV8OGHsNtuYUckkliqcIuIiEhSXX01vP8+jBoFBx4YdjQiiacuJSIiIpI0b73VhPvvh3/+0993WyQbqMItIiIiSfH553DPPW3IzYU77ww7GpHkUYVbREREEu6nn/wgyV133cILL2iQpGQXVbhFREQkobZu9ffY/vFHuO22/9GoUdgRiSSXfl+KiIhIQl17Lbz3HowcCS1arA07HJGkUwu3iIiIJMxzz8GwYdC/P5x1VtjRiIRDFW4RERFJiK++gnPPhU6dfKVbJFupwi0iIiKVbtUq6N4dGjSAMWNgp53CjkgkPOrDLSIiIpVq2zbo3RuWLvUPuGnSJOyIRMKlCreIiIhUqhtvhLfegsceg0MOCTsakfCpS4mIiIhUmjFj4I474IIL4Lzzwo5GJDWowi0iIiKV4n//g7PPhr/+Fe6/P+xoRFKHKtwiIiKyw375xQ+SrFcPXnwRqlcPOyKR1KE+3CIiIrJDtm2DvDxYtAjy8+EPfwg7IpHUogq3iIiI7JCBA+GNN+Dhh313EhEpTl1KREREpMLGjYPBg/0Dbi64IOxoRFKTKtwiIiJSIbNnw5lnQseO8MADYBZ2RCKpSRVuERERKbfffoNu3aB2bXjpJahZM+yIRFKX+nCLiIhIuRQUwBlnwHffwTvvQPPmYUckktpU4RYREZFyGTQIXn0V/vtfOOKIsKMRSX3qUiIiIiJxe+UVf1eSs86CSy4JOxqR9KAKt4iIiMRlzhzfleTPf/a3ANQgSZH4qMItIiIiZVqzxj9Jsnp1GDsWatUKOyKR9KE+3CIiIlKqggJ/+7+5c2HSJNhzz7AjEkkvqnCLiIhIqYYOhfHjYdgw6NIl7GhE0o+6lIiIiEiJ3ngDbroJ+vSBK64IOxqR9KQKt4iIiMQ0b56vaOfkwGOPaZCkSEWpwi0iIiLbWbvWD5KsUgXGjfNPlBSRilEfbhERESnGOTjnHJg1CyZOhL32CjsikfSmCreIiIgUc9ddMGYM3HknHHNM2NGIpD91KREREZEib70F118Pp54KV18ddjQimUEVbhEREQHgu++gVy/Yf3944gkNkhSpLKpwi4iICOvXQ48evv/2uHFQp07YEYlkDvXhFhERyXLOwfnnw4wZ8PrrsM8+YUckkllU4RYREcly994Lzz4LQ4bAcceFHY1I5lGXEhERkSz27rtwzTX+ntvXXx92NCKZSRVuERGRLLVoEZx2GrRpA089pUGSIokSSoXbzI4zszlmNs/Mrovxfp6ZzQj+PjSz9mHEKSIi8VFeTx+jRo2iZcuWmNWmTZuvWbduM+PHQ926YUcmkrmSXuE2s6rAg0BXoB3Q28zaRc22AOjsnMsBBgHDkxuliIjES3k9fYwaNYp+/fqxaNEi4GE2b/4TBQW9+eyzUWGHJpLRwmjh7gjMc85955zbDDwPnBQ5g3PuQ+fcL8HLj4HmSY5RRETip7yeJgYMGMD69euB/sBZwC1s2jSWAQMGhByZSGYL4y4lzYDvI14vAQ4pZf5zgTdivWFm/YB+AI0aNSI/P7+SQkxta9euzYqyZks5QWWVtKe8voOSdV4sXrwYOAL4D/Ay/mKDn56sbZ0tOSBbygnZVdaKCqPCHWtIhos5o1kXfGI+PNb7zrnhBJcl27Zt63JzcyspxNSWn59PNpQ1W8oJKqukPeX1HZSs82L33Q9h6dIXgHnAmRTupj333DNp52W25IBsKSdkV1krKowK9xJgj4jXzYFl0TOZWQ4wAujqnPs5SbGJiEj5Ka+ngY0boUaNV4HqQGdgNQC1a9dmyJAhYYYmkvHC6MP9GdDazPYys+pAL+CVyBnMbE9gLHCGc25uCDGKiEj8lNdTnHNwySXw3XcNueKKL2nRYiNmRosWLRg+fDh5eXlhhyiS0ZLewu2c22pm/YE3garAE865mWZ2YfD+I8DNwG7AQ+ZvCrrVOXdQsmMVEZGyKa+nvkcfhSeegBtvhEGDOvOf/ywMOySRrBLKo92dc68Dr0dNeyTi/+cB5yU7LhERqRjl9dT1wQdw2WVw/PEwcGDY0YhkJz1pUkREJEMtWwYnnwwtWsCoUVC1atgRiWSnUFq4RUREJLE2bfKV7TVrYNIk2HXXsCMSyV6qcIuIiGSgyy+Hjz6CF16AP/4x7GhEspu6lIiIiGSYESP8QMnrroNTTgk7GhFRhVtERCSDfPKJvwXgscfC4MFhRyMioAq3iIhIxvjhB+jZE5o1g+ee0yBJkVShPtwiIiIZYPNm333kl1983+0GDcKOSEQKqcItIiKSAa68EqZO9S3bOTlhRyMikdSlREREJM09+SQ8+CBcdRX06hV2NCISTRVuERGRNPbZZ3DRRXDkkXDHHWFHIyKxqMItIiKSplasgB49oGlTGD0aqqmjqEhK0qkpIiKShrZsgVNPhZUr4YMPoGHDsCMSkZKowi0iIpKGrrkGJk+Gp5+GDh3CjkZESqMuJSIiImnm//4P7rsPLrsMzjgj7GhEpCyqcIuIiKSRL7+Efv2gc2e4++6woxGReKjCLSIikiZWroTu3X1/7RdegJ12CjsiEYmH+nCLiIikga1b/T22f/gBpkyBxo3DjkhE4qUKt4iISBq4/np45x3/kJuDDw47GhEpD3UpERERSXHPP+/7a19yCfTtG3Y0IlJeqnCLiIiksK++gnPOgcMPh2HDwo5GRCpCFW4REZEUtWqVHyRZvz6MGQPVq4cdkYhUhPpwi4iIpKBt26B3b1i61D/gpmnTsCMSkYpShVtERCQF3XgjvPUWDB8Ohx4adjQisiPUpURERCTFvPgi3HGHf8DN+eeHHY2I7ChVuEVERFLIzJn+TiSHHgr33x92NCJSGVThFhERSRG//grdukHduvDSS1CjRtgRiUhlUB9uERGRFFBQAHl5sHAh5OfD7ruHHZGIVBZVuEVERFLAwIHw+uvw0ENw2GFhRyMilUldSkREREI2depuDBoEZ58NF14YdjQiUtlU4RYREQnRN9/A0KH7cfDBvnXbLOyIRKSyqcItIiISktWr/SDJGjUKeOklqFkz7IhEJBFU4RYREQlBQQGceSbMmwc33zyTPfYIOyIRSRQNmhQREQnBkCHw8stw332Qk/Nb2OGISAKphVtERCTJXnsNbrkFTj8dLr007GhEJNFU4RYREUmiuXOhTx844AAYPlyDJEWygSrcIiIiSbJmDXTvDjvtBOPGQa1aYUckIsmgPtwiIiJJ4Bz07etvAzhpErRoEXZEIpIsqnCLiIgkwdChMHYs3H03HHlk2NGISDKpS4mIiEiCTZwIN94IvXvDlVeGHY2IJJsq3CIiIgk0f76vaOfkwIgRGiQpko1CqXCb2XFmNsfM5pnZdTHeNzO7P3h/hpl1CCNOERGJj/J6bOvW+UGSVar4QZK1a4cdkYiEIekVbjOrCjwIdAXaAb3NrF3UbF2B1sFfP+DhpAYpIiJxU16PzTk45xyYOROeew722ivsiEQkLGG0cHcE5jnnvnPObQaeB06Kmuck4GnnfQzsamZ/SHagIiISF+X1GO6+G154AW6/HY49NuxoRCRMYdylpBnwfcTrJcAhcczTDFgeOZOZ9cO3lABsMrP/VW6oKashsDLsIJIgW8oJKmsmaht2AEmkvF6K667zf2XIlvMCsqes2VJOyK6yVii3h1HhjjVcxFVgHpxzw4HhAGY2zTl30I6Hl/qypazZUk5QWTORmU0LO4YkUl7fQSpr5smWckL2lbUiy4XRpWQJsEfE6+bAsgrMIyIiqUF5XUSkFGFUuD8DWpvZXmZWHegFvBI1zyvAmcGo9kOB35xzy6NXJCIiKUF5XUSkFEnvUuKc22pm/YE3garAE865mWZ2YfD+I8DrwPHAPGA9cHYcqx6eoJBTUbaUNVvKCSprJsqWciqvVw6VNfNkSzlBZS2TObddFzoREREREakketKkiIiIiEgCqcItIiIiIpJAaVfhzpbHB8dRzrygfDPM7EMzax9GnJWhrLJGzHewmW0zs5OTGV9liqesZpZrZtPNbKaZTU52jJUhjuN3FzN71cy+CsoZT3/elGRmT5jZipLuF50pOSmRsiWvQ/bkduX17eZJ+7wO2ZPbE5LXnXNp84cfjDMf2BuoDnwFtIua53jgDfw9Xw8FPgk77gSV869A/eD/XdOxnPGWNWK+d/EDr04OO+4E7tddgVnAnsHrxmHHnaBy3gD8O/h/I2AVUD3s2CtY3iOADsD/Sng/7XNSChwvGbENsyW3K69nXl4vR1kzIrcnIq+nWwt3tjw+uMxyOuc+dM79Erz8GH9P23QUzz4FuBR4CViRzOAqWTxl7QOMdc4tBnDOpWN54ymnA+qamQE745Py1uSGWTmcc+/j4y9JJuSkRMqWvA7Zk9uV14vLhLwOWZTbE5HX063CXdKjgcs7T6orbxnOxf/SSkdlltXMmgHdgUeSGFcixLNf2wD1zSzfzD43szOTFl3liaecDwD74R988jVwuXOuIDnhJV0m5KREypa8DtmT25XXi8uEvA7K7ZHKnZPCeLT7jqi0xwenuLjLYGZd8En58IRGlDjxlPVe4Frn3Db/ozltxVPWasCfgaOAWsBHZvaxc25uooOrRPGU82/AdOBIYB9gkplNcc6tTnBsYciEnJRI2ZLXIXtyu/J6cZmQ10G5PVK5c1K6Vbiz5fHBcZXBzHKAEUBX59zPSYqtssVT1oOA54Ok3BA43sy2OufGJyXCyhPv8bvSObcOWGdm7wPtgXRKzPGU82zgDuc7w80zswXAvsCnyQkxqTIhJyVStuR1yJ7crry+/TzpntdBuT1SuXNSunUpyZbHB5dZTjPbExgLnJGGv5IjlVlW59xezrmWzrmWwIvAxWmYlCG+4/dloJOZVTOz2sAhwOwkx7mj4innYnxrD2bWBGgLfJfUKJMnE3JSImVLXofsye3K68VlQl4H5fZI5c5JadXC7RL3+OCUEmc5bwZ2Ax4KWgi2OucOCivmioqzrBkhnrI652ab2URgBlAAjHDOxbwtUaqKc58OAkaa2df4S3PXOudWhhb0DjCz54BcoKGZLQFuAXaCzMlJiZQteR2yJ7crr2deXofsyu2JyOt6tLuIiIiISAKlW5cSEREREZG0ogq3iIiIiEgCqcItIiIiIpJAqnCLiIiIiCSQKtwiIiIiIgmkCreIiIiISAKpwi0iIiIikkCqcIuIiIiIJJAq3JL1zKyVmW0xs1ujpj9sZmvMLK2e8iYiku2U1yXVqMItWc85Nw8YAfzTzBoCmNnNwDlAd+fctDDjExGR8lFel1SjR7uLAGbWFJgPPAR8AwwHejvnXgg1MBERqRDldUkl1cIOQCQVOOd+MLN7gavw58VlSsoiIulLeV1SibqUiPzuW6AG8JFz7sGwgxERkR2mvC4pQRVuEcDMjgQeBT4CDjOz9iGHJCIiO0B5XVKJKtyS9cysAzAeP8AmF1gM3B5iSCIisgOU1yXVqMItWc3MWgFvAG8BlzrnNgO3Aseb2RGhBiciIuWmvC6pSHcpkawVjGD/EN/y8Tfn3KZgelXgf8Avzrm/hhiiiIiUg/K6pCpVuEVEREREEkhdSkREREREEkgVbhERERGRBFKFW0REREQkgVThFhERERFJIFW4RUREREQSSBVuEREREZEEUoVbRERERCSBVOEWEREREUmg/wcoDz9cYFGv4gAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "def plot_raw_data(xi, yi, ax):\n", " \"\"\"plot x vs y on axes ax, \n", " add axes labels and turn on grid\n", " \"\"\"\n", " ax.plot(xi, yi, 'ko', label='raw data')\n", " ax.set_xlabel('$x$', fontsize=16)\n", " ax.set_ylabel('$y$', fontsize=16)\n", " ax.grid(True)\n", "\n", "# set up our figs for plotting - we want two subplots arranged in a 1x2 grid\n", "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))\n", "fig.tight_layout(w_pad=5) # add some padding otherwise the axes labels overlap\n", "\n", "#### single data point example\n", "xi = [0.5]\n", "yi = [0.5]\n", "\n", "# plot the raw data on our first axis (ax1)\n", "plot_raw_data(xi, yi, ax1)\n", "\n", "# set up a grid of equally spaced x points to plot our functions over\n", "x = np.linspace(0., 1., 100)\n", "\n", "# np.ones_like(x) gives us a numpy array of 1's of the same size as x\n", "# we need arrays of the same length for plotting\n", "y = yi*np.ones_like(x)\n", "\n", "# plot the p/w constant line and give it a label for use in a legend\n", "# use the first of our two axis labels (ax1)\n", "ax1.plot(x, y, 'b', label='Constant')\n", "\n", "# add a figure title\n", "ax1.set_title('Polynomial approx to single data point', fontsize=16)\n", "# Add a legend\n", "ax1.legend(loc='best', fontsize=14)\n", "# set bounds on the view\n", "ax1.set_xlim(0, 1)\n", "ax1.set_ylim(0, 1)\n", "\n", "#### two data point example\n", "xi = [0.3, 0.7]\n", "yi = [0.2, 0.8]\n", "\n", "# plot the raw data\n", "plot_raw_data(xi, yi, ax2)\n", "\n", "# redefine x - just in case we wanted to edit for second plot\n", "x = np.linspace(0., 1., 100)\n", "\n", "# these are the mathematical expressions derived above\n", "# for the polynomial (linear) coefficients\n", "a0 = yi[0] - ((yi[1] - yi[0]) / (xi[1] - xi[0])) * xi[0]\n", "a1 = ((yi[1] - yi[0]) / (xi[1] - xi[0]))\n", "# we'll form and solve the linear system for the 3x3 case next\n", "\n", "# form the linear at the x locations using array operations\n", "y = a0 + a1*x\n", "\n", "# plot on the second of our two axis labels (ax2)\n", "ax2.plot(x, y, 'b', label='Linear')\n", "\n", "# add a figure title\n", "ax2.set_title('Polynomial approx to two data points', fontsize=16)\n", "# Add a legend\n", "ax2.legend(loc='best', fontsize=14)\n", "# set bounds\n", "ax2.set_xlim(0, 1)\n", "ax2.set_ylim(0, 1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fitting three data points\n", "\n", "The polynomial that fits the three data points $\\{(x_0,y_0),(x_1,y_1),(x_2,y_2)\\}$ is of course a quadratic,\n", "but things start to get a bit more complicated now if we actually want to solve for the polynomial coefficients.\n", "\n", "Substituting the data into the quadratic \n", "\n", "$$ y = f(x; \\boldsymbol{a} ) \\equiv a_0 + a_1\\,x + a_2\\,x^2$$\n", "\n", "leads to a system of three simultaneous equations, or a $3\\times 3$ linear system to solve. \n", "\n", "This is a lot more tricky to solve by hand than the $2\\times 2$ system for the linear coefficients that we could solve through substitution quite easily.\n", "\n", "However, you should be able to show that this system takes the form (and now you should spot a pattern and be able to write down what this would be in the $N+1$ dimension case)\n", "\n", "$$\n", "\\begin{pmatrix}\n", "1 & x_0 & x_0^2 \\\\\n", "1 & x_1 & x_1^2 \\\\\n", "1 & x_2 & x_2^2\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "a_0\\\\\n", "a_1\\\\\n", "a_2\n", "\\end{pmatrix}\n", "=\n", "\\begin{pmatrix}\n", "y_0\\\\\n", "y_1\\\\\n", "y_2\n", "\\end{pmatrix} \\;\\;\\;\\;\\;\\;\\;\\text{or equivalently in matrix notation} \\;\\;\\;\\;\\;\\; V\\boldsymbol{a} =\\boldsymbol{y}.\n", "$$\n", "\n", "If we solve this system by inverting the matrix ($V$) we have our quadratic polynomial coefficients: $\\boldsymbol{a} = V^{-1}\\boldsymbol{y}$. Although note in general we prefer to use a linear \"solver\" to obtain the solution rather than explicitly forming the inverse matrix.\n", "\n", "Let's form and solve the matrix system using appropriate NumPy functions.\n", "\n", "The matrix $V$ is what's called the Vandermonde matrix and there is a Numpy function for it." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "V = \n", "[[1. 0.3 0.09]\n", " [1. 0.5 0.25]\n", " [1. 0.8 0.64]]\n", "\n", " Our coefficients a = \n", "[-1.8 8.86666667 -7.33333333]\n", "\n", "The output from np.polyfit(x, y, 2) = \n", " [-7.33333333 8.86666667 -1.8 ]\n", "\n", "Which agrees with us as long as we reverse the order of our coefficients:\n", "np.flip(a, 0) = \n", "[-7.33333333 8.86666667 -1.8 ]\n" ] }, { "data": { "text/plain": [ "(0.0, 1.0)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAGKCAYAAAAi8hcxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABD00lEQVR4nO3dd5xU1f3/8deHsiC9CGJAiiIoIihIb4ugUjQqdrAQNdgTk2hMgiY2zNdEiRo1CXZ/YkQNNgRFylJEmtIEREDpWBEEFgR2z++PM8suw5bZ2dm5U97Px2Mfs3Pnzp3PnLkzn3vKPdecc4iIiJRWhaADEBGR5KQEIiIiUVECERGRqCiBiIhIVJRAREQkKkogIiISlbRNIGY23Mxcgb+dZrbEzG42s0ql3FaWmWWVU6hxYWbNQ+UwPIrnJv37Lw9mdp6Z/TbG28w0s7vNrELY8rzP79pYvl68mNk6M3s+iufFvIxjLfSZOTPLjOK5Cf3+0jaBFHAR0A24AJgP/BP4c6ARBWMrvhzeDTqQFHIeEOsvfybwF/TdzXMesS/jWPsE/936JIrnnkcCv79SHWmnqMXOuTWh/yebWUvgVtIsiTjnfgLmBh1HvJhZldB7TntmZkBl59y+oGNJRc65H0nR75aOYg63AKhpZg0BzGyAmX1kZnvMbIeZvWlmrYt6spk1MrN9ZvbrQh6728yyzaxu6H6Wmc02s/5m9knosU/N7LxCnltiHAW2N8DMFofWXWRmXcyskpk9YGZbzWybmT1vZtULPPewJiwz62Rmr5vZptC2VoW2cUQU5YqZ3RN6nzvM7Dszm2ZmXcPWyavuXxCK8Qcz+9HMxppZ/bB1nZmNMrORBWKcaWanFFEu54TK4yfgxtBjnc1sipntMrPdZjbVzDoXeO79oc+zU4Fl1UNl8VFRzZ2h5pirgMYFmknXFXi8tZm9YWbbQ3HPNbMBJZTf3fjaB8D+vO2GrVbRzO4Nfc7bzewdM2sStp11ZvaSmV1tZp8B+4DBocfam9nboXLfY2YfmlmvQmLpEyqrnaFye9/M2hYXf4Hn/joUw14zW1jE9huY2X/M7PPQ92Kjmb1sZo0LrPM8RZSxmVU1s3+Evk+7zOyrUFmcEEF8pdkHa5nZ42a2xcx+Cu0XvzEzK2R7mQWWlfjdL+H91TCzf5rZhtDrfh3aj0t8fzHlnEvLP2A44ICWYctfAw4A1YABQA7wAfBzYCiwBvgWaFzgOVlAVoH7rwIrwrZbEdgIPB/2vK3AcuDy0Ot9EHr9lgXWK00cXwHLgEuBs4EVwNfAU8BzwFn4GtZ+4G8Fnts8VB7DCyy7ALgztJ0++B/dr4BXwt7bIe+/mDJ/GrgC6Bva5iv4H692BdbJDMWxMRTvAOAWYCcwPWx7eet9iK/qXwKsAr4H6oXF9w3wJXB16DXahf72AB8DF4be74LQsvah51YC5gCrgRqhZc8DO4AWxbzX4/DNgd8AXUN/p4Ye+1nos/si9LmfA7wX+owHFrPNJqEydECPvO2GfX7rgJeBgfgfn++AGWHbWQdsBj4FLgP6heLtAOwGZofKYxDwNvAT0LHA8wfj99G3gHNDf3OAH4BjStgHrgnFmffZ3gxsCpVnwe9Ga+DR0GfSG78/LwjFXjWCMq4dKqtL8fvu+fjvz3agUQkxZhLBPog/AJ8VKrPfAWeGYnbAA4VsL7M03/0S3t9T+O/1NaHyOR94KG9/iNvvaDxfLJH+yE8grfE/EnWB6/Bf4jdD6yzE/3BUKvC8Fvgf39FhO0NWITtMrwLLfh5a1jXsefuB4wssaxiK4U8FlpUmjv3AsYW87pSw9z8e+LLA/eaEJZCw9S1UTpcDuUD9ot5/hOVfMbS9VcCjhZTde2HrDwst71dgmcP/QFYPex/7gfvC4ssFTgnb5uv4H5Q6BZbVArYB48O2uR14Af+D64ChEbzH54FNhSx/iMMPEiqGyuKTErZ5d+j1K4Utz/v8wpPFbaHlPyuwbB2QTdgPKTAVWAlkhMW1ktB3IrRsDTA17Lm1Qp/FI8XEXgH/oxz+2V4SivH5EvaXY0LrnV9SGRfx/Gr4JPCbEtaNaB/EHwQd9p3BJ66fgCPDtpdZyHe1pO9+UfvQpxT47gf1pyYs+Az/QW4DngTGAlebb97pAIxzzh3IW9k59yX+iLdPURt0zmXhj/yvK7D4OmCpcy68LXS1c251ged+gz/iaAq+uaSUcXzunPsi7P0BvF/I+25SsKodLlQ9f9DM1uK/EPuB/4dPJscX9bxittffzKab2ff4H9D9QCt8Eg/3atj91/BJoFvY8onOud15d5xz6/DtzeHrrXPOLQ5b1huY4JzbXuD5P+KPuvsUWLYOuB64En9E+qJz7uWi3mcEegNzXX7fG865HOC/wClmVqsM2w4fBLEsdNs0bPlc59xXeXfMN0v2IVTO5ps8K+E/6ymhmDGz4/FHxmPz1gmtlw18lLdeEZqE/sI/2//h94dDmNkN5kdG7go9viH0UJFNyGHPv9jM5pnZ9tDzdwM1In1+IXGG74O9Q/f/G7beS0AGh++D4Yr97pdgATDczP5kZqeZWcUInhNzSiC+6tcJOAF/JHulc24bvkZi+GpmuK+AeiVs91/AhWZW38ya4auo/y5kvW2FLPsJqBr6v7Rx/BB2f18xyyvhj8yK8hz+h/Mx4Ax8Od0UeqxqUU8qjJl1ACYCu/DV7q6h7S0pYltfF7zjfAfvD0Dj4tYrsCx8vcLKr14Ry7/Cl3tB7+KbxqoA/yjkOaVR3OtaIa9dGuH7U95AgfAyDn/9evh94S58Yi/4dzNQ1/zQ4Yah9Z8pZL2zgfoU7ejQbfhnewBftgeZ2S34A7opwBCgM36fKey9HMbMzgHG4WtPQ4Eu+P3t20ieX0Sc4ftgPWCbO3wwxlcFHi9OSd/94twC/AffJLsA+CbU51MtgufGjEZhwacFjwQL+AFf7WxUyGONCNvhC/Ei8Fd8U1ldfLv62CjiK2scUTGzqvi27budc48WWH5ylJu8AH8UOMQ5t7/A9urim4fCHRUWTwa+HDcXt16BZeHruULW20bR5Rr+5X4C/wO7FhhjZj0Kvo9SKu51XSGvXR7Cy2M7/mj6Cfy+e/gTnMsN1R4B/oj/cQ9X3EiuvKQV/tlW4vDEcym+mex3BdZrUcy2w10KrHHODS/w/MqU/KNeUEn74DagnplluENHsOV9tuXy3QRwzu3CfwZ/DB2gXgj8H7787yiv1w2nGkgRQs0iHwMXFawehj6s7sCMEp7/Iz5hXIc/Sng5tCyucZRBFfwPZviP5PAot1cN37578IfLzE6n6Or6xWH3L8Lvrx+FLR9kYaPJ8Eeq4esVZgYw2MxqFnh+TXyn9owCy4biO/9H4NvrTwHujWD7PwGFjVibAXQNxZr3GhVD217knNtZwjYpYrtRC+1ns4D2+H6YheF/oVVX4ftQTipsHefc0mJeZhO+DyT8s72Aww9mq3H4vveLQrZZVBlX4/BmsSsovsYdrqR9cEbo/kVh6w3D/5DHYuhuUe/vIOfceufcw/jmyohGwsWKaiDFuwvfdDHBzJ7Et5/egx8x8nAEz3+S/H6Qwpqv4hVHqTnndpjZXOB3ZrYV30F6NYc3DUXqPfzor+fN7Dl838ddHF5TyHNSaL1XQuuOwncQTw1bbw/+/J2/45PePcCPRNbMdB++2WWqmT2IT2534H987oWDR73/Ap5xzr0WWjYS+D8zm+ycm17M9lfgj1BvwA+E2OucWxaKbTjwgZn9JRTvjaH3ObiEmFeEbn9nZpOAnAI/7mX1W2Am8L6ZPYOvMRyJ74Or6Jz7g3POmdlNwFuhI/JX8fvGUfgDmg3OudGFbTxUg7kHeLrAZ9sSfyQdfnD1HnCHmf0Jf4Lv6fij7HBFlfF7wHlm9g9gAtAR+BWF13aLUtI+OAk/Yu3fZtYAP6JqEHAt8Ffn3HeleK2iFPr+zOwjfF/dMnyzcB988n8hBq8ZuaB78YP6o4hhvIWsNwB/xLEH/4P9FtA6bJ0sihiFhD9iW1DEY1nA7EKWryNsREop4pgdtqx56H1eG7b8bgqM5qHwYbzN8V+SnfjOvcfxP3CFjSgp9P2HveYt+KG0e/Dttv3Dn0v+iJUh+BEo20Ov/zKhUS0F1nX4L/Wf8Ee3e/FH0adEUs6hx7rgm2J24TtZpwKdQ49VCpX5Kg4d6WXA5NBr1i/m/VbHd7DmNUOuK/BYa+DN0Ge5F3+0OiCCMqyIb2b6Bt/k5Er4nPPKs+DntQ54qYjtn4j/wfwGf/S7Cf9DNShsvW74H+YfQvGvCz2vWwTv4dfA+tDzFgI9Cdvn8Ufd/8L3WewMvVaL0Hu5u6QyxtcM7ge24Dv4ZwCnhr9OEfGVZh+shf9ebMXXOj4HfgNYCZ9BFhF894t5fw8Ci0L7z258IvlVSWUf6z8LBSPlwMxa4Uc7/dI590zQ8SSD0MlW04EznHOFtbEXXNcBo5xzd8YhNEkTpdkH052asMqB+TN/W+KbU7bij1xERFJK3DvRzexZM/vGzD4t4nEzs8fMbI2ZLQ0N/0w21wLT8O3CQ51zewKOR0Qk5uLehGVmvfHtzS865w4bMWBmg/Bt5YPw7dOPOue6xDVIEREpUdxrIM65mRQ/zv1cfHJxzp+1XcfMji5mfRERCUAingfSGD9WPM8moh86KiIi5SQRO9ELm5up0HY2MxuBP7mLqlWrdmzaNJIpZFJfbm4uFSok4rFB/Kks8qks8qks8n3++effOecaRPPcREwgm/CzbuZpgh/LfRjn3BhgDEDr1q3dqlWryj+6JJCVlUVmZmbQYSQElUU+lUU+lUU+M1sf7XMTMQW/DVwZGo3VFdjhnCts4jkREQlQ3GsgZvZf/JmZR5rZJvwV1ioDOOf+jZ+xdRD+mgPZFD7/jYiIBCzuCcQ5d1kJjzvypwwXEZEElYhNWCIikgSUQEREJCqJOAor5nJzc/nuu+/Yvn07OTk5QYdT7mrXrs3KlSuDDqPcVaxYkTp16nDkkUdqSKZIANIigWzatAkzo3nz5lSuXJliLgOeEnbu3EnNmjVLXjGJOefYv38/X3/9NZs2bULnAInEX1octu3evZvGjRuTkZGR8skjXZgZGRkZNG7cmN27dwcdjkhaSosEAqiJI0XpcxUJjr59IiISFSUQERGJihKIROT1119X/5GIHEIJRMpN8+bNeeihh4IOQ0TKiRJIKYwdO5bmzZtToUIFmjdvztixY+P6+vv27Yvr64mIFEcJJEJjx45lxIgRrF+/Hucc69evZ8SIEeWaRDIzM7nhhhu47bbbaNCgAT169ABg9OjRtGvXjurVq9O4cWOuvfZatm/ffvB5LVu2ZNy4cQfv9+jRg5o1a3LgwAEAVq9ejZmxefPmIl/7xRdfpFmzZlSrVo2zzz6br7/++pDH165dy7nnnkujRo2oXr06HTp0YMKECYfEvn79em6//XbM7GDz1/fff89ll11GkyZNOOKIIzjppJN47rnnylxWIhJ/SiARGjlyJNnZ2Ycsy87OZuTIkeX6ui+99BLOOWbNmsWLL74I+KGrjzzyCMuXL+fll19m/vz53HLLLQef07NnT6ZPn34wxoULF1KlShUWLlwI+GshtGzZksaNC7/Q47x58xg+fDgjRoxg8eLFnHPOOfz5z38+ZJ1du3YxcOBAPvjgA5YsWcIFF1zAkCFD+OyzzwAYP348TZo04c9//jNbt25l61Y/I//evXsPJpvly5fz61//muuuu46pU6fGtuBEpPw551Lir1WrVq4oK1asKPKxSJmZw18Z8ZA/MyvztovSp08fd/LJJ5e43qRJk1xGRobLyclxzjk3evRol1cekydPdieccIK78sor3QMPPOCcc27o0KHu2muvLXJ7l112mevfv/8hy6655hrnd5eidenSxd13330H7zdr1sz9/e9/LzH+Sy65xF1zzTUlrleU4j7f6dOnR73dVKOyyKeyyAcsdFH+7qoGEqGipsoo7yk0OnbseNiyadOmccYZZ9CkSRNq1qzJkCFD2LdvH1999RUAvXr14vPPP2fLli1kZWXRt29fMjMzycrKAmDGjBnFXo1t5cqVdOvW7ZBl4fd3797N73//e9q0aUPdunWpUaMGCxcuZMOGDcW+n5ycHEaNGkW7du2oX78+NWrUYPz48SU+T0QSjxJIhEaNGkW1atUOWVatWjVGjRpVrq9bvXr1Q+6vX7+ewYMHc+KJJ/Laa6/x8ccf8+yzzwL5neytW7fmqKOOIisr62AC6du3Lx9++CErVqxg8+bNxSYQf1BSvNtuu43XXnuN++67jxkzZrB48WI6d+5cYkf/Qw89xMMPP8ztt9/O1KlTWbx4Meedd54GCIgkobSYTDEWhg0bBvi+kA0bNtC0aVNGjRp1cHm8LFy4kH379vGPf/yDihUrAhzSeZ2nT58+vPvuuyxcuJA+ffrQsGFDjjzySP72t78V2/8B0KZNG+bOnXvIsvD7s2fP5sorr+SCCy4AfN/G2rVradWq1cF1MjIyDpv9ePbs2ZxzzjlcccUVgE9Wn3/+OXXq1Im8EEQkIagGUgrDhg1j3bp15Obmsm7durgnD4Djjz+e3NxcHnnkEb788kv++9//8sgjjxy2XmZmJuPGjeP444+nYcOGgE8qL730UrG1D4Bf/epXTJkyhb/+9a+sXr2ap556ijfeeOOQdVq1asUbb7zBJ598wrJly7j88svZu3fvIes0b96cWbNmsXnzZr777ruDz5s6dSqzZ8/ms88+4+abb+bLL7+MvkBEJDBKIEmmXbt2PProo4wePZo2bdrw9NNPF3qyXt++fcnJyTkkWRS2rDBdu3blmWee4V//+hft2rVj/Pjx3H333YesM3r0aBo2bEivXr0YOHAgXbt2pVevXoesc++997Jx40aOO+44GjRoAMCdd95J586dGThwIL1796Z69eqBJGIRKTuLpL07GbRu3dqtWrWq0MdWrlzJiSeeGOeIgpMO1wMpqLjPNysrq8SEmS5UFvlUFvnM7GPn3GnRPFc1EBERiYoSiIiIREUJREREoqIEIiIiUVECERGRqCiBiIhIVJRAREQkKkogIiISFSUQERGJihKIFOu7777DzA5OBR9rWVlZmNnBubJEJHkogSSBzZs3M2LECJo0aUJGRgaNGzfml7/8JZs2bQo6tFJp3rz5YfN2de/ena1bt1K/fv2AohKRaCmBJLgvv/yS0047jU8//ZQXXniBNWvW8NJLL7F8+XI6derEunXrgg6R/fv3R/3cjIwMGjVqdPCa6SKSPJRAEtxNN91EhQoVmDJlCv369aNp06b07duXKVOmUKFCBW666SbAT99+8803H/Lc4cOHc/bZZx+8/95779GrVy/q1q1LvXr1OOuss1i5cuUhz1mwYAEdO3akatWqnHrqqcybN++Qx/OanCZOnEjnzp3JyMjg/fffZ+3atZx77rk0atSI6tWrH7zueZ7MzEzWr1/P7bffjpkdTBiFNWHNnTuX008/nerVq1O7dm369evHli1bYlOgac45/ycSC2l7Qalbb4XFi+P7mqecAoVcuqNI27Zt47333uP+++8v9GqIN954I3fddRc//PBDRNvbvXs3t956K+3atWPPnj3cf//9nHPOOaxYsYKMjAx2797N4MGD6dOnDy+88AKbN2/m1ltvLXRbd9xxBw8//DAtW7akZs2abNmyhYEDB3L//fdzxBFHMG7cOIYMGcLSpUs54YQTGD9+PO3bt+fqq6/mhhtuKDLGJUuW0LdvX6644gpGjx5NlSpVmDlzJgcOHIi43NLd/v0wfz588on/W7wYvv0Wtm/vyZ49UKECHHUUNGoEjRtD587Qq5e/rVo16OglmaRtAkkGq1evxjlX5FTlbdq0wTnH6tWrI9pe3tUD8zz33HPUqlWL+fPn07NnT8aOHcu+fft47rnnqFGjBm3btmXkyJEHrx5Y0N13382ZZ5558H6DBg1o3779wfsjR47knXfe4fXXX+fOO++kXr16VKxYkZo1a9KoUaMiY/zb3/5G+/btGTNmzMFl6TQVf7QOHICsLBg3DsaPh23b/PKGDeHUU/3fjh1fccIJTcjNha+/hq++gtWr4e23/boZGdC/P/ziF3DOOVClSmBvR5JE2iaQ0tQEglZU/0DetVwyMjIi2s7atWu56667mDdvHt9++y25ubnk5uayYcMGwF9Xo127dtSoUePgc7p161botk477dDLB+zevZt77rmHCRMmsHXrVvbv38/evXtp165dRLHlWbRoEeeff36pnpPO9uyBZ5+FBx+EjRuhRg34+c/hgguga1c4+mjI232ystaQmdnksG1s2wYffggzZsArr8BFF0H9+jB8ONx+u6+tiBRGfSAJ7Pjjj8fMWL58eaGPr1y5kkqVKtGiRQsqVKhA+MXBwju3zznnHL799lv+85//MG/ePBYtWkSlSpXYt28fwGHPL0716tUPuX/bbbfx2muvcd999zFjxgwWL15M586dD247UqlygbPy9tNPMHo0HHss3HwzHHMMvP46fPMNjB0LQ4bAz36WnzyKU6+er3E89BCsXw+TJkHfvv4g67jj4M47Yfv28n5HkoyUQBJYvXr1GDBgAE8++STZ2dmHPJadnc0TTzzB+eefT+3atWnQoAFbt249ZJ0lS5Yc/P/7779n5cqV/OlPf6J///6ceOKJ7Ny585C+hTZt2rBs2TJ27959cNncuXMjinX27NlceeWVXHDBBbRr144mTZqwdu3aQ9bJyMggJyen2O106NCBadOmRfSa6WrWLN+f9rvfwUknwfTpMHu2r3UccUTZtl2xIgwYAK+9BitWwNlnw6hRPlE99ZQ64OVQSiAJ7oknniAnJ4f+/fszbdo0Nm7cSFZWFmeccQaVK1fmscceA+D0009n0qRJvP3226xevZrf/va3bNy48eB26taty5FHHslTTz3FmjVrmDFjBtdffz2VKuW3Yg4dOpRKlSpx9dVXs3z5cj744ANGjRoVUZytWrXijTfe4JNPPmHZsmVcfvnl7N2795B1mjdvzqxZs9i8eXORJw7efvvtLFq0iBEjRrBkyRJWrVrF008/fbCZLZ398ANcey307g1798LEiTBlCmRmRlbTKK1WrXyT1iefQPv2MGIEDBzom8pEQAkk4bVo0YKFCxdy0kknccUVV9C8eXP69u1LhQoVWLx48cEO6auvvvrg3xlnnEGNGjUO6UuoUKEC48aNY+nSpbRt25abbrqJ++67jyoFekpr1KjBhAkTWL16NR06dOC2227jwQcfjCjO0aNH07BhQ3r16sXAgQPp2rUrvXr1OmSde++9l40bN3LcccfRoEGDQrdzyimnMGXKFD777DO6du1Kly5deOWVV6hcuXJpiy6lfPyx7wh//nnfL/Hpp/7HPB5OPRWmToUnnvC1n7Zt4aWX4vPakuCccynx16pVK1eUFStWFPlYMnrsscdc5cqV3Ztvvlno4z/++GOcIwpWcZ/v9OnT4xdIOcjNdW7MGOcyMpw75hjn5s6NfluxKIs1a5zr1cufTXLrrc7t31/mTQYi2feLWAIWuih/d1UDSUK33HILL730EitWrGDPnj1BhyPlZP9+32Q1YoRvpvrkE+jSJdiYjjsOpk2DX//ad7IPGpQ/ZFjST9oO4012F198cdAhSDnavdsPp500Ce66C/7yF9/BnQgqVfLJo107uP56fwLi5Mm+o13Si2ogIglm2zY44wx4/30/8uneexMneRR09dX+5MUffvBnsn/2WdARSbwpgYgkkK1b/Y/xxx/7obTXXht0RMXr3t0nkQMH/OiwpUuDjkjiKW0SiNMA9pSUSp/r99/7msf69fDee/5kwGRw8skwc6afCiUz0yc/SQ9pkUAqV66szuYUtWfPnpQY4vvjj/4EvjVr/NxUffsGHVHptG7th/jWquXfx+efBx2RxENaJJCGDRuyefNmsrOzU+qINZ0558jOzmbz5s00bNgw6HDKZM8eP5XI4sW+2er004OOKDotWvjOdICzzvLNcZLa0mIUVq1atQDYsmVLmS5+lCz27t1L1TSYl7ty5cocddRRBz/fZJSTA5de6o/ex471iSSZtWrlz5Dv29ef6DhjBtSuHXRUUl7SIoGATyLJ/ENTGllZWZx66qlBhyERGDnSN1k99hhcdlnQ0cRGp05+Svmzz4bzz/ejyVKglVEKkRZNWCKJ6MUX/TTs11/vZ9RNJWee6aeZnz4dfvvboKOR8pI2NRCRRPLRR/DLX/pRS489Vj6TIQbt8sth0SI/7XzHjv76IpJaVAMRibOtW33TTt41PFK5eefBB6FfP1/LWrAg6Ggk1pRAROIoJweGDYOdO+Gtt/yV/1JZpUr+MrtHH+2T5jffBB2RxJISiEgcjRrl+wWeeMJfDCod1K8Pb7zhT5QcPlwXpUolSiAicTJjBtxzj+8buOqqoKOJr1NO8ZfMnTQJ/vnPoKORWAkkgZjZADNbZWZrzOwPhTxe28zeMbMlZrbczH4RRJwisfLttzB0KLRsCU8+mZqd5iW58UY/tPf22zVnVqqIewIxs4rAE8BAoA1wmZm1CVvtJmCFc649kAk8bGYZcQ1UJAbGjh1Ls2bNadhwPFu2/MRVV02kZs2gowqGmR/aW6+eP+dFswslvyBqIJ2BNc65L5xz+4BXgHPD1nFATTMzoAawDTgQ3zBFymbs2LGMGDGCDRu6AUOAuxg16iLGjh0bdGiBadDAn/+yYgX8/vdBRyNlFUQCaQxsLHB/U2hZQY8DJwJbgGXAr51zufEJTyQ2Ro4cSXZ2TfzuPBd4mOzsbEaOHBlwZME64wx/RcPHH/dTuEjyCuJEwsJaf8PHZZwFLAZOB44DPjCzWc65Hw/ZkNkIYARAgwYNyMrKinmwyWjXrl0qi5Agy2L9+g3AeKA6MBzwx0AbNmwIJKZE2i/OOqsCr77aiaFDHU8/vZAqVeJ7fJhIZZHUor2YerR/QDfg/QL3/wj8MWydd4FeBe5PAzoXt91WrVqV9dryKWP69OlBh5AwgiyL+vV/5fyg1d85/EGSA1yzZs0CiSfR9ospU5wD5+64I/6vnWhlESRgoYvy9zyIJqwFwPFm1iLUMX4p8HbYOhuAfgBmdhTQGvgirlGKlMH338NPPz1IhQrzgH8cXF6tWjVGjRoVXGAJpF8/uOYaP7xXF6FKTnFPIM65A8DNwPvASuBV59xyM7vezK4PrXYf0N3MlgFTgTucc9/FO1aRaN1xB+zZU5VRo76lWbNjMDOaNWvGmDFjGDZsWNDhJYyHHoKGDf311dPgSgspJ5DJFJ1zE4GJYcv+XeD/LcCZ8Y5LJBbmzIFnnoHbboM//OFs/vCHs4MOKWHVqePPyh8yxN/eemvQEUlp6Ex0kRg6cABuuAGaNIG//CXoaJLDeef5y+D+5S/w1VdBRyOloQQiEkOPP+7Psn70UahRI+hokoOZL689e3zTnyQPJRCRGNmyBe66y1/K9fzzg44mubRqBb/7nT/J8MMPg45GIqUEIhIjd94J+/al7gWiytudd/qmv5tv9tPeS+JTAhGJgcWL4fnn4Ve/8hMmSulVrw4PP+zL8qmngo5GIqEEIlJGzvnml3r1IM1nKSmziy6CXr18h/rOnUFHIyVRAhEpowkTYNo0uPtuPyxVomcGf/+7v3Lhww8HHY2URAlEpAz27/fXt2jdGq67LuhoUkOXLr4m8tBD/vrxkriUQETKYMwYWLXKHzVXrhx0NKnjgQf8gIR77gk6EimOEohIlHbvhnvvhT59/JX2JHZatvQnZD79NKxcGXQ0UhQlEJEo/fOfvq3+gQc0bLc83HmnH5n1xz8GHYkURQlEJArbt8Pf/gaDBkH37kFHk5oaNPD9S2+9BQsXBh2NFEYJRCQK//gH/PAD3H9/0JGktl//2g+P1rxiiUkJRKSUvvsORo+GCy+EU08NOprUVrOmv3b6xIkwd27Q0Ug4JRCRUnrwQcjO9h3oUv5uusk3Z6kWkniUQERK4euv/XUrhg2DE08MOpr0UKOGn6V38mSYPTvoaKQgJRCRUvjHP+Cnn/wIIYmfG26Ao45SLSTRKIGIRGjbNl/7uOQSP/24xE+1an4477RpMHNm0NFIHiUQkQg99hjs2gV/+lPQkaSnX/7S94X89a9BRyJ5lEBEIvDjjz6BnHcetG0bdDTpqVo1f830996DRYuCjkZACUQkIv/6lz/vQ9O1B+vGG/3Q3v/7v6AjEVACESlRdrY/7+Oss+C004KOJr3VqeOTyOuvw+rVQUcjSiAiJXj2WT/nlWofieE3v4GMDD+VjARLCUSkGDk5fuhut27+SnkSvKOOgquvhhdegM2bg44mvSmBiBTjzTfhiy/8JWslcdx+O+Tm+uQuwVECESnGQw/Bscf60VeSOJo3h4svhqee0rXTg6QEIlKEOXP8BH6//S1UrBh0NBLuN7/xw6uffTboSNKXEohIER56COrWheHDg45ECtOpE/ToAY8+6vuqJP6UQEQKsXq17/+48UZ/VTxJTL/5DXz5pb/olMSfEohIIR55BCpXhptvDjoSKc555/n+EHWmB0MJRCTM9u3w/PMwdCg0ahR0NFKcihX9VQtnz4YFC4KOJv0ogYiEef55f/b5LbcEHYlE4uqr/fQmqoXEnxKISAG5uX7K9u7doUOHoKORSNSqBddeC6+9Blu2BB1NelECESng/fdhzRr1fSSbG2+EAwf8eSESP0ogIgU8/rjv97jggqAjkdJo2RIGDIAxY2D//qCjSR9KICIha9bApElw3XV+sj5JLjfe6JuwNKQ3fpRAREKefNKP6hkxIuhIJBqDBkGzZv5zlPhQAhEBdu/2U2JceCH87GdBRyPRqFgRbrgBpk+HFSuCjiY9KIGIAOPGwY4dcNNNQUciZXH11b75UbWQ+FACEcF3vrZp4+dWkuTVoAFccgm8+KJm6Y0HJRBJe0uWwLx5vu/DLOhopKxuvNEnj5dfDjqS1KcEImlvzBioUgWuuCLoSCQWunSBdu10Tkg8KIFIWtu9G156yV+cqF69oKORWDDzZ6Z//DEsWhR0NKlNCUTS2quv+osSaehuarn8cl+rfPrpoCNJbUogktbGjIETT1TneaqpW9cPyR471k+MKeVDCUTS1tKl/pK1112nzvNUdO21fmj2668HHUnqUgKRtPX00+o8T2V9+vg5stSMVX6UQCQt/fSTb944/3x1nqeqvM70WbPgs8+CjiY1KYFIWnrnHdi2DYYPDzoSKU9XXQWVKqkWUl6UQCQtPf88NGkC/fsHHYmUp0aN4Oyz/VDtAweCjib1KIFI2tm61U/bfuWVfgI+SW3Dh8PXX/uLhUlsKYFI2vl//89fuvaqq4KOROJh4EA48khf65TYUgKRtOKc/yHp0QNatQo6GomHjAwYOhTeftv3e0nsKIFIWpk/H1auVOd5urnqKti3z0/bL7GjBCJp5bnn4Igj/NxXkj5OPRXatoUXXgg6ktSiBCJpY+9eeOUVuOACqFUr6Ggknsx8LWTePFi1KuhoUocSiKSNiRP91BY68zw9DRvmR92pFhI7gSQQMxtgZqvMbI2Z/aGIdTLNbLGZLTezGfGOUVLPSy/58wJOPz3oSCQIRx8Nbdtu5m9/20Lfvv1p3rw5Y8eODTqspBb3BGJmFYEngIFAG+AyM2sTtk4d4Eng5865k4CL4h2npJYffoB334XLLvNnJkv6GTt2LCtX/pGcnJ8BvVm/fj0jRoxQEimDIGognYE1zrkvnHP7gFeAc8PWGQqMd85tAHDOfRPnGCXFvP66H4Vz+eVBRyJBGTlyJPv2vQ7sxP/EQHZ2NiNHjgw0rmQWxLFYY2BjgfubgC5h67QCKptZFlATeNQ592L4hsxsBDACoEGDBmRlZZVHvEln165dKouQvLJ4/PFTaNq0Mjt2LCBdiybd94sNGzYADhgPXAjcBOxjw4YNaV0uZRFEAinsygsu7H4loCPQDzgC+MjM5jrnPj/kSc6NAcYAtG7d2mVmZsY+2iSUlZWFysLLysqiRYtMli6F+++Hvn0zgw0oQOm+XzRt2pT169cDLwNX4VvR36Jp06ZpXS5lEUQT1ibgmAL3mwBbClnnPefcbufcd8BMoH2c4pMU89//+tuhQ4ONQ4I1atQoqlWrBkwFvgGGUq1aNUaNGhVwZMkriASyADjezFqYWQZwKfB22DpvAb3MrJKZVcM3ca2Mc5ySApzzc1/17AktWgQdjQRp2LBhjBkzhmbNmgCvYvZzHnnkWYYNGxZ0aEkr7gnEOXcAuBl4H58UXnXOLTez683s+tA6K4H3gKXAfOBp59yn8Y5Vkt/atTVYscKfAyAybNgw1q1bx+OPd8e5qmRkXBJ0SEktkAGNzrmJwMSwZf8Ou/934O/xjEtSz7RpDalUCS7SQHApoE2bH2nRAl5+WbMyl4XORJeU5RxMn96A/v2hfv2go5FEYub7xKZM8dcKkegogUjKWrAAvvrqCC5RK4UUYuhQf12YV18NOpLkpQQiKWvcOKhcOZfzzgs6EklEbdrAyScrgZSFEoikpNxcn0A6ddpGnTpBRyOJ6uKLYfZs2Lw56EiSkxKIpKQ5c/yPQt++mgVHipY3uOL114ONI1kpgUhKGjcOqlaF7t2/DzoUSWCtW0P79mrGipYSiKScnBx47TUYPBiqVcsJOhxJcBdf7GusGzeWvK4cSglEUs6MGX5o5qWXBh2JJAM1Y0VPCURSzquvQvXqMGhQ0JFIMjj+eH/NdDVjlZ4SiKSUnBx44w04+2yoVi3oaCRZXHwxzJ0L69cHHUlyUQKRlDJ7NnzzDVxwQdCRSDJRM1Z0lEAkpfzvf3701cCBQUciyeS446BDBzVjlZYSiKSM3FwYPx4GDIAaNYKORpLNRRfB/PkajVUaSiCSMubP9ycPqvlKojFkiL99881Aw0gqSiCSMv73P6hc2Xegi5RWq1Zw0km+FiuRUQKRlOCcTyD9+6O5ryRqQ4bAzJnw7bdBR5IclEAkJSxeDF9+qeYrKZshQ3xf2tvhF9mWQimBSEr43/+gYkU499ygI5Fk1r49tGihZqxIKYFI0nPOj9/v0weOPDLoaCSZmflayJQpsGNH0NEkPiUQSXqffQarVuWPohEpiyFDYN8+mDgx6EgSnxKIJL233vK3ar6SWOjaFRo1UjNWJJRAJOm99RZ07AhNmgQdiaSCChXgvPN8DWTPnqCjSWxKIJLUtm71k+DpuucSS0OGQHY2TJ4cdCSJrcQEYma6qoIkrHfe8bdqvpJY6tMHatXK37+kcJHUQF40s2lm1qbcoxEppTffhGOPhbZtg45EUklGhr+ezDvv+EsESOEiSSAdgcrAIjN7yMw0TZ0khJ07YepUX/swCzoaSTU//7m/NMD8+UFHkrhKTCDOuWXOuV7ACOByYJWZXVbukYmU4P33/XBLNV9JeRgwACpV0lnpxYm4E9059wLQGngT+H9mNt3MTiqvwERK8uabUL8+9OgRdCSSiurWhd6984eJy+FKNQrLObfDOXcT0Ak4Et+s9bCZ1SyX6ESKsH8/vPuun3m3UqWgo5FUde65sHIlrF4ddCSJKaIEYmaVzayzmf3KzF4G/gecBFQCbgI+M7Ofl2OcIoeYORO2b1fzlZSvc87xtxqNVbhIhvHOAX4EPgIeBloB7wCXAE2AhsArwOtmdn35hSqS7513oEoVOPPMoCORVNaiBZx8svpBihJJ5X8X8FfgQ2Cuc253Iev8zsy+Bv4E/DuG8YkcxjmfQPr1g+rVg45GUt2558IDD8D33/s+N8kXySisM51z9zrnphaRPPLMxNdIRMrVqlXwxRe68qDEx89/7q8RoskVDxfLqUyWAGqRlnI3YYK/HTw42DgkPXTsCEcfrWaswsQsgTjn9jjn1NUk5W7CBGjXDpo2DToSSQcVKviDlcmT/eg/yafJFCWpbN8Os2er+Uria/Bg+PFHv+9JPiUQSSrvv+/nJlICkXjq39/Pj/Xuu0FHkliUQCSpTJjgL1vbuXPQkUg6qVHDz9CrBHIoJRBJGjk5fiTMoEFQsWLQ0Ui6GTzYXz75iy+CjiRxKIFI0pg7F7ZtU/OVBCNv1J9qIfmUQCRpTJjg573S2ecShJYtoVUrJZCClEAkaUyYAL16Qe3aQUci6WrwYMjKgt3FnVKdRpRAJCls3Aiffur7P0SCMngw/PSTv5CZKIFIknjvPX87cGCwcUh669ULatZUM1YeJRBJCpMmwTHHQJs2QUci6SwjA844wycQ54KOJnhKIJLw9u+HKVN87UPXPpegDRoEmzfD8uVBRxI8JRBJeHPmwM6dar6SxHDWWf42r1k1nSmBSMKbNMkP3+3XL+hIRKBJE2jbVgkElEAkCUyaBD17+s5LkUQwYADMmgW7dgUdSbCUQCShbd4MS5eq+UoSy4ABsG+fPycknSmBSELT8F1JRD17QrVqasZSApGENmkSNG7s25xFEkWVKnD66UogSiCSsPbvhw8+0PBdSUwDBsDatbBmTdCRBEcJRBLWvHn+KnADBgQdicjh8vbLdK6FKIFIwpo82V+PWsN3JREdd5yfoVcJRCQBTZ4MXbpAnTpBRyJSuAEDYPp02Ls36EiCEUgCMbMBZrbKzNaY2R+KWa+TmeWY2YXxjE+Ct20bLFiga39IYhswALKz/Tkh6SjuCcTMKgJPAAOBNsBlZnbYFHmh9R4E3o9vhJIIpk2D3FwlEElsmZl+gsXJk4OOJBhB1EA6A2ucc1845/YBrwDnFrLeLcD/gG/iGZwkhsmToVYt6Nw56EhEila9OnTv7kcLpqNKAbxmY2BjgfubgC4FVzCzxsD5wOlAp6I2ZGYjgBEADRo0ICvdTwsN2bVrV1KXhXPw9ttdad9+J7Nnl23K02Qvi1hSWeSLZVkcd1xTnnnmWMaP/5B69fbHZJvJIogEUtiI/vCZ9R8B7nDO5VgxJwA458YAYwBat27tMjMzYxRicsvKyiKZy+Lzz+Hrr+Huu6uW+X0ke1nEksoiXyzLonp1eOYZ2Lu3B+lWvEE0YW0CjilwvwmwJWyd04BXzGwdcCHwpJmdF5foJHB57cnq/5Bk0KED1KuXnv0gQdRAFgDHm1kLYDNwKTC04ArOuRZ5/5vZ88AE59ybcYxRAjR5sh9jf+yxQUciUrKKFf25Sh984Jtf02nWhLjXQJxzB4Cb8aOrVgKvOueWm9n1ZnZ9vOORxLJvnx9Xr9qHJJMzzoAtW2DlyqAjia8gaiA45yYCE8OW/buIdYfHIyZJDHPn+mssKIFIMjnjDH/7wQfQ5rCTElKXzkSXhDJ5sm8S6Ns36EhEIte8uZ/WJN2G8yqBSEKZMsWf+1G7dtCRiJTOmWf6C0zt2xd0JPGjBCIJY8cOP32JJk+UZHTGGbB7t2+GTRdKIJIwZszw05f07x90JCKl17evb35Np2YsJRBJGFOmwBFHQNeuQUciUnq1a/vmVyUQkQBMnQq9evnLhYoko/79fTPsjh1BRxIfSiCSELZuhRUr1P8hye30030z7MyZQUcSH0ogkhCmTvW36v+QZNatG1Stmr8/pzolEEkIU6f6+YROOSXoSESiV6UK9Ozpr2eTDpRAJHDO+QTSt6+/BrpIMuvXD5Ytg2/S4EpG+rpK4Favho0b1XwlqeH00/3t9OnBxhEPSiASuLz2YnWgSyro0MEP6U2HfhAlEAnc1KlwzDF+LiGRZFepEvTpowQiUu5yc31Vv1+/9LqOgqS2fv3giy9g3bqgIylfSiASqKVLYdu2/HZjkVSQtz+n+mgsJRAJVF5Ho6Zvl1Ry0knQsKESiEi5mj7d9300aRJ0JCKxY+ZrIVOn+mHqqUoJRAKTk+OnfFDtQ1JRv37w1Vfw2WdBR1J+lEAkMIsW+UnnlEAkFeXt11lZgYZRrpRAJDB5X6zMzCCjECkfxx7rm2aVQETKwfTpcMIJcPTRQUciEntm/uAoKyt1+0GUQCQQBw7ArFmqfUhqy8z0c2Klaj+IEogE4uOPYedO9X9Iasvbv1N1XiwlEAlE3hdKNRBJZS1a+Gl6UrUfRAlEAjF9ev7JViKpKtX7QZRAJO727YPZs9V8JekhMxO+/RZWrgw6kthTApG4W7gQsrOVQCQ95DXTpmIzlhKIxF3eF6l370DDEImLVO4HUQKRuJs5E9q2hSOPDDoSkfKXyv0gSiASVwcOwIcfqvYh6SWvH2TFiqAjiS0lEImrRYtg1y5/xTaRdJGq54MogUhczZjhb1UDkXTSvLnvB5k5M+hIYksJROJq5kxo1QoaNQo6EpH4MfMHTTNnplY/iBKIxE1Ojp//Ss1Xko5694avv4Y1a4KOJHaUQCRuPv0Utm9X85Wkp7z9PpWasZRAJG7y+j9UA5F01Lo1NGigBCISlRkz8jsTRdJNwX6QVKEEInHhnP/iqPYh6ax3b1i3DjZsCDqS2FACkbhYuRK++079H5Le8vb/WbOCjSNWlEAkLvKq7aqBSDo7+WSoXTt1mrGUQCQuZsyAn/0Mjj026EhEglOxIvTsqQQiEjHnfJW9Vy/fkSiSznr39tdI/+aboCMpOyUQKXfr18PmzT6BiKS7VOoHUQKRcjd7tr9VAhGBDh2gWrXUaMZSApFyN2uW7zg86aSgIxEJXkYGdOumBCISkdmzoUcP34EoIr42vnQp7NgRdCRlowQi5er77/1FdHr2DDoSkcTRsyfk5sLcuUFHUjZKIFKuPvzQ36r/QyRfly6+Rp7XP5islECkXM2a5dt8Tzst6EhEEkeNGnDqqUogIsWaPRs6dYKqVYOORCSx9OgB8+bB/v1BRxI9JRApN9nZsHChmq9ECtOzJ+zZA4sWBR1J9JRApNzMnw8HDqgDXaQwPXr422RuxlICkXIze7afuqR796AjEUk8Rx8Nxx2nBCJSqFmzoG1bqFs36EhEElPPnj6BOBd0JNFRApFyceAAzJmj5iuR4vToAd9+C6tXBx1JdJRApFwsWwa7dimBiBQn7/uRd75UsgkkgZjZADNbZWZrzOwPhTw+zMyWhv7mmFn7IOKU6M2Z42/zOgpF5HAnnAD16ydvP0jcE4iZVQSeAAYCbYDLzKxN2GpfAn2cc+2A+4Ax8Y1SymLs2LH8/vdvAZvp3bs5Y8eODTokkYRk5g+ylEAi1xlY45z7wjm3D3gFOLfgCs65Oc65H0J35wJN4hyjRGns2LGMGDGC7Oz2wIds2LCeESNGKImIFKFnT/j88+S8wFQQCaQxsLHA/U2hZUW5BphUrhFJzIwcOZLs7DpAc8C3Y2VnZzNy5MgAoxJJXHnNvMnYD1IpgNcs7KKmhQ5iM7O++ARSaFesmY0ARgA0aNCArKysGIWY3Hbt2hVYWWzYsAG4IHTvw0OWBxFTkGWRaFQW+RKpLPbtMypX7sW4cZuoW/eLoMMplSASyCbgmAL3mwBbwlcys3bA08BA59z3hW3IOTeGUP9I69atXWZmZsyDTUZZWVkEVRZNmzZl/foeQDaw+JDlQcQUZFkkGpVFvkQri06dYNOmpmRmNg06lFIJoglrAXC8mbUwswzgUuDtgiuYWVNgPHCFc+7zAGKUKI0aNYoKFXoC84EDAFSrVo1Ro0YFGpdIIuve3c8b99NPQUdSOnFPIM65A8DNwPvASuBV59xyM7vezK4PrfZnoD7wpJktNrOF8Y5TonP++cMw60CtWssxM5o1a8aYMWMYNmxY0KGJJKxu3XzySLaJFYNowsI5NxGYGLbs3wX+vxa4Nt5xSdktWAA5ORV4+eWbGDz4pqDDEUkK3br52zlzoGvXYGMpDZ2JLjGVN5Ik7wshIiU7+mho0SL/BNxkoQQiMTVnDpx4ItSrF3QkIsmle3f//UmmiRWVQCRmcnP9F0DTt4uUXvfusHUrbNgQdCSRUwKRmFm1Cn74QfNfiUQj78ArmZqxlEAkZvL6P1QDESm9tm2hRg0lEElTH33k+z5atQo6EpHkU6kSdOmiBCJp6qOP/BBEK2yyGhEpUffusGSJv5ZOMlACkZjYvh1WrtTwXZGy6N4dcnL8+VTJQAlEYmLePH+rBCISvS5d/G2yNGMpgUhMzJ3rm646dQo6EpHkVbcutGmjBCJp5qOP/CiSWrWCjkQkuXXr5mv0yXBCoRKIlFlurq+BJNMcPiKJqmtX+P57WLs26EhKpgQiZbZqFezYof4PkVjIOxCbOzfYOCKhBCJl9tFH/lY1EJGyO/FEqFlTCUTSxNy5UKcOtG4ddCQiya9iRejcWQlE0sRHH/nhhxW0N4nERNeu/oTC7OygIymevvJSJj/+CMuXq/9DJJa6doUDB+CTT4KOpHhKIFIm8+f74Ybq/xCJnbwTCvP6FxOVEoiUSd4OnrfDi0jZNWgAxx2X+P0gSiBSJnPn+jNn69QJOhKR1NK1qz9AS+QTCpVAJGrO+TNmVfsQib2uXf0VCjdtCjqSoimBSNTWrvVnzKr/QyT2kuGEQiUQiVreDLyqgYjEXrt2ULWqEoikqHnzoFo1OOmkoCMRST0ZGdCxoxKIpKh58/wOXqlS0JGIpKauXeHjj2HfvqAjKZwSiETlp59g8WI1X4mUpy5d/Hdt6dKgIymcEohEZckSf1SkBCJSfvK+X/PnBxtHUZRAJCrqQBcpf8ccA0cdpQQiKWbePDj6aGjSJOhIRFKXmZ+ZN++ALdEogUhU8k4gNAs6EpHU1qULfPaZv2hbolECkVLbtg3WrPFHRiJSvvK+ZwsXBhtHYZRApNTy2mPV/yFS/jp18reJ2IylBCKlNm+eb7o67bSgIxFJfXlX+0zEjnQlECm1efP8DLy1agUdiUh6yOtIT7SZeZVApFSc80dCar4SiZ/OneGrrxJvZl4lECmVvBl4lUBE4idRTyhUApFSWbDA32oElkj8tGvnJ1dUApGkNn++n2JaM/CKxE+VKnDKKYk3EksJREplwQLo0AEqVw46EpH00qWLPxckJyfoSPIpgUjEDhyATz7JH5cuIvHTuTPs3g0rVwYdST4lEInYihWwZ48SiEgQ8jrSE6kZSwlEIpbXga4EIhJ/LVv6kwrzvoeJQAlEIjZ/PtSu7XdkEYmvvNkflEAkKS1Y4HfgCtprRAJx2mmwbBns3Rt0JJ5+CiQie/f6HVfNVyLB6dQJ9u9PnEvcKoFIRBYv9qOwdAKhSHDyDuASpRlLCUQiog50keA1aQINGyqBSJKZPx8aNYLGjYOORCR9mfmDuES5uJQSiERkwQK/4+oStiLB6tTJn0y4a1fQkSiBSAR27IBVq9R8JZIITjsNcnP9rBBBUwKREn38sb9VAhEJXt73MBGasZRApETqQBdJHA0bQtOmidGRrgQiJVqwAFq0gPr1g45ERMAfzCmBSFL4+GPVPkQSyWmn+auD/vBDsHEogUixvvsO1q2Djh2DjkRE8iRKP4gSiBQrrwP9tNOCjUNE8uUd0AXdjBVIAjGzAWa2yszWmNkfCnnczOyx0ONLzaxDEHFKfgLpoE9AJGHUqQPHH5+GCcTMKgJPAAOBNsBlZtYmbLWBwPGhvxHAv+IapBy0cGH+dQhEJHEkwhnpQdRAOgNrnHNfOOf2Aa8A54atcy7wovPmAnXM7Oh4Byq+BqLmK5HE07EjbNoE33wTXAxBJJDGwMYC9zeFlpV2HSln334LGzaoA10kEeUd2OU1MwehUgCvWdhsSi6KdTCzEfgmLoCfzOzTMsaWKo4EvovVxm6/3f8lqZiWRZJTWeRLmbIYNKjMm2gd7RODSCCbgGMK3G8CbIliHZxzY4AxAGa20DmnxhZUFgWpLPKpLPKpLPKZWdQ9KUE0YS0AjjezFmaWAVwKvB22ztvAlaHRWF2BHc65rfEOVEREihb3Gohz7oCZ3Qy8D1QEnnXOLTez60OP/xuYCAwC1gDZwC/iHaeIiBQviCYsnHMT8Umi4LJ/F/jfATeVcrNjYhBaqlBZ5FNZ5FNZ5FNZ5Iu6LMz/VouIiJSOpjIREZGoJF0C0TQo+SIoi2GhMlhqZnPMrH0QccZDSWVRYL1OZpZjZhfGM754iqQszCzTzBab2XIzmxHvGOMlgu9IbTN7x8yWhMoiJftbzexZM/umqFMdov7ddM4lzR++030tcCyQASwB2oStMwiYhD+XpCswL+i4AyyL7kDd0P8D07ksCqw3Dd//dmHQcQe4X9QBVgBNQ/cbBh13gGXxJ+DB0P8NgG1ARtCxl0NZ9AY6AJ8W8XhUv5vJVgPRNCj5SiwL59wc51zeFQPm4s+nSUWR7BcAtwD/AwKc/KHcRVIWQ4HxzrkNAM65VC2PSMrCATXNzIAa+ARyIL5hlj/n3Ez8eytKVL+byZZANA1KvtK+z2vwRxipqMSyMLPGwPnAv0ltkewXrYC6ZpZlZh+b2ZVxiy6+IimLx4ET8ScqLwN+7ZzLjU94CSWq381AhvGWQcymQUkBEb9PM+uLTyA9yzWi4ERSFo8AdzjncvzBZsqKpCwqAR2BfsARwEdmNtc593l5BxdnkZTFWcBi4HTgOOADM5vlnPuxnGNLNFH9biZbAonZNCgpIKL3aWbtgKeBgc657+MUW7xFUhanAa+EkseRwCAzO+CcezMuEcZPpN+R75xzu4HdZjYTaA+kWgKJpCx+Afyf8x0Ba8zsS+AEYH58QkwYUf1uJlsTlqZByVdiWZhZU2A8cEUKHl0WVGJZOOdaOOeaO+eaA68DN6Zg8oDIviNvAb3MrJKZVQO6ACvjHGc8RFIWG/A1MczsKPzEgl/ENcrEENXvZlLVQJymQTkowrL4M1AfeDJ05H3ApeAEchGWRVqIpCyccyvN7D1gKZALPO2cS7mZrCPcL+4DnjezZfhmnDuccykxS29BZvZfIBM40sw2AX8BKkPZfjd1JrqIiEQl2ZqwREQkQSiBiIhIVJRAREQkKkogIiISFSUQERGJihKIiIhERQlERESiogQiIiJRUQIRKQdm1tLM9pvZPWHL/2VmO80s5WYEkPSjBCJSDpxza/CTWP7GzI4EMLM/A1cD5zvnFgYZn0gsaCoTkXJiZo3wV8R7EvgMGANc5px7NdDARGIkqSZTFEkmzrmvzOwR4Hf479qvlDwklagJS6R8rQaqAB85554IOhiRWFICESknZnY68B/gI6CHmbUPOCSRmFICESkHZtYBeBPfkZ6Jv3DRAwGGJBJzSiAiMWZmLYFJwGTgFufcPuAe/GV0ewcanEgMaRSWSAyFRl7Nwdc4znLO/RRaXhH4FPjBOdc9wBBFYkYJREREoqImLBERiYoSiIiIREUJREREoqIEIiIiUVECERGRqCiBiIhIVJRAREQkKkogIiISFSUQERGJyv8HytujdGlsQmAAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# three data point example\n", "xi = [0.3, 0.5, 0.8]\n", "yi = [0.2, 0.8, 0.6]\n", "\n", "# use a function to construct the matrix above\n", "# note than numpy already has a function to do this\n", "V = np.vander(xi, increasing=True)\n", "\n", "print('V = \\n{}'.format(V))\n", "\n", "# use a numpy linear algebra solver to solve the system\n", "# uses an LU algorithm - see Lecture 3 for details!\n", "a = np.linalg.solve(V, yi)\n", "\n", "# output the coefficients for our quadratic we have computed\n", "print('\\n Our coefficients a = \\n{}\\n'.format(a))\n", "\n", "# show that they are the same as is obtained from \n", "# numpy's polyfit function (for a quadratic)\n", "# (which of course they should be, given we argued that this polynomial is unique)\n", "print('The output from np.polyfit(x, y, 2) = \\n {}'.format(np.polyfit(xi, yi, 2)))\n", "\n", "# Note that the order is reversed because numpy.poly* assumes decreasing\n", "# rather than the increasing powers of x which we have used above\n", "\n", "# set up figure\n", "fig = plt.figure(figsize=(6, 6))\n", "ax1 = fig.add_subplot(111)\n", "\n", "# plot the raw data\n", "plot_raw_data(xi, yi, ax1)\n", "\n", "# x locations at which to evaluate and plot the quadratic polynomial\n", "x = np.linspace(0., 1., 100)\n", "\n", "# Set up a polynomial from the coefficients using numpy rather than writing out.\n", "# Use numpy.flip to reverse the coefficients as poly1d assume decreasing rather than\n", "# increasing powers - look at documentation\n", "p2 = np.poly1d(np.flip(a, 0))\n", "print('\\nWhich agrees with us as long as we reverse the order of our coefficients:')\n", "print('np.flip(a, 0) = \\n{}'.format(np.flip(a, 0)))\n", "\n", "# the p2 here is a function so evaluate it at our x locations\n", "y = p2(x)\n", "\n", "# and plot\n", "ax1.plot(x, y, 'b', label='Quadratic')\n", "\n", "# add a figure title\n", "ax1.set_title('Polynomial approx to three data points', fontsize=16)\n", "# Add a legend\n", "ax1.legend(loc='best', fontsize=14)\n", "# set bounds\n", "ax1.set_xlim(0, 1)\n", "ax1.set_ylim(0, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Notation reconciliation\n", "\n", "Note that to reconcile this with the notation we introduced earlier ...\n", "\n", "Our scalar problem \n", "\n", "$$ y = f(x; \\boldsymbol{a} ) \\equiv a_0 + a_1\\,x + a_2\\,x^2$$\n", "\n", "and data points $\\{(x_0,y_0),(x_1,y_1),(x_2,y_2)\\}$ have been combined into the system\n", "\n", "\\begin{align}\n", "\\begin{pmatrix}\n", "y_0\\\\\n", "y_1\\\\\n", "y_2\\\\\n", "\\vdots\n", "\\end{pmatrix} &= \n", "\\begin{pmatrix}\n", "f(x_0; \\boldsymbol{a} )\\\\\n", "f(x_1; \\boldsymbol{a} )\\\\\n", "f(x_2; \\boldsymbol{a} )\\\\\n", "\\vdots\n", "\\end{pmatrix}\\\\[10pt]\n", "\\iff \n", "\\boldsymbol{y} &= \\boldsymbol{f}(\\boldsymbol{x}; \\boldsymbol{a} ) \\\\\n", "&= V\\boldsymbol{a}\n", "\\end{align}\n", "\n", "which is equivalent to the notation\n", "\n", "$$ G \\boldsymbol{m} = \\boldsymbol{d} $$\n", "\n", "where the data is $\\boldsymbol{d}=\\boldsymbol{y}$, the model parameters are $\\boldsymbol{m}=\\boldsymbol{a}$, and where the governing \"model\" is linear and given by the Vandermonde matrix ($G=V$). \n", "\n", "So even though our original model $y = f(x; \\boldsymbol{a} )$ wasn't in this form, by considering all of the data together we do end up needing to solve a linear (in $\\boldsymbol{a}$) problem." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Problem leading to a non-square linear/matrix system\n", "\n", "In this module it's very very important we understand and are able to deal with situations that result in non-square problems.\n", "\n", "As an example, let's suppose we have a case where our model is quadratic, i.e. it has three model parameters:\n", "\n", "$$ y = f(x; \\boldsymbol{a} ) \\equiv a_0 + a_1\\,x + a_2\\,x^2,$$\n", "\n", "but let's further suppose we have been given six distinct pieces of data, i.e. we now have a mismatch.\n", "\n", "We can write this mathematically as\n", "\n", "$$\n", "\\begin{pmatrix}\n", "1 & x_0 & x_0^2 \\\\\n", "1 & x_1 & x_1^2 \\\\\n", "\\vdots & \\vdots & \\vdots \\\\\n", "1 & x_5 & x_5^2\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "a_0\\\\\n", "a_1\\\\\n", "a_2\n", "\\end{pmatrix}\n", "=\n", "\\begin{pmatrix}\n", "y_0\\\\\n", "y_1\\\\\n", "y_2\\\\\n", "y_3\\\\\n", "y_4\\\\\n", "y_5\n", "\\end{pmatrix},\n", "$$\n", "\n", "or equivalently $V\\boldsymbol{a} =\\boldsymbol{y}$ where now we have a non-square version of the Vandermonde matrix (read the `numpy.vander` docs).\n", "\n", "In both the initial problem description, as well as this linear/matrix system, what does it mean to be a solution to this problem?\n", "\n", "\n", "
\n", "\n", "Consider the simpler case of a *linear* (in $x$) model asked to pass through *three* data points. If the three points happended to lie along a line then we can solve the problem, i.e. we can invert for the model parameters. But if not there is no solution (at least no exact solution) that fits all the data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Least squares solution\n", "\n", "In the language of our Numerical Methods module we are now in the world of curve-fitting rather than interpolation - we can't find a model (curve) that fits all the data exactly, but can still obtain a potentially useful model if we appropriately define \"useful\".\n", "\n", "
\n", "\n", "Recall that in Numerical Methods we used the command `numpy.polyfit` to fit a polynomial to data where the polynomial wasn't of a high enough degree to go through all the data points (or some of the data points weren't at distinct $x$ values - i.e. two $y$ values for the same $x$, think a cloud of points).\n", "\n", "
\n", "\n", "Instead of requiring the curve to go through all data points exactly (as for the cases above), we used *least squares* fitting to construct a function $f(x;\\boldsymbol{a})$, e.g. a polynomial in $x$ of degree $N$, (equivalently finding the parameters $\\boldsymbol{a}$) which minimises the sum of the squares of the differences between $M+1$ data points provided and the polynomial approximation, i.e. it minimises the quantity:\n", "\n", "$$E := \\sum_{i=0}^{M} (f(x_i;\\boldsymbol{a}) - y_i)^2,$$\n", "\n", "where $f(x_i;\\boldsymbol{a})$ is the output of our model evaluated at point $x_i$, and the $y_i$ are the corresponding data values. \n", "\n", "Note that for the \"cloud of points\" example we will be in the case where $M+1 > N$.\n", "\n", "\n", "\n", "*(Wikipedia: https://en.wikipedia.org/wiki/Linear_least_squares) We're computing the sum of the squares of the distances indicated in green.*\n", "\n", "
\n", "\n", "
\n", "\n", "Note that this error $E$ can be defined equivalently as\n", "\n", "$$E=\\| \\boldsymbol{f}(\\boldsymbol{x}; \\boldsymbol{a} ) - \\boldsymbol{y}\\|_2^2\n", "= || V\\boldsymbol{a} - \\boldsymbol{y}||_2^2,$$\n", "\n", "where $\\| \\cdot \\|_2$ is the 2-norm or the Euclidean norm acting on vectors.\n", "\n", "
\n", "\n", "[Why the square of $E$ here? As the 2 norm (Euclidean norm) includes a square root. We know that $E$ can't be negative (why?) and so minimising $E$ is completely equivalent to minimising $E^2$, i.e. we are minimising the 2-norm of the difference between the observed data and the model's prediction of the data.].\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will show in a later lecture that the coefficients ($\\boldsymbol{a}$) of the polynomial that minimises $E$ are given by the solution to the linear system\n", "\n", "$$V^TV\\boldsymbol{a} = V^T\\boldsymbol{y},$$\n", "\n", "where $V$ is again the Vandermonde matrix. \n", "\n", "
\n", "\n", "This is a very important result/equation in this module we will return to.\n", "\n", "
\n", "\n", "[As we have seen, $V$ is no longer square in the case where $M+1 > N$. \n", "What is the shape of $V$, and indeed all the vectors and matrices appearing in this equation - is it \"dimensionally\" consistent?]\n", "\n", "Let's check that this is true by forming and solving the matrix system $V^TV\\boldsymbol{a} = V^T\\boldsymbol{y}$ for $\\boldsymbol{a}$ and comparing with the result we get using `numpy.polyfit`." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "a = \n", " [ 0.75909819 -0.43193108 0.09889271 -0.00552147]\n", "\n", "poly_coeffs = \n", " [-0.00552147 0.09889271 -0.43193108 0.75909819]\n", "\n", "Our _a_ vector = output from np.polyfit (as long as we flip the order)? True\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhQAAAGKCAYAAABU/thLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABQ3UlEQVR4nO3dd3xT1f/H8ddp2RtkqCAtCDjw6wJBFLGCAoKi4gCtfgXBioKCE7E/FEdFvw4QBbEiOKh7i4gDKTJkqSgKDsAWGYqIILTI6vn9cdKSpukiaW/Tvp+PRx6Q25ubT05Obj456xprLSIiIiKhiPI6ABEREYl8SihEREQkZEooREREJGRKKERERCRkSihEREQkZEooREREJGQlmlAYYwYYY6wxplVJPk9xGGPGGGO6eh2HlD3GmFRjTKoHzzvCGNM3yPYxxpgyO6/bGHO0MeZzY8w/vs/5hV7HVJCino+MMbG+/QaE4TmDvreRIPDzYIw50VcnGwTZ1xpjxhzEc8T5HhsXSqxFeJ6DjW+AMeaaEggpbPzqdexBPjZsr68itlDcAyihkGBu8N1K2wgg2JfOFKBT6YZSLI8DLYHLcHHO9TacsNmEez0fhuFYIwj+3kaiE3HnzzwJBa68ppRqNKVjAFCmEwpcPe2Eq7fFNYAwvr5K4TqQeMsYY4DK1to9XsdSGkri9VprV4brWOFgrV0PrPc6jgIcA3xhrZ3ldSDhZK3dDSzyOo5IYq1VeXnEWvsn8KfXcQBgrS2xGy77sUCrQva7FvgW+BfYAjwHNAjYZxjwJbAV2Ib7wPcO2KcScD+wxu9Y84HOvr/bILcxBcTVBngH2Ow73jrgDaCS3z4nAfN8f98AjAbudUWbs0+s77kGBBw/zrc9zm9bd2AmLtvMBL4HbgWiAx6bBkzHZZc/AnuBi3x/OwF4H/gb2AUsAM4IePwpwKfAX77nWQtMKuR9qgaM88W0E/gd+AA4Op/3vQvwrm/fv4CJQPUg5XID7tfuZl8sM4DYYrzenr66sQvY7nvOo/we+5KvzsT4bTsM9yF8w29bKpAa5P25EHgGV/f+9pVBtK8M5wMZwA9AjyBl/CYuKdgF/AQ8GFAGaeStk8/7/jYGv3rk21YHeArYCOz2HfNmwASJu49v3y2+1zodqFeEz21l4AFfbHt8/z6AS+D8j5/rVsDxGvnK72ff+/sb8DLQtIjnkRa+9/B332teCzyR3/sWULbPh1AvAz+vZ+I+M9t97/m3wKAC4s73vS1KvS3guM/76lR7YKFf3ert+/stvuf+B3gPaHSQ56KccvUru8BbrN+5dYzfY8f4tv0HmON73zcB9wFRBT2vb3tf3Dk+E/fZfQNoXoSyicbV1ezzZyrQNkh8rXB16ldf+a0FngbqB7z+wNebXR4HXacpYj30O0+9iPsM7wa+A67M53ixftvScJ/3/sAqXH1dhu+7sAiv71DgBQ6cZzbhzsuNC3ptnrdQGGMewn1hTgBuB5riKsRxxpjTrLX7fbvG4prU0nCJw/nADGNML2vtR759RuJOronActzJtz0Hmug64T7Az+MqAxT8C3AGrjJfj3tDmwK98HUVGWMaAp/jTnRX4wr+dqB58Uohl5bAbOBJXJLSHvfhbATcGbDvWbhmyHtxX8ZpxpiTcQnON7hELRMYAnzmK8+vjDG1gI+BJbjKuANXvqcVEltVoDYHPrANcMnAImPM0dba3wP2nw68DkwCOgB3AzV9z+lvFO79Ggg0xn3pfmKMaWut3VvI6+2Ja/L7HOgH1MKdtOYbY0601m7wxdgJeNkY0wX3wZnuK5trC3nNAOOBt33H7wL8H64Ong08gksk/w942xgTY63d4ntcc9/reh5Xxm19ZdAS90EHuAiXQH6Le58hn18bxpgo32s92XecFUBvXDLWCLgr4CFP4OrwFcBRwP+A/bi6WpAXcN0YD+ISpk6+19fSd6yvfdveB5bikviCNMDV5VG+13Y47jO/wFdv/s3vgcaYFrh6molrbv8FOAKXeB+sotZL/zguAN7CJefX4c4HbYGYAp4n3/e2iPW2IHVwXzSP4k76icBbxpiJuB9CQ4EmuLo7Efd+huJD3Of+/4BLOXDeLKyZ/V1gKjAW6IH7wZXFgfLIwxgzBPflPg1XJrV9+881xhxvrd1RwPONwX0OHgc+wZ0/3w+y3+G+1zAC9yOhpe9xMznQzXgDrq5E495zcEkahFCn/RRYD40xNXHdiPV9sf0GXAm8ZIypYa1NLuT4Z+A+96N9sd6P+86MtdZuK+T1vYSr27f7nrcJ0A2oUeAzFuUXwsHeKKSFAvclth+4O2D76b7HXZjP46JwJ/RPgPf8ts8A3i4kJgs8UITYG/r27VPAPkm4X3DN/bbVxJ1sbMDrLNKvgoC/G9/rTMRVev/MPg13kj004DGzcRlplYCsfRXwru9+e9/zHh/i+xvtq2A7gJuDvO+TA/ZP9L3fbQLKZWXAa8t+/wcV4fUuw33J+LcatcC1YDzut62D7726F/fh3EfeVptUgrdQTA3Y72vfdv9s/3jftqsLeS+vxJ1QDwl4bdODPGZMQD06L596NAWXzDYMiPuFgP2ewp1YTLAYffscR5CWO9wXSa46gzshP3+Q9eYI3/EuKmTfF3G/4A4vYJ9c71tAuT7vd7+49XKA33uX5qtrUYW9viAxBHtvi1Rv8znm8774ugSpfz/h15qJ+2Ldm70t8LUFqetxfttylSsFnM8D6wwHWijuDNjvWdz5ol6w58UlVtvJ+5mLxX1+RxRQLvV9dSXw/R0ZrE4H7FMJ6Ozb76SAMpgf5jpd1Ho4LPA98W3/DPeDKjrgeLEB9e5vcre4ZJ/3ryjs9fnK8abi1HVrreeDMs/BJQcpxphK2TdgMS5T6pK9ozGmnTFmhjHmD9yXwV7f44/yO95SoJcxJskY09kYUyWE2P7CNYM9ZIy51hjTOsg+nYBF1tp12RustRm4boCDYow5zBjzjDEmHfcB2ov7ZVAP9+vd3yLr1ypgjKmOa5Z9A8jyK0+Dq4TZ5fkLruXlGWPMlcaYI4oR32XGmMXGmG249yEDdxI4KsjurwfcfxX3fncI2P6mtTYr+461dgHuyypwQGLg662J+7X+mrV2n9/jf8X9kjzTb9sS3C+ARFxSkWStnVfoC3Y+Crj/I5BhrZ0fsA3cSSU7vjrGmIeNMWtwX/h7cZm/AYLVp8J0wSUjrwRsnw5UIW95BQ4qXIFrZWpSyHNkHzPwOcCvTIvDGHO9MeZbY8xOXL3J/swEqzf+ugMzrLUbD+Z581HUepntKNyvtSn+9fRgFafeFiDDWvuF3/3s+veZPdCqm729Eq7p3AvByroWLnENphOu9SXwO2E97rV0yedx4LpXaubznLkYY6oYY+4yxvxojNmF+2xmnw8Kq5PZxzjYOp2tsHrYBdhgrU0N2G86rkXy2EKO/6W19m+/+yt8/xalBX0pcLsxZrgx5j++MWuF8jqhyP6CXI17Q/1vdYBDAHxfeLNxzUw34prmTwFm4fr1sz2Iaxbtg6scfxljpvm6JorFujTtHNwvibHAz8aYtcaY6/12Owz4I8jDg20rlK9J+33cL9EHcLNRTsG1hEDu1wp5mxsb4DLl0eQtz2FAfWNMlLV2O677YCOuuW2dMeZ7Y8zFhcR3PvAarrXjCqCjL74/g8QGecsh+37TQvbL3ha4X+DrrY/7cg7W7Po7eUejv8yBvsKJQR6Tn78D7u/BJWQ57IHBof7lMA3X3TQBV5dOwTVFB+5XVA2ArdYNGvT3u9/f/W0NuJ/9uIKeO/sYgWWa33MUyhhzI66efYbrG+8AnFqEWMCdA8I9MLWo9dI/BsIYR3HrbTDb/O/41b9gdRUOrr6FQ3HLOvs74TPynsP+w4H3IpjspCm/5/Q3FteKMh3XbdiBA7NxCi2rEOt0fnEFlk0D8q8j2X8vSK7Pv995oyjx9cN9F92BG7exwRhzt+87Kl9ej6H4y/dvd/J+EPz/3hOoC1xm3ch3AIwxufpzrOtvfxh42BhzKO6L+XFcs3y/4gZnrV0L/NeXnZ2A+1KeZIxJs27cxiaC/9oL3JbdnxbYYhL44TgS1yx1lbU25xei74s8aIgB97fhfsFOxDUV532A7xeWtXY5cLEv+2+P6wt83RhzgrX2+3yerz+w2lo7wC+2yuRfsZvgBiv63wc35iBwv2CPXR4YfsD9v33bDg3y+EM5UH+yk7UXcF8KtYHJlOB0PmNMNeACXDPrE37b/xPCYbcCDYwxVWzu2S3Zr/+vII85mOfIPuaaMD1Hf2C2tfbW7A2+sRFFkT12qSD/4n6ABAq1XvrHQBHiKKoi19sSUNRzUbg0wbX0+t+H/Ms6+7UPIPd7lK2g8RPZX775vb/++gMvWmsfyN7gG1tWVKHUaf+4CqqHWwne2hHOz3tQ1trNuB8/Q40xR+HGXd2L+/H4dH6P87qF4lPcF2Bza+2yILdffftlJw45A/SMMW1wfe1BWWt/t9ZOwWWQ/s1re4DqxQnSOstxo6fxO96XwKn+XQa+5szABOAP3K/DwGa+3gH3g73OykB8EePMwLXMnAB8HaxMgzxmn3VTvkbj6sMxBTxFDVzTnr+rcK0iwQQOBOuPe7+XBGy/xD/zNcacDjTDlW++fK/3K+BSY0xODMaYGFwrlv+6CKNwfaRX4GaKXGSMuY6SUxVXLnsDtg8Isu9uilYn5+Leo0sDtsfj6nU4pu5ll1n/gO3ZdfALiq8GecthYBEf+wlwnjGmoCb7dKCNfxenb/Bt7Xz2L2q9zPYzrk96cFGbfv3keW+LWW/DrajnomCyf+EW5/wZrKx34maKBbMQlzS0yuc74acCnus7XBdssOcMVNQ6md9nM5Q6na2wejgXaOY7H/q7AjeGYlUxny+YQs891tqfrLV34RLh/LqqgNJroehpjAmcAbDdWvupMeZh4ClfFjQXl0EfgWsinmKtnYNLCvYBLxpjHsM1bd2L67Py/yJ6Dzei+mvciz8J17rxDAesBHobY2b59tkYrH/WGHM8bpT8a7gumWjcl8E+3MhscNMHb8DNSBjDgVkeu/yPZa21xpjXgEHGmJ/xTfHCDUjytwp3ckwyxuzHVdibA2MrxC24k/7HxpjncFl7Q1yfbbS19k5jzHlAAm4E9q+4fsebcB/kgr7EZwEXGmPG4QbAtvM9bls++/cyxjyC+1LogOuOetFa+3PAfrWBd40xz+D6BsfixnkEbWUJMBo3VmCGMWYSrn/2XtzArscAjDEdcc2b91prv/RtmwQ8boz5wlobjg9mLtba7caYRcCtxphNuF+51xD8V+5K4Azf+/I7sMVamxZkv49wsy4mG2Ma4X7d9AIGA2PtgdklocT9gzHmFWCMr/VqIa5fezTwirX2u4M47CxgpDHmLtzJsitwSREfew/us7LQGPMg7rPYFOhprb3St8+ruPo81RjzPG5w4y24OhBMUeslkPP5HYGb6fO5MWYy7pfaMbhpdPcUEH9+722h9bYkFONcFEz2Oi1DjTEv4M5P39mC14K51vdjYSlulsdgXKvdtnzi+8cYczsw0VfHP8KVSVPc2JJUa+3L+Tx2m+/clGiM2YF7f08BBgXZfRZwtTFmBa5O9SX4LLeVwA3GmH64FrsdvqQmlDqdrbB6+DwwHDd7LBHXuhqP+268LmCszMHK8/pw9fQzIIUDU/QvwHXVfVLg0Yo7irM4N/Kfu2yB7/32uwr36yoDl72uwo1Ib+a3z2W+F/cv7kTaH1fgaX773Oo7zl8cmJs9Bt/8ed8+p+N+HfxLASN/cX15L3BgnvFWXMITuNZA9jTNfNeh8O1XDzcgb4vvWJNxH+TAkdUn4r40MnEV6D7chzDYKN48o8d9fzsGd5LdjEty1uP6w3r5/n4ULlH61Rf3n7jpUh0LeT+jcGM7Nvrim4tL2tLIf77/e773dCuFr0Pxp++4HwItAp67oNcbOJ//PXzz+XHJyhpfrP4zSarhBiktB6raAyOeU/32ifPFd3bA8z0PrA8Sh8VvBpHv9X2E+5BuxtXpYO/50b46lEnR16HYhGuV+Jn816EIjDv7fYkNVo5++2WvQ5GOO5mk47cOhd9+RZrlgfsF9LTv/d2BS0ZbUMjIe7/HH4kbiJo9F38tMC5gn+twSeguXBLUjtDr5YCA5+iKW1Nhp+/2LTCwkNiDvreF1dtCjlmk+hfwmlv5batH0c5FqQTMnsF96W3AzUbIqUuB7yUHZnkc5yuzXbgvqvsp2joUvXyP+8f32NW46afHFlI22etQ/O57XCpu8GJgfA1x58i/fbcUXPKR633HdS/MxNVby4F1Gg66The1Hvr2PczvvSr2OhT51JExBb0+XOvqM7jv2Z2+92ApfrND8rsZ30EljHytFfdYa4vbPFouGHcNhGlAa2vt6gL2i8UlNdda1z0lIuVA9jkQl4QGdpNWaEU9P0Yir8dQiIiISDmghEJERERCpi4PERERCZlaKERERCRkSihEREQkZF6vlBk2DRs2tLGxsWE7XkZGBjVr1gzb8cRRuYafyrRkqFzDT2Uafl999dUWa20jr+OAcpRQxMbGsmxZnoUgD1pqaipxcXFhO544KtfwU5mWDJVr+KlMw8+4C0mWCeryEBERkZApoRAREZGQKaEQERGRkCmhEBERkZApoRAREZGQlZtZHgXJyspiy5YtbNu2jf37i3bF17p167JqVdival3hqVzDr6yUaXR0NPXq1aNhw4ZERem3ikhFUyESivXr12OMITY2lsqVK2NM4RcB3bFjB7Vr1y6F6CoWlWv4lYUytdayd+9e/vjjD9avX0/z5s09jUdESl+F+BmRkZFB06ZNqVKlSpGSCREpHmMMVapUoWnTpmRkZHgdjoh4oEIkFICaYEVKgT5nIhWXPv0iIiISMiUUIiIiEjIlFCLFlJWVxXXXXcchhxyCMYbU1FQGDBjAeeedd1DH+/vvv2nSpAlr1qwJW4xFiac4MW/evJlGjRqxfv36cIQnIuVQhZjlEakGDBjAli1bmDFjRoV8/rJq5syZTJs2jdTUVFq2bEmDBg046aSTsNbm7BMXF8dxxx3HU089VejxHnzwQXr16sWRRx5ZkmHn8cQTT+SKuSCNGzfmv//9L/fccw/PPfdcCUcmIpFICYWUW3v27KFKlSphP+7q1as57LDDOO2003K2HezzZGZmMmXKFD744INwhVdkdevWLdb+AwcOpH379jzyyCM0aNCghKISkUilLo8ItnLlSnr37k3t2rVp3Lgxl19+Ob///nvO35cuXUr37t1p2LAhderUoXPnznz55Ze5jvHMM8/Qpk0bqlWrRqNGjejRowf79u1jzJgxvPDCC3z44YcYY3Ka9oNZsWIF3bp1o06dOtSuXZsTTjiBOXPm5Px91qxZHH300VSrVo0ePXrw8ssvY4whLS0NgOeff55atWrlOmZqairGGLZs2QLAX3/9xeWXX06zZs2oXr06bdu2Zdq0abkeExcXx/XXX89tt91Go0aNOP3004tUToXF72/AgAHcfPPNrFu3Lmdtk+zt2d0HAwYMYO7cuUycODGn7LJfa6CZM2cSFRWVE2u2H3/8kT59+lC3bl1q1apFp06dWLFiRZ7nyjZmzBiOO+64PMd/4IEHaNKkCbVq1WLgwIHs2rUr12vxP461lscee4zWrVtTtWpVmjVrxqhRo3L+ftxxx3H44Yfz9ttvB30tIlKxVdgWihEjYPny/P++f391oqPD+5wnngjjx4fnWJs2baJLly4MGjSIRx99lL1795KYmEifPn1YtGgRUVFR7Nixg6uuuoonnngCYwxPPfUUvXr14pdffqFhw4YsW7aMoUOH8sILL9C5c2e2bdvG559/DsBtt93GqlWr2Lp1Ky+99BJAvr9Kr7jiCk444QSWLFlCpUqVWLFiBdWqVQPgt99+48ILL+Taa69l6NChLF68mDvuuKPYr/fff//l5JNPZuTIkdSpU4fPPvuM6667jubNm9OtW7ec/aZPn05CQgLz5s3DWlukcioo/kBPPPEEMTExTJ06laVLlxIdpJI88cQT/Pzzzxx99NE8+OCDADRq1Cjo8ebNm0e7du1yrY+yceNGOnfuzOmnn86nn35KvXr1WLJkSZFXec02d+5cqlevzuzZs9mwYQPXXHMNI0eOZMKECUH3v+uuu3j66ad5/PHH6dKlC3/++SfffPNNrn06dOjA3LlzGTx4cLFiEZHyr8ImFJHu6aef5oQTTuDhhx/O2fbiiy/SoEEDli1bRocOHejatWuuxzz55JO89dZbzJo1iyuvvJJ169ZRs2ZN+vTpQ+3atYmJieGEE04AoFatWlSvXp2qVaty6KGHFhhLeno6t912G0cffTQArVq1yhVn8+bNmTBhAsYYmjZtym+//cbo0aOL9XqbNm3K7bffnnM/ISGBzz//nFdeeSVXQtGiRQsee+yxnPt33313oeVUUPyB6tatS+3atYmOjs63XOrWrUuVKlWoUaNGkcrusMMOy7Vt4sSJ1KxZkzfeeCOnK6VNmzYFHieY6Ohopk2bRq1atTjuuON4+OGHGTRoEGPHjqVmzZq59t25cyfjxo1j/PjxXHPNNYArh06dOuXa7/DDD2fp0qXFjkVEyr8Km1AU1lKwY8cuz5czLshXX33FF198kaerAGDNmjV06NCBzZs3M3r0aObMmcMff/zB/v372bVrF+vWrQPgnHPOISYmhhYtWtCjRw+6d+9O3759i/26b7nlFgYPHswLL7xAt27duPjii3O+nFetWsWpp56a6xd44JdUUezfv5+HHnqI1157jQ0bNrB792727NlDXFxcrv3atWuX635Ryqmg+Evarl27aNKkSa5t33zzDZ07dw55/Mfxxx+f63V36tSJPXv2sGbNGo4//vhc+65cuZLdu3fnSs6CqV69eq5uExGRbBpDEaGysrLo3bs3y5cvz3X75ZdfcvrFr776apYuXcq4ceNYuHAhy5cvp1mzZuzZsweA2rVr8/XXX/P666/TvHlzxo4dy9FHH83GjRuLFcuYMWNYuXIlF154IQsXLuT4449n6tSpAEWaRRAVFZVnv7179+a6/+ijj/LYY49x++23M3v2bJYvX86FF16Y81qyBf7yLko5FRR/SWvYsCF///13rm2FlVlRyqu4ijrbY+vWrfl234hIxaaEIkKdfPLJ/PDDD8TExNCqVatct+wWhvnz53PjjTfSu3dv2rZtS+3atdm0aVOu41SqVImuXbsyduxYvvvuOzIyMnKmiVapUqXI/fatW7fmpptu4sMPP2TQoEFMmTIFgGOPPZbFixfn+sJatGhRrsc2atSIzMxM/vnnn5xtywMGuMyfP5/zzz+fq666ihNPPJEjjzySn3/+OSzlVFD8B6uoZXfSSSexcuXKPDHPnz8/T7KUrVGjRnnex8DyAjfY1P+6GosWLaJKlSpBp6cee+yxVK1aldmzZxcY7/fff8/JJ59c4D4iUjEpoSjj/vnnnzy/rtPS0hg6dCjbt2+nX79+LF68mLVr1/LZZ5+RkJDAjh07ANfvPn36dFauXMnSpUvp379/rmb0GTNm8MQTT/DNN9+Qnp7Oyy+/zI4dOzjmmGMAiI2N5fvvv+enn35iy5YtQX8F79q1i6FDh5KamkpaWhqLFy9m/vz5HHvssQAMGTKEtLQ0RowYwU8//cS7777L5MmTcx2jY8eO1KxZk1GjRrF69WreeustJk2alGufNm3aMHv2bObPn8+PP/7IsGHD+PXXXwstv8LKqbD4D1ZsbCxLliwhLS2NLVu2kJWVFXS/Hj16sGrVKv7666+cbTfccAM7d+7ksssuY+nSpaxevZpXXnklJ2no2rUr33zzDVOnTmX16tWMHz+eBQsW5Dn2vn37uOaaa/jhhx/49NNPufPOO7n22mvztOKAa60aPnw4o0aNYtq0aaxZs4YlS5bw9NNP5+yTmZnJV199Rc+ePUMqGxEpp6y15eLWrl07m5+VK1fm+7f8/PPPP8V+TLhdffXVFshzu/jii6211v7888/24osvtvXq1bPVqlWzbdq0scOGDbO7d++21lq7fPly26FDB1utWjXbsmVL++KLL9q2bdvae+65x1pr7bx582xcXJxt0KCBrVatmm3btq2dOnVqzvNv3rzZnnPOObZWrVoWsHPmzMkT4+7du+3ll19umzdvbqtUqWIPO+wwe+2119rt27fn7DNjxgzbpk0bW7VqVduxY0c7ffp0C9hff/01Z593333Xtm7d2larVs12797dvvTSSxawf/75p7XW2q1bt9qLLrrI1qpVyzZq1Mjefvvt9vrrr7dnnnlmzjHOPPNMO3To0DwxFlRORYk/0COPPGJjYmLyvFe9e/fOuf/TTz/ZU0891VavXj3Paw106qmn2qeeeirXtu+//96ee+65tmbNmrZWrVq2U6dOdsWKFTl/v+eee+yhhx5q69SpYwcNGmRHjRpl27Ztmyeee++91zZq1MjWrFnT/ve//7UZGRn5xrx//347duxY26JFC1u5cmXbrFkze9ddd+X8/eWXX7ZHHXVUvq8j28F83sqiYPVdQqMyDT9gmS0D38HWWiUU+SkLCUV59M8//9ilS5cW+iVbkXz00Ue2TZs2dt++fQf1+NKqq6eccopNSUkpdD8lFKVn+vTpNiYmxhpjbExMjJ0+fbrXIRUoEso00pSlhEJdHiIe69mzJ0OHDi3T18nYvHkzl1xyCZdffrnXoYhPSkoKCQkJpKenY60lPT2dhIQEUlJSvA5NKiglFCJlwE033URMTIzXYeSrcePG3HHHHbmm/4q3EhMTyczMzLUtMzOTxMREjyKSiq7CrkMh3mnfvr3rbxORg5a9nkxRt4uUNLVQiIhEoObNmxdru0hJU0IhIhKBkpKSqFGjRq5tNWrUICkpyaOIpKJTQiEiEoHi4+NJTk4mJiYGYwwxMTEkJycTHx/vdWhSQWkMhYhIhIqPj1cCIWWGWihEREQkZEooREREJGRKKOSgDBs2LM+lw0VEpOJSQlGGDRgwAGMMxhgqVapE8+bNuf766/Nc7joSpKWlYYzh66+/9joUEREpAUooiiElJYXY2FiioqKIjY0tlSVuzz77bDZt2kRaWhpTpkzhgw8+4IYbbijx5xURESkOJRRF5NW6+VWrVuXQQw+lWbNmdO/enX79+vHJJ5/k/H3//v0MGjSIFi1aUL16dVq3bs3//ve/nMtlr1q1CmMMv//+O+CW5q1SpQrnnntuzjGeffZZWrdunW8M+/fv57bbbqN+/frUr1+fESNGsH///lz7zJo1izPOOIP69evToEGDnMtyZ2vRogUAcXFxGGNyukuWLl1K9+7dadiwIXXq1KFz5858+eWXoRWaiIiUOiUURVQW1s1fu3Yts2bNonLlyjnbsrKyaNq0Ka+//jqrVq0iKSmJBx98kGnTpgFwzDHH0KRJE1JTUwFYsGABdevWZf78+ezbtw+A1NTUAsdDPPbYYzz77LM888wzfPnll+zfvz9PIpWRkcGIESNYsmQJqamp1K1bl/PPP589e/YAsGTJEgDefvttNm3axNtvvw3Ajh07uOqqq5g3bx5LlizhxBNPpFevXmzZsiUsZSYiIqXE68udhutW0pcvN8ZYIM/NGFPsYxfV1VdfbaOjo23NmjVttWrVcp7z8ccfL/BxI0eOtN26dcu5f9lll9mEhARrrbV33XWXHTJkiI2JibELFy601lrbtGnTAi97fNhhh9kHHngg5/7+/ftt69at7ZlnnpnvY3bu3GmjoqLsvHnzrLXW/vrrrxawqampBcaelZVlDz30UPvSSy8VuJ8cUFqXLy8qXb5c8qMyDT/K0OXLtbBVETVv3pz09PSg20tSly5dSE5OZteuXTz77LOsWbOGm266Kdc+kydPZsqUKaSnp7Nr1y727t2b68qVcXFxjB8/HnCtEcOHDyczM5PU1FQaNmzIhg0b8m2h2L59O5s2baJTp04526KioujYsSO//fZbzrY1a9YwevRoFi9ezJ9//klWVhZZWVmFXqho8+bNjB49mjlz5vDHH3+wf/9+du3apQsciUipysqC7dthyxZ327oV9u4Fa93frIUePaB2ba8jLbuUUBRRUlISCQkJubo9SmPd/Bo1atCqVSsAJkyYwFlnncX999/PmDFjAHjttdcYMWIEjz76KKeddhp16tRh4sSJvPPOOznHiIuL44YbbuCXX35h2bJlxMXFkZGRwSuvvELDhg1p1aoVTZs2DSnO888/n6ZNm/LMM8/QtGlTKlWqxLHHHpvT5ZGfq6++mj/++INx48YRGxtL1apV6datW6GPExE5GFlZ8MsvsGQJrFjh/v/LL7BmDfz7b8GP/eknJRQFUUJRRNnL2yYmJrJu3TqaN29OUlJSqS97e88993DuueeSkJDA4Ycfzvz58+nYsSPDhg3L2WfNmjW5HpM9jiIpKYlWrVrRuHFjzjrrLIYNG0a9evUKHD9Rt25dDjvsMBYtWkTXrl0B1022ZMkSDjvsMAD++usvVq1axcSJEznrrLMA+Prrr3PGaABUqVIFIM9gzvnz5zNhwgR69+4NwB9//MGmTZsOsnRERHLLyoJvvoFPPoE5c2DpUti2zf2tShU48kho3Rp69oRmzaBhQzjkEGjQwP09KgqMcTe/hl8JQglFMZSFdfPj4uJo27YtDzzwAJMmTaJNmzY8//zzfPTRR7Rq1YpXX32VuXPnUr9+/VyPO/PMM5k+fTpDhgwBIDY2lkaNGvH222/zwgsvFPicw4cPZ+zYsbRp04b//Oc/TJo0iU2bNuUkFPXr16dhw4Y8++yzHHHEEWzYsIHbb7+dSpUOVK/GjRtTvXp1Zs+eTdu2balWrRp169alTZs2TJ8+nY4dO5KRkcEdd9yRk3yIiByMXbtg1ix4802XSGSP8T7+eOjXDzp0cLdjjoHoaG9jLU80yyMC3XLLLTz33HOkp6dz3XXXcdlll3HFFVdwyimnkJaWxq233prnMWeddRb79+/P1RoRFxeXZ1swt956KwMHDmTw4MF07NiRrKysXIlVVFQUr732Gt999x3HHXccQ4cO5f7776dq1ao5+1SqVIkJEybw4osvcvjhh3PBBRcAMHXqVHbu3Em7du3o378/11xzDbGxsSGVj4hUPPv2wYwZEB8PjRtD374umejZE156CTZtgm+/hcmT4Zpr4LjjlEyEm3GDRCNf+/bt7bJly4L+bdWqVRxzzDHFOt6OHTuorc6ysFO5hl9ZK9OD+byVRYVNp5biK4kyXbcOnnvO3TZscF0Vffu6loi4OKhUztvhjTFfWWvbex0HqMtDREQijLUwfz48+ih88IHb1qMHPPkknHce+C3VI6VICYWIiESE/fvhnXdcIrF4sRs8edddMHgwqKfUe0ooRESkTMvKgrffhnvugZUr3cyMiRNhwACoUcPr6CSbBmWKiEiZZK3r0jj5ZLj0Unf/1VfdehA33KBkoqypMAlFeRl8KlKW6XMm4fL113DWWdCnD2RkuJkaK1a4wZaanVE2VYiEonLlyuzatcvrMETKvV27duW6eJ1IcW3c6Loy2rd33RtPPw2rVsGVVyqRKOsqRELRuHFjNmzYQGZmpn5BiZQAay2ZmZls2LCBxo0bex2ORKB9+2DcODjqKHjlFbjtNrck9pAh5X/qZ3lRId6mOnXqALBx40b27t1bpMf8+++/VKtWrSTDqpBUruFXVsq0cuXKNGnSJOfzJlJUixfDdde5had69YIJE9zAS4ksFSKhAJdUFOdEl5qaykknnVSCEVVMKtfwU5lKpNq5E+68EyZNgsMOc0tl9+3rrpshkafCJBQiIlJ2pKa6JbDT0uDGG+GBB3Qlz0hXIcZQiIhI2ZCRAcOGuRkc0dEwdy488YSSifLAk4TCGNPTGPOTMWa1MebOIH+va4z5wBjzrTHmB2PMQC/iFBGR8Pnpp1qcfLLr4hg+3I2ZOOMMr6OScCn1hMIYEw1MBM4FjgUuN8YcG7DbUGCltfYEIA54zBija1qLiESgrCz43/9g2LCTyciA2bNh/PiyuTBVSkoKsbGxREVFERsbS0pKitchRQwvxlB0AFZba9cCGGNeBS4AVvrtY4HaxhgD1AK2AvtKO1AREQnNpk1w1VUuiejS5S/eeacRDRp4HVVwKSkpJCQkkJmZCUB6ejoJCQkAxMfHexlaRPCiy6Mp8Jvf/fW+bf6eAo4BNgIrgOHW2qzSCU9ERMIhNRVOOgkWLoQpU2DMmB/KbDIBkJiYmJNMZMvMzCQxMdGjiCKLFy0UwSYEBa421QNYDnQFjgQ+NcbMs9b+k+tAxiQACQBNmjQhNTU1bEHu3LkzrMcTR+UafirTkqFyPXhZWfDKK82ZOrUFTZvuYuLEH2jRIqPMl+m6devy3V6W4y4rvEgo1gNH+N1vhmuJ8DcQeMi6ZS1XG2N+BY4GlvjvZK1NBpIB2rdvb+Pi4sIWZGpqKuE8njgq1/BTmZYMlevB2bbNLZP94YfQvz8kJ9egdu1TgLJfps2bNyc9PT3o9rIcd1nhRZfHUqC1MaaFb6Blf+D9gH3WAd0AjDFNgKOAtaUapYiIFMuqVdChA3z8MTz1FLz8cmRNB01KSqJGwEjRGjVqkJSU5FFEkaXUEwpr7T5gGPAxsAp43Vr7gzFmiDFmiG+3+4HTjDErgNnASGvtltKOVUREiua996BjR/jnH5gzB4YOjbwVL+Pj40lOTiYmJgZjDDExMSQnJ2tAZhF5slKmtXYmMDNg22S//28Eupd2XCIiUjzWulUu777bXSH0nXegWTOvozp48fHxSiAOkpbeFhGRg/LvvzB4MKSkuHETyclQvbrXUYlXlFCIiEix/fknXHihmxKalASjRkVeF4eElxIKEREpllWroHdvt2jV66/DpZd6HZGUBUooRESkyObPhz59oEoVd2GvDh28jkjKCl1tVEREiuStt+Dss6FxY/jySyUTkpsSChERKdSTT7qujZNPhgULoEULryOSskYJhYiI5MtaSEyEm26CCy5wF/k65BCvo5KySGMoREQkqP373QJVzzwDCQkwaRJER3sdlZRVaqEQEZE89uyB+HiXTIwaBZMnK5mQgqmFQkREcsnMhIsvhlmz4JFH4LbbvI5IIoESChERybFjB5x/PsybB1OmwKBBXkckkUIJhYiIAO7S4+eeC0uXuuW0+/f3OiKJJEooRESELVuge3f4/nt48023rLZIcSihEBGp4DZvhm7dYPVqeP996NnT64gkEimhEBGpwDZvhq5dYe1a+PBD93+Rg6GEQkSkgvrjD5dA/PqrSybOOsvriCSSKaEQEamAspOJtDSYORPi4ryOSCKdEgoRkQrmzz9zJxNnnul1RFIeKKEQEalAtm6Fc85xYyaUTEg4KaEQEakgtm1zU0NXrYIPPtCYCQkvJRQiIhXAjh1u0arvvoN33nGJhUg4KaEQESnndu1yy2kvXeoWrerd2+uIpDxSQiEiUo7t2eMu9PXFFzB9ulbAlJKjhEJEpJzat89dgvyjj9xlyK+4wuuIpDyL8joAEREJv6wsSEhwXRyPPeb+L1KSlFCIiJQz1sJtt8G0aXD33XDLLV5HJBWBEgoRkXLmoYdg3DgYNgzGjPE6GqkolFCIiJQjzz4Ld90Fl18OTzwBxngdkVQUSihERMqJt96CIUPcehPPPw9ROsNLKVJ1ExEpB1JT3SyOU091AzGrVPE6IqlolFCIiES4b7+FCy6AVq3ckto1angdkVRESihERCJYWprr4qhTB2bNggYNvI5IKiotbCUiEqG2bIGePd3S2vPnwxFHeB2RVGRKKEREIlBmprs+R3o6fPYZtG3rdURS0SmhEBGJMPv3uwGYixe7mR2nn+51RCJKKEREIoq1cNNN8N578OSTcNFFXkck4mhQpohIBHnkEZg0CW6/3a2EKVJWKKEQEYkQr7wCI0dC//5ueW2RskQJhYhIBJg3DwYMgC5dtAqmlE2qkiIiZdzPP8OFF0KLFvDOO1C1qtcRieSlhEJEpAz780/o1cu1SHz4oRaukrJLszxERMqof/91LRMbNsDnn8ORR3odkUj+lFCIiJRB1sLAgbBwIbzxBnTq5HVEIgVTl4eISBk0Zgy8+iqMHQuXXOJ1NCKFU0IhIlLGTJ8O993nWihGjvQ6GpGiUUIhIlKGzJsHgwZBXBxMngzGeB2RSNEooRARKSPWrHFLacfGumt0VKnidUQiRaeEQkSkDNi+3V09NCsLZszQ9FCJPJrlISLisX37oF8/+OUX+OQTaN3a64hEik8JhYiIx269FT7+GJKT4ayzvI5G5OCoy0NExEOTJ8OECXDzzXDttV5HI3LwlFCIiHjk88/dJch79XKXJReJZEooREQ8sHq1W7DqqKPcZcmjo72OSCQ0SihERErZ9u3Qp49bY+L996FOHa8jEgmdBmWKiJSi/fvh8ssPzOjQBb+kvFBCISJSiu68Ez76CJ5+WjM6pHxRl4eISCl58UV49FEYOhSGDPE6GpHwUkIhIlIKFi+GhATXKjFunNfRiISfEgoRkRK2YYO7Rsfhh8Mbb0Dlyl5HJBJ+niQUxpiexpifjDGrjTF35rNPnDFmuTHmB2PM3NKOUUQkHHbtcsnEjh1uRschh3gdkUjJKPVBmcaYaGAicA6wHlhqjHnfWrvSb596wCSgp7V2nTGmcWnHKSISKmvhuutg6VJ45x047jivIxIpOV60UHQAVltr11pr9wCvAhcE7HMF8La1dh2AtXZzKccYskWL4LLL4N9/vY5ERLwyfjy89BLcdx9ceKHX0YiULC8SiqbAb3731/u2+WsD1DfGpBpjvjLG/LfUoguTHTtcX+kHH3gdiYh44dNP4bbboG9fSEz0OhqRkufFOhQmyDYbcL8S0A7oBlQHvjTGLLLW/pzrQMYkAAkATZo0ITU1NWxB7ty5M6TjRUXBIYd0Yvz4HTRq9H3Y4op0oZar5KUyLRmhlOuGDdW4/vp2xMTsZvDgb/jii/3hDS5Cqa6Wb14kFOuBI/zuNwM2Btlni7U2A8gwxnwBnADkSiistclAMkD79u1tXFxc2IJMTU0l1OMNGABPPFGV446Lo2HDsIQV8cJRrpKbyrRkHGy57tjhLvhVuTJ89lllWrY8I/zBRSjV1fLNiy6PpUBrY0wLY0wVoD/wfsA+7wFnGGMqGWNqAB2BVaUcZ8iuugr27YPXXvM6EhEpDVlZcPXVsGqV+9y3bOl1RCKlp9QTCmvtPmAY8DEuSXjdWvuDMWaIMWaIb59VwCzgO2AJMMVaG3H9BiecAP/5D0yf7nUkIlIaHnzQzeZ49FE4+2yvoxEpXZ5cy8NaOxOYGbBtcsD9R4BHSjOuknDllTBypLsQUOvWXkcjIiXlgw/g7rvdZ37ECK+jESl9WimzhF1xhbtEsVopRMqvH390icRJJ0FysvvMi1Q0SihKWLNm0LWrSyhs4FwWEYl427e7NSaqVnXdHdWrex2RiDeUUJSCq66CtWvhyy+9jkREwikry32+16xx6840b+51RCLeUUJRCvr2db9aXnrJ60hEJJzuu8+NnXj8cTjzTK+jEfGWEopSULu2axJ97TUtxS1SXrz3Htx7r5smOmyY19GIeE8JRSkZMAD+/ttdbVBEItuqVa6ro317mDxZgzBFQAlFqenWzQ3QnDbN60hEJBTbt7vLkVerBm+/7f4VESUUpSY62rVSfPIJbNjgdTQicjCysuC//3WDMN98E444ovDHiFQUSihK0YAB7oT04oteRyIiB+OBB1y35eOPQ5cuXkcjUrYooShFRx7pTkJTp2pNCpFIM2MG3HOPGzuhQZgieSmhKGUDB8Lq1bBggdeRiEhR/fwzxMe7lTCfeUaDMEWCUUJRyi65BGrW1OBMkUixY4eb9l25slbCFCmIEopSVqsW9OsHr78OGRleRyMiBbHWjX366Sf3mY2J8ToikbJLCYUHBg6EnTvdKHERKbseeshNDf3f/9w1eUQkf0ooPHD66e5S5lOmeB2JiORnyZL6JCZC//5wyy1eRyNS9imh8IAxcO21MH++W3FPRMqWtWvhgQeO5T//cYm/BmGKFE4JhUeuvtoN8nr2Wa8jERF/GRluJUxwgzBr1vQ2HpFIoYTCI40bu5PWCy/ogmEiZYW1rvVwxQr4v/9bRcuWXkckEjmUUHgoIQG2bnWDvkTEe+PHwyuvuBUxO3TY6nU4IhFFCYWHzjrLrZ6ZnOx1JCIyZw7cfrtrORw1yutoRCKPEgoPRUW55tW5c+HHH72ORqTiWrcOLrvMzb56/nkNwhQ5GEooPDZgAFSqpMGZIl7ZtQv69oU9e+Ddd6FOHa8jEolMSig81qSJW9ZXgzNFSp+1cP318NVX8NJLcNRRXkckErmUUJQBCQnw118anClS2iZNcsn83XdDnz5eRyMS2ZRQlAHdukGrVu7kJiKlY/58GDECzjvPXZZcREJTaEJhjOlfGoFUZFFRrtl1wQJYvtzraETKvw0b3JV/W7RwXR1R+mklErKifIxeNMZ8bow5tsSjKcdSUlKIjY0lKiqK2NhYUlJScv194EB3WeSJEz0KUKSC2L0bLr7YXaDvnXegXj2vIxIpH4qSULQDKgPfGGMeNcbUKuGYyp2UlBQSEhJIT0/HWkt6ejoJCQm5kor69SE+HlJS4O+/PQxWpJy76SZYvNiNnWjb1utoRMqPQhMKa+0Ka+0ZQAJwJfCTMebyEo+sHElMTCQzMzPXtszMTBITE3NtGzrUTWF7/vlSDE6kAklOdrdRo1wrhYiET5F7Dq21LwBHAe8CLxlj5hhjlN8Xwbp164q0/cQT4bTTXLdHVlYpBCZSgSxaBDfeCD16wP33ex2NSPlTrKFI1trt1tqhwClAQ1w3yGPGmNolEl050bx58yJvHzYM1qyBTz4p6ahEKo5Nm9ziVc2awcsvQ3S01xGJlD9FSiiMMZWNMR2MMTcZY14G3gLaApWAocCPxhjN4s5HUlISNWrUyLWtRo0aJCUl5dn34ovdYlcanCkSHnv2uBkd27e7lTAbNPA6IpHyqSjTRhcC/wBfAo8BbYAPgH5AM6Ax8CrwpjFmSMmFGrni4+NJTk4mJiYGYwwxMTEkJycTHx+fZ98qVdz1PT78ENau9SBYkXLmpptg4UKYNg3+8x+voxEpv4rSQrETGAt0B+pZa9tba4dba9+w1m601v5jrb0V+D/grpIMNpLFx8eTlpZGVlYWaWlpQZOJbEOGuCbZCRNKMUCRcujZZ+GZZ2DkSHfxLxEpOUWZ5dHdWnuftXa2tTajgF2/wLVYSIiaNoV+/eC551wzrYgU38KFbuZU9+4QpHdRRMIsnOvDfQtcEMbjVWg33+wW3nnuOa8jEYk8Gze68UhHHAGvvKJBmCKlIWwJhbV2l7X2g3Adr6Jr1w7OOMN1e+zb53U0IpFj9243o2PHDg3CFClNWsG+DLv5ZkhPdydFESmcta6bI3slTA3CFCk9SijKsD59oGVLGDfO60hEIsPTT7tuwsRErYQpUtqUUJRh0dEHprwtWeJ1NCJl2xdfwPDh0KsX3Huv19GIVDxKKMq4a66BOnXUSiFSkHXr3OJVRx6plTBFvKKEooyrXdstdPXGG5CW5nU0ImVPZiZceKEbjPnee1C3rtcRiVRMSigiwPDhYAw8/rjXkYiULdbC4MGwfLlrmTjqKK8jEqm4lFBEgCOOgCuvhClTYMsWr6MRKTseecStM/HAA9C7t9fRiFRsSigixB13wK5d8OSTXkciUjbMnAl33umW1B41yutoREQJRYQ45hg3jfSppyCjoAXQRSqAH3+Eyy+HE0+EqVNdl6CIeEsJRQS5807YutV1fYhUVNu2wQUXQLVqbtG3mjW9jkhEQAlFROnUyS3H/fjjsHev19GIlL79+13LxK+/wltvQfPmXkckItmUUESYkSPdnPtXX/U6EpHSd8cdMGsWTJwInTt7HY2I+FNCEWF69XLXJxg7FrKyvI5GpPRMnepa52680a3NIiJlixKKCGOMu07BqlWuyVekIliwAIYMgbPP1nosImWVEooIdMklbtbHffeplULKv/R0uOgiiI2F11+HSpW8jkhEglFCEYGio+H//g++/16XNpfybedON6Njzx54/32oX9/riEQkP0ooIlS/ftCmjWulsNbraETCLyvLrRC7YoUbhHz00V5HJCIFUUIRoaKj3ViKb7+FDz7wOhqR8EtMdBf7GjcOevb0OhoRKYwSigh2xRXQsqVaKaT8efFFeOghuO46N6tDRMo+JRQRrFIl9yvuq6/cdQ1EyoMFC9y00K5d3bVrtKy2SGRQQhHhrrrKtVKMHq0ZHxL51q51MzqaN4c33oDKlb2OSESKSglFhKtcGcaMgW++0boUEtm2bYPzzoN9++DDD6FBA68jEpHi8CShMMb0NMb8ZIxZbYy5s4D9TjHG7DfGXFKa8UWaK66AY4+Fu+92J2ORSLN3r7sM+S+/wNtvuxlMIhJZSj2hMMZEAxOBc4FjgcuNMcfms9/DwMelG2HkiY6GBx5wl3SePt3raIJLSUkhNjaWrl27EhsbS0pKitchSRlhLdx0E3z6KSQnQ1yc1xGJyMHwooWiA7DaWrvWWrsHeBW4IMh+NwJvAZtLM7hIdeGF0L696/7YvdvraHJLSUkhISGB9PR0rLWkp6eTkJCgpEIAGD8eJk92F74bONDraETkYHmRUDQFfvO7v963LYcxpilwETC5FOOKaMZAUpJbpnjKFK+jyS0xMZHMzMxc2zIzM0lMTPQoIikr3nkHbr0VLr4YHnzQ62hEJBRerIofbBJY4CoK44GR1tr9poA5Y8aYBCABoEmTJqSmpoYpRNi5c2dYj1caKleG448/kdGja9Cy5SKqVy8b0z7WrVuX7/ZIK+OyKBLrKsCPP9ZmxIgTOfroDK69djlffFE26mu2SC3XskxlWr4ZW8orIhljOgFjrLU9fPdHAVhrx/rt8ysHEo+GQCaQYK19N7/jtm/f3i5btixscaamphIXgZ25CxZA585w//3ueh9lQWxsLOnp6Xm2x8TEkJaWVvoBlTORWFfT0uDUU6F6dVi0CJo08TqivCKxXMs6lWn4GWO+sta29zoO8KbLYynQ2hjTwhhTBegPvO+/g7W2hbU21lobC7wJ3FBQMiEHnH469O3rVhn8/Xevo3GSkpKoUaNGrm01atQgKSnJo4jES9u2Qe/ebqzPzJllM5kQkeIr9YTCWrsPGIabvbEKeN1a+4MxZogxZkhpx1MePfSQO1mPGeN1JE58fDzJycnExMRgjCEmJobk5GTi4+O9Dk1K2e7dLuHNnh56zDFeRyQi4eLFGAqstTOBmQHbgg7AtNYOKI2YypPWreGGG+Cpp9x0vGPzTMotffHx8cTHx6vJswKzFgYNgjlz4KWX4KyzvI5IRMJJK2WWU6NHQ+3acMcdXkci4vzf/0FKipuNdOWVXkcjIuGmhKKcatjQXTjsww9h9myvo5GKLjnZTQu99loYNcrraESkJCihKMduvBFiYuC222D/fq+jkYrq/ffh+uuhVy+YNElXDxUpr5RQlGPVqsHDD8Py5fDcc15HIxXRl19C//7Qrh289hpU8mTUloiUBiUU5dxll8GZZ8Jdd8HWrV5HIxXJjz+6q4c2beq63mrV8joiESlJSijKOWNgwgT4+283UFOkNGzcCD16uBaJjz+GRo28jkhESpoSigrg+ONh6FB3Aably72ORsq7v/+Gnj1di9jMmdCypdcRiUhpUEJRQdx7LzRo4AZqlvJq61KBZGbC+ee77o6333ZjJ0SkYlBCUUHUrw9jx8L8+fDyy15HI+XR3r3Qrx8sXOjWmzjnHK8jEpHSpISiArnmGjjlFHe56L//9joaKU+ysmDwYJgxw00NvfRSryMSkdKmhKICiYqCZ56BLVtg5Eivo5Hywlq31smLL8J998EQXZFHpEJSQlHBnHQS3HwzPPsszJvndTRSHtx3H4wbB8OHu+W1RaRiUkJRAY0Z41bQTEhwV38UOVjjx7v6NHAgPP64VsEUqciUUFRANWvC00+7kfgPP+x1NBKppk51rV0XX+yu1RGls4lIhaZTQAV17rlw+eXuyo8//uh1NBJpXn3VXeirRw83o0NLaouIEooKbNw411pxzTW6eJgU3TvvuMuPd+7s1pqoWtXriESkLFBCUYE1aQJPPeUu4PTYY15HI5Fg5ky31sQpp7gpojVqeB2RiJQVSigquMsvd33go0fD9997HY2UZZ99Bn37uqXcP/oIatf2OiIRKUuUUFRwxrgBmnXrwtVXu9UORQKlpkKfPtCmjbvYV716XkckImWNEgqhUSN34bCvv3bLc4v4S02FXr3cRb4++wwOOcTriESkLFJCIYBryo6Ph/vvh2XLvI5Gyoq5c6F3b5dMfP45NG7sdUQiUlYpoZAcTz4Jhx3mxlXs2OF1NOK1uXNdy0RsrJIJESmcEgrJUb++W1Ng7VoYOtTraMRLn33m1ipRMiEiRaWEQnI54wy45x546SV3k4pn5kw47zxo1QrmzHHTi0VECqOEQvJITIQuXeD66+GXX7yORkrTu+/ChRdC27YumVDLhIgUlRIKySM62nV9VK0K/fvDv/96HZGUhldfhUsugZNPhtmzNZtDRIpHCYUE1awZTJvmppLeeKPX0UgwKSkpxMbG0rVrV2JjY0lJSTnoYyUnwxVXwOmnw6efap0JESk+JRSSrz594K67YMoUePZZr6MRfykpKSQkJJCeno61lvT0dBISEg4qqXj4YbjuOjejY9YsrYApIgdHCYUU6L77oHt3GDYMlizxOhrJlpiYSGZmZq5tmZmZJCYmFvkY1sKoUXDnnW6q8DvvQPXq4Y5URCoKJRRSoOhoePllOPxwd82PzZu9jkgA1q1bV6ztgfbtg8GD4aGHYMgQN6OncuVwRigiFY0SCinUIYfAW2/Bli1w2WWwZ4/XEUnz5s2Ltd1fRoabyTF1qrso3KRJLnEUEQmFEgopkpNPdmMp5s6FhATXXC7eSUpKokbAtcNr1KhBUlJSgY/bsgW6dXNXC5082XVpGVOSkYpIRaGEQoosPt4tevXCC7qImNfi4+NJTk4mJiYGYwwxMTEkJycTHx+f72NWr3azOL791rU4XXddKQYsIuVeJa8DkMhyzz1usavERDjySOjXz+uIKq74+Hji4+NJTU0lLi6uwH0XLIALLnD//+wzl1iIiISTWiikWIyB555zX0hXXw0LF3odkRTmlVega1do0AAWLVIyISIlQwmFFFu1am6J5iOOcJe2/u47ryOSYKx1l6O/4go49VT48kt3fQ4RkZKghEIOSsOGbkXFmjXdOhWrV3sdkfjLyHDdUXffDVddBZ98oqW0RaRkKaGQgxYb65KKffvg7LNhwwavIxKA9HTXrfHWW/DII24QbdWqXkclIuWdEgoJyTHHuOWat26Fc86BP//0OqKKLTUV2reHtDT48EO47TZNCxWR0qGEQkLWvj28/z78+ivExcHvv3sdUcVjLfzvf66l6JBDYPFi6NnT66hEpCJRQiFhERfnfhGnpbn/b9zocUAVyM6d0fTtCyNHQt++sHQpHHWU11GJSEWjhELCpmtX1/2xYQOceSb89pvXEZV/X38NQ4a0Y8YMGDcOXntNVwsVEW8ooZCwOuMMN1Dzzz+hSxf46SevIyqfrHUJxKmnwu7d0cyZAyNGaLyEiHhHCYWE3amnwuzZburiaae5VRolfDZvhvPOg1tugV69YMqUpXTu7HVUIlLRKaGQEtGunVtI6ZBD3MWo3nrL64jKhxkz4IQTXMI2cSK88w7UrbvP67BERJRQSMk58ki3NPfJJ8Oll8Ljj+sqpQdr+3YYNAjOPx8aN4YlS+CGG9TFISJlhxIKKVENG7pf0xddBLfeCklJx5CZ6XVUkeWzz+D44+H5591F2ZYudfdFRMoSJRRS4qpXhzfegAcegM8/b0ynTrBmjddRlX1btrgLsJ1zjivDhQtdGVap4nVkIiJ5KaGQUhEV5X5dP/TQCn77zS2G9d57XkdVNlnrlss++mh4+WVXbt98Ax07eh2ZiEj+lFBIqerQYSvLlkGLFnDhhXDttbBzp9dRlR3Ll7uFwQYMcItTLV/uWiWqV/c2LhGRwiihkFLXsqWbATJyJDz3HJx4ortfkW3eDAkJbgDrypXwzDMwbx60bet1ZCIiRaOEQjxRtSo89JC7mNW+fdC5s1tXYccOryMrXRkZ8OCD0Lo1TJsGw4fDL7+45CJKn04RiSA6ZYmnunSB775zXR/jx7txA6+/Xv6nl+7eDU8+6VprEhMPlMO4cVCvntfRiYgUnxIK8VydOjB5suv2aNIE+vWDHj3cQMTyJjMTnnoK2rSBm25yl39fsAA++MD9X0QkUimhkDKjY0e3YNOECbBsmRtP0K9f+bgeyNatbnBlTAzceCM0awYffwxz5rjlyUVEIp0SCilTKlVyX7hr18Lo0e6S6Mce69Zj+PZbr6Mrvq++citcNmvmXk/Hjm6w5YIF0L27VroUkfJDCYWUSfXqwX33ucRi+HB48003G6RrV9c9kJXldYT5++svN0ujQwe33sarr8KVV7qEaMYMdCEvESmXlFBImda4sbsGyPr18L//werV0KeP6zoYORJWrPA6Qmf7drcI1fnnw6GHwpAhbgbHhAmwcSMkJ2u5bBEp35RQSESoXx9uv90t2f3666614vHH3Zf08ce7mRJffAF795ZOPFlZLpl55BG3EFXDhhAf7xaiuvlm+Ppr+P57131Tt27pxCQi4qVKXjypMaYn8AQQDUyx1j4U8Pd4YKTv7k7gemttBPagS7hVruyuXHrppe5aF6+/7roUHn7YredQu7b7gu/QwV1CvV0718oRiqws+O03WLXKXZhr4UJYtAi2bXN/P+EEl+z06uUGWGr9CBGpiEo9oTDGRAMTgXOA9cBSY8z71tqVfrv9Cpxprf3bGHMukAzoSgaSS8OG7hLeN9zguhxmz3YzJ1JT3TiLbI0bQ2zsgVujRlCrlks+atZ0CcPevbBnj5vWuXkz/P67u6WlwY8/knOFVGPc6pWXXQadOsHZZ7sBlyIiFZ0XLRQdgNXW2rUAxphXgQuAnITCWrvQb/9FgE7ZUqC6daFvX3cD+Ocft47FsmUuIUhLc/fffdclDoVp0MCtidGsGQwe7NaIOOYY1xqhhadERPLyIqFoCvzmd389Bbc+DAI+KtGIpNypUwfOPNPd/FnrBkvu2OFuGRkQHe26UqpUgWrVXAuGLhEuIlI8XiQUwWbeB11o2RhzFi6hCDrRzhiTACQANGnShNTU1DCFCDt37gzr8cSJhHL95RevIyieSCjTSKRyDT+VafnmRUKxHjjC734zYGPgTsaY44EpwLnW2r+CHcham4wbX0H79u1tXFxc2IJMTU0lnMcTR+UafirTkqFyDT+VafnmxXj0pUBrY0wLY0wVoD/wvv8OxpjmwNvAVdbanz2IUURERIqh1FsorLX7jDHDgI9x00anWmt/MMYM8f19MnA3cAgwybi1ifdZa9uXdqwiIiJSNJ6sQ2GtnQnMDNg22e//g4HBpR2XiIiIHBwtwSMiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIiIiFTQiEiIiIhU0IhIiIiIVNCISIlLiUlhdjYWKKiooiNjSUlJcXrkEQkzCp5HYCIlG8pKSkkJCSQmZkJQHp6OgkJCQDEx8d7GZqIhJFaKESkRCUmJuYkE9kyMzNJTEz0KCIRKQlKKESkRK1bt65Y20UkMimhEJES1bx582JtF5HIpIRCREpUUlISNWrUyLWtRo0aJCUleRSRiJQEJRQiUqLi4+NJTk4mJiYGYwwxMTEkJydrQKZIOaNZHiJS4uLj45VAiJRzaqEQERGRkCmhEBERkZApoRAREZGQKaEQERGRkCmhEBERkZApoRAREZGQKaEQERGRkCmhEBERkZApoRAREZGQKaEQERGRkCmhEBERkZApoRAREZGQKaEQERGRkCmhEBERkZApoRAREZGQKaEQERGRkCmhEBERkZApoRAREZGQKaEQERGRkCmhEBERkZApoRAREZGQKaEQERGRkBlrrdcxhIUx5k8gPYyHbAhsCePxxFG5hp/KtGSoXMNPZRp+MdbaRl4HAeUooQg3Y8wya217r+Mob1Su4acyLRkq1/BTmZZv6vIQERGRkCmhEBERkZApochfstcBlFMq1/BTmZYMlWv4qUzLMY2hEBERkZCphUJERERCpoQiCGNMT2PMT8aY1caYO72OJ9IZY44wxswxxqwyxvxgjBnudUzliTEm2hjzjTFmhtexlAfGmHrGmDeNMT/66mwnr2MqD4wxN/s+/98bY14xxlTzOiYJLyUUAYwx0cBE4FzgWOByY8yx3kYV8fYBt1prjwFOBYaqTMNqOLDK6yDKkSeAWdbao4ETUNmGzBjTFLgJaG+tPQ6IBvp7G5WEmxKKvDoAq621a621e4BXgQs8jimiWWs3WWu/9v1/B+4E3dTbqMoHY0wzoDcwxetYygNjTB2gC/AcgLV2j7V2m6dBlR+VgOrGmEpADWCjx/FImCmhyKsp8Jvf/fXoyy9sjDGxwEnAYo9DKS/GA3cAWR7HUV60BP4Epvm6kaYYY2p6HVSks9ZuAB4F1gGbgO3W2k+8jUrCTQlFXibINk2FCQNjTC3gLWCEtfYfr+OJdMaY84DN1tqvvI6lHKkEnAw8ba09CcgANI4qRMaY+riW3hbA4UBNY8yV3kYl4aaEIq/1wBF+95uhprmQGWMq45KJFGvt217HU06cDvQxxqThuua6GmOmextSxFsPrLfWZregvYlLMCQ0ZwO/Wmv/tNbuBd4GTvM4JgkzJRR5LQVaG2NaGGOq4AYOve9xTBHNGGNwfdKrrLWPex1PeWGtHWWtbWatjcXV08+ttfrVFwJr7e/Ab8aYo3ybugErPQypvFgHnGqMqeE7H3RDg13LnUpeB1DWWGv3GWOGAR/jRiJPtdb+4HFYke504CpghTFmuW/bXdbamd6FJJKvG4EU3w+KtcBAj+OJeNbaxcaYN4GvcbO+vkGrZpY7WilTREREQqYuDxEREQmZEgoREREJmRIKERERCZkSChEREQmZEgoREREJmRIKERERCZkSChEREQmZEgoREREJmRIKESkWY0wrY8xeY8y9AdufNsbsMMa09yo2EfGOEgoRKRZr7WpgCnCzMaYhgDHmbuAa4CJr7TIv4xMRb2jpbREpNmPMocAaYBLwI+66DJdba1/3NDAR8YwuDiYixWat/d0YMx64FXceuUnJhEjFpi4PETlYvwBVgS+ttRO9DkZEvKWEQkSKzRjTFXgG+BI43RhzgschiYjHlFCISLEYY04G3sUNzIwD1gEPehiSiJQBSihEpMiMMa2Aj4BPgButtXuAe4FexpgungYnIp7SLA8RKRLfzI6FuBaJHtba3b7t0cD3wN/W2tM8DFFEPKSEQkREREKmLg8REREJmRIKERERCZkSChEREQmZEgoREREJmRIKERERCZkSChEREQmZEgoREREJmRIKERERCZkSChEREQnZ/wPPD9np+beqGAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Our data from Lecture 1:\n", "x = np.array([0.5, 2.0, 4.0, 5.0, 7.0, 9.0])\n", "y = np.array([0.5, 0.4, 0.3, 0.1, 0.9, 0.8])\n", "\n", "# Consider a polynomial of degree 3 - so not high enough to go through all the data\n", "N = 3\n", "\n", "# Use a numpy function to construct the Vandermonde matrix\n", "V = np.vander(x, N+1, increasing=True)\n", "\n", "# Form the matrix A by transposing V and multiplying by V:\n", "A = V.T @ V # same as A = np.transpose(V) @ V\n", "\n", "# Use a function from SciPy's linalg sub-package to find the inverse:\n", "invA = sl.inv(A)\n", "\n", "# Form the RHS vector:\n", "rhs = V.T @ y\n", "\n", "# Multipy through by the inverse matrix to find a:\n", "a = invA @ rhs\n", "print('a = \\n', a)\n", "\n", "# Compare against the coefficient that numpy's polyfit gives us\n", "poly_coeffs = np.polyfit(x, y, N)\n", "print('\\npoly_coeffs = \\n', poly_coeffs)\n", "# they're the same, we just set up our algorithm to return the coefficient in the \n", "# opposite order to polyfit - we need to remember this when we evaluate the polynomial\n", "\n", "print('\\nOur _a_ vector = output from np.polyfit (as long as we flip the order)? ', \n", " np.allclose(np.flip(a), poly_coeffs))\n", "# set up figure\n", "fig = plt.figure(figsize=(8, 6))\n", "ax1 = fig.add_subplot(111)\n", "ax1.margins(0.1)\n", "\n", "xx = np.linspace(0.4, 9.1, 100)\n", "yy = a[0] + a[1]*xx + a[2] * xx**2 + a[3] * xx**3 \n", "\n", "ax1.plot(xx, yy, 'b', label='Least squares fit (cubic)')\n", "\n", "# Overlay raw data\n", "ax1.plot(x, y, 'ko', label='Raw data')\n", "ax1.set_xlabel('$x$', fontsize=16)\n", "ax1.set_ylabel('$y$', fontsize=16)\n", "ax1.set_title('Least squares approximation of a cubic to multiple data points', fontsize=16)\n", "ax1.grid(True)\n", "ax1.legend(loc='best', fontsize=14);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The key result was that solving the problem\n", "\n", "$$V^TV\\boldsymbol{a} = V^T\\boldsymbol{y}$$\n", "\n", "i.e. forming\n", "\n", "$$\\boldsymbol{a} = (V^TV)^{-1}V^T\\boldsymbol{y}$$\n", "\n", "gave us a solution for the parameters which had the property that it fit the data with the minimum least-squares error.\n", "\n", "It can be shown that if the matrix (here $V$, later $G$) is of full \"column rank\" (the columns are all linearly independent; for square matrices full column rank == full row rank, but not the case for non-square matrices) then $(V^TV)^{-1}$ exists.\n", "\n", "We will return to the least squares approximation later." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Summary\n", "\n", "\n", "- We have introduced inversion and optimisation problems.\n", "\n", "\n", "- We have explained how they are largely equivalent, and how formulating as one can be used to solve the other.\n", "\n", "\n", "- We have reviewed some linear algebra theory.\n", "\n", "\n", "- We have reminded ourselves of some Python based linear algebra.\n", "\n", "\n", "- We have started our thinking on over- and under-determined systems (i.e. the non-square case where $M\\ne N$), ...\n", "\n", "\n", "- and in the over-determined case introduced the least squares solution." ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.13" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": true, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": true }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 1 }