{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Regression" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Preliminaries\n", "\n", "- Goal \n", " - Introduction to Bayesian (Linear) Regression\n", "- Materials \n", " - Mandatory\n", " - These lecture notes\n", " - Optional\n", " - Bishop pp. 152-158 \n", " - In this and forthcoming lectures, we will make use of some elementary matrix calculus. The most important formulas are summarized at the bottom of this notebook in an [OPTIONAL SLIDE on matrix calculus](#matrix-calculus). For derivations, see Bishop Appendix C. \n", " " ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Regression - Illustration\n", "\n", "\n", "

\n", "\n", "Given a set of (noisy) data measurements, find the 'best' relation between an input variable $x \\in \\mathbb{R}^M$ and input-dependent outcomes $y \\in \\mathbb{R}$.\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Regression vs Density Estimation\n", "\n", "\n", "- Observe $N$ IID data **pairs** $D=\\{(x_1,y_1),\\dotsc,(x_N,y_N)\\}$ with $x_n \\in \\mathbb{R}^M$ and $y_n \\in \\mathbb{R}$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Assume that we are interested in (a model for) the responses $y_n$ for **given inputs** $x_n$?, I.o.w. we are interested in building a model for the conditional distribution $p(y|x)$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Note that, since $p(x,y)=p(y|x)\\, p(x)$, building a model $p(y|x)$ is similar to density estimation with the assumption that $x$ is drawn from a uniform distribution." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Bayesian Linear Regression\n", "\n", "- Next, we discuss (1) model specification, (2) Inference and (3) a prediction application for a Bayesian linear regression problem. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "##### 1. Model Specification\n", "\n", "\n", "- In a traditional _regression_ model, we try to 'explain the data' by a purely deterministic function $f(x_n,w)$, plus a purely random term $\\epsilon_n$ for 'unexplained noise':\n", "$$\n", " y_n = f(x_n,w) + \\epsilon_n\n", "$$ " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- In a _linear regression_ model, i.e., linear w.r.t. the parameters $w$, we assume that \n", "$$f(x_n,w)= \\sum_{j=0}^{M-1} w_j \\phi_j(x_n) = w^T \\phi(x_n)$$\n", "where $\\phi_j(x)$ are called basis functions.\n", " - For notational simplicity, from now on we will assume $f(x_n,w) = w^T x_n$, with $x_n \\in \\mathbb{R}^M$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- In _ordinary linear regression_ , the noise process $\\epsilon_n$ is zero-mean Gaussian with constant variance, i.e.\n", "$$\n", "\\epsilon_n \\sim \\mathcal{N}(0,\\beta^{-1}) \\,.\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Hence, given a data set $D=\\{(x_1,y_1),\\dotsc,(x_N,y_N)\\}$, the likelihood for an ordinary linear regression model is \n", "\n", "$$\\begin{align*}\n", "p(y\\,|\\,\\mathbf{X},w,\\beta) &= \\mathcal{N}(y\\,|\\,\\mathbf{X} w,\\beta^{-1} \\mathbf{I}) \\\\\n", " &= \\prod_n \\mathcal{N}(y_n\\,|\\,w^T x_n,\\beta^{-1}) \\tag{B-3.10}\n", "\\end{align*}$$\n", "\n", "where $w = \\left(\\begin{matrix} w_1 \\\\ w_2 \\\\ \\vdots \\\\ w_{M} \\end{matrix} \\right)$, the $(N\\times M)$-dim matrix $\\mathbf{X} = \\left(\\begin{matrix}x_1^T \\\\ x_2^T \\\\ \\vdots \\\\ x_N^T \\end{matrix} \\right) = \\left(\\begin{matrix}x_{11},x_{12},\\dots,x_{1M}\\\\ x_{21},x_{22},\\dots,x_{2M} \\\\ \\vdots \\\\ x_{N1},x_{N2},\\dots,x_{NM} \\end{matrix} \\right) $ and $y = \\left(\\begin{matrix} y_1 \\\\ y_2 \\\\ \\vdots \\\\ y_N \\end{matrix} \\right)$.\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- For full Bayesian learning we should also choose a prior $p(w)$:\n", "\n", "$$\\begin{equation}\n", "p(w\\,|\\,\\alpha) = \\mathcal{N}(w\\,|\\,0,\\alpha^{-1}\\mathbf{I}) \\tag{B-3.52}\n", "\\end{equation}$$\n", "\n", "- For simplicity, we will assume that $\\alpha$ and $\\beta$ are fixed and known. \n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "##### 2. Inference\n", "\n", "- We'll do Bayesian inference for the parameters $w$. \n", "\n", "$$\\begin{align*}\n", "p(w|D) &\\propto p(D|w)\\cdot p(w) \\\\\n", " &= \\mathcal{N}(y\\,|\\,\\mathbf{X} w,\\beta^{-1} \\mathbf{I}) \\cdot \\mathcal{N}(w\\,|\\,0,\\alpha^{-1}\\mathbf{I}) \\\\\n", " &\\propto \\exp \\big( -\\frac{\\beta}{2} \\big( {y - \\mathbf{X}w } \\big)^T \\big( {y - \\mathbf{X}w } \\big) - \\frac{\\alpha}{2}w^T w \\big) \\qquad \\text{(B-3.55)} \\\\\n", " &= \\exp\\big( -\\frac{1}{2} w^T\\big(\\underbrace{\\beta \\mathbf{X}^T\\mathbf{X} + \\alpha \\mathbf{I}}_{\\Lambda_N}\\big)w + \\big(\\underbrace{\\beta \\mathbf{X}^Ty}_{\\eta_N}\\big)^T w - \\frac{\\beta}{2}y^T y \\big) \\\\\n", " &\\propto \\mathcal{N}_c\\left(w\\,|\\,\\eta_N,\\Lambda_N \\right)\n", "\\end{align*}$$\n", "\n", "with natural parameters (see the [natural parameterization of Gaussian](https://nbviewer.jupyter.org/github/bertdv/BMLIP/blob/master/lessons/notebooks/The-Gaussian-Distribution.ipynb#natural-parameterization)):\n", "\n", "$$\\begin{align*}\n", "\\eta_N &= \\beta\\mathbf{X}^Ty \\\\\n", "\\Lambda_N &= \\beta \\mathbf{X}^T\\mathbf{X} + \\alpha \\mathbf{I}\n", "\\end{align*}$$\n", "\n", "- Or equivalently (in the moment parameterization of the Gaussian):\n", "$$\\begin{align*}\n", "p(w|D) &= \\mathcal{N}\\left(w\\,|\\,m_N,S_N \\right) \\qquad &&\\text{(B-3.49)} \\\\\n", "m_N &= \\beta S_N \\mathbf{X}^T y \\qquad &&\\text{(B-3.53)}\\\\\n", "S_N &= \\left(\\alpha \\mathbf{I} + \\beta \\mathbf{X}^T \\mathbf{X}\\right)^{-1} \\qquad &&\\text{(B-3.54)}\n", "\\end{align*}$$\n", "\n", "- Note that B-3.53 and B-3.54 combine to\n", "$$\n", "m_N = \\left(\\frac{\\alpha}{\\beta}\\mathbf{I} + \\mathbf{X}^T \\mathbf{X} \\right)^{-1} \\mathbf{X}^T y\\,.\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ " - (Bishop Fig.3.7) Illustration of sequential Bayesian learning for a simple linear model of the form $y(x, w) =\n", "w_0 + w_1 x$. (Bishop Fig.3.7, detailed description at Bishop, pg.154.)\n", "\n", "

" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "##### 3. Application: Predictive Distribution\n", "\n", "- Assume we are interested in the distribution $p(y_\\bullet \\,|\\, x_\\bullet, D)$ for a new input $x_\\bullet$. This can be worked out as (exercise B-3.10)\n", "\n", "$$\\begin{align*}\n", "p(y_\\bullet \\,|\\, x_\\bullet, D) &= \\int p(y_\\bullet \\,|\\, x_\\bullet, w) p(w\\,|\\,D)\\,\\mathrm{d}w \\\\\n", "&= \\int \\mathcal{N}(y_\\bullet \\,|\\, w^T x_\\bullet, \\beta^{-1}) \\mathcal{N}(w\\,|\\,m_N,S_N)\\,\\mathrm{d}w \\\\\n", "&= \\int \\mathcal{N}(y_\\bullet \\,|\\, z, \\beta^{-1}) \\underbrace{\\mathcal{N}(z\\,|\\,x_\\bullet^T m_N,x_\\bullet^T S_N x_\\bullet)\\,\\mathrm{d}z}_{=\\mathcal{N}(w\\,|\\,m_N,S_N)\\,\\mathrm{d}w} \\quad \\text{(sub. }z=x_\\bullet^T w \\text{)} \\\\\n", "&= \\int \\underbrace{\\underbrace{\\mathcal{N}(z \\,|\\, y_\\bullet, \\beta^{-1})}_{\\text{switch }z \\text{ and }y_\\bullet} \\mathcal{N}(z\\,|\\,x_\\bullet^T m_N,x_\\bullet^T S_N x_\\bullet)}_{\\text{Use Gaussian product formula SRG-6}}\\,\\mathrm{d}z \\\\\n", "&= \\int \\mathcal{N}\\left(y_\\bullet\\,|\\, m_N^T x_\\bullet, \\sigma_N^2(x_\\bullet) \\right) \\underbrace{\\mathcal{N}\\left(z\\,|\\, \\cdot, \\cdot\\right)}_{\\text{integrate this out}} \\mathrm{d}z\\\\\n", "&= \\mathcal{N}\\left(y_\\bullet\\,|\\, m_N^T x_\\bullet, \\sigma_N^2(x_\\bullet) \\right)\n", "\\end{align*}$$\n", "with\n", "$$\\begin{align*}\n", "\\sigma_N^2(x_\\bullet) = \\beta^{-1} + x^T_\\bullet S_N x_\\bullet \\tag{B-3.59}\n", "\\end{align*}$$\n", "\n", "- So, the uncertainty $\\sigma_N^2(x_\\bullet)$ about the output $y_\\bullet$ contains both uncertainty about the process ($\\beta^{-1}$) and about the model parameters ($x^T_\\bullet S_N x_\\bullet$). \n", "\n", "- (See the [OPTIONAL SLIDE](#change-of-variable-derivation) below for the step in this derivation where $\\mathcal{N}(w\\,|\\,m_N,S_N)\\,\\mathrm{d}w$ gets replaced $\\mathcal{N}(z\\,|\\,x_\\bullet^T m_N,x_\\bullet^T S_N x_\\bullet)\\,\\mathrm{d}z$.)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Example Predictive Distribution\n", "\n", "- As an example, let's do Bayesian Linear Regression for a synthetic sinusoidal data set and a model with 9 Gaussian basis functions \n", "$$\\begin{align*}\n", "y_n &=\\sum_{m=1}^9 w_m \\phi_m(x_n) + \\epsilon_n \\\\\n", "\\phi_m(x_n) &= \\exp\\left( \\frac{x_n-\\mu_m}{\\sigma^2}\\right) \\\\\n", "\\epsilon_n &\\sim \\mathcal{N}(0,\\beta^{-1})\n", "\\end{align*}$$\n", "\n", "

\n", "\n", "- The predictive distributions for $y$ are shown in the following plots (Bishop, Fig.3.8)\n", "

\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "- And some plots of draws of posteriors for the functions $w^T \\phi(x)$ (Bishop, Fig.3.9) \n", "\n", "

" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Maximum Likelihood Estimation for Linear Regression Model\n", "\n", "- Recall the posterior mean for the weight vector\n", "$$\n", "m_N = \\left(\\frac{\\alpha}{\\beta}\\mathbf{I} + \\mathbf{X}^T \\mathbf{X} \\right)^{-1} \\mathbf{X}^T y\n", "$$\n", "where $\\alpha$ is the prior precision for the weights." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- The Maximum Likelihood solution for $w$ is obtained by letting $\\alpha \\rightarrow 0$, which leads to \n", "\n", "$$\\begin{equation*}\n", "\\hat w_{\\text{ML}} = (\\mathbf{X}^T \\mathbf{X})^{-1} \\mathbf{X}^T y\n", "\\end{equation*}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- The matrix $\\mathbf{X}^\\dagger \\equiv (\\mathbf{X}^T \\mathbf{X})^{-1}\\mathbf{X}^T$ is also known as the **Moore-Penrose pseudo-inverse** (which is sort-of-an-inverse for non-square matrices)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Note that if we have fewer training samples than input dimensions, i.e., if $Ngradient \n", "$ \\frac{\\partial \\left( {y - \\mathbf{X}w } \\right)^T \\left( {y - \\mathbf{X}w } \\right)}{\\partial w} = -2 \\mathbf{X}^T \\left(y - \\mathbf{X} w \\right)\n", "$ to zero yields the so-called **normal equations** \n", "$\\mathbf{X}^T\\mathbf{X} \\hat w_{\\text{LS}} = \\mathbf{X}^T y$ and consequently\n", "\n", "$$\n", "\\hat w_{\\text{LS}} = (\\mathbf{X}^T \\mathbf{X})^{-1} \\mathbf{X}^T y\n", "$$\n", "\n", "which is the same answer as we got for the maximum likelihood weights $\\hat w_{\\text{ML}}$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- $\\Rightarrow$ Least-squares regression ($\\hat w_{\\text{LS}}$) corresponds to the (probabilistic) maximum likelihood solution ($\\hat w_{\\text{ML}}$) if the probabilistic model includes the following assumptions:\n", " 1. The observations are independently and identically distributed (**IID**) (this determines how errors are combined), and\n", " 1. The noise signal $\\epsilon_n \\sim \\mathcal{N}(0,\\,\\beta^{-1})$ is **Gaussian** distributed (determines the error metric) \n", " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- If you use the Least-Squares method, you cannot see (nor modify) these assumptions. The probabilistic method forces you to state all assumptions explicitly! " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Not Identically Distributed Data\n", "\n", "- Let's do an example regarding changing our assumptions. What if we assume that the variance of the measurement error varies with the sampling index, $\\epsilon_n \\sim \\mathcal{N}(0,\\beta_n^{-1})$?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- The likelihood is now (using $\\Lambda \\triangleq \\mathrm{diag}(\\beta_n)$ )\n", "$$\n", "p(y\\,|\\,\\mathbf{X},w,\\Lambda) = \\mathcal{N}(y\\,|\\,\\mathbf{X} w,\\Lambda^{-1} ) \\,.\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Combining this likelihood with the prior $p(w) = \\mathcal{N}(w\\,|\\,0,\\alpha^{-1}\\mathbf{I})$ leads to a posterior\n", "\n", "$$\\begin{align*}\n", "p(w|D) &\\propto p(D|w)\\cdot p(w) \\\\\n", " &= \\mathcal{N}(y\\,|\\,\\mathbf{X} w,\\Lambda^{-1} \\mathbf{I}) \\cdot \\mathcal{N}(w\\,|\\,0,\\alpha^{-1}\\mathbf{I}) \\\\\n", " &\\propto \\exp \\left\\{ \\frac{1}{2} \\left( {y - \\mathbf{X}w } \\right)^T \\Lambda \\left( {y - \\mathbf{X}w } \\right) + \\frac{\\alpha}{2}w^T w \\right\\} \\\\\n", " &\\propto \\mathcal{N}\\left(w\\,|\\,m_N,S_N \\right) \n", "\\end{align*}$$\n", "with\n", "$$\\begin{align*}\n", "m_N &= S_N \\mathbf{X}^T \\Lambda y \\\\\n", "S_N &= \\left(\\alpha \\mathbf{I} + \\mathbf{X}^T \\Lambda \\mathbf{X}\\right)^{-1} \n", "\\end{align*}$$\n", "\n", "- And maximum likelihood solution \n", "$$\n", "\\hat{w}_{\\text{ML}} = \\left. m_N\\right|_{\\alpha \\rightarrow 0} = \\left(\\mathbf{X}^T \\Lambda \\mathbf{X}\\right)^{-1} \\mathbf{X}^T \\Lambda y\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- This maximum likelihood solution is also called the **Weighted Least Squares** (WLS) solution. (Note that we just stumbled upon it, the crucial aspect is appropriate model specification!)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Note also that the dimension of $\\Lambda$ grows with the number of data points. In general, models for which the number of parameters grow as the number of observations increase are called **non-parametric models**." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Code Example: Least Squares vs Weighted Least Squares\n", "\n", "- We'll compare the Least Squares and Weighted Least Squares solutions for a simple linear regression model with input-dependent noise:\n", "\n", "$$\\begin{align*}\n", "x &\\sim \\text{Unif}[0,1]\\\\\n", "y|x &\\sim \\mathcal{N}(f(x), v(x))\\\\\n", "f(x) &= 5x - 2\\\\\n", "v(x) &= 10e^{2x^2}-9.5\\\\\n", "\\mathcal{D} &= \\{(x_1,y_1),\\ldots,(x_N,y_N)\\}\n", "\\end{align*}$$" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using Pkg; Pkg.activate(\"../.\"); Pkg.instantiate();\n", "using IJulia; try IJulia.clear_output(); catch _ end" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdd3wT9f8H8M9l7zYdadK96aLQMgQRKKNsyhAZ4gBF4YtbRL76/eHEiQu3iAKKygZFhoDIHrJXaQtNd9OVptnrcvf7IyWEdFDaNJc27+fj+wc5Lndv+o159e4+788HI0kSAQAAAL6KRnUBAAAAAJUgCAEAAPg0CEIAAAA+DYIQAACAT4MgBAAA4NMgCAEAAPg0CEIAAAA+DYIQAACAT4MgBAAA4NMgCAEAAPi0LhmEL730EkEQbdzZZrN1ajGgKYIgYOo+z4OPuue1/YsIuJHbP+pYV/zC4vF4SqWSy+W2ZWetVisUCju7JODMZDIxGAwGg0F1Ib4FPuqep9freTwehmFUF+Jb3P5R75JXhAAAAIC7QBACAADwaRCEAAAAfBoEIQAAAJ8GwxkAAAB0ASdPnrx08YI4IDAzM9O9g2UgCAEAAHi1urq62VMn+RvrM/0ZJTb0bol63jPPP/X8i+46PgQhAAAArzZ31vSHgyxZkVH2l0+khc5f9VVyz17DR4xwy/HhGSEAAADvVVNTo64oyYoMdGxh0LBFvaU/fPmZu04BQQgAAMB7VVRURIpcp0+J8eeVlJS46xS+cmvUaDTm5uZSXYUn0On0Xr16wVQXAIDuQSqVKnRml43lGlNoaKi7TuErQbh58+YXXnghOjqa6kI6XW5u7unTp1NTU6kuBAAA3EAmk2F+gWer1H2kfvYtJEJfXq6a8+ZH7jqFrwQhjuM5OTk//vgj1YV0ul69esHkywCA7mT1+k0zcsanl2n6B7IbzPiWYs3IqTMm5kxy1/F9JQgBAAB0UREREUfPXti5c+f5f08GSkK+Gjykd+/ebjw+BCEAAABvR6PRJk6cOHHiRISQVqt188HdezgAAACga4EgBAAA4NMgCAEAAPg0CELqFRUVVVdXO15eunSptLS02T3Pnz+vUCg8VRcAAPgECEKKHTt2bOvWrZMnT7a/rK+vP3bsWGRkZLM7Z2Rk/PzzzziOe7BAAADo5iAIKbZy5crx48evWrXK/nLFihUPPfRQK/v7SDckAAB4DAQhxXJzcxMTE+0TweA4rlKpnNfZ0uv1CCGCIAwGg31LUlLS+fPnKSkVAAC6JQhCyhiNxjVr1phMpi1btuh0OoTQ+fPnY2JiHDts3779woULTz311G+//fboo4/m5+fbt+M4bg9IAAAAHee7DfW1JnRRSXrmXAl+KErgOgs2l8tNSEjIysp64IEH7FvKy8tDQkLsf7527VqfPn0iIiKefvrpjz/+OCIiwjFRalBQUGVlZUJCgmeKBwCA7s13g/B4NfFlLuGZc02Joi1MaWY5iAsXLvTq1cvx0mQyMZlM+5+Tk5MRQkqlMjg4mMPhDBkyxLEbi8UyGo2dXDIAAPgK3w3CSVG0SVEU3xm+ePHi/PnzHS9DQkJu3Lhh/7PZbEYIHTx4cMCAAfY9HZGpUqmkUqnHiwUAgO4JnhFSKTc313m9pJSUlIKCAvufFy1adPjw4dOnT4tEooqKirq6OsduDQ0NwcHBnq4VAAC6KQhCyuA4LhKJOByOY4tUKnX0CL7xxhsIobfeeisrK0ulUo0YMcK+XafThYaGwrq7AADgLhCE1MjJyTlw4MDMmTNdtk+ePPnAgQMIoaCgoOzsbBaL1bdv37S0NMcOv/zyy8KFCz1aKwAAdGsQhNSYPXu2Xq9/+OGHXbZnZWXJ5fL6+vpm33Xt2jWZTNbSvDMAAADawXcHy1BrxowZLf3VvHnz7G2FTUVERNhHkwIAAHAXCMK2KioqOnDgQF1dnUgkGjJkiPMgF7cTCAR3tR0AAEC7URCEarV63759RUVFUql0ypQpzl/ue/bsuXjxYmJi4uTJk71nPIhKpVo4/8mt27b3Cg+Scpn1ZtuLz9cOHjRo1Zq1cJcSAAC6OgqeEWZlZa1du1apVP7yyy8pKSlVVVX27UuXLn322WeNRuObb765YMECzxfWLIPBMGLokMqzRw8/MmjrpF5fj0pZP7HnqTn3+dXIBw24p92LIpWWlm7evPnkyZMIocrKyn379rm1agAAAG1FQRDu379/x44d77///u7du8PCwtatW4cQamho+PTTT3///fc33nhj7969P/30U0lJiedra+qj5R8S9dWrx/UME97qcwjgsj4ZmdxTRH950aL2Hfaff/4JCAjYvn07Qmjbtm0tLUAIAACgs1EQhIGBgfY/YBjG4XAYDAZC6NixY1Kp1D4SRCKR9OnT5++///Z8bU39+P3KZzLCWXTXHxSG0KJ+0Ztvzpd9tyZOnLh9+/bx48cjhA4dOpSVldXxUgEAALQDlYNldu/efeXKlQ0bNiCEFAqFTCZz/JVUKq2srGzpjTiOv/rqq/YERQj169dv0qRJLe1sNptZLFb7FrPVaDQlFYo+45Ka/dsegQIWg1ZQUJCZmXm3Rw4ICDhx4sRnn31GkmRhYWFcXFw7ymuJxWKxz9BGFbPZbLPZbDYbhTX4IPtHneoqfIvZbKbT6d4zoMFH3NVHnclk0mh3uOSjLAjPnDkzZ86cX3/9VSKRIIQwDCPJW2tBkCTZ+mdLLBY7gpDFYrXy76TRaDQarX2fVJPJhBDiMFo8OJvBaN/81yRJCoVCGo124sSJlJSUdhyhFfZ/snuP2Y4CqK3BB8HP3PM68vUC2s3tH3VqgvDixYsTJkxYuXJldna2fYtMJnMeeFJVVRUaGtrS2xkMxuLFi7lcblvOxWQymUwmnU5vR51BQUF8LqeowdBTImr6t2qztV5niIqKaseRMQx7/PHH9+zZs379esf0ae7CYDAcq1hQwmazMRgMx28qwDPsH3Wqq/At9p85BKGHuf2jTsHvj3l5eePGjfvss8+c72fed999NTU1V65cQQgpFIpz586NHDnS87W5oNFoY8eM3pDX/NDQjbmVPVOSwsPD23HklStXxsTEjBo1SqlUOtYjBAAA4HkU/M4+ZcoUOp2+devWrVu3IoSys7OfeOIJkUj08ssv5+TkzJgxY8eOHfPmzYuIiPB8bU29uezdfn369AoSPJAS5rz9SKny43+Ltv7+R/sOSxCEWq3+4osv3n33XT6f745KAQAAtAcFQbh8+XLn52qOcSJLly4dPHjw+fPnP/roo9GjR3u+sGalpKRs3rp11ozpf8jrx0aLI0TcWoN5f2nD3/Kaz7/8ctSoUe077IIFCywWy9ixY91bLQAAgLtFQRBOmDChpb/KysrywkaCsWPH5hVc/+brr3/fs0uRrwgMCBg6dtqnC5/q4FBPGOAHAADeAIYztIlUKn3zrbfefOstqgsBAADgZjDYGgAAgE+DK0JqXLt2LSYmxnl5eoSQxWI5d+6cUqkMDQ3t2bMntB8AAIAHwFctNXr16nXy5EnnKWkKCwuzs7OlUml4eHhhYSGbzT5+/DiFFQIAgI+AIPQWy5YtGzly5MqVK+0vq6urqa0HAAB8BDwj9BYqlUooFDpehoSEUFgMAAD4Dh++IiwtRadOeehcqanoThOKLliwYOrUqadOncrKysrOzh4yZAjM2wQAAB7gw0GYl4c2bfLQuXD8jkE4ZsyY/Pz8rVu3HjlyZMWKFdnZ2Vu2bIEsBACAzubDQThqFGrvvDCdJCIi4rnnnnvuuecKCwuTkpJOnjw5cOBAqosCAIBuDp4Regvn1fsiIyPZbLbFYqGwHgAA8BE+fEVItW+//VYqldr/nJOT88EHHwQGBvbv35/JZP7yyy/R0dH9+/entkIAAPAFEITUePvtt50vAel0+nvvvbdjx44zZ84ghCZOnPjII4+0ccFFAAAAHQFBSI0lS5Y03fjCCy94vhIAAPBx8IwQAACAT4MgBAAA4NMgCKlx4cIFnU5n/7PBYDh79qxjsWKtVnvhwgWEkEKhKCwsbPregoKC7du379y5s6SkxGMFAwBAdwVBSI158+ZtutnOv2PHjr59++7evdv+cv369QsWLEAIrVq16qWXXnJ543PPPTds2LBt27atWbNmwIABe/bs8WTZAADQ/cBgGWpkZWUdOnRo7ty5CKGDBw+OGjXq0KFDU6dOtb8cNmxYs++6fv36N998U1ZWZp+JFMdxx3UkAACA9oErQmoMHTr0wIED9j8fOnTotddeO3jwoOPl0KFDm32XRqOh0WiOedcYDIbzPN0AAADawXevCAtVhX/L//bMuQaED0gPSXfeMmTIkMrKyuLiYh6PZzabBw0apNFo6urqGhoaqqurBw0a1OxxMjIyhg8f3qNHj5EjRw4ePHjq1Knh4eEe+RcAAEC35btBWKYuO6s465lz+XP8XYLQz8+vd+/eBw8e5PP5Q4YMQQgNHjz4yJEjSqWyb9++LV3n0Wi0nTt3Hjx4cP/+/evWrXv11Vd37NjR0n1UAAAAbeG7QZgVnZUVnUVlAVlZhw4dcgThkCFDDh06pFQqWw82DMOGDRs2bNiwd955Z968eR988AEEIQAAdAQ8I6TM0KFDDx06dPDgQfsTQXsQHj58uKUHhAghi8VCkqTjZXh4OEEQnqgVAAC6L9+9IqTckCFDSktLQ0NDY2NjEUKJiYnV1dV1dXXODwivXLni6KDw9/cfNmzY/Pnzp02bFh0dfePGjRUrVqxevZqa6gEAoLuAIKSMn5/fV199FRgY6Njy2WefaTQagUBgfzl69Gjnh4VCobB///4rVqw4duzYmTNnQkJCjh07lp6e7npcAAAAdwOCkErz5893fjl9+nTnl/3792+6EtOIESNGjBjR6ZUBAIDPgGeEAAAAfBoEIQAAAJ8GQQgAAMCnQRACAADwaRCEAAAAfBoEIQAAAJ/mQ+0T+fn5K1eupLqKTqdUKqkuAQAAuhJfCcKBAweeOnXq7FkPzbJNofvvvz86OprqKgAAoMvwlSBMSkr69ttvqa4CAACA14FnhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAAHwaBCEAAACfBkEIAADAp0EQAgAA8GkQhAAAALoMnEBWws3HZLj5eAAAAIBbWQhUridLdahUS5bqyckyLMDPnceHIAQAAOB1dFZUoSfL9EiuJRV6kuzMc0EQAgAA8AoqMyrVkfb/1Zo8d14IQgAAANQgSLLOhMr0SK4hi3VIb+3UC78WQRACAADwHIuNrDLar/xQqQ6ZbNSEnzMKgjA/P3/Lli3nzp0LCwtbsWKFY3tVVdWiRYsuXLiQlJT00UcfxcTEeL42AAAAbqfHUbmOLNOjUh1ZoSe9IPtuQ0EQXrx4sby8nMPhnD592nn77Nmz4+Li/vjjj1WrVuXk5Fy6dAnDMM+XBwAAoOO01sYHfmV6pDAgsnPHu3QIRlVxq1ev/v77748fP25/mZeXl5GRUVtbKxAIbDabTCbbvHnzkCFDmn0vj8dTKpVcLrctJ9JqtUKh0G11gzYwmUwMBoPBgBvvHgUfdc/T6/U8Hg9+ZbdzPPAr1ZElOtRg7qxwmRZmTJMK3HhAb/mqunLlSo8ePQQCAUKITqdnZmZevny5pSAEAADgDWwkWW1Aci1ZqkNletKIU11Qu3hLENbW1vr7+zteisXi6urqlnY2m83JycmO38LGjBmzfPnylnbW6/Xw+5qHwRUhJeCj7nkGg4EgCF/7sZttpMJEK9NjZTpUbsRwd8/zckcGg0Gna+vOHA7njt9F3vJV5efnp9frHS+1Wq1YLG5pZxaLtWvXLg6HY38ZEhLC5/Nb2pkkSfuFJvAYxk1UF+Jb4KPueRiG+citUa3VPsiTLNMj5/Z2GgOxPF4Mj8dz70fdW76qYmJiCgsLcRy3f3sWFBTMmzevpZ0xDIuJiWnjM0IAAADtQFV7u+dREIRWq1Wn0xkMBhzHVSoVk8kUCAQDBgwIDAxcu3bt448/vnPnTpVKNWbMGM/XBgAAPosgUZWBLNWTZTpUpCUNXfOBXztQEIQHDx6cMWOG/c9xcXHjxo1bt24dhmE//fTTzJkz33jjDRzHf/75Z8edTwAAAJ3E0d4u16AyPbIS3tvk0HkoCMLs7Oz6+vqm2wcOHFhUVKRSqfz9/el0uucLAwAAX6CzkhV65LXt7Z7nLc8I7Wg0WmBgINVVAABAd2Nvb5dryFIdWWemrIPcO3lXEAIAAHAL5/b2Yi2ptjj/JaTgbSAIAQCgm7ASqFJPfL/qh30b1qjqleLw+N6PvS5LzqS6Lm8HQQgAAF2Y2UZWGJB99fYSHbnlmdF9iaqvewYH8aJya+tff3N67YzF6VPmU11mR5k0qoqrpw0NdSEJ6Sgszr0H75J3imGuUS8HM8tQAj7qnkfVXKO3zWft1N6ef2Arc9Pr794b6djTYiNG/Z73wLqLbL7Iw0W60bWda87+8FZ2hF8QkzxeZ8aCwjZt3drKpCt3C76qAACgC3C0t8u1SNXCfNblR7a/FHNb4LHotKxwv/LLp+IGZHukTPcrv3yq6OdleyYmchh0hNB8hPYWK+fOmr59zz53nQKCEAAAvNHt7e3IgN/57p3NbOSJXHvP+HRMb+nCE8Pkbvj0//qE2FPQblR04Jq916urq0NCQtxyCghCAADwFhYCletJ+wO/drS3B6QMOHFpXXLQbXfIj1fr7ktId2uZHqWpLIqP9nfZGOfPKykpgSAEAIDuQGdFFXrH6u3I1oFxGz2nPLFm+7cpgaoBoWKEEE6QX12opMf39ZNFua9eT+MFSBQ6jYh92yzbCr1ZIpG46xQQhAAA4GnO81m7sb2dzRdN+vLvd9+fbzydF8xnl6kNCWMeyn78NbccnCoJE+d9/tN/v8q6FYS5dVoDix8dHe2uU8CoUeB+MGqUEvBR97y2jxq9vb0dqS2d+8Vrs1oMqlqhJOyOe9YUXr38y/KG0nxhSETipAXR/Yd3amHtc+TTF/Sn/3w0XiTmME9UG/+ptWzduSs+Pt5dx4cgBO4HQUgJ+Kh7XutBaCWQwmC/7EOlOtJk83B1d3Zx89elmz55NUOSGCgo1xg/u1xrTh2etfhrqutqRs2NK/LDv1tV1eLUgf/3yPjeEW7rnUBwaxQAANzIbEMVBlKuQaU6stKAcC9ezMGgqr3220e7J/Zg0mkIITGH+cMI0cN7/6648m9YWn+qq3MliU+TxKfZ/8xgGt17cAhCAICPIghi7Y8/7t62WaPV9Ok/4PklrwQHB7fjOLe1txtQV7nNVnTm4IRIkT0FHR6JF/16aJsXBmGngiAEAPgik8k0YeSwNJr22dgAQRjnRN7BkQO2rPptU7/+bcoAlRnJtWRBHVZlsamtnp5Zxi2sRr0/03WjkM3AdSoqyqESBCEAwBd9+dkng1j6J9LD7S8nxksyJMJn5s09eelqs/s7t7fLtaQRRwghq5XGZHbJFEQIBcemnPzdOuf2jadrjP5DfetyEEEQAgB8065tW1dk3HYjNFzEFSJrZWVlaGiofYtze3upnsQJKgrtNKGp/Y7TxL9fr5mU0NiQd1bRsLnMOHP0TGoL8zwIQgCAL9Ib9EKW6zLgfmxGtUqr5ZFleiTXks7zWXc/GIZN/OTPX5Y//eX2YwmBwnKNyRYYOXHFX0wOj+rSPA2CEADgi5JTUs9XlfcLvTV3F0GSF6u1WzURdFP3uvRrGUfoP+qtdVajXlVRlBISzhG6zmTmI2h33gUAALoXgiTnLn7trbMKha5xNmqcIN8+VRo9Ygadxaa2Ns9jcvmS+DSfTUEEV4QAAB9hsZFVRnSzvR2ZsNSM/6598MMFMjYpZNHzarWJE+be+9hSqssEFIAgBAB0W3oclesc81mTttuf+EVmDp69/qqmptyi1/aJiKcxmjQTAN8AQQgA6Fbutr1dJAn3TGHAa0EQAgC6Nuf5rEt0qKGF1dsBaAkEIQCg67GRZLUBubS3A9A+EIQAgK7BbCMrDKi7trcDCkEQAgC8l9bauIZRmR517/Z2QCEIQgCAd3Fevb3WRHU1wAdAEAIAKOY8n3WRljTAAz9KKfLO18qv8sWS8PQBbL6I6nI8ofkg3Lp169ixY9u4BDwAANwt5/b2Eh1p9r7V232Qvr56zyvTw6x19wQwqiy0jcs1/Re802NU95+Du/kgXLhwoclkmjFjxlNPPZWenu7hmgAA3ZLOSlboUUvt7YByu16e+locMSg82v7yxd4h079/1T8mNSShJ6V1dbrm5xr9559/FixYsHnz5l69evXt23flypUGg8HDlQEAugGtFV1VkTtKiK+u2j6+TP5WSBytIkp1kIJeR1VeGGipHxQe4NjCZ9L/21uSu+VLCqvyjOaDMDk5+f3336+oqNi4caNYLJ4/f35YWNj8+fMvX77s4foAAF0LQZI1RvJsHbmtmPj0su3jS7ZNcuJsHVlruvMkL12RSaM6tPypX2emrpuetOeVafVlN6iuqJ0aFKUJfhyXjfEBfE3ZdUrq8aTWVp/gcDgPPPDAvn37Ll68OHXq1JUrV/bq1WvYsGF//PFHt/xAAwDax0aSlXryaBXx6w1i+SXy61xiRwlxUUmqLVRX1sl0dVUbH+s/TX9679iofybGvSxW/PVcdtmFY1TX1R78AEmlweqysVJr4gXJKKnHk+68DNOVK1dWrVq1detWBoORk5Oj1+snTZo0fvx4HIehXQD4LrONlGvJgwrypwLivQvkyjxifwVZoCaNuA/9lnzm+6X/lx4wKS6YQcMwhPrL/H8eGXv0o6eorqs9gmNTrhtQoUrv2EIitOJybcLEJyisyjNabJ8wGo0bN25cuXLl8ePHQ0NDn3322SeffDIsLAwhtGvXrpycnP37948ZM8aDpQIAKHbbfNbQ3o6Q4tLxETnxzltChRyOVW8x6Fg8AVVVtQ+GYaPf2fjYf6dODmP3DeLWGa2rr6tDRsyO7ptFdWmdrvkgXLJkycqVK9Vq9fDhwzdt2jRp0iQm89YCJePGjYuLi6uoqPBUkQAAyjja2+VapIL5rG+HIUTHMJeNTDqNwF3vMXYJwbEpD667mLt/04W805zIsEFPTg6MTKS6KE9oPgj/+OOPOXPmLFiwoEePHs3u8Pnnnycm+sQPCABfc3t7OzL40q3OuyWOSjpfVZ8h9XNs0VlwlYXkiMQUVtURdBa757iH0LiHqC7Eo5oPwosXL7JYrFbeNnr06M6pBwBAAQuByvWkfT7rMj2yEhB+bZL55NuLX53y5X20pCAhQqjWYH72cEm/J96iui5wd5oPwtZTEADQDdze3o5s8Mjv7gXHpgz/4I8Xli/ElXkMGs3C4PZf+FncfeOprgvcHZhrFAAf4jyfdZ0Zgz6ojpPEp93/3WHChpM2G53Fproc0B4QhAB0Z86rtxdrkdrinHw+l4JGtfLY5y9VXTxKIwm6QNzvP+/FDsh2y5FpdAaiw9dpVwX/zwFApf379v324/cVFeVJqWlPL3o5Pj7+zu+5EyuBFAb7ZR8q1ZEmmM8aIYSQUVO/ad69r6T5j52UiBCq0ple/GKhrnJR+tQFVJcGKAZBCABlFj4+t/r88XlJgbIkTm71+dnjRrzw+jszZ7dpwN6VK1dOHD9GkujeQYPS0tLMNlSsx6o1ZKmOrDQgHEa7NHHu5+VP9xCOjQ2yv5QKOKuz40f+9EFazuM0BrP19wIvgRGEX3UZk2ZF0uY7GtoHghAAavzzzz/V54+tGBJrfzkkMrCvzP+B1/43PmeSUChs5Y04ji+Y80jpxdMjpFwSoac/X06L73vfq2twgs5iER6pvUtSnPtn5D23dTWw6bS0YGFdcb4kPo2qqkBLaDbcX1ESUFYYUHYjoLwwoOxGQFmhf2WxwT/IPGsWyvjQjeeCIASAGjs2b5gW7ee8hcekDw0THT16dOzYsS29S2VG//e/18Vll98YEWff8lAq+uzc1ZOr3uz72GudW3H3BJfO1KPjVlF1mbiiSFwuF1cUiSvk4gp5sPwazubUxiTXxqWqwmLK0geqwmLrohItPMG0MGOwWwuAIASAGjqN2o/tekfOj440Go3zlqbt7Xu2/Pb3xATnfZ7uHTpixwYIwtbJMoftL903vUeIY4vZRlyt02VEu+0mm6am/MK65cqC8/xAafTIWYnDprjryN0G3WoR1ZQ7Mk8ivxosvyasrdQGh6rCYlThsaqwmNzkaaqw2NqYJCuH55mqIAgBoEZK7z4X9lxNDxE5b7zQYJmRmtp6ezudsDFot03rxaBhDBKGxNxB5sOLv3xsq5BFHxsThBCq1ptfPFrS++El7npAWHRiz6mPFi5OD8rMENUaqr7/bemfu9aO/3Ab1mQONgerUc/k8t1ydi/EsJiFtRXiiiJJ4dXgomv25HPJvIvjH1aFxdbEpeAs1xWgPFoqhecGwJc9+tjjWd981TdEmxLU+ERwS0F1gzDsOCNly/nWlq3FMRpOkM5ZiBOktQ0ryfg4rihg2qpjaz9f9MHvx+3tE/2f+jJmoHsmySJs+OHlT/0xNl7MYSKEgnnsTwcLXz1emLd/Y3L2DJeddcqqIx89W59/hseka3EybdrTGTOewWh0t1RCCYbFFFAuD5bniivkjqs9l8wruG98TVyKWhZFeN+/FIIQAGqIxeJV2/Y8P+8R2rnCED4rt1YnTh88ZNkXlfo7PLWKGz7tu8t/PdUr1LHlu8uKuGH3d3K93QHPP2jka2s748hV+Rd7B/HtKejwcKL4td8H73oAACAASURBVL9dg9Bi0G5dkPVGL//hU5IRQibc9v7RHw6V5We9/E1nFOZ2XI3K8RjPJfNqY5NrYlMrkzKvjpymCotpkEWTtK7x+xkEIQCe42hvl2vIYh3SW+Ozvz6ur6/R1ikmh8e1ceGeAU++9dfSG6f2X50YxsEwtKPcpJWmjJ7/dmtXkaCTWQxaf7brhY4fm2nRq102Xtq28pEY7vCoQPtLDoP+xoDIsX/s1ddX8wNCkDdpmnmSwqsMi6k+PFYVFqsKi3Vknio0BrV8B9j7QRAC0LksNrLKiG62tyNTk7ziB0j4AZK2H5DOZI17f7Mi7/zf5w8hhOIfHipLykAI2SzdfT14LxYU3eNAjc5l4/lqtTghy2Vj3cWjQ0JFLhvvCxUp8i7E30vZYgaOzAuW59rvcAaW3iDodFVYjGvmhcVSVWTngSAEwP30OCrXkTfnsyY741pNlpRhzz/gDQRBMmZcxqrLNx7vKbNfGZWoDcsvKXO+e95lTxqTaSFc2z3NNpLuqaZ+rkZ1M+0ar/aCSgpsDKYj8+T9R9TGPlMTl2oS+numJMpBEALgHret3m5AMJ+1r8l+fe3BFYt+27Y7LVhUa7JW21gj3t0ikoS77CYbOOGPvR+lBd+6KLTaiMMV6ulp/dxbD82G+1WVujbnFeXhLLY982pjU3JHTlOFxSoj481814tUnwJBCEA7Oc9nXaJDDbB6uweRJKlWFBM4Lg6P9ZLxlgw2N+vlr61GvbL0epQ4qGkE2qWOfXDTtm955yufSAvhM+lFDYZXTpanzXyBxWttOqHWNduQLinMtXK4joZ0e3OevSG93SfqrrrkOiw8Hk+pVHK53LbsrNVqW5+wCridyWRiMBgMRjf8Ncu5vV2uJY041QU5sVgsPrKSaME/W09+uSRGxKbT0I16Q6+HXu51PzUTZ1utVibzrm9pErj13K+fFe771WrSCyURGfPejMy4r43vbUtDuv0Opycb0juJwVKrNSnUxjKNqVxrUqiNpVpThcZUmR095Nf7v3XjiSAIgft1syA028gKA7K3t5fqSdxbp/P0kSCUH99d+OUzK4fH2uflMeK2Fw8XM3NeTJ/ypOeLaV8QtlFbGtLtmUd5Q3q7kSSpt1RpTBVaU6XaWKoxVmrNFWpjqdak0BjLmXS+iBsq4kSIuGFCdqgfN1LICRVxwh5LkPYJc+ckaxCEwP26QRBqrY1rGJXpkULfNf4j8ZEg3PTYPav7CkKFt773DVbb2F3yhzcXeL4YdwXhHRvSa2OSa+JSVWEx3tmQfkcmq0plKNKaKrUmhf0POrNCa1Y0GIrpNLaQIxOyZWJ+rIDd+AchRybihLMZzT+5nBZmTJO68wZvF/6qAsC9nFdvrzVRXQ1ogUWtDBUGOW/hMek8GmGzmLvEAvHdsiEdIWQjLAZLnc5cpTLItaZKranKnnYqQ5HaWMpiCMW8GCFHJuSEinkxwcJkISdUyJH5c6OYdOonmYMgBL7r9vmsSYM3PfADLSEwjCBJ2u3t20arzQvXFHTJvODCqxJ5LolhzTTndZGGdJww2a/qdGaFPe3ssaczV2lNlRymWMiRiXmxQo5MyJHZ007Mi/HjRtIwr84ary4OALe7Y3s78HJhmUN3FV6aEH/rEdG5qgZheAJG6cVTd2pIxwmj1qSwp53KILf/2f4Hk1Ul5IQK2FL79ZyQIwv1nyZgy4QcmT8vCkNd756tHTwjBO7nbc8IdVayQo86tb3dG3TFZ4RmnfrE16+U/7uPRuAkk9NzxvM9Jz/ReqQZ1cotC7KmhzFzYvyZdNr+UtV3+ZqcL/b6h8Z4pmbnhnT/shsBimKXhnRVWExtbEpNfJpJ4Hfnw1HEZFVpzQqtSeG4gXnzAZ4ct5kcaSfmxQo5jX8QsKUCtqyVxTTciCSR3ob0VqSxIr2V1OJIZ0U6K9LhpM6KDQ/BN41y5+AgCELgft4QhPb2drmGLNWRdeYu+Tm/W50XhARuvbDxi4oTuy1GXUj6oD6PvsL1C+z4YW1Wy/o5/Z6KZd2fKMEQ0ltty/4tL4sbmrX4qzu+8cLGzytP7SVwa0jG0MzZL7Vxmta70paGdEVUD21kgtc2pDvSTqWXa80KnbnxD2pDGUFa7fctBezG53aOB3gcptgDteEE0uKkzorprKQWd8o8K9LhyIAjHh3xmUjERHwmJmQiPgMJmUjAwARMck60KTPUnf+Pd8kvCAhCL0dJEDq3txdrSbXvzbvZSUFo1mu2zB86JQSbEufPZzIOlqk+u6Ic/eEfkvi0Dh754u8/xhz8/MXMMOeN43fkj/zykFAS1tK7OkNLDelGkbg2NsW5UcGlIb1T2yfaqNkBmSpDkdpYxmII7nZAphvZo05rwXQ4qbUgLY60FmQiSK0Vaa3IhCMOAwmZ9v9hAibi0pCQhQQMTMgi/ZgYreUrTxg1CsAtNpKsNiC5lizVoTI9MuJd77c673f6h7eejKLPTGpcGGFKgqRnEH/hO49NX/1vB49ce+7AvDDXr7NhYYKK3DNJnRaE3rlCeuuaDsh0DFGhcECmjUQGnDTZMK0Vaa2kzoq0jf8jTTZMbSUJwp5whJCJ2QMvWIS4dEzIxARMUsBs5R6rp8cNQRCCLqartLd3GyXHd08dE+m8JV7MZxoqLAZtR2YFQwhhNJqtyR0pG4ncNezF3pAuKcwNLsptabXYi+Mfro1NqYtKJOgUfxk6D8h0DFFpeUDmeCEn1J8TXbZvX/n+PywGXXBqYuYjS3j+QXc+012U1HhJZ7KRWiuyP6jTWhsv6fRWxGYgIQMJmaSAgQlZSMxCkQIkYNC4DOTHxFiNQ2eaTTXvGiILQQi6gNvms+4i7e3dBmGzsuiuySRgMSxGfQeDUHrPmF07P8iU3lrigCDJA+WacWn33O2hONqGgHJ5s815joZ0+6BNahvS7QMyb3WUtzogMzJgUCsDMnGzccvC4cMFxkXxYgGLfazi4Cdzt4x8Z5MspU/b6zHhSIvfvIzDG+9e6nBSa8U0VtJ285KOy2i8eynjokRR00s670q1doAgBF7K0d4u15IqM9XV+DD/0Ji8Om1S0K3Ms9oIhc7MF9/FGorNShk1Y8OGFcGXqx5LkTDptDqD5f9OloWPfLD11Rm9vyG97QMyxbyYyIBB7RuQee6Xj6cFmOanN95DnpQgyZAI57772Kx1Fx374CQy4qTOijVexuHIdDP2dFZMbSVZdCRkYPZLOi6j8e6lkEkTMpGIibG7ziVdB0EQAm9xe3s7MsADP++Q+cTbi9+a9cPwGAmfjRAy4bZXj5em3v+fjt/AxGj0B747fHrt++v3bEBWC0PgnzHnvcRhUxw7eHNDuj3tVLoyrbm09QGZMlFmomS82wdkFh3atnzwbfNtRvpx+TbzpgtVJoFEa8WMNvLmgBTSMSDFcfdSyCL9mLSbA1K6Vaq1A4waBe7X9lGjFgJVGchSHSnXoDI9shJd79PoPTqvfaL43/1HP3pGxsG4TNp1pT595vMZM59z7ynu2JDuPM20e0/dijsOyBSwpAGCuM4bkNnagJT/pp2eHE2/Pf4f+ruE9dL2oIi4NgxI6cJg1Cjo8m5vb0dNh0vckUnbIP/3b52iWBydHDcg2wvn1upmovuPjN54TVtTYTUbB4bFdvBasK0rpHukIb2DAzI73j7R+oAU+yUdl97MgJQL8ckXq6ucn7DaSLJMrX+wRySdaY+/bhmCnQKCEHiC83zWHWxvv3Fw+8kvFk2OFEbysEsniV+/WJT99oaQxHQ3VtsKkiAu//Hj5d8+xqwmgsYIv2fUwIXvsb2ymdrt7ra3r/WG9NqYlNq4FHujgjIqwdyxcTeta8eATDfOkOk8IMV+SWfEbw5IsZA2stUBKQzHc0PXVGPPe/3VVyevGcGRCjgIIZwg3/63LG7ULDqzi80u5A3g1ihwP5PJRKPTG3DGzfZ2pLa452OmVpTsfjpr+7hEAavxG6pEbXj4YMXs36545r//g8ufCpcfWtovnM+kkwhtyq/5ptg6c/W/3vDtQ+EUa3fRkB7dw8LtlOa2uxqQKebFumWGTKvVijGYLgNS7Jd0Rhw1HZAiZCEhw/7cDhMykYiF2B24ui45e+jI8oVhbCRg0a/VapJy5vWb8yq1c656BtwaBd7LSiCFgSzVkYUqrMKILKT7W/yu7V63ICnAkYIIoSg/3pAQbsn5I7H9R7j9dC60NRWq03+tm9jD/hJDaHoPSZmu/MruX3rlzO3ss3sJ54Z0yc07nB5rSG92QKaqQd5gLiEwXMgJu3kbs0MDMl00O0OK/ZLOaKOZcMJlQIr9ko5Dxzp7QEpUn6FR669qqsssBl2fiHh4RtBuEISgQ8w2VGEgb7a3I5wgEUI4jmg01Bm/mBqriqJErpPtxvOxM1Vl7j9ZE5XXzmaFut5dyA4XvXv2b9Qdg7AtDemlvQe5vSH9LmbIxGJq/ziUYDT3p0crddixasPgxUuj+2Tf1elsBDLYGi/pjDbkPCDFfklHQ83MkGLvMWCQhJjbSvx44imdKCTCA2fp3iAIwV2jsL2dGxJdVnS+3+0bCw2kZ6amxLBmZ0IhPTMff6dqpTmvkxrS2z5DpkyUmSKb1nRAJkmSvz3SZ0V6QN/ExjE19UbLtA+eFH99yE9621Q4Lpd0RqKZASn2SzoO7bYBKXe8pLNaO/6TANSDIARtojLbp/QkS3VIZabsuXLS2NnfPPPDmNhgHrPxu7hcYzykMMzOHOKBs4el9dv1uWZJ3zDnb8RdpVrp+LEeOLu7eKwhvbNnyKzKO9+DbekrlTm2BHBZz6UGfP3r98LpbzkGpKgtJEHedknHYbQ0IKWpLv8rDmgLCELQPO9sb/cPjcn4z4fjvl7yQIwoms+4rLbuqTRlL9tAZ7E9cHZ+QEjYsBlP/bN92YDwAC4LJ8gfr1Yd1HEeyJ7ugbO3gwca0j0zIBMnkNHmOiCl8rx8ptD1IIkBfGtZnpiFZLybM6SwsJsDUrr/DCleqCr/wr9fL9FUFtGYrKhBE/o/tpTZOaOlOgKCENxiIVC5vvGBn9e2tyeOfCCi/4iCY3vOV8r9Y9NmDRrrmRS0u/ep9/IP9J2+5h2bQYMYrLjhD9y/7BUa1fM1I3tzXnGBpKasM1ZIb+cMmdxoDGvrBaW9x6DxMu72ASlanMRtrjOkyLjIPzxYccT1I1qlM4eHh94jgXjzCvn7N+avfOXDgWFJfROsNuK3gv0/zP1zxo8nOjhLrdtB+4Sv01lRhZ7sSHt7UziO02g0mg8M46ZESw3p9aHRDeFxqrDY2tjk2tjU+oi4u2pI79Qly+84IMXeY8Chk44BKQJmY4+Bvd+gmWNaLetmpGwZFW2f+w0hZCPJGbsLer22UZac2fZ/eEd4w3qEXoskbD/dn7h7QrzzMO81V6uOxk8aMO+1jhwZ2ieAGzi3t9eaqK4GtKDZ5rxWGtLb0kfY9gGZTjNkxnKY/q0fFrV1Fdb2DEhp8efDZA1/be30tx+dHcPvHSxQ6Izf5asjxz/hsRQErVOWXk8Uc51TECE0MTZgw8k9qGNB6HYQhD7BsXq7XEMW65De2vVuA3RvLqvFiivkEnluYEmBXhzsaEi3Z15bGtLbMSDTjxvBot/hxknTGVJurcLa3ICUSIH9kq4TV2GN6D3ogZ/Ond/966GCs5yY6KFPTA+ITGj30YB7ETZb0wW8WDSaDbdQUk8rIAi7LYuNrDLar/xQqQ6ZbBB+XqGNDenHwmJrY5Ot7Obv/zcdkFlVe6Gm4qwJ0+J8KxPjBfr1uNsBmc0OSGmyCivW/CqsLIxF0YAUNl+UOW1B5x0ftFtgRPyeWg1OkAzarQ/A4fJ6ac97KayqWRCE3Yq9vV2usT/wIz2ffYQNL7t4XFlW6CeNiM4c4g0Tj1Gogw3pOGHSGuRtGZBpKK6yHjm3KDw4yy+ZXop9f1V5iRWU8+l2l6d3Jhw1mJpfhVVrJXGimQEp3XIVVuAZdBY7Zep/nj24+v17o0RsBkLotKLhw8uqKd+9QnVprmCwTJd3W3u7AVH4f2j19cv7X5vVz5+eIsSu68ljNaaspWvC0we65eB5+zdd+e1jk0bFEfqnPPBs8pgHvaqNvS0N6TVxqU0b0ts5Q6bTgEyryfDbzNS/JvXgMm4ddsmxkkvZb/Dvub+tA1KYiEPZsu1dGAyWuaOru34+v+ZdLsItBCGI6DF48Vf+oTEdPKbbB8t4VxDqdLqCgoLY2Fh//9Yezvt4EDoe+JXqyBIdaqCuvd0Zbjb++mD6z8PCo/wap5es0plm7C2atvYMVxTQwYMfePeJkOLjr/aRBfPYSqPlg3OKIkmv0W/90uGq26OlzFOFx9bEpjivnOdoSHfXgMxmB6Sorhwa8OdznwwKd97zUrXmhZrw2Jd+FbKQgGGf9BKjedFvDt0EBGEb4WYjncVx1y+v3XnU6I4dO+bOnZuQkFBQUPDJJ588+uijVFfkRZzb2+Va0ohTXVATN07sHRvGc6QgQkgq4DwY73f5wNbek+d15Mg1hVdtVw9/Oibe/jKQy/pwUNSj+89V5p4JTenboaLvpJmG9MKrJI3WbEO6CW9wGpB5TFe/WVV2dwMynVdhrTGQOrXjER1psmFqK0kQzUx62cA2BrJchyTwWXR/QufUTgcZCKjEaOFpt5fwliDEcXzhwoWrVq2aPHnyiRMnxowZM3Xq1G52JXe3zDaywoBuzmdN4u5fy8GdtJXygULXr+MefqyTpXkdPHLpmQOTI13/K5oayfvz371uDEJ75gXLrwbLr9mTL7DkOsFguGReaVRYGcfgNCDzmM68WZVfpL7QpgGZzquwVlpRvhLprGRbVmHlMpAfE2M13r28LdU06WmHf9C5/HNOVWkCUu5u7mnfoa+vqSvO5/kHBkYlesNkCIBy3vIhOHbsmMViycnJQQgNHDgwMjJy9+7d06d76cxVnUdrtQ/ypGA+6w7iBsrKja71lumsnPjIZvdvO8JqYTe5qcemY4TZ3L4DCmsrg4uuOTfnOa+QXhMedXLwIHlYTrEfXo8abs6QeVRn3txQU8JWiVqYITOaSeeh23sManEk1yBtHdLhhNaKaaykjWh1FdZ2DUgRhUQw4vt8dSHvP71CaRiGELpUrfk2T/3Af//Tvp9PN2Yx6A5+sMCQdzI9WFBssv2lwYf+99uIzKFU1wUo5i1BWFJSEhMT45iLJDY2tqSkpKWdCYI4cOAAm904nURcXFxMTEefvlKoLe3tNqvlwvrPK07ttpoMIen39ZnzSscfvLlX/MDRm797dU6KxI/d+MjEYLWtva4a/+L9HTyyNPWevw+unpp028a/q8yS0YNaf2MrDelVkVG5CdFnoiWlPcJLhNIqXr8GXKk1VerM57SmPzlMsVArE9uamSGTQAznVVhVOFKYkFZrH5BCuqzCymXctmSPiImxm7uku6n9dy+zX1t74rulG7ZuiA0Q1hkshDh0woq/vO0T4g3+WjprNlcxfULjipI1evOsZXPGfL5fHB5HbWGAWt4ShAaDwRFsCCEul6vX61vaGcfxDz/8kE5v/FK59957lyxZ0tLOOp3rXSPKESSqNqJyI1aux0r0mNF2h29Ak0694+mRD4TSl6aIuUzhwdKDnz66ZeT724Jikj1TcFvQOPz+z62YtOK5h+NEif6cIo1pzXVNryeWsf2DLZYO9c9KU/ufZgb/eEUxJ1VKwzASoV+vVV20CidnDnUcmW61+NVWBFQUBVQWiyuKAiqLQoqu8RXX5TLx+aSws1JRmYRTGYdVc/yraAlaS7UJzxOw1fYBKQK2VMgJ78EdLGBJBRwZnxWlJ2g6C6azIa0V0+GoyoLpjEhXg5lspMlGcOikgIkEDFLAQAImEtJRKIcU0JGARYoYqLUBKTZksXXkJ9EKrP/8Zb0e/Z+xvqq3OJjFFSCEOvhj7350dQqsMm/6uETHFgmf/UqG5MdfPr7vhc/ad0yr1UoQhFcNYPYFer1eq23r/TIOh3PHAU3eEoQhISEqlcrxUqlUSqXSlnZmsVh79uxp46hRhJB7nzVu3bL5s/eWadVqBos9ZfqMF5e8wuG4LhXbVIvt7XTEoiOjWnn59x+08kvckKiE0bODY1Oc33ty9dtPxbGmJUrsL6ckSlIDec8uXzBt1Qk3/ruc6etrSi8cNWtUksRebX8Olzh0YmTGfef2bzoovyLomzxpyf38AIlb6sn5dOfhla+v2bZZxGaYTXhGxtCXHp0WvPOngLxzvOoCk7pQY6mRh/rly/xuBDMqI1FNgkWRVWNFDCFHKGDzhByJfUBmPCc0kxtLZ4Qw6KFaHHPMkFJtRTeMpE6Daawk1rgK680eAxaS8ht7DDj01mdIoZ4wugfVJXgvbVVJaqDrfAI9g4XaS9fuODVdSzAMg1Gjnsfn84XC7jhqNCMjIz8/v76+PiAgwGKxnD59+t1336W6qGa8/sqSvL3bvuwbHsCV4gS59sjvY3bv+vvYScflqTM9jsp1jvmsW2xvL/53/7EP5j+Z6JcayK+qyf/6fxuDh8++54k3HDuUnto3aVy081sSAwWkOg83GztjLNaF9SvyNq2YGCUSM9HRHdYTzOBxH2zh+gW25b0ckThj6pPumnSbYTEJayslhbnckrO98TJTz6A6c6kKqUv5v6+8tqWcY65LIvFUxLVwAkP6+Qlj7DcwI9myaIaMzoizID/HKqy1VlJuQVq984AUwmVAipBFipg0emuTXnpzCII74Aj8y8yul+R1BgtHFExJPcB7eEsQRkdHT5gwYe7cuS+88MLq1avT0tL69et357fdPZPJdPXqVYPBkJaWJhaL7+q9VVVVuzdv2DI+yf51yKBhj6dJ686Ubdq4YeasB+372Nvb5RqyVEfWme/cpombjYffn799TGwAl4UQSkeikTHBs/b8UjloYmhKn8adCBuzyZR9fBbT2glBWPTv36o/v9yTk2SfFekRhPYXK1csfXDS53+590QuuBoVu+ySufKUtS7PqC5SmksbCGUd01AYSC/n41wRy98/QMSW+olGssQJxXv3ZFTXvRUXGmngCkjGPyXKxb9V097+U4EEmgbS1jjpJcllkM0MSLm1CmvTVIOc684k8Wl/qczVenMI/9ZTmB/ylDGzXqCwKuANvCUIEUJr1qx577333n///ZSUlE8++aQzTrHh11/ee/3/0oP4XDrtTJV6VM6UZcs/bvZirlmnTp0aEiZ0+bLMDhdu27UrIXtWc+3td76LXXrx+BCZwJ6CdjQMW5AcsOavdY4gFEojrtfrEwJu3dUx24hag5UjvLsgb4u8TZ+/3UfmPDfgyOjAz69eNzTU8fyDOnhwG2E2WJTW+gJz5WmLMs+oKVaay1SEUsk0lPGtQis91CrypwcIQmV8wQjcL4kM7BnLiIukh+txptZKanBUacXUZcVJh3/5PKcHutlPMiwq8DGV6dC5tb0feBpWYQUtwWi0Yf+3esZbDy9MFvcJEdYbLd/lqTRRfbOzJlNdGqCYFwWhUCjs1NuhB/7+e+Wypeuz4+zLghCkbMWZfa8tWfzOR20NXYIgmmYmHcOuq2w7StrZ5WdsUMZzXL+dg/ksU73C8TLzibcXvfPwDyNignlshJARty05VpI+49nOeESvriqNSQlx3kIixCGtuxdNMOm1ARHx6XP+Lyy1tYt1nDA1GMr0liq9tVpXf9VcX2DUlmjMCjWhrGUYxEZSpqeFmUUCeiCPHebHG2EUJSkDeouCBuoIRq0VK7WRN5fsQUICCUlMgDVe0nHomJBFVtVckoW7PukZEiY8VHA0mPO0238goDuJ6D3o/tWn9276ckP+WY44JPap2QP6ZlFdFKCeFwVhZ/vsvWWv9QtzLI5Fw7DnMsInbN385vsfMhh3/jmYbaQkqc8/lfpnet+2fV+5VjJiZLurEofHXVG7zhNzTWkQxfV0vIzoda9t0bfTPn5GxqFxGLTCen3vhxZnTFvY7pO2QhgkrdAa48W3kubZPZciOfQX+vClgoC8OtUb7zxYM/GpnjPmu8yQadCWaLQFGnOVntRJzGypFotUmpI1DB49iMkKt/Lu1QhTavz7XIhIu8iXXnYZkMJEsUzHgBQkZLYS8Fgdi2VqMvbShBM0pufWqQddF9cvcMC816muAngXHwrC8rKyuLR45y00DAsXcauqqsLDw5t9y23zWetJEoVxe4/83/Ej/+sXzmPSSYQ259fsrMNmjnmw3VXJkjMPWdjHy+rvjWjs+qrSmT4+eWPMzBHOu0Xfkx29MU9bW4mbjYNCY7BOW/w9YdL8j9f+9+vhcRhCapr1z6qq+lDrlL5Bv9MUpTRDtdCsjzXvIl7evesVMekXauRLNShWaUqoUkkMTC4twsAdpBSmXAlJvhGZcrV33BU/P/sqrEIGsk96OYpFPnBr0sv23L2M6Dlg68fq//YJdb5/u61YEz5tktt+CgAAX+JDQcjlcXUW3GW55Hqjxc/Pz3mLo71driVVTaYuyVr81aXtK8et/4xhs9gwetg9o6Z9+25HFhvCMCxiwJi3/vw2QshJkwgVWtOVWu2L/WNWf/FS+MqjLjsLg0PbfaKWuC5ZHqgoGm9NNB8gBTYGidEkKDGYfVJXFqolByrJUCUeoTCFqTAFL/J6wkB5ZHKtLLY+IfZAZAJdKLy5fA8pQ7ZwekujRjt6O5cjEidNe+ahnd+83T8sIYDfYLJ+fbn6PBYyZdiUDh4ZAOCbvGv1iTZq3+oTy99dptm/cX76rSy5VK35vBLbdfCYYz7rIi1paDKftUFVW3hyn7G2XBybFjdwlH1yQpIk3fWIbv3DGduGSqoNZrlKL+VzUoKFDBo2bkd+2Oi5JUe2WfRa//C4zCfevjWI9O41XbJca1bUG+RaYzmDLmAzpUyGVGiTiHUMSQMZW4snlFYMKbjeu66unsHAAri8YCHux8P9uDY/nlXM//Jq1fXRr6SNJSC+aQAAIABJREFUmtHS6dzVPtGKsgvHLvz4ZoOimCsUJ4ybkz51fuddJXcVFoul3f1woH1g9QlKdPNlmNqofUFoNpvvHz8mRF8zOUrEZdCOKrQbSwyPf7fHFBBrJVr8IVzbuebCj2/dHy0K5dLON9gO1eFj3tvslildSMKmU1YLAqXrZiQfyol3+dtp2871kgU+nxEmYDPy6rT/O1UZ+eAraTmPt3Q0+4BM5yXLlXq52lips1TpjCUMuojNjqHRZYguQ1ikQMcOVdtS6vFBFVVp5flRNfJglaI+UFYni1WGxWoiYoyRsQ1hsbWxyaf//Cn5+DdP9Q5zPtesvYW93t4eFJ3UUjEeCELQFAShh2lrK6tuXBEGSCRxqTRGF45DfX01YbN1xg2nTgJBiFB7g1BnJSv0aMPmrcd3bdPqDYHp96VPebL1Prxaee6xJRO3jE9k32zju16vn3e85qHfrnTk+sOoqT/y8XO1l46GCLkKjYFA2FeDw9MlIscOBElm/XzswEODHI/BTLht1O/50387ZyRV9iXLlXq5ylipNlZpTZUGi8JoUdBpYjpDRmPE2GgyCyZjsGL9aYGRRlq/6oY0RUF0jTy8pkimKBIrK7XBoQ0318yrjUmpiUtRSyObrpCO7Iu+Ppz5bqb4vvAAe2HfX1LsRpETPvq9lX8gBGHrGiqL6orz+QEhIfFpbvwChSBsRfHpf0r+2WiqrxEn9+t1/3/YAr87v6dlFoPun/eeNF8/nRnMV5qJSyrTfS9+ETNwtLuq9ZgbR3ac+GJxCBujYVil3tp//rKklu/0eA8IQoTuMgivVevyjbxSHVl/92sVHFmx6KGGI9mxt0088ezRssBnf2i9haAVhA1fP6f/S/HMcbFBCCESoY15VZ+dLNz74D1CFgMhZMRsyy5er6AZJmRKa2jmapq5BjOX0AzXcYOZQ2OwwjCajKDJCHoskynlsUMFbCmfFRaFRIl1JTG18ojqotBqeYiiKMBphfTa2JSa2BR78rmskH5H2trKg+/PNxRfDRZwKtSGmOHTBsx/u/VfICAIW2JUK/e/8Qi79kZGIK/SaLvUYMn678qIzCFuOTgEYbNIktz7+kP+ZecfS/QL4LL+rdKuKlCPWLahI2t4/fnSpAc5ldN7NM4gqDRaZv11Y/hHu7xq+t87kh/ffeOLZ74ZFhPIZSGE1Gbr0weLQh59Jynb25f96c4L83aS6xp0QdPOsDcoiiOlrt/4sXx6WXV5u4Pw+pGd94nwQfFBBZiuGjNX08zK3mZaCNbbeJTHpun5FpxG0DLpGQzBUZoyhGRHEtx+hH8UwfvxtEoxfkVk3+EhpobIGnlYdVFAsVxccUBcLg+W53I1KscK6crUPjdGP+C8Qnr7lJ4/WpN/jiUUD1n8lSBQqlfVDAsOg/mFO2Ln4skvRVhHZjbeCa/Rm2cue3T814f8pB1drAq05Nq+DbE1Fz8Y3rhATY9AwbAI/9lvPvLQ+qvt+zDrlFVEee708bcm7w7kspZmhny7/tNhr6x0T9Eecfq7pT8PiQq8OZuHH5v5ZVbM5B/e9P4gdLvuH4QdwZNFlzaU9Qi87VePQh0eJAlr6S3OdBZVnb6yTl9Vo5OrTZVas0JvKjIqL9BGq7ai6xKSLSXZEhs7hGQPIEVH+COCI4cLbX5RPQaV/v7FxLLfH0yW0cxWhtpIVxsYaqX1YkWfqpcD3y5uaYV05L6IMjTU7Vw8KRFTDw9iNljR1jVvRI55tP/jr7nr+L6pruiaxKwcGX1ryTAJn/1CWuDWrd8OWuiNM+t2D8V//fJB8m0z5YaLuHF8mrIkv5Xn3K1QVRSlBLhO6ZASLFSdyW1/lVTAdSqp4LbZM/zYTLrVSBKErw09gyBsTdLExz9bsm1IVKDTM0LdZQ0x++ZNFROO6kwqhbao3lChNVapjXK9WWGyKiwWuc1ajjABjSFlMkM57BgeSybk9YkKmqa9/sfc8iMz4m57Lr22ooIe3DMrOCVYfjV4y0ei4jzaiXzpgas0Og3341pE3CMG/Hpy34a5r9bGpmiDZJ39D9+79MElUbbhUVH2l/N6yp74+5cbCRnxQyZ29qm7sfpyeZrYtes/JUiwpugKJfX4CJOmPoDLc9kYzGEYNapm978jjsCv3OQ6uLzOYGnjxPTeo9llAKwE6WspiCAIWyeOSYl66LXsn9+ckMjnimyncNMFqwV7fMJHh2dZrArcpkB4KUYTsVgxbKaMw5KJeLEhwmQxL0wqjJEJI3jNjYNQWANPvLXrEaaSoTEy1Ab7Bd+LdVry37cbIn6zX+cVDp9SNGXewk2f15QXCNhMldmWMnNh5szntHfzbK8p3GysKrhkaKiTxKX6h0a3tJtRU49qiob3v3Xnh4Zhr/aRLdryFQRhR3BF4uomk+LUGsxsP1j9oBP5Ryfl1l4dHHlbSuUq9SPbuxhvUEzyX3pbudYYLrz13GRlbl3M9C42w19Ict9DpSVDnX4y56oa/CJ9cSUvnwhCZUn++TXv1suv8MWS2LGPJo+e6fgrnEBanNRaMLXFWKdXqIyVGpNCb5YbzQqzRWHDFRhbQc6t+crGZVr5HHa0v6S3iBcWxI8N5EmlglAxL5KGtfgzbH6FdPk1E2GrKasShfrRpH5FoewvbYyKUXMynned8nT4feNIwmbWaTgiN0yufePg9hNfvNQ/hB/Eop2oMZARaSNeW83mi5ruqaurChe5PhmN8uOpa0o7XoYvC0vtt67GUGewBPEan8qQCH2bWx/3/KPUFta99Zy1aNni8b8GCx0Pw369VsWMTm/3YpkYhg1/7afZS2fOS/S7RypUGi3f5zfoo/uNHDHNfVV7wr3Pf7p04bAndPiE2AA6hv1VXP95rmrSl+uorosC3X/U6DMfrTnw1Rtv9JGkh/hd1+o/Lqi4JJaQk/5jsFRbLHIaocAIBYkrCELFZsk4TBmfHRrAj/XjSoP4oWKOTMiR+XOjMewO9wroVouoptw58yTyXHG53D5o0z5c0361Vxfdw8LlXz/8x7VNn6ury8Wy6NTZi2P6j2j9+B1UXXDp+P8mrx+dIGQ3xvbW6zVr9dKJH+9ourNRrdw7f8AOp7EACKGiBsPz15kTP9/bltPBqNGWlJ49ePidx+f38OstEVTrzV9fU/H6T7zvuY/dcnAYNdqSohN/Hf3kmXQxJ5hDP12t5SYNGPbf75hc1+d8d8WsU1/c8o3y6kl+UGj0yJmRbhr662FWo/7MT+9Xnt5PEoQsM6vvnFc72FjiGdA+gdDdBKHeYJDMDUmPZ1XRTGXIiAjEM9BxNcmihSQOejjEP86PGyriRIg4YRymfxvPzrCYhbUV9syTyHOD5blip0YF58yrjU22dsLCue3w91tznmXmDwwPcN6Ys7NgxNfHmv29eNtTI5eEGYZENF6JkggtOFDo/+j7CW1bsAaCsBVGTf2lrd+pC87zJOFxYx6WJWW468gQhK0gcGtN4VWDWimJSxUESt11WJhZhhLQPnF3zp45M8AoftQY/M6ugrW9MofKghBCpAitv1r58+UTfb5+p/W3MyymgHJ5sDxXXCF3XO25ZJ68/4iauJS7bc7zsIbSgh79XT83SQE8Vbm82SAc9fYvby/KSS8pHSphqi3k+iKtbPisNqYgaB1XFHDPnFeorsLn0BhMaY/ed97Px9gsZhqDgXnxd5dndPMgNBgMySq/uiuWqUGh9hRECGEIzUoN3f9PsSLvvOP3ca5G5XiM1zTzamNTHI0KHWzOowTXP7DOoHFe/hchVG3EU/2bH+fGDwiZ/uPJ4tP/7Lh2miUKGPbcSP/QmGb3BAB0RfLje059tYRp0eMkyQyQ3ffS1yGJ6VQXRZluHoSpqalvVOssAnJ03K12GXtz3pM2g37NB6kicdOG9E5qzqNQ7JhHvtn4xqeDb10UFjUYSk20wZEJLb0Fw7CY/sNj+g/3SIEAAM/J37+x7Mf//TYkSsIPRwgVKHUL/ztp+Ic7JPFpVJdGjW7+jBAh9MygoZEFF0aLeQkIMRoMDLWBpGH4/7d373FRlPsfwJ+9cNtluV8XuSsogmlAEoqKYmLS8X7UMhMz0PqZpZ08Wp30Z2nq+Ykdq5MvS8sjP0tPR/NSiDcgUfGCyiJq3CJuCiJ3dmF3Zs4fkxMtqFx3WObz/mt2eGb3yzDMZ5955mIjy6ZF+UETJGP/dN994H133yYbh94um0cMw5zesFh6O/1lP2s7c9OMioZ/5ddP+uig86Cgxy/ceRgj5AXGCA3PSMcI/zXb/8gkLyuz3ztCqoq690qsYhKO8VhVx2GMsJMYZrWZ/LKp3eXKygFP++qs5TobGW0m1dHMC4dvxaza3uVTqI2LSCSa8M6XJaqMr05921xTYRsSNm9dbDfPmnsYraapsbbayrHXr/oHgC5oaapXSOjWKUgICXKyqjn/M18l8a6/B6FIdHnH/sw6ixNrF3z/68U3Ha3cpeKbd2r/98odv1mvCyQFOQOCRg4IGtl77383V5X2UZyk4Z7CVFparxk2Z/mIuctxb1KAPkUskWopWm8mzTBMtx+abbz6exASEuZIQpTi2H17Txw7vGXXP+9klrj7+i/5bLv30GAdQzQ6RssQHU00OvLbBMVoaZGWZpr17wHSA7Tqxku7Pii5dIJqaXYe+lRo3HqGoSvybphbWjv7DTMx178RlBGpLi04tWrqjrHug+z8CCEaHfV+ys4LtVVPL1nPd2kA8DupmQVlbqV3Z5zUX++7BIbxWBW/+v8YIfc8wq7R0oyGEulodoLoGKKlyG8Tej+iiZZ+8COK0VAiHUPUut9Xb+P9iv8sGRs3UD7V195UIv6p+P57P+XJZZaRnnY1WjrjTkPokg8HPzP3EcX0BboWzc1T39XlXjV1cBs45k+2D+5TdfrDxa8w2ZGev5+GSjHM+IM58769JTU156lYYcEYoeEZ6RhhadaF1LXzPgpThrraUDSTVHhvk+r+jH+mKjr2OAHeYYzQ0EzEIpPfzvloe9xA1GaiHVqa0dFEx4hWxL/716GKyQ+ebjjBy8HXRrbq1I01IUpCSEOLbuHud4b4enkPf5pNUB1DdLSoUcfQfea7yt2fs5LfmT3d3Xy4nXnlnZbdhz7xeHZxyMLVhJDK21dGjvvDnewlIlGAg1V1SYGjTwBP9QJAO9yGhU3+x8nNn666d0kllkiVI8b8efeHFlZ2j1+yn0IQ9jouSjPPpb07yaf1j7xsZFqaVusoC6nE0lT6fogy8ZvN7/+5nYe/d7xjqtYRHU1aR6maYnT6IwJdwdB08rtz9oxVelr/dgh35mDX54/vKnkycsCwMKmpRZOWkpn84crcRi0lNdV/3gIA8M52gG/0xn/zXUVfgSA0HJqhJW3OHDGTSLQUYyElhJAhDorcM7ntLtsjHdPuHONtppjS29cCraVcChJCxCLR8kCHT45+MWBYmPvomP/k/Dsu6PeTRe81tRQ16iLcfNorBwCgr0AQGo63t0/OvfoAh98HLBu1VI1Gy53HXNnUbG/XW0cnuh+lh8srzij0NxilwlxWUhrtLopYueKtad9pr5c/7+9gaSK5VF677kr5zPd3OlmIerZjCgDQsxCEhvPuhk1L5kzfNtrLw9qCEFLbrF15Invxk55cg903K2YtfJO/Ah/D18trd4NWb2Z+dWNgwLAwJzEh8ouXLn32j20rvvt3XX39EyNGHDm1z8dHvzvY7Y4poY3w9C4A6Mtw1qhBXTh//u1lS6nGOlOJpEqjbdI0z/d3HOlsWdusPVBYJx8Y9NU3+/vyDVkiQkas8DULdf3tSR0NLboFJ3K/Ppw0ZMiQ1s00Go1UKpVKe+VrFnfykVrHPBgNJTqG6GjCdjr1f6Q/Ykp0fefsox6Fs0YNz0jPGjV2eAwTIcYchKzGxsbm5mY7Ozu1Wv31rl1Xzv1k6+Dw7LSZ4yIj+S7tMUpKShbMnuGkawiyktzTiU6X1r3/0ZYZs2brNevVIOwR/bJjiiA0PAQhLxCEhBh/EBq7jIyMnJwcFxeXUaNGWVm184z7vh+E3dcHO6YIQsNDEPIC1xEC/0aOHDlyZC/eqs0ocCcfKUy6ePIR6XbHVKMzwq+xAH0PghCANz14uwa1jqmp05nLzTvVMW3SMRSyFAQPQQhgxFp3TC10RGHZOjUN1TGliDGOsABwEIQAQtezHdMujJg26QiFKAX+IAgBoLv6xIgpOqbQVQhCAOgT0DEFviAIAaCfMHzHtEHN0BJCEVG7T14DY4EgBAD4Xac6po2NtEwmFv3xZvqtO6Zdu8F9P733Ud+FIAQA6End75h2+3YNuMF95yAIAQD6lr5w8lHfvI9gL0EQAgD0Q7yffGREN7hHEAIAQDuE0zFFEAIAQG/pfsf0jxnJaGki7+lTcxGEAADQd5lLCJFwr0SEkPr6Hv6IvvsMWAAAAANAEAIAgKAhCAEAQNAQhAAAIGgIQgAAEDQEIQAACBqCEAAABA1BCAAAgoYgBAAAQUMQAgCAoCEIAQBA0BCEAAAgaAhCAAAQNAQhAAAIGoIQAAAEDUEIAD2jpaWlqKhIq9XyXQhA5yAIAaC7KisrF/x5ZljAoP+ZHh062Hfpopdqamr4Lgqgo/CEegDoFq1W+1xU5FIfiw0xQ9g53+eqZk6JPnn2vEgk4rc2gI5AjxAAuuXI4cMhCmaCpz03Z+ogJ1dtbVpaGo9VAXQcghAAuuX6lUvBduZ6M0NsTa5fvcpLPQCdhSAEgG6RyS2bdJTezEYdI5PLeakHoLMQhADQLc88O+VIcUPrOTTD/FDaMCEqiq+SADoFQQgA3TJixIghY6LeSCvIr27U0cyte/Xxp/Mnz5nv7e3Nd2kAHYKzRgGguxI+/Tzpxx8/+WTbr5nFPj7eaz7dGDFmDN9FAXQUghAAekD05MnRkyfzXQVAV+DQKAAACBqCEAAABA1BCAAAgoYgBAAAQUMQAgCAoCEIAQBA0BCEAAAgaAhCAAAQNAQhAAAIGoIQAAAEDUEIAACChiAEAABBQxACAICgIQgBAEDQEIQAACBoCEIAABA0Hh7M29LSolKpVCqVra3t1KlTufk0Te/duzcrK8vPzy82NtbExMTwtQEAgNDwEIQff/zx559/rlAoZDJZ6yBcvnx5enr64sWLExMTT5069e233xq+NgAAEBoRwzAG/kiapsVi8e7du3fu3Hnu3Dl2ZmVlpYeHx82bN728vGpra11dXa9du+bn59fuO8hksqqqKgsLi458XH19vUKh6LHqoQM0Go1UKpVKefiaJWTY1A2vsbFRJpOJRCK+CxGWHt/UeRgjFIvb+dALFy54enp6eXkRQqytrUNDQ9PS0gxdGQAACE9f+c5+584dR0dH7qWzs3NZWdnDGmu12qVLl3IdjuDg4NjY2Ic11mg0GG40MPQIeYFN3fA0Go1YLEaP0MA6tambmJhIJJJHt+mVXdX27dvffPNN/U+SSjUazUPrkEopiuJearXaR/yeYrE4ODiYazBkyJBHNDYxMcHewcAoikIQGh42dcNj1zmC0MA6tam3ewxST6/sqpYtW7Zs2bJOLaJUKktLS7mXZWVlbm5uD2sskUgWL17cwTFCiUTy2K8D0LMkD/BdiLBgnRseu84RhAbW45t6X7mOMCIioq6uLiMjgxBSUFCgUqkmTZrEd1EAAND/8XDw6uLFi++8805ZWVlpaenEiRPDw8PXrVsnk8nWr18/bdq0KVOmnDx58i9/+Yuzs7PhawMAAKHh4fKJ2travLw87qWNjY2vry87nZ2dnZWV5e/vHxwc/Ih3wOUTfRxOluEFNnXDw+UTvOjxTZ2HXZW1tfXDci4wMDAwMNDA9QAAgJD1lTFCAAAwRhfOn588dvQIP5+QAP+3ly+rq6vju6JOQxACAEAX/f+/9qxa9Pwab/GRZ/2+m+jllpseGRZaW1vLd12dgyAEAICuoChqw/vv7Rw/0NtGRgiRiEQz/ZzmeVj84/+28F1a5yAIAQCgK27fvj3EXi4z+cMlfc942qWdPMFXSV2DIAQAgK6gKEoi1j9jVioWa3U6XurpMgQhAAB0hb+/f3ZFnZaiW89M/fX+06Mj+CqpaxCEAADQFaampvHL3ngzraBao2XnnC25//nPNW/+dQ2/hXUWLnkGAIAuWrZipdLD45V171PNah3DDB024tiZ/zg5OfFdV+f0/x5hUlIS3yUITlZWVnFxMd9VCItWqz116hTfVQjOhQsXqqqq+K6CZzNnzb6gyjmfk3s195e93x0cMGBAr35cfX392bNne/Y9+3kQajSaBQsW8F2F4OzateuHH37guwphKSoqWrlyJd9VCM62bdvS09P5rqJPMNgtFTMzMzds2NCz79nPgxD4Yvh72ALwApt6P4AgBAAAQUMQAgCAoPHwGKbuMzExGTNmjFj8+BSnaTo1NTUyMtIAVQHn1q1blpaWvT1mDq2p1eqrV6+Gh4fzXYiwXL9+XalUOjo68l2IgNTU1OTl5YWEhHSw/fTp01999dVHtzHKINy3b1/Ht7zCwkJvb+9erQf0VFZWWlhYWFpa8l2IgDAMU1RU5OXlxXchwlJWVmZvb29mZsZ3IQKi0+nKy8vd3d072N7b25t75O3DGGUQAgAA9BSMEQIAgKAhCAEAQNAQhAAAIGgIQgAAEDSjv+l2Q0NDUlKSTqeLjo62sbFp24BhmNTU1MLCwvDwcH9/f25+ZWVlcnKyTCaLjo62sLAwYMn9wY0bNy5evDhw4MCIiHaet9LS0pKenl5cXOzu7j527Fj2Qheapk+fPs218fDw8PPzM1zFxu/+/fvHjx83NTWNjo6Wy+V6P62pqbl8+TL3MigoyNnZmZ0uLi4+c+aMg4PDxIkTTUxMDFdxv3D58mWVShUYGBgaGtr2p6dOnWp9vqG7u7u/v79arW5937VBgwZ5enoaotb+oqSk5Oeffx46dCi3DeupqKg4ceKEXC6Pjo42Nzfn5p89ezY3NzckJCQoKKhzH8kYs8rKSl9f35iYmNmzZyuVyqKiorZtFixYEBAQEBcX5+jo+M0337Azc3JyHBwc5s+fHxUVNWzYsPr6esMWbtx2797t5OQUHx/v7++/dOnStg28vb3Dw8MXLlwYEBAwatQotVrNMIxGoyGEREZGRkVFRUVFffbZZwYv3Ijl5+c7OzvPmTNn8uTJ/v7+1dXVeg1SU1MtLS2jHjhz5gw7/6effrKzs1u0aFFYWFhkZKROpzN06cZsw4YN7u7u8fHxHh4eH3zwQdsG0dHR3Do3NzffuHEjwzD5+flSqZSbv3//foMXbsSCgoIUCoWZmRm3u9aTnZ1tb2//4osvjh8/fvjw4Q0NDez85cuXDxw4MD4+3sXFZceOHZ36UOMOwvXr18fExLDTsbGxy5cv12uQlZVlbW1dVVXFMMyhQ4d8fHwoimIYZv78+StWrGAYhqKo0aNHf/rpp4Yt3IhptVqlUnn8+HGGYe7evSuXy3Nzc/Xa5OXlsRNqtdrT0zMxMZF5EIT4ztE1S5YsiY+PZ6cnTZq0efNmvQapqamBgYFtF4yMjNy6dSvDMM3NzX5+focOHertUvuN6upquVyenZ3NMMzNmzdlMtn9+/cf1riwsFAqlf76668Mw+Tn59vY2Biu0P6loKCAoqgnnnjiYUE4b968t99+m2EYiqLCw8PZzPvll18sLCxKS0sZhklNTXV0dNRoNB3/UOMeIzx69OisWbPY6VmzZh09erRtg/Hjx9vZ2RFCnn322fLy8pycnNYLisXiGTNmtF0QHiYzM7OpqSkqKooQ4uTkFBERcezYMb023OWr5ubmtra2LS0t3I/S09NTUlJqamoMVnD/cPTo0ZkzZ7LTM2fObHeLbW5uTk5OzsjIYL9zEEIaGxtTUlLYBU1NTZ977jls6h135swZd3f3oUOHEkIGDx7s4+Nz8uTJhzXetWvXM888w13lzQ4EnD17tqGhwUDl9hfe3t6PvmsY978gFounT5/ObtI//vjjU089pVQqCSEREREikSgjI6PjH2rcQVhaWurm5sZOu7m5sV8H9BpwN/oyMTFxcnIqLS1tamqqqanRW9CQZRu10tJSV1dXbkt99No7fPhweXn5c889x750cXHZvn37mjVrvL29Dx48aIhy+wWapu/cucNtyQ9b52Kx+JNPPlm0aFFAQEB2djYhpKysjBDC7h0esSC0q/Xegzxy7dE0/fXXXy9atIibY2dnt23bNvZgXVpaWq/XKhh1dXX19fVt/xda/7FEIpFSqezUpm7cJ8tQFMXtkSUSCUVReg10Ol3rux9JpVKdTsc2E4lE3II6nc4g9fYHFEVxq448cu1duXLllVde2bt3r729PSHEzMyspKREIpEQQvbs2RMbGztlyhRTU1PDlG3UaJqmafrRW+yoUaNu3bpFCGEYZtmyZa+//vrp06fZPxY29a7R29TZvUe7LZOTkxsbG2NiYtiXXl5eBQUF7LIbN258+eWXc3NzDVCwEDxs793xP1a7jLtH6OrqWlFRwU7fvXvX1dW19boghCiVSq4BTdMVFRVKpVKhUMjl8srKSm5B7iszPJarqyu36siD1d62mUqliomJ2bFjx8SJE7mZbAoSQubOnVtXV1dYWNjb1fYPUqnU0dHx0Vsst25FItHcuXOvXbtGCHFxcaFp+t69e9yC7f6xoF2tdy/kkTuKXbt2vfTSS9x3brFYzO2Inn/++by8PBwg7Sm2trbm5uZt/xc6/sdql3EH4bhx444fP85OJycnjxs3jp2urq7WarVsg5SUFHaM6vz58xYWFuwR/3HjxiUnJ7ddEB5r+PDhLS0tmZmZhBC1Wp2WlsY+3KOlpYUb+bt9+/bkyZO3bt06bdq0dt/k6tWrYrGYOzoNj/WwTb2qqqrtgZDMzEx2sMrGxmb48OHsps4wzIkTJ/Aklo4bPXr0rVu3ysvLCSF3795VqVTsxUIajaY4mgBGAAAEWUlEQVSuro5rVlVVdfjw4ZdeeqndN7ly5YqdnR1uQN9Nzc3NtbW17HS7/wvjxo07d+4c+4UjJyenurq63ctdHqrb5/jwqaCgwNbW9q233vrb3/5mbW19/fp1dr6Dg8P333/PTo8ZMyY6Onrr1q2+vr5btmxhZ6alpVlZWX3wwQevvfaai4vL3bt3+fkFjNPatWv9/f0TEhImTJgQHR3Nzty3b5+npyc77ebmNmjQoLgH2L9FYmLiCy+88NFHH61atcrBweG9997jq35jdPnyZSsrq7Vr165YscLBwYE9O7G5uZkQcvnyZYZhVq9evXTp0i1btsTHx1taWnJnhx44cMDR0XHz5s0vvPCCn59fU1MTn7+GsYmLiwsODt62bVtoaOjLL7/Mzvz73/8eHh7OtUlISBg5cmTrpRISEhYtWrRp06Y33njDysoKVwp1yrZt2+Li4uzt7SdMmBAXF3f79m2GYXbu3BkQEMA2SElJsba2/vDDD1999VX2ABU7f+rUqWPHjk1ISAgICFizZk2nPtTonz5RWFiYmJhIUdScOXMGDx7Mzty7d++YMWM8PDwIIWq1+quvviouLh41atSUKVO4Ba9du3bw4EELC4sFCxbg0GhnHTp0KCMjw9vbmzsiVFBQcPHixblz5xJCvvjiC5qmucZPPvlkSEjInTt3jhw5UlhYqFAoIiIiRo8ezVv1xunGjRsHDhwwNTWdP38+u23TNP3ll19Onz7dwcHh5s2bycnJ5eXlTk5OMTExrW9WcPbs2aSkJDs7u4ULF7JnUEMH0TS9b98+9oL6efPmscefVSrVL7/8wp0ClpSUZG9v37r/UVRUdOzYseLiYltb24kTJ44YMYKf6o3TDz/8UFJSwr2MiYlRKpW3b99WqVTcNQJXr149ePCgXC5/8cUXub13S0vLnj178vPzQ0NDp0+frjdM9mhGH4QAAADdYdxjhAAAAN2EIAQAAEFDEAIAgKAhCAEAQNAQhAAAIGgIQgAAEDQEIQAACBqCEAAABA1BCAAAgoYgBAAAQUMQAhixgwcP2tnZ7d27l5uzbt06Jycn9vEgANARuNcogHGLjY09cODApUuXhgwZkpKSEhUVtX79+tWrV/NdF4DRQBACGDe1Wh0WFkbT9NGjR59++unBgwefOHGCe04vADwWghDA6N24ceOpp56SSqVyufzatWtOTk58VwRgTDBGCGD0hg4dOn78+Lq6utdeew0pCNBZ6BECGL3ExMT58+cHBQUVFRVlZmb6+vryXRGAMUEQAhi3vLy84ODgGTNmbN++PSQkxNLSMj093czMjO+6AIwGghDAiDU3N4eHh9fX11+5ckWhUGRlZYWFhcXHxyckJPBdGoDRwBghgBFbuXJlTk7O/v37FQoFIWTYsGGbNm36+OOPDx06xHdpAEYDPUIAABA09AgBAEDQEIQAACBoCEIAABA0BCEAAAgaghAAAAQNQQgAAIKGIAQAAEH7L/DgwBpAGTFQAAAAAElFTkSuQmCC", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using Plots, Distributions, LaTeXStrings, LinearAlgebra\n", "\n", "# Model specification: y|x ~ 𝒩(f(x), v(x))\n", "f(x) = 5*x .- 2 \n", "v(x) = 10*exp.(2*x.^2) .- 9.5 # input dependent noise variance\n", "x_test = [0.0, 1.0]\n", "plot(x_test, f(x_test), ribbon=sqrt.(v(x_test)), label=L\"f(x)\") # plot f(x)\n", "\n", "# Generate N samples (x,y), where x ~ Unif[0,1]\n", "N = 50\n", "x = rand(N)\n", "y = f(x) + sqrt.(v(x)) .* randn(N)\n", "scatter!(x, y, xlabel=\"x\", ylabel=\"y\", label=L\"y\") # Plot samples\n", "\n", "# Add constant to input so we can estimate both the offset and the slope\n", "_x = [x ones(N)]\n", "_x_test = hcat(x_test, ones(2))\n", "\n", "# LS regression\n", "w_ls = pinv(_x) * y\n", "plot!(x_test, _x_test*w_ls, color=:red, label=\"LS\") # plot LS solution\n", "\n", "# Weighted LS regression\n", "W = Diagonal(1 ./ v(x)) # weight matrix\n", "w_wls = inv(_x'*W*_x) * _x' * W * y\n", "plot!(x_test, _x_test*w_wls, color=:green, label=\"WLS\") # plot WLS solution\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Some Final Remarks\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Aside from the above example, we also recommend that you read through the [RxInfer Bayesian Linear Regression example](https://biaslab.github.io/RxInfer.jl/stable/examples/basic_examples/Bayesian%20Linear%20Regression%20Tutorial/). \n", "- In this lesson, we focussed on modelling the map from given inputs $x$ to uncertain outputs $y$, or more formally, on the distribution $p(y|x)$. What if you want to fit the best curve through a data set $\\{(x_1,y_1),\\dotsc,(x_N,y_N)\\}$ where both variables $x_n$ and $y_n$ are subject to errors? In other words, we must now also fit a model $p(x)$ for the inputs, leading to a generative model $p(y,x) = p(y|x) p(x)$. \n", "- While this is a very common problem that occurs throughout the sciences, a proper solution to this problem is still hardly covered in statistics textbooks. Edwin Jaynes discusses a fully Bayesian solution in his 1990 paper on [Straight Line Fitting - A Bayesian Solution](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Jaynes-1990-straight-line-fitting-a-Bayesian-solution.pdf). (Optional reading)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "##
OPTIONAL SLIDES
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Some Useful Matrix Calculus\n", "\n", "When doing derivatives with matrices, e.g. for maximum likelihood estimation, it will be helpful to be familiar with some matrix calculus. We shortly recapitulate used formulas here. \n", "\n", "- We define the **gradient** of a scalar function $f(A)$ w.r.t. an $n \\times k$ matrix $A$ as\n", "$$\n", "\\nabla_A f \\triangleq\n", " \\begin{bmatrix}\n", "\\frac{\\partial{f}}{\\partial a_{11}} & \\frac{\\partial{f}}{\\partial a_{12}} & \\cdots & \\frac{\\partial{f}}{\\partial a_{1k}}\\\\\n", "\\frac{\\partial{f}}{\\partial a_{21}} & \\frac{\\partial{f}}{\\partial a_{22}} & \\cdots & \\frac{\\partial{f}}{\\partial a_{2k}}\\\\\n", "\\vdots & \\vdots & \\cdots & \\vdots\\\\\n", "\\frac{\\partial{f}}{\\partial a_{n1}} & \\frac{\\partial{f}}{\\partial a_{n2}} & \\cdots & \\frac{\\partial{f}}{\\partial a_{nk}}\n", " \\end{bmatrix}\n", "$$\n", " \n", "\n", " \n", "- The following formulas are useful (see Bishop App.-C)\n", "$$\\begin{align*}\n", "|A^{-1}|&=|A|^{-1} \\tag{B-C.4} \\\\\n", "\\nabla_A \\log |A| &= (A^{T})^{-1} = (A^{-1})^T \\tag{B-C.28} \\\\\n", "\\mathrm{Tr}[ABC]&= \\mathrm{Tr}[CAB] = \\mathrm{Tr}[BCA] \\tag{B-C.9} \\\\\n", "\\nabla_A \\mathrm{Tr}[AB] &=\\nabla_A \\mathrm{Tr}[BA]= B^T \\tag{B-C.25} \\\\\n", "\\nabla_A \\mathrm{Tr}[ABA^T] &= A(B+B^T) \\tag{B-C.27}\\\\\n", " \\nabla_x x^TAx &= (A+A^T)x \\tag{from B-C.27}\\\\\n", "\\nabla_X a^TXb &= \\nabla_X \\mathrm{Tr}[ba^TX] = ab^T \\notag\n", "\\end{align*}$$" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Derivation Predictive Distribution\n", "\n", "- In the [derivation of the predictive distribution](#predictive-distribution), we replaced $\\mathcal{N}(w\\,|\\,m_N,S_N)\\,\\mathrm{d}w$ with $\\mathcal{N}(z\\,|\\,x_\\bullet^T m_N,x_\\bullet^T S_N x_\\bullet)\\,\\mathrm{d}z$. Here we discuss why that is allowed. \n", "\n", "- Since $z = x^T w$ (drop the bullet for notational simplicity), we have\n", "$$p(z) = \\mathcal{N}(z|m_z,\\Sigma_z)$$\n", "with\n", "$$\\begin{aligned} m_z &:= E[z] = E[x^T w] = x^T E[w] = x^T m_N \\\\ \\Sigma_z &:= E[(z-m_z)(z-m_z)^T] \\\\   &= E[(x^T w - x^T m_N)(x^T w - x^T m_N)^T] \\\\   &= x^T E[(w - m_N)(w - m_N)^T]x \\\\   &= x^T S_N x \\end{aligned}$$\n", "\n", "- Then we equate probability masses in both domains:\n", "$$ \\mathcal{N}(z|m_z,\\Sigma_z)\\mathrm{d}z = \\mathcal{N}(w|m_N,S_N)\\mathrm{d}w$$\n", "\n", "$$ \\Rightarrow \\mathcal{N}(z|x^T m_N,x^T S_N x)\\mathrm{d}z = \\mathcal{N}(w|m_N,S_N)\\mathrm{d}w $$" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "slideshow": { "slide_type": "skip" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "open(\"../../styles/aipstyle.html\") do f\n", " display(\"text/html\", read(f, String))\n", "end\n" ] } ], "metadata": { "@webio": { "lastCommId": null, "lastKernelId": null }, "anaconda-cloud": {}, "celltoolbar": "Raw Cell Format", "kernelspec": { "display_name": "Julia 1.9.3", "language": "julia", "name": "julia-1.9" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.9.3" } }, "nbformat": 4, "nbformat_minor": 4 }