{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Gaussian Process Regression and Active Subspaces\n", "\n", "**Author**: Spencer Lyon\n", "\n", "**Date**: March 29, 2017" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The purpose of this notebook is to explain the key numerical tools from [\"Machine learning for high-dimensional dynamic stochastic economies\" by Scheidegger and Bilionis (2017)](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2927400)) (henceforth, S&B).\n", "\n", "The main methodological contribution of this paper is the description of a novel function approximation technique that combines two tools from the Machine Learning literature: [Gaussian process regression](https://en.wikipedia.org/wiki/Gaussian_process) (GPR) and [active subspaces](http://activesubspaces.org/research/) (AS).\n", "\n", "I will describe the basic mathematical theory behind these tools (meaning I leave out some of the details), then replicate a few of the illustrative examples S&B cover in their paper.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Theory" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Gaussian process regression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "GPR is a tool from machine learning we can use to approximate functions. The notes in this section are derived from the treatment in [chapter 2](http://www.gaussianprocess.org/gpml/chapters/RW2.pdf) of [Williams and Rasmussen (2006)](http://www.gaussianprocess.org/gpml), with some ideas from S&B.\n", "\n", "GPR is a Bayesian regression method in function space. We will compute interpolated values as a function of the posterior mean, where the accuracy of the approximation is expressed as a function of the posterior (co)variance. As is typical with Bayesian methods, we will construct the posterior distribution by combining a prior and a likelihood. Below, I describe how to characterize the prior and likelihood, how to combine them into the posterior, as well as how to compute interpolated values." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Setup\n", " \n", "Let $f : \\mathbb{R}^D \\rightarrow \\mathbb{R}$ represent a function we want to approximate. In economic examples this might be a value or a policy function.\n", "\n", "Suppose that we have $N$ input-output observations of the form of the form $X := \\begin{bmatrix} x_1 & x_2 & \\cdots & x_N \\end{bmatrix}$ and $V := \\begin{bmatrix} v_1 & v_2 & \\cdots & v_N \\end{bmatrix}^T$, where $X$ is $D \\times N$, $V$ is $N \\times 1$, and $v_i$ is our candidate value for $f(x_i)$. I will sometimes refer to $(X, V)$ as the _training data_. Also, I will use notation $F$ to refer to the true value of $f$ evaluated at all the points in $X$: $F \\stackrel{\\Delta}{=} \\begin{bmatrix} f(x_1) & f(x_2) & \\cdots & f(x_N) \\end{bmatrix}^T$\n", "\n", "> NOTE: I use the language \"candidate value\" because in our applications $f$ is typically not known exactly (we are solving for it!) and we will only be able to provide a guess for what $f(x)$ should be at our input points $x \\in X$.\n", "\n", "*Definition*: A **Gaussian Process** is a collection of random variables, any finite number of which have a joint multivariate normal distribution.\n", "\n", "As the mean and covariance are sufficient statistics for a multivariate normal distribution, a gaussian process is fully specified by a mean and covariance function. We can write the mean and covariance functions of our function $f$ as\n", "\n", "$$m(x) = E[f(x)]$$\n", "$$k(x, x') = E[(f(x) - m(x))(f(x') - m(x'))]$$\n", "\n", "and we would write the gaussian process as\n", "\n", "$$\n", "f(x) \\sim \\mathcal{G}\\mathcal{P}(m(x), k(x,x')).\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prior\n", "\n", "In GPR, the formation of a prior happens when a mean and covariance function are chosen.\n", "\n", "S&B use the zero-mean function ($m(x) = 0 \\; \\forall x$) and choose the [squared Euclidean (or radial basis function) kernel](http://scikit-learn.org/stable/modules/gaussian_process.html#radial-basis-function-rbf-kernel) defined as:\n", "\n", "$$\n", "k(x,x') = s^2 \\exp \\left(-\\frac{1}{2} \\sum_{i=1}^D \\left(\\frac{x_i - x_i'}{l_i}\\right)^2 \\right)\n", "$$\n", "\n", "where $\\theta \\stackrel{\\Delta}{=} (s, \\{l_i\\}_{i=1}^{D})$ are the _hyperparameters_ of the prior distribution. \n", "\n", ">NOTE: The zero-mean and squared Euclidean kernel are also the reccomendations in Williams and Rasmussen (W&R). W&R prove that the use of the squared Euclidean kernel makes Gaussian process regression equivalent to Bayesian linear regression with an infinite number of basis functions -- so it seems like a good choice to me!\n", "\n", "If $\\hat{X} \\in \\mathbb{R}^{D \\times n}$ and $\\hat{X}' \\in \\mathbb{R}^{D \\times n'}$ are sets of $n$ and $n'$ input points, let $K(\\hat{X},\\hat{X}')$ be the $n \\times n'$ cross-covariance matrix. The $i,j$ element is $k(\\hat{x}_i, \\hat{x}_j')$. Also denote $m(X) = \\begin{bmatrix} m(x_1) & m(x_2) & \\cdots & m(x_N) \\end{bmatrix}^T$\n", "\n", "Given this setup, the prior (here a distribution over the samples of $f$) is represented as:\n", "\n", "$$\n", "F \\bigm\\vert X, \\theta \\sim N \\big(m(X), K(X,X) \\big)\n", "$$\n", "\n", "Notice a few things about the prior:\n", "\n", "\n", "1. The prior is a function of the training _inputs_ $X$, but not the proposed values $V$.\n", "2. A sample from the prior is a vector in $\\mathbb{R}^N$.\n", "3. To evaluate the prior we need values for the hyperparameters. These are usually solved for using maximum likelihood given the training data (I skip a detailed discussion of this step).\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Likelihood\n", "\n", "The likelihood is a distribution for the proposed funciton values, given the data and hyperparameters $V \\bigm\\vert X, \\theta$.\n", "\n", "To construct the likelihood, we need to make an assumption about the distribution of $V$. We will follow S&B (who, in turn, follow R&W) and assume\n", "\n", "$$\n", "v_i = f(x_i) + \\sigma_n \\epsilon\n", "$$\n", "\n", "where $\\epsilon \\stackrel{\\text{i.i.d.}}{\\sim} N(0, 1)$ is noise and $\\sigma_n$ is the standard deviation of the _noise_ in our estimates of $f$ in the samples $(X, V)$. Note that $\\sigma_n$ is another hypterparaemter we need to optimize over, so it is included as part of $\\theta$.\n", "\n", "Because we've assumed that $\\epsilon$ is i.i.d., the distribution of all the samples is given by \n", "\n", "$$\n", "V \\bigm\\vert F, \\theta \\sim N\\left(F, \\sigma_n^2 I_N\\right)\n", "$$\n", "\n", "where $I_N$ is the $N \\times N$ identity matrix. Finally, we write the likelihood of observing the training data $(X, V)$ as\n", "\n", "$$\n", "V \\bigm\\vert X, \\theta \\sim N \\bigm(m(X), K(X, X) + \\sigma_n^2 I_N \\bigm)\n", "$$\n", "\n", "> NOTE: this last step is still a little to fuzzy to me. This is how it is presented both in S&B and W&R, so for now I am trusting that it is correct." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Posterior\n", "\n", "The posterior distribution is given by Bayes' Law:\n", "\n", "$$\n", "\\text{posterior} \\propto \\text{prior} \\times \\text{likelihood}\n", "$$\n", "\n", "As shown above, the prior and likelihood are both Gaussian. A Gaussian prior and likelihood form a [conjugate pair](https://en.wikipedia.org/wiki/Conjugate_prior), which means that the posterior is known in closed form and in our case is also Gaussian. Armed with this information, we compute the posterior distribution at a point $x \\in \\mathbb{R}^D$ in closed form as\n", "\n", "$$\n", "f(x) \\bigm\\vert X, V, \\theta \\sim N \\left(\\tilde{m}(x), \\tilde{k}(x, x)\\right),\n", "$$\n", "\n", "where\n", "\n", "\n", "\n", "$$\\tilde{m}(x) \\stackrel{\\Delta}{=} m(x) + K(x, X)\\bigm(K(X,X) + \\sigma_n^2 I_N\\bigm)^{-1}(V - m(X))$$\n", "$$\\tilde{k}(x,x') \\stackrel{\\Delta}{=} k(x, x') - K(x, X)\\bigm(K(X,X) + \\sigma_n^2 I_n\\bigm)^{-1} K(X, x').$$\n", "\n", "$\\tilde{m}(x)$ and $\\tilde{k}(x, x)$ are known as the _predictive mean_ and _predictive variance_, respectively. Note that $\\tilde{m}(x) \\in \\mathbb{R}$ and $\\tilde{k}(x, x) \\in \\mathbb{R}$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Interpolation\n", "\n", "We can use the predictive mean to construct approximate of $f$ for any $x \\in \\mathbb{R}^D$. \n", "\n", "To see how, define\n", "\n", "$$\n", "\\alpha \\stackrel{\\Delta}{=} \\bigm(K(X,X) + \\sigma_n^2 I_n \\bigm)^{-1}(V - m(X)).\n", "$$\n", "\n", "and notice that the equation for $\\tilde{m}(x)$ can be written as the sum of $\\tilde{m}(x) = m(x) + K(x, X) \\cdot \\alpha$.\n", "\n", "With this insight, we can consider the elements of $K(x, X)$ as the evaluation of $N$ symmetric Gaussian basis functions centered at each observation input (centered at each $x \\in X$) while $\\alpha$ is the corresponding coefficient vector.\n", "\n", "\n", "\n", "We compute an approximated value of $f(x)$ as $\\hat{f}(x) \\stackrel{\\Delta}{=} m(x) + K(x, X) \\cdot \\alpha = m(x) + \\sum_{i=1}^N k(x, x_i) \\alpha_i$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Comments\n", "\n", "A few observations:\n", "\n", "- Given our choice of the squared euclidean kernel and zero mean function, approximation following this scheme is interpolating (meaning $\\hat{f}(x_i) = v_i$).\n", "- The cost of evaluating $\\hat{f}(x)$ increasing in both $D$ and $N$ as computing $\\tilde{m}(x)$ requires that we compute $K(x, X)$, which is the squared Euclidean distance between $x$ and all $N$ points in $X$. The cost of each of these distance calculations is increasing in $D$.\n", "- There is scope for parallelization when computing $\\hat{f}(x)$ because each term of $\\sum_{i=1}^N k(x_i, x) \\alpha_i$ can be computed simultaneously.\n", "- We can use the predictive variance as a measure of the accuracy of the approximation. Specifically, the distribution of $\\hat{f}(x)$ is $N \\left(f(x), \\tilde{k}(x, x) \\right)$. So, if we wanted to plot $\\hat{f}(x)$ with 95% confidence bands we could plot $\\hat{f}(x)$ as well as $\\hat{f}(x) \\pm 1.96\\sqrt{\\tilde{k}(x, x)}$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Active Subspaces" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These notes are my summary of the exposition in S&B.\n", "\n", "Active subspaces are a set of tools used for dimensionality reduction. The core concept is to suppose that the function $f$ we want to approximate is itself well approximated by\n", "\n", "$$\n", "f(x) \\approx h(W^T x)\n", "$$\n", "\n", "where the matrix $W \\in \\mathbb{R}^{D \\times d}$ projects from the high dimensional input space $\\mathbb{R}^D$ to the lower dimensional _active subspace_ $\\mathbb{R}^d$ and $h: \\mathbb{R}^d \\rightarrow \\mathbb{R}$ is called the _link_ function. \n", "\n", "> NOTE: All matrices $W$ that span the same subset of $\\mathbb{R}^d$ work identically. S&B find it useful to focus on matrices with orthogonal columns. This allows us to interpret the columns of $W$ as the directions in $\\mathbb{R}^D$ as directions of steepest change for $f$. As these directions encoded in the gradient of f, $\\nabla f$, we can use what we know about $\\nabla f$ to find an appropriate matrix $W$.\n", "\n", "This leaves us with three main problems:\n", "\n", "1. How to determine $d$\n", "2. How to find the matrix $W$\n", "3. What to choose as the function $h$" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "### Determining $d$ and $W$\n", "\n", "The algorithms for finding $d$ and $W$ are related, so we talk about them together.\n", "\n", "Suppose that in addition to $X$ and $V$ we also have the gradient of $f$ at each point in $X$. That is we have a matrix $G \\in \\mathbb{R}^{D \\times N}$ where\n", "\n", "$$ \n", "G = [g_1 \\; g_2 \\; \\cdots \\; g_N]\n", "$$\n", "\n", "and $g_i = \\nabla f(x_i) = \\begin{bmatrix} \\frac{\\partial f}{\\partial x_{i1}} & \\frac{\\partial f}{\\partial x_{i2}} & \\cdots & \\frac{\\partial f}{\\partial x_{iD}} \\end{bmatrix}^T$.\n", "\n", "Now let $\\rho(x)$ be some PDF on $\\mathbb{R}^D$, such as the PDF of a uniform distribution. Then define \n", "\n", "$$ \n", "C = \\int (\\nabla f(x)) (\\nabla f(x))' \\rho(x) dx\n", "$$\n", "\n", "$C$ is symmetric and positive definite and can be written as $C = V \\Lambda V^T$, where $\\Lambda$ is a diagonal matrix whose elements are the eigenvalues of $C$ (meaning $\\Lambda = \\text{diag}(\\lambda_1, \\lambda_2, \\cdots, \\lambda_D)$ and $\\lambda_i$ is the $i$th eigenvalue) and the columns of $V$ are the associated eigenvectors.\n", "\n", "Now consider reordering the columns of $\\Lambda$ according to the size of the eigenvalues so that the largest eigenvalues appear in the first columns. Re-order the columns of $V$ appropriately and write them as\n", "\n", "\\begin{align*}\n", "\\Lambda &= \\begin{bmatrix} \\Lambda_{1} & 0 \\\\ 0 & \\Lambda_2 \\end{bmatrix} \\\\\n", "V &= \\begin{bmatrix}V_1 & V_2 \\end{bmatrix}\n", "\\end{align*}\n", "\n", "where \n", "\n", "\\begin{align*}\n", "\\Lambda_1 &= \\text{diag}(\\lambda_1, \\lambda_2, \\cdots, \\lambda_d) \\\\\n", "\\Lambda_2 &= \\text{diag}(\\lambda_{d+1}, \\lambda_{d+2}, \\cdots, \\lambda_D) \\\\\n", "V_1 &= \\begin{bmatrix} v_1 & v_2 & \\cdots & v_d]\\end{bmatrix} \\\\\n", "V_2 &= \\begin{bmatrix}v_{d+1} & v_{d+2} & \\cdots & v_D \\end{bmatrix}\n", "\\end{align*}\n", "\n", "We can then choose $d$ by examining the elements of $\\Lambda$ and looking for when they approach zero (this will be obvious in examples).\n", "\n", "The matrix $W$ is then $V_1$.\n", "\n", "Now, in practice we can't compute $C$ directly. Instead we will make the _assumption_ that $\\rho(X)$ is uniform and compute our approximation of $C$ as\n", "\n", "$$\n", "C_N = \\frac{1}{N} \\sum_{i=1}^{N} g_i g_i^T\n", "$$\n", "\n", "$C_N$ will be symmetric and positive definite, so we can use either the eigen decomposition or SVD to compute $\\Lambda$ and $V$. In practice I've seen that the SVD is a better choice both because it is faster and also because the eigenvalues will be ordered.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Choosing $h$\n", "\n", "Choosing $h$ is not actually an issue when we want to apply this technique. What we do in practice is use the gradient information to construct $W$ and pick $d$. Then we transform all the input arguments $Y = W^T X$ and the use GPR to approximate $h$ from the transformed training data $(Y, V)$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Replications" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this section I will replicate some key figures from the paper." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from scipy import linalg\n", "from sklearn.gaussian_process import GaussianProcessRegressor\n", "from sklearn.gaussian_process.kernels import RBF\n", "\n", "%matplotlib inline\n", "%config InlineBackend.figure_format = 'svg'\n", "%config InlineBackend.figure_formats = ['svg', 'pdf']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Figure 1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I start by replicating part of the top right panel of figure 1. This demonstrates that the average and maximum error in a 2d Gaussian process regression is decreasing in the number of training points.\n", "\n", "Note that I use the formula S&B call the average error, which is given by:\n", "\n", "$$\n", "e = \\sqrt{\\sum_{i=1}^N \\left(f(x_i) - \\tilde{m}(x_i) \\right)^2 / f(x_i)^2}.\n", "$$\n", "\n", "> NOTE: it is not obvious to me why this is a measure of average error, but this is the equation and nomenclature from S&B, so I use it in the replication.\n", "\n", "We will use the function" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def f1(x):\n", " return np.abs(0.25 - x[0]**2 - x[1]**2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and will use a randomly generated set of $N$ points on $\\Delta^2$ (a two dimensional simplex)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def randDelta2(N):\n", " x = np.random.rand(N)\n", " return np.column_stack([x, 1-x])" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def figure1():\n", " np.random.seed(39)\n", " num_Ns = 20\n", " ave_errs = np.inf * np.ones(num_Ns)\n", " max_errs = np.inf * np.ones(num_Ns)\n", " Nvals = np.logspace(1, 3, num_Ns)\n", " \n", " # construct test points\n", " X_test = randDelta2(1000)\n", " f1_test = f1(X_test.T)\n", " \n", " for i, N in enumerate(Nvals):\n", " # training points\n", " X = randDelta2(int(N))\n", " V = f1(X.T)\n", " \n", " # fit GP. note that the call to `gp.fit` below does\n", " # maximum likelihood optimization of priors for us\n", " gp = GaussianProcessRegressor(RBF())\n", " gp.fit(X, V)\n", " \n", " # compute error\n", " m_tilde = gp.predict(X_test)\n", " ave_errs[i] = np.sqrt(np.sum((f1_test - m_tilde)**2 / (f1_test**2)))\n", " max_errs[i] = np.max(np.abs(f1_test - m_tilde))\n", " \n", " fig, ax = plt.subplots(figsize=(8, 5.5))\n", " ax.loglog(Nvals, ave_errs, \".-\", ms=15, label=\"Average error\")\n", " ax.loglog(Nvals, max_errs, \".-\", ms=15, label=\"Maximum error\")\n", " ax.legend()\n", " \n", " return np.column_stack([ave_errs, max_errs]), fig" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that when we run the function above some scary looking red warnings will appear. This OK. What is happening is that when the GPR routine does maximum likelihood to it wanders into regions of the state space where the log-likelihood function that is being used is not well defined. When this happens the optimizer simply chooses another search direction." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 4, 'warnflag': 2, 'grad': array([ 0.00091152]), 'funcalls': 59}\n", " \" state: %s\" % convergence_dict)\n", "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 2, 'warnflag': 2, 'grad': array([ 0.00327253]), 'funcalls': 45}\n", " \" state: %s\" % convergence_dict)\n", "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 0, 'warnflag': 2, 'grad': array([-7.53496059]), 'funcalls': 21}\n", " \" state: %s\" % convergence_dict)\n", "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 5, 'warnflag': 2, 'grad': array([-0.00061803]), 'funcalls': 51}\n", " \" state: %s\" % convergence_dict)\n", "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 5, 'warnflag': 2, 'grad': array([ 0.00377541]), 'funcalls': 56}\n", " \" state: %s\" % convergence_dict)\n" ] }, { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+\nCmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9TaGFkaW5nIDYgMCBSIC9FeHRHU3RhdGUg\nNCAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1hP\nYmplY3QgNyAwIFIKL1BhdHRlcm4gNSAwIFIgPj4KZW5kb2JqCjEwIDAgb2JqCjw8IC9UeXBlIC9Q\nYWdlIC9Bbm5vdHMgWyBdCi9Hcm91cCA8PCAvVHlwZSAvR3JvdXAgL1MgL1RyYW5zcGFyZW5jeSAv\nQ1MgL0RldmljZVJHQiA+PiAvUmVzb3VyY2VzIDggMCBSCi9NZWRpYUJveCBbIDAgMCA0OTUuMyAz\nMzQuODggXSAvQ29udGVudHMgOSAwIFIgL1BhcmVudCAyIDAgUiA+PgplbmRvYmoKOSAwIG9iago8\nPCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDExIDAgUiA+PgpzdHJlYW0KeJy9msuOFUcS\nhvfnKWrpWZDk/bIE2YNkyQvbrZnFaBao3WZANBaNsOfx54u6ZEZ1Q1dZHugWcOo/f0VGRMYtq3DT\nm8vTZ2569WGy0xv+/DH9a/o3//4yuenF9PTbm99fX9/89OL5dP3hYsFvL7ElE/j0dv0UQjS1cmnH\nx/9cLu8uSIX2AkGvLqEaP/nEXwio0eTl4u16EXw0Tm6cef0KMb+inl/Ue4UkVDRVKSny+eaSqonN\n8rstogBnCrIuzzdJbvrj8h7t7PTEchHbypzZzhpfqvNpur69PL8Sm43N0WaQq18uT//uoExXv16+\ncX+brt5csgnZBxtzLQmJUL6x6xcxVWsXWcFUP39YRRQl4bury4/dSDHu4rMzcTNkvTgwwicPqz3U\nvZWcvoTu/jO6xxxNEE+6vt0KOdqKlBbqQv+K5oRPmJNRjnisPreaN2sUEOYAxhgiyBMgrXaTB9BJ\nzjWTis+h54FGBm12QXCxR7JGBq2iX4ohpU5TSKd5a01w1ZfSw0khg+bwXY21dN00Mmi+mZZrqyM4\nFTJokW0O2apFFTJohKxzKdRuqUY6LXgiu7riQ98ChQwaCe+d9101BQxSDSblEKPazoGM/bR89Cg8\nNlQhg8b++RZa7pppZNCCN7VUW/pWaWTQIoUwJqcWVcigEQ3WxzhYA+ikJRXnWlqzySGE4sTYSPoN\n4JFELHKnN7n4CNXnv1oMnTfBU9a9k28+nYJpS8H3sootk7Sb5RO3V08KLOnbFimoRHt68/IfH39+\n+e7Dk9vX7z5+mL79jQy+V5JmP7hET0k2pbo5QiEHnoBFP2oZFSwt4iv4In5RX3jqcM6lxLj5QiEH\nvvCuSG2rVuyqf7Uyn/FFOOULBo9V/p/0RW3sbWyhJ4hCjnzB7EIVai3EZPPX8IX/Er7Imy8Ck1C0\ntQZxRd4Ba/kES84E0iC3jTQARWoGF6TqOqkDg5Rxha259OUGMEjFSVcruW6kAShSNq5EmlondWCQ\n6I+MFb4L6teKQjOIoaXYOR0YJGezacwTfbEBKJJjTojFp9xZA1E0Txyh5jBPIYoWJBxSTd3pCtE0\nIrc4pb9CFC0Gk5u3rjteIZpWDC075mHCQBSN/bfZudj3WiGKVqKJJWmHDETRquQHnW/oNhBFa2Re\na41hZ6MNZNCwyIQoybbRFKJp9MTalAkKUTTnKH3Nd9J2rSnJOB9zCoPUEU2rTH41jIBViKIxNrXm\nlTMUomiJWKiOebPTBqJomQJDMCgrB6JpxGm2PoxFB6Jo7B5eCiOfFKJp1ciIW8cODETRZNiIVN3O\n6oAmZTI2JadYHRk0DhAm2abcoRBFo+WFJl7qtIHMtPfTOLUyIMoprFXDUH93M/1zejf56fvJmcRp\nlPGAfc41e6nphgRZfgqTWLGpVh8Si/90/0CtTqmBg+ec3qVRJmggDrUrTdkWK353VoKtZiftOxk6\ntZXAY8BlfiUl47JNLVgnVkpGN7yMLd7OeZadZIbLWFOZKelzthhbMkEguIyFyORm2o+cN2QKf8tx\nodJdfGVc9DYxOcrAITiDf8jM4JKgyG8Ud5HjGZUKzBLBUQG7ZW988azVIpOkaJxtdHO/YFg2MVDu\n6ozHmmOWYOS0QzYXh9M4ECA+zNYG9onjUKONUF5MwAvz1odUDEFQiVBHH6Q/tDkKA7FWZWpHG/xb\nqL5F+JFDEK02BdQhNqpNMUni0XvZBBcq6uAdDFmaUYyeCk8GtLnY55psmR9ejKOtwzvsjp036/Iz\nY8D3nw+LaRcWd/JgYx9El8fY6xgiU0iPHx1D17fT0x+stPmNJoGNANl2Ct8Taojnt1jr6jGbTGwu\nZ6lNx2zCjn2gDhdHMzmUjf8ifcDKkHGsuASOlQip4YTenMXYcE6woZygU38Dl17cn4+Fc+mI5ix3\nlGM6+5JdcOKaM16RiYoAhJ9PCXeuFFsZRE9ozqzCvEreWBL2RKgwt2Tyw4V4hs2sEWsqtNgz7Dmm\nY242nbGyZTySWyonQpxyw7BFnOR2Sm8bErGFC094kNOD59zXolTXwWamns63DCcFoVUrBTVJiuPj\nRvnNttInLp/tE57hKhZOX2nXKwQOtcjxdNcrHIM3lXwZP1SzcKjlKCf3mwWzACMBdXbfK1yan3+U\nuSfoXuGJbJYt+1bh0JfazTx2r1V4qd+sH++1Cg+f6hfrvldwQmcZKa+7VoFeMZDaYd8p4jwB0rj2\nnQLPtBzrvT4Rk2m0lZL2bQIZobKk33cJVMVBdT4SqiYhO8zcPM9PqkcQA6Xt20PE1ujT7JC1OzwW\nAWtLeJzy2T4wYiRKX5tD1D0a0ER+LjmIfofsJiNoCYlxNR6yxRWMhIwxKZzQJGVDDnJOpOa4Qzrz\nNyN65Jzs2rEqT4KMlFTVmhF/rDmhEolFdvqEUwKqlETEOVtPKE6klGRpTFuTfEx2CZRV5gtGg3qs\nipNilh2ZjfbHwkngHMg28rkcCy+GokCQW4n+Q9kM55LtOMWd2B6yilziaJDKsezM5mRKAJX4eCuf\nUEbwd6UguRMu4ZBbJKk5mR6zZSslry1nSkVemsH6iEWqvn3w6mpf4vULsv1LMMqFemN2e/+NGV9/\n9uXa9t1K/tS9T5+F5e2aFKRtEK3rWzvORRxL50ecctSiSHhpO7EUWaMjVDoqKJWXHwVH2pPQSCTr\nxfMbUFfC9WVBOIMu+myXLOpmYUpucPJYblrWXi+ulYIrRDXPSCvzgisWcu6kLlqDZROWu7W126bR\nbopaZBj8KW9dy0vM/hBx7v9EwjID9JeY584JD8aCUGw3gayvy+6E9hD94kcS4qyvyskprJESKpZa\nOWUqmCZOPpK+C1eKIDOAvIlxMtvJsTcGSev57QxjDGPDfIaHV+f7o/FpMVCe4TmZ7zW4WS0CBioj\n4aKJkuoC1XJRpa8PVoIUw4vWlDmDqXm+SRkFuu79sL9jCCCgqX6zgoNKUEv39HupBDYNO+/WB3Np\ncY/StGRTHhilQGW/QoerlFTl1b6+8r/SVO2UMkpt6sP938e+Xf87gLyuf341jQfjoVV1a6rrk/Nv\nnv1+c/fy1c10c3f32508+J6+u9ol0olB+n6V7RnjG7Mbg2/Zp4yC/4+DmsoODt4m523dEUkKV1En\nQ69NDKptxKdg3FHWAO+x7BuxlFPTCeIbozKzfg47sNuoM4R53cg5nPPsTmqVFxZVr9+MZaBb11e6\nyjP+xvC5yxCGaLfZNUKkg/scUeQRelrwFqR7JUZAD3V75Gu7FKh9oODhLy20e3asrzdB6zo2bNi1\n29xPRMKfThS5ty3qS6b88PK/r28/3u4z5cfL/wAWqi0uCmVuZHN0cmVhbQplbmRvYmoKMTEgMCBv\nYmoKMjQ5MwplbmRvYmoKMTggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA5\nMCA+PgpzdHJlYW0KeJxNjUESwCAIA++8Ik9QRND/dHrS/1+r1A69wE4CiRZFgvQ1aksw7rgyFWtQ\nKZiUl8BVMFwL2u6iyv4ySUydhtN7twODsvFxg9JJ+/ZxegCr/XoG3Q/SHCJYCmVuZHN0cmVhbQpl\nbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMTAgPj4Kc3Ry\nZWFtCnicNVDLDUMxCLtnChaoFAKBZJ5WvXX/a23QO2ER/0JYyJQIeanJzinpSz46TA+2Lr+xIgut\ndSXsypognivvoZmysdHY4mBwGiZegBY3YOhpjRo1dOGCpi6VQoHFJfCZfHV76L5PGXhqGXJ2BBFD\nyWAJaroWTVi0PJ+QTgHi/37D7i3koZLzyp4b+Ruc7fA7s27hJ2p2ItFyFTLUszTHGAgTRR48eUWm\ncOKz1nfVNBLUZgtOlgGuTj+MDgBgIl5ZgOyuRDlL0o6ln2+8x/cPQABTtAplbmRzdHJlYW0KZW5k\nb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA0ID4+CnN0cmVh\nbQp4nD2SO5LDMAxDe52CF8iM+JPk82Qnlff+7T4yyVaASYkAKC91mbKmPCBpJgn/0eHhYjvld9ie\nzczAtUQvE8spz6ErxNxF+bKZjbqyOsWqwzCdW/SonIuGTZOa5ypLGbcLnsO1ieeWfcQPNzSoB3WN\nS8IN3dVoWQrNcHX/O71H2Xc1PBebVOrUF48XURXm+SFPoofpSuJ8PCghXHswRhYS5FPRQI6zXK3y\nXkL2DrcassJBaknnsyc82HV6Ty5uF80QD2S5VPhOUezt0DO+7EoJPRK24VjufTuasekamzjsfu9G\n1sqMrmghfshXJ+slYNxTJkUSZE62WG6L1Z7uoSimc4ZzGSDq2YqGUuZiV6t/DDtvLC/ZLMiUzAsy\nRqdNnjh4yH6NmvR5led4/QFs83M7CmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRl\nciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMzggPj4Kc3RyZWFtCnicRVJLcsUwCNvnFFwgM+Zn4/O8\nTlfp/beVcDrdPPQMCAkyPWVIptw2lmSE5BzypVdkiNWQn0aORMQQ3ymhwK7yubyWxFzIbolK8aEd\nP5elNzLNrtCqt0enNotGNSsj5yBDhHpW6MzuUdtkw+t2Iek6UxaHcCz/QwWylHXKKZQEbUHf2CPo\nbxY8EdwGs+Zys7lMbvW/7lsLntc6W7FtB0AJlnPeYAYAxMMJ2gDE3NreFikoH1W6iknCrfJcJztQ\nttCqdLw3gBkHGDlgw5KtDtdobwDDPg/0okbF9hWgqCwg/s7ZZsHeMclIsCfmBk49cTrFkXBJOMYC\nQIqt4hS68R3Y4i8Xroia8Al1OmVNvMKe2uLHQpMI71JxAvAiG25dHUW1bE/nCbQ/KpIzYqQexNEJ\nkdSSzhEUlwb10Br7uIkZr43E5p6+3T/COZ/r+xcWuIPgCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBv\nYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA4OCA+PgpzdHJlYW0KeJw1jLsNwDAI\nRHumuBH4OID3iVKR/dsQWy64e9IT5znAyD4PR+jELWRD4aZ4STmhPlE0Wm86tkhzv7xkMxcV/TPc\nfYXt5oBkNEnkIdHlcuVZKXo+l9obfgplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0\nZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ3ID4+CnN0cmVhbQp4nE1Ru21EMQzr3xRc4ADra3me\nC1Jd9m9DyQiQwiChLymnJRb2xksM4QdbD77kkVVDfx4/MewzLD3J5NQ/5rnJVBS+FaqbmFAXYuH9\naAS8FnQvIivKB9+PZQxzzvfgoxCXYCY0YKxvSSYX1bwzZMKJoY7DQZtUGHdNFCyuFc0zyO1WN7I6\nsyBseCUT4sYARATZF5DNYKOMsZWQxXIeqAqSBVpg1+kbUYuCK5TWCXSi1sS6zOCr5/Z2N0Mv8uCo\nunh9DOtLsMLopXssfK5CH8z0TDt3SSO98KYTEWYPBVKZnZGVOj1ifbdA/59lK/j7yc/z/QsVKFwq\nCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0\naCA2OCA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWc\ngohbQjRBlIJYEKVmJmYQSTgDIpcGAMm0FeUKZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAv\nRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE3MSA+PgpzdHJlYW0KeJxNkE0OQiEQg/ecohcw\nofMDj/NoXOn9t3bw+eKC9EshQ6fDAx1H4kZHhs7oeLDJMQ68CzImXo3zn4zrJI4J6hVtwbq0O+7N\nLDEnLBMjYGuU3JtHFPjhmAtBguzywxcYRKRrmG81n3WTfn67013UpXX30yMKnMiOUAwbcAXY0z0O\n3BLO75omv1QpGZs4lA9UF5Gy2QmFqKVil1NVaIziVj3vi17t+QHB9jv7CmVuZHN0cmVhbQplbmRv\nYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDUgPj4Kc3RyZWFt\nCnicRVC7jUMxDOs9BRcIYP0se553SJXbvz1KRnCFIVo/kloSmIjASwyxlG/iR0ZBPQu/F4XiM8TP\nF4VBzoSkQJz1GRCZeIbaRm7odnDOvMMzjDkCF8VacKbTmfZc2OScBycQzm2U8YxCuklUFXFUn3FM\n8aqyz43XgaW1bLPTkewhjYRLSSUml35TKv+0KVsq6NpFE7BI5IGTTTThLD9DkmLMoJRR9zC1jvRx\nspFHddDJ2Zw5LZnZ7qftTHwPWCaZUeUpnecyPiep81xOfe6zHdHkoqVV+5z93pGW8iK126HV6Vcl\nUZmN1aeQuDz/jJ/x/gOOoFk+CmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAv\nRmxhdGVEZWNvZGUgL0xlbmd0aCAxNTcgPj4Kc3RyZWFtCnicRZC5EUMxCERzVUEJErAI6rHH0Xf/\nqRf5SrRvAC2HryVTqh8nIqbc12j0MHkOn00lVizYJraTGnIbFkFKMZh4TjGro7ehmYfU67ioqrh1\nZpXTacvKxX/zaFczkz3CNeon8E3o+J88tKnoW6CvC5R9QLU4nUlQMX2vYoGjnHZ/IpwY4D4ZR5kp\nI3Fibgrs9xkAZr5XuMbjBd0BN3kKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvVHlwZSAv\nWE9iamVjdCAvQkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0xlbmd0aCAzNyAvU3VidHlw\nZSAvRm9ybQovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjQwUzA2NVXI5TI3NgKz\ncsAsI3MjIAski2BBZNMAAV8KCgplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9GaWx0ZXIg\nL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTcgPj4Kc3RyZWFtCnicMza0UDCAwxRDLgAalALsCmVuZHN0\ncmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNTUg\nPj4Kc3RyZWFtCnicRZFLkgMgCET3noIjgPzkPJmaVXL/7TSYTDZ2l6j9hEojphIs5xR5MP3I8s1k\ntum1HKudjQKKIhTM5Cr0WIHVnSnizLVEtfWxMnLc6R2D4g3nrpxUsrhRxjqqOhU4pufK+qru/Lgs\nyr4jhzIFbNY5DjZw5bZhjBOjzVZ3h/tEkKeTqaPidpBs+IOTxr7K1RW4Tjb76iUYB4J+oQlM8k2g\ndYZA4+YpenIJ9vFxu/NAsLe8CaRsCOTIEIwOQbtOrn9x6/ze/zrDnefaDFeOd/E7TGu74y8xyYq5\ngEXuFNTzPRet6wwd78mZY3LTfUPnXLDL3UGmz/wf6/cPUIpmiAplbmRzdHJlYW0KZW5kb2JqCjMx\nIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjE0ID4+CnN0cmVhbQp4nD1Q\nuxFDMQjrPQUL5M587TfPy6XL/m0knKRCNkISlJpMyZSHOsqSrClPHT5LYoe8h+VuZDYlKkUvk7Al\n99AK8X2J5hT33dWWs0M0l2g5fgszKqobHdNLNppwKhO6oNzDM/oNbXQDVocesVsg0KRg17YgcscP\nGAzBmROLIgxKTQb/rXL3UtzvPRxvooiUdPCu+eX0y88tvE49jkS6vfmKa3GmOgpEcEZq8op0YcWy\nyEOk1QQ1PQNrtQCu3nr5N2hHdBmA7BOJ4zSlHEP/1rjH6wOHilL0CmVuZHN0cmVhbQplbmRvYmoK\nMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA4MCA+PgpzdHJlYW0KeJxF\njLsNwDAIRHumYAR+JmafKJWzfxsgStxwT7p7uDoSMlPeYYaHBJ4MLIZT8QaZo2A1uEZSjZ3so7Bu\nX3WB5npTq/X3BypPdnZxPc3LGfQKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVy\nIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDg3ID4+CnN0cmVhbQp4nDVNuRHAMAjrmYIRzKPY7JNL5ezf\nBuy4QTp9IJQba+QBguGdbyH4pi8ZhHUITyq7JTpsoYazCpKJ4Vc2eFWuiva1konsbKYx2KBl+tHO\nt0nPB6XeG5gKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29k\nZSAvTGVuZ3RoIDI0OCA+PgpzdHJlYW0KeJwtUTmSA0EIy+cVekJz0++xy5H3/+kKygGDhkMgOi1x\nUMZPEJYr3vLIVbTh75kYwXfBod/KdRsWORAVSNIYVE2oXbwevQd2HGYC86Q1LIMZ6wM/Ywo3enF4\nTMbZ7XUZNQR712tPZlAyKxdxycQFU3XYyJnDT6aMC+1czw3IuRHWZRikm5XGjIQjTSFSSKHqJqkz\nQZAEo6tRo40cxX7pyyOdYVUjagz7XEvb13MTzho0OxarPDmlR1ecy8nFCysH/bzNwEVUGqs8EBJw\nv9tD/Zzs5Dfe0rmzxfT4XnOyvDAVWPHmtRuQTbX4Ny/i+D3j6/n8A6ilWxYKZW5kc3RyZWFtCmVu\nZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVh\nbQp4nDWMsRHAMAgDe6bQCDZYYO+TS0X2b0N8TgMvHQ+XosFaDbqCI3B1qfzRI125KUWXY86C4XGq\nX0gxRj2oI+Pex0+5X3AWEn0KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9G\nbGF0ZURlY29kZSAvTGVuZ3RoIDMzOCA+PgpzdHJlYW0KeJw1Ujmu3UAM630KXSCAds2c5wWpfu7f\nhpRfCkO0VoqajhaVafllIVUtky6/7UltiRvy98kKiROSVyXapQyRUPk8hVS/Z8u8vtacESBLlQqT\nk5LHJQv+DJfeLhznY2s/jyN3PXpgVYyEEgHLFBOja1k6u8Oajfw8pgE/4hFyrli3HGMVSA26cdoV\n70PzecgaIGaYlooKXVaJFn5B8aBHrX33WFRYINHtHElwjI1QkYB2gdpIDDmzFruoL/pZlJgJdO2L\nIu6iwBJJzJxiXTr6Dz50LKi/NuPLr45K+kgra0zad6NJacwik66XRW83b309uEDzLsp/Xs0gQVPW\nKGl80KqdYyiaGWWFdxyaDDTHHIfMEzyHMxKU9H0ofl9LJrookT8ODaF/Xx6jjJwGbwFz0Z+2igMX\n8dlhrxxghdLFmuR9QCoTemD6/9f4ef78Axy2gFQKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8\nPCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDg3ID4+CnN0cmVhbQp4nD2OuxHAMAhDe6Zg\nBPMJhn1yqZz924A/adBDOk64CTYMz9Htws6BN0HuRS+wLm+ACiGLJBn59s7FolaZxa+mReqENEmc\nN1HoTotK5wPj9A54Prp1HXoKZW5kc3RyZWFtCmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9u\ndCAvRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXQovRm9udE1h\ndHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvV2lkdGhzIDE0IDAgUiAvQ2hhclByb2NzIDE3\nIDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBbIDMyIC9zcGFj\nZSA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIDY1IC9BIDc3IC9NIDk3IC9h\nIDEwMSAvZQoxMDMgL2cgMTA1IC9pIDEwOSAvbSAxMTEgL28gMTE0IC9yIDExNyAvdSAvdiAxMjAg\nL3ggXQo+PgovRm9udERlc2NyaXB0b3IgMTUgMCBSIC9CYXNlRm9udCAvRGVqYVZ1U2FucyAvU3Vi\ndHlwZSAvVHlwZTMKL05hbWUgL0RlamFWdVNhbnMgL0xhc3RDaGFyIDI1NSA+PgplbmRvYmoKMTUg\nMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9GbGFncyAzMiAvRm9udE5hbWUgL0RlamFW\ndVNhbnMgL0l0YWxpY0FuZ2xlIDAKL1N0ZW1WIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0\nIDEyMzMgXSAvTWF4V2lkdGggMTM0MiAvQ2FwSGVpZ2h0IDAKL0FzY2VudCA5MjkgL1hIZWlnaHQg\nMCAvRGVzY2VudCAtMjM2ID4+CmVuZG9iagoxNCAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAg\nNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2\nMDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQw\nMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3\nIDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4Mzgg\nODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYg\nNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2\nODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYz\nNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTky\nIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTgg\nMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMx\nOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEg\nMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEg\nMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIg\nOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2\nMzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4\nNyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgy\nIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIg\nNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxNyAw\nIG9iago8PCAvZm91ciAxOCAwIFIgL3plcm8gMTkgMCBSIC9hIDIwIDAgUiAvZyAyMSAwIFIgL0Eg\nMjIgMCBSIC9maXZlIDIzIDAgUgovaSAyNCAwIFIgL2UgMjYgMCBSIC9yIDI3IDAgUiAvb25lIDMy\nIDAgUiAvc3BhY2UgMjkgMCBSIC91IDI1IDAgUiAvbyAzMSAwIFIKL20gMzAgMCBSIC94IDMzIDAg\nUiAvdHdvIDM0IDAgUiAvdiAzNSAwIFIgL3RocmVlIDM2IDAgUiAvTSAzNyAwIFIgPj4KZW5kb2Jq\nCjMgMCBvYmoKPDwgL0YxIDE2IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTIgPDwgL1R5cGUg\nL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PgovQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAv\nY2EgMSA+PgovQTMgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMC44IC9jYSAwLjggPj4gPj4KZW5k\nb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwg\nL00xIDEzIDAgUiAvTTAgMTIgMCBSIC9EZWphVnVTYW5zLW1pbnVzIDI4IDAgUiA+PgplbmRvYmoK\nMTIgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL0JCb3ggWyAtNC4yNSAtNC4yNSA0LjI1IDQuMjUg\nXSAvTGVuZ3RoIDEzNQovU3VidHlwZSAvRm9ybSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJl\nYW0KeJxtkDEOgCAMRfeeggt8AkgRVkev4WJMvP+qEiwKLNCW/kd/rTrIqJXuA5OeWZ1kdEqerSsF\nq5OP3vGTTuzjHJTTgW24K2+wkbxB+rMaQsupya1PUMtFKLoXWuEfZGluqfnaqPkNvRA9Hf0I+M8I\nmR2NKQx8Y7AeDLaIjo9iYyda6AKS9k3fCmVuZHN0cmVhbQplbmRvYmoKMTMgMCBvYmoKPDwgL1R5\ncGUgL1hPYmplY3QgL0JCb3ggWyAtNC4yNSAtNC4yNSA0LjI1IDQuMjUgXSAvTGVuZ3RoIDEzNQov\nU3VidHlwZSAvRm9ybSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxtkDEOgCAMRfee\nggt8AkgRVkev4WJMvP+qEiwKLNCW/kd/rTrIqJXuA5OeWZ1kdEqerSsFq5OP3vGTTuzjHJTTgW24\nK2+wkbxB+rMaQsupya1PUMtFKLoXWuEfZGluqfnaqPkNvRA9Hf0I+M8ImR2NKQx8Y7AeDLaIjo9i\nYyda6AKS9k3fCmVuZHN0cmVhbQplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0NvdW50\nIDEgL0tpZHMgWyAxMCAwIFIgXSA+PgplbmRvYmoKMzggMCBvYmoKPDwgL0NyZWF0b3IgKG1hdHBs\nb3RsaWIgMi4wLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChtYXRwbG90bGli\nIHBkZiBiYWNrZW5kKSAvQ3JlYXRpb25EYXRlIChEOjIwMTcwMzMwMTAzOTI1KzExJzAwJykKPj4K\nZW5kb2JqCnhyZWYKMCAzOQowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAK\nMDAwMDAxMDU3MiAwMDAwMCBuIAowMDAwMDA5NzQ4IDAwMDAwIG4gCjAwMDAwMDk3ODAgMDAwMDAg\nbiAKMDAwMDAwOTkyMiAwMDAwMCBuIAowMDAwMDA5OTQzIDAwMDAwIG4gCjAwMDAwMDk5NjQgMDAw\nMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzkwIDAwMDAwIG4gCjAwMDAwMDAyMDgg\nMDAwMDAgbiAKMDAwMDAwMjk1OCAwMDAwMCBuIAowMDAwMDEwMDMyIDAwMDAwIG4gCjAwMDAwMTAz\nMDIgMDAwMDAgbiAKMDAwMDAwODQ2MiAwMDAwMCBuIAowMDAwMDA4MjYyIDAwMDAwIG4gCjAwMDAw\nMDc4NDQgMDAwMDAgbiAKMDAwMDAwOTUxNSAwMDAwMCBuIAowMDAwMDAyOTc5IDAwMDAwIG4gCjAw\nMDAwMDMxNDEgMDAwMDAgbiAKMDAwMDAwMzQyNCAwMDAwMCBuIAowMDAwMDAzODAxIDAwMDAwIG4g\nCjAwMDAwMDQyMTIgMDAwMDAgbiAKMDAwMDAwNDM3MiAwMDAwMCBuIAowMDAwMDA0NjkyIDAwMDAw\nIG4gCjAwMDAwMDQ4MzIgMDAwMDAgbiAKMDAwMDAwNTA3NiAwMDAwMCBuIAowMDAwMDA1Mzk0IDAw\nMDAwIG4gCjAwMDAwMDU2MjQgMDAwMDAgbiAKMDAwMDAwNTc5NCAwMDAwMCBuIAowMDAwMDA1ODgz\nIDAwMDAwIG4gCjAwMDAwMDYyMTEgMDAwMDAgbiAKMDAwMDAwNjQ5OCAwMDAwMCBuIAowMDAwMDA2\nNjUwIDAwMDAwIG4gCjAwMDAwMDY4MDkgMDAwMDAgbiAKMDAwMDAwNzEzMCAwMDAwMCBuIAowMDAw\nMDA3Mjc0IDAwMDAwIG4gCjAwMDAwMDc2ODUgMDAwMDAgbiAKMDAwMDAxMDYzMiAwMDAwMCBuIAp0\ncmFpbGVyCjw8IC9TaXplIDM5IC9JbmZvIDM4IDAgUiAvUm9vdCAxIDAgUiA+PgpzdGFydHhyZWYK\nMTA3ODAKJSVFT0YK\n", "image/svg+xml": [ "\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "figure1();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The accuracy of using the posterior mean of our Gaussian process as a form of function approximation improves with the number of training points." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Figure 3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I now move on to figure 3, demonstrates the procedure for constructing the active subspace and then compares the Gaussian process regression on the original subspace to the GPR on the active subspace.\n", "\n", "For this example I will use a function" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def f3(x):\n", " return np.exp(0.3*x[0] + 0.7*x[1])\n", "\n", "def df3(x):\n", " \"gradient of f3\"\n", " val = f3(x)\n", " return np.column_stack([0.3*val, 0.7*val])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I will be evaluating this function on points in $\\Omega \\equiv [-1, 1]^2$" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def randOmega(N, D=2):\n", " \"random points on Ωᴰ := [-1, 1]ᴰ\"\n", " return 2 * (np.random.rand(N, D) - 0.5)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def figure3():\n", " np.random.seed(43)\n", " Nvals = np.array([4, 8, 16, 32])\n", " num_Ns = len(Nvals)\n", " ave_errs_as = np.inf * np.ones(num_Ns)\n", " ave_errs = np.inf * np.ones(num_Ns)\n", " \n", " # construct test points\n", " X_test = randOmega(1000, 2)\n", " f1_test = f3(X_test.T)\n", " \n", " for i, N in enumerate(Nvals):\n", " # training points\n", " X = randOmega(N, 2)\n", " V = f3(X.T)\n", " G = df3(X.T)\n", " \n", " # find active subspace\n", " CN = (G.T @ G) / N\n", " vals, vecs = linalg.eigh(CN)\n", " W = vecs[:, 1]\n", " Y = X @ W\n", " \n", " # fit GP on active subspace\n", " gp_as = GaussianProcessRegressor(RBF())\n", " gp_as.fit(Y[:, None], V)\n", " m_tilde_as = gp_as.predict((X_test @ W)[:, None])\n", " ave_errs_as[i] = np.sqrt(np.sum((f1_test - m_tilde_as)**2 / (f1_test**2)) / N)\n", " \n", " # fit gp on input domain\n", " gp_as = GaussianProcessRegressor(RBF())\n", " gp_as.fit(X, V)\n", " m_tilde = gp_as.predict(X_test)\n", " ave_errs[i] = np.sqrt(np.sum((f1_test - m_tilde)**2 / (f1_test**2)) / N)\n", "\n", " \n", " fig, ax = plt.subplots(1, 2, figsize=(9, 5))\n", " ax[0].plot([1, 2], W, \".\", ms=15)\n", " ax[0].set_xlabel(\"Input dimension\")\n", " ax[0].set_xticks([1, 2])\n", " ax[0].set_ylabel(\"Magnitude of W\")\n", " \n", " ax[1].semilogy(Nvals, ave_errs_as, \".-\", label=\"ASGP\", ms=15)\n", " ax[1].semilogy(Nvals, ave_errs, \".-\", label=\"GP\", ms=15)\n", " ax[1].legend()\n", " fig.tight_layout()\n", " return fig\n", " " ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 5, 'warnflag': 2, 'grad': array([ -2.15820037e-05]), 'funcalls': 55}\n", " \" state: %s\" % convergence_dict)\n", "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 1, 'warnflag': 2, 'grad': array([-9.64908174]), 'funcalls': 29}\n", " \" state: %s\" % convergence_dict)\n", "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 4, 'warnflag': 2, 'grad': array([-0.00597919]), 'funcalls': 49}\n", " \" state: %s\" % convergence_dict)\n" ] }, { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+\nCmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9TaGFkaW5nIDYgMCBSIC9FeHRHU3RhdGUg\nNCAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1hP\nYmplY3QgNyAwIFIKL1BhdHRlcm4gNSAwIFIgPj4KZW5kb2JqCjEwIDAgb2JqCjw8IC9UeXBlIC9Q\nYWdlIC9Bbm5vdHMgWyBdCi9Hcm91cCA8PCAvVHlwZSAvR3JvdXAgL1MgL1RyYW5zcGFyZW5jeSAv\nQ1MgL0RldmljZVJHQiA+PiAvUmVzb3VyY2VzIDggMCBSCi9NZWRpYUJveCBbIDAgMCA2NDAuMzc4\nMTI1IDM1Mi4xNDM3NSBdIC9Db250ZW50cyA5IDAgUiAvUGFyZW50IDIgMCBSID4+CmVuZG9iago5\nIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTEgMCBSID4+CnN0cmVhbQp4\nnL1ZS28VNxTez6/wsl3E8fHbS1DbqEiVSonKouoChZAGJaGQUv5+vzMzd3zsmZsMlJYQwf3i4/Pw\neYfU2+H0Camre2XUW3x/Ur+p3/Hva0XqTJ1+d/n39cXlL2dP1cX9YIDfDtEb7VImG/DxRn50wWry\nLgXApvn0xzDcDeACijNcfDUM3uk0UyUdxmO3gyOgsaQYK3rToJ60n2Fxg0DB6Q00spNGV2AGrXQW\nerEI+MkQojbB+GSlBAL02sx3Dk8PN5L6NDw9V6c/kCKjzt8M0DEnk7LPhkhZq0tQ56+Hb+hbdf5W\nfX++iMIiDM5A0JI85UZrge5iakvWjlJyJltXBFd74NocJ4fjJlAJjq1VtJ3MxhQ/3v358S/1+vr2\n8u7++t3dptTV0MHrEELOFlLj2W2PPiw1bkuUoVwwOoViTAzO5VEMo/0jrMkkTSFG3/IW8E7mZJx2\ngb0plpl3eIx3KBA4edPxrvBe3iHoEk2wMftoZ+7xEe6WLATOFFvuAt7J3ZqsQyQLhzXWz9zTY9yj\nh3flYjvuFd7LPZKGsIgtCjnO3PMj3OHl8K0SO48T8E7uUw6x2SRHPs3cy2a4GNxzQkgR5HUqbnw2\ng7DMY2Ce345pkC/46dXV3fVfH19fqndv1MvDXe/VRmqzsehSCLkyMgwXLOrDpXqp7tSiOVk4co42\nhQHhEOc/CT9JhqML+RREV81JdfzkL2fDzpMfrob3AyvMmi8J0HFCCwQfSy5Hhy91catOfzLqu3fL\naesDMm8sPhprk7Ip4okOX+L88+G5eq+sejZVmSYftzVmsy5sp/rhxWbBuD1WMHD+M6pOe3q55qHb\nl0L6TE217lOtd0g5Gi8BL5KsowXaCXTToIKDvOKLKp7LeHfrU2lkkOiu8uMSEwfi/O1l0TuSRy3X\nB3hI80YS3cUV5VGjhNlV0SOzzTai6mXf1VqJ7mMbrTYWEYH8hRAQbLe1DUjLlrxFXMm2QqD7+goT\ndA4ef71xXpb4bW0Dxy3O4j0kW4HuY+vZxThRUHFWsj2ibSnIsC4H17AV6D62xelA3qdUshNc3bay\nIhASfDdSDFwfHLSlHu75TpnuhJOXs3m+hgWMEU2SSZAClQWpCxJyptcI9BCnbqkKPPV3A0giHBJO\nxQl2qirjD7g2W2e9nYqPzvbQcfEdabzCjSfHioHyYtKYUqf/gTxbM3ZrXuf5foiEVPn21a8fX7y6\nuz+5vb77eD/n1qPmQa2DGxE8qDFPhXebB528dsWg2UnoI/8P+9j/wT4WWah4MojLxkAC320hi26j\nZGMKwbBJWMjMZf+rW4h2WQi0ZbrlSyzkDPKfQ1yk1kICf8hCiH9hoYIcgcqJUpPQsP9LH4raB5h7\nrpObBjIHAwn9YqsfmkqTcRGN6qU1PGonCbj1ROFzHcEC9wTB6RiKNx2HCq8I8vh2JncEC9wTRKdt\nMTmXlqDCK4KkrUM16kSqcE+A/FxiKrEjqHBPwHNZ8NyutxQCX5GgqSf4RezUEPiKBGEafTDOdiQV\nX5E4lAYPs/RcKr4mwdRmSimxJ1nwFQkvRbIJqXtDga9JPEd86Q22wCuCRGjVU/S9JhVfkWTYBUll\nZeKKr0iK1za6ElJHUvGexBrSyfEOpaGo8JoA5cjl1FtL4CsSJDi2fA4dScXXJEn7ZGyMPcmCr0gs\n2mw8c+kFq/iKxKNN9cm5zl4CX5FgMk64sjexwFckiQczF32vS8XXJJw/0Az2ulR8RQK3KJbbwY6k\n4iuSwokTU3OXHgW+JuFJoJTge5IFX5NkzdGde8Eq3pM4G5Dc+JlbEoGvSBwsQ4iNnqTiIwmPtpWM\nC91nzP08E5MOKEq75/puYsY8RryLydywl6ATlZz8uC9JGF1sRK5lH8e9xbD50WZrgpkyghj2KoQv\nnjo5DpA4EG3L2D/NtTzQ7t4lcAuwf5shNg+LGhTRu1dVXAg0rRJIrh6QfE3EKFe8wzB3Ai9G51US\nz2ph43hikTC2Z95lqBMq6F4Qvx49ycZpztMW5Ry9ioXLnnDDjNIYHKe8LAief6XXV8RmQvvoyhjw\niBbMYoZKRFw++OTO4ml5WdM/OSwSDaUxe9Qn59TlMiXr2ifnYgZByYT65qSOyzQ/9MNHjr/uIjUv\nlvxkUPvA82IuNTaj/4QTb5zuXhexim4cgzOMkDaO98+b4WqeImXezIrz+9dW2+udIxsbmHdr8XN7\nbPHDi67PWB91x8Xq6vj9p0+cXF1NMZwPy6uAVjKZcReKTlzTeJVdrhohXlth2IcfhjxDDIgzacoA\nDYp5Ql0MEsLE4Jc9mIDxRCmZ7JRk4+J8WIqzgBeN4At8AxjTVYoGQ0iFA5fbw9mFW4N6cXGFRyVu\nWmjRVrIajbKy5QVv8JYBaswIcLYpKywbvN0739YtQwwLMwyrbtzO5B77z5P8EJLgiUdlMVIE5DIC\ndIKQhjARIRXxR5R9NPHAIGDEHB7GQo0Ad6P1U+Gp2gZOJHg/g9yDBwTqw8wkQbHgc27RUd2JfsGK\ndt6Dm5K3YhqwXB1CI4JDZ20oTlccpHX87D4n1qCq5Cy/tVB7BJgOrkEeeWo5w+tkU0J7k+MePq2Y\n8u98vC+llZBT61obgS56C0xYqN4qrVlFaEy/iCufqSo1v2X33q2Tm/k3wbyUXi0CBV2e1oBPXpz9\n3C0CReF8oAKtCqeMB3KHjNHGRMW/YiFsIsAUbp9nzsJnKl69i8y0ByqNIxJ8wkaf+mAgnhjQ0UgH\nx0QUMwhdiy5atuFAnHLR5aQmHAgOYGNoo4EiREMJbcMBLYWBcxpqwoEbjYNqwtQL2gaGPL24m7y4\numYrRHVkIXH1eamcQBtTCFwYTly8mFgI0TxHlVi8XVVOvvKWT3xBqIAR1d/p12B5PvwDXQbhdApl\nbmRzdHJlYW0KZW5kb2JqCjExIDAgb2JqCjIxNjUKZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIg\nL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjEwID4+CnN0cmVhbQp4nDVQyw1DMQi7ZwoWqBQCgWSeVr11\n/2tt0DthEf9CWMiUCHmpyc4p6Us+OkwPti6/sSILrXUl7MqaIJ4r76GZsrHR2OJgcBomXoAWN2Do\naY0aNXThgqYulUKBxSXwmXx1e+i+Txl4ahlydgQRQ8lgCWq6Fk1YtDyfkE4B4v9+w+4t5KGS88qe\nG/kbnO3wO7Nu4SdqdiLRchUy1LM0xxgIE0UePHlFpnDis9Z31TQS1GYLTpYBrk4/jA4AYCJeWYDs\nrkQ5S9KOpZ9vvMf3D0AAU7QKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9G\nbGF0ZURlY29kZSAvTGVuZ3RoIDI0NCA+PgpzdHJlYW0KeJxNUUluxDAMu/sV/MAAlqzFeU+Kntr/\nX0s6GEwPhhhZ4hJ3Jyay8LKFuhY6HF828krYDPwetObEz4gV/9E2Iq18UG2BTMREkdBF3WjcI93w\nKkT7kQpeqd5jzUfcrTRv6djF/t4H05pHqqwK8ZBjc2QjaV0TRcf30HrPDdEpgwT63IQcEKU5ehWS\n/toP17XR7NbazJ6oDh6ZEsonzWpoTqeYIbOhHrepWMko3KgyKu5TGWk9yNs4ywBOjhPKaD3ZsZqI\n8lOfXEJeTMQJu+zsWIhNLGwc0n5U+sTy860342+mtZZCxgWJ0BSX3s94j+8/3VhbtAplbmRzdHJl\nYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzE3ID4+\nCnN0cmVhbQp4nDVSS3JDMQjbv1Nwgc6Yv32edLJq7r+thCcrsC1AQi4vWdJLftQl26XD5Fcf9yWx\nQj6P7ZrMUsX3FrMUzy2vR88Rty0KBFETPfgyJxUi1M/U6Dp4YZc+A68QTikWeAeTAAav4V94lE6D\nwDsbMt4Rk5EaECTBmkuLTUiUPUn8K+X1pJU0dH4mK3P5e3KpFGqjyQgVIFi52AekKykeJBM9iUiy\ncr03VojekFeSx2clJhkQ3SaxTbTA49yVtISZmEIF5liA1XSzuvocTFjjsITxKmEW1YNNnjWphGa0\njmNkw3j3wkyJhYbDElCbfZUJqpeP09wJI6ZHTXbtwrJbNu8hRKP5MyyUwccoJAGHTmMkCtKwgBGB\nOb2wir3mCzkWwIhlnZosDG1oJbt6joXA0JyzpWHG157X8/4HRVt7owplbmRzdHJlYW0KZW5kb2Jq\nCjIyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzMyID4+CnN0cmVhbQp4\nnC1SOY4kMQzL/Qp+YADr8vGeHkzU+/90SVUFBapsyzzkcsNEJX4skNtRa+LXRmagwvCvq8yF70jb\nyDqIa8hFXMmWwmdELOQxxDzEgu/b+Bke+azMybMHxi/Z9xlW7KkJy0LGizO0wyqOwyrIsWDrIqp7\neFOkw6kk2OOL/z7FcxeCFr4jaMAv+eerI3i+pEXaPWbbtFsPlmlHlRSWg+1pzsvkS+ssV8fj+SDZ\n3hU7QmpXgKIwd8Z5Lo4ybWVEa2Fng6TGxfbm2I+lBF3oxmWkOAL5mSrCA0qazGyiIP7I6SGnMhCm\nrulKJ7dRFXfqyVyzubydSTJb90WKzRTO68KZ9XeYMqvNO3mWE6VORfgZe7YEDZ3j6tlrmYVGtznB\nKyV8NnZ6cvK9mlkPyalISBXTugpOo8gUS9iW+JqKmtLUy/Dfl/cZf/8BM+J8AQplbmRzdHJlYW0K\nZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTMxID4+CnN0\ncmVhbQp4nEWPyw0EIQxD71ThEvIZPqmH1Z7Y/q/rMJpBQvhBIjvxMAis8/I20MXw0aLDN/421atj\nlSwfunpSVg/pkIe88hVQaTBRxIVZTB1DYc6YysiWMrcb4bZNg6xslVStg3Y8Bg+2p2WrCH6pbWHq\nLPEMwlVeuMcNP5BLrXe9Vb5/QlMwlwplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9GaWx0\nZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzkyID4+CnN0cmVhbQp4nD1SS24FMQjbzym4QKXwTXKe\nqd7u3X9bm8xUqgovA7YxlJcMqSU/6pKIM0x+9XJd4lHyvWxqZ+Yh7i42pvhYcl+6hthy0ZpisU8c\nyS/ItFRYoVbdo0PxhSgTDwAt4IEF4b4c//EXqMHXsIVyw3tkAmBK1G5AxkPRGUhZQRFh+5EV6KRQ\nr2zh7yggV9SshaF0YogNlgApvqsNiZio2aCHhJWSqh3S8Yyk8FvBXYlhUFtb2wR4ZtAQ2d6RjREz\n7dEZcVkRaz896aNRMrVRGQ9NZ3zx3TJS89EV6KTSyN3KQ2fPQidgJOZJmOdwI+Ge20ELMfRxr5ZP\nbPeYKVaR8AU7ygEDvf3eko3Pe+AsjFzb7Ewn8NFppxwTrb4eYv2DP2xLm1zHK4dFFKi8KAh+10ET\ncXxYxfdko0R3tAHWIxPVaCUQDBLCzu0w8njGedneFbTm9ERoo0Qe1I4RPSiyxeWcFbCn/KzNsRye\nDyZ7b7SPlMzMqIQV1HZ6qLbPYx3Ud577+vwBLgChGQplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2Jq\nCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYxID4+CnN0cmVhbQp4nEWQSxLDIAxD\n95xCR/BHBnyedLpK77+tIU2zgKexQAZ3JwSptQUT0QUvbUu6Cz5bCc7GeOg2bjUS5AR1gFak42iU\nUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlHcPVf9Uex7pzNxMBk5Q6EZvUp7nyb\nHVFd3WR/0mNu1mt/FfaqsLSspeWE285dM6AE7qkc7f0FqXM6hAplbmRzdHJlYW0KZW5kb2JqCjI2\nIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzM4ID4+CnN0cmVhbQp4nDVS\nS5JbQQjbv1PoAq5q/s15nJrV5P7bCOysoIEWEpAWOMjESwxRjXLFH3mC8TqBv+vlafw+3oXUgqci\n/cC1aRvvx5o1UbA0YinMPvb9KCHHU+PfEOi5SBNmZDJyIBmI+7U+f9abTDn8BqRpc/ooSXoQLdjd\nGnZ8WZBB0pMaluzkh3UtsLoITZgbayIZObUyNc/HnuEynhgjQdUsIEmfuE8VjEgzHjtnLXmQ4Xiq\nFy9+vY3XMo+pl1UFMrYJ5mA7mQmnKCIQv6AkuYm7aOoojmbGmtuFhpIi9909nJz0ur+cRAVeCeEs\n1hKOGXrKMic7DUqgauUEmGG99oVxmjZKuFPT7V2xr99nJmHc5rCzUjINznFwL5vMESR73TFhEx6H\nmPfuEYzEvPldbBFcucy5JtOP/SjaSB8U1+dcTZmtKOEfquSJFdf4//zez88/kDd9sQplbmRzdHJl\nYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzIwID4+\nCnN0cmVhbQp4nDVRu3HFMAzrNQUX8J34lTSPc6/K278NQDsVYRoEQKq8ZEq5XOqSVbLC5EeH6hRN\n+T5gpvwO9ZDj6B7ZIbpT1pZ7GAjLxDyljlhNlnu4BYEvDE2JuYXz9wjoKwajMBOBusXfP0CzJDBp\ncPBTkGutWmKJDjwsFlizK8ytGilUyFV8Oza5BwVycbPQpxyaFLfcgvBliGRHarGvy2Up8rv1CRiE\nFeaITxSJheeBDmYi8ScDYnv22WJXVy+qERnWSYcHUgTSbG4SMDRFsuqDG9hXxzU/T0fZwclBv4rB\n+DY4mS9JeV8FoRCPF/4Oz9nIsZJDJBTyfbXAiCNsgBGhT+0jEGUgNEX37plSPiZViu8ARiEcfapX\nMrwXkdlqhs3/GV3ZKgoGVVkfn0ZwJoNJOPNkowrTUrXTv/vc4/MHY2N6gAplbmRzdHJlYW0KZW5k\nb2JqCjI4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ1ID4+CnN0cmVh\nbQp4nEVQu41DMQzrPQUXCGD9LHued0iV2789SkZwhSFaP5JaEpiIwEsMsZRv4kdGQT0LvxeF4jPE\nzxeFQc6EpECc9RkQmXiG2kZu6HZwzrzDM4w5AhfFWnCm05n2XNjknAcnEM5tlPGMQrpJVBVxVJ9x\nTPGqss+N14GltWyz05HsIY2ES0klJpd+Uyr/tClbKujaRROwSOSBk0004Sw/Q5JizKCUUfcwtY70\ncbKRR3XQydmcOS2Z2e6n7Ux8D1gmmVHlKZ3nMj4nqfNcTn3usx3R5KKlVfuc/d6RlvIitduh1elX\nJVGZjdWnkLg8/4yf8f4DjqBZPgplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9GaWx0ZXIg\nL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjI3ID4+CnN0cmVhbQp4nDVPO7IDIQzrOYUukBmMbWDPs5lU\nL/dvn2SyDRL+SPL0REcmXubICKzZ8bYWGYgZ+BZT8a897cOE6j24hwjl4kKYYSScNeu4m6fjxb9d\n5TPWwbsNvmKWFwS2MJP1lcWZy3bBWBoncU6yG2PXRGxjXevpFNYRTCgDIZ3tMCXIHBUpfbKjjDk6\nTuSJ52KqxS6/72F9waYxosIcVwVP0GRQlj3vJqAdF/Tf1Y3fSTSLXgIykWBhnSTmzllO+NVrR8dR\niyIxJ6QZ5DIR0pyuYgqhCcU6OwoqFQWX6nPK3T7/aF1bTQplbmRzdHJlYW0KZW5kb2JqCjMwIDAg\nb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggODAgPj4Kc3RyZWFtCnicRYy7DcAw\nCER7pmAEfiZmnyiVs38bIErccE+6e7g6EjJT3mGGhwSeDCyGU/EGmaNgNbhGUo2d7KOwbl91geZ6\nU6v19wcqT3Z2cT3Nyxn0CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxh\ndGVEZWNvZGUgL0xlbmd0aCA4OSA+PgpzdHJlYW0KeJw9jbsRwDAIQ3umYARjftY+uVTO/m1sH+cG\nPQQnYYAbq64BD84OfoTWvukjKZrUR7CILjJFee6tKJvWdVTKJFzK3PmTwnDU4By6v9XacbrE0Vv3\n/tOJHZkKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAv\nTGVuZ3RoIDI0OCA+PgpzdHJlYW0KeJwtUTmSA0EIy+cVekJz0++xy5H3/+kKygGDhkMgOi1xUMZP\nEJYr3vLIVbTh75kYwXfBod/KdRsWORAVSNIYVE2oXbwevQd2HGYC86Q1LIMZ6wM/Ywo3enF4TMbZ\n7XUZNQR712tPZlAyKxdxycQFU3XYyJnDT6aMC+1czw3IuRHWZRikm5XGjIQjTSFSSKHqJqkzQZAE\no6tRo40cxX7pyyOdYVUjagz7XEvb13MTzho0OxarPDmlR1ecy8nFCysH/bzNwEVUGqs8EBJwv9tD\n/Zzs5Dfe0rmzxfT4XnOyvDAVWPHmtRuQTbX4Ny/i+D3j6/n8A6ilWxYKZW5kc3RyZWFtCmVuZG9i\nagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkwID4+CnN0cmVhbQp4\nnE2NQRLAIAgD77wiT1BE0P90etL/X6vUDr3ATgKJFkWC9DVqSzDuuDIVa1ApmJSXwFUwXAva7qLK\n/jJJTJ2G03u3A4Oy8XGD0kn79nF6AKv9egbdD9IcIlgKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9i\nago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDg3ID4+CnN0cmVhbQp4nD2OuxHAMAhD\ne6ZgBPMJhn1yqZz924A/adBDOk64CTYMz9Htws6BN0HuRS+wLm+ACiGLJBn59s7FolaZxa+mReqE\nNEmcN1HoTotK5wPj9A54Prp1HXoKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVy\nIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMwNCA+PgpzdHJlYW0KeJw9kjuSwzAMQ3udghfIjPiT5PNk\nJ5X3/u0+MslWgEmJACgvdZmypjwgaSYJ/9Hh4WI75XfYns3MwLVELxPLKc+hK8TcRfmymY26sjrF\nqsMwnVv0qJyLhk2TmucqSxm3C57DtYnnln3EDzc0qAd1jUvCDd3VaFkKzXB1/zu9R9l3NTwXm1Tq\n1BePF1EV5vkhT6KH6UrifDwoIVx7MEYWEuRT0UCOs1yt8l5C9g63GrLCQWpJ57MnPNh1ek8ubhfN\nEA9kuVT4TlHs7dAzvuxKCT0StuFY7n07mrHpGps47H7vRtbKjK5oIX7IVyfrJWDcUyZFEmROtlhu\ni9We7qEopnOGcxkg6tmKhlLmYlerfww7bywv2SzIlMwLMkanTZ44eMh+jZr0eZXneP0BbPNzOwpl\nbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGgg\nNjggPj4Kc3RyZWFtCnicMzM2UzBQsDACEqamhgrmRpYKKYZcQD6IlcsFE8sBs8wszIEsIwuQlhwu\nQwtjMG1ibKRgZmIGZFkgMSC60gBy+BKRCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKPDwgL1R5\ncGUgL1hPYmplY3QgL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9MZW5ndGggMzcgL1N1\nYnR5cGUgL0Zvcm0KL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUy\nNzYCs3LALCNzIyALJItgQWTTAAFfCgoKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago8PCAvRmls\ndGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMzOCA+PgpzdHJlYW0KeJxFUktyxTAI2+cUXCAz5mfj\n87xOV+n9t5VwOt089AwICTI9ZUim3DaWZITkHPKlV2SI1ZCfRo5ExBDfKaHArvK5vJbEXMhuiUrx\noR0/l6U3Ms2u0Kq3R6c2i0Y1KyPnIEOEelbozO5R22TD63Yh6TpTFodwLP9DBbKUdcoplARtQd/Y\nI+hvFjwR3Aaz5nKzuUxu9b/uWwue1zpbsW0HQAmWc95gBgDEwwnaAMTc2t4WKSgfVbqKScKt8lwn\nO1C20Kp0vDeAGQcYOWDDkq0O12hvAMM+D/SiRsX2FaCoLCD+ztlmwd4xyUiwJ+YGTj1xOsWRcEk4\nxgJAiq3iFLrxHdjiLxeuiJrwCXU6ZU28wp7a4sdCkwjvUnEC8CIbbl0dRbVsT+cJtD8qkjNipB7E\n0QmR1JLOERSXBvXQGvu4iRmvjcTmnr7dP8I5n+v7Fxa4g+AKZW5kc3RyZWFtCmVuZG9iagozOSAw\nIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NSA+PgpzdHJlYW0KeJxFjzsS\nAyEMQ3tOoSOAf8B5NpOKvX8byztJGiyMJT+iKzr2ysNjY8rGa7SxJ8IW7lI6DaepjK+Kp2ddoMth\nMlNvXM1cYKqwyG5OsfoYfKHy7OaEr1WeGMKXTAkNMJU7uYf1+lGclnwkuwuC6pSNysIwR9L3QExN\nm6eFgfyPL3lqQVCZ5o1rlpVHaxVTNGeYKlsLgvWBoDql+gPT/0ynvT/c9D2ACmVuZHN0cmVhbQpl\nbmRvYmoKNDAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzYgPj4Kc3Ry\nZWFtCnicTVBLbkQhDNtzilzgSSQhAc5D1VXn/tuxw1TtKoYYf0gP6bJVHutTYnWJ7PKlTZfKMnkV\nqOVP2/9RDAJu/9DIQbS3jJ1i5hLWxcIkPOU0Ixsn1ywfjztPG2aFxsSN450uGWCfFgE1W5XNgTlt\nOjdAupAat6qz3mRQDCLqQs0Hky6cp9GXiDmeqGBKdya1kBtcPtWhA3FavQq5Y4uTb8QcWaHAYdBM\ncdZfAdaoybJZyCBJhiHOfaN7lAqNqMp5KxXCD5OhEfWG1aAGlbmFoqnlkvwd2gIwBbaMdekMSoGq\nAMHfKqd9vwEkjV1TCmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVE\nZWNvZGUgL0xlbmd0aCA0OSA+PgpzdHJlYW0KeJwzsjRVMFCwtAAShpbmCuZGlgophlxAPoiVywUT\nywGzDIA0WGkOTEUOVxoApUQM5AplbmRzdHJlYW0KZW5kb2JqCjQyIDAgb2JqCjw8IC9GaWx0ZXIg\nL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjU1ID4+CnN0cmVhbQp4nEWRS5IDIAhE956CI4D85DyZmlVy\n/+00mEw2dpeo/YRKI6YSLOcUeTD9yPLNZLbptRyrnY0CiiIUzOQq9FiB1Z0p4sy1RLX1sTJy3Okd\ng+IN566cVLK4UcY6qjoVOKbnyvqq7vy4LMq+I4cyBWzWOQ42cOW2YYwTo81Wd4f7RJCnk6mj4naQ\nbPiDk8a+ytUVuE42++olGAeCfqEJTPJNoHWGQOPmKXpyCfbxcbvzQLC3vAmkbAjkyBCMDkG7Tq5/\ncev83v86w53n2gxXjnfxO0xru+MvMcmKuYBF7hTU8z0XresMHe/JmWNy031D51ywy91Bps/8H+v3\nD1CKZogKZW5kc3RyZWFtCmVuZG9iago0MyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAv\nTGVuZ3RoIDIxNCA+PgpzdHJlYW0KeJw9ULsRQzEI6z0FC+TOfO03z8uly/5tJJykQjZCEpSaTMmU\nhzrKkqwpTx0+S2KHvIflbmQ2JSpFL5OwJffQCvF9ieYU993VlrNDNJdoOX4LMyqqGx3TSzaacCoT\nuqDcwzP6DW10A1aHHrFbINCkYNe2IHLHDxgMwZkTiyIMSk0G/61y91Lc7z0cb6KIlHTwrvnl9MvP\nLbxOPY5Eur35imtxpjoKRHBGavKKdGHFsshDpNUENT0Da7UArt56+TdoR3QZgOwTieM0pRxD/9a4\nx+sDh4pS9AplbmRzdHJlYW0KZW5kb2JqCjQ0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2Rl\nIC9MZW5ndGggODggPj4Kc3RyZWFtCnicNYy7DcAwCER7prgR+DiA94lSkf3bEFsuuHvSE+c5wMg+\nD0foxC1kQ+GmeEk5oT5RNFpvOrZIc7+8ZDMXFf0z3H2F7eaAZDRJ5CHR5XLlWSl6PpfaG34KZW5k\nc3RyZWFtCmVuZG9iago0NSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEz\nMyA+PgpzdHJlYW0KeJxNj0ESwzAIA+9+hZ6AsQHznnR6Sv5/LZA27gXtjICRhjAIPGIM6zAlvHr7\n4VWkS3A2jvklGUU8CGoL3BdUBUdjip342N2h7KXi6RRNi+sRc9O0pHQ3USptvZ3I+MB9n94fVbYk\nnYIeW+qELtEk8kUCc9hUMM/qxktLj6ft2d4fZj4z1wplbmRzdHJlYW0KZW5kb2JqCjQ2IDAgb2Jq\nCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTcxID4+CnN0cmVhbQp4nE2QTQ5CIRCD\n95yiFzCh8wOP82hc6f23dvD54oL0SyFDp8MDHUfiRkeGzuh4sMkxDrwLMiZejfOfjOskjgnqFW3B\nurQ77s0sMScsEyNga5Tcm0cU+OGYC0GC7PLDFxhEpGuYbzWfdZN+frvTXdSldffTIwqcyI5QDBtw\nBdjTPQ7cEs7vmia/VCkZmziUD1QXkbLZCYWopWKXU1VojOJWPe+LXu35AcH2O/sKZW5kc3RyZWFt\nCmVuZG9iago0NyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE3ID4+CnN0\ncmVhbQp4nDM2tFAwgMMUQy4AGpQC7AplbmRzdHJlYW0KZW5kb2JqCjQ4IDAgb2JqCjw8IC9GaWx0\nZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNjggPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYK\nKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIW0I0QZSCWBClZiZmEEk4AyKXBgDJtBXlCmVuZHN0cmVh\nbQplbmRvYmoKNDkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0OSA+Pgpz\ndHJlYW0KeJwzNrRQMFAwNDAHkkaGQJaRiUKKIRdIAMTM5YIJ5oBZBkAaojgHriaHKw0AxugNJgpl\nbmRzdHJlYW0KZW5kb2JqCjUwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGgg\nMjQ3ID4+CnN0cmVhbQp4nE1Ru21EMQzr3xRc4ADra3meC1Jd9m9DyQiQwiChLymnJRb2xksM4Qdb\nD77kkVVDfx4/MewzLD3J5NQ/5rnJVBS+FaqbmFAXYuH9aAS8FnQvIivKB9+PZQxzzvfgoxCXYCY0\nYKxvSSYX1bwzZMKJoY7DQZtUGHdNFCyuFc0zyO1WN7I6syBseCUT4sYARATZF5DNYKOMsZWQxXIe\nqAqSBVpg1+kbUYuCK5TWCXSi1sS6zOCr5/Z2N0Mv8uCounh9DOtLsMLopXssfK5CH8z0TDt3SSO9\n8KYTEWYPBVKZnZGVOj1ifbdA/59lK/j7yc/z/QsVKFwqCmVuZHN0cmVhbQplbmRvYmoKNTEgMCBv\nYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMzggPj4Kc3RyZWFtCnicNVI5rt1A\nDOt9Cl0ggHbNnOcFqX7u34aUXwpDtFaKmo4WlWn5ZSFVLZMuv+1JbYkb8vfJCokTklcl2qUMkVD5\nPIVUv2fLvL7WnBEgS5UKk5OSxyUL/gyX3i4c52NrP48jdz16YFWMhBIByxQTo2tZOrvDmo38PKYB\nP+IRcq5YtxxjFUgNunHaFe9D83nIGiBmmJaKCl1WiRZ+QfGgR61991hUWCDR7RxJcIyNUJGAdoHa\nSAw5sxa7qC/6WZSYCXTtiyLuosASScycYl06+g8+dCyovzbjy6+OSvpIK2tM2nejSWnMIpOul0Vv\nN299PbhA8y7Kf17NIEFT1ihpfNCqnWMomhllhXccmgw0xxyHzBM8hzMSlPR9KH5fSya6KJE/Dg2h\nf18eo4ycBm8Bc9GftooDF/HZYa8cYIXSxZrkfUAqE3pg+v/X+Hn+/AMctoBUCmVuZHN0cmVhbQpl\nbmRvYmoKMTcgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0x\nMDIxIC00NjMgMTc5NCAxMjMzIF0KL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0g\nL1dpZHRocyAxNSAwIFIgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNv\nZGluZwovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgNDYgL3BlcmlvZCA0OCAvemVybyAvb25lIC90\nd28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggL3NldmVuCi9laWdodCAvbmluZSA2NSAvQSA3MSAv\nRyA3MyAvSSA3NyAvTSA4MCAvUCA4MyAvUyA4NyAvVyA5NyAvYSAxMDAgL2QgL2UgL2YKL2cgMTA1\nIC9pIDEwOSAvbSAvbiAvbyAvcCAxMTUgL3MgL3QgL3UgXQo+PgovRm9udERlc2NyaXB0b3IgMTYg\nMCBSIC9CYXNlRm9udCAvRGVqYVZ1U2FucyAvU3VidHlwZSAvVHlwZTMKL05hbWUgL0RlamFWdVNh\nbnMgL0xhc3RDaGFyIDI1NSA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlw\ndG9yIC9GbGFncyAzMiAvRm9udE5hbWUgL0RlamFWdVNhbnMgL0l0YWxpY0FuZ2xlIDAKL1N0ZW1W\nIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvTWF4V2lkdGggMTM0MiAvQ2Fw\nSGVpZ2h0IDAKL0FzY2VudCA5MjkgL1hIZWlnaHQgMCAvRGVzY2VudCAtMjM2ID4+CmVuZG9iagox\nNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAw\nIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAg\nNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAz\nOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYz\nNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3\nMCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1\nIDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAg\nNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2\nMTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYz\nNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0\nMDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAg\nMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3\nCjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUw\nMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0\nIDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUg\nNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2\nMzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3\nOCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0\nIDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvemVybyAxOSAwIFIgL3AgNDAg\nMCBSIC9zaXggMjEgMCBSIC9kIDI5IDAgUiAvZSAyOCAwIFIgL1MgMjYgMCBSCi9uaW5lIDI3IDAg\nUiAvZWlnaHQgMjQgMCBSIC9zIDIyIDAgUiAvdGhyZWUgNTEgMCBSIC9vbmUgMzAgMCBSIC9XIDMx\nIDAgUgovdHdvIDMyIDAgUiAvZm91ciAzMyAwIFIgL2YgNDUgMCBSIC9NIDM0IDAgUiAvYSAzNSAw\nIFIgL3NldmVuIDM2IDAgUgovQSA0NCAwIFIgL2cgMzggMCBSIC9QIDM5IDAgUiAvSSA0MSAwIFIg\nL20gNDIgMCBSIC9vIDQzIDAgUiAvRyAyMCAwIFIKL24gMjUgMCBSIC91IDQ2IDAgUiAvc3BhY2Ug\nNDcgMCBSIC9pIDQ4IDAgUiAvcGVyaW9kIDQ5IDAgUiAvZml2ZSA1MCAwIFIKL3QgMjMgMCBSID4+\nCmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8\nIC9UeXBlIC9FeHRHU3RhdGUgL0NBIDAgL2NhIDEgPj4KL0EzIDw8IC9UeXBlIC9FeHRHU3RhdGUg\nL0NBIDAuOCAvY2EgMC44ID4+Ci9BMiA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+\nID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAg\nb2JqCjw8IC9NMSAxMyAwIFIgL0RlamFWdVNhbnMtbWludXMgMzcgMCBSIC9NMCAxMiAwIFIgL00y\nIDE0IDAgUiA+PgplbmRvYmoKMTIgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL0JCb3ggWyAtNC4y\nNSAtNC4yNSA0LjI1IDQuMjUgXSAvTGVuZ3RoIDEzNQovU3VidHlwZSAvRm9ybSAvRmlsdGVyIC9G\nbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxtkDEOgCAMRfeeggt8AkgRVkev4WJMvP+qEiwKLNCW/kd/\nrTrIqJXuA5OeWZ1kdEqerSsFq5OP3vGTTuzjHJTTgW24K2+wkbxB+rMaQsupya1PUMtFKLoXWuEf\nZGluqfnaqPkNvRA9Hf0I+M8ImR2NKQx8Y7AeDLaIjo9iYyda6AKS9k3fCmVuZHN0cmVhbQplbmRv\nYmoKMTQgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL0JCb3ggWyAtNC4yNSAtNC4yNSA0LjI1IDQu\nMjUgXSAvTGVuZ3RoIDEzNQovU3VidHlwZSAvRm9ybSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+Pgpz\ndHJlYW0KeJxtkDEOgCAMRfeeggt8AkgRVkev4WJMvP+qEiwKLNCW/kd/rTrIqJXuA5OeWZ1kdEqe\nrSsFq5OP3vGTTuzjHJTTgW24K2+wkbxB+rMaQsupya1PUMtFKLoXWuEfZGluqfnaqPkNvRA9Hf0I\n+M8ImR2NKQx8Y7AeDLaIjo9iYyda6AKS9k3fCmVuZHN0cmVhbQplbmRvYmoKMTMgMCBvYmoKPDwg\nL1R5cGUgL1hPYmplY3QgL0JCb3ggWyAtNC4yNSAtNC4yNSA0LjI1IDQuMjUgXSAvTGVuZ3RoIDEz\nNQovU3VidHlwZSAvRm9ybSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxtkDEOgCAM\nRfeeggt8AkgRVkev4WJMvP+qEiwKLNCW/kd/rTrIqJXuA5OeWZ1kdEqerSsFq5OP3vGTTuzjHJTT\ngW24K2+wkbxB+rMaQsupya1PUMtFKLoXWuEfZGluqfnaqPkNvRA9Hf0I+M8ImR2NKQx8Y7AeDLaI\njo9iYyda6AKS9k3fCmVuZHN0cmVhbQplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0Nv\ndW50IDEgL0tpZHMgWyAxMCAwIFIgXSA+PgplbmRvYmoKNTIgMCBvYmoKPDwgL0NyZWF0b3IgKG1h\ndHBsb3RsaWIgMi4wLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChtYXRwbG90\nbGliIHBkZiBiYWNrZW5kKSAvQ3JlYXRpb25EYXRlIChEOjIwMTcwMzMwMTAzOTM1KzExJzAwJykK\nPj4KZW5kb2JqCnhyZWYKMCA1MwowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAg\nbiAKMDAwMDAxNDYyMiAwMDAwMCBuIAowMDAwMDEzNTE3IDAwMDAwIG4gCjAwMDAwMTM1NDkgMDAw\nMDAgbiAKMDAwMDAxMzY5MSAwMDAwMCBuIAowMDAwMDEzNzEyIDAwMDAwIG4gCjAwMDAwMTM3MzMg\nMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzk4IDAwMDAwIG4gCjAwMDAwMDAy\nMDggMDAwMDAgbiAKMDAwMDAwMjYzOCAwMDAwMCBuIAowMDAwMDEzODEyIDAwMDAwIG4gCjAwMDAw\nMTQzNTIgMDAwMDAgbiAKMDAwMDAxNDA4MiAwMDAwMCBuIAowMDAwMDEyMDgzIDAwMDAwIG4gCjAw\nMDAwMTE4ODMgMDAwMDAgbiAKMDAwMDAxMTQwNiAwMDAwMCBuIAowMDAwMDEzMTM2IDAwMDAwIG4g\nCjAwMDAwMDI2NTkgMDAwMDAgbiAKMDAwMDAwMjk0MiAwMDAwMCBuIAowMDAwMDAzMjU5IDAwMDAw\nIG4gCjAwMDAwMDM2NDkgMDAwMDAgbiAKMDAwMDAwNDA1NCAwMDAwMCBuIAowMDAwMDA0MjU4IDAw\nMDAwIG4gCjAwMDAwMDQ3MjMgMDAwMDAgbiAKMDAwMDAwNDk1NyAwMDAwMCBuIAowMDAwMDA1MzY4\nIDAwMDAwIG4gCjAwMDAwMDU3NjEgMDAwMDAgbiAKMDAwMDAwNjA3OSAwMDAwMCBuIAowMDAwMDA2\nMzc5IDAwMDAwIG4gCjAwMDAwMDY1MzEgMDAwMDAgbiAKMDAwMDAwNjY5MiAwMDAwMCBuIAowMDAw\nMDA3MDEzIDAwMDAwIG4gCjAwMDAwMDcxNzUgMDAwMDAgbiAKMDAwMDAwNzMzNCAwMDAwMCBuIAow\nMDAwMDA3NzExIDAwMDAwIG4gCjAwMDAwMDc4NTEgMDAwMDAgbiAKMDAwMDAwODAyMSAwMDAwMCBu\nIAowMDAwMDA4NDMyIDAwMDAwIG4gCjAwMDAwMDg2NzAgMDAwMDAgbiAKMDAwMDAwODk3OSAwMDAw\nMCBuIAowMDAwMDA5MTAwIDAwMDAwIG4gCjAwMDAwMDk0MjggMDAwMDAgbiAKMDAwMDAwOTcxNSAw\nMDAwMCBuIAowMDAwMDA5ODc1IDAwMDAwIG4gCjAwMDAwMTAwODEgMDAwMDAgbiAKMDAwMDAxMDMy\nNSAwMDAwMCBuIAowMDAwMDEwNDE0IDAwMDAwIG4gCjAwMDAwMTA1NTQgMDAwMDAgbiAKMDAwMDAx\nMDY3NSAwMDAwMCBuIAowMDAwMDEwOTk1IDAwMDAwIG4gCjAwMDAwMTQ2ODIgMDAwMDAgbiAKdHJh\naWxlcgo8PCAvU2l6ZSA1MyAvSW5mbyA1MiAwIFIgL1Jvb3QgMSAwIFIgPj4Kc3RhcnR4cmVmCjE0\nODMwCiUlRU9GCg==\n", "image/svg+xml": [ "\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "figure3();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Somewhat surprisingly the GPR is more accurate on the 1-dimensional active subspace instead of the 2-dimensional input space. I'm not 100% sure why this happens. However, one reason for this could be that the squared Euclidean kernel loses power as the number of dimensions increases. If we have some dimensions that really don't influence the value of $f$ very much, keeping them around only hurts the power of our chosen kernel, so the GPR might struggle to achieve high accuracy." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Figure 4\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example, we move to a higher dimensional function $f4 : \\mathbb{R}^10 \\rightarrow \\mathbb{R}$. The function is just the exponential of a coefficient vector and a point $x \\in \\mathbb{R}^10$. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def f4(x):\n", " return np.exp(0.01*x[0] + 0.7*x[1] + 0.02*x[2] + 0.03*x[3] + 0.04*x[4] + \n", " 0.05*x[5] + 0.06*x[6] + 0.08*x[7] + 0.09*x[8] + 0.1*x[9])\n", "\n", "def df4(x):\n", " val = f4(x)\n", " coefs = np.array([0.01, 0.7, 0.02, 0.03, 0.04, 0.05, 0.06, 0.08, 0.09, 0.1])\n", " return val[:, None] * coefs[None, :]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that the coefficient on the second component of $x$ is an order of magnitude larger than all the other coefficients, so the value of the second component will largely drive the value of $f4$. \n", "\n", "This means we should expect to see a 1 dimensional active subspace. \n", "\n", "To verify if our intuition is correct, we will plot both the set of eigenvalues of the matrix $C_N$ as well as the principal eigenvector of $C_N$." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def figure4():\n", " np.random.seed(41)\n", " N = 300\n", " X = randOmega(N, 10)\n", " V = f4(X.T)\n", " G = df4(X.T)\n", " CN = (G.T @ G) / N\n", "\n", " # find active subspace\n", " vals, vecs = linalg.eigh(CN)\n", " W = vecs[:, -1]\n", " \n", " fig, ax = plt.subplots(1, 2, figsize=(9, 5))\n", " x = np.arange(1, 11)\n", " ax[0].plot(x, vals, \".\", ms=15)\n", " ax[0].set_xlabel(\"Sorted eigenvalues\")\n", " ax[0].set_xticks(x)\n", " ax[0].set_ylabel(\"$\\lambda$\")\n", " \n", " ax[1].plot(x, W, \".\", ms=15)\n", " ax[1].set_xlabel(\"Input dimension\")\n", " ax[1].set_xticks(x)\n", " ax[1].set_ylabel(\"Magnitude of W\")\n", " \n", " fig.tight_layout()\n", " return fig\n", " " ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+\nCmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9TaGFkaW5nIDYgMCBSIC9FeHRHU3RhdGUg\nNCAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1hP\nYmplY3QgNyAwIFIKL1BhdHRlcm4gNSAwIFIgPj4KZW5kb2JqCjEwIDAgb2JqCjw8IC9UeXBlIC9Q\nYWdlIC9Bbm5vdHMgWyBdCi9Hcm91cCA8PCAvVHlwZSAvR3JvdXAgL1MgL1RyYW5zcGFyZW5jeSAv\nQ1MgL0RldmljZVJHQiA+PiAvUmVzb3VyY2VzIDggMCBSCi9NZWRpYUJveCBbIDAgMCA2MzguMzc4\nMTI1IDM1Mi4xNDM3NSBdIC9Db250ZW50cyA5IDAgUiAvUGFyZW50IDIgMCBSID4+CmVuZG9iago5\nIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTEgMCBSID4+CnN0cmVhbQp4\nnO2YT28bNxDF7/speGwPpjkkZzg8NkhrNEBQJDGaQ9GDY8uuDUuu4zj5+n3USlpSWDlMYRQI0AiB\nrecRfxz+mZ0nMjfD8U9krh6MMzf4/8X8Yf7EzwtD5sQcv1x8vj5fvD15Yc4fBgd9OUhQG5KSZ7y9\nrd8G9pZiSAzZNe/+GobVAAo+cYKBr4YhBps2n0qW12HLITi2mp3U6m2jRrJxI1cjVCpIl8jIjxld\nAYasrFZ5lSngLwOzlexDqCcwadG6zYjDi+14ZL4ML07N8S9kyJnTywEZMnmJ3kVm473NbE4vhh/o\nR3N6Y34+3U2kTGBQrIdoiDVw0rqASW1WFfYUpQb6WSA5tYKFada4EruQhNWPEkW8j6lmhnlmYEve\nBWmYk9jHDGRzdJJ8iFoz4zxTyHKWkBrmJPYxWW0kEfUx5prJ80xV6zA/bZiT2MfUiJPtceSYXc2U\nWSZ2HSgNuWZWYhfTE9kgmpwXppqZ5pm4Ww5L0iB3Wh8RH9UYE/nEvibqPFFAci5Sg5zEPqZEGzz2\nwiuHmpnnmTnYnHCRG+Yk9jE1WaGEWlBedTlwW2i7+14sJ+FS2LL1Yy0r8e/uPn5aXJjF9dVi9fns\n9nHxMDvpqQAytpRTUMGso9sONalPTxujJVJkF5G7OCYKgqtaZuKs+wpay8wlU0PeiZ1gRYlR53DZ\nE1Z7BM/X0AmM02QpS0SxrtGV3AknH2zImVmDxrShzxfUii6oaMSZU0uf5F46ylSg7JQ08zb3+dI6\n0VFKrQu49Q18UnvZGfuE8+E4OtYNe77EVmwAsVi4iy18kjvppSyg6JLXTBI29PliW9HxBMQhYd+u\neyX30hOjVOP2+Yj6u6HPl92q0SCxCQ8mbo97JXfSA6E45ESJc0hxQ98V4Ht0T2SOCG0KinvKpfKQ\nio3r386XGHUoYZifK0Fu94uN6zYJIejcbs5+f3x3tno4+u3D7fX94+Lo9mz54eLMvLwb3uB1b2Z6\nMC9oSNYKDlM2HxfmvVmZ3XKQJxYVLN4Almz+Jfwl4QSpR8+HD101keZw5NuToTPy4xVWZcyVzKZL\n85lL27Urc+uqdb40x69dyXEbjSobkzgqLwj/B5QAlGec3u2risbJMPfGm1ejH2g659YNzHbw8035\n8G62tV8eau0R/w3+oI3eDfPU6DvL88qMruTL5EwCajiPT+UKjfNpZW9Ct41aEeoh/pU3CSlhs7I2\n5qQSu/oRVBYbnKNOexLLjcdTsGn1KrGLGR2ubRLqdCgx4MoH0r0ztBP7mNgED2CnQ4ksNrukjUOp\nxD4m48R5pU6Hgv7cBvQXjUOpxD6m4omDFrjTobDLVhl2sTG4k9jncJ1YUUedDoWD2BBYG4dSiX3M\nENAYiO90KIi0StQid1ofkbOV4H2nQ+HS32rKjUOpxD4mHujw4r7ToeBK2SQhNw6lEruYgt2PG3/S\n41DKrSAOtG9Qfl39/fjJXFwvF6uH67vV7ISrGogSiLvlqfRsATvj9uX9SY+P+qPy1ArIzOs6taBo\nTWASypLR2AZVc11/CzUWezfmNHqXYWyo9jul/R5peb16fNi2RocSyeXyeNa9RCa5L5Es1jHDDzu/\n7li+kgim8dyJECf0lWm0L1Umld6XCjEOHjoK2BfKHZuy6bGfMxdfLgH8nt/LpdL7cinfjOQokeHJ\nYte+xGfPJQWUW5Use7lMemcugjOZ8aig2LUr/rkzCYFQUmFH9jKp9Kebl+it8NrwYAethoTz5XFt\n9r+XaD61803BK56V45FGcBxX5XS5/g67jPD67Gp1/enxYmHuLs377WCl953r/74LW7TtDze+qCyb\nBDd+mxPHFp8OGoKjOUdwOBwx1oumFJPbuq4nBi9FImTUSPi27yVYEZwyTD16tP9kGv3Wa96iHHAd\nsDtz5mV5yLwUs/YNFmgvvLJfh8d/M/wDpiSAkwplbmRzdHJlYW0KZW5kb2JqCjExIDAgb2JqCjE0\nNTQKZW5kb2JqCjE4IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9CQm94IFsgLTEwMTYgLTM1MSAx\nNjYwIDEwNjggXSAvTGVuZ3RoIDEwMgovU3VidHlwZSAvRm9ybSAvRmlsdGVyIC9GbGF0ZURlY29k\nZSA+PgpzdHJlYW0KeJw1jjsKxEAMQ3ufQhcIjOXvnCeQKrl/u5MJWwg9JLAsloUsxyPejYFbbNrn\n2vDhi0J3cPhXcBaCb8EeSOpLZcgkGES2gVrbT9Eam27RDlTuC05UzD2ysijYUnHi/84p1w/uyx2e\nCmVuZHN0cmVhbQplbmRvYmoKMTYgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL0ZpcnN0Q2hhciAwIC9G\nb250QkJveCBbIC0xMDE2IC0zNTEgMTY2MCAxMDY4IF0KL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAg\nMC4wMDEgMCAwIF0gL1dpZHRocyAxNCAwIFIgL0NoYXJQcm9jcyAxNyAwIFIKL0VuY29kaW5nIDw8\nIC9UeXBlIC9FbmNvZGluZyAvRGlmZmVyZW5jZXMgWyBdID4+IC9Gb250RGVzY3JpcHRvciAxNSAw\nIFIKL0Jhc2VGb250IC9EZWphVnVTYW5zLU9ibGlxdWUgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9E\nZWphVnVTYW5zLU9ibGlxdWUKL0xhc3RDaGFyIDI1NSA+PgplbmRvYmoKMTUgMCBvYmoKPDwgL1R5\ncGUgL0ZvbnREZXNjcmlwdG9yIC9GbGFncyA5NiAvRm9udE5hbWUgL0RlamFWdVNhbnMtT2JsaXF1\nZQovSXRhbGljQW5nbGUgMCAvU3RlbVYgMCAvRm9udEJCb3ggWyAtMTAxNiAtMzUxIDE2NjAgMTA2\nOCBdIC9NYXhXaWR0aCAxMzUwCi9DYXBIZWlnaHQgMCAvQXNjZW50IDkyOSAvWEhlaWdodCAwIC9E\nZXNjZW50IC0yMzYgPj4KZW5kb2JqCjE0IDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAg\nNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2\nMDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2\nMCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2\nIDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4Mzgg\nNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcg\nODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAz\nOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3\nOCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4\nIDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAw\nIDUwMCA1MDAgNTAwIDEzNTAgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUx\nOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyOCA2MDAgNTI1IDYxMSAzMTgg\nNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTcgODM4IDM2MSAxMDAw\nIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxNyA5Njkg\nOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2\nMzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDcz\nMiA3MzIgNzMyIDczMiA2MTEgNjA4CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5OTUgNTUw\nIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIg\nNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE3IDAgb2Jq\nCjw8ID4+CmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIx\nMCA+PgpzdHJlYW0KeJw1UMsNQzEIu2cKFqgUAoFknla9df9rbdA7YRH/QljIlAh5qcnOKelLPjpM\nD7Yuv7EiC611JezKmiCeK++hmbKx0djiYHAaJl6AFjdg6GmNGjV04YKmLpVCgcUl8Jl8dXvovk8Z\neGoZcnYEEUPJYAlquhZNWLQ8n5BOAeL/fsPuLeShkvPKnhv5G5zt8DuzbuEnanYi0XIVMtSzNMcY\nCBNFHjx5RaZw4rPWd9U0EtRmC06WAa5OP4wOAGAiXlmA7K5EOUvSjqWfb7zH9w9AAFO0CmVuZHN0\ncmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDgg\nPj4Kc3RyZWFtCnicLVE5kgNBCMvnFXpCc9PvscuR9//pCsoBg4ZDIDotcVDGTxCWK97yyFW04e+Z\nGMF3waHfynUbFjkQFUjSGFRNqF28Hr0HdhxmAvOkNSyDGesDP2MKN3pxeEzG2e11GTUEe9drT2ZQ\nMisXccnEBVN12MiZw0+mjAvtXM8NyLkR1mUYpJuVxoyEI00hUkih6iapM0GQBKOrUaONHMV+6csj\nnWFVI2oM+1xL29dzE84aNDsWqzw5pUdXnMvJxQsrB/28zcBFVBqrPBAScL/bQ/2c7OQ33tK5s8X0\n+F5zsrwwFVjx5rUbkE21+Dcv4vg94+v5/AOopVsWCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoK\nPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzYgPj4Kc3RyZWFtCnicTVBLbkQhDNtz\nilzgSSQhAc5D1VXn/tuxw1TtKoYYf0gP6bJVHutTYnWJ7PKlTZfKMnkVqOVP2/9RDAJu/9DIQbS3\njJ1i5hLWxcIkPOU0Ixsn1ywfjztPG2aFxsSN450uGWCfFgE1W5XNgTltOjdAupAat6qz3mRQDCLq\nQs0Hky6cp9GXiDmeqGBKdya1kBtcPtWhA3FavQq5Y4uTb8QcWaHAYdBMcdZfAdaoybJZyCBJhiHO\nfaN7lAqNqMp5KxXCD5OhEfWG1aAGlbmFoqnlkvwd2gIwBbaMdekMSoGqAMHfKqd9vwEkjV1TCmVu\nZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAz\nMTcgPj4Kc3RyZWFtCnicNVJLckMxCNu/U3CBzpi/fZ50smruv62EJyuwLUBCLi9Z0kt+1CXbpcPk\nVx/3JbFCPo/tmsxSxfcWsxTPLa9HzxG3LQoEURM9+DInFSLUz9ToOnhhlz4DrxBOKRZ4B5MABq/h\nX3iUToPAOxsy3hGTkRoQJMGaS4tNSJQ9Sfwr5fWklTR0fiYrc/l7cqkUaqPJCBUgWLnYB6QrKR4k\nEz2JSLJyvTdWiN6QV5LHZyUmGRDdJrFNtMDj3JW0hJmYQgXmWIDVdLO6+hxMWOOwhPEqYRbVg02e\nNamEZrSOY2TDePfCTImFhsMSUJt9lQmql4/T3AkjpkdNdu3Csls27yFEo/kzLJTBxygkAYdOYyQK\n0rCAEYE5vbCKveYLORbAiGWdmiwMbWglu3qOhcDQnLOlYcbXntfz/gdFW3ujCmVuZHN0cmVhbQpl\nbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NSA+PgpzdHJl\nYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKeBgCffQy1CmVuZHN0cmVh\nbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzOTIgPj4K\nc3RyZWFtCnicPVJLbgUxCNvPKbhApfBNcp6p3u7df1ubzFSqCi8DtjGUlwypJT/qkogzTH71cl3i\nUfK9bGpn5iHuLjam+FhyX7qG2HLRmmKxTxzJL8i0VFihVt2jQ/GFKBMPAC3ggQXhvhz/8Reowdew\nhXLDe2QCYErUbkDGQ9EZSFlBEWH7kRXopFCvbOHvKCBX1KyFoXRiiA2WACm+qw2JmKjZoIeElZKq\nHdLxjKTwW8FdiWFQW1vbBHhm0BDZ3pGNETPt0RlxWRFrPz3po1EytVEZD01nfPHdMlLz0RXopNLI\n3cpDZ89CJ2Ak5kmY53Aj4Z7bQQsx9HGvlk9s95gpVpHwBTvKAQO9/d6Sjc974CyMXNvsTCfw0Wmn\nHBOtvh5i/YM/bEubXMcrh0UUqLwoCH7XQRNxfFjF92SjRHe0AdYjE9VoJRAMEsLO7TDyeMZ52d4V\ntOb0RGijRB7UjhE9KLLF5ZwVsKf8rM2xHJ4PJntvtI+UzMyohBXUdnqots9jHdR3nvv6/AEuAKEZ\nCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0\naCAxNTcgPj4Kc3RyZWFtCnicRZC5EUMxCERzVUEJErAI6rHH0Xf/qRf5SrRvAC2HryVTqh8nIqbc\n12j0MHkOn00lVizYJraTGnIbFkFKMZh4TjGro7ehmYfU67ioqrh1ZpXTacvKxX/zaFczkz3CNeon\n8E3o+J88tKnoW6CvC5R9QLU4nUlQMX2vYoGjnHZ/IpwY4D4ZR5kpI3Fibgrs9xkAZr5XuMbjBd0B\nN3kKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVu\nZ3RoIDMzOCA+PgpzdHJlYW0KeJw1UkuSW0EI279T6AKuav7NeZya1eT+2wjsrKCBFhKQFjjIxEsM\nUY1yxR95gvE6gb/r5Wn8Pt6F1IKnIv3AtWkb78eaNVGwNGIpzD72/Sghx1Pj3xDouUgTZmQyciAZ\niPu1Pn/Wm0w5/AakaXP6KEl6EC3Y3Rp2fFmQQdKTGpbs5Id1LbC6CE2YG2siGTm1MjXPx57hMp4Y\nI0HVLCBJn7hPFYxIMx47Zy15kOF4qhcvfr2N1zKPqZdVBTK2CeZgO5kJpygiEL+gJLmJu2jqKI5m\nxprbhYaSIvfdPZyc9Lq/nEQFXgnhLNYSjhl6yjInOw1KoGrlBJhhvfaFcZo2SrhT0+1dsa/fZyZh\n3Oaws1IyDc5xcC+bzBEke90xYRMeh5j37hGMxLz5XWwRXLnMuSbTj/0o2kgfFNfnXE2ZrSjhH6rk\niRXX+P/83s/PP5A3fbEKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0\nZURlY29kZSAvTGVuZ3RoIDMyMCA+PgpzdHJlYW0KeJw1UbtxxTAM6zUFF/Cd+JU0j3Ovytu/DUA7\nFWEaBECqvGRKuVzqklWywuRHh+oUTfk+YKb8DvWQ4+ge2SG6U9aWexgIy8Q8pY5YTZZ7uAWBLwxN\nibmF8/cI6CsGozATgbrF3z9AsyQwaXDwU5BrrVpiiQ48LBZYsyvMrRopVMhVfDs2uQcFcnGz0Kcc\nmhS33ILwZYhkR2qxr8tlKfK79QkYhBXmiE8UiYXngQ5mIvEnA2J79tliV1cvqhEZ1kmHB1IE0mxu\nEjA0RbLqgxvYV8c1P09H2cHJQb+Kwfg2OJkvSXlfBaEQjxf+Ds/ZyLGSQyQU8n21wIgjbIARoU/t\nIxBlIDRF9+6ZUj4mVYrvAEYhHH2qVzK8F5HZaobN/xld2SoKBlVZH59GcCaDSTjzZKMK01K107/7\n3OPzB2NjeoAKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29k\nZSAvTGVuZ3RoIDI0NSA+PgpzdHJlYW0KeJxFULuNQzEM6z0FFwhg/Sx7nndIldu/PUpGcIUhWj+S\nWhKYiMBLDLGUb+JHRkE9C78XheIzxM8XhUHOhKRAnPUZEJl4htpGbuh2cM68wzOMOQIXxVpwptOZ\n9lzY5JwHJxDObZTxjEK6SVQVcVSfcUzxqrLPjdeBpbVss9OR7CGNhEtJJSaXflMq/7QpWyro2kUT\nsEjkgZNNNOEsP0OSYsyglFH3MLWO9HGykUd10MnZnDktmdnup+1MfA9YJplR5Smd5zI+J6nzXE59\n7rMd0eSipVX7nP3ekZbyIrXbodXpVyVRmY3Vp5C4PP+Mn/H+A46gWT4KZW5kc3RyZWFtCmVuZG9i\nagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDgwID4+CnN0cmVhbQp4\nnEWMuw3AMAhEe6ZgBH4mZp8olbN/GyBK3HBPunu4OhIyU95hhocEngwshlPxBpmjYDW4RlKNneyj\nsG5fdYHmelOr9fcHKk92dnE9zcsZ9AplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9GaWx0\nZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjI3ID4+CnN0cmVhbQp4nDVPO7IDIQzrOYUukBmMbWDP\ns5lUL/dvn2SyDRL+SPL0REcmXubICKzZ8bYWGYgZ+BZT8a897cOE6j24hwjl4kKYYSScNeu4m6fj\nxb9d5TPWwbsNvmKWFwS2MJP1lcWZy3bBWBoncU6yG2PXRGxjXevpFNYRTCgDIZ3tMCXIHBUpfbKj\njDk6TuSJ52KqxS6/72F9waYxosIcVwVP0GRQlj3vJqAdF/Tf1Y3fSTSLXgIykWBhnSTmzllO+NVr\nR8dRiyIxJ6QZ5DIR0pyuYgqhCcU6OwoqFQWX6nPK3T7/aF1bTQplbmRzdHJlYW0KZW5kb2JqCjM1\nIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggODkgPj4Kc3RyZWFtCnicPY27\nEcAwCEN7pmAEY37WPrlUzv5tbB/nBj0EJ2GAG6uuAQ/ODn6E1r7pIyma1EewiC4yRXnurSib1nVU\nyiRcytz5k8Jw1OAcur/V2nG6xNFb9/7TiR2ZCmVuZHN0cmVhbQplbmRvYmoKMzYgMCBvYmoKPDwg\nL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicTY9BEsMwCAPvfoWe\ngLEB8550ekr+fy2QNu4F7YyAkYYwCDxiDOswJbx6++FVpEtwNo75JRlFPAhqC9wXVAVHY4qd+Njd\noeyl4ukUTYvrEXPTtKR0N1Eqbb2dyPjAfZ/eH1W2JJ2CHlvqhC7RJPJFAnPYVDDP6sZLS4+n7dne\nH2Y+M9cKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAv\nTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDWMsRHAMAgDe6bQCDZYYO+TS0X2b0N8TgMvHQ+XosFaDbqC\nI3B1qfzRI125KUWXY86C4XGqX0gxRj2oI+Pex0+5X3AWEn0KZW5kc3RyZWFtCmVuZG9iagozOCAw\nIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkwID4+CnN0cmVhbQp4nE2NQRLA\nIAgD77wiT1BE0P90etL/X6vUDr3ATgKJFkWC9DVqSzDuuDIVa1ApmJSXwFUwXAva7qLK/jJJTJ2G\n03u3A4Oy8XGD0kn79nF6AKv9egbdD9IcIlgKZW5kc3RyZWFtCmVuZG9iagozOSAwIG9iago8PCAv\nRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDg3ID4+CnN0cmVhbQp4nD2OuxHAMAhDe6ZgBPMJ\nhn1yqZz924A/adBDOk64CTYMz9Htws6BN0HuRS+wLm+ACiGLJBn59s7FolaZxa+mReqENEmcN1Ho\nTotK5wPj9A54Prp1HXoKZW5kc3RyZWFtCmVuZG9iago0MCAwIG9iago8PCAvRmlsdGVyIC9GbGF0\nZURlY29kZSAvTGVuZ3RoIDMwNCA+PgpzdHJlYW0KeJw9kjuSwzAMQ3udghfIjPiT5PNkJ5X3/u0+\nMslWgEmJACgvdZmypjwgaSYJ/9Hh4WI75XfYns3MwLVELxPLKc+hK8TcRfmymY26sjrFqsMwnVv0\nqJyLhk2TmucqSxm3C57DtYnnln3EDzc0qAd1jUvCDd3VaFkKzXB1/zu9R9l3NTwXm1Tq1BePF1EV\n5vkhT6KH6UrifDwoIVx7MEYWEuRT0UCOs1yt8l5C9g63GrLCQWpJ57MnPNh1ek8ubhfNEA9kuVT4\nTlHs7dAzvuxKCT0StuFY7n07mrHpGps47H7vRtbKjK5oIX7IVyfrJWDcUyZFEmROtlhui9We7qEo\npnOGcxkg6tmKhlLmYlerfww7bywv2SzIlMwLMkanTZ44eMh+jZr0eZXneP0BbPNzOwplbmRzdHJl\nYW0KZW5kb2JqCjQxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNjggPj4K\nc3RyZWFtCnicMzM2UzBQsDACEqamhgrmRpYKKYZcQD6IlcsFE8sBs8wszIEsIwuQlhwuQwtjMG1i\nbKRgZmIGZFkgMSC60gBy+BKRCmVuZHN0cmVhbQplbmRvYmoKNDIgMCBvYmoKPDwgL1R5cGUgL1hP\nYmplY3QgL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9MZW5ndGggMzcgL1N1YnR5cGUg\nL0Zvcm0KL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LA\nLCNzIyALJItgQWTTAAFfCgoKZW5kc3RyZWFtCmVuZG9iago0MyAwIG9iago8PCAvRmlsdGVyIC9G\nbGF0ZURlY29kZSAvTGVuZ3RoIDMzOCA+PgpzdHJlYW0KeJxFUktyxTAI2+cUXCAz5mfj87xOV+n9\nt5VwOt089AwICTI9ZUim3DaWZITkHPKlV2SI1ZCfRo5ExBDfKaHArvK5vJbEXMhuiUrxoR0/l6U3\nMs2u0Kq3R6c2i0Y1KyPnIEOEelbozO5R22TD63Yh6TpTFodwLP9DBbKUdcoplARtQd/YI+hvFjwR\n3Aaz5nKzuUxu9b/uWwue1zpbsW0HQAmWc95gBgDEwwnaAMTc2t4WKSgfVbqKScKt8lwnO1C20Kp0\nvDeAGQcYOWDDkq0O12hvAMM+D/SiRsX2FaCoLCD+ztlmwd4xyUiwJ+YGTj1xOsWRcEk4xgJAiq3i\nFLrxHdjiLxeuiJrwCXU6ZU28wp7a4sdCkwjvUnEC8CIbbl0dRbVsT+cJtD8qkjNipB7E0QmR1JLO\nERSXBvXQGvu4iRmvjcTmnr7dP8I5n+v7Fxa4g+AKZW5kc3RyZWFtCmVuZG9iago0NCAwIG9iago8\nPCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMzMiA+PgpzdHJlYW0KeJwtUjmOJDEMy/0K\nfmAA6/Lxnh5M1Pv/dElVBQWqbMs85HLDRCV+LJDbUWvi10ZmoMLwr6vMhe9I28g6iGvIRVzJlsJn\nRCzkMcQ8xILv2/gZHvmszMmzB8Yv2fcZVuypCctCxosztMMqjsMqyLFg6yKqe3hTpMOpJNjji/8+\nxXMXgha+I2jAL/nnqyN4vqRF2j1m27RbD5ZpR5UUloPtac7L5EvrLFfH4/kg2d4VO0JqV4CiMHfG\neS6OMm1lRGthZ4OkxsX25tiPpQRd6MZlpDgC+ZkqwgNKmsxsoiD+yOkhpzIQpq7pSie3URV36slc\ns7m8nUkyW/dFis0UzuvCmfV3mDKrzTt5lhOlTkX4GXu2BA2d4+rZa5mFRrc5wSslfDZ2enLyvZpZ\nD8mpSEgV07oKTqPIFEvYlviaiprS1Mvw35f3GX//ATPifAEKZW5kc3RyZWFtCmVuZG9iago0NSAw\nIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIxNCA+PgpzdHJlYW0KeJw9ULsR\nQzEI6z0FC+TOfO03z8uly/5tJJykQjZCEpSaTMmUhzrKkqwpTx0+S2KHvIflbmQ2JSpFL5OwJffQ\nCvF9ieYU993VlrNDNJdoOX4LMyqqGx3TSzaacCoTuqDcwzP6DW10A1aHHrFbINCkYNe2IHLHDxgM\nwZkTiyIMSk0G/61y91Lc7z0cb6KIlHTwrvnl9MvPLbxOPY5Eur35imtxpjoKRHBGavKKdGHFsshD\npNUENT0Da7UArt56+TdoR3QZgOwTieM0pRxD/9a4x+sDh4pS9AplbmRzdHJlYW0KZW5kb2JqCjQ2\nIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNDkgPj4Kc3RyZWFtCnicM7I0\nVTBQsLQAEoaW5grmRpYKKYZcQD6IlcsFE8sBswyANFhpDkxFDlcaAKVEDOQKZW5kc3RyZWFtCmVu\nZG9iago0NyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI1NSA+PgpzdHJl\nYW0KeJxFkUuSAyAIRPeegiOA/OQ8mZpVcv/tNJhMNnaXqP2ESiOmEiznFHkw/cjyzWS26bUcq52N\nAooiFMzkKvRYgdWdKeLMtUS19bEyctzpHYPiDeeunFSyuFHGOqo6FTim58r6qu78uCzKviOHMgVs\n1jkONnDltmGME6PNVneH+0SQp5Opo+J2kGz4g5PGvsrVFbhONvvqJRgHgn6hCUzyTaB1hkDj5il6\ncgn28XG780Cwt7wJpGwI5MgQjA5Bu06uf3Hr/N7/OsOd59oMV4538TtMa7vjLzHJirmARe4U1PM9\nF63rDB3vyZljctN9Q+dcsMvdQabP/B/r9w9QimaICmVuZHN0cmVhbQplbmRvYmoKNDggMCBvYmoK\nPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzEgPj4Kc3RyZWFtCnicRY/LDQQhDEPv\nVOES8hk+qYfVntj+r+swmkFC+EEiO/EwCKzz8jbQxfDRosM3/jbVq2OVLB+6elJWD+mQh7zyFVBp\nMFHEhVlMHUNhzpjKyJYytxvhtk2DrGyVVK2DdjwGD7anZasIfqltYeos8QzCVV64xw0/kEutd71V\nvn9CUzCXCmVuZHN0cmVhbQplbmRvYmoKNDkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUg\nL0xlbmd0aCAxNjEgPj4Kc3RyZWFtCnicRZBLEsMgDEP3nEJH8EcGfJ50ukrvv60hTbOAp7FABncn\nBKm1BRPRBS9tS7oLPlsJzsZ46DZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6\nFd7touRypu7O+Udw9V/1R7HunM3EwGTlDoRm9SnufJsdUV3dZH/SY27Wa38V9qqwtKyl5YTbzl0z\noATuqRzt/QWpczqECmVuZHN0cmVhbQplbmRvYmoKNTAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVE\nZWNvZGUgL0xlbmd0aCAxNzEgPj4Kc3RyZWFtCnicTZBNDkIhEIP3nKIXMKHzA4/zaFzp/bd28Pni\ngvRLIUOnwwMdR+JGR4bO6HiwyTEOvAsyJl6N85+M6ySOCeoVbcG6tDvuzSwxJywTI2BrlNybRxT4\n4ZgLQYLs8sMXGESka5hvNZ91k35+u9Nd1KV199MjCpzIjlAMG3AF2NM9DtwSzu+aJr9UKRmbOJQP\nVBeRstkJhailYpdTVWiM4lY974te7fkBwfY7+wplbmRzdHJlYW0KZW5kb2JqCjUxIDAgb2JqCjw8\nIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTcgPj4Kc3RyZWFtCnicMza0UDCAwxRDLgAa\nlALsCmVuZHN0cmVhbQplbmRvYmoKNTIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xl\nbmd0aCA2OCA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyA\ntCWcgohbQjRBlIJYEKVmJmYQSTgDIpcGAMm0FeUKZW5kc3RyZWFtCmVuZG9iago1MyAwIG9iago8\nPCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ5ID4+CnN0cmVhbQp4nDM2tFAwUDA0MAeS\nRoZAlpGJQoohF0gAxMzlggnmgFkGQBqiOAeuJocrDQDG6A0mCmVuZHN0cmVhbQplbmRvYmoKNTQg\nMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDcgPj4Kc3RyZWFtCnicTVG7\nbUQxDOvfFFzgAOtreZ4LUl32b0PJCJDCIKEvKaclFvbGSwzhB1sPvuSRVUN/Hj8x7DMsPcnk1D/m\nuclUFL4VqpuYUBdi4f1oBLwWdC8iK8oH349lDHPO9+CjEJdgJjRgrG9JJhfVvDNkwomhjsNBm1QY\nd00ULK4VzTPI7VY3sjqzIGx4JRPixgBEBNkXkM1go4yxlZDFch6oCpIFWmDX6RtRi4IrlNYJdKLW\nxLrM4Kvn9nY3Qy/y4Ki6eH0M60uwwuileyx8rkIfzPRMO3dJI73wphMRZg8FUpmdkZU6PWJ9t0D/\nn2Ur+PvJz/P9CxUoXCoKZW5kc3RyZWFtCmVuZG9iago1NSAwIG9iago8PCAvRmlsdGVyIC9GbGF0\nZURlY29kZSAvTGVuZ3RoIDMzOCA+PgpzdHJlYW0KeJw1Ujmu3UAM630KXSCAds2c5wWpfu7fhpRf\nCkO0VoqajhaVafllIVUtky6/7UltiRvy98kKiROSVyXapQyRUPk8hVS/Z8u8vtacESBLlQqTk5LH\nJQv+DJfeLhznY2s/jyN3PXpgVYyEEgHLFBOja1k6u8Oajfw8pgE/4hFyrli3HGMVSA26cdoV70Pz\necgaIGaYlooKXVaJFn5B8aBHrX33WFRYINHtHElwjI1QkYB2gdpIDDmzFruoL/pZlJgJdO2LIu6i\nwBJJzJxiXTr6Dz50LKi/NuPLr45K+kgra0zad6NJacwik66XRW83b309uEDzLsp/Xs0gQVPWKGl8\n0KqdYyiaGWWFdxyaDDTHHIfMEzyHMxKU9H0ofl9LJrookT8ODaF/Xx6jjJwGbwFz0Z+2igMX8dlh\nrxxghdLFmuR9QCoTemD6/9f4ef78Axy2gFQKZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iago8PCAv\nVHlwZSAvRm9udCAvRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMg\nXQovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvV2lkdGhzIDE5IDAgUiAvQ2hh\nclByb2NzIDIyIDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBb\nIDMyIC9zcGFjZSA0NiAvcGVyaW9kIDQ4IC96ZXJvIC9vbmUgL3R3byAvdGhyZWUgL2ZvdXIgL2Zp\ndmUgL3NpeCAvc2V2ZW4KL2VpZ2h0IC9uaW5lIDczIC9JIDc3IC9NIDgzIC9TIDg3IC9XIDk3IC9h\nIDEwMCAvZCAvZSAvZiAvZyAxMDUgL2kgMTA4IC9sIC9tCi9uIC9vIC9wIDExNCAvciAvcyAvdCAv\ndSAvdiBdCj4+Ci9Gb250RGVzY3JpcHRvciAyMCAwIFIgL0Jhc2VGb250IC9EZWphVnVTYW5zIC9T\ndWJ0eXBlIC9UeXBlMwovTmFtZSAvRGVqYVZ1U2FucyAvTGFzdENoYXIgMjU1ID4+CmVuZG9iagoy\nMCAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZsYWdzIDMyIC9Gb250TmFtZSAvRGVq\nYVZ1U2FucyAvSXRhbGljQW5nbGUgMAovU3RlbVYgMCAvRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3\nOTQgMTIzMyBdIC9NYXhXaWR0aCAxMzQyIC9DYXBIZWlnaHQgMAovQXNjZW50IDkyOSAvWEhlaWdo\ndCAwIC9EZXNjZW50IC0yMzYgPj4KZW5kb2JqCjE5IDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYw\nMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAw\nIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTgg\nNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAz\nMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgz\nOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1\nNiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjEx\nIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUg\nNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1\nOTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUx\nOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTgg\nMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYx\nMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2\nMSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYx\nMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMy\nIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4Mzgg\nNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5\nODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYx\nMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjIy\nIDAgb2JqCjw8IC9JIDQ2IDAgUiAvemVybyAyMyAwIFIgL3AgMjUgMCBSIC9zaXggMjYgMCBSIC9s\nIDI3IDAgUiAvZSAzMiAwIFIKL3IgMjkgMCBSIC9vbmUgMzMgMCBSIC9TIDMwIDAgUiAvbmluZSAz\nMSAwIFIgL2VpZ2h0IDI4IDAgUiAvcyA0NCAwIFIKL2QgMzQgMCBSIC9XIDM1IDAgUiAvZiAzNiAw\nIFIgL2ZvdXIgMzggMCBSIC9NIDM5IDAgUiAvYSA0MCAwIFIKL3NldmVuIDQxIDAgUiAvdHdvIDI0\nIDAgUiAvZyA0MyAwIFIgL28gNDUgMCBSIC92IDM3IDAgUiAvbSA0NyAwIFIKL3RocmVlIDU1IDAg\nUiAvbiA0OSAwIFIgL3UgNTAgMCBSIC9zcGFjZSA1MSAwIFIgL2kgNTIgMCBSIC9wZXJpb2QgNTMg\nMCBSCi9maXZlIDU0IDAgUiAvdCA0OCAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDIxIDAg\nUiAvRjIgMTYgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMiA8PCAvVHlwZSAvRXh0R1N0YXRl\nIC9DQSAxIC9jYSAxID4+Ci9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAwIC9jYSAxID4+ID4+\nCmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2Jq\nCjw8IC9NMSAxMyAwIFIgL00wIDEyIDAgUiAvRGVqYVZ1U2Fucy1taW51cyA0MiAwIFIKL0RlamFW\ndVNhbnMtT2JsaXF1ZS1sYW1iZGEgMTggMCBSID4+CmVuZG9iagoxMiAwIG9iago8PCAvVHlwZSAv\nWE9iamVjdCAvQkJveCBbIC00LjI1IC00LjI1IDQuMjUgNC4yNSBdIC9MZW5ndGggMTM1Ci9TdWJ0\neXBlIC9Gb3JtIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nG2QMQ6AIAxF956CC3wC\nSBFWR6/hYky8/6oSLAos0Jb+R3+tOsiole4Dk55ZnWR0Sp6tKwWrk4/e8ZNO7OMclNOBbbgrb7CR\nvEH6sxpCy6nJrU9Qy0Uouhda4R9kaW6p+dqo+Q29ED0d/Qj4zwiZHY0pDHxjsB4MtoiOj2JjJ1ro\nApL2Td8KZW5kc3RyZWFtCmVuZG9iagoxMyAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvQkJveCBb\nIC00LjI1IC00LjI1IDQuMjUgNC4yNSBdIC9MZW5ndGggMTM1Ci9TdWJ0eXBlIC9Gb3JtIC9GaWx0\nZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nG2QMQ6AIAxF956CC3wCSBFWR6/hYky8/6oSLAos\n0Jb+R3+tOsiole4Dk55ZnWR0Sp6tKwWrk4/e8ZNO7OMclNOBbbgrb7CRvEH6sxpCy6nJrU9Qy0Uo\nuhda4R9kaW6p+dqo+Q29ED0d/Qj4zwiZHY0pDHxjsB4MtoiOj2JjJ1roApL2Td8KZW5kc3RyZWFt\nCmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvQ291bnQgMSAvS2lkcyBbIDEwIDAgUiBd\nID4+CmVuZG9iago1NiAwIG9iago8PCAvQ3JlYXRvciAobWF0cGxvdGxpYiAyLjAuMCwgaHR0cDov\nL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKG1hdHBsb3RsaWIgcGRmIGJhY2tlbmQpIC9DcmVh\ndGlvbkRhdGUgKEQ6MjAxNzAzMzAxMDM5NDErMTEnMDAnKQo+PgplbmRvYmoKeHJlZgowIDU3CjAw\nMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDE1MjI3IDAwMDAwIG4g\nCjAwMDAwMTQ0MDEgMDAwMDAgbiAKMDAwMDAxNDQ0NCAwMDAwMCBuIAowMDAwMDE0NTQzIDAwMDAw\nIG4gCjAwMDAwMTQ1NjQgMDAwMDAgbiAKMDAwMDAxNDU4NSAwMDAwMCBuIAowMDAwMDAwMDY1IDAw\nMDAwIG4gCjAwMDAwMDAzOTggMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAxOTI3\nIDAwMDAwIG4gCjAwMDAwMTQ2ODcgMDAwMDAgbiAKMDAwMDAxNDk1NyAwMDAwMCBuIAowMDAwMDAy\nNzAxIDAwMDAwIG4gCjAwMDAwMDI0OTMgMDAwMDAgbiAKMDAwMDAwMjE4NCAwMDAwMCBuIAowMDAw\nMDAzNzU0IDAwMDAwIG4gCjAwMDAwMDE5NDggMDAwMDAgbiAKMDAwMDAxMjk2NyAwMDAwMCBuIAow\nMDAwMDEyNzY3IDAwMDAwIG4gCjAwMDAwMTIyOTkgMDAwMDAgbiAKMDAwMDAxNDAyMCAwMDAwMCBu\nIAowMDAwMDAzNzc2IDAwMDAwIG4gCjAwMDAwMDQwNTkgMDAwMDAgbiAKMDAwMDAwNDM4MCAwMDAw\nMCBuIAowMDAwMDA0Njg5IDAwMDAwIG4gCjAwMDAwMDUwNzkgMDAwMDAgbiAKMDAwMDAwNTE5NiAw\nMDAwMCBuIAowMDAwMDA1NjYxIDAwMDAwIG4gCjAwMDAwMDU4OTEgMDAwMDAgbiAKMDAwMDAwNjMw\nMiAwMDAwMCBuIAowMDAwMDA2Njk1IDAwMDAwIG4gCjAwMDAwMDcwMTMgMDAwMDAgbiAKMDAwMDAw\nNzE2NSAwMDAwMCBuIAowMDAwMDA3NDY1IDAwMDAwIG4gCjAwMDAwMDc2MjYgMDAwMDAgbiAKMDAw\nMDAwNzgzMiAwMDAwMCBuIAowMDAwMDA3OTc2IDAwMDAwIG4gCjAwMDAwMDgxMzggMDAwMDAgbiAK\nMDAwMDAwODI5NyAwMDAwMCBuIAowMDAwMDA4Njc0IDAwMDAwIG4gCjAwMDAwMDg4MTQgMDAwMDAg\nbiAKMDAwMDAwODk4NCAwMDAwMCBuIAowMDAwMDA5Mzk1IDAwMDAwIG4gCjAwMDAwMDk4MDAgMDAw\nMDAgbiAKMDAwMDAxMDA4NyAwMDAwMCBuIAowMDAwMDEwMjA4IDAwMDAwIG4gCjAwMDAwMTA1MzYg\nMDAwMDAgbiAKMDAwMDAxMDc0MCAwMDAwMCBuIAowMDAwMDEwOTc0IDAwMDAwIG4gCjAwMDAwMTEy\nMTggMDAwMDAgbiAKMDAwMDAxMTMwNyAwMDAwMCBuIAowMDAwMDExNDQ3IDAwMDAwIG4gCjAwMDAw\nMTE1NjggMDAwMDAgbiAKMDAwMDAxMTg4OCAwMDAwMCBuIAowMDAwMDE1Mjg3IDAwMDAwIG4gCnRy\nYWlsZXIKPDwgL1NpemUgNTcgL0luZm8gNTYgMCBSIC9Sb290IDEgMCBSID4+CnN0YXJ0eHJlZgox\nNTQzNQolJUVPRgo=\n", "image/svg+xml": [ "\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "figure4();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the left panel in this figure it is clear that there is a single active subspace.\n", "\n", "From the right panel we see that it is the second dimension." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Figure 5" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example we modify the function used to produce figure 4 by multiplying it by $x_1$ and $x_2$. This effectively makes the function have a 3 dimensional active subspace.\n", "\n", "This example is slightly different than what S&B present in that I plot the maximum absolute error on the 1,000 random testing points instead of what they refer to as the average error." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def f5(x):\n", " return x[1]*x[2] * f4(x)\n", "\n", "def df5(x):\n", " _f4 = f4(x)\n", " val = np.atleast_1d(x[1] * x[2] * _f4)\n", " coefs = np.array([0.01, 0.7, 0.02, 0.03, 0.04, 0.05, 0.06, 0.08, 0.09, 0.1])\n", " out = val[:, None] * coefs[None, :]\n", " out[:, 1] += x[2] * _f4\n", " out[:, 2] += x[1] * _f4\n", " return out" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def figure5():\n", " np.random.seed(42)\n", " Nvals = np.array([10, 30, 100, 250, 500, 1000])\n", " num_Ns = len(Nvals)\n", " max_errs = np.inf * np.ones((num_Ns, 3))\n", " max_errs_gp = np.inf * np.ones(num_Ns)\n", " \n", " # construct test points\n", " X_test = randOmega(1000, 10)\n", " f_test = f5(X_test.T)\n", "\n", " for i, N in enumerate(Nvals):\n", " # training points\n", " X = randOmega(int(N), 10)\n", " V = f5(X.T)\n", " G = df5(X.T)\n", " \n", " CN = (G.T @ G) / N\n", " vals, vecs = linalg.eigh(CN)\n", " for d in range(3, 0, -1):\n", "\n", " # find active subspace of dimension d\n", " W = vecs[:, -d:]\n", " Y = X @ W\n", "\n", " # fit GP on active subspace\n", " gp_as = GaussianProcessRegressor(RBF(), n_restarts_optimizer=0)\n", " gp_as.fit(Y.reshape(N, d), V)\n", " m_tilde = gp_as.predict((X_test @ W).reshape(1000, d))\n", " max_errs[i, d-1] = np.max(np.abs(f_test - m_tilde))\n", " \n", " gp = GaussianProcessRegressor(RBF())\n", " gp.fit(X, V)\n", " m_tilde = gp.predict(X_test)\n", " max_errs_gp[i] = np.max(np.abs(f_test - m_tilde))\n", " \n", " fig, ax = plt.subplots(1, 2, figsize=(9, 5))\n", " x = np.arange(1, 11)\n", " ax[0].semilogy(x, vals, \".\", ms=15)\n", " ax[0].set_xlabel(\"Sorted eigenvalues\")\n", " ax[0].set_xticks(x)\n", " ax[0].set_ylabel(\"$\\lambda$\")\n", " \n", " ax[1].semilogy(Nvals, max_errs, \".-\", ms=15)\n", " ax[1].semilogy(Nvals, max_errs_gp, \".-\", ms=15)\n", " ax[1].legend([str(i) + \"d AS\" for i in range(1, 4)] + [\"Full GP\"])\n", " ax[1].set_xlabel(\"# of points\")\n", "\n", " fig.tight_layout()\n", " return fig, max_errs" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 3, 'warnflag': 2, 'grad': array([-0.00025588]), 'funcalls': 49}\n", " \" state: %s\" % convergence_dict)\n", "/home/john/anaconda3/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py:427: UserWarning: fmin_l_bfgs_b terminated abnormally with the state: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'nit': 5, 'warnflag': 2, 'grad': array([ -7.00950623e-05]), 'funcalls': 50}\n", " \" state: %s\" % convergence_dict)\n" ] }, { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+\nCmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9TaGFkaW5nIDYgMCBSIC9FeHRHU3RhdGUg\nNCAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1hP\nYmplY3QgNyAwIFIKL1BhdHRlcm4gNSAwIFIgPj4KZW5kb2JqCjEwIDAgb2JqCjw8IC9UeXBlIC9Q\nYWdlIC9Bbm5vdHMgWyBdCi9Hcm91cCA8PCAvVHlwZSAvR3JvdXAgL1MgL1RyYW5zcGFyZW5jeSAv\nQ1MgL0RldmljZVJHQiA+PiAvUmVzb3VyY2VzIDggMCBSCi9NZWRpYUJveCBbIDAgMCA2MzcuMzIz\nNDE3OTY4NyAzNTIuMTQzNzUgXSAvQ29udGVudHMgOSAwIFIgL1BhcmVudCAyIDAgUiA+PgplbmRv\nYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDExIDAgUiA+PgpzdHJl\nYW0KeJzFW12PFbkRfe9f0VJeNg9jXLbLH4+gzSKtFCUsaPMQ5YEdBgKagQDL8vdzqvteu9zd904z\nYSaLVsyccblc5VNfvgON74ZHj2l883m04zv8/3X85/gv/P1qpPHp+OjHqz/eXl798vTJePl5sMBv\nhuiT8c4Hyvj2Wn/r2RkKPjFg233372F4P0ALJJ5i4zfDwGxyooyfQp6nZTeDp2w8U8qpodcdGsiE\nA6x2UCg0vYZFbrboDZTBKpOVXXIE/GSIyWTrUtYHaFgw9rDj8OS4H41fhycvxkc/0Uh2fPF6iMFE\n521xXLiMzpnC44tXww/05/HFu/EvL+pB5ABDCcYTF+u1RgXuUlnIkI+O2HpySqXbVEkOf7tiXedm\nje5SSpRMDMX7FGJkpdVvaw2gRHD4q9Oq0H1aQzAUXeBSLNjVtIZtrcmbzOx6pQ3cpzNZExNzdjBW\nqeRNlc5a42P2xWmdGt2ntCRDOScbUnJeaY3bWl00ORNTd6ka3aXVOW8SHOuSsykqrWlbK3vjS0C4\nd1oVuk8rWwMOlFBicDpm8rbWVEyxCefTShu4T2eKJrlsxdCsg6Zs6vQ2muBszt2tanSXVg8a2IhA\nhZO9DhqyR7U9C3wGVQNuhBkxUgzyELlZ5PmHT79fvRqv3r65ev/Hy+svV583T95SIhJZYstw0c2A\nPOyX6PLkH5Gd7Xhh8Y0kh8Nyb1wu0Wfcchwvb3BeqQGmpMhxPlk7/pz0hgjqIyWGKBvY6fCTuUg6\nUiYcEsCUspGAjtbJHkltEUyAPvKy4LDFRI4B5n6EemvTKKVo/gr7ZifJkMeA5D0rxllRut69/PXL\n85fvP1/cvH3/5fP444fhGf6c8BgRIT0hX/QuU/Aun5FFlgPVSmaIKafZGOzDOo13OQ2blnn7uziN\ngyleSNI7rcH7nIakEBIVRgID9f+fTHMPwLSSDbuAOOyd1uBzTgttG6Sh6ImKLewewmnl/n2DAxgi\ndtT7RsG7fOOQuCMHFAqk6fwQvokP4JvMKNgotQvfNHifbzIZNFXBF7giPIRv/P37xjvUy5LY975R\n8BnfeNu2wSl8RMdZfCz/q2/QMHCGlfmQZzdcY4+ueYYDyVku5FQUzMTckTL2mL6aj3L0IR1dOH1h\n5sqycNvffrt++/HL1cX1y5vfXr08+u/juDF5ueiQf0shR7YE+MObMn66Gv8xvh+rw/FDjjm6xAM0\nxsN/uEywD4nbYd6D0Jtu5Xh65S9Ph50rP705XJb459jQx5JxqYzizMnn6OPUoYyP/mrF0uNq9MRM\nYJcLntFYo8FCL5PQ4Md862oU/sBcXMzh9rXI3XAik/M5pVuXezbRY+gLGVS7/djGevAI/fJ0ycfF\nuM3x4+jGn+fJvZtx+7l9c9beHp+H55tD+M2pIRzrv2GS71fXbc7tXh8nfh7n94Ov7Q3BY/4muMh2\n00GE9+LhQaKp1qjSoLe40yuCBx9jDNSPnRrdNyxg7gwZ6guXpIcFu9noI92ajBCwQWvV6C6tQbK3\ntwn9LSJKvybYE3pxTJs4ZOr0KnSf3mgNGi9cRulHz3BCL4MfyJ6EaNEUVuguvWyzSRjiSkTC1ON9\nPKU3Iv9mzPIdwzS6Ty9HUzDcJ48ORr9k5BN6IypALMgn3VuRRve9T1lC5fIpupy6YZus3R5Cg7yA\niYOoEMaXbgj90/jh9fifD2/f/749fapYkgejhLIeZC7wuO24ws/VY+/njTgzyhER5lAkP4uxyj7I\nHHpvQ4BykbMoAslFot5FCt/vIoeLTkQRd535AafOexgutYsKsijCdWp4tYsaftZFpXNRxrdIFr6E\nUvhhW7tqYOwNRFpAA0QuT/a5NTyZpwQSSWMkj1idQIOXAtkhY8T5HUgJNHgpUDAcOEIR6QUavBLI\ncKs8W/brj+hyOVlkLnjcLgQUvhbBzaVsXViKVHwlgpQRU8p5KdLwlYi8NufAZXmwhq9EuMhjsQtp\nIdLwlYi8L8doAy1EGr4WgZkFwb08WMNXIhgb0FjHwguRhq9ECtpMWImppxdp+Fok4csU8kqk4ksR\nyVMxJB8WIgpfiXis4mB5KdLwlUiQjytCooXHFL4S4SwtOdvFVSp8JYJSSdJILA/W8JVIwpdI7XFx\n+wpfiWSZ8tABLJis8LUIG9CiMC9FKr4WAWMpZlqJVHwp4p2MbxjQFiIKX4nI3IO2Ii08pvBJROaa\nKjZn728ZVWUkIsNItbtH0cXAhHY8IhuFgCLs0TUhlP0csfB6RgHBSRWOMQKUT4irKPazQadSnDAm\noEyjAGKQ69eHEiR1edfDusHTOOYgGYB2z8tSCvdP7Gq6rnZbsEQfwqG7n2dP0oNqkErPyKjEUh/X\nCyjKuFdysmhXMcmiE8aFYmbf2s1LdgoY0HNEUru4ZTX6ASxFq4qpKW8rBxUZvCouy1Sulzz7biQb\nSZyLocvjolmcazkUS5gx8i3Mwm1j5ktzAtnDLM2g78osGk/bcKDT+SVnOHS0Eu0AH55G3EkOZQMF\nEXRzxW2s7elkN1b0HNpasckbdztv3PfljSQneGT2obSGKAJsp+G9+8ktFGKTvS1zu9Yo5FJCv+YT\nxZ5CzqFnEFeHnkKEDhIte5jWNwpJh4FxLbqeQfXVrUtNe4xpqaliw7nV52h1tBzg4Q3Nn6TVhUcZ\nt+Cg8zHxxuqeWBcBPORoLUnzv7F8kak4menjbNDcb6zuGXchHZX3ln3h6RPC1Vl6+l04jOQkr9iS\ny9X678rFPMU1NHi5A4Qf6IV2fPpmqiEBc+UthRLdlbdEYZHOQBw0ENnnRTpD+y5eC67n4tSm5OIj\nd1x06JAzoR6EnozSbqUSkw2ajbvMqWxsq4dzq88VyoPt8g59IEw4zUaZkCmgLoNyW6uXbDQY6rmw\nfKy8sXpZNoNJaL5TTJ7zxvIFG+FJn+SVByK0dZYFGz3aTpysJBv1Yfa/Rm+/2p54iMWFbr3n3px6\nz5X36294FV4sVy/Sp/d/9NjrF+kDg+qbNDLpYbvg2gupF9rnDhXSI79aAvsaKthxpTeIXbTFHRgO\nCy8HhZJ1xk0vLtc97A3HgNgblS4ZtefF6lgNvNQ2NFjeygtoawk5rsEC1rVVW4e6tnGDqx3XHVpN\nVsqqb7Z8eylv9fUJaMpo4N/cmdW3+t2fSC2Ymnx9z0CMHYhaVuC9N+hIp02pMz7PJ8lgLhItvHUE\nPXKnLVS8ICmjBoQJSwUJBLIerRjqhFzF9MkikrIbJcXKQ7yII/GUWQ2SsFS03IEHg2fxI4hxF0Mf\nuVHviQKKeuS4U59xpkKzfD1ogQxaFYSfsgeXP9vT7D5AIiuTYy5InW0dmFHYZur2k45Vfp2q04ym\nJmUpDOqMwPzKGgU2uxXYPNS2VK5syrXX2znVBTV76j2u7rsnuj38Lq18BNU/4FvS7QCko5x8evR/\nNT5+vni5V2PMmf5+VfdVZMCiMD+gdrHR4O84ZOg4QIeLYnrQ25jT4EaxlA3R1CYqMgLM5EtaBAOu\nktGCatbn6ZeTSEYnBVb7umDAVYZoSX4dsu2JQwcKtosFOTH0pD4Y8twnWNLBgJ+6o0nNvxXsAkKv\nrVTTu1ZWdgeoBG5HbURXNilQ26/g5qu6Z3NpVd47vx5UX1Q1Sd/oxu3fLS6SdOzHZ/8f3NnIuOug\npoIkxsMvdHQxUtF7H6N04ER/+Oiwi5uKNjICQoeYU1dCpm7JUR80AHNg77oKEkUGmbirINXkLmqk\nhc+SDHXUxOmzWkpd2KArCIFDFzQYDEAVNGw6aAAemgZFGlubBs2vtrIRUW1ZSat0V26rQ7YgUOYo\nUFmu0OaktqfyZlOu/F6PqW+o2qNucn3nd4sW+UXgciwj/pZgudskqYJF/t3I/KFhFy0Nvvc5T4dL\nyMdfJuvipcGNiMBkti1dnQlgTbLOLeoMS4j60EUMw+kuYYTswGp2FzLs5S3f567QsHyGJv+UpKln\nmQan13gdMsw4VEzc1Rn1qznN/AZ2QaPXVjrqXStxuwNUjteTtlBQFilQW6/g5qm2ZfNoVd47vx5U\nX1Q1Sd/oxu3fLXIgH+fzS+j89OX6enz692PwPBv+Cw+Q580KZW5kc3RyZWFtCmVuZG9iagoxMSAw\nIG9iagozMDUzCmVuZG9iagoyMSAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvQkJveCBbIC0xMDE2\nIC0zNTEgMTY2MCAxMDY4IF0gL0xlbmd0aCAxMDIKL1N1YnR5cGUgL0Zvcm0gL0ZpbHRlciAvRmxh\ndGVEZWNvZGUgPj4Kc3RyZWFtCnicNY47CsRADEN7n0IXCIzl75wnkCq5f7uTCVsIPSSwLJaFLMcj\n3o2BW2za59rw4YtCd3D4V3AWgm/BHkjqS2XIJBhEtoFa20/RGptu0Q5U7gtOVMw9srIo2FJx4v/O\nKdcP7ssdngplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9GaXJzdENo\nYXIgMCAvRm9udEJCb3ggWyAtMTAxNiAtMzUxIDE2NjAgMTA2OCBdCi9Gb250TWF0cml4IFsgMC4w\nMDEgMCAwIDAuMDAxIDAgMCBdIC9XaWR0aHMgMTcgMCBSIC9DaGFyUHJvY3MgMjAgMCBSCi9FbmNv\nZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcgL0RpZmZlcmVuY2VzIFsgXSA+PiAvRm9udERlc2NyaXB0\nb3IgMTggMCBSCi9CYXNlRm9udCAvRGVqYVZ1U2Fucy1PYmxpcXVlIC9TdWJ0eXBlIC9UeXBlMyAv\nTmFtZSAvRGVqYVZ1U2Fucy1PYmxpcXVlCi9MYXN0Q2hhciAyNTUgPj4KZW5kb2JqCjE4IDAgb2Jq\nCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRmxhZ3MgOTYgL0ZvbnROYW1lIC9EZWphVnVTYW5z\nLU9ibGlxdWUKL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL0ZvbnRCQm94IFsgLTEwMTYgLTM1MSAx\nNjYwIDEwNjggXSAvTWF4V2lkdGggMTM1MAovQ2FwSGVpZ2h0IDAgL0FzY2VudCA5MjkgL1hIZWln\naHQgMCAvRGVzY2VudCAtMjM2ID4+CmVuZG9iagoxNyAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2\nMDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYw\nMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4\nIDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTgg\nMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4\nMzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2\nNTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYx\nMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1\nIDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQg\nNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1\nMTggMTAwMCA1MDAgNTAwIDUwMCAxMzUwIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4\nIDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjggNjAwIDUyNSA2\nMTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjE3IDgzOCAz\nNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2\nMTcgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYz\nMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4\nIDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwOAo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMg\nOTk1IDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2\nMTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoy\nMCAwIG9iago8PCA+PgplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xl\nbmd0aCAyMTAgPj4Kc3RyZWFtCnicNVDLDUMxCLtnChaoFAKBZJ5WvXX/a23QO2ER/0JYyJQIeanJ\nzinpSz46TA+2Lr+xIgutdSXsypognivvoZmysdHY4mBwGiZegBY3YOhpjRo1dOGCpi6VQoHFJfCZ\nfHV76L5PGXhqGXJ2BBFDyWAJaroWTVi0PJ+QTgHi/37D7i3koZLzyp4b+Ruc7fA7s27hJ2p2ItFy\nFTLUszTHGAgTRR48eUWmcOKz1nfVNBLUZgtOlgGuTj+MDgBgIl5ZgOyuRDlL0o6ln2+8x/cPQABT\ntAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5n\ndGggOTAgPj4Kc3RyZWFtCnicTY1BEsAgCAPvvCJPUETQ/3R60v9fq9QOvcBOAokWRYL0NWpLMO64\nMhVrUCmYlJfAVTBcC9ruosr+MklMnYbTe7cDg7LxcYPSSfv2cXoAq/16Bt0P0hwiWAplbmRzdHJl\nYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ0ID4+\nCnN0cmVhbQp4nE1RSW7EMAy7+xX8wACWrMV5T4qe2v9fSzoYTA+GGFniEncnJrLwsoW6FjocXzby\nStgM/B605sTPiBX/0TYirXxQbYFMxESR0EXdaNwj3fAqRPuRCl6p3mPNR9ytNG/p2MX+3gfTmkeq\nrArxkGNzZCNpXRNFx/fQes8N0SmDBPrchBwQpTl6FZL+2g/XtdHs1trMnqgOHpkSyifNamhOp5gh\ns6Eet6lYySjcqDIq7lMZaT3I2zjLAE6OE8poPdmxmojyU59cQl5MxAm77OxYiE0sbBzSflT6xPLz\nrTfjb6a1lkLGBYnQFJfez3iP7z/dWFu0CmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0Zp\nbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDcgPj4Kc3RyZWFtCnicTVG7bUQxDOvfFFzgAOtr\neZ4LUl32b0PJCJDCIKEvKaclFvbGSwzhB1sPvuSRVUN/Hj8x7DMsPcnk1D/muclUFL4VqpuYUBdi\n4f1oBLwWdC8iK8oH349lDHPO9+CjEJdgJjRgrG9JJhfVvDNkwomhjsNBm1QYd00ULK4VzTPI7VY3\nsjqzIGx4JRPixgBEBNkXkM1go4yxlZDFch6oCpIFWmDX6RtRi4IrlNYJdKLWxLrM4Kvn9nY3Qy/y\n4Ki6eH0M60uwwuileyx8rkIfzPRMO3dJI73wphMRZg8FUpmdkZU6PWJ9t0D/n2Ur+PvJz/P9CxUo\nXCoKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVu\nZ3RoIDQ1ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp4G\nAJ99DLUKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAv\nTGVuZ3RoIDM5MiA+PgpzdHJlYW0KeJw9UktuBTEI288puECl8E1ynqne7t1/W5vMVKoKLwO2MZSX\nDKklP+qSiDNMfvVyXeJR8r1samfmIe4uNqb4WHJfuobYctGaYrFPHMkvyLRUWKFW3aND8YUoEw8A\nLeCBBeG+HP/xF6jB17CFcsN7ZAJgStRuQMZD0RlIWUERYfuRFeikUK9s4e8oIFfUrIWhdGKIDZYA\nKb6rDYmYqNmgh4SVkqod0vGMpPBbwV2JYVBbW9sEeGbQENnekY0RM+3RGXFZEWs/PemjUTK1URkP\nTWd88d0yUvPRFeik0sjdykNnz0InYCTmSZjncCPhnttBCzH0ca+WT2z3mClWkfAFO8oBA7393pKN\nz3vgLIxc2+xMJ/DRaaccE62+HmL9gz9sS5tcxyuHRRSovCgIftdBE3F8WMX3ZKNEd7QB1iMT1Wgl\nEAwSws7tMPJ4xnnZ3hW05vREaKNEHtSOET0ossXlnBWwp/yszbEcng8me2+0j5TMzKiEFdR2eqi2\nz2Md1Hee+/r8AS4AoRkKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0\nZURlY29kZSAvTGVuZ3RoIDE1NyA+PgpzdHJlYW0KeJxFkLkRQzEIRHNVQQkSsAjqscfRd/+pF/lK\ntG8ALYevJVOqHyciptzXaPQweQ6fTSVWLNgmtpMachsWQUoxmHhOMaujt6GZh9TruKiquHVmldNp\ny8rFf/NoVzOTPcI16ifwTej4nzy0qehboK8LlH1AtTidSVAxfa9igaOcdn8inBjgPhlHmSkjcWJu\nCuz3GQBmvle4xuMF3QE3eQplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9GaWx0ZXIgL0Zs\nYXRlRGVjb2RlIC9MZW5ndGggMzM4ID4+CnN0cmVhbQp4nDVSS5JbQQjbv1PoAq5q/s15nJrV5P7b\nCOysoIEWEpAWOMjESwxRjXLFH3mC8TqBv+vlafw+3oXUgqci/cC1aRvvx5o1UbA0YinMPvb9KCHH\nU+PfEOi5SBNmZDJyIBmI+7U+f9abTDn8BqRpc/ooSXoQLdjdGnZ8WZBB0pMaluzkh3UtsLoITZgb\nayIZObUyNc/HnuEynhgjQdUsIEmfuE8VjEgzHjtnLXmQ4XiqFy9+vY3XMo+pl1UFMrYJ5mA7mQmn\nKCIQv6AkuYm7aOoojmbGmtuFhpIi9909nJz0ur+cRAVeCeEs1hKOGXrKMic7DUqgauUEmGG99oVx\nmjZKuFPT7V2xr99nJmHc5rCzUjINznFwL5vMESR73TFhEx6HmPfuEYzEvPldbBFcucy5JtOP/Sja\nSB8U1+dcTZmtKOEfquSJFdf4//zez88/kDd9sQplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8\nIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTcgPj4Kc3RyZWFtCnicMza0UDCAwxRDLgAa\nlALsCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xl\nbmd0aCAyNDUgPj4Kc3RyZWFtCnicRVC7jUMxDOs9BRcIYP0se553SJXbvz1KRnCFIVo/kloSmIjA\nSwyxlG/iR0ZBPQu/F4XiM8TPF4VBzoSkQJz1GRCZeIbaRm7odnDOvMMzjDkCF8VacKbTmfZc2OSc\nBycQzm2U8YxCuklUFXFUn3FM8aqyz43XgaW1bLPTkewhjYRLSSUml35TKv+0KVsq6NpFE7BI5IGT\nTTThLD9DkmLMoJRR9zC1jvRxspFHddDJ2Zw5LZnZ7qftTHwPWCaZUeUpnecyPiep81xOfe6zHdHk\noqVV+5z93pGW8iK126HV6VclUZmN1aeQuDz/jJ/x/gOOoFk+CmVuZHN0cmVhbQplbmRvYmoKMzYg\nMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA4MCA+PgpzdHJlYW0KeJxFjLsN\nwDAIRHumYAR+JmafKJWzfxsgStxwT7p7uDoSMlPeYYaHBJ4MLIZT8QaZo2A1uEZSjZ3so7BuX3WB\n5npTq/X3BypPdnZxPc3LGfQKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvRmlsdGVyIC9G\nbGF0ZURlY29kZSAvTGVuZ3RoIDIyNyA+PgpzdHJlYW0KeJw1TzuyAyEM6zmFLpAZjG1gz7OZVC/3\nb59ksg0S/kjy9ERHJl7myAis2fG2FhmIGfgWU/GvPe3DhOo9uIcI5eJCmGEknDXruJun48W/XeUz\n1sG7Db5ilhcEtjCT9ZXFmct2wVgaJ3FOshtj10RsY13r6RTWEUwoAyGd7TAlyBwVKX2yo4w5Ok7k\niediqsUuv+9hfcGmMaLCHFcFT9BkUJY97yagHRf039WN30k0i14CMpFgYZ0k5s5ZTvjVa0fHUYsi\nMSekGeQyEdKcrmIKoQnFOjsKKhUFl+pzyt0+/2hdW00KZW5kc3RyZWFtCmVuZG9iagozOCAwIG9i\nago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxNj0ESwzAI\nA+9+hZ6AsQHznnR6Sv5/LZA27gXtjICRhjAIPGIM6zAlvHr74VWkS3A2jvklGUU8CGoL3BdUBUdj\nip342N2h7KXi6RRNi+sRc9O0pHQ3USptvZ3I+MB9n94fVbYknYIeW+qELtEk8kUCc9hUMM/qxktL\nj6ft2d4fZj4z1wplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVj\nb2RlIC9MZW5ndGggMjQ4ID4+CnN0cmVhbQp4nC1ROZIDQQjL5xV6QnPT77HLkff/6QrKAYOGQyA6\nLXFQxk8Qlive8shVtOHvmRjBd8Gh38p1GxY5EBVI0hhUTahdvB69B3YcZgLzpDUsgxnrAz9jCjd6\ncXhMxtntdRk1BHvXa09mUDIrF3HJxAVTddjImcNPpowL7VzPDci5EdZlGKSblcaMhCNNIVJIoeom\nqTNBkASjq1GjjRzFfunLI51hVSNqDPtcS9vXcxPOGjQ7Fqs8OaVHV5zLycULKwf9vM3ARVQaqzwQ\nEnC/20P9nOzkN97SubPF9Phec7K8MBVY8ea1G5BNtfg3L+L4PePr+fwDqKVbFgplbmRzdHJlYW0K\nZW5kb2JqCjQwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzIgPj4Kc3Ry\nZWFtCnicNYyxEcAwCAN7ptAINlhg75NLRfZvQ3xOAy8dD5eiwVoNuoIjcHWp/NEjXbkpRZdjzoLh\ncapfSDFGPagj497HT7lfcBYSfQplbmRzdHJlYW0KZW5kb2JqCjQxIDAgb2JqCjw8IC9GaWx0ZXIg\nL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQ4ID4+CnN0cmVhbQp4nE2QORYEIQhEc0/BEVhE8D7zJnLu\nnw4u2J3o14Iq1MUBwWwujcDI4UNFiaBWhF+R1heNIuzAUThKNTqUdXEXPrM5OtxASacqmOTbepSm\nPQn7UWfwQ9tPOyYpnjRDfRPyyqBD1RHWJHFau/JRJJI2caRPjU4t3+6wfWBlcO1JdxYSedGeWe47\n5L4tf2OU7x/FgEPlCmVuZHN0cmVhbQplbmRvYmoKNDIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVE\nZWNvZGUgL0xlbmd0aCA3NCA+PgpzdHJlYW0KeJwzNTdVMFCwtAASpobmCuZGlgophlxAPoiVywUT\nywGzzEzMgCxDS2SWibEhkGViYYbEMjaxgMoiWAZAGmxNDsz0HK40AANxGJMKZW5kc3RyZWFtCmVu\nZG9iago0MyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMwNCA+PgpzdHJl\nYW0KeJw9kjuSwzAMQ3udghfIjPiT5PNkJ5X3/u0+MslWgEmJACgvdZmypjwgaSYJ/9Hh4WI75XfY\nns3MwLVELxPLKc+hK8TcRfmymY26sjrFqsMwnVv0qJyLhk2TmucqSxm3C57DtYnnln3EDzc0qAd1\njUvCDd3VaFkKzXB1/zu9R9l3NTwXm1Tq1BePF1EV5vkhT6KH6UrifDwoIVx7MEYWEuRT0UCOs1yt\n8l5C9g63GrLCQWpJ57MnPNh1ek8ubhfNEA9kuVT4TlHs7dAzvuxKCT0StuFY7n07mrHpGps47H7v\nRtbKjK5oIX7IVyfrJWDcUyZFEmROtlhui9We7qEopnOGcxkg6tmKhlLmYlerfww7bywv2SzIlMwL\nMkanTZ44eMh+jZr0eZXneP0BbPNzOwplbmRzdHJlYW0KZW5kb2JqCjQ0IDAgb2JqCjw8IC9GaWx0\nZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNjggPj4Kc3RyZWFtCnicMzM2UzBQsDACEqamhgrmRpYK\nKYZcQD6IlcsFE8sBs8wszIEsIwuQlhwuQwtjMG1ibKRgZmIGZFkgMSC60gBy+BKRCmVuZHN0cmVh\nbQplbmRvYmoKNDUgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL0JCb3ggWyAtMTAyMSAtNDYzIDE3\nOTQgMTIzMyBdIC9MZW5ndGggMzcgL1N1YnR5cGUgL0Zvcm0KL0ZpbHRlciAvRmxhdGVEZWNvZGUg\nPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWTTAAFfCgoKZW5kc3RyZWFt\nCmVuZG9iago0NiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMzOCA+Pgpz\ndHJlYW0KeJxFUktyxTAI2+cUXCAz5mfj87xOV+n9t5VwOt089AwICTI9ZUim3DaWZITkHPKlV2SI\n1ZCfRo5ExBDfKaHArvK5vJbEXMhuiUrxoR0/l6U3Ms2u0Kq3R6c2i0Y1KyPnIEOEelbozO5R22TD\n63Yh6TpTFodwLP9DBbKUdcoplARtQd/YI+hvFjwR3Aaz5nKzuUxu9b/uWwue1zpbsW0HQAmWc95g\nBgDEwwnaAMTc2t4WKSgfVbqKScKt8lwnO1C20Kp0vDeAGQcYOWDDkq0O12hvAMM+D/SiRsX2FaCo\nLCD+ztlmwd4xyUiwJ+YGTj1xOsWRcEk4xgJAiq3iFLrxHdjiLxeuiJrwCXU6ZU28wp7a4sdCkwjv\nUnEC8CIbbl0dRbVsT+cJtD8qkjNipB7E0QmR1JLOERSXBvXQGvu4iRmvjcTmnr7dP8I5n+v7Fxa4\ng+AKZW5kc3RyZWFtCmVuZG9iago0NyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVu\nZ3RoIDMzMiA+PgpzdHJlYW0KeJwtUjmOJDEMy/0KfmAA6/Lxnh5M1Pv/dElVBQWqbMs85HLDRCV+\nLJDbUWvi10ZmoMLwr6vMhe9I28g6iGvIRVzJlsJnRCzkMcQ8xILv2/gZHvmszMmzB8Yv2fcZVuyp\nCctCxosztMMqjsMqyLFg6yKqe3hTpMOpJNjji/8+xXMXgha+I2jAL/nnqyN4vqRF2j1m27RbD5Zp\nR5UUloPtac7L5EvrLFfH4/kg2d4VO0JqV4CiMHfGeS6OMm1lRGthZ4OkxsX25tiPpQRd6MZlpDgC\n+ZkqwgNKmsxsoiD+yOkhpzIQpq7pSie3URV36slcs7m8nUkyW/dFis0UzuvCmfV3mDKrzTt5lhOl\nTkX4GXu2BA2d4+rZa5mFRrc5wSslfDZ2enLyvZpZD8mpSEgV07oKTqPIFEvYlviaiprS1Mvw35f3\nGX//ATPifAEKZW5kc3RyZWFtCmVuZG9iago0OCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29k\nZSAvTGVuZ3RoIDE2NSA+PgpzdHJlYW0KeJxFjzsSAyEMQ3tOoSOAf8B5NpOKvX8byztJGiyMJT+i\nKzr2ysNjY8rGa7SxJ8IW7lI6DaepjK+Kp2ddoMthMlNvXM1cYKqwyG5OsfoYfKHy7OaEr1WeGMKX\nTAkNMJU7uYf1+lGclnwkuwuC6pSNysIwR9L3QExNm6eFgfyPL3lqQVCZ5o1rlpVHaxVTNGeYKlsL\ngvWBoDql+gPT/0ynvT/c9D2ACmVuZHN0cmVhbQplbmRvYmoKNDkgMCBvYmoKPDwgL0ZpbHRlciAv\nRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzYgPj4Kc3RyZWFtCnicTVBLbkQhDNtzilzgSSQhAc5D1VXn\n/tuxw1TtKoYYf0gP6bJVHutTYnWJ7PKlTZfKMnkVqOVP2/9RDAJu/9DIQbS3jJ1i5hLWxcIkPOU0\nIxsn1ywfjztPG2aFxsSN450uGWCfFgE1W5XNgTltOjdAupAat6qz3mRQDCLqQs0Hky6cp9GXiDme\nqGBKdya1kBtcPtWhA3FavQq5Y4uTb8QcWaHAYdBMcdZfAdaoybJZyCBJhiHOfaN7lAqNqMp5KxXC\nD5OhEfWG1aAGlbmFoqnlkvwd2gIwBbaMdekMSoGqAMHfKqd9vwEkjV1TCmVuZHN0cmVhbQplbmRv\nYmoKNTAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMjAgPj4Kc3RyZWFt\nCnicNVG7ccUwDOs1BRfwnfiVNI9zr8rbvw1AOxVhGgRAqrxkSrlc6pJVssLkR4fqFE35PmCm/A71\nkOPoHtkhulPWlnsYCMvEPKWOWE2We7gFgS8MTYm5hfP3COgrBqMwE4G6xd8/QLMkMGlw8FOQa61a\nYokOPCwWWLMrzK0aKVTIVXw7NrkHBXJxs9CnHJoUt9yC8GWIZEdqsa/LZSnyu/UJGIQV5ohPFImF\n54EOZiLxJwNie/bZYldXL6oRGdZJhwdSBNJsbhIwNEWy6oMb2FfHNT9PR9nByUG/isH4NjiZL0l5\nXwWhEI8X/g7P2cixkkMkFPJ9tcCII2yAEaFP7SMQZSA0RffumVI+JlWK7wBGIRx9qlcyvBeR2WqG\nzf8ZXdkqCgZVWR+fRnAmg0k482SjCtNStdO/+9zj8wdjY3qACmVuZHN0cmVhbQplbmRvYmoKNTEg\nMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzEgPj4Kc3RyZWFtCnicRY/L\nDQQhDEPvVOES8hk+qYfVntj+r+swmkFC+EEiO/EwCKzz8jbQxfDRosM3/jbVq2OVLB+6elJWD+mQ\nh7zyFVBpMFHEhVlMHUNhzpjKyJYytxvhtk2DrGyVVK2DdjwGD7anZasIfqltYeos8QzCVV64xw0/\nkEutd71Vvn9CUzCXCmVuZHN0cmVhbQplbmRvYmoKNTIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVE\nZWNvZGUgL0xlbmd0aCAyMTQgPj4Kc3RyZWFtCnicPVC7EUMxCOs9BQvkznztN8/Lpcv+bSScpEI2\nQhKUmkzJlIc6ypKsKU8dPktih7yH5W5kNiUqRS+TsCX30ArxfYnmFPfd1ZazQzSXaDl+CzMqqhsd\n00s2mnAqE7qg3MMz+g1tdANWhx6xWyDQpGDXtiByxw8YDMGZE4siDEpNBv+tcvdS3O89HG+iiJR0\n8K755fTLzy28Tj2ORLq9+YprcaY6CkRwRmryinRhxbLIQ6TVBDU9A2u1AK7eevk3aEd0GYDsE4nj\nNKUcQ//WuMfrA4eKUvQKZW5kc3RyZWFtCmVuZG9iago1MyAwIG9iago8PCAvRmlsdGVyIC9GbGF0\nZURlY29kZSAvTGVuZ3RoIDg4ID4+CnN0cmVhbQp4nDWMuw3AMAhEe6a4Efg4gPeJUpH92xBbLrh7\n0hPnOcDIPg9H6MQtZEPhpnhJOaE+UTRabzq2SHO/vGQzFxX9M9x9he3mgGQ0SeQh0eVy5Vkpej6X\n2ht+CmVuZHN0cmVhbQplbmRvYmoKNTQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xl\nbmd0aCAxNjEgPj4Kc3RyZWFtCnicRZBLEsMgDEP3nEJH8EcGfJ50ukrvv60hTbOAp7FABncnBKm1\nBRPRBS9tS7oLPlsJzsZ46DZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7t\nouRypu7O+Udw9V/1R7HunM3EwGTlDoRm9SnufJsdUV3dZH/SY27Wa38V9qqwtKyl5YTbzl0zoATu\nqRzt/QWpczqECmVuZHN0cmVhbQplbmRvYmoKNTUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNv\nZGUgL0xlbmd0aCAxNzEgPj4Kc3RyZWFtCnicTZBNDkIhEIP3nKIXMKHzA4/zaFzp/bd28PnigvRL\nIUOnwwMdR+JGR4bO6HiwyTEOvAsyJl6N85+M6ySOCeoVbcG6tDvuzSwxJywTI2BrlNybRxT44ZgL\nQYLs8sMXGESka5hvNZ91k35+u9Nd1KV199MjCpzIjlAMG3AF2NM9DtwSzu+aJr9UKRmbOJQPVBeR\nstkJhailYpdTVWiM4lY974te7fkBwfY7+wplbmRzdHJlYW0KZW5kb2JqCjU2IDAgb2JqCjw8IC9G\naWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNjggPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrm\nZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIW0I0QZSCWBClZiZmEEk4AyKXBgDJtBXlCmVuZHN0\ncmVhbQplbmRvYmoKNTcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMTcg\nPj4Kc3RyZWFtCnicNVJLckMxCNu/U3CBzpi/fZ50smruv62EJyuwLUBCLi9Z0kt+1CXbpcPkVx/3\nJbFCPo/tmsxSxfcWsxTPLa9HzxG3LQoEURM9+DInFSLUz9ToOnhhlz4DrxBOKRZ4B5MABq/hX3iU\nToPAOxsy3hGTkRoQJMGaS4tNSJQ9Sfwr5fWklTR0fiYrc/l7cqkUaqPJCBUgWLnYB6QrKR4kEz2J\nSLJyvTdWiN6QV5LHZyUmGRDdJrFNtMDj3JW0hJmYQgXmWIDVdLO6+hxMWOOwhPEqYRbVg02eNamE\nZrSOY2TDePfCTImFhsMSUJt9lQmql4/T3AkjpkdNdu3Csls27yFEo/kzLJTBxygkAYdOYyQK0rCA\nEYE5vbCKveYLORbAiGWdmiwMbWglu3qOhcDQnLOlYcbXntfz/gdFW3ujCmVuZHN0cmVhbQplbmRv\nYmoKNTggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMzggPj4Kc3RyZWFt\nCnicNVI5rt1ADOt9Cl0ggHbNnOcFqX7u34aUXwpDtFaKmo4WlWn5ZSFVLZMuv+1JbYkb8vfJCokT\nklcl2qUMkVD5PIVUv2fLvL7WnBEgS5UKk5OSxyUL/gyX3i4c52NrP48jdz16YFWMhBIByxQTo2tZ\nOrvDmo38PKYBP+IRcq5YtxxjFUgNunHaFe9D83nIGiBmmJaKCl1WiRZ+QfGgR61991hUWCDR7RxJ\ncIyNUJGAdoHaSAw5sxa7qC/6WZSYCXTtiyLuosASScycYl06+g8+dCyovzbjy6+OSvpIK2tM2nej\nSWnMIpOul0VvN299PbhA8y7Kf17NIEFT1ihpfNCqnWMomhllhXccmgw0xxyHzBM8hzMSlPR9KH5f\nSya6KJE/Dg2hf18eo4ycBm8Bc9GftooDF/HZYa8cYIXSxZrkfUAqE3pg+v/X+Hn+/AMctoBUCmVu\nZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL0ZpcnN0Q2hhciAwIC9Gb250\nQkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0KL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4w\nMDEgMCAwIF0gL1dpZHRocyAyMiAwIFIgL0NoYXJQcm9jcyAyNSAwIFIKL0VuY29kaW5nIDw8IC9U\neXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgMzUgL251bWJlcnNpZ24gNDgg\nL3plcm8gL29uZSAvdHdvIC90aHJlZSAvZm91ciAvZml2ZSAvc2l4Ci9zZXZlbiAvZWlnaHQgL25p\nbmUgNjUgL0EgNzAgL0YgL0cgODAgL1AgODMgL1MgOTcgL2EgMTAwIC9kIC9lIC9mIC9nIDEwNSAv\naQoxMDggL2wgMTEwIC9uIC9vIC9wIDExNCAvciAvcyAvdCAvdSAvdiBdCj4+Ci9Gb250RGVzY3Jp\ncHRvciAyMyAwIFIgL0Jhc2VGb250IC9EZWphVnVTYW5zIC9TdWJ0eXBlIC9UeXBlMwovTmFtZSAv\nRGVqYVZ1U2FucyAvTGFzdENoYXIgMjU1ID4+CmVuZG9iagoyMyAwIG9iago8PCAvVHlwZSAvRm9u\ndERlc2NyaXB0b3IgL0ZsYWdzIDMyIC9Gb250TmFtZSAvRGVqYVZ1U2FucyAvSXRhbGljQW5nbGUg\nMAovU3RlbVYgMCAvRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9NYXhXaWR0aCAx\nMzQyIC9DYXBIZWlnaHQgMAovQXNjZW50IDkyOSAvWEhlaWdodCAwIC9EZXNjZW50IC0yMzYgPj4K\nZW5kb2JqCjIyIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAw\nIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAg\nNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3\nODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYz\nNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4\nNiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAz\nIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4Mzgg\nNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5\nNzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYz\nNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEz\nNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAx\nMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2\nIDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQw\nMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0\nIDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUg\nMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2\nMTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYx\nNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0\nIDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjI1IDAgb2JqCjw8IC9BIDUzIDAgUiAv\nemVybyAyNiAwIFIgL251bWJlcnNpZ24gNDEgMCBSIC9HIDI4IDAgUiAvZml2ZSAyOSAwIFIKL2wg\nMzAgMCBSIC9lIDM1IDAgUiAvciAzMiAwIFIgL29uZSAzNiAwIFIgL1MgMzMgMCBSIC9zcGFjZSAz\nNCAwIFIKL2VpZ2h0IDMxIDAgUiAvcyA0NyAwIFIgL3RocmVlIDU4IDAgUiAvZCAzNyAwIFIgL3R3\nbyAzOSAwIFIgL2ZvdXIgMjcgMCBSCi9GIDQyIDAgUiAvYSA0MyAwIFIgL3NldmVuIDQ0IDAgUiAv\nZiAzOCAwIFIgL2cgNDYgMCBSIC9QIDQ4IDAgUiAvdiA0MCAwIFIKL3UgNTUgMCBSIC9vIDUyIDAg\nUiAvcCA0OSAwIFIgL24gNTQgMCBSIC9pIDU2IDAgUiAvbmluZSA1MCAwIFIgL3NpeCA1NyAwIFIK\nL3QgNTEgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAyNCAwIFIgL0YyIDE5IDAgUiA+Pgpl\nbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAvY2EgMSA+Pgov\nQTMgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMC44IC9jYSAwLjggPj4KL0EyIDw8IC9UeXBlIC9F\neHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYg\nMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL00zIDE1IDAgUiAvRGVqYVZ1U2Fucy1PYmxp\ncXVlLWxhbWJkYSAyMSAwIFIgL00yIDE0IDAgUiAvTTQgMTYgMCBSCi9EZWphVnVTYW5zLW1pbnVz\nIDQ1IDAgUiAvTTEgMTMgMCBSIC9NMCAxMiAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBl\nIC9YT2JqZWN0IC9CQm94IFsgLTQuMjUgLTQuMjUgNC4yNSA0LjI1IF0gL0xlbmd0aCAxMzUKL1N1\nYnR5cGUgL0Zvcm0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicbZAxDoAgDEX3noIL\nfAJIEVZHr+FiTLz/qhIsCizQlv5Hf606yKiV7gOTnlmdZHRKnq0rBauTj97xk07s4xyU04FtuCtv\nsJG8QfqzGkLLqcmtT1DLRSi6F1rhH2Rpbqn52qj5Db0QPR39CPjPCJkdjSkMfGOwHgy2iI6PYmMn\nWugCkvZN3wplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9CQm94\nIFsgLTQuMjUgLTQuMjUgNC4yNSA0LjI1IF0gL0xlbmd0aCAxMzUKL1N1YnR5cGUgL0Zvcm0gL0Zp\nbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicbZAxDoAgDEX3noILfAJIEVZHr+FiTLz/qhIs\nCizQlv5Hf606yKiV7gOTnlmdZHRKnq0rBauTj97xk07s4xyU04FtuCtvsJG8QfqzGkLLqcmtT1DL\nRSi6F1rhH2Rpbqn52qj5Db0QPR39CPjPCJkdjSkMfGOwHgy2iI6PYmMnWugCkvZN3wplbmRzdHJl\nYW0KZW5kb2JqCjE2IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9CQm94IFsgLTQuMjUgLTQuMjUg\nNC4yNSA0LjI1IF0gL0xlbmd0aCAxMzUKL1N1YnR5cGUgL0Zvcm0gL0ZpbHRlciAvRmxhdGVEZWNv\nZGUgPj4Kc3RyZWFtCnicbZAxDoAgDEX3noILfAJIEVZHr+FiTLz/qhIsCizQlv5Hf606yKiV7gOT\nnlmdZHRKnq0rBauTj97xk07s4xyU04FtuCtvsJG8QfqzGkLLqcmtT1DLRSi6F1rhH2Rpbqn52qj5\nDb0QPR39CPjPCJkdjSkMfGOwHgy2iI6PYmMnWugCkvZN3wplbmRzdHJlYW0KZW5kb2JqCjE1IDAg\nb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9CQm94IFsgLTQuMjUgLTQuMjUgNC4yNSA0LjI1IF0gL0xl\nbmd0aCAxMzUKL1N1YnR5cGUgL0Zvcm0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic\nbZAxDoAgDEX3noILfAJIEVZHr+FiTLz/qhIsCizQlv5Hf606yKiV7gOTnlmdZHRKnq0rBauTj97x\nk07s4xyU04FtuCtvsJG8QfqzGkLLqcmtT1DLRSi6F1rhH2Rpbqn52qj5Db0QPR39CPjPCJkdjSkM\nfGOwHgy2iI6PYmMnWugCkvZN3wplbmRzdHJlYW0KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9Y\nT2JqZWN0IC9CQm94IFsgLTQuMjUgLTQuMjUgNC4yNSA0LjI1IF0gL0xlbmd0aCAxMzUKL1N1YnR5\ncGUgL0Zvcm0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicbZAxDoAgDEX3noILfAJI\nEVZHr+FiTLz/qhIsCizQlv5Hf606yKiV7gOTnlmdZHRKnq0rBauTj97xk07s4xyU04FtuCtvsJG8\nQfqzGkLLqcmtT1DLRSi6F1rhH2Rpbqn52qj5Db0QPR39CPjPCJkdjSkMfGOwHgy2iI6PYmMnWugC\nkvZN3wplbmRzdHJlYW0KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9Db3VudCAxIC9L\naWRzIFsgMTAgMCBSIF0gPj4KZW5kb2JqCjU5IDAgb2JqCjw8IC9DcmVhdG9yIChtYXRwbG90bGli\nIDIuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAobWF0cGxvdGxpYiBwZGYg\nYmFja2VuZCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDE3MDMzMDEwMzk1MCsxMScwMCcpCj4+CmVuZG9i\nagp4cmVmCjAgNjAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAw\nMTc5MjAgMDAwMDAgbiAKMDAwMDAxNjIwOCAwMDAwMCBuIAowMDAwMDE2MjUxIDAwMDAwIG4gCjAw\nMDAwMTYzOTMgMDAwMDAgbiAKMDAwMDAxNjQxNCAwMDAwMCBuIAowMDAwMDE2NDM1IDAwMDAwIG4g\nCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDQwMiAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAw\nIG4gCjAwMDAwMDM1MzAgMDAwMDAgbiAKMDAwMDAxNjg0MCAwMDAwMCBuIAowMDAwMDE3NjUwIDAw\nMDAwIG4gCjAwMDAwMTY1NzAgMDAwMDAgbiAKMDAwMDAxNzM4MCAwMDAwMCBuIAowMDAwMDE3MTEw\nIDAwMDAwIG4gCjAwMDAwMDQzMDQgMDAwMDAgbiAKMDAwMDAwNDA5NiAwMDAwMCBuIAowMDAwMDAz\nNzg3IDAwMDAwIG4gCjAwMDAwMDUzNTcgMDAwMDAgbiAKMDAwMDAwMzU1MSAwMDAwMCBuIAowMDAw\nMDE0NzcwIDAwMDAwIG4gCjAwMDAwMTQ1NzAgMDAwMDAgbiAKMDAwMDAxNDA5NCAwMDAwMCBuIAow\nMDAwMDE1ODIzIDAwMDAwIG4gCjAwMDAwMDUzNzkgMDAwMDAgbiAKMDAwMDAwNTY2MiAwMDAwMCBu\nIAowMDAwMDA1ODI0IDAwMDAwIG4gCjAwMDAwMDYxNDEgMDAwMDAgbiAKMDAwMDAwNjQ2MSAwMDAw\nMCBuIAowMDAwMDA2NTc4IDAwMDAwIG4gCjAwMDAwMDcwNDMgMDAwMDAgbiAKMDAwMDAwNzI3MyAw\nMDAwMCBuIAowMDAwMDA3Njg0IDAwMDAwIG4gCjAwMDAwMDc3NzMgMDAwMDAgbiAKMDAwMDAwODA5\nMSAwMDAwMCBuIAowMDAwMDA4MjQzIDAwMDAwIG4gCjAwMDAwMDg1NDMgMDAwMDAgbiAKMDAwMDAw\nODc0OSAwMDAwMCBuIAowMDAwMDA5MDcwIDAwMDAwIG4gCjAwMDAwMDkyMTQgMDAwMDAgbiAKMDAw\nMDAwOTQzNSAwMDAwMCBuIAowMDAwMDA5NTgxIDAwMDAwIG4gCjAwMDAwMDk5NTggMDAwMDAgbiAK\nMDAwMDAxMDA5OCAwMDAwMCBuIAowMDAwMDEwMjY4IDAwMDAwIG4gCjAwMDAwMTA2NzkgMDAwMDAg\nbiAKMDAwMDAxMTA4NCAwMDAwMCBuIAowMDAwMDExMzIyIDAwMDAwIG4gCjAwMDAwMTE2MzEgMDAw\nMDAgbiAKMDAwMDAxMjAyNCAwMDAwMCBuIAowMDAwMDEyMjI4IDAwMDAwIG4gCjAwMDAwMTI1MTUg\nMDAwMDAgbiAKMDAwMDAxMjY3NSAwMDAwMCBuIAowMDAwMDEyOTA5IDAwMDAwIG4gCjAwMDAwMTMx\nNTMgMDAwMDAgbiAKMDAwMDAxMzI5MyAwMDAwMCBuIAowMDAwMDEzNjgzIDAwMDAwIG4gCjAwMDAw\nMTc5ODAgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA2MCAvSW5mbyA1OSAwIFIgL1Jvb3QgMSAw\nIFIgPj4Kc3RhcnR4cmVmCjE4MTI4CiUlRU9GCg==\n", "image/svg+xml": [ "\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig5, max_errs = figure5();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the left panel we see that there are at most three dimensions to the active subspace (the largest eigenvalue is still 3 orders of magnitude larger than the 3rd largest, which I why we say \"at most\").\n", "\n", "The right panel shows that only the 3d active subspace version shows a monotonically decreasing error as the number of training points rises.\n", "\n", "> NOTE: This figure doesn't fully line up with Figure 5 in S&B. Notice the scale on the horizontal axes is on the order of 10^-3 in this figure, but 10^-7 in Figure 5 in S&B. I played with this for a while and was never able to achieve results on the same scale as theirs. My hunch is that the choice of where to locate the training points is increasingly important as the number of input dimensions increases. Here I'm just sampling from $[-1, 1]^{10}$ uniformly, which isn't necessarily an ideal choice of training points. Unfortunately S&B don't discuss how they choose training points, so we can just guess here. \n", "\n", ">For sake of completeness, in addition to uniformly sampled points I also tried using Sobol, Neiderreiter, Weyl, and Haber sequences to choose training points. I didn't see significant changes in the performance of the Active Subspace GPR, so I opted to keep things simple in the final version and stick to randomly sampling points. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "_draft": { "nbviewer_url": "https://gist.github.com/d884a4f7ef9e9bd5862428303179b2bd" }, "gist": { "data": { "description": "ASGP_py.ipynb", "public": false }, "id": "d884a4f7ef9e9bd5862428303179b2bd" }, "kernel_info": { "name": "python3" }, "kernelspec": { "display_name": "Python [default]", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autocomplete": false, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false } }, "nbformat": 4, "nbformat_minor": 2 }