{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Numpy and Matplotlib"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Numpy\n",
"\n",
"NumPy is a linear algebra library in Python, with computationally expensive methods written in FORTRAN for speed. \n",
"\n",
"* The reference manual is at . \n",
"* A nice tutorial can be found at \n",
"* or: \n",
"* If you already know Matlab, a comparison is at "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Importing libraries\n",
"\n",
"To import a library in Python, you only need to use the keyword `import` at the beginning of your script / notebook (or more exactly, before you use it).\n",
"\n",
"```python\n",
"import numpy\n",
"```\n",
"\n",
"Think of it as the equivalent of `#include ` in C/C++ (if you know Java, you will not be shocked). You can then use the functions and objects provided by the library using the namespace of the library:\n",
"\n",
"```python\n",
"x = numpy.array([1, 2, 3])\n",
"```\n",
"\n",
"If you do not want to type `numpy.` everytime, and if you are not afraid that numpy redefines any important function, you can also simply import every definition declared by the library in your current namespace with:\n",
"\n",
"```python\n",
"from numpy import *\n",
"```\n",
"\n",
"and use the objects directly:\n",
"\n",
"```python\n",
"x = array([1, 2, 3])\n",
"```\n",
"\n",
"However, it is good practice to give an alias to the library when its name is too long (numpy is still okay, but think of matplotlib...):\n",
"\n",
"```python\n",
"import numpy as np \n",
"```\n",
"\n",
"You can then use the objects like this:\n",
"\n",
"```python\n",
"x = np.array([1, 2, 3])\n",
"```\n",
"\n",
"Remember that you can get help on any NumPy function:\n",
"\n",
"```python\n",
"help(np.array)\n",
"help(np.ndarray.transpose)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Vectors and matrices\n",
"\n",
"The basic object in NumPy is an **array** with d-dimensions (1D = vector, 2D = matrix, 3D or more = tensor). They can store either integers or floats, using various precisions.\n",
"\n",
"In order to create a vector of three floats, you simply have to build an `array()` object by providing a list of floats as input:\n",
"\n",
"```python\n",
"A = np.array( [ 1., 2., 3.] )\n",
"```\n",
"\n",
"Matrices should be initialized with a list of lists. For a 3x4 matrix of 8 bits unsigned integers, it is:\n",
"\n",
"```python\n",
"B = np.array( [ \n",
" [ 1, 2, 3, 4],\n",
" [ 5, 6, 7, 8],\n",
" [ 4, 3, 2, 1] \n",
" ] , dtype=np.uint8)\n",
"```\n",
"\n",
"Most of the time, you won't care about the type (the default floating-point precision is what you want for machine learning), but if you need it, you can always specify it with the parameter `dtype={int32, uint16, float64, ...}`. Note that even if you pass integers to the array (`np.array( [ 1, 2, 3] )`), they will be converted to floats by default."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The following attributes of an array can be accessed:\n",
"\n",
"- `A.shape` : returns the shape of the vector `(n,)` or matrix `(m, n)`.\n",
"\n",
"- `A.size` : returns the total number of elements in the array.\n",
"\n",
"- `A.ndim` : returns the number of dimensions of the array (vector: 1, matrix:2).\n",
"\n",
"- `A.dtype.name` : returns the type of data stored in the array (int32, uint16, float64...)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Q:** Define the two arrays $A$ and $B$ from above and print those attributes. Modify the arrays (number of elements, type) and observe how they change. \n",
"\n",
"*Hint:* you can print an array just like any other Python object."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Internally, the values are stored sequentially as a vector, even if your array has more than one dimension. The apparent shape is just used for mathematical operations. You can **reshape** a matrix very easily with the `reshape()` method:\n",
"\n",
"```python\n",
"B = np.array( [ \n",
" [ 1, 2, 3, 4],\n",
" [ 5, 6, 7, 8],\n",
" [ 4, 3, 2, 1] \n",
"]) # B has 3 rows, 4 columns\n",
"\n",
"C = B.reshape((6, 2)) # C has 6 rows, 2 columns\n",
"```\n",
"\n",
"The only thing to respect is that the total number of elements must be the same. Beware also of the order in which the elements will be put.\n",
"\n",
"**Q:** Create a vector with 8 elements and reshape it into a 2x4 matrix."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Initialization of an array\n",
"\n",
"Providing a list of values to `array()` would be tedious for large arrays. Numpy offers constructors that allow to construct simply most vectors or matrices.\n",
"\n",
"`np.zeros(shape)` creates an array of shape `shape` filled with zeros. Note: if you give a single integer for the shape, it will be interpreted as a vector of shape `(d,)`. \n",
"\n",
"`np.ones(shape)` creates an array of shape `shape` filled with ones. \n",
"\n",
"`np.full(shape, val)` creates an array of shape `shape` filled with `val`. \n",
"\n",
"`np.eye(n)` creates a diagonal matrix of shape `(n, n)`.\n",
"\n",
"`np.arange(a, b)` creates a vector of integers whose value linearly increase from `a` to `b` (excluded).\n",
"\n",
"`np.linspace(a, b, n)` creates a vector of `n` values evenly distributed between `a` and `b` (included).\n",
"\n",
"\n",
"**Q:** Create and print:\n",
"\n",
"* a 2x3 matrix filled with zeros.\n",
"* a vector of 12 elements initialized to 3.14.\n",
"* a vector of 11 elements whose value linearly increases from 0.0 to 10.0.\n",
"* a vector of 11 elements whose value linearly increases from 10 to 20."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Random distributions\n",
"\n",
"In many cases, it is useful to initialize a vector or matrix with random values. **Random number generators** (rng) allows to draw numbers from any probability distribution (uniform, normal, etc.) using pseudo-random methods. \n",
"\n",
"In numpy versions before 1.16, the `numpy.random` module had direct methods allowing to initialize arrays:\n",
"\n",
"```python\n",
"A = np.random.uniform(-1.0, 1.0, (10, 10)) # a 10x10 matrix with values uniformly taken between -1 and 1\n",
"```\n",
"\n",
"Since numpy 1.16, this method has been deprecated in favor of a more explicit initialization of the underlying rng:\n",
"\n",
"```python\n",
"rng = np.random.default_rng()\n",
"A = rng.uniform(-1.0, 1.0, (10, 10))\n",
"```\n",
"\n",
"The advantages of this new method (reproducibility, parallel seeds) will not matter for these exercises, but let's take good habits already.\n",
"\n",
"The generator has many built-in methods, covering virtually any useful probability distribution. Read the documentation of the random generator:\n",
"\n",
"\n",
"\n",
"**Q:** Create:\n",
"\n",
"* A vector of 20 elements following a normal distribution with mean 2.0 and standard devation 3.0.\n",
"* A 10x10 matrix whose elements come from the exponential distribution with $\\beta = 2$.\n",
"* A vector of 10 integers randomly chosen between 1 and 100 (hint: involves `arange` and `rng.choice`)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Manipulation of matrices: indices, slices\n",
"\n",
"To access a particular element of a matrix, you can use the usual Python list style (the first element has a rank of 0), once per dimension:\n",
"\n",
"```python\n",
"A = np.array(\n",
" [ \n",
" [ 1, 2, 3, 4],\n",
" [ 5, 6, 7, 8],\n",
" [ 9, 10, 11, 12]\n",
" ]\n",
")\n",
"\n",
"x = A[0, 2] # The element on the first row and third column\n",
"```\n",
"\n",
"For matrices, the first index represents the rows, the second the columns. [0, 2] represents the element at the first row, third column.\n",
"\n",
"**Q:** Define this matrix and replace the element `12` by a zero using indices:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is possible to access complete row or columns of a matrix using **slices**. The `:` symbol is a shortcut for \"everything\":\n",
"\n",
"```python\n",
"b = A[:, 2] # third column\n",
"c = A[0, :] # first row\n",
"```\n",
"\n",
"**Q:** Set the fourth column of A to 1."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As for python lists, you can specify a range `start:stop` to get only a subset of a row/column (beware, stop is excluded):\n",
"\n",
"```python\n",
"d = A[0, 1:3] # second and third elements of the first row\n",
"e = A[1, :2] # first and second elements of the second row\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use boolean arrays to retrieve indices:\n",
"\n",
"```python\n",
"A = np.array( \n",
" [ [ -2, 2, 1, -4],\n",
" [ 3, -1, -5, -3] ])\n",
"\n",
"negatives = A < 0 # Boolean array where each element is True when the condition is met.\n",
"A[negatives] = 0 # All negative elements of A (where the boolean matrix is True) will be set to 0\n",
"```\n",
"\n",
"A simpler way to write it is:\n",
"\n",
"```python\n",
"A[A < 0] = 0\n",
"```\n",
"\n",
"**Q:** print A, negatives and A again after the assignment:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Basic linear algebra \n",
"\n",
"Let's first define some matrices:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"A = np.array( [ [ 1, 2, 3, 4],\n",
" [ 5, 6, 7, 8] ])\n",
"\n",
"B = np.array( [ [ 1, 2],\n",
" [ 3, 4],\n",
" [ 5, 6],\n",
" [ 7, 8] ])\n",
"\n",
"C = np.array( [ [ 1, 2, 3, 4],\n",
" [ 5, 6, 7, 8],\n",
" [ 9, 0, 1, 1],\n",
" [ 13, 7, 2, 6] ])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Transpose a matrix \n",
"\n",
"A matrix can be transposed with the `transpose()` method or the `.T` shortcut:\n",
"\n",
"```python\n",
"D = A.transpose() \n",
"E = A.T # equivalent\n",
"```\n",
"\n",
"**Q:** Try it:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`transpose()` does not change `A`, it only returns a transposed copy. To transpose `A` definitely, you have to use the assigment `A = A.T`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Multiply two matrices \n",
"\n",
"There are two manners to multiply matrices:\n",
"\n",
"- element-wise: Two arrays of **exactly** the same shape can be multiplied *element-wise* by using the `*` operator:\n",
"\n",
"```python\n",
"D = A * B\n",
"```\n",
"\n",
"- algebrically: To perform a **matrix multiplication**, you have to use the `dot()` method. Beware: the dimensions must match! `(m, n) * (n, p) = (m, p)`\n",
"\n",
"```python\n",
"E = np.dot(A, B)\n",
"```\n",
"\n",
"**Q:** Use the matrices `A` and `B` previously defined and multiply them element-wise and algebrically. You may have to transpose one of them."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Multiplying a matrix with a vector\n",
"\n",
"`*` and `np.dot` also apply on matrix-vector multiplications $\\mathbf{y} = A \\times \\mathbf{x}$ or vector-vector multiplications.\n",
"\n",
"**Q:** Define a vector $\\mathbf{x}$ with four elements and multiply it with the matrix $A$ using `*` and `np.dot`. What do you obtain? Try the same by multiplying the vector $\\mathbf{x}$ and itself."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Inverting a matrix\n",
"\n",
"Inverting a Matrix (when possible) can be done using the `inv()` method whitch is defined in the `linalg` submodule of NumPy.\n",
"\n",
"```python\n",
"inv_C = np.linalg.inv(C)\n",
"```\n",
"\n",
"**Q:**\n",
"\n",
"1. Invert `C` and print the result.\n",
"2. Multiply `C` with its inverse and print the result. What do observe? Why is Numpy called a *numerical computation* library?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Summing elements \n",
"\n",
"One can sum the elements of a matrix globally, row-wise or column-wise:\n",
"\n",
"```python\n",
"# Globally\n",
"S1 = np.sum(A)\n",
"\n",
"# Per column\n",
"S2 = np.sum(A, axis=0) \n",
"\n",
"# Per row\n",
"S3 = np.sum(A, axis=1) \n",
"```\n",
"\n",
"**Q:** Try them:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You also have access to the minimum (`np.min()`), maximum (`np.max()`), mean (`np.mean()`) of an array, also per row/column. \n",
"\n",
"**Q:** Try them out:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Mathematical operations \n",
"\n",
"You can apply any usual mathematical operations (cos, sin, exp, etc...) on each element of a matrix (element-wise):\n",
"\n",
"```python\n",
"D = np.exp(A)\n",
"E = np.cos(A)\n",
"F = np.log(A)\n",
"G = (A+3) * np.cos(A-2)\n",
"```\n",
"\n",
"**Q:** Try it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Matplotlib\n",
"\n",
"Matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms.\n",
"\n",
"* Reference: \n",
"* Tutorial by N. Rougier: \n",
"\n",
"This is the default historical visualization library in Python, which anybody should know, but not the nicest. If you are interested in having better visualizations, have a look at:\n",
"\n",
"* `seaborn` \n",
"* `ggplot2` \n",
"* `bokeh` \n",
"* `plotly` \n",
"\n",
"We will nevertheless stick to matplotlib in these exercises.\n",
"\n",
"The `pyplot` module is the most famous, as it has a similar interface to Matlab. It is customary to use the `plt` namescape for it:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### `plt.plot()`\n",
"\n",
"The `plt.plot()` command allows to make simple line drawings:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAgNElEQVR4nO3deWBU5b3/8fc3OyFhCSTsEPZVEQgQ5LpbtVqL9laLVgTZbF3qtbUV/fX+sLet1/ZnF6utLSISFFBEbsWlFotWa1Eg7EtYIgIJCVkIJCEL2Z77R6b9IQWBTCYnM/N5/TPJSWbOZyD5cDjnOc9jzjlERCS0RHgdQEREmp/KXUQkBKncRURCkMpdRCQEqdxFREJQlNcBADp37uxSU1O9jiEiElQ2bNhQ7JxLPt3XWkW5p6amkpmZ6XUMEZGgYmYHzvQ1nZYREQlBKncRkRCkchcRCUEqdxGREKRyFxEJQSp3EZEQpHIXEQlBZy13M1tgZoVmtv2kbUlm9q6Z7fU9djzpa4+YWbaZ7TazawMVXEQk2C36eD9rsosD8trncuS+ELjulG1zgNXOuYHAat/nmNkwYDIw3Pec35lZZLOlFREJEWuyi3ls5Q5eycwJyOuftdydcx8CJadsngRk+D7OAG46afvLzrkTzrnPgGxgXPNEFREJDYdLq7l/6Sb6JSfw+M0XBGQfTT3n3sU5lw/ge0zxbe8BnPzPUK5v278ws9lmlmlmmUVFRU2MISISXGrrG7h3yUaqa+v5/R1jaBsbmFlgmvuCqp1m22nX8XPOzXPOpTnn0pKTTzvvjYhIyHn87Sw2HDjKz75+IQNSEgK2n6aWe4GZdQPwPRb6tucCvU76vp5AXtPjiYiEjpVb8njh7/u5a2IqX7mwe0D31dRyXwlM9X08FXj9pO2TzSzWzPoCA4F1/kUUEQl+uw+X8/DyrYxN7cij1w8N+P7OerLHzJYClwOdzSwXmAs8ASwzsxnAQeAWAOfcDjNbBuwE6oB7nXP1AcouIhIUyqpr+dZLG0iIi+K3t48mOjLwtxidtdydc7ed4UtXneH7fwr81J9QIiKhoqHB8dCyLeSUVLJ0djop7eJaZL+6Q1VEJICe/eBTVu0s4JHrhzI2NanF9qtyFxEJkA/2FPHkqt1Muqg70yemtui+Ve4iIgGQU1LJd5ZuYnCXRP77axdgdrqR4oGjchcRaWZVNfXc/eIGnHP8YcoY4mNafrnqVrFAtohIqHDOMWfFVrIOl7Fg2lj6dGrrSQ4duYuINKPnP/qM1zfn8dA1g7licMrZnxAgKncRkWayJruY//7TLq4b3pV7Lu/vaRaVu4hIM8g9Wsl9SzfRt3Nbnrx1ZItfQD2Vyl1ExE9VNfXMXrSB2voG5k0ZQ0KAZno8H94nEBEJYs45fvCa7wLq1LH0Sw7cTI/nQ0fuIiJ+mPfhPt7Y4ruAOsS7C6inUrmLiDTRB3uK+Nk7u7jhgm6eX0A9lcpdRKQJ9hUd574lGxnctR3/75YLPb+AeiqVu4jIeSqrrmXmokyiIyN47k5v7kA9G5W7iMh5qG9wPLB0EwePVPK7b46mZ8d4ryOdlspdROQ8/PzPu3h/dxFzvzqc9H6dvI5zRip3EZFztGJjLn/4YB/fHN+bKel9vI7zhVTuIiLnYNPBo8xZsY30fkk89tXhXsc5K5W7iMhZHC6t5u4XN9ClXSy/++aYFlkD1V+tP6GIiIeqauqZtSiTihN1zL9zLEltY7yOdE5a3/gdEZFWoqHB8dCrW9ieV8r8O9MY3DXR60jnTEfuIiJn8NTqvby1LZ9HvjyEq4Z28TrOeVG5i4icxhtb8nhq9V5uGdOTWZf08zrOeVO5i4icYtPBozz06hbGpnbkJzePaHVTC5wLlbuIyEkOHati1qINpLSL5Q9T0oiNivQ6UpPogqqIiM/xE3XMWLieE7X1LJ01PmhGxpyOyl1EhP8/Z8zewuMsmDaWgV2CZ2TM6ei0jIgI8NO3sli9q5DHbhzGZYOSvY7jN5W7iIS9Fz/ez4K/f8ZdE1OZMiHV6zjNQuUuImHtr7sLeeyNnVw5JIUf3jDM6zjNRuUuImErK7+M+5ZsYlCXRH5z2ygiI4JvyOOZ+FXuZvagme0ws+1mttTM4swsyczeNbO9vseOzRVWRKS5FJRVM33hetrGRrJgWhoJsaE1vqTJ5W5mPYDvAGnOuRFAJDAZmAOsds4NBFb7PhcRaTUqTtQxfeF6SqtqeX7qWLq1b+N1pGbn72mZKKCNmUUB8UAeMAnI8H09A7jJz32IiDSb+gbHAy9vIiu/jGduH8WIHu29jhQQTS5359wh4EngIJAPlDrnVgFdnHP5vu/JB1JO93wzm21mmWaWWVRU1NQYIiLnzDnHYyt38JesQn701eFcOSS4JgM7H/6clulI41F6X6A70NbM7jjX5zvn5jnn0pxzacnJwT+mVERav+f+to8XPznA7Ev7hcyQxzPx57TM1cBnzrki51wtsAK4GCgws24AvsdC/2OKiPjnra35PP72Lm64oBtzrhvidZyA86fcDwLpZhZvjVOmXQVkASuBqb7vmQq87l9EERH/rN9fwoPLNjOmT0d+cetIIkJoyOOZNHnsj3NurZktBzYCdcAmYB6QACwzsxk0/gNwS3MEFRFpiuzC48zMyKRnhzbMvzONuOjgnOXxfPk1sNM5NxeYe8rmEzQexYuIeKqwvJqpC9YRHWlkTB9HxyCe5fF8hdaofRERn+O+sewlFTW8cnc6vZLivY7UolTuIhJyausbuGfxRrLyy3nuzjFc2LOD15FanOaWEZGQ4pxjzmvb+HBPEY/fPCKkx7J/EZW7iISUX6zaw2sbc3nw6kF8Y2xvr+N4RuUuIiEjY81+nnk/m9vG9eI7Vw3wOo6nVO4iEhLe2prPY2/s4OqhXfjxpBE03n4TvlTuIhL01nxazIOvbGZM7448c/sooiJVbfoTEJGgtv1QKbMXbaBPp3jmTw2fm5TORuUuIkFrf3EF015YR7u4KBbNGEeH+PC5SelsVO4iEpQKy6u5c8E66hsci2aMD8kFN/yhm5hEJOiUVdcydcF6io+fYMmsdAakJHgdqdXRkbuIBJWqmnpmLswku7CcZ+8Yw0W9OngdqVXSkbuIBI3a+gbuW7KR9QdK+M3kUVw2SAv9nImO3EUkKDQ0OB5evpXVuwr58aQR3Diyu9eRWjWVu4i0es45/uvNnazYdIiHrhnEHel9vI7U6qncRaTV+9W7e1i4Zj+zLunLvVeE97QC50rlLiKt2vy/7eM372XzjbRePHr90LCfVuBcqdxFpNV6Zf1BfvJWFtdf0JXHv3aBiv08qNxFpFV6Y0sec1Zs47JByfzqGxcRGQaLWjcnlbuItDp/2VnAg69sZmxqEr+/YwyxUZov5nyp3EWkVVmTXcw9SzYyrHs7np+aRpsYFXtTqNxFpNXI3F/CjIxM+nZqS8Zd40iMi/Y6UtBSuYtIq7A19xjTXlhPt/ZxvDRzPB3baoZHf6jcRcRzWfll3LlgHR3io1k8azzJibFeRwp6KncR8VR2YTl3zF9LXFQkS2ama+reZqJyFxHPfFZcwe3PrSUiwlgyazy9O8V7HSlkqNxFxBM5JZXc/twn1DU4lswcT79kzcnenFTuItLico9WMnneJ1TW1PPSjPEM7JLodaSQo3IXkRaVd6yK2577hPLqWhbPHM+w7u28jhSSVO4i0mIOl1Zz23OfcKyilhdnjGdEj/ZeRwpZfpW7mXUws+VmtsvMssxsgpklmdm7ZrbX99ixucKKSPD6R7EfOV5DxoxxjNTyeAHl75H7U8A7zrkhwEggC5gDrHbODQRW+z4XkTBWUNZY7IVl1WRMH8vo3jrmC7Qml7uZtQMuBZ4HcM7VOOeOAZOADN+3ZQA3+RdRRIJZQVk1t81rLPZFM8Yxpk+S15HCgj9H7v2AIuAFM9tkZvPNrC3QxTmXD+B7TDndk81stpllmllmUVGRHzFEpLU6XFrN5HmfUFBWTcZ0FXtL8qfco4DRwLPOuVFABedxCsY5N885l+acS0tO1grmIqEmv7SKyfM+pqj8BItmjCMtVcXekvwp91wg1zm31vf5chrLvsDMugH4Hgv9iygiwSbvWBWT531C8fEaHbF7pMnl7pw7DOSY2WDfpquAncBKYKpv21Tgdb8SikhQySmp5BvzPqbkeI3vHLsunnohys/n3w8sNrMYYB9wF43/YCwzsxnAQeAWP/chIkHiwJHGuWLKq2t5ceZ4LtJwR8/4Ve7Ouc1A2mm+dJU/rysiwWdf0XFuf24t1XX1LJmVrhuUPObvkbuICHsKyvnm/LXUNziWzkpnaDdNKeA1lbuI+GVHXilTnl9HZITxyux0TQLWSmhuGRFpss05x7ht3ifERUWw7O4JKvZWREfuItIka/cdYfrC9SQlxLBkZjq9krTQRmuicheR8/bBniLufjGTHh3asHhmOl3bx3kdSU6hcheR8/LO9sPcv3QjA1MSWTRjHJ0TtJh1a6RyF5Fz9tqGXH7w2lYu7NmehdPG0T4+2utIcga6oCoi5yRjzX6+9+oW0vsl8dKM8Sr2Vk5H7iLyhZxz/Pb9bJ5ctYcvDevC07eNIi460utYchYqdxE5o4YGx0/fzuL5jz7j5lE9+PnXLyQ6Uv/hDwYqdxE5rbr6Bh5+bRuvbcxl2sWp/N+vDCMiwryOJedI5S4i/6K6tp77l27i3Z0FfPdLg7j/ygGYqdiDicpdRD6ntKqWWRmZrD9Qwn9NGs6dE1K9jiRNoHIXkX8qLKvmzgXr+LToOL+ZPIobR3b3OpI0kcpdRIDGKXvvXLCOkooaFkwbyyUDtfxlMFO5iwibc44xfeF6AJbOSmekFtkIeip3kTD3/u5C7nlpI50TY1g0fTx9O7f1OpI0A5W7SBh7NTOHR1ZsY3DXRF64aywpiZoALFSo3EXCkHOOp9/L5pfv7uHfBnTm2TtGkxin6QRCicpdJMzU1Tfwn69vZ+m6HL42ugdPfO1CYqJ012moUbmLhJHjJ+q4b8lG/rq7iHuv6M9D1wzWzUkhSuUuEiYKyqqZvnA9uw6X8/jNF3D7+N5eR5IAUrmLhIHdh8uZvnA9RytrmD81jSsGp3gdSQJM5S4S4j7cU8S9izfSJiaSZXdPYESP9l5HkhagchcJYUvXHeSHf9zOwJQEFkwbS/cObbyOJC1E5S4SguobHD9/Zxd/+HAflw9O5pnbR5MQq1/3cKK/bZEQU1lTxwMvb+bdnQVMSe/D3BuHEaUFNsKOyl0khOSXVjEzI5Os/DIeu3EY0yb29TqSeETlLhIiNuccY/aiTCpr6nl+6liuGKIRMeFM5S4SAlZuyeP7r24hpV0sL80cz6AuiV5HEo+p3EWCWEOD41d/2cPT72UzLjWJZ+8YTaeEWK9jSSvgd7mbWSSQCRxyzn3FzJKAV4BUYD9wq3PuqL/7EZHPO36ijgdfabxwemtaT3580whioyK9jiWtRHNcQn8AyDrp8znAaufcQGC173MRaUY5JZX8++/WsDqrgLk3DuNn/36hil0+x69yN7OewA3A/JM2TwIyfB9nADf5sw8R+by/Zxdz4zMfkV9aRcb0cdw1sa8m/5J/4e9pmV8DPwBOvnrTxTmXD+Ccyzez016yN7PZwGyA3r01gZHI2TjnWPD3/Tz+dhb9k9syb0oaqVo1Sc6gyUfuZvYVoNA5t6Epz3fOzXPOpTnn0pKTtRCvyBeprq3ne69u4cdv7uTqoSmsuGeiil2+kD9H7hOBr5rZ9UAc0M7MXgIKzKyb76i9G1DYHEFFwlXu0Uq+9dIGduSV8eDVg7j/ygFEROg0jHyxJh+5O+cecc71dM6lApOB95xzdwArgam+b5sKvO53SpEw9dHeYm58+iMOHKnk+alpPHD1QBW7nJNAjHN/AlhmZjOAg8AtAdiHSEhzzvHsB5/y5J93MyAlgT9MSaOvTsPIeWiWcnfO/RX4q+/jI8BVzfG6IuGorLqWh5ZtYdXOAm4c2Z0nvnYBbTWjo5wn/cSItCJZ+WXcs3gjOSWVzL1xGNMuTtUwR2kSlbtIK/FqZg4//ON22reJZunsdMamJnkdSYKYyl3EY9W19Ty2cgcvr8/h4v6deGryKJITNT+M+EflLuKhfUXHuWfxRnYdLufeK/rz3S8NJlKjYaQZqNxFPLJySx6PvLaVmKgIFt41lssHa/51aT4qd5EWVl1bz4/e2MnSdQcZ06cjT982SgtXS7NTuYu0oOzCcu5dvIndBeV8+/L+fPdLg4jW+qYSACp3kRbgnOPVzFzmrtxBfEykTsNIwKncRQKsrLqWR1ds482t+Uzo14lfT76ILu3ivI4lIU7lLhJAGw6U8MDLm8kvreb71w7mW5f112gYaREqd5EAqKtv4Jn3s3n6vWy6tY9j2d0TGNOno9exJIyo3EWaWU5JJQ++spnMA0e5eVQPfjRpOO3ior2OJWFG5S7STJxzrNh4iLkrd2DAU5MvYtJFPbyOJWFK5S7SDI5W1PDDP27nrW35jOubxC9vHUnPjvFex5IwpnIX8dP7uwt5ePlWjlbW8PB1Q5h9aT9dNBXPqdxFmqjiRB0/fTuLJWsPMrhLIi/cNZbh3dt7HUsEULmLNMkn+47w/eVbyD1axd2X9uO71wwiNirS61gi/6RyFzkPVTX1/PzPu3jh7/vp0ymeZXdP0Lzr0iqp3EXO0brPSvjB8i3sP1LJ1Al9ePjLQ4iP0a+QtE76yRQ5i8qaOn7+zm4yPt5Pz45tWDJrPBf37+x1LJEvpHIX+QJ/zy5mzoqt5JRUMe3iVL5/7WAtVi1BQT+lIqdRWlXL429l8UpmDn07t2XZ3RMY11fn1iV4qNxFTuKc40/bDzN35Q5KKmr49uX9eeCqgcRFaySMBBeVu4hPfmkV//nHHfwlq4Dh3dvxwrSxjOihcesSnFTuEvbqGxwZa/bzi1W7qXeOR68fwvSJfYnSCkkSxFTuEta25Zby6P9sY9uhUi4blMxPbhpBryTNCSPBT+UuYam0qpZfrNrNi58coHNCLM/cPoobLuiGmeaEkdCgcpew4pzjj5sP8dO3siipqGHqhFS+e80gzbcuIUflLmFjZ14Zc1duZ/3+o4zs1YGFd43TBVMJWSp3CXmllbX86i97WPTxftq3ieaJr13ArWm9iNC0vBLCmlzuZtYLWAR0BRqAec65p8wsCXgFSAX2A7c65476H1Xk/NQ3OF5ef5An/7yb0qpabh/fm4euGUyH+Bivo4kEnD9H7nXA95xzG80sEdhgZu8C04DVzrknzGwOMAd42P+oIufu40+P8OM3d7Izv4xxfZOYe+MwzbUuYaXJ5e6cywfyfR+Xm1kW0AOYBFzu+7YM4K+o3KWFHDxSyeNvZ/HOjsP06NCGp28bxVcu1CgYCT/Ncs7dzFKBUcBaoIuv+HHO5ZtZyhmeMxuYDdC7d+/miCFhrLSylmfe30vGmgNERRoPXTOImZf007QBErb8LnczSwBeA/7DOVd2rkdIzrl5wDyAtLQ0528OCU81dQ0sXnuAp1bvpbSqlq+P7slD1w6mS7s4r6OJeMqvcjezaBqLfbFzboVvc4GZdfMdtXcDCv0NKXIq5xxvbs3nyVW7OXCkkokDOvHo9UN1Xl3Ex5/RMgY8D2Q553550pdWAlOBJ3yPr/uVUOQUa7KL+dk7u9iSW8qQroksvGsslw1K1nl1kZP4c+Q+EZgCbDOzzb5tj9JY6svMbAZwELjFr4QiPttyS/n5n3fxt73FdG8fx5O3jOTmUT2I1Hh1kX/hz2iZj4Az/VZd1dTXFTnVnoJyfvXuHv60/TAd46P54Q1DuSO9jy6WinwB3aEqrdZnxRU89Zc9vL4lj7YxUXznqoHMuqQviZoHRuSsVO7S6hw4UsFvVmfzx82HiI40Zl/aj29d2p+ObXVnqci5UrlLq7G/uILfvp/Nik2HiIowpl2cyt2X9SMlUcMaRc6Xyl08l114nN+9/48j9QimpPfhnsv7k6Kx6iJNpnIXz2w/VMrv/prNn7YfJi4qkhn/1pdZl+pIXaQ5qNylRTnn+GRfCb//4FM+2FNEYmwU914+gLsmptIpIdbreCIhQ+UuLaK+wbFqx2F+/8GnbMktpXNCDN+/djBTJvTRKkgiAaByl4CqrKlj+YZc5v/tMw6WVNKnUzw/uWkEXx/TU+PURQJI5S4BkV9aRcaaAyxdd5DSqlpG9e7AI18ewjXDu+qOUpEWoHKXZuOcY+PBoyxcc4A/bcunwTmuG9GV6RP7kpaa5HU8kbCiche/VdfWs3JLHos+3s/2Q2UkxkUx7eJUpl6cSq+keK/jiYQllbs02f7iCl765ACvbsiltKqWASkJ/OSmEdw8qgdtY/WjJeIl/QbKeampa2DVzsMsWXuQNZ8eISrCuHZ4V+5I70N6vyRNuyvSSqjc5ZzsLSjnlfU5/M+mQxypqKFHhzY8dM0gbknrpVWPRFohlbucUWlVLW9uzWP5hlw2HTxGVIRx9dAuTB7Xi0sGJmvUi0grpnKXz6mtb+Bve4tYsfEQq3YWUFPXwMCUBP7P9UO5eXQPOusuUpGgoHIXnHNsyjnGys15vLEljyMVNXSIj2by2F58fUxPLujRXufSRYKMyj1MOefIyi/nza15vLE1j5ySKmKiIrh6aAo3XdSDywenEBMV4XVMEWkilXsYcc6xI6+Md7Yf5q1t+XxWXEFkhDFxQGe+c+VArh3RVfO8iIQIlXuIq29ovGt01Y7DvLPjMDklVUQYTOjfiVmX9OPa4V00G6NICFK5h6DjJ+r4aG8xq7MKeG9XIUcqaoiObDxCv++KAVw9VIUuEupU7iHAOce+4go+2F3E+7sLWbuvhJr6BhLjorhicApfGtaFywYn65SLSBhRuQep0spa1nxazN+yi/lwTxG5R6sA6J/clmkTU7licAppqR2JjtRFUZFwpHIPEhUn6li/v4SP9x3h40+PsP1QKQ0OEmKjSO/XiW9d1p/LBiVroi4RAVTurVZJRQ0bDhxl/f4S1n5WwvZDpdQ3OKIjjVG9OnL/lQO5ZGBnRvbqoKNzEfkXKvdWoK6+gT0Fx9mcc4xNB4+y4eBR9hVVABATGcHIXu359mX9Gd8vibQ+SbSJ0QpGIvLFVO4trK6+gX3FFezIK2VrbinbckvZkVdGVW09AB3joxnduyNfH9OTtD5JXNizvZajE5HzpnIPEOccxcdr2FtYzu7D5ezKL2dXQTm78ss4UdcAQFx0BMO7t+cbY3txUa8OjOrdgd5J8brVX0T8pnL3U3VtPTkllew/Usm+ouPsK6pgX/Fx9hYe51hl7T+/L6ltDEO6JjIlvQ/De7RjWLf29E9uS5TOl4tIAKjcv4BzjtKqWgrKTpBXWsXh0mryjlVx6GgVuUerOFhSyeGy6s89p3NCDH07t+XLI7oxqEsCA1ISGNw1keSEWB2Ri0iLCYtyr29wVNbUUVVTT0VNPcer6yivrqWsuo7SqhqOVdZytLKWoxU1HKk4QfHxGorKT1B0/AQ1vlMo/xBh0K19G3p0bMPFAzqR2qktfTrF0zspnn6dE2gfrxuFRMR7ASt3M7sOeAqIBOY7555o7n3sOlzGfUs20eAczkGDc9TVO+oaGqird5yoa+BEXT219e6srxUdaXSMj6FTQiydE2JI7RRPl3ZxJCfG0qVdHN07xNG1fRtSEmM19FBEWr2AlLuZRQK/Bb4E5ALrzWylc25nc+4nLiqSwV0SMQMzI8IgMsKIjoggKtKIjYokNjqC2KgI2sZE0SYmkviYSBLjokmMiyIxLor2baLpGB9DfEykTpuISMgI1JH7OCDbObcPwMxeBiYBzVruqZ3b8ttvjm7OlxQRCQmBOr/QA8g56fNc37Z/MrPZZpZpZplFRUUBiiEiEp4CVe6nO7/xuRPfzrl5zrk051xacnJygGKIiISnQJV7LtDrpM97AnkB2peIiJwiUOW+HhhoZn3NLAaYDKwM0L5EROQUAbmg6pyrM7P7gD/TOBRygXNuRyD2JSIi/ypg49ydc28Dbwfq9UVE5Mx0N46ISAhSuYuIhCBz7uy35gc8hFkRcMCPl+gMFDdTnGAQbu8X9J7Dhd7z+enjnDvtWPJWUe7+MrNM51ya1zlaSri9X9B7Dhd6z81Hp2VEREKQyl1EJASFSrnP8zpACwu39wt6z+FC77mZhMQ5dxER+bxQOXIXEZGTqNxFREJQUJe7mV1nZrvNLNvM5nidJ9DMrJeZvW9mWWa2w8we8DpTSzGzSDPbZGZvep2lJZhZBzNbbma7fH/fE7zOFEhm9qDvZ3q7mS01szivMwWCmS0ws0Iz237StiQze9fM9voeOzbHvoK23E9ayu/LwDDgNjMb5m2qgKsDvuecGwqkA/eGwXv+hweALK9DtKCngHecc0OAkYTwezezHsB3gDTn3AgaJxuc7G2qgFkIXHfKtjnAaufcQGC173O/BW25c9JSfs65GuAfS/mFLOdcvnNuo+/jchp/4Xt88bOCn5n1BG4A5nudpSWYWTvgUuB5AOdcjXPumKehAi8KaGNmUUA8Ibr+g3PuQ6DklM2TgAzfxxnATc2xr2Au97Mu5RfKzCwVGAWs9ThKS/g18AOgweMcLaUfUAS84DsVNd/M2nodKlCcc4eAJ4GDQD5Q6pxb5W2qFtXFOZcPjQdwQEpzvGgwl/tZl/ILVWaWALwG/IdzrszrPIFkZl8BCp1zG7zO0oKigNHAs865UUAFzfRf9dbId455EtAX6A60NbM7vE0V/IK53MNyKT8zi6ax2Bc751Z4nacFTAS+amb7aTz1dqWZveRtpIDLBXKdc//4X9lyGss+VF0NfOacK3LO1QIrgIs9ztSSCsysG4DvsbA5XjSYyz3slvIzM6PxPGyWc+6XXudpCc65R5xzPZ1zqTT+Hb/nnAvpozrn3GEgx8wG+zZdBez0MFKgHQTSzSze9zN+FSF8Afk0VgJTfR9PBV5vjhcN2EpMgRamS/lNBKYA28xss2/bo75VryS03A8s9h247APu8jhPwDjn1prZcmAjjSPCNhGi0xCY2VLgcqCzmeUCc4EngGVmNoPGf+huaZZ9afoBEZHQE8ynZURE5AxU7iIiIUjlLiISglTuIiIhSOUuIhKCVO4iIiFI5S4iEoL+F0zePjHb1/+NAAAAAElFTkSuQmCC",
"text/plain": [
"