Lecture 4.3 (Linear Algebra for Machine Learning: Part-I)
\n",
""
]
},
{
"cell_type": "markdown",
"id": "c9fff9cb",
"metadata": {},
"source": [
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "47900344",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "e996a5d6",
"metadata": {},
"source": [
"## Learning agenda of this notebook\n",
"\n",
"**Section I: (Overview of Linear Algebra: Vectors)**\n",
"1. Overview of Vectors \n",
" - Scalars vs Vectors\n",
" - Mathematical and Graphical Representation of Vectors in $\\mathbb{R}^2$\n",
" - Mathematical and Graphical Representation of Vectors in $\\mathbb{R}^3$\n",
" - Hands on Implementation in Python \n",
"2. Magnitude of a Vector (Vector Norms)\n",
"3. Direction of a Vector\n",
"4. Components of a Vector\n",
" - Components of a Vector in $\\mathbb{R}^2$\n",
" - Components of a Vector in $\\mathbb{R}^3$\n",
"5. Two Fundamental Vector Operations\n",
" - Vector Addition\n",
" - Multiplying a Vector with Scalar value\n",
"6. Basis and Unit Vectors\n",
"7. Linear combination and Span of Vectors\n",
"8. Vector to Vector Multiplication\n",
" - Vector Dot Product\n",
" - Vector Cross Product\n",
"\n",
"**Section II: (Overview of Linear Alagebra: Matrices)**\n",
"1. Overview of Matrices\n",
"2. Matrices and its Types\n",
" - Row Vector\n",
" - Column Vector\n",
" - Zero Matrix\n",
" - Ones Matrix\n",
" - Random Integer Matrix\n",
" - Square Matrix\n",
" - Symmetric Matrix\n",
" - Triangular Matrix\n",
" - Diagonal Matrix\n",
" - Identity Matrix\n",
" - Scalar Matrix\n",
" - Orthogonal Matrix\n",
"3. Matrix Operations\n",
" - Matrix Addition\n",
" - Matrix-Scalar Multiplication\n",
" - Matrix Multiplication (Hadamard Product)\n",
" - Matrix Multiplication (Dot Product)\n",
" - Matrix-Vector Multiplication\n",
" - Frobenius Norms\n",
" - Transpose of a Matrix\n",
" - Determinant of a Matrix\n",
" - Inverse of a Matrix\n",
" - Trace of a Matrix\n",
" - Rank of a Matrix\n",
"\n",
"**Section III: (Solving System of Linear Equations)**\n",
"1. An overview of Linear Equations\n",
" - What is a Linear Equation?\n",
" - What is a system of Linear Equations?\n",
"2. How to solve a system of Linear Equations?\n",
" - Substitution strategy\n",
" - Elimination strategy\n",
" - Graphing strategy\n",
" - Consistent vs Inconsistent System of Linear Equations\n",
" - Plotting a Linear Equation with Three Variables\n",
" - Solving set of Three Linear Equations with Three variables\n",
"3. Solving System of Linear Equations using Matrix Algebra\n",
" - Writing a system of Linear Equations in Matrix form\n",
" - Solving system of Linear Equations using Gaussian Elimination Method\n",
" - Solving system of Linear Equations using Gauss Jordan Method\n",
" - Solving system of Linear Equations using Cramer's Rule\n",
" - Solving system of Linear Equations using Matrix Inverse Method\n",
" - Limitations of Matrix Inversion Method\n",
"4. Categories of System of Linear Equations\n",
" - Standard systems\n",
" - Overdetermined systems\n",
" - Underdetermined systems\n",
"5. Solving Inconsistent Overdetermined System of Linear Equations using Least Squares Method\n",
" - Modeling Linear Equations in Machine Learning with `2` variables\n",
" - Modeling Linear Equations in Machine Learning with `m` variables\n",
" - Simple Linear Regression using Least Squares Method\n",
" - Multiple Linear Regression using Ordinary Least Squares (OLS) Method\n",
"\n",
"\n",
"**Section IV: (Linear Transformation and Matrices)**\n",
"\n",
"**Section V: (Eigen Decomposition and its Applications)**\n",
"\n",
"**Section VI: (Singular Value Decomposition and its Applications)**"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d787763a",
"metadata": {},
"outputs": [],
"source": [
"# Unlike the other modules, we have been working so far, you have to download and install...\n",
"# To install this library in Jupyter notebook\n",
"import sys\n",
"!{sys.executable} -m pip install -q --upgrade pip"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "3587d30f",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import numpy.linalg\n",
"import math\n",
"import scipy\n",
"from matplotlib import pyplot as plt\n",
"from plot_helper import * # Helper functions: plot_vector, plot_linear_transformation, plot_linear_transformations"
]
},
{
"cell_type": "markdown",
"id": "02af6b34",
"metadata": {},
"source": [
"# Section 1: (Overview of Linear Alagebra: Vectors) \n",
"Some of the codes of this notebook are adapted from:\n",
"- [Jon Krohn's](https://github.com/jonkrohn).\n",
"- [Frank Cleary's](https://gist.github.com/frankcleary).\n",
"- [Engineers Code](https://github.com/engineersCode/EngComp4_landlinear)."
]
},
{
"cell_type": "markdown",
"id": "1feba2dc",
"metadata": {},
"source": [
"## 1. Overview of Vectors"
]
},
{
"cell_type": "markdown",
"id": "e9a72e17",
"metadata": {},
"source": [
"### a. Scalar vs Vectors\n",
"\n",
"- A quantity that has magnitude but no particular direction is called scalar. For example, length, speed, mass, density, pressure, work, power, temperature, area, volume.\n",
"- A quantity that has magnitude as well as direction is called vector. For example, displacement, velocity, weight, force.\n",
"- For example, to describe a body’s velocity completely, we will have to mention its magnitude and direction. This means that we will have to mention how fast it is going in terms of distance covered per unit time and describe what direction it is headed. So, if we say a car is moving at 40 km/hr. This statement only describes the speed of the body. If someone says a car is moving at 40 km/hr and is headed North. This statement is describing the velocity of the car. It tells us the magnitude by which the car is moving and the direction in which it is headed.\n",
"\n",
"\n",
"\n",
"- We come accross the concept of vectors in the domains of physics, engineering, mathematics, computer science and more. Each field's interpretation of what a vector is a bit different:\n",
" - In **physics**, we represent a vector as an arrow of specific length, representing its magnitude; and drawn at a specific angle, representing its direction. It can represent directional quantities like velocity, force, acceleration.\n",
" - In **computer science**, a vector is an ordered list of numbers, stored in order. For example the price, area and number of bedrooms in a house. Or may be the age, weight and blood pressure of a person.\n",
" - In **mathematics**, vectors are generic objects that behave in a certain way, when they are added or scaled: $\\mathbf{u}+\\mathbf{v}$, $\\alpha\\mathbf{v}$.\n",
"\n",
"- Vectors can be 2-dimensional, 3-dimensional and so on to N-dimensional. The two and three dimensional vecgtors are pretty easy to visualize. If we are able to understand and visualize the vector operations in 2-dimensions, we can map the concepts to larger dimensions as well. For example, to model the age, weight, daily hours of sleep, weekly hours of exercise, and blood pressure of an individual, we need a five dimensional vector."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "931c5502",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "a29bf5e9",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "ae5c7095",
"metadata": {},
"source": [
"### b. Mathematical and Graphical Representation of Vectors in $\\mathbb{R}^2$\n",
"#### Algebraically\n",
"- Algebraically, vectors are often represented using a lowercase character, having comma separated list of numbers written horizontally or may be numbers written from top to bottom. \n",
"- The **length** of the vector is the number of scalar values in the vector, and is also called the `order/rank/degree/dimension of the vector`. \n",
"- For example, in $\\mathbb{R}2$ space, algebraically a vector $\\overrightarrow{\\rm v}$ can be written as: \n",
"\n",
"$\\hspace{2 cm}\\overrightarrow{\\rm v} = (a, b) = \\begin{bmatrix} a \\\\ b \\end{bmatrix} \\hspace{2 cm}\\overrightarrow{\\rm v} = (2, 5) = \\begin{bmatrix} 2 \\\\ 5 \\end{bmatrix}$\n",
"\n",
"$\\hspace{2 cm}\\overrightarrow{\\rm v} = a\\hat{i} + b\\hat{j} \\hspace{3 cm} \\overrightarrow{\\rm v} = 2\\hat{i} + 5\\hat{j}$\n",
"\n",
"$\\hspace{2 cm}\\overrightarrow{\\rm v} = a\\begin{bmatrix} 1 \\\\ 0 \\end{bmatrix} + b\\begin{bmatrix} 0 \\\\ 1 \\end{bmatrix} \\hspace{1.5 cm} \\overrightarrow{\\rm v} = 2\\begin{bmatrix} 1 \\\\ 0 \\end{bmatrix} + 5\\begin{bmatrix} 0 \\\\ 1 \\end{bmatrix} $\n",
"\n",
"\n",
"\n",
"#### Graphically/Geometrically\n",
"- One can think of a vector as a point in space. Graphically/Geometrically, vectors can be represented by a directed line segment in cartesian coordniate system, whose length is the magnitude of the vector and the angle represents its direction.\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67dbbc98",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "d64aded7",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "d31de3ff",
"metadata": {},
"source": [
"### c. Mathematical and Graphical Representation of Vectors in $\\mathbb{R}^3$\n",
"\n",
"#### Algebraically\n",
"\n",
"- Algebraically, vectors of rank 3 consists of three scalar values written as comma separated list of numbers horizontally or may be from top to bottom. \n",
"- The **length** of the vector is the number of scalar values in the vector, and is also called the `order/rank/degree/dimension of the vector`, in this case it is three. \n",
"\n",
"- Algebraically, in $\\mathbb{R}3$ space, a vector $\\overrightarrow{\\rm v}$ can be written as: \n",
"\n",
"$\\hspace{2 cm}\\overrightarrow{\\rm v} = (a, b, c) = \\begin{bmatrix} a \\\\ b \\\\ c \\end{bmatrix} \\hspace{5 cm}\\overrightarrow{\\rm v} = (2, 5, 3) = \\begin{bmatrix} 2 \\\\ 5 \\\\ 3 \\end{bmatrix}$\n",
"\n",
"$\\hspace{2 cm}\\overrightarrow{\\rm v} = a\\hat{i} + b\\hat{j} + c\\hat{k} \\hspace{5 cm} \\overrightarrow{\\rm v} = 2\\hat{i} + 5\\hat{j} + 3\\hat{k}$\n",
"\n",
"$\\hspace{2 cm}\\overrightarrow{\\rm v} = a\\begin{bmatrix} 1 \\\\ 0 \\\\0 \\end{bmatrix} + b\\begin{bmatrix} 0 \\\\ 1 \\\\0 \\end{bmatrix} + c\\begin{bmatrix} 0 \\\\ 0 \\\\1 \\end{bmatrix} \\hspace{3 cm} \\overrightarrow{\\rm v} = 2\\begin{bmatrix} 1 \\\\ 0 \\\\0 \\end{bmatrix} + 5\\begin{bmatrix} 0 \\\\ 1 \\\\0 \\end{bmatrix} + 3\\begin{bmatrix} 0 \\\\ 0 \\\\1 \\end{bmatrix}$\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"#### Graphically/Geometrically\n",
"- A 3-D coordinate system has 3 dimensions or can be regarded as having 3 perpendicular axes: x, y, and z-axes. Such a system is called a 3-dimensional rectangular coordinate system.\n",
"- Note, the third axis is the Z-axis, and all the three axis are perpedicular to each other.\n",
"- The vector $\\overrightarrow{\\rm v}$ is shown in bold red having its three components a, b and c along the x, y and z axis respectively.\n",
"- We can visualize vectors upto three dimensions, however, beyond three dimensions, we normally use algebraic notations."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6f5e3581",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "f07fe69d",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "a5c1609d",
"metadata": {},
"source": [
"### d. Hands on Implementation in Python"
]
},
{
"cell_type": "markdown",
"id": "67bbd2fb",
"metadata": {},
"source": [
"**Example 1:** Creating Vector $[3,2]$ having tail at origin"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "6e03dde5",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAAFkCAYAAAAXG0EgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAB7CAAAewgFu0HU+AAAgEElEQVR4nO3df3RU5eHn8c9Mwo9kklBl3URjFJWhaUAWCGIRMUEU0C6g3doNmC+0fhXxB+pRaDlYDlhRs9ZaBVFQUBBcKazHIuJaiyV63KI2iVpNshKrVBITV1DM5AYSmbn7B5KvSAgzIfc+D+H9OodzZubeuc8ncycfnrlzZxJwXVcAAPOCpgMAAA6gkAHAEhQyAFiCQgYAS1DIAGAJChkALEEhA4AlKGQAsASFDACWoJABwBIUMgBYgkIGAEtQyABgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWIJChu8CgcDNgUDADQQChaazADahkOGrQCBwmqQ5pnMANqKQ4bclku41HQKwEYUM3wQCgYmSvpH0Z9NZABslmw6AE0MgEAhJukfSeEm9OrmN04+ySpKkUyQ1SGpwXXd/Z8YBTKGQ4Ze7JS1zXbc+EAj06+Q2diawbo6k2k6OAxgRcF3XyMAvvviim5qaqmDQnqMmsVhMzc3NIld84s21fft2LV68WIsXL1YwGFRDQ4OmTJmiP/zhDxoyZEjc440ZMybuddetW6fMzMy41/fa8b4P/WZjrlgsposvvjjg5RjGZsgpKSnKy8tTKBQyFeEwjuOosrKSXHGKN9eWLVuUnJysBQsWSJL27dsnSXryySfVp08fLVmyROecc85Rx6uuru5weUNDQ1tph8NhhcPheH8Uzx3v+9BvNuZyHMfzMYwVclJSkkKhkNLT001FaBe5EhNPrkWLFmnRokVt13fs2KGzzjpLixcvVmFhYdxj5ebmdrg8LS2t7XJqaupx+ViZQC572PFaAABAIcNft912m4qKig67DICzLOCzhx56yHQEwFrMkAHAEhQyAFiCQgYAS1DIAGAJChkALEEhA4AlKGQAsASFDACWoJABwBIUMgBYgkIGAEtQyABgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWIJCBgBLUMgAYAkKGQAsQSEDgCUoZACwBIUMAJagkAHAEhQyAFgi2XQAnBg2btyoZcuWqbW1VS0tLWpubtacOXM0ZcoU09EAa1DI8MVjjz2mqVOnatq0aZKkTZs2afLkyRo4cKAGDx5sOB1gBw5ZwBf33HOPpk6d2na9sLBQruvq448/NpgKsAszZPgiPz+/7fI333yjBx54QHl5ebrkkkvi3kZtbW2Hy+vr6zudD7CBsUKORqNyHMfU8O1yHIdcCehMrttvv10bNmxQbm6unnvuObmuq0gkEtd9c3Jy4h6nubk57u36oTvtQz/YmMtxHKWnp3s6RsB1XU8HOJIXX3zRTU1NVTBoz1GTWCym5uZmkSs+nc0VjUb11FNPacuWLVq6dKn69u0b1/3GjBkT9xjr1q1TZmZm3Ot7rbvtQ6/ZmCsWi+niiy8OeDmGsRlySkqK8vLyFAqFTEU4jOM4qqysJFecjiXX8OHDNXDgQL3++utatGhRXPeprq7ucHlDQ0NbaYfDYYXD4YQyeak77kMv2ZjLj9m6sUJOSkpSKBTy/CVAosiVmHhztba2qmfPnofc9sMf/lAfffRR3D9Tbm5uh8vT0tLaLqemph63j5XfyGUPO14LoNsbNmzYYbfV19frtNNOM5AGsBOFDF9UVVVp8+bNbdfXrl2rDz/8UNOnTzeYCrALp73BFw8//LDuuece3XfffYrFYgoEAnrhhRd04YUXmo4GWINChi9mzZqlWbNmmY4BWI1DFgBgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWIJCBgBLUMgAYAkKGQAsQSEDgCUoZACwBIUMAJagkAHAEhQyAFiCQgYAS1DIAGAJChkALEEhA4AlKGQAsASFDACWoJABwBIUMgBYgkIGAEtQyABgCQoZACyRbDoAThzr16/XihUrFI1G1djYqH79+ul3v/ud+vXrZzoaYAVmyPBNcXGx7rjjDr366qt66623lJKSogkTJqilpcV0NMAKFDJ8M3nyZI0fP16SFAwGdcstt+jDDz9URUWF4WSAHThkAd9s2LDhkOu9e/eWpLhnyLW1tR0ur6+v71wwwBLGCjkajcpxHFPDt8txHHIl4Fhzbd26VaeeeqoGDx6sSCRy1PVzcnLi3nZzc3Nc2/RLd92HXrExl+M4Sk9P93QMY4W8d+9eVVVVKRi056hJLBYjVwKOJVdra6vuv/9+3XDDDfrHP/7R5dlqamrU2NjY5dvtrO64D7val017tW17nd771//TNYXnKq1HwIpcB8ViMWVlZXk6hrFCTklJUV5enkKhkKkIh3EcR5WVleSK07HkmjlzpqZMmaLbbrst7vtUV1d3uLyhoUFjxoyRJIXDYYXD4YQyeak77sNj5bqu/u/Oz/XSW5Xa/NYHKtv+qYLBgJ5bcJ1+POB06x4vP2brxgo5KSlJoVDI85cAiSJXYjqTa+7cuerTp4/uv//+hMbKzc3tcHlaWlrb5dTU1G7xWPnBz1z7o1G98f5H2vh/3tULf3tPH3+265Dlf7jx57riouGKRCLWPl5e4k09+KqkpEQ7d+7UmjVrJEnl5eWSpPz8fJOx4IOX3nxf/3bfk/qysf2Z5nU/Ga1ZP73Y51R2sePgDE4Iy5Yt09q1azVr1ixVVFSorKxMmzZt0vvvv286Gnxw+Y/P1f8uuUWZJ2UctuyiwWE9cusUBQIBA8nswQwZvohEIrrpppsUi8U0cuTIQ5Y99dRThlLBT5/t2qO712zW518d+mZrv6y+eu63M9WzB3XEIwBfpKenKxqNmo4BA1zX1dq/vKlblvxRe5qaJR0o4R0Nu5WW0kub7r1Z/6nPiXOcuCMUMgDPfLZrj65/cK1e3Hbg1MYfpKVq8az/rjP+88kac/uD+p+/uVaDzso2nNIeFDKALtferHjiBYO1/PZindr3B/r0892677orNfGC/2I4qV0oZABd6kiz4uJLf9z2pt3pp5ykXxWNNxnTShQygC5xtFnxd9ny6TvbUMgAjlk8s2IcHYUMoNMSmRXj6ChkAJ3CrLjrUcgAEsKs2DsUMoC4MSv2FoUM4KiYFfuDQgbQIWbF/qGQAbSLWbH/KGQAh2FWbAaFDKANs2KzKGQAkpgV24BCBk5wruvq2a1l+vUTG5kVG0YhAyew+t1f6851r2nb9s8kMSs2jUIGTkBtx4oXr9MeZ68kZsU2oJCBE8z3jxWn9e6h39/wM103sZBZsWEUMnCCaO8MistG5OnfL/yhxhUOp4wtQCEDJ4AjnUEx+ccDVVFRYTgdDqKQgW7saOcVRyIRwwnxXRQy0E1xXvHxh0IGuhk+bXf8opCBboRZ8fGNP/0KX7W2tmru3LlKTk7Wjh07TMfpNlzX1ZpXtmngLxe2lfHECwaratVC/du4kZTxcYIZMnyzY8cOTZkyRQMGDFA0GjUdp9tgVtx9UMjwTVNTk9asWaPa2lo9/fTTpuMc9zhW3P1QyPDNoEGDJEm1tbWGkxz/mBV3TxQyjhtHK/L6+nqfkpjDrLh7M1bI0WhUjuOYGr5djuOQKwGdzdXcfKBImpqaEvpgQk5OTkJj2PShh67Yh/W7v9atj/4vvfz3KknSD0Ip+h8zrlBRYb4CgUCnft7u9tzykuM4Sk9P93QMY4W8d+9eVVVVKRi050SPWCxGrgR0Ntf27dslSR988IF27drlSbaamho1NjZ6su3OOJZ96Lqu/vKPHVrycpma9n0jSbpgQLZu/68j1Dc9cEwffe5uzy0vxWIxZWVleTqGsUJOSUlRXl6eQqGQqQiHcRxHlZWV5IpTZ3MdnCEPGjRIZ555Ztz3q66u7nB5Q0ODxowZI0kKh8MKh8Nxb9trnX2sjjYrNpXLazbm8mO2bqyQk5KSFAqFPH8JkChyJaYzuVJTUyVJaWlpCd0vNze3w+VpaWmHjHE8P1Z+HivuTs+t4x1v6gGW+f4ZFCelp2rxrCJdfcn5nEHRzVHI8E1ra6vGjRunPXv2SJKKioqUk5OjDRs2mA1mCc6gAIUM3/Ts2VOlpaWmY1iJWTEkChkwilkxvotCBgxhVozvo5ABnzErxpFQyICP6nd/rTvuW82sGO2ikAEfuK6rV977RI898Lz2OHslMSvG4ShkwGOf7dqjf79/Vdun7ZgV40goZMAj7R0rvmxEnlb+6hfMitEuChnwwOHfV5yiGy8dol//4r8pIyPDcDrYikIGutCRzqD4/YwrVfdJDYco0CEKGegiHZ1X3NTUpLpPDAeE9Shk4BhxXjG6CoUMHAM+bYeuRCEDncCsGF6gkIEEMSuGVyhkIE7MiuE1ChmIA7Ni+IFCBjrArBh+opCBI2BWDL9RyMD3MCuGKRQy8B3MimEShQyIWTHsQCHjhMesGLagkHHCYlYM21DIOCExK4aNKGScUJgVw2YUMk4YzIphOwoZvnr++ed17733qnfv3goGg3r00Uc1cOBAT8dkVozjBYUM37z99tuaPn26ysvLFQ6H9fTTT2v8+PGqrq5Wenq6J2MyK8bxJGg6AE4cJSUl+slPfqJwOCxJKi4u1v79+7Vq1aouH8t1Xa15ZZsG/nJhWxlPvGCwKp9aqOJLf0wZw0oUMnzz6quvavjw4W3Xg8Gg8vPztWXLli4f68bFf9S0+57SnqZmnZSeqjXzrtHGRTdxiAJW8/2QRSAQSJaUtX79etXV1SktLc3vCEfU1NSkL774glxxSiTXV199pcbGRvXo0UO1tbVtt6enp+u999475LYjqa+v73D5559/3na59K13pV6pumT4j1Qy46fKPKmP6urqjjqGV7rDPvSTjbmampr0ox/96HRJDa7r7vdijIDrul5s98gDBgKnS9rp66AA0HVyXNc9+gyiE0wcssgyMCYAdBXPOszEWRZfSFJWVpZWr16tM844w0CEw33xxRf6+c9/LknkikNnco0YMUI33HCDfvnLX7bddv311ys5OVlLly496v0bGho6XL59+3Zdf/31kqS1a9cqPz//qNv0Q3fah36wMdfBTN8+B7/wahwThRyVDvxynXHGGcrNzTUQ4XBpaWltv/DkOrrO5Lrkkku0c+fOtnVd19X27dt15513xnX/o60TCoXaLmdnZx/Xj5UfyBW/72bStx3mBc6ygG/mzp2rzZs366OPPpIkPfPMM0pKStL06dMNJwPswAdD4JsRI0Zo1apVKioqUkpKioLBoP785z979qEQ4HhDIcNXV155pa688krTMQArccgCACxBIQOAJShkALAEhQwAlvD9Tb1vP3IYKC0tdbOzs/0e/ohOP/10NTY2qry8XOQ6OhtznXrqqW2Xs7Ls+UCojY+VRK5EHMyUnp7u6dcEMkMGAEtQyABgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWMKaQo7FYjr//PPVr18/01HU0tKi+fPn68ILL9TYsWM1dOhQXXHFFW3f42vKl19+qYULF+rCCy9UYWGhhg4dqnvvvVf793vy9xYTVlNTowsuuECFhYWmo+jqq69WQUGBKisrTUdRa2urFixYoLFjx+pf//qX6TiSpPXr12vcuHGaOHGiZs6cqWnTpmnHjh1GM23cuFGXXXaZxo4dq3HjxmnGjBnasGGD0UzfFwgEbg4EAm4gECj0YvvWfP3m0qVLtX37dvXp08d0FO3Zs0crV67UO++8o8zMTMViMRUVFamoqEhlZWXGcr3yyitav369tm3bpj59DvwV5WHDhqm1tVULFy40lkuS1qxZo0cffVRJSUnGMrzzzjttl5955hm9/fbbGj9+vKqrq4195/KOHTs0ZcoUnXXWWYrFYkYytKe4uFibNm3SBRdcoL///e964oknNGHCBL333nvq1auXkUyPPfaYpk6dqmnTpikSiWjJkiW69tprNXz4cA0ePNhIpu/69q+ez/FyDCtmyHV1dVq5cqVmzJhhOook6aSTTtLmzZuVmZkpSQoGgxo9erTxGfLJJ5+s2bNnt/2nlZ2drauuukrPPvus0VyS1LdvX7322mvq37+/sQyPPvroIdeLi4u1f/9+rVq1ykwgHfjT8WvWrFFxcbGxDO2ZPHmyxo8fL+nA83vmzJn68MMPVVFRYSzTPffco6lTp7ZdHzJkiFzX1ccff2ws03fNmTNHku71cgwrCvmWW25RSUmJUlJSTEeRJPXs2VNDhw5tu15XV6fVq1fr1ltvNZhKGjdunK655ppDbuvdu7daWloMJfoPl19+uXr27Gk0wxtvvHHI9WAwqPz8fG3ZssVQImnQoEFG/5M6ku8fCujdu7ckGX0u5efnKzn5wIv2b775Rn/84x+Vm5urSy65xFimgzZt2nQw25+9HMd4IR/8QSdMmGA6ymEOHhI455xzNH78eN11112mIx1m27ZtbX+h90S2e/duNTU1HXZ7VlaWPvnkEwOJji9vv/22TjvtNI0aNcp0FN100006++yzVV5erueff15paWlG8ziOozvvvFMlJSWej2W0kJuamjRv3jw99NBDJmMcUXZ2tioqKvTPf/5TL7/8sq677jrTkQ7x17/+VbW1tfrNb35jOopxzc3N7d7eq1evIy7DAa2trXr44Yf1yCOPqEePHqbjaOnSpfrkk080ZMgQjRs37uCxW2Pmz5+vmTNn+vINgp4UciAQKPn2ncgj/vv000+1aNEizZw585CvTfTS3LlzFQgEjvgvIyNDn3766WH3y87OVklJiVasWOHJu/adyVVXV6cbb7xRGzduVEZGRpdn6mwuU1JTU9u9vaWl5YjLcMCDDz6on/70p1b9rcPk5GRdc801isVievDBB43lqKio0FtvvaWZM2f6Mp5XZ1ncK+mRjlbIzs7e+frrr+v9999vO561Y8cONTQ0qLCwUP3799eKFSu6NNS8efN08803H3F5U1OT6urqFI1GFY1GDzljIDc3V5JUVVWlgQMHGsl10O7du3XFFVdo+fLlGjJkSJdmOZZcJvXt21fp6emKRCKH3N7Q0KCzzz7bUCr7LViwQL169dL8+fNNR1Fra+sh70MEg0H1799fVVVVxjJt3rxZe/fu1cUXX6xoNCpJ675d9FAgENgj6VrXdbvs3X5PCtl13UZJjR2tU1paqr/97W+HnI60cOFCrVq1SqWlpV7EUkZGRoezyUgkooaGBq1bt05NTU2aPXt227KDL5tOO+00Y7kOXp40aZIWLFiggoICSdLjjz/uyRkqieSywahRo/Tyyy+3XXddVxUVFbrzzjsNprJXSUmJamtr296sLi8vl3TgzTUThg0bpg8++OCQ2xoaGjR69GgjeaQDhysO/mcViUSUkZFRJOkTSbe5rlva1eMZf1PPVk8++aR27dolSdq3b5/uvvtuDRo0SOedd56xTPv27dOkSZM0cuRIZWVlqaysTGVlZVq+fLmxTDa58cYbD7n+zDPPKCkpSdOnTzeUyF7Lli3T2rVrdf3116umpkYVFRXatGmT3n//fWOZqqqqtHnz5rbrf/nLX1RTU3Ni7T/XdY3827p1q9vY2Oi6ruvW19e7BQUF7plnnun26tXLLSgocJ966inXb42Nje7WrVvdqqoqd9asWe6QIUPc0aNHu0OHDnWLi4vdTz/91PdM3831wAMPuJLa/WcyV2Njo7tx40a3oKDAzczMdPv06eMWFBS4K1as8DXPzp072x6PYcOGuRdddJH7wQcf+Jrh+1paWtyCggL33HPPdSW5w4cPd3/2s58ZzdTY2OgGg8F2n0cmfu8OWrx4sTty5Eh31KhR7ogRI9yBAwe669evN5bn+2644QZX0pvfPlbvSlrndnEvBlzX9fm/gANKS0vd/Px8Y5+gak8kElF5ebnIFR/bctXW1ionJ0eSVF1d3Xbc3wa2PVYHkSt+kUiEv6kHACcKChm+senLhwAbUcjwxZo1azRt2jQFgzzlgCPhtwO+sOHLhwDbWfP1m+jeLr/88mPeRm1tbYfLTX/EFjhWxgo5Go3KcRxTw7fLcRxyJaAzub755htFo9HDPlEXj4NnUMSjubm5U2N4pTvtQz/YmMtxHM/P+DBWyHv37lVVVZVVxxRjsRi5EtCZXLt37247pclLNTU1amzs8MOivupO+9APNuaKxWKef8GQsUJOSUlRXl6eQqGQqQiHcRxHlZWV5IrTvHnz9MgjHX5licrKyjRgwIC263379pXjOJ36eG51dXWHyxsaGjRmzBhJUjgcVjgcTngMr9i6D8kVPz9m68YKOSkpSaFQyJqTvg8iV/zmzp2riy66SOeee+4Rv7M2Kyur7UvHJalHjx5KSkrq1M9xtA96fDdDamqqVY+VZOc+lMhlE97UQ6dlZGTolFNOUXZ29gn1SwN4xY6DMwAAChn+eOGFF1RYWKiXX35Z7777rgoLC7Vy5UrTsQCrcMgCvpg0aZImTZpkOgZgNWbIAGAJChkALEEhA4AlKGQAsASFDACWoJABwBIUMgBYgkIGAEtQyABgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWIJCBgBLUMgAYAkKGQAsQSEDgCUoZACwBIUMAJagkAHAEhQyAFiCQgYAS1DIAGAJChkALJFsOgC6vy+//FKLFy/Wli1blJycrK+//lpXXXWVfvWrXyk5macgcBC/DfDcSy+9pPXr12vbtm3q06eP6urqNGzYMLW2tmrhwoWm4wHW4JAFPNe3b1/Nnj1bffr0kSRlZ2frqquu0rPPPms4GWAXZsjw3GWXXXbYbb1791ZLS0tC26mtre1weX19fULbA2xjrJCj0agcxzE1fLscxyFXAo4l1xtvvKErrrhCkUgk7vvk5OTEvW5zc3NC2/Zad9yHXrIxl+M4Sk9P93QMY4W8d+9eVVVVKRi056hJLBYjVwI6m6uiokI7duzQ/PnzVV5e7km2mpoaNTY2erLtzuhu+9BrNuaKxWLKysrydAxjhZySkqK8vDyFQiFTEQ7jOI4qKyvJFad58+bpkUce6XCdsrIyDRgwoO36Z599pmXLlum5557T4MGDExqvurq6w+UNDQ0aM2aMJCkcDiscDie0fS/Zug/JFT8/ZuvGCjkpKUmhUMjzlwCJIlf85s6dq4suukjnnnuu0tLS2l0nKyur7dS23bt3q7i4WE888YRGjRqV8Hi5ubkdLv9uhtTUVKseK8nOfSiRyya8qYdOy8jI0CmnnKLs7Oyj/tJEIhFNmjRJCxYsUEFBgSTp8ccf14wZM/yIChwX7Dg4g25t3759mjRpkkaOHKmsrCyVlZWprKxMy5cvNx0NsAozZHhu5cqVKi0tVWlpqX7/+9+bjgNYixkyPHfTTTfJdd12/wH4DxQyAFiCQgYAS1DIAGAJChkALEEhA4AlKGQAsASFDACWoJABwBIUMgBYgkIGAEtQyABgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWIJCBgBLUMgAYAkKGQAsQSEDgCUoZACwBIUMAJagkAHAEhQyAFgi2XQAdH8tLS1atGiRtm7dql69eunLL7/UmWeeqQceeED9+/c3HQ+wBjNkeO6rr77SypUr9dxzz+nVV19VeXm5evbsqaKiItPRAKtQyPDcySefrM2bNyszM1OSFAwGNXr0aH300UeGkwF2oZDhuZ49e2ro0KFt1+vq6rR69WrdeuutBlMB9uEYMnxTV1eniRMnqqqqSnfccYfuuuuuhO5fW1vb4fL6+vpjiQcYZ6yQo9GoHMcxNXy7HMchVwISzZWRkaHXXntNn332mYqKilRXV6clS5bEPV5OTk7c6zY3NysSicS9vte6yz70i425HMdRenq6p2MYK+S9e/eqqqpKwaA9R01isRi5ErB8+XKtW7euw3VWr16tM84447Dbr776as2ZM0cFBQU666yzujxbTU2NGhsbu3y7nWXrPiRX/GKxmLKysjwdw1ghp6SkKC8vT6FQyFSEwziOo8rKSnLF6be//a0mT56sAQMGKDU1td11MjMzFQgEJElJSUmH3D5nzhwlJSUpPz8/rvGqq6s7XN7Q0KAxY8ZIksLhsMLhcFzb9YOt+5Bc8fNjtm6skJOSkhQKhTx/CZAociUmKytL4XC4w1yrVq3Srl27NHv27LbbDh5OOOecc+L+mXJzcztcnpaW1nY5NTXVusfK1n1ILnvY8VoA3d6TTz6pXbt2SZL27dunu+++W4MGDdJ5551nOBlgD86ygOfGjh2riooKXXrppUpPT1dTU5MGDhyol156ST179jQdD7AGhQzP5eTkaPHixaZjANbjkAUAWIJCBgBLUMgAYAkKGQAsQSEDgCUoZACwBIUMAJagkAHAEhQyAFiCQgYAS1DIAGAJChkALEEhA4AlKGQAsASFDACWoJABwBIUMgBYgkIGAEtQyABgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWIJCBgBLUMjwVSwW0/nnn69+/fqZjgJYh0KGr5YuXart27ebjgFYiUKGb+rq6rRy5UrNmDHDdBTAShQyfHPLLbeopKREKSkppqMAVko2HQAnhk2bNik5OVkTJkzQm2++2alt1NbWdri8vr6+U9sFbGGskKPRqBzHMTV8uxzHIVcC4s3V1NSkuXPn6k9/+pMikYhaWlrkuq4ikUhC4+Xk5MS9bnNzc8Lb99Lxvg/9ZmMux3GUnp7u6RgB13U9HQDdVyAQKJH066Os9iNJ10v6yHXdpd/eb6GkX7iu2y/B8RJ5sua4rtvxlBqwDIWMTgsEAhmSMo6yWoOkcklfS4p9e1s/SVmS3tSBor42zvFOP8oqSZJO+XbMBtd198ezXcAWFDJ819kZMtDdcZYFAFiCQoZvAoFAViAQKJX0C0lZgUCgNBAI/MJoKMAiHLIAAEswQwYAS1DIAGAJChkALEEhA4AlKGQAsASFDACWoJABwBIUMgBYgkIGAEtQyABgCQoZACxBIQOAJShkALAEhQwAlqCQAcASFDIAWIJCBgBLUMgAYAkKGQAsQSEDgCUoZACwBIUMAJb4/3yKMOHL8FQNAAAAAElFTkSuQmCC\n",
"text/plain": [
"