{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Hyperbolic Embedding of Graphs and Clustering" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lead authors: Thomas Gerald and Hadi Zaatiti.\n", "\n", "## Introduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From social networks to parse trees, knowledge graphs to protein interaction networks, Graph-Structured Data is endemic to a wide variety of natural and engineered systems. Often, understanding the structure and/or dynamics of these graphs yields insight into the systems under investigation. Take, for example, the problems of finding key influencers or distinct communities within social networks. \n", "\n", "The goal of graph embedding is to find a way of representing the graph in a space which more readily lends itself to analysis/investigation. One approach is to identify points in a vector space with nodes of the graph in such a way that important relations between nodes are preserved via relations between their corresponding points.\n", "\n", "There are a wide variety of methods which approach this problem in different ways and for different aims, say for clustering or for link prediction. Recently, the embedding of Graph Structured Data (GSD) on manifolds has received considerable attention. In particular, much work has shown that hyperbolic spaces are beneficial for a wide variety of tasks with GSD [[ND2017]](#References). This tutorial shows how to learn such embeddings using the Poincaré Ball manifold and the well-known 'Karate Club' social network dataset with `geomstats`. This data and several others can be found in the `datasets.data` module of the project's github repository. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![KarateEmbedding](figures/karate_embedding_iterations.gif \"segment\")\n", "*Learning a Poincaré disk embedding of the Karate club graph dataset*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\n", "\n", "\n", "We start by importing standard tools for logging and visualization, allowing us to draw the embedding of the GSD on the manifold. Next, we import the manifold of interest, visualization tools, and other methods from `geomstats`." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys\n", "import warnings\n", "\n", "sys.path.append(os.path.dirname(os.getcwd()))\n", "warnings.filterwarnings(\"ignore\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO: Using numpy backend\n" ] } ], "source": [ "import logging\n", "import matplotlib.pyplot as plt\n", "\n", "import geomstats.backend as gs\n", "import geomstats.visualization as visualization\n", "\n", "from geomstats.datasets.utils import load_karate_graph\n", "from geomstats.geometry.poincare_ball import PoincareBall" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Parameters and Initialization\n", "\n", "We define the following parameters needed for embedding:\n", "\n", "| Parameter | Description |\n", "|:------|:------|\n", "| random.seed | An initial manually set number for generating pseudorandom numbers|\n", "| dim | Dimensions of the manifold used for embedding |\n", "|max_epochs|Number of iterations for learning the embedding |\n", "|lr| Learning rate|\n", "|n_negative| Number of negative samples|\n", "|context_size| Size of the considered context for each node of the graph|\n", "\n", "Let us discuss a few things about the parameters of the above table.\n", "The number of dimensions should be high (i.e., 10+) for large datasets \n", "(i.e., where the number of nodes/edges is significantly large). \n", "In this tutorial we consider a dataset that is quite small with only 34 nodes. The Poincaré disk of only two dimensions is therefore sufficient to\n", "capture the complexity of the graph and provide a faithful representation.\n", "Some parameters are hard to know in advance, such as `max_epochs` and `lr`. These should be tuned specifically for each dataset.\n", "Visualization can help with tuning the parameters. Also, one can perform a grid search to find values of these parameters which maximize some performance function. In learning embeddings, one can consider performance metrics such as\n", "a measure for cluster seperability or normalized mutual \n", "information (NMI) or others.\n", "Similarly, the number of negative samples and context size can also be thought of as hyperparameters and will\n", "be further discussed in the sequel. An instance of the `Graph` class is created\n", " and set to the Karate club dataset." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "gs.random.seed(1234)\n", "dim = 2\n", "max_epochs = 100\n", "lr = 0.05\n", "n_negative = 2\n", "context_size = 1\n", "karate_graph = load_karate_graph()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Zachary karate club network was collected from\n", "the members of a university karate club by Wayne Zachary\n", "in 1977. Each node represents a member of the club,\n", "and each edge represents an undirected relation between\n", "two members. An often discussed problem using this dataset\n", "is to find the two groups of people into which the\n", "karate club split after an argument between two teachers.\n", "\n", "Some information about the dataset is displayed to provide\n", "insight into its complexity." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO: Number of vertices: 34\n", "INFO: Mean edge-vertex ratio: 4.588235294117647\n" ] } ], "source": [ "nb_vertices_by_edges = [len(e_2) for _, e_2 in karate_graph.edges.items()]\n", "logging.info(\"Number of vertices: %s\", len(karate_graph.edges))\n", "logging.info(\n", " \"Mean edge-vertex ratio: %s\",\n", " (sum(nb_vertices_by_edges, 0) / len(karate_graph.edges)),\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Denote $V$ as the set of nodes and $E \\subset V\\times V$ the set \n", "of edges. The goal of embedding GSD is to provide a faithful and exploitable representation \n", "of the graph structure. It is mainly achieved by preserving *first-order* proximity \n", "that enforces nodes sharing edges to be close to each other. It can additionally \n", "preserve *second-order* proximity that enforces two nodes sharing the same context \n", "(i.e., nodes that share neighbors but are not necessarily directly connected) to be close. \n", "Let $\\mathbb{B}^m$ be the Poincaré Ball of dimension $m$ equipped with the distance function $d$.\n", "The below figure shows geodesics between pairs of points on $\\mathbb{B}^2$. Geodesics are\n", "the shortest path between two points. The distance function $d$ of two points is\n", "the length of the geodesic that links them.\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Declaring an instance of the `PoincareBall` manifold of two dimensions in `geomstats` is straightforward:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "hyperbolic_manifold = PoincareBall(2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*first* and *second-order* proximities can be achieved by optimising the following loss functions:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loss function.\n", "\n", "To preserve first and second-order proximities we adopt a loss function similar to (Nickel, 2017) and consider the negative sampling approach as in (Mikolov, 2013) :\n", "\n", "$$ \\mathcal{L} = - \\sum_{v_i\\in V} \\sum_{v_j \\in C_i} \\bigg[ log(\\sigma(-d^2(\\phi_i, \\phi_j'))) + \\sum_{v_k\\sim \\mathcal{P}_n} log(\\sigma(d^2(\\phi_i, \\phi_k'))) \\bigg]$$\n", "\n", "where $\\sigma(x)=\\frac{1}{1+e^{-x}}$ is the sigmoid function and $\\phi_i \\in \\mathbb{B}^m$ \n", "is the embedding of the $i$-th node of $V$, $C_i$ the nodes in the context of the \n", "$i$-th node, $\\phi_j'\\in \\mathbb{B}^m$ the embedding of $v_j\\in C_i$ and \n", "$\\mathcal{P}_n$ the negative sampling distribution over $V$: \n", "$\\mathcal{P}_n(v)=\\frac{deg(v)^{3/4}}{\\sum_{v_i\\in V}deg(v_i)^{3/4}}$. \n", "Intuitively one can see that to minimizing $L$, the distance between $v_i$ and $v_j$ should get smaller, while the one between\n", "$v_i$ and $v_k$ would get larger.\n", "" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## Riemannian optimization.\n", "Following the idea of (Ganea, 2018) we use the following formula to optimize $L$:\n", "\n", "$$ \\phi^{t+1} = \\text{Exp}_{\\phi^t} \\left( -lr \\frac{\\partial L}{\\partial \\phi} \\right) $$\n", "\n", "where $\\phi$ is a parameter of $L$, $t\\in\\{1,2,\\cdots\\}$ is the epoch iteration number \n", "and $lr$ is the learning rate. The formula consists of first computing the usual gradient of the loss function\n", "giving the direction in which the parameter should move. \n", "The Riemannian exponential map $\\text{Exp}$ is a function that takes a base point $\\phi^t$ and some \n", "direction vector $T$ and returns the point $\\phi^{t+1}$ such that $\\phi^{t+1}$ belongs to the geodesic\n", "initiated from $\\phi{t}$ in the direction of $T$ and the length of the geoedesic curve between $\\phi^t$ and $\\phi^{t+1}$ is of 1 unit. \n", "The Riemannian exponential map is implemented as a method of the `PoincareBallMetric` class in the `geometry` module of `geomstats`.\n", "\n", "Therefore to minimize $L$ we will need to compute its gradient. Several steps are required to do so,\n", "1. Compute the gradient of the squared distance\n", "2. Compute the gradient of the log sigmoid\n", "3. Compute the gradient of the composision of 1. and 2." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For 1., we use the formula proposed by (Arnaudon, 2013) which uses the Riemannian logarithmic map to compute the gradient of the distance.\n", "This is implemented as" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "def grad_squared_distance(point_a, point_b):\n", " \"\"\"Gradient of squared hyperbolic distance.\n", "\n", " Gradient of the squared distance based on the\n", " Ball representation according to point_a\n", "\n", " Parameters\n", " ----------\n", " point_a : array-like, shape=[n_samples, dim]\n", " First point in hyperbolic space.\n", " point_b : array-like, shape=[n_samples, dim]\n", " Second point in hyperbolic space.\n", "\n", " Returns\n", " -------\n", " dist : array-like, shape=[n_samples, 1]\n", " Geodesic squared distance between the two points.\n", " \"\"\"\n", " hyperbolic_metric = PoincareBall(2).metric\n", " log_map = hyperbolic_metric.log(point_b, point_a)\n", "\n", " return -2 * log_map" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "For 2. define the `log_sigmoid` corresponding as follows:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "def log_sigmoid(vector):\n", " \"\"\"Logsigmoid function.\n", "\n", " Apply log sigmoid function\n", "\n", " Parameters\n", " ----------\n", " vector : array-like, shape=[n_samples, dim]\n", "\n", " Returns\n", " -------\n", " result : array-like, shape=[n_samples, dim]\n", " \"\"\"\n", " return gs.log((1 / (1 + gs.exp(-vector))))" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "The gradient of the logarithm of sigmoid function is implemented as:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "pycharm": { "name": "#%% \n" } }, "outputs": [], "source": [ "def grad_log_sigmoid(vector):\n", " \"\"\"Gradient of log sigmoid function.\n", "\n", " Parameters\n", " ----------\n", " vector : array-like, shape=[n_samples, dim]\n", "\n", " Returns\n", " -------\n", " gradient : array-like, shape=[n_samples, dim]\n", " \"\"\"\n", " return 1 / (1 + gs.exp(vector))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For 3., apply the composition rule to obtain the gradient of $L$. The following function given $\\phi_i$, $\\phi'_j$ and $\\phi'_k$ returns the total value of $L$ and its gradient vector at $\\phi_i$. For the value of $L$ the loss function formula is simply applied. For the gradient, we apply the composition of `grad_log_sigmoid` with `grad_squared_distance` while paying attention to the signs." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "def loss(example_embedding, context_embedding, negative_embedding, manifold):\n", " \"\"\"Compute loss and grad.\n", "\n", " Compute loss and grad given embedding of the current example,\n", " embedding of the context and negative sampling embedding.\n", " \"\"\"\n", " n_edges, dim = negative_embedding.shape[0], example_embedding.shape[-1]\n", " example_embedding = gs.expand_dims(example_embedding, 0)\n", " context_embedding = gs.expand_dims(context_embedding, 0)\n", " positive_distance = manifold.metric.squared_dist(\n", " example_embedding, context_embedding\n", " )\n", " positive_loss = log_sigmoid(-positive_distance)\n", "\n", " reshaped_example_embedding = gs.repeat(example_embedding, n_edges, axis=0)\n", " negative_distance = manifold.metric.squared_dist(\n", " reshaped_example_embedding, negative_embedding\n", " )\n", " negative_loss = log_sigmoid(negative_distance)\n", "\n", " total_loss = -(positive_loss + negative_loss.sum())\n", "\n", " positive_log_sigmoid_grad = -grad_log_sigmoid(-positive_distance)\n", "\n", " positive_distance_grad = grad_squared_distance(example_embedding, context_embedding)\n", "\n", " positive_grad = (\n", " gs.repeat(positive_log_sigmoid_grad, dim, axis=-1) * positive_distance_grad\n", " )\n", "\n", " negative_distance_grad = grad_squared_distance(\n", " reshaped_example_embedding, negative_embedding\n", " )\n", "\n", " negative_distance = gs.to_ndarray(negative_distance, to_ndim=2, axis=-1)\n", " negative_log_sigmoid_grad = grad_log_sigmoid(negative_distance)\n", "\n", " negative_grad = negative_log_sigmoid_grad * negative_distance_grad\n", " example_grad = -(positive_grad + negative_grad.sum(axis=0))\n", "\n", " return total_loss, example_grad" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## Capturing the graph structure\n", "At this point we have the necessary bricks to compute the resulting gradient of $L$. We are ready to prepare\n", "the nodes $v_i$, $v_j$ and $v_k$ and initialise their embeddings $\\phi_i$, $\\phi^{'}_j$ and $\\phi^{'}_k$.\n", "First, initialize an array that will hold embeddings $\\phi_i$ of each node $v_i\\in V$ with random points belonging to the Poincaré disk. " ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "embeddings = gs.random.normal(size=(karate_graph.n_nodes, dim))\n", "embeddings = embeddings * 0.2" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Next, to prepare the context nodes $v_j$ for each node $v_i$, we compute random walks initialised from each $v_i$ \n", "up to some length (5 by default). The latter is done via a special function within the `Graph` class. The nodes $v_j$ will be later picked from the random walk of $v_i$.\n", "\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "random_walks = karate_graph.random_walk()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Negatively sampled nodes $v_k$ are chosen according to the previously defined probability distribution function\n", "$\\mathcal{P}_n(v_k)$ implemented as" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "negative_table_parameter = 5\n", "negative_sampling_table = []\n", "\n", "for i, nb_v in enumerate(nb_vertices_by_edges):\n", " negative_sampling_table += (\n", " [i] * int((nb_v ** (3.0 / 4.0))) * negative_table_parameter\n", " )\n", "\n", "negative_sampling_table = gs.array(negative_sampling_table)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## Numerically optimizing the loss function\n", "Optimising the loss function is performed numerically over the number of epochs. \n", "At each iteration, we will compute the gradient of $L$. Then the graph nodes are moved in the direction\n", "pointed by the gradient. The movement of the nodes is performed by following geodesics in the gradient direction.\n", "The key to obtain an embedding representing accurately the dataset, is to move the nodes smoothly rather\n", "than brutal movements. This is done by tuning the learning rate, such as at each epoch\n", "all the nodes made small movements." ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "A *first level* loop iterates over the epochs, the table `total_loss` will record the value of $L$ at each iteration\n", "and help us track the minimization of $L$." ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "A *second level* nested loop iterates over each path in the previously computed random walks. Observing these walks, notice that nodes having \n", "many edges appear more often. Such nodes\n", "can be considered as important crossroads and will therefore be subject to a greater number of embedding updates. \n", "This is one of the main reasons why random walks have proven to be effective\n", "in capturing the structure of graphs. The context of each $v_i$ will be the set of nodes $v_j$ belonging \n", "to the random walk from $v_i$. The `context_size` specified earlier will limit the length of the walk to be considered. Similarly, we use\n", "the same `context_size` to limit the number of negative samples. We find $\\phi_i$ from the `embeddings` array.\n" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "\n", "A *third level* nested loop will iterate on each $v_j$ and $v_k$. From within, we find $\\phi'_j$ and $\\phi'_k$ then call the `loss` function to compute the gradient.\n", "Then the Riemannian exponential map is applied to find the new value of $\\phi_i$ as we mentioned before." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO: iteration 0 loss_value 1.826876\n", "INFO: iteration 1 loss_value 1.774560\n", "INFO: iteration 2 loss_value 1.725700\n", "INFO: iteration 3 loss_value 1.663358\n", "INFO: iteration 4 loss_value 1.655706\n", "INFO: iteration 5 loss_value 1.615405\n", "INFO: iteration 6 loss_value 1.581097\n", "INFO: iteration 7 loss_value 1.526418\n", "INFO: iteration 8 loss_value 1.507913\n", "INFO: iteration 9 loss_value 1.505934\n", "INFO: iteration 10 loss_value 1.466526\n", "INFO: iteration 11 loss_value 1.453769\n", "INFO: iteration 12 loss_value 1.443878\n", "INFO: iteration 13 loss_value 1.451272\n", "INFO: iteration 14 loss_value 1.397864\n", "INFO: iteration 15 loss_value 1.396170\n", "INFO: iteration 16 loss_value 1.373677\n", "INFO: iteration 17 loss_value 1.390120\n", "INFO: iteration 18 loss_value 1.382397\n", "INFO: iteration 19 loss_value 1.404103\n", "INFO: iteration 20 loss_value 1.395782\n", "INFO: iteration 21 loss_value 1.389617\n", "INFO: iteration 22 loss_value 1.410152\n", "INFO: iteration 23 loss_value 1.390600\n", "INFO: iteration 24 loss_value 1.374832\n", "INFO: iteration 25 loss_value 1.367194\n", "INFO: iteration 26 loss_value 1.323190\n", "INFO: iteration 27 loss_value 1.389616\n", "INFO: iteration 28 loss_value 1.361034\n", "INFO: iteration 29 loss_value 1.384930\n", "INFO: iteration 30 loss_value 1.340814\n", "INFO: iteration 31 loss_value 1.349682\n", "INFO: iteration 32 loss_value 1.317423\n", "INFO: iteration 33 loss_value 1.346869\n", "INFO: iteration 34 loss_value 1.327198\n", "INFO: iteration 35 loss_value 1.363809\n", "INFO: iteration 36 loss_value 1.352347\n", "INFO: iteration 37 loss_value 1.317670\n", "INFO: iteration 38 loss_value 1.320039\n", "INFO: iteration 39 loss_value 1.323888\n", "INFO: iteration 40 loss_value 1.341444\n", "INFO: iteration 41 loss_value 1.312259\n", "INFO: iteration 42 loss_value 1.315983\n", "INFO: iteration 43 loss_value 1.305483\n", "INFO: iteration 44 loss_value 1.325384\n", "INFO: iteration 45 loss_value 1.328024\n", "INFO: iteration 46 loss_value 1.306958\n", "INFO: iteration 47 loss_value 1.303357\n", "INFO: iteration 48 loss_value 1.303790\n", "INFO: iteration 49 loss_value 1.324749\n", "INFO: iteration 50 loss_value 1.328376\n", "INFO: iteration 51 loss_value 1.313816\n", "INFO: iteration 52 loss_value 1.325978\n", "INFO: iteration 53 loss_value 1.317516\n", "INFO: iteration 54 loss_value 1.353495\n", "INFO: iteration 55 loss_value 1.331988\n", "INFO: iteration 56 loss_value 1.346874\n", "INFO: iteration 57 loss_value 1.348946\n", "INFO: iteration 58 loss_value 1.324719\n", "INFO: iteration 59 loss_value 1.330355\n", "INFO: iteration 60 loss_value 1.331077\n", "INFO: iteration 61 loss_value 1.305729\n", "INFO: iteration 62 loss_value 1.311746\n", "INFO: iteration 63 loss_value 1.347637\n", "INFO: iteration 64 loss_value 1.326300\n", "INFO: iteration 65 loss_value 1.309570\n", "INFO: iteration 66 loss_value 1.313999\n", "INFO: iteration 67 loss_value 1.346287\n", "INFO: iteration 68 loss_value 1.300901\n", "INFO: iteration 69 loss_value 1.323723\n", "INFO: iteration 70 loss_value 1.320784\n", "INFO: iteration 71 loss_value 1.313709\n", "INFO: iteration 72 loss_value 1.312143\n", "INFO: iteration 73 loss_value 1.309172\n", "INFO: iteration 74 loss_value 1.320642\n", "INFO: iteration 75 loss_value 1.308333\n", "INFO: iteration 76 loss_value 1.325884\n", "INFO: iteration 77 loss_value 1.316740\n", "INFO: iteration 78 loss_value 1.325933\n", "INFO: iteration 79 loss_value 1.316672\n", "INFO: iteration 80 loss_value 1.312291\n", "INFO: iteration 81 loss_value 1.332372\n", "INFO: iteration 82 loss_value 1.317499\n", "INFO: iteration 83 loss_value 1.329194\n", "INFO: iteration 84 loss_value 1.305926\n", "INFO: iteration 85 loss_value 1.304747\n", "INFO: iteration 86 loss_value 1.342343\n", "INFO: iteration 87 loss_value 1.331992\n", "INFO: iteration 88 loss_value 1.295439\n", "INFO: iteration 89 loss_value 1.332853\n", "INFO: iteration 90 loss_value 1.332004\n", "INFO: iteration 91 loss_value 1.357248\n", "INFO: iteration 92 loss_value 1.342234\n", "INFO: iteration 93 loss_value 1.329379\n", "INFO: iteration 94 loss_value 1.313617\n", "INFO: iteration 95 loss_value 1.310320\n", "INFO: iteration 96 loss_value 1.320590\n", "INFO: iteration 97 loss_value 1.315822\n", "INFO: iteration 98 loss_value 1.328819\n", "INFO: iteration 99 loss_value 1.339718\n" ] } ], "source": [ "for epoch in range(max_epochs):\n", " total_loss = []\n", " for path in random_walks:\n", "\n", " for example_index, one_path in enumerate(path):\n", " context_index = path[\n", " max(0, example_index - context_size) : min(\n", " example_index + context_size, len(path)\n", " )\n", " ]\n", " negative_index = gs.random.randint(\n", " negative_sampling_table.shape[0], size=(len(context_index), n_negative)\n", " )\n", " negative_index = negative_sampling_table[negative_index]\n", "\n", " example_embedding = embeddings[one_path]\n", " for one_context_i, one_negative_i in zip(context_index, negative_index):\n", " context_embedding = embeddings[one_context_i]\n", " negative_embedding = embeddings[one_negative_i]\n", " l, g_ex = loss(\n", " example_embedding,\n", " context_embedding,\n", " negative_embedding,\n", " hyperbolic_manifold,\n", " )\n", " total_loss.append(l)\n", "\n", " example_to_update = embeddings[one_path]\n", " embeddings[one_path] = hyperbolic_manifold.metric.exp(\n", " -lr * g_ex, example_to_update\n", " )\n", " logging.info(\n", " \"iteration %d loss_value %f\", epoch, sum(total_loss, 0) / len(total_loss)\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plotting results\n", "Once the `max_epochs` iterations of epochs is achieved, we can plot the resulting `embeddings` array and the true labels shown\n", "as two colors. At 100 epochs we can see that the two group of nodes with different labels are moving away from each other\n", "on the manifold. If one increases the `max_epochs`, then further separability is achieved." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdAAAAHUCAYAAACKzHkWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdd3wUZf7A8c8TCMEEgvQWBYQ0E5LQg0iVpjRBqYpwlLMAnqiInt6BHCjlOEWJov7gKKcBkSIHGqVIEYjUEEMwCR5IQugCCSGQTXZ+f8wmLiG9zZbv+/XaV3ZnZme+szvZ7zzPPPM8StM0hBBCCFE8LkYHIIQQQtgjSaBCCCFECUgCFUIIIUpAEqgQQghRApJAhRBCiBKQBCqEEEKUgCTQCqKU+qtS6v+MjqM8KaVmKqX+Y3neVCmlKaUqV8B2lyulZpfRusYqpX4sYP5OpdQEy/OnlFLfl8V2i0MpdY9S6r9KqetKqbVFfE9O3OJOxflsHPVzrMj/V0ciCbSYlFKnlVLpSqkbSqkLSql/K6WqFfY+TdPe0TTNpv/xlFLdlFJmy77dUEqdVUq9XU7bsv4csx+Ly2Nb5UXTtM81TettwKafBOoDtTVNG5p7pvWJTHmwfHc9rV6PUEpdVUp1LYdtlfqHXSlVxfKZJCil0izxL1NKNS27SPPcbjdL7GG5pv+olBpbxHVoSqkW5RKgKDVJoCUzQNO0akBroB3wlsHx5KmEPzrJmqZVs+zfw8B4pdTjZRxatgHZ27I8JpfTdhxNEyBe07RMowNRSo0BwoB+mqbtKsH7K5V9VHf5ChgIjAJqAMHAYeCRCth2GvBMeSfr0pBSZ8lJAi0FTdPOAt8CgQBKqUZKqU1Kqd+VUieVUhOzl82nenOMUuqMUuqyUupNq2UrWap8f1VKpSqlDiul7rPMW6SUSlRKpVimd861ja+UUv9RSqUAY5VSLkqp1y3ruqKU+lIpVauI+3cK2Ac8aLWNfLdfVizVqHuVUu8ppa4ppf6nlHrIMj1RKXXR8sNtrY5Saqvl89qllGpitT4/y7zflVJxSqlhVvNqW76zFKXUAaB5rlh6KaV+sVSXLgZUrjh/tHqtKaWes5R0riqlwpRSyjKvklJqoeW7PqWUmlxQyUop5a/06sJrSqnjSqmBlulvA38HhltK7eNzva8v8Fer+cesZjexfK6pSqnvlVJ1rN4XqpTaZ9neMaVUtwK/JP09fwYWAn00TdtnNX2tUuq85TPbrZQKsJq3XCn1sVLqG6VUGtBdKdVPKXXU8h0kKqVmWm1mt+XvNcv+dLSsZ5xS6oTlc/7O+vvOFWNPoBcwSNO0g5qmZWqadl3TtDBN05bmsfwdpXeVdwm4uVLqgGX/vi7k/+kasByYkd8C+e2LUip7349Z9n245dh+wjL/YUtsj2Xvq1IqyvLcRSn1llLqN8v/y0qlVI1c+zReKXUG2JFHTE8ovaQeWMC+CU3T5FGMB3Aa6Gl5fh9wHPiH5fUu4COgKhACXAIescybCfzH8rwpoAGfAfegnxHfBvwt86cBPwO+6D/YwejVdQBPA7WBysArwHmgqtU2TMDj6CdH9wAvAZGAF+AGfAKE57Nv3YAkq9fewFmgh9W0wrafex8rF/Y55jFvLJAJ/AmoBMwGzqCXdNyA3kAqUM2y/HLL6y6W+YuAHy3zPIBEy7oqo9caXAYCLPNXA19algu07G/2e+sAKehVpq7AVEtcE6zi/NEqbg3YDNwL3G/5/vta5j0HxFq+h5rAtvw+H8u2TqInwipAD8v++eb+nPP5/O6aD+wEfgV8LMfFTmCuZV5j4ArwmOW46WV5XbeA724dcAEIzmP+OKC65bt4H4iymrccuA50smyrKvpx19LyOsiy3sfzO47Qj++TgL/lO30L2JdPrHOBXYX8T++0+k7v+Oxyb9+y7FnLseJh+Rzy/C4s+5UENLAcR9nf34/A2KLsi2XbLaxezwI+tDz/q+U7nWc1b5HVd3ASeACoBqwHVuXap5WWfbjHej/R/1dOWm9XHvkcO0YHYG8Py4/HDfQzy9/QE+Y96Mk0C6hutey7wHLL85x/TKuD1ctq2QPACMvzOPQz5qLEcxXLj5hlG7tzzT+BJYlbXjdET7J5/XB3A8yWfUuxxLgeqFKM7RcngWZ/jtmPiZZ5Y4EEq2VbWtZV32raFSDE8nw5sNpqXjXLd3EfMBzYk2vbn6CXCCpZPgs/q3nv8EcCfQaItJqn0H8QC0qgD1u9/hJ43fJ8B/Cs1bye+X0+QGf0ExMXq2nhwMzcn3M+n+1d89F/+N+yev0CEGF5Ph3Lj6vV/O+AMQV8dynA19Yx5rPsvZb9rGH1Xa0s5D3vA+/ldxyh1/qMt3rtAtwEmuSxrs+sj418treT4iXQuVbzHwQygEr5/D8lWZ7PB9ZYnlsn0AL3hbsT6CNAtOV5BDAByzGKfgI/xPJ8O/CC1ft8sfzfW+3TA3ns56tYTvQK+szkoT+kCrdkHtc07V5N05pomvaCpmnpQCPgd03TUq2W+w397D4/562e30T/4Qf9h//XvN6glHrFUt1zXSl1Df2aTh2rRRJzvaUJsMFSNXcNPaFmoTdCyUuyZd880X/80oEVxdh+cWR/jtmPz6zmXbB6ng6gaVruadaNt3L2W9O0G8Dv6N9JE6BD9v5bYn4KvVRQF/0Hxfoz+83qeaNc69W4+/PNLb/v9I51FbKeRkCipmnmXHEVdCwVRX6xNQGG5vqMHkY/2crPc+il2f/LrqaGnKrquUq/ZJCCnmyhgGNUKdVBKfWDUuqSUuq6Zd0FHVNNgEVWsf6OfnKT1+dzpZD9KIncx4srhf8PzAP6KKWCc00vzr4A7Ad8lFL10Wu5VgL3Warj2/NHlXcj7jyWf0M/1q3/7/M6BqcBYZqmJRWyPwK5BlqWkoFaSqnqVtPuR6/uKa5Ecl2LA1D69cbpwDCgpqZp96JXhymrxbQ81vVorkRVVdOv3xZI07TrwBfAgGJs3yj3ZT9ReqvoWujfSSJ6FZ71/lfTNO159CrWTOv3on9n2c7lWq/KtWxxnEOvvr0r3jwko/8oWv9/FudYyn0MFCYRvQRq/Rl5aJo2t4D3XEQvDXVGr4XJNgoYhF7CroFesoGCj9EvgE3AfZqm1QCWWC2f174kopfmreO9R7O6DmtlG9BeKeWVx7y8pAHuVq8b5LFM7uPFhH5ZIF+apl1BL1n/I9es4uwLmqbdRG8A9RcgRtO0DPR2Ci8Dv2qalh1HMnpyto4zkztPTPP6bHsDb2VfZxUFkwRaRjRNS0Q/kN9VSlVVSgUB44HPS7C6/wP+oZTyVrogpVRt9OtKmeg//JWVUn8HPAtZ1xJgjlXDhLpKqUFFCcKSiEagX+elhNuvKI9ZGlVUQf+R+snynWxGP2MfrZRytTzaKaX8NU3LQq+inqmUcldKPQiMsVrnFiBAKTXE0ojkRfL+QS2KL4G/KKUaK6XuRT8Ryc9P6D/kr1ni7YZ+ErO6iNu6ADTNlYAL8h9ggFKqj6UEWVXpt2AUmHQ0TUtGvz7bVyn1nmVydfTr+VfQE9E7Rdh+dfTam1tKqfboSTjbJfTLCg9YTVsCvKEsjZOUUjWUUnfdzmOJcRuwFb0Wpo1SqrJSqrrSG3uNy+MtUUAXpdT9lkY3b+SxzNNKqQeVUu7o1x2/shxLhfkX8BD69c6i7suFXPsOelXtZMtf0KuVrV+DXuU/VSnVzPJ//A56FXJhLbePA32BMGVpuCbyJwm0bI1EP+NOBjYAMzRN21qC9fwL/Qf3e/RrTUvRr7N+h37NJB69SuYWhVcpLkI/u/9eKZWK3qCoQwHLN7K0+Lth2UYt9CpPSrj9gvxX3Xkf6IZSrOsL9OuavwNtsmO2VKn3Rj8RSEavxpyH3sAF9B+eapbpy4F/Z6/QcjY/FL0hyhX0RlV7SxjfZ+jfZzRwFPgG/WTkrh9eS6liIPAoesnmI+AZTdN+KeK2sjtXuKKUOlLYwpYTjUHojVIuoX+n0yjC74PlvT2AJ5VS76JXKf6GXlqORT/eCvMCMMtyfP4d/djPXv9NYA6w11LNGapp2gb073C1pZo4Bv2zys+T6J/3GvQakxigLXrpNPf+bLUsF41e0tucx/pWoR8r59EbQb1YhH1E07QU9GuhtaymFbYvM4EVln3Pbj2+C/2kY3c+rwGWWeLcDZxC/1+dUsQ4jwH9gc+UUgV9rk5P6Zd1hBAVyfLDtETTtCaFLiyEsElSAhWiAii9+73HLFWIjdFLy6UpcQshDCYlUCEqgOV62S7AD70F8RbgL5ZqPSGEHZIEKoQQQpSAVOEKIYQQJVCsToTr1KmjNW3atJxCEUIIIWzL4cOHL2uaVjevecVKoE2bNuXQoUNlE5UQQghh45RSv+U3T6pwhRBCiBKQBCqEEEKUgCRQIYQQogRkJHIhhHBQJpOJpKQkbt26ZXQoNq9q1ap4eXnh6upa5PdIAhVCCAeVlJRE9erVadq0KVajzolcNE3jypUrJCUl0axZsyK/T6pwhRDCQd26dYvatWtL8iyEUoratWsXu6QuCVQIIRyYJM+iKcnnJAlUCCGEKAFJoEII4SQaNAClyu7RoAjDy1+4cIFRo0bxwAMP0KZNGzp27MiGDRU7ENEvv/xCx44dcXNz45///GeZrVcaEQkhhJO4cKFi16dpGo8//jhjxozhiy++AOC3335j06ZNdy2bmZlJ5crlk5Jq1arFBx98wMaNG8t0vVICFUIIUS527NhBlSpVeO6553KmNWnShClTpgCwfPlyhg4dyoABA+jduzeapjFt2jQCAwNp2bIla9asAWDnzp30798/Zx2TJ09m+fLlgN7F7PTp02nfvj3t27fn5MmTd8VRr1492rVrV6xbVIpCSqBCCCHKxfHjx2ndunWBy+zfv5/o6Ghq1arFunXriIqK4tixY1y+fJl27drRpUuXQrfj6enJgQMHWLlyJS+99BKbN28uq10okJRAhRBCVIhJkyYRHBxMu3btcqb16tWLWrVqAfDjjz8ycuRIKlWqRP369enatSsHDx4sdL0jR47M+bt///7yCT4PkkCFEEKUi4CAAI4cOZLzOiwsjO3bt3Pp0qWcaR4eHjnPNU3Lcz2VK1fGbDbnvM59v6b1LSgVeduOJFAhhBDlokePHty6dYuPP/44Z9rNmzfzXb5Lly6sWbOGrKwsLl26xO7du2nfvj1NmjQhNjaW27dvc/36dbZv337H+7Kvla5Zs4aOHTuWz87kQa6BCiGEk6hfv2xb4tavX/B8pRQbN25k6tSpzJ8/n7p16+Lh4cG8efPyXH7w4MHs37+f4OBglFLMnz+fBpZ7ZYYNG0ZQUBDe3t60atXqjvfdvn2bDh06YDabCQ8Pv2u958+fp23btqSkpODi4sL7779PbGwsnp6eJdvx7P3Lr8icl7Zt22oyoLYQQtiHEydO4O/vb3QY5app06YcOnSIOnXqlHpdeX1eSqnDmqa1zWt5qcIVQgghSkCqcIUQQtit06dPG7ZtKYEKIYQQJSAJVAghhCgBSaBCCCFECUgCFUIIIUpAEqgQQjgLA8Yzs4XhzD7//HOCgoIICgrioYce4tixY2WyXkmgQgjhLCp4PLPs4cy6dOnC//73Pw4fPszq1atJSkq6a9nMzMyyjc1Ks2bN2LVrF9HR0fztb3/jz3/+c5msVxKoEEKIcmErw5k99NBD1KxZE4DQ0NA8E3hJyH2gQgghyoUtDme2dOlSHn300WLvS16kBCqEEKJCGD2c2Q8//MDSpUvz7Yu3uCSBCiGEKBe2NJxZdHQ0EyZM4Ouvv6Z27drF25F8SAIVQghRLmxlOLMzZ84wZMgQVq1ahY+PTxntnVwDFUII51HB45nZynBms2bN4sqVK7zwwguAXqIti5HFZDgzIYRwUDKcWfHIcGZCCCFEBZAqXCGEEHbLyOHMJIEKUY4yMjK4cOEC586d49y5c1y5coUbN26QmprKjRs3ch7Wr2/evInZbMZsNpOVlYXZbEYphYuLS86jUqVKeHh4UK1aNapXr061atXyfF63bl0aNmxIw4YNqVu3LpUqVTL6IxEVTNO0fFumij8U53JmNkmgQpRQVlYWiYmJJCQkkJCQwJkzZ3IS5blz5zh//jzXr1+/I4nVrl2b6tWr5zwaNWp0V+Jzd3enUqVKOcky+8cvO6mazWYyMzO5efPmHck3++/Vq1dJTEwkJSWFS5cu5cRz9epV6tatS4MGDXLiadiwIU2bNsXb2xtvb28aNGggP7YOpGrVqly5coXatWvL91oATdO4cuUKVatWLdb7JIEKUYgLFy4QGxubkyjj4+NJSEjg1KlT1KlTBx8fH7y9vWnSpAl+fn53JKg6derg4mIbTQ1MJhMXL168I8EnJyezZ88eli1bRkJCAunp6TnJ1PoREBBAjRo1jN4FUUxeXl4kJSXdcd+lyFvVqlXx8vIq1nukFa4QFpqmkZiYyJEjR3IeR48eJT09nYCAgJxE6e3tjY+PD82bN8fd3d3osMvUtWvXck4Usk8W4uPjOXHiBA0bNqR169a0bt2aVq1a0bp16zJp+SiELSuoFa4kUOG0Ll++zO7duzlw4EBOwqxcuTJt2rS5I1E0adLE6au/srKyiIuL4+jRo3ecXHh6euZ8Vh07dqRjx45Uq1bN6HCFKDOSQIUAzp07x+7du9m1axe7du0iKSmJTp060bFjx5wk0LBhQ6PDtBtms5lTp07lJNQff/yRo0ePEhgYSNeuXenatSudOnWSql9h1ySBCqd0/vx5tm7dmpM0L1++TOfOnXN+3IODg6lcWZoBlKX09HR++umnnJOUAwcO4OvrS5cuXejWrRvdu3fH09PT6DCFKDJJoMIpmM1mjh49yubNm9myZQsJCQn06NGDbt260bVrVwIDA22mQY+zuH37NocOHWLXrl3s3LmT/fv3ExoaSv/+/enfvz/Nmzc3OkQhCiQJVDisGzdusH379pyk6enpmfPj3KlTJ1xdXY0OUVi5ceMG27Zty/m+7r33Xvr16yffl7BZkkCFQ7l27RobNmzgyy+/ZO/evXTo0IH+/fvTr18/WrRoYXR4ooisaww2b97Mr7/+St++fRk+fDh9+/bFzc3N6BCFkAQq7F96ejqbN2/miy++YMeOHTzyyCOMGDGCvn37yjU1B3Hu3Dk2bdpEeHg4P//8M4MHD2bkyJF069ZNelAShpEEKuySyWRi27ZtfPHFF2zevJl27doxcuRIBg8ezL333mt0eKIcJSUlsWbNGsLDw0lOTmbYsGGMHDmS9u3bO/0tRaJiSQIVdiUmJoZPPvmENWvW0KJFC0aOHMmwYcOoX8jYg8IxxcfHEx4eTnh4OCaTiTFjxjB+/HgaN25sdGjCCchwZsLmpaens3LlSjp16kTfvn2pVasWP/30E/v27WPKlCmSPJ2Yj48PM2bM4MSJE3z55ZecP3+eli1bMnjwYCIiIjCbzUaHKJyUlECFoX755Rc++eQTVq1aRbt27Xjuuefo16+f3J8pCnTjxg3Cw8P55JNPuHLlChMnTmTcuHE0aNDA6NCEg5ESqLApJpOJ1atX59xY7+7uzsGDB/n2228ZNGiQJE9RqGrVqjFx4kQOHTrE2rVrOXXqFP7+/gwdOpQdO3aUaGgqIYpLSqCiwty4cYP/+7//47333qNZs2ZMnjyZgQMHUqVKFaNDEw4gJSWFzz//nA8//JB77rmHadOm8eSTT8oJmSgVKYEKQ124cIG33nqLpk2bsnfvXtauXcvOnTt58sknJXmKMuPp6cnzzz9PTEwMM2fOJCwsDG9vbxYvXkxaWprR4QkHJAlUlJuEhASee+45/Pz8uHLlCpGRkaxdu5b27dsbHZpwYC4uLgwYMIA9e/bw+eefs337dpo1a8bMmTNlXExRpiSBijJ37NgxnnzySR566CHq1atHXFwcH3/8sfQSJCrcQw89xIYNG9izZw/Jycn4+PgwefJkzp49a3RowgFIAhVlJi4ujhEjRtCnTx86derEqVOnmDVrFvXq1TM6NOHkfH19+fTTT4mNjeWee+4hKCiIV155RUqkolQkgYpS++233xg3bhwPP/wwQUFBnDx5kqlTp8rAysLmNGzYkAULFvDzzz9z69Yt/Pz8+Nvf/sa1a9eMDk3YIUmgosTOnTvH5MmTad26NY0aNSIhIYG//vWvkjiFzWvUqBFhYWEcPnyYpKQkvL29effdd6WxkSgWSaCi2K5du8b06dMJDAykSpUqnDhxgtmzZ0v/tMLuNG3alH//+9/s2bOHqKgoWrRowQcffIDJZDI6NGEHJIGKIsvKyuLTTz/NaVV77Ngx/vWvf8k1TmH3/Pz8WLNmDd9++y3ffPMNQUFBfPfdd0aHJWyc3GEsiuTHH3/kxRdfxN3dnW+++YbWrVsbHZIQZS4kJIRvv/2WzZs3M2nSJB588EH+9a9/SQtykScpgYoCJSUlMWrUKEaOHMm0adPYs2ePJE/h0JRSDBgwgOPHj9OpUydCQ0N5/fXXSU1NNTo0YWMkgYo83bp1izlz5hASEkLz5s355ZdfGDlypIzFKJyGm5sb06dPJzo6mnPnzuHn58eqVatk9BeRQxKouMvWrVsJCAjgyJEjHDx4kH/84x94eHgYHZYQhmjUqBErVqxg3bp1fPjhh3Tu3JkTJ04YHZawAZJARY6rV68ybtw4JkyYQFhYGOvWraNZs2ZGhyWETQgNDSUyMpKnn36aLl26MHv2bGmt6+QkgQoANmzYQGBgIO7u7sTExNC3b1+jQxLC5ri4uPD8889z+PBh9u3bR9u2bTl8+LDRYQmDSCtcJ3f+/HmmTJlCdHQ0q1evpnPnzkaHJITNu//++9myZQuff/45jz32GGPHjmXmzJncc889RocmKpCUQJ2UpmmsWLGC4OBgvL29OXbsmCRPIYpBKcXTTz/Nzz//zOnTpwkODmb37t1GhyUqkJRAndDFixcZN24cZ8+e5dtvv5XbUoQohXr16rFmzRo2btzIqFGjePLJJ5k7dy5Vq1Y1OjRRzqQE6mS+//57QkJCaNmyJQcOHJDkKUQZefzxx4mOjiYxMZHQ0FBpqesEJIE6iYyMDF599VXGjx/Pf/7zH959911cXV2NDksIh1KrVi2++uorJk2aRJcuXfj000/RNM3osEQ5kQTqBOLi4ujYsSMJCQkcPXqUHj16GB2SEA5LKcXEiRPZs2cPH3/8MU8++SS///670WGJciAJ1IFpmsayZct4+OGHmTBhAhs3bqROnTpGhyWEU/Dz8yMyMpL777+fkJAQdu7caXRIooxJIyIHlZqaysSJEzl+/Dg//PADgYGBRockhNNxc3Pjvffeo3fv3owaNYoJEyYwY8YMKlWqZHRoogxICdQBJSQkEBoaSvXq1Tlw4IAkTyEM9uijj3L06FF2797NgAEDuHr1qtEhiTIgCdTBbNmyhU6dOvGXv/yFzz77TG7sFsJG1K9fn61bt+Lj40P79u2JiYkxOiRRSlKF6yDMZjPvvPMOH3/8MRs3buShhx4yOiQhRC6urq68//77tGnThu7du+c0MhL2SRKoA0hJSWHMmDFcuHCBgwcP0qhRI6NDEkIUYPTo0Tz44IMMGTKEw4cPM3v2bLkuaoekCtfOxcXF0aFDB+rXr88PP/wgyVMIO9GmTRsOHTpEZGQk/fr1k1td7JAkUDu2fft2OnfuzMsvv8ySJUtwc3MzOiQhRDHUrVuXrVu34u/vT4cOHTh58qTRIYlikCpcO7Vq1SpeffVV1q5dS9euXY0ORwhRQpUrV+a9997Dz8+Pzp07s3HjRjp06GB0WKIIJIHaGU3TmDt3LkuWLGHHjh0EBAQYHZIQogw8++yzNG7cmP79+7N06VIGDhxodEiiEJJA7UhmZiZTpkxh//797N+/X653CuFg+vfvz5YtWxg0aBBnz57l+eefNzokUQBJoHYiLS2NkSNHkp6ezu7du/H09DQ6JCFEOWjfvj0//vgjjz76KGfOnGHOnDm4uEhzFVsk34oduHjxIt27d6dmzZps2bJFkqcQDq558+bs27ePnTt38swzz5CRkWF0SCIPkkBt3JkzZ+jUqRO9e/dm+fLlVKlSxeiQhBAVoE6dOmzfvp20tDT69etHWlqa0SGJXCSB2rCTJ0/SpUsXJk2axOzZs1FKGR2SEKICubu789VXX+Hl5UXfvn1JSUkxOiRhRRKojYqNjaVbt268+eabvPTSS0aHI4QwSKVKlVi6dClBQUE88sgj0uGCDZEEaoOOHTvGI488wty5c5k4caLR4TiNcePGUa9evTxHr/nnP/+JUorLly8bEJlwdi4uLixevJhu3brRvXt3Ll26ZHRIAkmgNicqKoo+ffrwwQcf8PTTTxsdjnNISYGkJMYOHUpERMRdsxMTE9m6dSv333+/AcEJoVNKMX/+fAYMGECPHj24ePGi0SE5PUmgNuTIkSP07duXsLAwhg4danQ49suSEMeNHn1XiTIqKorQ0FBCgoNp+8ADHPDxgdq1wdeXLgMHUuvpp+H6dTCZct4zdepU5s+fL9egheGUUvzjH/9gyJAhdO/enQsXLhgdklOTBGojjh49yqOPPsrHH3/ME088YXQ49sdkgtWroVWrnIQ4NjyciJo170iIr732GjNefpmoypWZde4cryUkQGYm3Lyp/z1xApKTITQUrl5l06ZNNG7cmODgYIN3UAidUoq3336bYcOG0b17dymJGkg6UrABcXFx9OvXjyVLljB48GCjw7E/V69Cz54QHw83bujTMjPpApyOjwcXFz0hbtuGAlKmT4fkZK5nZJBnX05mM8TEcLNHD+a4uvL91q0Vty9CFNGMGTMwm8307duXH374gRo1ahgdktOREqjBEhMT6d27N++8844kz5IwmfTkGRPzR/LMzZIQ6dmT9x95hGm//cZ9GRm8Cryb33ozMvg1Lo5TcXEEBwfTtGlTkpKSaN26NefPny+nnRGieGbOnEmnTp0YMGAA6enpRofjdCSBGujSpUv06tWLl156ibFjxxodjn1atw7i4qCwnloyMiAujo/nzOE9TSMReA8YX8BbWqanc7F5c06fPs3p06fx8iMrmpQAACAASURBVPLiyJEjNGjQoAx3QIiSU0qxaNEi7r//foYOHYrJ6tq9KH+SQA2SkpJC3759GTp0KFOnTjU6HPs1bx4UtYeWtDRWpKYyxPJyKHDAavZIoCMQB3gBS0EvucrN68KGubi48O9//xulFGPHjsVsNhsdktOQBGqA9PR0Bg4cSGhoKLNmzTI6HPuVkqInuGJoBOyyPN8BeFvNCwfOASYgCUvp1NU1J4GePn2aOnXqlC5mIcqBq6srX375JUlJSbz44otommZ0SE5BEmgFM5lMDB8+nEaNGvHhhx/KrRGlkZICBfQNnFeJ8jPgFSAY+CvwaWHbMJlAOu8XduCee+5h06ZN7Nu3jxkzZhgdjlOQVrgVSNM0nn32WbKyslixYoUMUVRanp4FXvsMz2f64eJsIzBQEqiwGzVq1CAiIoLOnTtTv359Jk2aZHRIDk0SaAWaP38+UVFR7NmzB1dXV6PDsX+ennqCi4oq+nuaNoVLl4p23dTDA6ZPL3F4QhihXr16fPvttzz00EO0aNGCPn36GB2Sw5IiUAXZuHEjH374IZs2bcLDw8PocBzH9Ol6oisKDw+YPRt8fQus+gX0+X5+MGRIwcsJYYMeeOAB1q5dy+jRo4mNjTU6HIclCbQCHD16lIkTJ7Jhwwa8vLyMDsexPPFE8RLisGGwbZtecs0v8Xp4QMuWsHWr3ohICDvUuXNnFixYwIABA6Tz+XIiCbScJScnM2jQID766CPatWtndDiOx9W1+AmxZk2IjISlS/Wu/1xdwd1d/9uqlT59/359OSHs2JgxYxg2bBhDhgzh9u3bRofjcFRxmju3bdtWO3ToUDmG41hu3rxJ165dGTRoEG+99ZbR4Tg2kwnWr9fvC42J0ZOhyaQn1unT9arY/EqTKSn6w9NTGgwJh2M2m3nyySfx9PTMuV9UFJ1S6rCmaW3znCcJtHyYzWaGDx+Om5sbq1atkoO2IklCFOIOaWlpdO7cmeHDhzNdGsYVS0EJVFrhlpN33nmHs2fPsmPHDkmeFU0SpxB38PDw4L///S8dOnSgZcuWPPbYY0aH5BDkGmg52LFjB2FhYaxdu5aqVasaHY4QQtC4cWNWr17Nn/70J86cOWN0OA5BEmgZO3fuHE8//TSrVq2icePGRocjhBA5Hn74YV599VWGDRtGRmEDMIhCSQItQ5mZmYwYMYJnn32Wnj17Gh2OEELc5ZVXXqFevXq89tprRodi9ySBlqG///3vuLm5SYtbIYTNcnFxYcWKFXz99desW7fO6HDsmjQiKiNbtmxh1apVHD58mEqVKhkdjhBC5KtmzZp8+eWX9OvXj+DgYFq0aGF0SHZJSqBl4LfffmPcuHGEh4dTr149o8MRQohCtWvXjhkzZjB06FDS09ONDscuSQItpezrnq+++ioPP/yw0eEIIUSRvfDCC/j6+vLKK68YHYpdkgRaSvPmzcPDw0MOQCGE3VFK8cknn7Blyxa+++47o8OxO3INtBSioqJ4//33OXLkiIztKYSwSzVq1GDZsmWMHTuW6Ohoakof0EUmv/oldPv2bUaPHs3ChQu57777jA5HCCFK7JFHHmHw4MFMmTLF6FDsiiTQEpoxYwYtWrRg9OjRRocihBClNnfuXA4ePMhXX31ldCh2QxJoCezbt48VK1bwySefSD+3TmrcuHHUq1ePwMDAnGkzZ86kcePGhISEEBISwjfffGNghEIUj7u7OytWrGDy5MmcP3/e6HDsgiTQYkpLS2PMmDF89NFHcsuKM0pJgaQkxg4dSkRExF2zp06dSlRUFFFRUdJht7A7oaGhjB8/nj//+c8UZ6QuZyUJtJhee+01OnbsyODBg40ORVQUkwlWr9YH265dG3x96TJwILWefhquX9fnC+EgZsyYwZkzZ1i+fLnRodg8SaDFsG/fPjZs2MAHH3xgdCiioly9CqGhMHEiREVBZibcvKn/PXECkpP1+VevArB48WKCgoIYN24cVy3ThLAnVapUYcWKFUyfPp2LFy8aHY5NkwRaRJmZmTz//PMsXLiQe++91+hwnJOl+jQxNpbu3bvj7+9PQEAAixYtAmDatGn4+fkRFBTE4MGDuXbtWum2ZzJBz54QEwM3buS9jNmsz+/Zk+cnTODXX38lKiqKhg0byr3Bwm4FBwczevRo6XC+EJJAi+iDDz6gbt26jBgxwuhQnEse1aeVg4JYmJzMiRkziNyzh7CwMGJjY+nVqxcxMTFER0fj4+PDu+++W7ptr1sHcXFQ2LBPGRkQF0f9vXupVKkSLi4uTJw4kQMHDpRu+0IYaObMmWzfvp1du3YZHYrNkgRaBElJSbzzzjt89NFH0uq2IuVTfdowK4vW8fEwcSLVe/XCv0ULzp49S+/evalcWe8bJDQ0lKSkpNJtf948SEsr2rJpaZybPTvn5YYNG+5ooev0LLUHty5epH379gQHBxMQEMCMGTMA+P333+nVqxfe3t706tVLqr9tQPXq1Vm0aBHPP/+8jB2aD0mgRfDSSy8xadIkfHx8jA7FeRSl+vTGDU7//DNHt22jQ+vWd8xatmwZjz76aMm3n5KibzsfI4GOQBzgBSwFXjt+nJYBAQQFBfHDDz/w3nvvlXz7jiCP2gO3Ro3YcesWx954g6iDB4mIiCAyMpK5c+fyyCOPkJCQwCOPPMLcuXONjl4AgwcPplmzZvzrX/8yOhTbpGlakR9t2rTRnM2WLVu05s2ba+np6UaH4lzCwzXNw0PTIN9HKmitQVvn5qZpq1fnvHX27Nna448/rpnN5pJvPzFR09zdC9z+XQ93d/19QtN+/13TWrfWtGrV8v6sqlXT0kJCtFZBQVpkZKTm4+OjJScna5qmacnJyZqPj4/BOyCy/frrr1rt2rW1U6dOGR2KIYBDWj45UUqgBbh58yaTJ08mLCyMqlWrGh2Ocymk+tQEPAE8BQy5fVtfHlixYgWbN2/m888/L111u6dngdc+bwHtgWAgAJgBYDIxbf78sm3IZI8KqT3IAkJu3KBeVBS9LlygQ+vWXLhwgYYNGwLQsGFDaf1pQx544AGmTp3KlClT5N7QXCSBFmDevHm0bduWPn36GB2Kcymk+lQDxgP+wMvZE2NiiFi/nnnz5rFp0ybc3d1LF4OnJxRwDdMN2AEcA6KACCCyWTN69e9ftg2Z7FEhja8qoX9mScCBS5eIsbSiFrbr1Vdf5eTJk2zevNnoUGyKJNB8JCcns3jxYhYsWGB0KM4nJQWqVMl39l5gFXoCC7E8vnFxYfLLL5OamkqvXr0ICQnhueeeK10c06eDh0eesxRQzfLcBJhcXFCjR5d9QyZ7VMTGV/cC3cxmIt57j/r163Pu3DkAzp07J7182Rg3NzcWLlzIa6+9RmZmptHh2AxJoPmYOXMm48ePp0mTJkaH4nwKqT59GL0UGo1ekokCHjObORkdTWJiYk5XekuWLCldHE88Ac2b5zs7Cz151wN61a5Nh+nT75hf6oZM9qiQ2oNLQHaldjqwDfC7cIGBffqwYsUKQK+GHzRoUHlHKorp0UcfpWHDhixbtszoUGyGJNA8xMbGsnHjRt544w2jQ3FOhVSf5ikwUH9fWVNKf+ThjqrImzeJsUocc+bMoXLlyjz11FNlH5MtK6T24BzQHQgC2gG9gP5ubrw+cSJbt27F29ubrVu38vrrr1dMvKLIlFIsWLCAmTNnciO/lvFORhJoHqZPn84bb7whA8saqYDq07t4eOjLl7V16+DkSb3daAHuBbplZBCxcCFQhg2Z7FEhtQdBwFH02oMY4O8AJhO1mzRh+/btJCQksH37dmrVqlUh4YriadOmDd27d2eh5Vh3dpJAc9m5cycxMTG88MILRofi3J54Anx9CyzNAPp8Pz8YMqTsYyjgWt5dVZEmE3779xMREVF2DZnskS3VHohyMXv2bD744AMZ8gxJoHcwm81MmzaNd955Bzc3N6PDcW6urrBtm/7jml9J1MMDWraErVv15ctSIdfy8qyKTExk8gsvlG1DJntkC7UHotw0a9aMsWPH8vbbbxsdiuFUce7radu2rXbo0KFyDMdYa9asYcGCBRw4cAAXFzm3sAkmE6xfr5cGY2L0RGky6Yl1+nS95FnWyRMgKUkvAd+8WfT3uLvrt294eZV9PPbEZNK7YIyJKbgf4SpV9BOg/fvL5zsU5eb333/H19eXPXv24OfnZ3Q45UopdVjTtLZ5zpMEqsvMzMTPz49PP/2UHj16GB2OyEtKiv7w9Cz/Kr+UFL37ueI02Xd1hcuXpToS9H6Me/bUTyjyqgb38NCr3rduBWlrYJcWLFjATz/9xFdffWV0KOWqoAQqxSyL1atX07hxY0metszTUy/dVUSCkmt5pVOzJkRGwtKlel+4rq56Cd3VVX+9dKle8pTkabdeeOEF9uzZw/Hjx40OxTCSQNGvfc6ZM4e33nrL6FCELZFreaXj6grDh8ORI3rJPC5O/3vkiD49n2rbcePGUa9evTtGszl27BgdO3akZcuWDBgwgJSUlIraC5EPDw8PXnrpJefsbctCEiiwfv16PD096dmzp9GhCFtiCy2BDWIZfYyUFHjvvfcICAggMDCQkSNHcuvWreKvsAi1B9nbHDp0LBEREXfMmzBhAnPnzuXnn39m8ODBDtVDWPZ+jx5994nD8OHDCQkJISQkhKZNmxISEmJgpHebNGkSERERnDx50uhQjJFfL/N5PRxxNBaz2awFBwdrmzZtMjoUYYuyRxXJb2QYDw9Na9NGX87OZWTog+CEhGha5cr64DKVKiVpVao01VasuKllZGja0KFDtX//+9/lus3KlTXN3/+U5uUVoGVk6MtVr149Z3SdM2fOaP7+/mUWgxHy/qx3aT4+h+/Yb2svv/yy9vbbb1d8sIX4+9//ro0fP97oMMoNBYzG4vQJdNOmTVpwcHDphr4Sji0jQx8urVUrTXN11X/tXF3116tXa3n+2tmZ/EcfS9LAS/PwuKK1amXSevfup3333XflvE1Ng1Oai0uA1rq1vlzHjh21jRs3apqmaQsXLtSqVatWJjEYoTj7nc1sNmteXl5afHy8cYHn48qVK1qtWrW006dPGx1KuSgogTp1Fa6macyePZu33nrL+XqMEUVXwmt59qLg0ccaA6+SlnY/R4825ODBGnTv3ruct6kzm/X5PXvCJ58sIywsjDZt2pCamkqVwqrVbVRx99tk0qft2bOH+vXr4+3tXXHBFlGtWrWYOHEi8+fPNzqUCufUCXTbtm2kpqYyxIGuXYlyVpEtgStIwaOPXQW+Bk4ByaSkpDF16n/KeZt/yMjQl4uN9eP777/n8OHDjBw5kuYFdPJvy4q73+vX66/Dw8MZOXJk+QdYQi+//DLh4eEkJycbHUqFcuoEOm/ePF5//XXpNEE4tYJHH9sGNAPqAq5kZQ1hzZp95bzNO6WlwezZ+gDbZrOZ2bNn220PT4XvdzL6yYo/aWkBvPLKIjIzM/niiy/49NNPcXFxwRbvxa9Xrx6jR49mkZON7eq0meP48eMcP36cESNGGB2KEIYppMdC4H4gEriJPojcdn7/3Z/S3EVS+DZHAh2BOMALWEpsbDgtWvjg5+dHo0aN+NOf/lTyAAxS+H4DVAYaACeASM6eDSMsbCk+Pj5s2rSJLl26lHucJfXiiy+ybNky0tPTjQ6lwlQ2OgCjLF68mGeffdZur6UIURayRx/Lv8OlDsCTQGv0n4tWuLn9OadDqPLZZvhdU6pWhZ07/2LXvSQWvt8jgZ3AZfQTh7dxcfFn3bp1jB8/Hl9f3wqKtGSaN29Ohw4d+OKLLxg/frzR4VQIp0yg165dY/Xq1cTGxhodihCGKmT0MYu3LQ9dZmbpLgEXbZt3Mpns/7Jz4fud+8ThNGbz26xcGUPTpvax85MnT+aNN95g3LhxTtEw0ymrcJcvX54zuroQzsyIHgudtZfE4u33DeAJlHqfn36ynx3v3bs3aWlp7N271+hQKoTTJVCz2UxYWBhTpkwxOhQhbIIRPRY6ay+J06dD4QUzE/AE8BSaNoR588o/rrLi4uLC5MmT+fDDD40OpUI4XQKNiIjA09OT0NBQo0MRwiYY0WOhs/aS2KuX3l1C/jRgPOAPvAzoDY/sqevfsWPHsnXrVs6ePWt0KOXO6RLohx9+yJQpU5yifl6IojBi7HKjx0s3Sno63HNPQUvsBVYBO4AQIAQXl29Ys2YDXl5e7N+/n379+tGnT5+KCLdEPD09GTVqFEuWLDE6lHLnVOOB/vrrr3Ts2JEzZ85QtWpVo8MRwqYYMXa5UeOlG8VZhpk9ceIE3bp1IykpCVc7/wJlPFCLlStXMmrUKEmeQuTBiB4Lbb2XxIJGSikJZ2lA5e/vT4sWLe4aVcfROE0CNZvNrFy5kmeeecboUISweUb0WGgrvSSaTLB6tT7ud+3a+rXa8PCx1KwZwfXrf/RPW1LO0oDqmWeeYeXKlUaHUa6cJoHu3bsXDw8PWrVqZXQoQggbdfUqhIbCxIkQFaVXtd68CVlZXYiPr0Vysj7/6tWSb8NZGlANGzaM77//nqul+bBsnNMk0JUrVzJmzBhpPCSEyFNJR0opLmdpQFWzZk369OnDl19+aXQo5cYpEmh6ejrr1q3jqaeeMjoUIYSNKulIKSVRsyZERsLSpXpVsasruLvrf1u10qfv368vZ88cvRrXKRLopk2baNeuHY0aNTI6FCGEjSruCDGl7eDA1htQlYU+ffpw8uRJTp48aXQo5cIpEqg0HqpAlmaLibGxdO/eHX9/fwICAu4Y5ujDDz/E19eXgIAAXnvtNQODFUJXtJFS7lSWHRzYSgOqsubq6sqoUaNYtWqV0aGUC4fvTP7ChQvs27fPoevhDWcy6fVf2TfzValC5du3Wdi8Oa3ffpvU3r1pExpKr169uHDhAl9//TXR0dG4ublx8eJFo6MXokQjpbi6ji/VqDTO4plnnmHIkCHMnDnT4dqgOHwC3bhxI48++igeRW03Lorn6lW9RUV8/B8tLzIzaQg0jI+HiROp7uODf4sWnD17ls8++4zXX38dNzc3QB+IVwijFX+kFMcYIaYihISEUKVKFY4cOUKbNm2MDqdMOXwV7saNGxk8eLDRYTimojRbvHGD0z//zNFt2+jQujXx8fHs2bOHDh060LVrVw4ePFixMQuRB2fp4MAISikef/xxNm7caHQoZc6hE2hKSgp79+6lb9++RofimIrQbPEG8ITJxPsuLnhu20ZmZiZXr14lMjKSBQsWMGzYMIrTnaQQ5cVZOjgwgiRQO/Ttt9/SuXNnqlevbnQojqmQZot/DMoEQ27fhnnz8PLyYsiQISilaN++PS4uLly+fLmiIhYiX87SwYEROnTowOXLlx2uNa5DJ9CNGzfy+OOPGx2GY0hJIfHAAbp36aK3rPX3Z1F0NADHgI5AS2AAkEJegzIBMTE83qcPO3bsACA+Pp6MjAzq1KlTsfsiRB6cpYMDI7i4uDBw4EC+/vpro0MpUw6bQG/fvk1ERAQDBgwwOhT7latT0MrdurFw3z5OVK1K5IQJhGkascAEYC7wMzAYWEBegzLBNy4ujBs0iP/9738EBgYyYsQIVqxY4XAt84T9cpYODowwaNAgh0ugDjuc2Xfffcfbb7/Nvn37jA7FPuXVutaahweD0tKYjF5Nex1QQCLQB4jNa532OC6TcGopKeTcqiKHbencunWLBg0aEB8fb1et751yODOpvi2FIrSuPZ2WxlGgAxAIbLJMX4ueRPMkzRaFnXHUDg6MULVqVXr37s3mzZuNDqXMOGQC1TSNzZs3M3DgQKNDsU+FtK69gV7qfB/wBJYBYUAbIBXIsw2GNFsUwukNHDiQ//73v0aHUWYcMoFmt/Ty9fU1OBI7VUDr2jta1lqm+QHfA4fR+2tpnvtN0mxRCAH07NmTXbt2kZWVZXQoZcIhE+iOHTvo0aOHNE4piQI6Bc2zZS1wEcDdHTMwG3jO+k020mzx1q1btG/fnuDgYAICApgxYwYAf/vb3wgKCiIkJITevXuTnJxsWIxCOLoGDRrQsGFDoqKijA6lTDh0AhUlkN0paB7ybFkLhLu64lOtGn5ubjRyceFP99xjO80WLZ3bu92+zY4dOzh27BhRUVFEREQQGRnJtGnTiI6OJioqiv79+zNr1ixj4hTCSfTo0SPnVjZ753AJ1Gw288MPP9C9e3ejQ7FPBXQK+jB6KTQaiLI8HgP+AsQnJBB/6xZzr15FxccbOy5Trttv8PVF1alDtc6dYfVqTDdvYjKZUErhadU6JC0tTWothChnjpRAHa4z+ePHj1OjRg3uv/9+o0OxT9mdghanisW6da3R7f3z6dweICsqijZPPcVJYNLkyXTo0AGAN998k5UrV1KjRg1++OEHgwIXwjl07dqVMWPGkJGRQZXCun2ycQ5XAt2+fbtU35aWvXYKWsjtN5WAKLOZpEqVOLBsGTFHjwIwZ84cEhMTeeqpp1i8eHEFBy2Ec6lVqxbe3t4cOHDA6FBKzeESqFz/LAP22iloETq3B7jXZKLb7dtELFx4x/RRo0axbt268oxQCIHjVOM6VALNzMxk9+7ddOvWzehQ7Ju9dgpawO03l4BrlufpwDaTCb/9+0lISMhZZtOmTfj5+ZV7mEI4O0dJoA51DTQ2Npb69etTv359o0Oxf9mdgq5fryemmBg9UZpMemKdPl0vedpK8izg9huAc8AYIAswA8OA/omJPPHKK8T973+4uLjQpEkTlixZUjHxCuHEOnXqxKFDh8jMzKRyZftNQ/YbeR4OHjxIu3btjA7Dcbi66q1ohw+3/U5Bs2+/sTQYyi0IOJp7oqsr6z76SO+rTQhRYTw9PfHy8iI2NpagoCCjwykxh6rCPXTokCTQ8mLrnYIWcPtNvkwm290fUWqWW4AZPXoc9erVIzAwMGeedKBhvHbt2nHw4EGjwygVh0qgBw8epG3bPDvNF44u+/ab4pDO7R1OHrcAEx4+lpo1I7h+XZ8PSAcaNqBt27bYy+he+XGYBHr79m1iY2Np1aqV0aEIo9jr7TeiTFy9CqGhMHGifhtzZibcvAlZWV2Ij69FcrI+/+pVpAMNGyAlUBsSHR2Nt7c37u7uRocijGKvt9+IUivCCHyYzfr8nj315d98803uu+8+Pv/8cymBGiAkJITY2Fhu375tdCgl5jAJ9NChQ1J96+zs9fYbUWpFvAWYjAx9ufXrpQMNo7m7u+Pt7U10dLTRoZSYwyRQaYErgD9uv1m6VL8Q5uoK7u6207m9KBcF3AJ8l7Q0ffls0oGGcdq2bWvX1bgOcxvL4cOHeeGFF4wOQ9gCe7r9RpRaIbcA5+nnnxNISfHG01M60DCSJFAbkJWVRXx8PA8++KDRoQhbI4nT4RVyCzD6MO87gcuAF/A2mvYN7drF4eYmHWgYKTAwkJUrVxodRok5RAL97bffqFevnjQgEsIJFX4LcPhdU1xcxnPwoJxbGc3X15e4uDg0TbPLltAOcQ30l19+wdfX1+gwhBAGkFuA7VfdunXRNI3Lly8bHUqJOEQCjYuLkwQqhBOTW4Dtk1IKPz8/4uLijA6lRCSBCiHsntwCbL98fX355ZdfjA6jRBwmgUorOiGcl9wCbL+yr4PaI4dIoHINVAghtwDbJ3tOoHbfCjclJYXU1FQaN25sdChCCIPJLcD2x56vgdp9Ak1ISKB58+a4uDhEYVoIUUYkcdqH5s2bc/r0abscXNvus87Zs2e57777jA5DCCFECbi5uVGzZk0uXrxodCjFZvcJNDk5WapvhRDCjjVu3NguBzV3iATaqFEjo8MQQghRQo0aNZIEagRJoEIIYd8kgRpEEqgQQtg3SaAGkQQqhBD2TRKoQSSBloGUFEhKIjE2lu7du+Pv709AQACLFi0C4G9/+xtBQUGEhITQu3dvuzzQhRC2SxKoATIyMrh27Rp169Y1OhT7YzLB6tV6Fy21a4OvL5WDgliYnMyJGTOI3LOHsLAwYmNjmTZtGtHR0URFRdG/f39mzZpldPRCOISIiAh8fX1p0aIFc+fONTocw9hrArWvu1ZzuXjxInXq1JFOFIrr6lXo2RPi4+HGDX1aZiYNgYbx8TBxItV9fPBv0YKzZ8/eMVB5WlqaXY7bJ4StyO4hycMji0mTJrF161a8vLxo164dAwcOvOP/zVk0aNCAc+fOGR1Gsdl15rl+/To1atQwOgz7YjLpyTMm5o/kmduNG5z++WeObttGh9atAXjzzTe57777+Pzzz6UEKkQx5VHhQ926B7h0qQUHDjyAUlUYMWIEX3/9tdGhGsLT05OUlBSjwyg2u06gqampVK9e3egw7Mu6dRAXBxkZ+S5yA3jCZOJ9Fxc8t20DYM6cOSQmJvLUU0+xePHiCgpWCPt39SqEhsLEiRAVBZmZcPMmZGWdJTX1PiZO1OfXrOnF2bNnjQ7XEO7u7mRkZJCZmWl0KMUiCdTZzJsHaWn5zjYBTwBPAUNu39aXtzJq1CjWrVtXriEaKiWFWydP0r5NG4KDgwkICGDGjBkArF27loCAAFxcXDh06JDBgQp7UHCFjwbo02NiYO5c0DTnvDyilKJatWrcyK9WzEZJAnUmKSn6f2o+NGA84A+8nD0xJoaEo0dzltm0aZPjjb2aq37NLSiIHceOcczFhag33iDi22+JjIwkMDCQ9evX06VLF6MjFnai4AofLyAR0OcnJyfx++/Oe0dB9erVSU1NNTqMYrHrRkSSQIspJQWqVNHrkPKwF1gFtARCLNPecXFh6ZtvEnfmDC4uLjRp0oQlS5ZUUMAVII8GVSozk2oAUVGYnnsOk9mMSk3FPzTU0FCF/Sm4wqcdkACcAhpjMq0mKuqLCovN1kgCrWCSQIvJ07PAa58Pk12pZMVs5rHVqx1zXCjr+rVcn0sW0AY4mZbGJBcXOrz+OnTrpg84KUQRemt1MgAAIABJREFUFFLhg/7zuxjog37EjePXXwNyxjB1NvaYQKUK15l4ekJgYPHeExjouP/NBdSvVQKigCTggNlMzIkTsH59RUco7Fh2hU/BHgPigV+BN3F11d/njCSBVrDU1FQ8HfXHvbxMnw4eHkVb1sNDX95RFdKgCuBeoBsQkZ5+V4MqIQpSSIVPnkwmxz1fLYynp6ck0Ip08+ZN3N3djQ7DvjzxhH4TWmGnxlWqgJ8fDBlSMXFVtALq1y4B1yzP04FtgB/oyztr8UAUm1T4FI+Hh4e0wq1IWVlZVK5s15dxK56rK2zbpv+n5lcS9fCAli1h61bHveZXQP3aOaA7EITezKMX0B/YoBRevr7s37+ffv360adPnwoLV9gnqfApukqVKmE2m40Oo1jsOoGazWbpxq8kataEyEhYulS/dcPVFdzd9b+tWunT9+/Xl3NUBdSvBQFHgWggBvi7ZfpgTSMpLo7bt29z4cIFvvvuu4qJVdgtqfApOhcXF0mgFSkrK0sSaEm5usLw4bBzJxw7RuJ//0v30FD809MJmDWLRR99BMDw4cMJCQkhJCSEpk2bEhISUvB67YXUr4kKIBU+RWePCdSu6z+lBFpCJpPeAnXePP26XpUqVL59m4XNm9P67bdJ7d2bNqGh9OrVizVr1uS87ZVXXnGsvoenT4cJEwptSARI/ZoosewKn/Xr//iXc3XV/w0DA/XDasgQ506eoCfQrKwso8MoFrtOoDIqSAmUcCQWTdP48ssv2bFjh3Gxl7UnnoAFC/K8D/QOUr8mSim7wmf48D9GY/H0lAoNa5qm2V2ByL6izcUei/yGKuFILAB79uyhfv36eHt7V1CwFUDq14QBPD3By0uSZ272WKNoX9HmIgm0mEo4EgtAeHg4I0eOrIAgK5g0qBLCJthjArXrKlx7rDM3VElGYhk+nMzMTNavX8/hw4crKtKKJfVrQhjOHhOofUWbi6urKyaTyegw8paSQuKBA3Tv0gV/f38CAgJYtGgRADNnzqRx48Y5rVu/+eabComnJCOxkJLCtm3b8PPzw8vLq/zjNJqT1q+lpMCBA4l06dL9ruNViIqQmZlpd/f123UCtbnx43INi1W5WzcW7tvHiapViXz1VcIWLyY2NhaAqVOnEhUVRVRUFI899lj5x1ZIx5zZI7HsQB+JJQT4xsUFUlJYvXq1Y1bfOrlchyvdulVm376FVK16gldfjWTx4rCc41WI8paamkq1atWMDqNY7Cvd51K9enUuXLhgdBi6PFq3NrS0biUqiuovvoh/pUqc/eUXuH0bMjNJjI3lmUmTOH/+PC4uLvz5z3/mL3/5S84q//nPfzJt2jQuXbpEnTp1ShdfCUdiwdOT5cuXl27bwubk3Ri7IdCQqCh48cXqVKrkzy+//NESWziH3FcxsrKyaNu2LY0bN2bz5s3ltl17HBzErkugNtN7fxFat56+cYOj16/T4amnYN48Fr/zDn0CAvA8coR9r7xC5J49hIX9ccafmJjI1q1buf/++8smRuk4QFgUrTH2aa5fP8qsWR2w1askouzkro3w9dX/tmoFY8cuwtfXv9xjkARawWwmgRbSuvUGeuOc9wHPW7d4XtP4VdOIAQJTUnjl+eep3qtXzr2XoFfxzp8/v2zvdZWOOQVFaYz9xxF78qSnjOLm4K5ehdBQmDgRoqIgMxNu3tT/RkUlER6+hSNHJpT7iZQk0ApmMwm0gNatd7RstUyrjz7epAswETiQmXnHvZebNm2icePGBAcHl22c0jGnoLDG2HcesWlpMoqbIyu8NuIlsrLmc+qUC5GRlGsSlQRawapXr06K0cNLFdC6Nc+WreijfWTbgD5UVva9l5W3bGHOnDnMmjWr7GOVjgOcXsGNsfM+YmUUN9uVkgJJSfrfcePGUa9ePQKtLtVMmzYNPz8/goKCGDx4MNeuXbvj/QXXRmwG6gFtyMzUT7rKszYiJSVFEmhFsokSaAGtW/Ns2Qq8BrREH/VjO3CZP+69/PXddzl16hTBwcE0bdqUpKQkWrduzfnz58smXuk4wKkV3Bg77yPW1VUSqC3J73rlnj1jmTo1As2qNWCvXr2IiYkhOjoaHx8f3n333TvWVXBtxF5gE9AUGEFW1g6ef/7p8tglsrKySE9Px6Ool5hshNK0u9pe5qtt27baoUOHyjGc4omLi6N///78f3t3HlZ1mbh//H1QBEVN0zTItBwWDUglF0ZmXFI009TcGm20TB23Mr9NX7O+M5dTM0xSv6IybNQ0sXEpd0NzNzOTXHCJNETR3Cg3FMFUlvP74whZIssReM5yv67rXCgc4I7U+3ye5/k8T0pKirkQGRm2P705OaX+VCvwFHAntvlRwFZkZ88WLN6577772Llz5+2vwr0VbRzgVuz54/qbP5JiUGGrp29UrdpR8vJ6cOpU0k2vgZcuXcqiRYuYO3cuUNo/C18A/w9Pz/hy+bNw8eJFGjRoYP6CqBAWi2WX1WptWdjHnPoKtFatWjcNSVQ4e1a3XlfUvZcVxk03DnBXWoztvEqyevryZdtdcp073zxfOWvWLLp161bw+2JuDS9UeY1GXLhwgVq1apX9Fy5nTl2gd911FxcvXuRaUSdpVITSrG69Qf69l/uAPdcfj16/9zLf0aNHy+/qU9ySFmM7pxJsZQ2A1Wp73o3zlVFRUVSuXJknn3yy4H3F3Br+Gx2AeLKzy+fFVFpaGn5+fmX/hcuZUxeoh4cH9evXL7v5QXuVdHVrSejlvpQzLcYuH/kLek6fvkLr1q1p1qwZwcHBTJo0qUy+fjFbWf/Kjaun4+LiiI+PZ+7cub+6Lc6RRiNOnTqlAjXBz8+PU6dOmQ1RktWtJaGX+1IBtBi77BS2oMfPz4srVzby8st72bFjD6tXryYhIeG2vk8xW1kXKikJlixZTXR0NCtWrKBatWo3Pac0oxFVq8Jzz9mGW/v160eTJk1o2rQp27ZtK12wQqhADXGIAoVbr26tXNn2J69SpaI/Xy/3pQJpMfbtu9UGBLm5Fr79tjojRkBERDZXrmTf9oYoJZuvHAj8HkgGGmCxzOSFF57l0qVLREZG0rx5c0aNGvWrzyjN4NmVK/CXv0Djxs9z552PsHXr96xcuZeYmNibbp8BmDJlCkFBQQQHBzNhwoQiv7azFqhT74ULtgLN373HuFsdi5Wba5vVT04ufAzGx8dWnnq5LxVIp7jZ78YFPYXPI+aSmfkQe/Ycon79sYSFtbmt71ey+cr5v/qd1Qr79g0r8v9n/mhEUf883fj1cnIySE//kg8/nM2MGVC1ahWuXfsLv/vdC1y8OITsbNvX3LRpE8uXL2ffvn14eXlx+vTpIpOfPHmSDh06FPcf6HB0BVpeblzdqpf74uC0GLt0il/QUwnbssATnDmznXffLeX462+U53zlb/95qlQJfrlgvg/bXevNgZZAKnAXVutQoAU//zyc3NyHOHjwTk6dsl2Rp6fDBx98wMSJE/Hy8gKgXr16RWZw1itQpy/Qe+65xzEL9LfyX+4nJtpuqktOtr1NTLS9X1eeIk6j5At6apGX14GYmNW3/T1vd/V0YTsVLVy4kODgYLy8PDh0aCdWKwWPX2zC9mJgJ5ADJAKjgd2ADzAZsB3elJSUfzV7kC1bttCmTRvat2/Pjh07isx76tQp7rnnnpL9xzkQpy9Qh70CLYpe7os4reIX9JwB8u9P/xlYz08/Nbnt+yftXT2dvzq4f/+nWb3610UeEhLCRx8twcenHVFRtrncvLyivnqD64/8Iel+2ArV5to127XB+fM5pKenk5CQwJtvvsmAAQMoatMeZ70Cdfo50HvvvZcffvjBdAwRcRP5C3puvYNPGrY9xnKBPGAAXl49CuaY7VWS+cr85RSrVtmGmaOjbWVfpQpcu9aOgICjXLpEwVylv39TwsNtX6vwfrMAXa6/HQn8BbgX20KlIGybkf76vNisLPDwaECfPn2wWCy0bt0aDw8Pzp49y1133XXTd8jIyCA7O5vaTjiF5fQF2rhxY44dO8a1a9eoUhb3YYqIFKH4BT0PYhve/EVZbUCQP1+5ZMkv5ejpafv6ISG2YduOHaFbt98elm57e+AAeHjY5irXr4c1a2xlfOuLw62AH3AaiMR29MUUbLt3XwMaAx8BF3/1WVlZvfn884106NCBgwcPcu3atVtuCJOcnExgYGDZHt1YQZy+QL28vGjQoAGpqak0adLEdBwRcXH5C3r27Cn555TlBgRFrZ7OzraV461XB/96rjI3t7i53Pxh1XrA48B24EVs86H5BmLbK/cstuHdV/HyeoaUlGcICQmhSpUqxMXF3bIgk5OTCQoKKvkPwIE4/RwoQJMmTUhOTjYdQ0TcRHlsh5g/Vzl48M2Lfc6fP09kZCQBAQFERkaSnp4O3LycoqTb/eXPVX77bVHPygIu3fDrtUBhS4HnYxu2zgZOAMPIyanC7Nn/JSkpicTERB5++OFbfpfvv/9eBWpSUFAQ33//vekYIuImymo7xMJ2Mpo//2lq117NxYu/bAg/efJkOnXqREpKCp06dWLy5MmFfr3SbvdX9GFcP2HbsbsZ0BroDjxSoq9dmituXYEaFhQUpCtQEakwZbEd4q13Mmp3032Vy5cv56mnngLgqaeeYtmyZTd9veJXB/96pyKYidW69Pqvt2EryK43PL8xsPf64zvg/4r8mdz4312aHUlVoIapQEWkot3O/iglOZrsxrnKn376CV9fXwB8fX0L3dmn+O3+bh5qtVgev/7rq9iuONeU/AdQiNLuSJqbm0tKSgqBgYG39X1NcfpFRKA5UBExw97tEEs7V1n0UKtN6Y4ns/HwAC8v25VvaT6nsHtF7dmR9NixY9StW5fq1auXPIADcYkr0Hr16pGTk8PZs2dNRxERN1Wa/VFKO1eZm1uftLQ0wHZ2ZmFb49mz3V9oqK30ipvL9fS0PTctDebNK7sdSZ15+BZcpEAtFguhoaHs27fPdBQRkSLZczRZdnZPpk2LA2zne/bq1avQ55V2dfDEiSWby33wQdi8Ge6+u2x3JN27d+9Np7g4E5coUICWLVsWu9+iiIhpxc9V3rzYx8trIuvXryMgIIB169YxceLEQj/TntXB9s7llsWOpDt37qRVq1b2fwHDXGIOFKBVq1YsXbrUdAwRkSIVP1c5/6b35OTAqlUbii2r0mz3d+Ncpamj7Xbs2EFUVFT5fpNypCtQEZEKVJ5Hk8Htn55YUWddnDlzhgsXLuDv71++36gcuUyB+vv7c+HCBc6cOWM6iohIkcpjJ6MbOcPpiTt37uShhx7Cw8N5a8h5k/+Gh4cHDz30EDt37iz+ySIit5C/pd6uXck0b9684FGzZk3eeeedMvkeZbWTUUk46umJO3bscOr5T3ChAgUN44qIfQrbUi88PAiLZQ8TJ+4hIWEX1apV4/HHHy+T71cWOxk5u507d9KyZUvTMW6LSxVoq1atVKAiUiq32lIvJ8f2+xEjIDR0A40a/Y5GjRqV2fe93blKZ2a1Wl3iCtRlVuGCrUDHjh2L1Wp1yrPlRKRi3bil3q1WxmZmQmrqAq5cGVhwEHVZMbX61bSTJ0+Sk5NDw4YNTUe5LS51BdqwYUO8vb21rZ+IlEjJttS7Rl7eCs6f78+SJeWXxVHnKsvDF198Qbt27Zz+QselCtRisfDwww+zceNG01FExAmUbEu9z4EwLl+uT3R0BYRyAxs3bizyjFBn4VIFCqhARaRESr6l3nxsuwPZnp+RUZ6pXJ/VamXDhg0qUEfUsWNHNm3aRF5hxwWIiFxX/JZ6AJeBdYDtPhJPTxXo7Tpy5AjXrl2jSZMmpqPcNpcr0AYNGlC3bl1tLC+u6fpNisf376djx440bdqU4OBg3n33XQD27NlDeHg4zZs3p2XLlmzfvt1wYMdVsuO/qgHngDsA26Ijd5ijLE/5w7fOPv8JLligoGFccTGF3KRY+cEHeevUKQ5MmkTCli3Exsayf/9+JkyYwKRJk9izZw+vvfYaEyZMMJ3eYZX3lnpSOFeZ/wQVqIhju8VNir65uYQdPAgjRlAjMpKm/v6cPHkSi8VCxvUxxosXL+Ln52f4P8CxlfeWevJrVqvVpQrUYi3JUefXtWzZ0uoMW+WdOXMGf39/zp49i6crbuEh7iE721aeRd2kCBz19KQdkJSWxsnTp+natStWq5W8vDy+/vrrMr3539WU8EdMlSq2XYG2bXPNXYEqyv79++nevTtHjhwxHaXELBbLLqvVWuiWSS55BXrXXXfRuHFjEhISTEcRsV8JblLMBPpmZ/OOhwc116/ngw8+ICYmhuPHjxMTE8OwYcMqLq8T0pZ6FWvNmjV07tzZdIwy45IFCtCzZ0+WL19uOoaI/Yq5STEb6As8CfS5ehWio4mLi6PP9Z3H+/fvr0VEJeDOW+pVtOXLl9OrVy/TMcqMyxZo7969WbZsGaUZohZxGMXcpGgFhgFNgRfy35mUhN/dd7N582bAtlgjICCgnIO6Bmc4/svZnT17lt27d9OpUyfTUcqMS+2Fe6PmzZuTnZ3Nd999R0hpl9qJmJZ/k2JOTqEf3gp8DIQCza+/798eHsx4/XWe/+tfycnJwdvbm+nTp1dQYNfhDnvRmhAfH09kZCRVq1Y1HaXMuGyBWiyWgqtQFag4nWJuUvwDtqvQX8nLg86d2XU7h0eKlJNly5bRr18/0zHKlMsO4cIvw7giTkc3KYoLuXz5Mps2baJ79+6mo5Qply7QP/7xjxw5coTjx4+bjiJSerpJUVzEunXraNmyJbVdbCWWSxdo5cqV6dGjBytWrDAdRaT0+vaFoKDiN2ytUgWaNAEN3YqDWrZsGb179zYdo8y5dIGCbRh3SXke4idSXnSToriA7Oxs4uPjXer2lXwuX6CPPPIIu3fv5uTJk6ajiJSeblIUJ7d27VoCAwNp2LCh6ShlzuULtGrVqvTt25d58+aZjiJiH92kKE4sLi6OIUOGmI5RLly+QAGGDBlCXFycNlUQ51ezJjRooNW24hTS09NZs2YNAwYMMB2lXLhFgUZERHD58mV2795tOoqIiNtYuHAhXbp0cbnVt/ncokA9PDwYPHgwc+bMMR1FRMRtzJkzx2WHb8FNChRg8ODBzJ8/n+zsbNNRRERc3uHDhzl48CCPPPKI6Sjlxm0K1N/fH39/f9asWWM6ioiIy5szZw4DBw506TOZ3aZA4ZfFRCIiUn7y8vL4+OOPXXr4FtysQJ944gk2bNjAjz/+aDqKiIjLWr9+PTVq1CAsLMx0lHLlVgVaq1YtBgwYoCOeRETK0ZQpU3juueewWCymo5QrtypQgLFjx/Kf//yHa0UcFSUiIvZJTU1l27ZtDBo0yHSUcud2BRoaGkpQUJD2xxURKQdTp05l6NChVKtWzXSUcud2BQrw3HPPMWXKFNMxRERcSlZWFrNnz2bMmDGmo1QItyzQnj17cuLECRITE01HERFxGXPnziUiIoL777/fdJQK4ZYFWrlyZUaPHs37779vOoqIiEuwWq28//77PPfcc6ajVBi3LFCA4cOHs3TpUs6ePWs6ioiI0/vyyy/Jzs6mU6dOpqNUGLct0Lp169KnTx9iY2NNRxERcXpvvfUW48aNc/lbV27ktgUK8NJLL/H++++TkZFhOoqIiNPas2cPO3fuZOjQoaajVCi3LtDAwEAiIyP54IMPTEcREXFaUVFRvPjii3h7e5uOUqEspTlkumXLltadO3eWY5yKl5SUROfOnUlNTXWL+5ZERMrS/v376dixI6mpqfj4+JiOU+YsFssuq9XasrCPufUVKEBISAgRERHMmDHDdBQREafz+uuvM378eJcsz+K4/RUoQGJiIj179uTQoUNuNwQhImKvQ4cOER4eTmpqKjVr1jQdp1zoCrQYYWFhNGvWjNmzZ5uOIiLiNCZPnszYsWNdtjyLoyvQ67Zt28bAgQNJSUlx6QNgRUTKwrFjx2jevDkpKSnUqVPHdJxyoyvQEvj9739PQEAAH374oekoIiIO77XXXmPkyJEuXZ7FqWw6gCOJjo6me/fu/PnPf6ZGjRqm44iIOKSkpCRWrFjBwYMHTUcxSlegNwgLC6NTp0689dZbpqOIiDisiRMn8sorr1CrVi3TUYxSgf7Gv/71L6ZMmUJaWprpKCIiDmfTpk3s37+f0aNHm45inAr0N+677z6GDh3KP/7xD9NRREQcSl5eHv/7v//Lv//9b7y8vEzHMU4FWoj/+7//Y8mSJRw4cMB0FBERh/HJJ5/g4eHBgAEDTEdxCCrQQtSuXZuJEycyceJE01FcT0YGnDjB8evbfzVt2pTg4GDeffddAM6fP09kZCQBAQFERkaSnp5uOLCIAFy9epVXXnmFN954Aw8PVQeoQG9p7Nix7Nu3j82bN5uO4vyys2HBAmjRAurUgaAgKj/4IG+dOsWBSZNI2LKF2NhY9u/fz+TJk+nUqRMpKSl06tSJyZMnm04vIkBsbCwhISF06NDBdBSHoY0UivDJJ58QFRXFrl27tLmCvdLToXNnOHgQMjNv/nj16hAYSK/69Xn2f/6HZ599li+++AJfX1/S0tLo0KEDycnJFZ9bRAqkpaURGhrKli1baNq0qek4FUobKdhpwIAB1K9fn/fee890FOeUnW0rz6SkwssTIDOTo99+y+7162kTFsZPP/2Er68vAL6+vpw+fboCA4tIYV544QVGjBjhduVZHBVoESwWC7Gxsbz++uscP37cdBzns3gxJCfDtWu3fEom0Dc7m3c8PKi5fn3FZRORElm3bh0JCQn8/e9/Nx3F4ahAixEYGMizzz7L+PHjTUdxPtHRkJV1yw9nA32BJ4E+V69CdDT169cvuAc3LS2NevXqVUhUEbnZlStXGDt2LFOmTNF5yYVQgZbAxIkT2bdvH6tWrTIdxXlkZNiGbm/BCgwDmgIv5L8zKYmeXbsSFxcHQFxcHL169SrnoCJyK2+88QbBwcH06NHDdBSHpEVEJbR27VpGjRpFUlKSXomVxIkTEBQEly8X+uGvgD8CofzyKu7fXl602bGDAePHc+zYMRo2bMjChQu58847Kyi0iOTLP+szMTGRhg0bmo5jTFGLiFSgpfDEE0/g7+9PVFSU6SiOLyPDdstKTk7JP8fTE86eBTc9W1DEUVitVh555BEiIyN58cUXTccxSqtwy0hMTAzTp08nqYihSbmuZk0ICSnd54SEqDxFHMCCBQs4deoUzz//vOkoDk0FWgp+fn68/vrrDBkyhGtFrCyV6156CXx8SvZcHx/b80XEqFOnTjF+/HhmzZql+9+LoQItpWHDhuHn58e//vUv01EcX9++tnnQKlWKfl6VKtCkCfTpUzG5RKRQVquV4cOHM2rUKFq1amU6jsNTgZaSxWJhxowZTJs2jR07dpiO49g8PWH9etvQ7K2uRH18IDQU1q2zPV9EjPnwww/56aef+Nvf/mY6ilNQgdrB19eX9957jyFDhvDzzz+bjuPYateGhASYOdO2F66nJ1SrZnvbooXt/du22Z4nIsYcOXKEV155hTlz5mjotoS0Cvc2/OlPf8LX15eYmBjTUZxHRobtUbOmFgyJOIjc3Fw6duxIr169+Otf/2o6jkPRKtxyEhsby6effsqmTZtMR3EeNWtCgwYqTxEH8s477wBox7VSUoHehjp16jBjxgyGDh3KxYsXTccRESm17777jsmTJzN79mwqVapkOo5TUYHepkcffZRHH32U4cOHU5rhcBER07KyshgwYADR0dE0btzYdBynowItA2+//TapqalMmTLFdBQRAds8+4kTvBsdTUhICMHBwQXDlGJjtVoZPXo0rVq1YujQoabjOKXKpgO4Am9vbxYuXEh4eDht2rShTZs2piOJuJ/sbNsRetHRkJREUuXKzLhyhe0PPkiVCRN4ZNYsunfvTkBAgOmkDmHmzJkkJibyzTffYLFYTMdxSroCLSONGzdm+vTpPPHEE5w7d850HBH3kp4O4eEwYgTs2QM5ORy4coVwoNq+fVQeNYr2Bw6wdN4800kdwp49e3j55ZdZtGgRPiXdLUxuogItQ71796Zv374MGTKEvLw803FE3EN2NnTubDs+LzOz4N0hwJfAOeByZiarfvyR47Gxtue7sYsXL9K/f3/ee+89mjRpYjqOU1OBlrHJkyeTnp7Om2++aTqKiHtYvBiSk+E3+1M3BV4CIoFHgGZWK5XT02HJEgMhHUP+Vn2RkZEMHDjQdBynpwItY56ennzyySfExMSwefNm03FEXF90NGRlFfqhYUAitivRO4GAnBzb893UlClTSE1N5e233zYdxSWoQMvBvffey5w5cxg4cCBHjx41HUfEdWVk2IZub+H09bfHgCXAQLA9PyOj/LM5mI0bNxIVFcXChQvx9vY2HcclqEDLSZcuXXjppZd47LHHyHDDv6wiFSIjo8jTfvoCDwCPAbFAbbDtw+xmfycPHjzIwIED+eSTT3S/ZxlSgZajcePG8Yc//IGBAweSm5trOo6I66lZ86a5zxttAfYDe4FO+e/MznarrSTPnz9Pjx49iIqKokOHDqbjuBQVaDmyWCy89957XL16lRdffNF0HBHXU7Om7bi80ggJcZsCzc7Opl+/fvTs2ZPhw4ebjuNyVKDlzNPTk4ULF7Jq1SqmTZtmOo6I63nppVufN/tbPj6257sBq9XKmDFj8PHxIdqNF06VJxVoBahduzbx8fFMmjSJDRs2mI4j4lr69oWgoCLnQgHbx5s0gT59KiaXYTExMWzfvp158+Zpk/hyogKtIAEBASxYsIBBgwaRnJxsOo6I6/D0hPXrbUOzt7oS9fGB0FBYt872fBcXHx/PW2+9xWeffUaNGjVMx3FZKtAK1KFDB6Kjo+natSsnTpwwHUfEddSuDQkJMHMmtGhhK8lq1WxvW7SwvX/bNtvzXNzWrVsZOnS8B1LAAAAPOElEQVQoS5YsoWHDhqbjuDRtJl/Bnn76ac6ePUtkZCRbtmyhbt26piOJuAZPT3jiCdsjI8P2qFnTbRYMAezdu5fHH3+c//73vzrUogKoQA148cUXOX/+PN26dWPDhg3UdKO/4CIVws2KEyAlJYVu3boRGxtL165dTcdxCxrCNSQqKoqHHnqI3r17c+XKFdNxRMSJnTx5ki5duvDqq6/Sv39/03HchgrUEIvFQmxsLPXq1eNPf/oTOTk5piOJiBM6d+4cXbp0YfTo0YwYMcJ0HLeiAjWoUqVKzJkzh2vXrjF8+HAdgSYipXLp0iUeffRRHnvsMSZMmGA6jttRgRpWpUoVFi1axKFDh3j++eexWq2mI4mIE7h8+TK9e/emefPmvP7666bjuCUVqAOoVq0a8fHxbN++nXHjxqlERaRIWVlZ9OjRg3vuuYepU6disVhMR3JLKlAHUatWLdauXcuOHTt49tlnNZwrIoXKL8+GDRvy0UcfaZchg1SgDuSOO+5g7dq17N69mzFjxqhEReRX8uc877//fmbOnKnyNEwF6mBq1qzJmjVr2L9/P88884xW54oIABcuXKBLly4EBQXx4YcfqjwdgArUAdWoUYPPP/+cU6dO8eSTT5KdnW06kogYdPbsWR5++GHatGnDtGnT8PDQP92OQP8XHJSPjw8rVqzg559/pm/fvtpsQcRNpaWl0b59e7p160ZMTIwWDDkQFagD8/b2ZvHixfj4+BAZGcn58+dNRxKRCnTgwAHatm3L4MGDiYqKUnk6GBWog/P09GTu3LmEh4cTERHB0aNHTUcSkQqwZcsWOnTowKuvvsrEiRNNx5FCaDN5J+Dh4cGbb77JvffeS0REBJ999hlhYWGmY4lIOVm4cCFjx45l7ty5REZGmo4jt6ACdSLjxo2jQYMGdO3alTlz5tCtWzfTkUSkjMXExPD222+zbt06mjVrZjqOFEFDuE6mT58+LF++nKFDhzJz5kzTcUSkjOTm5jJ+/HhmzpzJ119/rfJ0AroCdUJt27Zl8+bNdOvWjWPHjvGPf/xDiwtEnNjPP//M4MGDOXfuHF999RW1atUyHUlKQFegTiooKIht27axZs0a+vfvz6VLl0xHEhE7/PDDD0RERODt7c3q1atVnk5EBerE6tevz+bNm6lVqxbh4eGkpKSYjiQipbBx40batGnDn//8Zz7++GO8vLxMR5JSUIE6OS8vL2bMmMG4ceOIiIhg1apVpiOJSDGsVitvv/02gwYNYt68ebzwwguahnFCmgN1ARaLhZEjRxIaGsqAAQMYPXo0L7/8srb7EnFAly9fZsSIERw4cIBvvvmGRo0amY4kdtK/sC6kbdu2bN++nZUrV9KvXz/Ni4o4mCNHjhAREYGHhwdbt25VeTo5FaiL8fPzY9OmTdSrV482bdqQlJRkOpKIAJ9//jm///3vGTp0KHPmzKFq1aqmI8ltUoG6IC8vL/7zn/8wYcIEOnbsyNSpU7FaraZjibilK1euMH78eEaOHMmnn37KuHHjNN/pIlSgLuzpp59m69atzJo1i969e3P27FnTkUTcyoEDBwgPD+f48ePs2bOHdu3amY4kZUgF6uICAwP5+uuvCQwMpHnz5mzcuNF0JBGXZ7VamT59On/84x8ZO3YsixYt4s477zQdS8qYVuG6gSpVqvDmm28SGRnJ4MGDGTx4MP/85z/x9PQ0HU3E5Zw/f54RI0Zw+PBhtmzZQtOmTU1HknKiK1A30qVLF3bv3k1SUhIREREcPHjQdCQRl7Jp0yaaN29Ow4YNSUhIUHm6OBWom6lXrx6fffYZQ4YMoW3btrzxxhvk5OSYjiXi1C5evMioUaMYPHgw06ZNIyYmBm9vb9OxpJypQN2QxWLh2WefZfv27axdu5bw8HD27t1rOpaIU1q5ciUhISHk5eXx3Xff6ZhBN6ICdWONGzdm3bp1jBkzhsjISP7+979z9epV07FEnMKZM2d48sknGTduHHFxcUyfPp077rjDdCypQCpQN2exWHjmmWfYs2cP3377LS1atGDbtm2mY4k4LKvVyoIFCwgNDeXuu+9m3759PPzww6ZjiQFahSuAbQejpUuXsmjRIvr06cOAAQP45z//Sc2aNU1HE3EYP/zwA8899xyHDx9m+fLltGnTxnQkMUhXoFLAYrHQv39/kpKSuHTpEk2aNOGjjz4iLy/PdDQRoy5fvsyrr75KWFgYDz30EImJiSpPUYHKzerUqcOsWbNYvnw506dPJzw8nISEBNOxRCqc1Wpl4cKFPPDAA3z33XckJiYyadIkndspgIZwpQitWrVi69at/Pe//6Vv37507tyZyZMn4+vrazqaSLnbt28fzz//POfPn2f27Nl06NDBdCRxMLoClSJ5eHgwZMgQvv/+e3x9fQkNDSU6OlqrdcVlnTt3jrFjxxIZGcmAAQPYtWuXylMKpQKVEqlRowaTJ08mISGBrVu38sADDzB37lxyc3NNRxMpE5cvX+aNN96gadOmeHh4cODAAUaPHk3lyhqok8KpQKVU/P39WbFiBTNnzmTq1Kk0a9aMpUuX6rg0cVpXr17l/fffx9/fn507d7J582amTJmizd+lWCpQsUuHDh346quviI6O5rXXXqN169asWbNGRSpOIycnh1mzZhEUFMTq1atZuXIln376qfavlRJTgYrdLBYL3bt3Z9euXUyYMIHx48fTvn17tmzZYjqayC3l5eWxYMECgoOD+fjjj5k3bx7x8fG0aNHCdDRxMipQuW0eHh7079+fb7/9lmHDhjFkyBC6du3Kpk2bdEUqDiMnJ4cFCxbQokULYmJiiI2NZePGjbRt29Z0NHFSKlApM5UrV+app54iOTmZ/v37M3r0aFq3bs3ChQu12EiMycrKYsqUKQQEBDB16lSioqJISEigc+fOWCwW0/HEialApcxVqVKF4cOHs3//fv72t78RExNDUFAQH3zwAT///LPpeOImzpw5w6RJk7j//vv54osvmD9/Pl9++SU9evRQcUqZUIFKufHw8KBXr158/fXXxMXFsXr1au677z5ee+01zp07ZzqeuKjDhw8zZswYAgMD+fHHH/nqq69YvHgx4eHhpqOJi1GBSoWIiIhg+fLlfPHFFxw7dgx/f3+GDRvG9u3bNU8qty03N5eVK1fSs2dP2rRpQ+3atTlw4ADTpk0jMDDQdDxxUZbS/OPVsmVL686dO8sxjriL06dP89FHHzFt2jTuuOMORo0axaBBg6hRo4bpaOJETp06xaxZs5gxYwZ33303I0eO5IknnsDHx8d0NHERFotll9VqbVnYx3QFKkbUq1ePl156iUOHDhEdHc3atWtp1KgRI0eOZPfu3abjiQPLy8tjzZo19OnTh5CQEE6cOMGyZcv45ptveOaZZ1SeUmG0R5UY5eHhQZcuXejSpUvB1UTv3r2pX78+zzzzDP369aNu3bqmY4oDSE1NZf78+cyaNatg1CIuLk6jFmKMhnDF4eTm5rJmzRrmzJnD559/TkREBIMGDaJXr176x9LNpKWl8emnnzJ//nxSU1Pp168fTz/9NK1atdJKWqkQRQ3hqkDFoWVmZrJ8+XLmz5/Pli1b6Nq1K4MGDaJbt246k9FFpaens2TJEubPn8+uXbt47LHHGDRoEJ06dcLT09N0PHEzKlBxCefOnWPx4sXMmzePffv20atXL3r16kXnzp2pXr266XhyG06fPs2qVatYtmwZmzZtolOnTgwaNIju3btTtWpV0/HEjalAxeWcOHGCxYsXEx8fT0JCAm3btqVHjx706NGD+++/33Q8KYbVamXv3r3Ex8cTHx/P999/T+fOnXnsscfo3bs3d9xxh+mIIoAKVFxcRkYG69atIz4+nlWrVlG3bl169OhB9+7dadu2rc5zdBCXL19mw4YNxMfHs3LlSry9vQte9LRr144qVaqYjihyExWouI28vDx27NjBypUriY+P5/Dhw0RERNC+fXvatWtHy5YtNY9WQTIzM9m2bRubN29m8+bN7N69m1atWhWUZmBgoBYCicNTgYrbOnPmDFu2bCn4R/zw4cOEh4cXFGrr1q3x9vY2HdMlXLx4ka1btxb8rJOSkmjRokXBz7pt27aaqxanowIVuS49PZ2vvvqKzZs38+WXX7J//36aNWtGixYtCAsLIywsjAceeEDDicXIyspi7969JCYmFjwOHz5Mq1ataN++Pe3bt6dNmzZaACROTwUqcguXLl1i165dBSWwe/dujhw5wgMPPEBYWFhBsQYHB7vt1dO5c+fYt29fwc8nMTGRo0ePEhwcXPDzadGiBS1atNALD3E5KlCRUsjKyioojPxHcnIytWrVIiAggICAAAIDAwt+/bvf/c7pr7QyMjJISUnh4MGDpKSk/OrXubm5hIaG3nSVrrlkcQcqUJHblJeXx8mTJwstmKNHj1K/fn0aNWqEr68vd999N76+vgWP/N/XqVMHD4+K3X46JyeH06dPk5aWxo8//khaWlrBI//3qampZGZmFrwg+O0LhLvuukuLfcRtqUBFylFOTg7Hjh3j+PHjvyqo35bWpUuXuPPOO6levTo1atQo9G316tWpVq0alSpVwsPDo+BhtVrJy8sreOTm5pKVlUVmZiaXLl265dsLFy5Qp06dm8r8xt/fd999+Pn5qSRFClFUgeoGOZHbVLlyZRo3bkzjxo2LfN7Vq1c5d+4cmZmZBY/fll5mZiZZWVk3leWNZWqxWKhUqRI1atTAz8+voHgLK+M6deroPliRcqK/WSIVxMvLCz8/P9MxRKSM6DxQERERO6hARURE7KACFRERsYMKVERExA4qUBERETuoQEVEROygAhUREbGDClRERMQOKlARERE7qEBFRETsoAIVERGxgwpURETEDipQERERO6hARURE7KACFRERsYMKVERExA4qUBERETuoQEVEROygAhUREbGDClRERMQOKlARERE7qEBFRETsoAIVERGxgwpURETEDipQERERO6hARURE7KACFRERsYMKVERExA4qUBERETuoQEVEROygAhUREbGDClRERMQOKlARERE7qEBFRETsYLFarSV/ssVyBvih/OKIiIg4lEZWq/Wuwj5QqgIVERERGw3hioiI2EEFKiIiYgcVqIiIiB1UoCIiInZQgYqIiNhBBSoiImIHFaiIiIgdVKAiIiJ2UIGKiIjY4f8DEIjwAWUsdw0AAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.patches as mpatches\n", "\n", "colors = {1: \"b\", 2: \"r\"}\n", "group_1 = mpatches.Patch(color=colors[1], label=\"Group 1\")\n", "group_2 = mpatches.Patch(color=colors[2], label=\"Group 2\")\n", "\n", "circle = visualization.PoincareDisk(point_type=\"ball\")\n", "\n", "fig, ax = plt.subplots(figsize=(8, 8))\n", "ax.axes.xaxis.set_visible(False)\n", "ax.axes.yaxis.set_visible(False)\n", "circle.set_ax(ax)\n", "circle.draw(ax=ax)\n", "for i_embedding, embedding in enumerate(embeddings):\n", " x = embedding[0]\n", " y = embedding[1]\n", " pt_id = i_embedding\n", " plt.scatter(x, y, c=colors[karate_graph.labels[pt_id][0]], s=150)\n", " ax.annotate(pt_id, (x, y))\n", "\n", "plt.tick_params(which=\"both\")\n", "plt.title(\"Poincare Ball Embedding of the Karate Club Network\")\n", "plt.legend(handles=[group_1, group_2])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In `geomstats`, several unsupervized clustering algorithms on manifolds are implemented such as $K$-means and Expectation-Maximization. \n", "\n", "Let us apply $K$-means to learn the node belonging of the two groups and see how well we predicted the true\n", "labels.\n", "Lets first import $K$-means" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "from geomstats.learning.kmeans import RiemannianKMeans" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Set the number of groups to 2." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "n_clusters = 2" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Initialize an instance of $K$-means." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "kmeans = RiemannianKMeans(\n", " metric=hyperbolic_manifold.metric,\n", " n_clusters=n_clusters,\n", " init=\"random\",\n", " mean_method=\"batch\",\n", ")" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Fit the embedded nodes" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "centroids = kmeans.fit(X=embeddings)\n", "labels = kmeans.predict(X=embeddings)" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "And plot the resulting labels provided by $K$-means" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "pycharm": { "name": "#%%\n" }, "scrolled": false, "tags": [ "nbsphinx-thumbnail" ] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdAAAAHUCAYAAACKzHkWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdeXxM1/vA8c/JPhFBxRKiqL2xpJEQYlf70tqXr510sVW/paL6K20pLS2qltLF0paipZYvpXZShAiVlKK2EGrfMiGT3N8fM0mTyC7JzSTP+/Wal8nc7bmLeeace+45StM0hBBCCJE5NnoHIIQQQlgjSaBCCCFEFkgCFUIIIbJAEqgQQgiRBZJAhRBCiCyQBCqEEEJkgSRQkW8ppaYopZZY3j+nlHqQxfVUVkrJ817pUErtU0oNyu55s1N2n0ulVIRSqlkq015USp1P9PcppVTj7Nq20J8k0DxOKXVeKfVior97K6VuK6Wa6hmXtdE07W9N01xyYt1pfYlmYFk7pZSmlKqQ6LNApdQVpVSNbAox8faSfKmL3KNpWjVN0/bqHYfIPnZ6ByAyTik1EPgM6KBpWpDe8Yjsp5SaDAwBmmiadiaTy9oAaJoWlwOhCSGSkRKolVBKvQJ8CrRJK3laqsY+UEodUEo9VEqtU0oVV0qtUErdU0odVEo9m2j+55VSvymlbimlTiqluiWa1lkpFaqUuq+UuqiU+r9E0ypbSk4DLCWw60qpwETT/ZRSIZZtXlNKzUgl3uJKqf9Zlr+tlNqglCqbbH+mKqUOK6XuKqXWKqWKJYshwFJiu6KUejOV7SSpulNKFVVKfauUirTE/0F8AlJK2SqlZimlbiqlzgJt0zjeK4AywGal1AOl1H8tn7+slApTSt1RSu1QSlVLbR2J1jUdGECi5JnB4/OhUup34CHwrFJqmFLqT8t5O6uUGmaZtwiwwTLPA8urpFLKRin1jmXeG0qplfHHOJU4u1qui3tKqTNKqdYpzJNQfZ7S8beoktJ5TWWbnZVSxyzHc59SqmaiaRFKqbFKqROWfVqklCqllPrVEuNWpVTRZOtL8ZpJ71gopQYppS5YpgUmW6ezUmq55TyFAXWTTU+oqbAcnxVKqe8s5+mEUso70bw+6t//eyuVUquV+ceVyEs0TZNXHn4B54GfgGtAnQzMvw/4C3gOKAacBE4BzTHXOPwALLbMWxi4jPlL2w7zf/ibQDXL9BZATcw/tOoAN4COlmmVAQ1YCDgB3sAjoIplejDQJ9F26qcSbwmgC2AAXIGfgTXJ9ucS8DxQCFgHLEkWw3LA2RLjTaCZZfqU5PMmWu9GYL5ludLAEWCoZdpIIAzwAIoDexIvm8I+RMRv0/J3DeCB5fjZA+9Yzol9CsvaWfZhjeU8eWTh+Jy3bNPesr5OlvOvLDEYgdqW+V8EzifbxlhgP1DWci6/Bpansq8NgTtAS8t1US7R9bIPGJT82Kdy/FM9ryls0xfz9e8L2GIuoZ8FHBId/yCgpOWc3QQOW64HJ2A3MDGD10yqxwKoZTmv/oAj8DlgSrTsTGAX5v935YHwxMc68XViOT5GoI1ln2YA+yzTHC3zjrSc0x5ADDBZ7+8jeSW7NvUOQF7pnCDzl+M94BfAJgPz7wPGJ/p7DrAh0d9dgMOW9/8BdiZb/uv4L5sU1v0FMMPyPv6LqHSi6SFAd8v7IOA9oHgm99cHuJ5sf6Yk+rs2EI05OcTHUDnR9M+ALy3vU0ygli9HI+CYaLn+wDbL+z3AsETT2pO5BPo+8EOiv22Aq0CjFJaNT6D3gFlZPD7vpbPMRmCE5X1KCfQ00DTR3+Uw/xh64nqzXB8z0rj2BiU/9smPf3rnNYX1LgYmJfvsLOCf6Pj3SjTtF2Buor/fxPKjIwPXTKrHAvgA+C7RNBcgln+T4kXgxUTTh5N2At2SbP8fWN63AC4m298DSALNcy+pwrUOrwFVga+UUir+Q6XUV4mq4t5ONP+1RO+NKfwd35imPOBvqRa7o5S6A/QC3C3rb6CU2mWpPrwLDAPcEgemadrVRH9GJVr3YMyli1NKqUNKqfYp7ZhSqpBlPy4qpe4BO5JvA3NJJd4FzL/Qn0ljepmUtpVIecs6riXa73lAKcv0MimsMzPKJF5GM9+TjMCcuFPTC+irElWTQ5aOD0qpjspcVX/Lsm+tU1gmsWeBDYmOxR+Yk0zJFOYthzl5ZYf0zmu88sD4ZNepO0mPZ0av+dS2HX/NpHUsklwXmqY9AG4lWo97CutNS/L/O4Us78tgvl5Si1fkEZJArcM/mKvMGmOudgRA07Rhmqa5WF6fZGG9l4DtmqYVTfRy0TRtpGX6SszVx+U0TSsCfIW55JcuTdNOaZrWG/MXz6fAT0oppxRmfRuoCNTTNM0V86/v5Molev8s5hLBrTSmX0knvEuYv7CeSbTfrpqm1bZMj0xhnWlJfm/vCuYvfSChcY8H5ury1PyJuXT4hlJqbKLPM3J8Et/bNWCuDp4GlNI0rSiwlX/PW0qPcEQArZJdB07JfhzFuwRUSmM/4j3EXEUar3QK86R3XhNv8/1k8TlrmrYqA3GkJrVrJq1jkeS6UEq5kDThX01hvVkRifl6SS1ekUdIArUSmqZdwfzl2VYpNSubVrse8FRK9VVK2Vte9dS/DV4KA7c0TYtWSvkBvTO6YqVUf6WUm6X0dRfzF3dKrUMLY05mt5VSxTFX+yY3QClVXSlVCHP16CpN0xIngv9TShmUUrWAgcCPacWmadolzPfFZiqlXC0NRyorpZpYZlkFjFFKlbXEND6d3b2G+Z5jvFVAZ6VUM6WUPTAOuA8cTCeuP4BWwDtKqfgfMRk5Pok5Ag7AdSBWKdUR84+vxLG6KaUKJ/psIfCRsjQuU+aGRZ1TWf/XwDClVHPLcfNQKTeQCgWaKqXKWRrwBKYwT3rnNd4iYIRSyleZuSilOlmWy6rUrpm0jsVq4CVLzYwj5mrYxPGuwnzuilqWH0nW7ANslVKvK/NjTt1I1iBJ5A2SQK2I5Yu/BdBdKTUtG9Z3F3Mjhn6Yf/VexVxycbTM8jowTSl1H3NDmMz84m8P/GlZdibme1SPU5jvM6AI5oYcQcDmFOZZDnxnidEWGJNs+j7gb8wlrWmapu3IQHz9MFeZhQO3MX85xpeSFgDbMVffBWMu0aXlI+B9S7XfGE3TwjB/KS/AnMjaAp01TYtJLyhN045a5p+ilAogY8cn8fJ3MN/zW4u5NNcd8z3Q+OknMNcqnLfEW9KyjS3Adsv5CsLcYCel9QcBAZgb0NwFdpJy6WiLJYY/gEOYf6wll955jd/mQczX4gLM5+ovzOfvaaR2zaR6LDRNOw68gfn/wWXM/18Sl9InWfblPObztCwrgWma9ghzW4XXMO9vT+B/mEvoIg9RKf/gEyJvUErtA77SNG1JCtMqA6c1TctQtbIQ1kopdQSYrWnacr1jEf+SEqgQQuQxlur/UpYq3KFAdcylZZGHSE9EQgiR99TAfF+2EOZWz900TbuW9iIit0kVrhBCCJEFUoUrhBBCZEGmqnDd3Ny0ChUq5FAoQgghRN5y5MiRG5qmlUhpWqYSaIUKFTh8+HD2RCWEEELkcUqpVHuUkipcIYQQIgskgQohhBBZIAlUCCGEyAJ5DlQIIZKJiYkhIiKC6OhovUMRucTJyQkPDw/s7e0zvIwkUCGESCYiIoLChQtToUIFEo0gKPIpTdO4efMmERERVKxYMcPLSRWuEEIkEx0dTfHixSV5FhBKKYoXL57pGgdJoEIIkQJJngVLVs63JFAhhBAiC+QeqBBCpKP0zNJce5h9fbmXKlSKq2OvpjmPra0ttWrVwmQyUaNGDZYuXYqzs3OWtrdr1y5mzpzJxo0bWb9+PeHh4QQGpjTGOdy5c4cffviB4cOHZ2obkydPxsXFhbFjxz4x7bvvvuOTTz4hNjYWOzs7fH19mTlzJkWLFs3S/mTFxIkTWbZsGbdv3+bBgwfZsk4pgQohRDqyM3lmdH0Gg4HQ0FBOnDiBg4MDCxcuTDJd0zTi4uIyve3OnTunmjzBnEDnz5+f6fWmZsuWLcyaNYvNmzcTFhZGSEgIDRs25Nq1J49BbGxstm03uU6dOnHo0KFsXackUCGEyOMaN27MmTNnOH/+PDVq1GD48OF4e3tz6dIltm7dSoMGDfD29qZHjx4JpastW7ZQvXp1GjVqxM8//5ywriVLljBy5EgArl27RpcuXahTpw516tQhKCiIwMBAzp49i5eXF+PGjQNgxowZ+Pr6Urt2bSZNmpSwrqlTp1KtWjVefPFFTp06lWLsU6dOZebMmZQtWxYwl6yHDBlCtWrVAHMXsR988AGNGjVi9erVhIaG4ufnR+3atenSpQu3b98GoFmzZgldyd64cYP4ftmXLFnCSy+9RNu2balWrRrvv/9+inH4+fnh7u6epeOfGkmgQgiRh5lMJjZv3kytWrUAOHXqFAMGDODo0aMUKlSIKVOm8NtvvxESEoKPjw+fffYZ0dHRBAQEsGHDBvbu3cvVqylXF48ePZqmTZty7NgxQkJC8PT0ZPr06VSqVInQ0FBmzJjB1q1bOX36NIcOHSI0NJQjR46wZ88ejhw5wsqVKzl69Cg///wzwcHBKW4jLCwMb2/vNPfRycmJffv20bt3bwYMGMDHH3/M8ePHqVWrVqoJMbFDhw7x/fffExoayurVq3Otz3ZJoEIIkQcZjUa8vLzw8fHh2WefZejQoQCUL18ePz8/AA4cOEB4eDj+/v54eXmxdOlSLly4wMmTJ6lYsSJVqlRBKUW/fv1S3MaOHTt4/fXXAXPJsEiRIk/Ms3XrVrZu3coLL7yAt7c3J0+e5PTp0+zdu5cuXbrg7OyMq6srnTt3Tnef/vjjD7y8vKhUqRI//vhjwue9evUC4O7du9y5c4emTZsCMHDgQPbs2ZPuelu1akXx4sUxGAx07dqVffv2pbtMdpBGREIIkQfF3wNNrlChQgnvNU2jVatWrFixIsk8oaGh2fYYjqZpTJgwgVdffTXJ57Nnz87QNjw9PQkJCaF58+bUqlWL0NBQRo4cidFoTJgn8T6lxs7OLuGeb/LnNZPHkVuPIEkJVAghrJSfnx/79+/nzJkzAERFRfHXX39RvXp1zp07x9mzZwGeSLDxWrZsyYIFCwBzA5579+5RuHBh7t+/nzBPmzZt+OabbxLurV6+fJl//vmHJk2asHbtWoxGI/fv32fDhg0pbmPChAmMHTuWiIiIhM8SJ8/EihQpQrFixdi7dy8Ay5cvTyiNVqhQgSNHjgCwZs2aJMtt27aNW7duYTQaWbduHf7+/mkctewjCVQIIdJRqlCpPLm+EiVKsGTJEvr06UPt2rXx8/Pj5MmTODk5sWjRIjp06ECjRo0oX758isvPmTOHnTt3UqtWLerWrUtYWBjFixfH39+fmjVrMm7cOFq3bk3fvn1p0KABtWrVonv37ty/fx9vb2969eqFl5cX3bp1o3Hjxiluo3379owePZp27drx/PPP07BhQ2xtbWnTpk2K8y9dupRx48ZRu3ZtQkNDee+99wAYO3YsCxYsoGHDhty4cSPJMo0aNaJ///4Jsfj4+Dyx3rfffhsPDw+ioqLw8PBg8uTJmTjSKVOapmV4Zh8fH00G1BZC5Hd//vknNWrU0DsMkQFLlizh8OHDfPHFF0+9rpTOu1LqiKZpT2ZkpAQqhBBCZIk0IhJCCGG1Bg0axKBBg3TZtpRAhRBCiCyQBCqEEEJkgSRQIYQQIgskgQohhBBZII2IhBAiHaX37+daTEy2ra+UvT1X03nYX4Yzyz5RUVH06NGDs2fPYmtrS6dOnZg+ffpTr1dKoEIIkY7sTJ4ZXZ8MZ5a9xo4dy8mTJzl69Cj79+9n8+bNT71OSaBCCJHHyXBmTzecmbOzM82bNwfAwcEBb2/vJF0LZpUkUCGEyMNkOLPsHc7szp07bNiwgZYtW6a73vRIAhVCiDxIhjPL/uHMTCYTffr0YfTo0Tz33HPprjc90ohICCHyIBnOLKnsGM7slVdeoUqVKowZMybd7WWElECFEMJKyXBmZhkZzuzdd9/l7t27zJ49O8VtZ4WUQIUQIh2l7O2z/TGW7JB4OLNHjx4BMGXKFKpWrZownJmbmxuNGjXixIkTTyw/Z84cXnnlFb7++mtsbW1ZsGABDRo0SBjOrF27dsyYMYM///yTBg0aAODi4sJ3332XZDiz8uXLpzmc2fXr12nXrh2xsbEULVqUmjVrpjmc2WuvvUZUVBTPPfcc3377LWBuRduzZ0+WL19OixYtkiwTP5zZmTNn6Nu37xPDmUVERDB16lSqV6+ecD925MiRDBs2LBNH+0kynJkQQiQjw5lZDxnOTAghhLAyUoUrhBDCauk5nJkkUCFy0OPHj7l27RqRkZFERkZy8+ZNHjx4wP3793nw4EHCK/HfUVFRxMXFERcXR2xsLHFxcSilsLGxSXjZ2tpSqFAhXFxcKFy4MC4uLim+L1GiBO7u7ri7u1OiRAlsbW31PiRC5BuSQIXIotjYWC5dusTp06c5ffo0Fy9eTEiUkZGRXL16lbt37yZJYsWLF6dw4cIJrzJlyjyR+JydnbG1tU1IlvFN8uOTalxcHCaTiaioqCTJN/7f27dvc+nSJe7du8f169cT4rl9+zYlSpSgdOnSCfG4u7tToUIFqlSpQpUqVShdunS2Pf4gRH4nCVSIdFy7do3w8PCERPnXX39x+vRpzp07h5ubG1WrVqVKlSqUL1+e6tWrJ0lQbm5u2NjkjaYGMTEx/PPPP0kS/JUrV9i7dy/ffPMNp0+fxmg0JiTTxC9PT88UH7IXoiCTBCqEhaZpXLp0iZCQkITX0aNHMRqNeHp6JiTKAQMGULVqVSpVqpTl0TH0YG9vT9myZRP6JE3JnTt3En4onD59mq1bt/LFF1/w559/4u7ujre3N97e3gm90ri5ueXiHgiRt0gCFQXWjRs32LNnD4cOHUpImHZ2dtStWxdvb2+GDh3KCy+8QPny5QtMtWbRokXx9fXF19c3yeexsbGcOnWKo0ePEhISwrRp0zh69Ciurq4JSbVBgwY0aNAAFxcXnaLPOftL7yfmWvY9B2pfyh7/q2kPZ3b16lXGjBlDcHAwjo6OVKhQgdmzZ+Pg4EDHjh1TfK4zPUuWLKF169aUKVMmq6Fnm127duHg4EDDhg0ztVxaw7G5uLgkdPiQGySBigIjMjKSPXv2sHv3bnbv3k1ERAT+/v40aNCAN954A29vb9zd3fUOM0+ytbXl+eef5/nnn+c///kPYL4ne+7cuYQfHx988AFHjx6lZs2aNG3alKZNm+Lv758vqn6zM3lmZH2aptGlSxcGDhzIypUrAXP3fNeuXaNcuXJZ3u6SJUuoWbNmphKoyWTCzi77U8WuXbtwcXFJMYGmtc3OnTtnqN/d3CAJVORbV69eZdu2bQlJ88aNGzRu3JimTZsydOhQ6tSpkyNfDAWFjY0NlSpVolKlSvTo0QMwd9F28OBBdu/ezcyZM+nZsyfVqlWjSZMmNGvWjObNm+Pq6qpz5Hnfzp07sbe357XXXkv4zMvLC4Dz588nfJa8E4GOHTsyduxYGjduzNChQzl8+DBKKYYMGUK5cuU4fPgw//nPfzAYDPz++++Eh4fz3//+lwcPHuDm5saSJUtwd3enWbNmNGzYkP3799O5c2feeuuthG0+ePCAUaNGJax70qRJdOvWja1btzJp0iQePXpEpUqV+Pbbb3FxcaFChQoMHDiQDRs2EBMTw+rVq3FycmLhwoXY2try3XffMXfuXL7++mueeeYZjh49ire3NxMnTmTIkCH8/fffODs7s2jRImrXrp1kn8+dO0ffvn0xmUy0bds2IcbIyEh69erFvXv3MJlMLFiwINWekp6GfHuIfCMuLo6jR4+yceNGNm3axOnTp2nRogXNmjVj1KhR1KxZM8806MmvDAYDzZo1o1mzZgA8evSIw4cPs3v3bubNm0e/fv3w8/OjY8eOdOzYkUqVKukbcB514sQJ6tatm+XlQ0NDuXz5ckI17507dyhatChffPEFM2fOxMfHh5iYGEaNGsUvv/xCiRIl+PHHH5k4cSLffPNNwjK7d+9+Yt0ffvghRYoU4Y8//gDg9u3b3LhxI2FYtUKFCvHxxx/z2Wef8d577wHg5uZGSEgI8+fPZ+bMmXz11Ve89tpruLi4MHbsWAC+/vpr/vrrL3777TdsbW0ZNWoUL7zwAuvWrWPHjh0MGDDgic7133jjDV5//XUGDBjAvHnzEj7/4YcfaNOmDRMnTiQ2NpaoqKgsH8u0SAIVVu3Bgwds3749IWm6urrSsWNHPvnkE/z9/bHPpj5HRdY4Ojri7++Pv78/77zzDg8ePOC3335j48aNTJ8+naJFi9KhQwc6duwo5ysbPffcc/z999+MGjWKDh060Lp16yfmOXXqFCdOnKBVq1aA+T534lsY8UOMJffbb78lVCsDFCtWjI0bNyYMqwbm55/j+84F6Nq1KwB169ZNMrh3cj169Eh4Vnnfvn389NNPALRo0YKbN29y9+7dJPPv378/YZ7+/fszfvx4AHx9fRkyZAgxMTG8/PLLCaX37CYJVFidO3fusHbtWlatWsX+/fupX78+HTt2ZPz48VSuXFnv8EQaXFxcePnll3n55ZeT1BiMGzeOs2fP0rZtW3r16kXbtm1xdHTUO1zdeHp6PjHiSEoSD/EF/w7zVaxYMY4dO8avv/7KvHnzWLVqVULJMp6maXh6evL777+nuO7UhhjTNO2JRnWpDasWL/5c2traYjKZUt2f5EO1JZdSY76UPmvSpAl79uxh06ZN9O/fn3HjxjFgwIBUt5tVUp8lrILRaGT16tV06dKF8uXLs2HDBgYPHkxERATbtm3jjTfekORpZWxsbKhbty6TJk0iODiYsLAwmjZtyqxZsyhTpgzDhg1j+/btxMbG6h1qrmvRogWPHj1i8eLFCZ8FBwc/UaVaoUIFQkNDiYuL49KlSxw6dAgwtzCPi4ujW7dufPjhh4SEhAAkGaqsWrVqXL9+PSGBxsTEEBYWlm5srVu3TtJx++3bt1MdVi0tyYdNS65JkyZ8//33gLnBkZub2xP3z/39/RNKw/HzAly4cIGSJUsSEBDA0KFDE/Y/u0kCFXlWTEwMmzdvpn///pQpU4bFixfTuXNnLly4wM8//0zPnj2lQUo+4u7uzquvvsquXbs4duwYNWrUYPz48ZQrV44xY8Zw8ODBFEslucG+VPZWLae3PqUUa9euZdu2bVSqVAlPT08mT578ROtZf39/KlasSK1atRg7dmzCUF2XL1+mWbNmeHl5MWjQIKZNmwaY+4197bXX8PLyIjY2ljVr1jB+/Hjq1KmDl5cXQUFB6cb+7rvvcvv2bWrWrEmdOnXYuXNnkmHVateujZ+fHydPnkxzPZ06dWLt2rV4eXkljP+Z2OTJkzl8+DC1a9cmMDCQpUuXPjHPnDlzmDdvHr6+vkmqd3ft2oWXlxcvvPACP/30E2+88Ua6+5UVMpyZyHNOnDjBl19+yY8//kjlypXp06cPPXv2pFSpUnqHJnTw119/sWLFClasWEFMTAwDBw5k6NChaXYI8bRkOLOCSYYzE1bJaDSybNky/P39adu2Lc888wwHDx4kKCiIUaNGSfIswKpWrcqkSZP4888/WbVqFVevXqVWrVp06dKFLVu2JLkHKERukgQqdHXy5EnefPNNypUrx4oVK3j77bc5f/4877//PhUrVtQ7PJGHKKWoW7cu8+fP5+LFi7Rv3553332XSpUq8dFHH3H16lW9QxQFjCRQketiYmJYuXJlwoP1zs7OBAcHs3nzZl566SXp3ECky8XFhYCAAA4fPszq1as5d+4cNWrUoEePHuzYsSNb7pXqdb9V6CMr51sSqMg1Dx48YPbs2VSuXJmFCxcycuRILly4wNSpU6W0KbLMx8eHxYsXc+HCBVq0aMHIkSPx8fFh5cqVaT4ykRYnJydu3rwpSbSA0DSNmzdv4uTklKnlpBGRyHHXrl1j7ty5LFy4kObNmzNu3Djq1aund1gin4qLi2PTpk188sknRERE8NZbbzF48OBUn2tMSUxMDBEREQnPVYr8z8nJCQ8Pjyc680irEZEkUJFjTp8+zaeffsqPP/5I7969eeutt+RZTZGrgoKCmDFjBvv372f48OGMGDGCEiVK6B2WsCLSClfkqmPHjtG9e3caNmxIyZIlOXXqFAsWLJDkKXJdw4YNWbt2LXv37uXKlStUrVqVkSNHcvnyZb1DE/mAJFCRbU6dOkXv3r1p06YN/v7+nDt3jg8++ICSJUvqHZoo4KpVq8aiRYsIDw/HYDBQu3Zt3nrrLa5fv653aMKKSQIVT+3ChQsMGTKERo0aUbt2bc6cOcObb76ZLwdWFtbN3d2dGTNm8McffxAdHU316tX5v//7P+7cuaN3aMIKSQIVWRYZGcnIkSPx9vamTJkynD59mnfeeUcSp8jzypQpw7x58zhy5AgRERFUqVKFadOm8fDhQ71DE1ZEEqjItDt37jB+/Hhq1qyJg4MDf/75J1OmTKFo0aJ6hyZEplSoUIFvv/2WvXv3EhoaSuXKlfn888+JiYnROzRhBSSBigyLjY1l0aJFVK9enZs3b3Ls2DE+++wzuccprF716tX58ccf2bx5M//73/+oXbs2v/76q95hiTxOunwRGbJv3z5Gjx6Ns7Mz//vf/xJGfRAiP/Hy8mLz5s1s3LiRESNG8Pzzz/PZZ59JC3KRIimBijRFRETQt29f+vTpw7hx49i7d68kT5GvKaXo1KkTYWFh+Pv74+fnR2BgYJpjV4qCSRKoSFF0dDRTp07Fy8uLSpUqcfLkSfr06ZPi6O9C5EeOjo6MHz+e48ePExkZSfXq1Vm+fLmM/iISSAIVT9i2bRuenp6EhIQQHBzMhx9+mKlu0ITIT8qUKcPSpUv56aefmDt3Lo0bN+bPP//UOyyRB0gCFQlu377NkCFDGDZsGPPmzeOnn36STt6FsPDz8+PAgQP069ePJk2aMGXKFGmtW8BJAhUArF27lpo1a+Ls7MyJEydo27at3iEJkefY2Njw+pOf690AACAASURBVOuvc+TIEYKCgvDx8eHIkSN6hyV0Iq1wC7irV68yatQojh8/zsqVK2ncuLHeIQmR5z377LNs2rSJ77//nvbt2zNo0CAmT56MwWDQOzSRi6QEWkBpmsbSpUupU6cOVapU4dixY5I8hcgEpRT9+vXjjz/+4Pz589SpU4c9e/boHZbIRVICLYD++ecfhgwZwuXLl9m8ebM8liLEUyhZsiQ//vgj69ato2/fvnTv3p3p06dnenBmYX2kBFrAbN26FS8vL2rVqsWhQ4ckeQqRTV5++WWOHz/OpUuX8PPzk5a6BYAk0ALi8ePHjB07lqFDh/Ldd98xbdq0J0ZeF0I8nWeeeYY1a9YwYsQImjRpwqJFi9A0Te+wRA6RBFoAnDp1igYNGnD69GmOHj1KixYt9A5JiHxLKUVAQAB79+5lwYIFdO/enVu3bukdlsgBkkDzMU3T+Oabb2jUqBHDhg1j3bp1uLm56R2WEAVC9erVOXDgAM8++yxeXl7s2rVL75BENpNGRPnU/fv3CQgIICwsjJ07d1KzZk29QxKiwHF0dGTWrFm0bt2avn37MmzYMCZNmoStra3eoYlsICXQfOj06dP4+flRuHBhDh06JMlTCJ21a9eOo0ePsmfPHjp16sTt27f1DklkA0mg+cymTZvw9/fnjTfeYPHixfJgtxB5RKlSpdi2bRtVq1alXr16nDhxQu+QxFOSKtx8Ii4ujo8++ogFCxawbt06GjZsqHdIQohk7O3tmT17NnXr1qV58+YJjYyEdZIEmg/cu3ePgQMHcu3aNYKDgylTpozeIQkh0tC/f3+ef/55unbtypEjR5gyZYrcF7VCUoVr5U6dOkX9+vUpVaoUO3fulOQphJWoW7cuhw8f5sCBA3To0EEedbFCkkCt2Pbt22ncuDH//e9/WbhwIY6OjnqHJITIhBIlSrBt2zZq1KhB/fr1OXPmjN4hiUyQKlwrtXz5csaOHcvq1atp2rSp3uEIIbLIzs6OWbNmUb16dRo3bsy6deuoX7++3mGJDJAEamU0TWP69OksXLiQHTt24OnpqXdIQohs8Oqrr1K2bFk6duzI119/TefOnfUOSaRDEqgVMZlMjBo1it9//53ff/9d7ncKkc907NiRTZs28dJLL3H58mVef/11vUMSaZAEaiUePnxInz59MBqN7NmzB1dXV71DEkLkgHr16rFv3z7atWvHxYsXmTp1KjY20lwlL5KzYgX++ecfmjdvTrFixdi0aZMkTyHyuUqVKhEUFMSuXbsYMGAAjx8/1jskkQJJoHncxYsX8ff3p3Xr1ixZsgQHBwe9QxJC5AI3Nze2b9/Ow4cP6dChAw8fPtQ7JJGMJNA87MyZMzRp0oQRI0YwZcoUlFJ6hySEyEXOzs6sWbMGDw8P2rZty7179/QOSSQiCTSPCg8Pp1mzZkycOJExY8boHY4QQie2trZ8/fXX1K5dm5YtW0qHC3mIJNA86NixY7Rs2ZLp06cTEBCgdzgFRtWqVbGxscHJyemJaR07dkQpxalTp3SITBR0NjY2fPHFFzRr1ozmzZtz/fp1vUMSSALNc0JDQ2nTpg2ff/45/fr10zucAiHiXgTBl4PpN6wfy5cvf2L6wYMHOXDggPRVKnSllOKTTz6hU6dOtGjRgn/++UfvkAo8eYwlDwkJCaF9+/bMmzePbt266R2O1Yq4F0Hk/Uh6Nu3Jhb8v4ODgQHR0NAA//vgjQ4cOJTY2FpNmgvZgqmNKWNZ+lz1xcXFEPY7C2cEZgG7dujF//nz69u2ry/4IEU8pxYcffoitrS3Nmzdnx44dlCpVSu+wCiwpgeYRR48epV27dixYsECSZxZEPY5i9ObRGKYaKDerHPW+qsf5Kuex7W6LKc5E1OMoAIYPH07AqADUuwqTnwnT/0xJ1hMTG0NsXCxuM9w4d/scEydOxM3NjZ49e+qxW0I8QSnF+++/T8+ePWnevLmURHUkJdA84NSpU3To0IGFCxfSpUsXvcOxOudun8NzvidGkzHpBD8wXTBBHLjNcCNseBgAc3fPJbZNLBiBJ293AmA0GakxowZ2n9sRHh6eszsgRBZMmjSJuLg42rZty86dOylSpIjeIRU4UgLV2aVLl2jdujUfffSRJM8siHoclXLyTMZoMuI535N6I+oReyAW3geCgTQK+4/OPuJh1EOee+457OzsiI2NxdPTk+PHj2frPgiRVZMnT8bf359OnTphNKb9f0BkP0mgOrp+/TqtWrVizJgxDBo0SO9wrFLg9sB0k2c8o8nIlkVboAEwCagH/JDGAs+D4UMDJpMJk8mEra0tYWFh1K5dOxsiF+LpKaWYM2cOzz77LD169CAmJkbvkAoUSaA6uXfvHm3btqVHjx68+eabeodjtRaHLM7cAteAFy3v2wDRiabNApYAsZhLqGvNSTfiXsRTxylETrGxseHbb79FKcWgQYOIi4vTO6QCQxKoDoxGI507d8bPz48PPvhA73CsVsS9CKJN0enPmJgtcNDy/neStgJ4E3PJdLLlX0uNeuT9SMA8Gk61atWyHrAQOcTe3p5Vq1YRERHB6NGj0TRN75AKBEmguSwmJoZevXpRpkwZ5s6dK93zPYX4xJaqFEqUtAG2A1OAXUCn9LfjXtj9KaIUIncYDAbWr19PUFAQkyZN0jucAkFa4eYiTdN49dVXiY2NZenSpTJE0VNKN7GlVjNeL+PbMNgZ8HD1yPgCQuioSJEibNmyhcaNG1OqVClGjBihd0j5miTQXPTJJ58QGhrK3r17sbe31zscq+fh6oGTnVOmqnFtlS2xWmyG5w/wlq4UhXUpWbIkmzdvpmHDhlSuXJk2bdroHVK+JUWgXLJu3Trmzp3L+vXrKVSokN7h5BuZTXCv1H0Fg50hQ/M62zkzreW0rIQlhK6ee+45Vq9eTf/+/eU55hwkCTQXHD16lICAANauXYuHh1QHZqfpLadnKiHObDWTsOFh6S7jbOfMieEnErrzE8LaNG7cmBkzZtCpUyfpfD6HSALNYVeuXOGll15i/vz5+Pr66h1OvuPs4JzphFixWEVujLvB6Hqjn1jOYGdgdL3RXB93nYrFKuZk6ELkuIEDB9KzZ0+6du3Ko0eP9A4n31GZae7s4+OjHT58OAfDyV+ioqJo2rQpL730Eu+++67e4eRrUY+jmLB9AotDFifpWMFgZyDAO4BpLaelWpqM73zevbC7NBgS+U5cXBzdu3fH1dU14XlRkXFKqSOapvmkOE0SaM6Ii4ujV69eODo6snz5crloc5EkRCGSevjwIY0bN6ZXr16MHz9e73CsSloJVFrh5pCPPvqIy5cvs2PHDkmeuczD1UMSpxCJFCpUiA0bNlC/fn1q1apF+/bt9Q4pX5B7oDlgx44dzJs3j9WrV+PklMpwH0IIkYvKli3LypUrGTx4MBcvXtQ7nHxBEmg2i4yMpF+/fixfvpyyZcvqHY4QQiRo1KgRY8eOpWfPnjx+/FjvcKyeJNBsZDKZ6N27N6+++iovvvhi+gsIIUQue+uttyhZsiRvv/223qFYPUmg2ei9997D0dFRWtwKIfIsGxsbli5dyi+//MJPP/2kdzhWTRoRZZNNmzaxfPlyjhw5gq2trd7hCCFEqooVK8aqVavo0KEDderUoXLlynqHZJWkBJoNLly4wJAhQ1ixYgUlS5bUOxwhhEiXr68vkyZNokePHhiNGRuUXiQlCfQpxd/3HDt2LI0aNdI7HCGEyLDhw4dTrVo13nrrLb1DsUqSQJ/Sxx9/TKFCheQCFEJYHaUUX375JZs2beLXX3/VOxyrI/dAn0JoaCizZ88mJCRExvYUQlilIkWK8M033zBo0CCOHz9OsWLF9A7Jasi3fhY9evSI/v378+mnn1KuXDm9wxFCiCxr2bIlXbp0YdSoUXqHYlUkgWbRpEmTqFy5Mv3799c7FCGEeGrTp08nODiYNWvW6B2K1ZAEmgVBQUEsXbqUL7/8Uvq5LaCqVq2KjY1Nkq4amzVrhq2tLQaDAYPBwPvvv69jhEJkjrOzM0uXLmXkyJFcvXpV73CsgiTQTHr48CEDBw5k/vz58shKARRxL4Lgy8H0G2burjG5du3aYTQaMRqNTJo0SYcIhcg6Pz8/hg4dyiuvvEJmRuoqqKQRUSa9/fbbNGjQgC5duugdisglUY+jCNweyOKQxUSbohM+t99lT1xcHFGPo1Ida1QIazNp0iTq1avHkiVLGDx4sN7h5GmSQDMhKCiItWvXEh4erncoIpecu30Oz/meSQbpjhcTGwNx4DbDjbDhYQBs2bIFg8FAuXLl+PXXX6lYsWJuhyzEU3FwcGDp0qW0atWKDh06SE1bGqQKN4NMJhOvv/46n376KUWLFtU7nAIpvvp0w/YNFCtWDEdHR5ycnOjatStg7lnF0dERg8FAmTJluHDhwlNtL+pxVKrJMzGjyYjnfE8++ewToqKiuH//PiVKlKBNmzZPtX0h9FKnTh369+8vHc6nQxJoBn3++eeUKFGC3r176x1KgRL1OIrRm0djmGqg3Kxy1PuqHp1Xd+ZBqwe8uu5Vwk6GsXHjRtavX0/Xrl25e/cuRqORcuXK0bNnz6faduD2wHSTZzyjycj3177HwcEBOzs7Pv74YxlzUVi1yZMns337dnbv3q13KHmWJNAMiIiI4KOPPmL+/PnS6jYXnbt9DrcZbsw9NDfJvUfcweRpYu6hudT6vhZFihUhPDycCRMmJLSKbdasGdeuXXuq7S8OWZyp+b/c9GXC+5kzZ0rVVyLxtQdhF8NwcXHBYDDg5ORE06ZNATh79izFixfHwcGB4sWLc+7cOZ0jFoULF2bOnDm8/vrrMnZoKiSBZsCYMWMYMWIEVatW1TuUAiPD1adnjdy4foNuvbol+XzJkiW0a9cuy9uPuBeRNGknNwtYAsQC7wNr4dH3j3B0MlchHzhwgPXr12d5+/lBSrUHNb+uSczIGAJ+DiDiagRHjhzhq6++ok+fPvj6+vL48WN8fX2lpieP6NKlCxUrVuSzzz7TO5Q8SRoRpeN///sfoaGhfPfdd3qHUqBkqPr0PrAcaABfnPyCORXnANCqVStsbW2ZN29elrcfeT8y7RneTPnjfcP24VvWN8vbzS9SbXxlA48Nj5l7aC6L9i5CxSqUUoSGhnLo0CHA/EB/vXr1dIhaJKeUYu7cudSrV4/evXtToUIFvUPKU1RmnvXx8fHRDh8+nIPh5C1RUVHUrFmTBQsWSIOQXGaYaki7BPgYcymwHNAXDHYGoiZGERAQwMqVKzl37hxubm5Z3n7EvQjKzUqji0Yj8BmgWV4ewGCo/UttToadxMbGhmLFivH7779Tvnz5LMdhjaIeR+E2wy31H0AmYLr5X+WheHD2AYUcCyV57tDGxoa4uLhciVekb+rUqQm1KgXtNpZS6oimaT4pTZMq3DR8/PHH+Pj4SPLMZelWn8YB84EiQF/zR0aTkXH/N45ly5Zx6NChp0qeAB6uHjjZOaU+gyMwGngXGA9cAftQe3r36J2tDZmsUbq1B3aYj9sboN3Q6PlhwTtG1mbs2LGcOXOGjRs36h1KniIJNBVXrlzhiy++YMaMGXqHUuCkW316GLgDXAemWF47Ydb0WcTGxuLt7Y3BYOD5559/qjgCvANSn2gDFLa8jwE0aFK+SbY3ZLJGGW58VQwoDZtXb8be3p7Q0FDAPMqRnZ3cXcpLHB0d+fTTT3n77bcxmUx6h5NnSBVuKl555RWKFi3KJ598oncoBU661aepuPTmJTxcPbItjqjHUTzzyTM8in2U8gyJqyLLKh78/SBJj0SlSpWia9euLFiwINtiyuvSPXfXMZdAiwFRwBygPtS+Xxv3ku5s2bKFtm3bcuvWrYR7oiJv0DSNli1b0rt3b1555RW9w8k1aVXhys+8FISHh7Nu3TpOnTqldygFUnz1aZrVuMkY7AzZmjzjpXm/J74q8jZoX2qs/2U9vXuYW49mR0Mma5Ru7cE/wFrM943BfA+7BbzX8j0COgfg4OCAi4sLwcHBORuoyDSlFDNmzKBTp0707dsXFxcXvUPSnVThpmD8+PFMmDBBBpbVUZrVp9kwf0YEbg/MWBK3VEVOmDnBHEtAAAcOHOD48eMFbqB198Luac/giflHx/9ZXoPMH9evXZ9bt27x+PFjbt26RaVKlXI0TpE1devWpXnz5nz66ad6h5InFKz/3Rmwa9cuTpw4wfDhw/UOpUCb3nI6BjtDhuZ1tnNmWstp2R5DmvfyrgO3Le+jgEiIcI5gypQp2daQyRql2/gqBTlVeyByxpQpU/j8889lyDMkgSYRFxfHuHHj+Oijj3B0dNQ7nALN2cGZsOFh6SZRZztnTgw/ke2joaTbEvgfYB7wIfAp5t6Rmpl4//33s7UhkzXKC7UHIudUrFiRQYMGyXi3SAJNYvXq1WiaRq9evfQORQAVi1XkxrgbjK43+olEarAzMLreaK6Pu07FYtk/4km69/JSqYoMOh+EyWRKGBO0II7ckxdqD0TOmjhxImvWrOHkyZN6h6IraYVrYTKZqF69OosWLaJFixZ6hyNSEHEvgsj7kbgXds/xKr+80hLYWqU1DFy8+NqDnPgBJHLejBkzOHjwIGvWrNE7lBwlHSlkwMqVKylbtqwkzzzMw9UD37K+uZKg5F7e09Gz9kDkjuHDh7N3717CwsL0DkU3kkAx3/ucOnUq7777rt6hiDxE7uU9HWcHZ+a0m0PUxCguvXmJQ8MOcenNS0RNjGJOuzmp3reuWrUqNjY2CR1SAKxatYrChQvj5OREqVKliIiIyK3dEKkoVKgQY8aMYdq0glsFLwkU+Pnnn3F1deXFF1/UOxSRhxTke3nxw49F3Ivg5ZdfxsnJCScnJ8qXL8+dO3cyvb6M1B7Eb7PfsH4sX748ybTBgwczZcoUoqOjad++PX369Ml0DHlVRHQ0wXfvUrFy5Sd+ODz77LMYDAYMBgN2dnYYDBm7HnPLiBEj2LJlC2fOnNE7FF0U+Hugmqbxwgsv8OGHH9KpUye9wxF5TEG6lxf1OIrA7YEsDln8bwvky8DXEPBDALO7zKZapWq0adOGr776Kue2CdhH2BO3NI57D+7h7OCMUorY2FhsbGw4ePAgTZo04dGjVHqIsgJRJhOBf//N4shIouO/g3/+GTtXV7QZM7j38CHOyboz9PHxwdXVlR07dugQceomTZrE5cuXs+2ayGvkHmga4jtH7tixo86RiLyooNzLS3XwcgANFu9fzDNTn8EYbcy2Tg7S2mZMbAyxcbG4zXDj3O1zuLi4JNxi+eijj6x6gOdzRiNuQUHMvXLl3+QJ0LUrplKliNU03IKCOGf890dbXFwcoaGhTJgwQYeI0/bGG2+wdu1aLly4oHcoua5Al0A1TcPPz49x48bRvXt3vcMRViA3WwLnlnSHH1sJWJ5WUEUVD649eOrnbtPd5gVgGfB/5h8q39X8joAhARiNRurXr8/u3butcrizKJMJt6AgjKnF/scf8NZbsHUrBhsbbjRsiLOdHXPnziUwMJCHDx/mbsAZFBgYyP379/Nl15VSAk3Fb7/9xv379+nataveoQgrkZstgXNLmsOP3QbOAyOAd0AzadTvUj9nt5mM0WRkt8Nubt68SVRUFIGBgTg7Z2/HGbkl8O+/U0+eyRjj4phw7hwACxcuzNNPCPz3v/9lxYoVXLlyRe9QclWBTqAff/wxgYGBBa6/UiESS7PLwoOAM1ACcACqQtjhp39sIcNDnll8uflLwPy89vDhw+nRo8dTx6CHxZHpdNBx/jzExECrVtC6NV8MHUp0dDTh4eH8+uuvKKVYtmxZrsSaGSVLlqR///7MmTNH71ByVYHNHGFhYYSFhdG7d2+9QxFCN+l2WVgSuAs8xDyQ+TnQimtE3Mv6YyTpbnMWsASIBd4H1sKjbY+wd7DH2dmZEiVK8PXXX2d5+3qJiI5Oes8zJXZ2YGsL27bBDz8Q9/vv9Bk0CGdnZ3755ReKFCmSO8FmwejRo/nmm28wGjNWs5AfFNjhzL744gteffVVHBwc9A5FCN2k22WhN3AMc3+/CvMg4i+Zl8tqNXa623wz5Y+DhgXhW9Y3S9vMCyLTazXcqxdcvw6aBi1bmkuhhQuz49dfeemll2jXrl3uBJpFlSpVon79+vzwww8MHTpU73ByRYFMoHfu3GHlypUFsp9SIRJLd/gxgMFZXO5ptpmNy+UV7ukNUPHjj0n//uMP2LaNjeHhNK5WLecCy0YjR45kwoQJDBkyJO2xdPOJAlmFu2TJEtq1a4e7u3X/hxTiaenRZWFB7SbRw8kJp4wmlVu3YNw46NGDNVbURqN169Y8fPiQ/fv36x1KrrCeM5NN4uLimDdvHqNGjdI7FCHyBD26LCyo3SQGZORHe3Q0DB4MdevCa6+l3/AoD7GxsWHkyJHMnTtX71ByRYFLoFu2bMHV1RU/Pz+9QxEiT9Cjy8KC2k3iqDJl0p4hLg6GDIGSJWHqVMD8OEtEdBqNrvKYQYMGsW3bNi5fvqx3KDmuwCXQuXPnMmrUqAJRPy9ERugxeLneA6br5U5sbNozrF8PkZFw8SK0aWN+LVnCO4GB2NracvfuXQYNGoSbm1vuBJwFrq6u9O3bl4ULF+odSo4rUD0RnT17lgYNGnDx4sUkHTYLIcy9A03YPoHFIYuTdHJgsDMQ4B3AtJbTsj2R6bFNPUVER1PuwIFML3fJzw8PK/rO+vPPP2nWrBkRERHY29vrHc5TSasnogKVQCdNmsTdu3eZPXu23qEIkafp0WVhXuwmMSI6mshHj+hZty4X/v4bBwcHop+yOtWwe3f6z4Mmnt/GhqgmTZ5qm3rw9/cnMDDQ6gfpSCuBFpjHWOLi4li2bBk//fST3qEIked5uHrkehLTY5spSXGklPbtsXN1xTRjBlEm0xMjpWRGgLs7czPR5V2GGh7lQQMGDGDZsmVWn0DTUmDuge7fv59ChQrxwgsv6B2KECKPyspIKZk1/bnnMGTw0RRnGxumVbTOkX569uzJ1q1buX37tt6h5JgCk0CXLVvGwIEDpfGQECJFUSYTnsHB6Xb2boyLwzM4mCiTKUvbcbazI8zXN90k6mxjwwlf36cq7eqpWLFitGnThlWrVukdSo4pEAnUaDTy008/8Z///EfvUIQQeVRWR0rJiooGAzcaNmR02bJPJFKDjQ2jy5blesOGVDRk7FGfvCq+Gje/ss6fNpm0fv16fH19KZPeM1hCiAIrsx0WLI6MZE6VKlnenrOdHXOqVGFOlSoJjZXcHR2tqrVtetq0acPQoUM5c+YMlStX1jucbFcgSqDLli1jwIABeodRIETciyD4cjAbtm+gWLFiODo64uTklGTM1e7du+Pg4ICTkxP16tXTMVohzDI0Ukoy2dnBgYeTE75FiuSr5Algb29P3759Wb58ud6h5Ih8XwK9du0aQUFB+boeXm9Rj6MI3B7I4pDF/w5TFQl2rex4ffDrvFHjDWpUrcH69es5c+YMO3bs4MaNG7i6uhIW9vRjSwrxtLI0UkpgIJGPHuW7pJfdBgwYQNeuXZk8eXK+a4OS7xPounXraNeuHYUKFdI7lHzp3O1zeM73TPIQPADuYHI3MffQXL4K+YoixYoQHh7OvHnzCAwMxNXVFQBPT08dohYiqUyPlJLR5QReXl44ODgQEhJC3bp19Q4nW+X7Ktx169bRpUsXvcPIl6IeR6WcPJMxnjVy4/oNuvXqxo0bN9i0aRMuLi4ULVqUpUuX5lK0QqQuUyOlWBhsbKT0mQFKKV5++WXWrVundyjZLl8n0Hv37rF//37atm2rdyj5UuD2wHSTJ/eB5UAD+OLkF2iaxp07d7h37x6zZs1i2LBhxGWw5aMQOSmzHRZYawcHepAEaoU2b95M48aNKVy4sN6h5EuLQxanPcNjYD7wHNDaPL+LiwuDBw/GxsaGwYPNIzWfOnUqp0MVIl0FpYMDPdSvX58bN25w5swZvUPJVvk6ga5bt46XX35Z7zDyhYh7EXy78VuKFC2Co6Mjjo6ORH9naTB0AvgI+BCYAdwF4jAnzyJAX/NsRpORRs0b8fPPPwPw66+/omka1apVy+W9EeJJBaWDAz3Y2NjQuXNnfvnlF71DyVb5NoE+evSILVu25Ot+GHNa1OMoRm8ejWGqgXKzyjFk4xDutbyHzXs2tPq8FfwFnAR+AVoA/wdUAX4CDgN3gOvAFMtrJ7w55U0uXbqEk5MTXbp04eOPP8Ymg7/6hchpBaWDAz289NJL+S6B5tufULt27aJGjRqULl1a71CsUoqta93Nr2hTNJuubgJnzAkyBoh/nNMH+BYYkuizRCq5V+LcU/TgIkROKwgdHOihRYsW9O3bl3/++YeSJUvqHU62yLc//aX6Nusy1Lr2AvAQqA04ADssn+8BUhkz2GBnyBOjbQiRUfm1gwM9ODk50bp1azZu3Kh3KNkmXyZQTdPYuHEjnTt31jsUq5Ru69pELWspAnQHjmCupn2c+mIB3gHZGKUQwtp07tyZDRs26B1GtsmXCTS+pZc0TsmaNFvXJmtZC0BVYDzwLuAPpDAAvbOdM9NaTsveQIUQVuXFF19k9+7dxMamUk1lZfJlAt2xYwctWrTId91G5YaIexH/dseXXAotawG4Zvk3FtgEJOtcyNnOmRPDT+Ds4JzN0WbcnTt3cHFxwWAw4OTkRNOmTQFo3LgxBoMBg8FA8eLFCQkJ0S1GIfK70qVL4+7uTmhoqN6hZIt82Yhox44dtG/fXu8wrFLk/TRGpIhvWWuLuboWzCXOa6BOKzRNMzc0stScG+wMBHgHMK3lNN2SZ8S9CCLvR1KqUCnOnDlD6dKliYqKomTJknz11VesWLECDw/zfdlu3brRr18/wsPDdYlViIKgRYsW7NixI19065fvEmhcXBw7d+5kxowZeodildwLp9G76N4qnQAAIABJREFUSj1SbFkLcPHNi3i4eiQkLPfC7ro1GEqxc3vAyc6JAO8AxtYZS1xcHEqphOQJ8ODBA6m1ECKHtWjRgkWLFjFu3Di9Q3lqSsvEED4+Pj7a4cOHczCcp/fHH3/QtWtXTp8+rXcoVssw1ZB6NW5K89sZiJoYlYMRZVyqndsDNtE2xM2MAxPUqVuH0MPmaiR/f38OHjyInZ0dR48epUaNGrkdthAFxq1bt6hQoQI3btzAwcFB73DSpZQ6ommaT0rT8t090O3bt9OiRQu9w7BqmW0tm1da16b1+E2Rh0VYO3stRd4sAm/AsbBjrFy9EoD9+/djMplo2rQpr732Wm6HLUSB8swzz1ClShUOHTqkdyhPLd8l0PgGRCLrprecjsEuYz2t5KXWtWk9ftPoZCNco13xP+kPxYDSMGHmhCTzvPfeexw8eDAXIhWiYIu/D2rt8lUCNZlM7Nmzh2bNmukdilVzdnAmbHhYukk0L7SuTSytx2/qHKzDVa7SNrQtRAGREOEcwbZt2xLmmTVrFm5ubrkQqRAFW35JoPmqEVF4eDilSpWiVKlSeodi9SoWq8iNcTeYsH0Ci0MWJynZ5YXWtcml9fiNi9EFdV0xkIFolzT4FCgHpmYm+g/oz53bd1BK4erqyqZNm3I3cCEKIH9/fw4fPozJZMLOijvlt97IUxAcHIyvr6/eYeQbzg7OzGk3hznt5uSJ1rVpSfz4zX/X/5em4U1RmFvU2sbZgh38GvMrUfZRoCD2aixMh8KOhXEf6E61L6XTDSFyi6urKx4eHoSHh1O7dm29w8myfFWFe/jwYUmgOcTD1QPfsr55MnlC0sdvVjVcxZ1Cd3CMcaRwdGGcHzvjHGMuKTvHOOP82JnC0YVxjHHEvqQ95d4qp1fYIgdFREcTfPcuFStXxsbGBqdE/dlKBxr68/X1JTg4WO8wnkq+SqDBwcH4+KTY2ljkcx6uHjjZmb8gI9wiGPb6MDZ5byLaPuVq3Wi7aLb4bqHBiQY4V80b1dDi6UWZTIz+6y8Mu3dT7sAB6h09yvn27bF95x1MmkaUyQTAihUrMBqNGI1GmjVrRr9+/XSOvODx8fEhrz8WmZ58k0AfPXpEeHg4L7zwgt6hCJ0kfpwmxi6Gzzt8zg+NfiDKIekzqlH2UfzQ+Adi34vFxiHf/Bco8M4ZjbgFBTH3yhWiEz/f3rUrplKliNU03IKCOGc0SgcaeYCUQPOQ48ePU6VKFZydpTRRUKX0+I3nJU+cH5uvCZMylz6cY5ypdblWnnn8Rjy9KJMJz+BgjHFxac5njIvDMziYKJMJf39/7Ozs2L17N2vWrMmlSEU8Ly8vwsPDefTokd6hZFm+SaCHDx+W6tsCLvnjN46PHfE65wWYq2z3Pr+XaDtzla7vBV8cYx11i1Vkr8C//043ecYzxsUx4dw56UBDZ87OzlSpUoXjx4/rHUqW5ZsEKi1wBfz7+M3oeqPxP++PQ6wDN11uMmbwGGb0mcHvM3/HvrQ9PIJbv97SO1yRTRZHpjEIQjrzSwca+vn/9u49Lsoy///4a8YRZkBQVtRAtCABj9+UHDR45BnNNDXzkG66WbKekty2Wqx229pcsbb4Kerm2WjNzA4esFJTMw8heE5UNMEDggkqgnIcZn5/oOQBgRkZ7jl8no/HPFS8x3lnOO+57+u6r6tz5852fRnXYW5j2bdvH5MnT1Y6hrABN2+/OXfiHBcKL+A+050ffH6omEFsGGcgNTKVojM1X+9X2K6MoqLbxzxroDApiYzQUPy0WllAQ0FSoDagrKyMEydO0LZtW6WjCBvS4uUWtHj57ltUNJ4a2q1qV8kzhD3Kqm4MbeRIyM4Gkwl694aICNi7l4C33qKeLKChqPbt2xMfH690DIs5RIGeOXOGpk2bygQiIZyQj2s1Y9mrVlX65bSuXfG75d5QUfeCg4NJTU3FZDLZ5UxohxgDPX78OMHBspKMEM7IT6tFa+abr06tlvK0AU2aNMFkMpGTk6N0FIs4RIGmpqZKgQrhxCJ9qtgIvhaOF9ahUqlo3bo1qampSkexiBSoEMLuxQQEoFPX7O3MTa1mpr+/lROJmgoODub48eNKx7CIwxRo69atlY4hhFCIm0ZDil5fbYm6qdUc0etxs+MdQBzNzXFQe+QQBSpjoEIIf52OnLAwopo3v6tIdWo1Uc2bkx0Whr+uZpvFi7phzwVq9x/D8vLyyM/Pp3nz5kpHEUIozE2jYXZgILMDA8koKiKruBgfV1eZMGTD7HkM1O4L9OTJkzz88MOoazj+IYRwDn5arRSnHXj44Yc5ffq0XW6ubfetc/78eVq0kP0chRDCHrm6uuLl5cXFixeVjmI2uy/QzMxMuXwrhBB2rHnz5mRmZiodw2wOUaC+vr5KxxBCCGEhX19fKVAlSIEKIYR9kwJViBSoEELYNylQhUiBCiGEfZMCVYgU6P3LyMsg+Xwy67esx8vLC1dXV7RaLUOHDgXg8ccfR6fTodPpaNy4Mfv371c4sRDCkUiBKqCkpITc3FyaNGmidBS7U1BSQNR3Uehm6GgR24LQxaEMWj2IaxHXmLBmAinHU0hISGDdunWsXLmSwsJCCgsL6dGjB88995zS8YVwCO+99x4uLi7Ur1+fJ554Quk4ipECVcDFixfx9vaWRRTMlH4lHe8PvIlLiqPIUPT7b/iAoZ2BuKQ4OqzoQEOvhhw9ehQ/P7+KQ65du2aX+/YJYSsyiopIvnqVtLw83nnnHTZt2sSVK1fYvn0769atUzqeIh544AGysrKUjmE2+1r24Q5Xr16lYcOGSsewKwUlBbSb345CQ2GVxxWeKqQwu5BnRj4DQHh4OHv27EGj0XDgwIG6iCqEwygwGIhOS2NRVhZFJlP5F9evBzc3vvb1JVSrpVu3bsybN49BgwYpG1YBnp6e5OXlKR3DbHZ96pafn4+Hh4fSMexK9JboasuTfOBT4DGYe3wuALt27cJgMNC9e3cmTpxo9ZxCOIr0wkK8d+8mLjPz9/IEOHcOPD2Jy8zEe/duGrdoYZdnYbXBzc2NkpISDAaD0lHMIgXqZBbtX1T1ASXAfCAA6Hv38f/4xz/Ys2ePteIpLiMvgy1Ht+Dm7oZOp0Or1dK9e3cAXnnlFbRaLSqVivj4eIWTCntQYDDQLjmZQqPx7t+8pUwLjUa+uHgR091HOQWVSkWDBg24du2a0lHMIgXqRDLyMm4f87yTkfLybAiMLv9SoaGQz9Z+VnFIbGws3t7e1oxZ5+6cUNVnVR8KJxXCW/D8yufZt28fixcvplevXnzzzTcybCBqLDotrfLyBGjZEm65bFmWlUWeE39veXh4kJ+fr3QMs9j1GKgUqHmy8qu5PLQXyAXqAe/d+Fo4TPvvNF4Y+QIqlQpPT082bNhg3aB1KP1K+t1jwmrAA4oMRSz4eQEUw6WCS4wfOF6xnMI+Larqkmy/fjB7Nhw6BIGBcPgwWW+/XXfhbIwUaB2TAjWPj4dP1QeE3njcYf+6/fh5+t39G3auyglVBiDmxo/N4Z38d5haMhU3F7c6TinsVUZR0e1jnndycYHnnoNXXy2/nNuxI6VhYWQUFTnlNmz2WKByCdeJ+Hn6odWY9w9Tp9E5ZHlCNROqNMBbwMvAJSg8XMj0LdPrMJ2wd1nFxdUf9PzzsHkz/PAD/Oc/NX+eA5ICrWP5+fl4enoqHcOuRIZEWvV4e1LthCoAL+AB4EANjxfiBh9X1zp9nr3z9PSUAq1LBQUFuLnJJTVzxPSOQafR1ehYN40bM3vPtHIiZVQ5oSobuHLj5wVAFuBTPqEqIy+jbgIKu+en1aI1c9ERnVrtlJdvAdzd3WUWbl0qKytDo7HrYdw65+biRsrklGpL1E3jxpHJRxx2zK/KCVUXgXnAv4APAR+gF7AZHvR6kKtXr/L888873GxkUfsifaqZd3CfxzuSevXqYbzXjGUbZdcFajQaZRk/C/h7+ZPzWg5RoVF3FalOoyMqNIrs17Lx9/JXKKH1VTmhqh3l459/v/F4/sbXI+DMlTOYTCaMRiM5OTnWjinsXExAALoavke5qdXM9Hfcf3PVUavVUqB1qaysTArUQm4ubszuP5sTU0+wftR6/tPmP3jGelL2bhkLnl7Ac8+WLxjfsmXLip1YNBoNOl3NLv/aOplQJeqCm0ZDil5fbYm6qdUc0etxc+IravZYoHb9f0vOQC1TUFJA9JZoFu1f9Ps4YBZo+mqYNG4SL7d5mTZBbVi3bh1nz56teF7nzp0datJWZEgkcUlxZh0vhLn8dTpywsKYnp7Ooqys2xZW0KnVRPr4MNPf36nLE8oLtKysTOkYZrHr/2OyK4j5Kl04AMp3YvEp34ll8f7FFTux3FzY2mg0cvDgQb777jsFUltHTO8YFu9fXP3awDj2hCphfW4aDbMDA5kdGEhGURFZxcX4uLpWOmHIWGJE7eJ8JwYmk8nuTojsK+0d7PGUX0nm7MSSk51TsRMLwLx583B1dSUiIsLaMeuMTKgSSvDTatE3bFhpeZZkl7D7gd2UZJcokExZ9nhF0b7S3kEK1DyW7sQC8PHHH9OrVy+r5lOCTKgStiRnTQ6GKwZy1jrfBDV7LFC7voRrj9fMlWTJTiyz+8+mqKiIY8eOsXz5cuuHVMDNCVWz+88mIy+DrPwsfDx8ZMKQqHMXll2o+NF3vK/CaeqWPRaofaW9Q/369SktLVU6RqUy8jJYlrCMho0a4urqilarZejQoQD06NGDevXqVcxufeedd+okjyU7sWTkZfCf//wHT09P9Hq91XMqzc/TD31zvdOVZ1Xfr6JulF4pJX9f+Uo8+XvzKc21zfc2azEYDHZ3X799pb2Dre0fd9fs1iygN2hDtIx6cBT/e+F/rFu3DoD+/fuTkJBQZ9ks3Ykla3wWy5cv58knn7RuQFHnavr9enMimahdqX9OJfvLbG5uAmoymFDVV2EqKf8xsUUiKs2NiZIqaDK8CcELgpULbGX5+fk0aNBA6RhmsesC9fDw4LffflM6BnCP2a0+5Y8iQxHLTi0DV9i5byfFhmLKispYv2U9Y4eNpaCgAJVKxZNPPsnXX39d8fSBAweyYcMGjh8/TnDw/f3DsXQnFh8PH3799df7em1he8z5fpUCtY4Wr7Yg96dcitKLMJXcvmuL8frvcztULiq0/lpa/LVFneS6c5ZwSUkJjRo1svr7rT1uDmLXl3BtZfX+Gs1uPQNchw/yPyAxI5GdW3cyqN8gcg25PPvfZ0k5nkJCQkLFGeqePXtITEykXr16tZJRFg4QN5nz/TqncA4FJQV1ls2ZuAW5oT+kxyfSB7Vb5W/Fap0a3z/7oj+sxy3IerPACwwGok6cQLd9Oy0SEwk9cIAWiYnotm+ndf/+eDdpYrXXvkkKtI7ZSoFWO7v1lpmtNASeBN4E3gC8IP61eDqs6FBx7yXAM888w/z582s1p+zEIsC879di92LZxs2K1K5qguYG0TK6JfUa3P5hWd1ATcvpLQmMC7TqfaHphYV4795NXGbmXfuXFh07RnpSElkDB1JW1d6mtUAKtI7ZSoFWObv1jpmtADSj/OJ5PaAPcPX2ey/ffPNNvL29GTFiRK3mlJ1YBJj//SrbuFlf3u48yq7duKPgxsCa8ZqRvMQ8q75ugcFAu+Tk21ZHus3f/w5Tp2JQqbhUWkqBwWC1LFKgdczDw4O8POt+g1WnytmtlcxsBcona9y0G3Cj4hP/+3veZ/bs2RWXcmuTLBwgLPl+lW3crKusoIwr28r3z1Pr1DQZ2gS1rvyt+cqWK5QV3vtWvYyiIpKvXiWjqIigoCDUajXaWxZo0Ov1uLq6otPp8PX15cyZM7c9Pzot7d7luWwZNGgAfftWfGl6erql/5nVysvLkwKtS7ZwBlrl7NabM1uzKZ/Z+h6wDVhF+VZZ/wLOUf6p/8Yn/k/WfkJBQQEBAQFoNBrKyspo164dhw8frpW8snCAc7Po+7W654n7cnnjZUzFJlx8XOi4vSPtVrWj4/aOuPi4YCo2cXnj5duOv9d4ZfoTT9Dn/fdvO3bo0KFcvXqVwsJCWrRocddVrUVZVX0/7IWzZ6F3b5g/H3JzievWrdb+u29VVlZGYWEh7u7uVvnzrcXuZ+EqXaBVzm69x8xWet740QjMAbRUfOIvbV3KudxzFZN3NBoNKSkp9z0L91aycIDzsuj7tbrniftSdLaIJiOaELw4GI1H+Vuyp96T0OOhpEamUnTm9ysG6YWF97zkahg6lM2//AJGI+mFhfjrdEyf/vv4dY8ePVi1alXFrzOKiu4a87zNvHm///zLL2HFCkwrV5JRVFTrm35fu3YNd3d3u1tIwa4LtFGjRuTm5iqa4ebs1ioXKbiXKu69rKsy8/P0k+J0IpZ8v8psbOtq8XILePnur2s8NbRb1a7i19WOV96iXXIyOWFht+3wsnz58tsWx8gqLrYob1Zxca0XaG5uLo0aNarVP7Mu2Ffd36FJkyZcvXqVkhJlF162eLZqKPBPyjdtfuvGo+ftn/YNBkOtnn0KIbOx7VOV45V3KDQabxuvjIiIoF69esy75azSx9W15i8+bBh88435z6uhrKwsfH3tb+lCuy5QtVpNs2bNuHDhgqI5zJndWh35tC+sTWZjW8fNCT0pFy7QoEEDdDodWq2W7t2718qfX+V4ZRXHR0ZGkpiYyOHDh2+7ROqn1aI1c0tInVpd62efAJmZmVKgSvD19SUzM1PRDDWd3VoT8mlfWJvMxq49lU3oaX/0KKXx8UQeOkTGxYvs27ePxYsX39frVDteWYlCo5HX/vlP4uPjSUpKwtvb+65jIn3MG9se1aQJZ86cwc/PD1dXV1xdXVm4cKFZf0ZlpEAVYgsFClXPbq0p+bQv6orMxr5/91yAQK2m5A9/IC4zE78ff6TMaERl5pnenWo0XjlyJLz8MpSWls+cjYkhdsYMysrKCAkJQafT0bZt29ueEhMQgM6MiTtLf/uNh8LC8NDrOXLpEpvS0vjXv/991+0zAMOGDcPFxQWtVkto6D1mp91grwVq15OIoLxAz58/r3QM4N6zW0vLSqtdOk0+7Yu6JrOxLVfthJ6SEnjqKYpLSlC1acOoP/3pvl6vRuOOt8ywven0mjVVXnJ102hI0etrPDmJ7Gz47TeOT51K0N695V8bNAiNpyeGDz6gwGDATaPho48+YuvWreTk5ODp6UlKSkqVf+z58+fp0aNH9a9vY+QM1Epu3RZLPu0LW+es27hZqtoJPS4usHEjrFyJ6dw5RixYcF+vZ83xSn+djpywMKKaN7/7bLR37/KFFPr1gyeegF9+gfr1YcyY8q+NGQM9e2Jo1owykwnv3btJLywkNjaW6OhoPD09AWjXrl0lr/w7ez0DtfsCbd68uU0W6J1uftoveLOAc385R9L4JM795RwFbxYwu/9sOfMUwo7UeELPAw9Aq1Z8t2LFfb+mueOVdx5f2UpFr7zyClqtFvf69Tn69deYKhtnXbKk/MPA99+DwQBFRTB6dPnXXF3h7bcrDi00GstvocnJYcOGDTRo0IBGjRrxySefVJk1MzOT5s2bm/XfZwvsvkBt9Qy0KvJpXwj7Ve2EnjNn4OadAXl5cOIExsBAMoosuFf8FuaMV7qp1cz096/Im3z1Ks9NnMinn35623G9evXi41WrwN2dH65erX6i0sMPg1oNAwaU/3rAADh37rZDCo1GSo1GcnNzycvLIzY2lvHjx2Os4ozdXs9A7X4MtEWLFnet7yiEENZS7YSe9HSYeWMyoMkE7drBuHH3vQBBTccr3dRqkjp1IjotjUVZWb+XYkgI9X/5BaPJVDFW2euJJ/DevfveL6pSQeSNOwMefxzefBO0WtizB7p0ge3boVmzu55WptMxbtw41Go148aN489//jOpqam0adPmrmPz8vIoLS3Fy8vLrL8PW2D3BRoQEMDZs2cpKSnBxcVF6ThCCAdX7YSeHj3KH+Y+rwZujldOT09nUVbWbUWqU6uJ9PHhzw88gP7AgUpLthTgxlhlil5P7LlzVY/lzp8PQUHlHwomToQ2beCvfy2/bGsylS82P3cu5OTc/rxHH2Xll18ybdo0Nm7ciMlkuueCMKmpqQQFBd33TGUl2H2Burq64ufnR1paGq1bt1Y6jhDCwd2c0GPOfZm1uQCBm0bD7MBAZt+4LJxVXIyPqyt+Wi0FBgPeu3dXO6P25lhlVZdVgfLyBPD3Ly/P5OTys+tevX4/ZuTI8tm5JlP5pKOICPjrXzk3aRJarRa1Ws2sWbPuuc5tamqq3a62ZvcFCtC6dWtSU1OlQIUQdSLSx4c4M+Ze1GQC0M0yHPHoo5xJS8PFxYWiG+Omp06dIjQ0tGLPzL179+Lv74+fVntbMZu73F+VrlyBsjLw9i7/+YkTMGrU3cdVcvsMQFJqao0+NBw/ftxuC9TuJxEBBAcHc/z4caVjCCGchKUTeu5U2UpGp598knpvvIHhxlglwKhRo9Dr9ZSUlKDX63n22Wcr/fPMXe6vSmfOwB//WH67ysiR0LZt+W0rNWDOGbecgSosODiYxMREpWMIIZyEORN6juj1t+2KctM9tyYbOhTDL7/cNlZ58OBBkpKSAIiJial0ZZ9qZwdXdqnVywu++AKMRoiJKd/CbO3a8uM7diy/VcUC5txyIwWqsODg4GrvMxJCiNpUkwk9M/39Ky3Pmm5NdnOssrS0lI4dOwLQsWNHDDfOTG9V7ezge1xqZcKEqp9npqrOuO9UVlbGyZMnCbo51mpnHKJAb46BCiFEXapqQk9VanWs8gZrbDNmrqrOuCtz9uxZvL29adCggZWTWYdDjIE2bdoUg8FAzp1TqYUQoo74abXoGzas0dif2WOVGg0HDx4E4ODBg2gqKShLlvvTqlQ1HsvVAgdDQipd8k+nVhPVvDnZYWH462q+mYY9X74FBylQlUpFhw4dOHz4sNJRhBCiSpZsTcbDDzPttdcAiI6Orriceydzl/v7s68vKXp9tSXqplZztEsXHvH0ZHZgIAXdunGua1eSOnXiXNeuFHTrxuzAwBqfed506NAh2rdvb9ZzbIlDFChA586dSU5OVjqGEEJUqdqxykq2JePNN9m/dy8uLi4kJSWxcuXKSp9qyezgqhaTr+rM0pwz7nvZu3cver3e4ucrzSHGQAH0ej3ffPON0jGEEKJK1Y5V3mOyz9GsrGrLytLZwZaO5d6v5ORkZsyYYdXXsCY5AxVCiDpkza3JoOrtyWoyVlkbZ5Y1kZ2dTW5uLq1atbLq61iTw5yBtmrVitzcXLKzs2nSpInScYQQ4p6ssZLRrZQ6ozTH3r17efTRR++5xJ89sN/kd1Cr1Tz66KPsvblLuhBCWODm9l+frluHTqereKhUKp5++ulaeY3aWsmoJurqjNJcycnJdj3+CQ5UoCCXcYUQlqlsSb2xnp7w/fdEHjpE1qVLqNVqXn311Vp5vZtjlTWZ/WrOfZX2ZO/evXTu3FnpGPfFoQpUr9dLgQohzJJeWIj37t3EZWbedXtJkclEXGYmTaZNQ+fmRnh4eK297v2OVdozk8nkEGegDvWxRq/XM2XKFEwmk13uLSeEqFs1XVLP8O23GDt3rtiIurbYw1ilNZw/fx6DwUDLli2VjnJfHOoMtGXLlmi1WlnWTwhRIzVaUq+gADIzMY4Zw/T0dKtlsdWxSmv48ccf6datm92f6DhUgapUKnr16sXWrVuVjiKEsAM1WlJvxQrw8ICAgNrdLsyJbd26lV63bsptpxyqQAEpUCFEjdR4Sb0ffoAbY5+FRiMZNza5FpYxmUxs2bJFCtQW9ezZk23btmGs4Q4GQgjnVO2SegC5ueV7aP7pT+Y9T9xTeno6JSUltG7dWuko983hCtTPzw9vb29ZWF44pIy8DJLPJ7N+y3q8vLxwdXVFq9UydOhQAFatWkWDBg3Q6XS4u7uzbNkyhRPbrhpt/9WoEWzdCs2amfc8cU83L9/a+/gnOGCBglzGFY6loKSAqO+i0M3Q0SK2BaGLQxm0ehDXIq4xYc0EUo6nkJCQwLp165g8eTKvv/46hYWFvP766/zlL39ROr7NsvaSeqJyjjL+CVKgQti09CvpeH/gTVxSHEWGW8befMDQzkBcUhwdVnSgoVdDjh49ikql4tKlSwDk5OTQqFEjhZLbB3OXyDP3eHE7k8nkUAWqMpmxL13nzp1N9rBUXnZ2Nq1atSInJ4f69esrHUcIixSUFOD9gTeFhsKqDzwDLIcTp06QmpLKkCFDgPI3q59++qlWb/53NAUGA967d1d/KwvlqwJlh4U55KpAdeXo0aMMGDCAdCveDlTbVCrVPpPJVOmSSQ55BtqkSRMCAgJITExUOooQFoveEl19eeYDnwKPwdzjc3nzzTf5y1/+gsFgICoqiqeeeqouototWVKvbm3cuJE+ffooHaPWOGSBAgwaNIi1a9cqHUMIiy3av6jqA0qA+UAA0Lf8+MOHDzNr1iwAPvzwQ65cuWLtmHbPmZfUq2tr165l8ODBSseoNQ5boEOGDGHNmjWYc4laCFuRkZdx+5jnnYyUl2dDYHT5lwoNhdR3qc+cOXOA8gLVyZt+jdxcUq+gWzfOde1KUqdOnOvalYJu3ZgdGChnnrUgJyeHAwcO0Lt3b6Wj1BqH/a7o2LEjpaWlpKSk0L59e6XjCGGWrPxqVrzZC+QC9YD3bnwtHKa9PY033niD6dOnU69ePRYuXGjdoA7IT6uVmbZWkJCQQEREhEN9qHPYAlWpVBVnoVKgwt74eFQz2zP0xuMOUS9F8f4b71slkxD3Y82aNQwbNkzpGLXKYS/hwu+XcYWwN36efmg15p0F6TQ6/Dz9rJRICMsVFBSwbds2BgwYoHSUWuXQBfr444+Tnp7OuXPnlI4ihNkiQyKterwQdWXz5s107twZLy8vpaPUKocuUI1Gw8CBA1lUavnqAAAb9klEQVS3bp3SUYQwW0zvGHSamo0XuWncmNl7ppUTCWGZNWvWVNyf7EgcukCh/DLu119/rXQMIczm5uJGyuSUakvUTePGkclHcHNxq6NkQtRcaWkpCQkJDnX7yk0OX6BPPPEEBw4c4Pz580pHEcJs/l7+5LyWQ1Ro1F1FqtPoiAqNIvu1bPy9/BVKKETVNm3aRFBQEC1btlQ6Sq1zyKX87hQZGUlQUBCvvfaa0lGEuC8ZeRlk5Wfh4+EjE4aEXRgxYgS9e/dmwoQJSkexSFVL+TlFge7YsYNJkybxyy+/OMQWOkIIYQ+uXLnCQw89xOnTp+12ApHTrYV7p/DwcAoKCjhw4IDSUYQQwmmsXr2avn372m15VscpClStVjNmzBji4+OVjiKEEE4jPj6esWPHKh3DapyiQAHGjBnDypUrKS0tVTqKEEI4vFOnTnHixAmeeOIJpaNYjdMUaKtWrWjVqhUbN25UOooQQji8+Ph4Ro0a5dB7MjtNgQKMHTuWTz75ROkYQgjh0IxGI59++qlDX74FJyvQkSNHsmXLFi5cuKB0FCGEcFg//PADHh4ehISEKB3FqpyqQBs1asSIESNkiychhLCiuLg4pk6d6vC3DTpVgQJMmTKFjz/+mJKSEqWjCCGEw0lLS+Pnn39m9OjRSkexOqcr0A4dOhAcHCzr4wohhBXMnz+fcePG4ebm+GszO12BAkydOpW4uDilYwghhEO5fv06y5cvZ/LkyUpHqRNOWaCDBg0iIyOD/fv3Kx1FCCEcxooVKwgPD8ff3zk2N3DKAtVoNEyaNIm5c+cqHUUIIRyCyWRi7ty5TJ06VekodcYpCxRg/PjxfPPNN+Tk5CgdRQgh7N5PP/1EaWkpvXv3VjpKnXHaAvX29mbo0KHMmzdP6ShCCGH3PvzwQ6Kiohz+1pVbOW2BAvztb39j7ty55OXlKR1FCCHs1sGDB9m7dy/jxo1TOkqdcuoCDQoKIiIigv/+979KRxFCCLs1Y8YMXn31VbRardJR6pRTbKhdlSNHjtCnTx/S0tKc4r4lIYSoTUePHqVnz56kpaXh7u6udJxa5/Qbalelffv2hIeHs2jRIqWjCCGE3Zk5cybTpk1zyPKsjtOfgQLs37+fQYMG8euvvzrdJQghhLDUr7/+SteuXUlLS8PT01PpOFYhZ6DVCAkJ4ZFHHmH58uVKRxFCCLsRExPDlClTHLY8qyNnoDf8/PPPjBo1ipMnTzr0BrBCCFEbzp49S8eOHTl58iSNGzdWOo7VyBloDTz22GMEBgayePFipaMIIYTNe/fdd5kwYYJDl2d1NEoHsCWzZs1iwIABPPfcc3h4eCgdRwghbNKRI0dYt24dJ06cUDqKouQM9BYhISH07t2bDz/8UOkoQghhs6Kjo3njjTdo1KiR0lEUJQV6h/fee4+4uDiysrKUjiKEEDZn27ZtHD16lEmTJikdRXFSoHd46KGHGDduHP/85z+VjiKEEDbFaDTy2muv8e9//xtXV1el4yhOCrQSb775Jl9//TXHjh1TOooQQtiMVatWoVarGTFihNJRbIIUaCW8vLyIjo4mOjpa6SgOJyMvg+Tzyazfsh4vLy9cXV3RarUMHToUgFOnTtG4cWNcXFxo3Lgx6enpCicWQgAUFxfzxhtv8P7776NWS3WAFOg9TZkyhcOHD7N9+3alo9i9gpICor6LQjdDR4vYFoQuDmXQ6kFci7jGhDUTSDmeQkJCAuvWrWPUqFHo9XpKSkrQ6/U8++yzSscXQgDz5s2jffv29OjRQ+koNkNuY7kHrVZLTEwMU6dOZd++fbK4goXSr6TTbn47Cg2Ft/+GDxh8DMQlxbF4/2IaejXk6NGjHDx4kKSkJKB8lZPQ0FAFUgshbpWVlcW///1vduzYoXQUmyJnoFUYMWIEzZo1Y86cOUpHsUsFJQWVl+cdCk8VkpOdwzMjn6G0tJSOHTsC0LFjRwwGQ11EFUJU4ZVXXiEyMpI2bdooHcWmyBloFVQqFfPmzSMsLIwRI0bQokULpSPZlegt0dWWJ/nAp8BjMPf43LqIJYQww+bNm0lMTGTJkiVKR7E5cgZajaCgIF566SWmTZumdBS7s2h/NVvElQDzgQCgb/nx9evX5+DBg0D5LvcajXzGE0IpRUVFTJkyhbi4ONkvuRJSoDUQHR3N4cOH+fbbb5WOYjcy8jIoMhTd+wAj5eXZEBhd/qVCQyFt2repmP0cHR1dcTlXCFH33n//fdq1a8fAgQOVjmKT5ON9DWi1WubNm8fEiRM5cuSIfBKrgaz8alZy2gvkAvWA9258LRz+MecfRA6KxMXFhQYNGpCcnGzdoEKISv3666/MmTOH/fv3Kx3FZkmB1lDfvn3R6/XMmDGDGTNmKB3H5vl4+FR9QOiNxx26/F8XLl++bJVMQoiaMZlMTJkyhejoaFq2bKl0HJsll3DNEBsby8KFCzly5IjSUWyen6cfWo3WrOfoNDr8PP2slEgIUVOff/45mZmZvPzyy0pHsWlSoGbw9fVl5syZjB07lpKSEqXj2LzIkEirHi+EqH2ZmZlMmzaNpUuXyv3v1ZACNdOLL76Ir68v7733XvUHO7mY3jHoNLoaHeumcWNm75lWTiSEqIrJZGL8+PFMnDgRvV6vdBybJwVqJpVKxaJFi1iwYIFMcKmGm4sbKZNTqi1RN40bRyYfwc1FJmcJoaTFixfz22+/8dZbbykdxS5IgVrAx8eHOXPmMHbsWAoLq1kowMn5e/mT81oOUaFRdxWpTqMjKjSK7Ney8ffyVyihEAIgPT2dN954g/j4eLl0W0Mqk8lU44M7d+5s2rt3rxXj2Jdnn30WHx8fYmNjlY5iNzLyMsjKz8LHw0cmDAlhI8rKyujZsyeDBw/mr3/9q9JxbIpKpdpnMpk6V/Z7chvLfZg3bx7/93//x6BBg+jZs6fSceyCn6efFKcQNub//b//ByArrplJLuHeh8aNG7No0SLGjRvH1atXlY4jhBBmS0lJISYmhuXLl1OvXj2l49gVKdD79OSTT/Lkk08yfvx4zLkcLoQQSrt+/TojRoxg1qxZBAQEKB3H7kiB1oKPPvqItLQ04uLilI4ihAAyiopIvnqV/oMHo9Vq0Wq1PP3000rHsikmk4lJkyah1+sZN26c0nHskoyB1gKtVsvq1avp2rUrXbp0oUuXLkpHEsLpFBgMRKelsSgriyKTCXbsgO++w2X1al7092dljx5s3ryZiIgIpaPahCVLlrB//3727NmDSqVSOo5dkjPQWhIQEMDChQsZOXIkly5dUjqOEE4lvbAQ7927icvMLC9PgMOHoVkzSho25L+XL5Pn78+/P/xQ2aA24uDBg0yfPp0vv/wSd3d3pePYLSnQWjRkyBCeeeYZxo4di9FoVDqOEE6hwGCgXXIyhXf+mwsJgQsX4Nw5yM3FeOwY20+epMBgUCaojbh69SrDhw9nzpw5tG7dWuk4dk0KtJbFxMRw5coVPvjgA6WjCOEUotPS7i5PgMceg969ITISnnsOmjbFpFYzPT297kPaiJtL9UVERDBq1Cil49g9KdBaVr9+fVatWkVsbCzbt29XOo4QDm9RVhV7z0ZHw/ffQ0ICuLuDn1/Vxzu4uLg40tLS+Oijj5SO4hCkQK2gRYsWxMfHM2rUKE6fPq10HCEcVkZR0e9jnpW5ebaZkgInTsCf/0yh0UhGUVHdBLQhW7duZcaMGaxevRqt1rytBkXlZBaulfTt25e//e1vPPXUU+zatQtPT0+lIwnhcLKKi6s+4KWXoLQUVCp48UXw8al4np8TlciJEycYNWoUq1atkvs9a5EUqBVFRUVx/PhxRo0axbp162SVDyFqmY+ra9UHbNhg2fMcyOXLlxk4cCAzZsygR48eSsdxKHIJ14pUKhVz5syhuLiYV199Vek4QjgcP60WrZn3MOrUaqc5+ywtLWXYsGEMGjSI8ePHKx3H4UiBWln9+vVZvXo13377LQsWLFA6jhAOJ/LGZVlrHW+vTCYTkydPxt3dnVmzZikdxyHJJdw64OXlRUJCAo8//jitWrWid+/eSkcSwmHEBASw+MKFym9luYObWs1Mf+fYezY2NpakpCR27twpw0dWImegdSQwMJDPP/+c0aNHk5qaqnQcIRyGm0ZDil6PTl3125mbWs0RvR43jeOfNyQkJPDhhx+yfv16PDw8lI7jsKRA61CPHj2YNWsW/fr1IyMjQ+k4QjgMf52OnLAwopo3v6tIdWo1Uc2bkx0Whr9Op1DCurNr1y7GjRvH119/TcuWLZWO49Ac/6OYjXn++efJyckhIiKCHTt24O3trXQkIRyCm0bD7MBAZgcGklFURFZxMT6urk4zYQjg0KFDPP300/zvf/+TTS3qgBSoAl599VUuX75M//792bJli9wjKkQt89Nqnao4AU6ePEn//v2ZN28e/fr1UzqOU5BLuAqZMWMGjz76KEOGDKHICVdFEULUnvPnz9O3b1/eeecdhg8frnQcpyEFqhCVSsW8efNo2rQpzz77LAYn3yFCCGGZS5cu0bdvXyZNmkRkZKTScZyKFKiC6tWrR3x8PCUlJYwfP162QBNCmCU/P58nn3ySp556itdff13pOE5HClRhLi4ufPnll/z666+8/PLLmKpaGFsIIW4oKChgyJAhdOzYkZkzZyodxylJgdoANzc3EhISSEpKIioqSkpUCFGl69evM3DgQJo3b878+fNRmbmcoagdUqA2olGjRmzatInk5GReeukluZwrhKjUzfJs2bIly5Ytk1WGFCQFakMaNmzIpk2bOHDgAJMnT5YSFULc5uaYp7+/P0uWLJHyVJgUqI3x9PRk48aNHD16lBdeeEFm5wohAMjNzaVv374EBwezePFiKU8bIAVqgzw8PPjuu+/IzMzkj3/8I6WlpUpHEkIoKCcnh169etGlSxcWLFiAupp1f0XdkP8LNsrd3Z1169ZRWFjIM888I4stCOGksrKy6N69O/379yc2NlYmDNkQKVAbptVq+eqrr3B3dyciIoLLly8rHUkIUYeOHTtGWFgYY8aMYcaMGVKeNkYK1MbVr1+fFStW0LVrV8LDwzl9+rTSkYQQdWDHjh306NGDd955h+joaKXjiErIYvJ2QK1W88EHH9CiRQvCw8NZv349ISEhSscSQljJ6tWrmTJlCitWrCAiIkLpOOIepEDtSFRUFH5+fvTr14/4+Hj69++vdCQhRC2LjY3lo48+YvPmzTzyyCNKxxFVkEu4dmbo0KGsXbuWcePGsWTJEqXjCCFqSVlZGdOmTWPJkiXs3r1bytMOyBmoHQoLC2P79u3079+fs2fP8s9//lMmFwhhxwoLCxkzZgyXLl1i586dNGrUSOlIogbkDNROBQcH8/PPP7Nx40aGDx9Ofn6+0pGEEBY4c+YM4eHhaLVavv/+eylPOyIFaseaNWvG9u3badSoEV27duXkyZNKRxJCmGHr1q106dKF5557jk8//RRXV1elIwkzSIHaOVdXVxYtWkRUVBTh4eF8++23SkcSQlTDZDLx0UcfMXr0aD777DNeeeUVGYaxQzIG6gBUKhUTJkygQ4cOjBgxgkmTJjF9+nRZ7ksIG1RQUEBkZCTHjh1jz549PPjgg0pHEhaSd1gHEhYWRlJSEhs2bGDYsGEyLiqEjUlPTyc8PBy1Ws2uXbukPO2cFKiD8fX1Zdu2bTRt2pQuXbpw5MgRpSMJIYDvvvuOxx57jHHjxhEfH49Op1M6krhPUqAOyNXVlY8//pjXX3+dnj17Mn/+fEwmk9KxhHBKRUVFTJs2jQkTJvDFF18QFRUl450OQgrUgT3//PPs2rWLpUuXMmTIEHJycpSOJIRTOXbsGF27duXcuXMcPHiQbt26KR1J1CIpUAcXFBTE7t27CQoKomPHjmzdulXpSEI4PJPJxMKFC3n88ceZMmUKX375JX/4wx+UjiVqmczCdQIuLi588MEHREREMGbMGMaMGcO//vUv6tevr3Q0IRzO5cuXiYyM5NSpU+zYsYM2bdooHUlYiZyBOpG+ffty4MABjhw5Qnh4OCdOnFA6khAOZdu2bXTs2JGWLVuSmJgo5engpECdTNOmTVm/fj1jx44lLCyM999/H4PBoHQsIeza1atXmThxImPGjGHBggXExsai1WqVjiWsTArUCalUKl566SWSkpLYtGkTXbt25dChQ0rHEsIubdiwgfbt22M0GklJSZFtBp2IFKgTCwgIYPPmzUyePJmIiAj+/ve/U1xcrHQsIexCdnY2f/zjH4mKiuKTTz5h4cKFNGzYUOlYog5JgTo5lUrFCy+8wMGDB/nll1/o1KkTP//8s9KxhLBZJpOJzz//nA4dOvDAAw9w+PBhevXqpXQsoQCZhSuA8hWMvvnmG7788kuGDh3KiBEj+Ne//oWnp6fS0YSwGWfOnGHq1KmcOnWKtWvX0qVLF6UjCQXJGaiooFKpGD58OEeOHCE/P5/WrVuzbNkyjEaj0tGEUFRBQQHvvPMOISEhPProo+zfv1/KU0iBirs1btyYpUuXsnbtWhYuXEjXrl1JTExUOpYQdc5kMrF69Wratm1LSkoK+/fv5+2335Z9OwUgl3BFFfR6Pbt27eJ///sfzzzzDH369CEmJgYfHx+lowlhdYcPH+bll1/m8uXLLF++nB49eigdSdgYOQMVVVKr1YwdO5bjx4/j4+NDhw4dmDVrlszWFQ7r0qVLTJkyhYiICEaMGMG+ffukPEWlpEBFjXh4eBATE0NiYiK7du2ibdu2rFixgrKyMqWjCVErCgoKeP/992nTpg1qtZpjx44xadIkNBq5UCcqJwUqzNKqVSvWrVvHkiVLmD9/Po888gjffPONbJcm7FZxcTFz586lVatW7N27l+3btxMXFyeLv4tqSYEKi/To0YOdO3cya9Ys3n33XUJDQ9m4caMUqbAbBoOBpUuXEhwczPfff8+GDRv44osvZP1aUWNSoMJiKpWKAQMGsG/fPl5//XWmTZtG9+7d2bFjh9LRhLgno9HI559/Trt27fj000/57LPPSEhIoFOnTkpHE3ZGClTcN7VazfDhw/nll1948cUXGTt2LP369WPbtm1yRipshsFg4PPPP6dTp07ExsYyb948tm7dSlhYmNLRhJ2SAhW1RqPR8Kc//YnU1FSGDx/OpEmTCA0NZfXq1TLZSCjm+vXrxMXFERgYyPz585kxYwaJiYn06dMHlUqldDxhx6RARa1zcXFh/PjxHD16lLfeeovY2FiCg4P573//S2FhodLxhJPIzs7m7bffxt/fnx9//JGVK1fy008/MXDgQClOUSukQIXVqNVqBg8ezO7du/nkk0/4/vvveeihh3j33Xe5dOmS0vGEgzp16hSTJ08mKCiICxcusHPnTr766iu6du2qdDThYKRARZ0IDw9n7dq1/Pjjj5w9e5ZWrVrx4osvkpSUJOOk4r6VlZWxYcMGBg0aRJcuXfDy8uLYsWMsWLCAoKAgpeMJB6Uy582rc+fOpr1791oxjnAWFy9eZNmyZSxYsICGDRsyceJERo8ejYeHh9LRhB3JzMxk6dKlLFq0iAceeIAJEyYwcuRI3N3dlY4mHIRKpdpnMpk6V/Z7cgYqFNG0aVP+9re/8euvvzJr1iw2bdrEgw8+yIQJEzhw4IDS8YQNMxqNbNy4kaFDh9K+fXsyMjJYs2YNe/bs4YUXXpDyFHVG1qgSilKr1fTt25e+fftWnE0MGTKEZs2a8cILLzBs2DC8vb2VjilsQFpaGitXrmTp0qUVVy0++eQTuWohFCOXcIXNKSsrY+PGjcTHx/Pdd98RHh7O6NGjGTx4sLxZOpmsrCy++OILVq5cSVpaGsOGDeP5559Hr9fLTFpRJ6q6hCsFKmzatWvXWLt2LStXrmTHjh3069eP0aNH079/f9mT0UFduXKFr7/+mpUrV7Jv3z6eeuopRo8eTe/evalfv77S8YSTkQIVDuHSpUt89dVXfPbZZxw+fJjBgwczePBg+vTpQ4MGDZSOJ+7DxYsX+fbbb1mzZg3btm2jd+/ejB49mgEDBqDT6ZSOJ5yYFKhwOBkZGXz11VckJCSQmJhIWFgYAwcOZODAgfj7+ysdT1TDZDJx6NAhEhISSEhI4Pjx4/Tp04ennnqKIUOG0LBhQ6UjCgFIgQoHl5eXx+bNm0lISODbb7/F29ubgQMHMmDAAMLCwmQ/RxtRUFDAli1bSEhIYMOGDWi12ooPPd26dcPFxUXpiELcRQpUOA2j0UhycjIbNmwgISGBU6dOER4eTvfu3enWrRudO3eWcbQ6cu3aNX7++We2b9/O9u3bOXDgAHq9vqI0g4KCZCKQsHlSoMJpZWdns2PHjoo38VOnTtG1a9eKQg0NDUWr1Sod0yFcvXqVXbt2VfxdHzlyhE6dOlX8XYeFhclYtbA7UqBC3HDlyhV27tzJ9u3b+emnnzh69CiPPPIInTp1IiQkhJCQENq2bSuXE6tx/fp1Dh06xP79+ysep06dQq/X0717d7p3706XLl1kApCwe1KgQtxDfn4++/btqyiBAwcOkJ6eTtu2bQkJCako1nbt2jnt2dOlS5c4fPhwxd/P/v37OX36NO3atav4++nUqROdOnWSDx7C4UiBCmGG69evVxTGzUdqaiqNGjUiMDCQwMBAgoKCKn7+8MMP2/2ZVl5eHidPnuTEiROcPHnytp+XlZXRoUOHu87SZSxZOAMpUCHuk9Fo5Pz585UWzOnTp2nWrBkPPvggPj4+PPDAA/j4+FQ8bv66cePGqNV1u/y0wWDg4sWLZGVlceHCBbKysioeN3+dlpbGtWvXKj4Q3PkBoUmTJjLZRzgtKVAhrMhgMHD27FnOnTt3W0HdWVr5+fn84Q9/oEGDBnh4eFT6Y4MGDXBzc6NevXqo1eqKh8lkwmg0VjzKysq4fv06165dIz8//54/5ubm0rhx47vK/NZfP/TQQ/j6+kpJClGJqgpUbpAT4j5pNBoCAgIICAio8rji4mIuXbrEtWvXKh53lt61a9e4fv36XWV5a5mqVCrq1auHh4cHvr6+FcVbWRk3btxY7oMVwkrkX5YQdcTV1RVfX1+lYwghaonsByqEEEJYQApUCCGEsIAUqBBCCGEBKVAhhBDCAlKgQgghhAWkQIUQQggLSIEKIYQQFpACFUIIISwgBSqEEEJYQApUCCGEsIAUqBBCCGEBKVAhhBDCAlKgQgghhAWkQIUQQggLSIEKIYQQFpACFUIIISwgBSqEEEJYQApUCCGEsIAUqBBCCGEBKVAhhBDCAlKgQgghhAWkQIUQQggLSIEKIYQQFpACFUIIISwgBSqEEEJYQApUCCGEsIAUqBBCCGEBKVAhhBDCAlKgQgghhAWkQIUQQggLSIEKIYQQFpACFUIIISwgBSqEEEJYQApUCCGEsIDKZDLV/GCVKhs4Y704QgghhE150GQyNansN8wqUCGEEEKUk0u4QgghhAWkQIUQQggLSIEKIYQQFpACFUIIISwgBSqEEEJYQApUCCGEsIAUqBBCCGEBKVAhhBDCAlKgQgghhAX+Pzn9PHR3GSL/AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "colors = [\"g\", \"c\", \"m\"]\n", "circle = visualization.PoincareDisk(point_type=\"ball\")\n", "fig2, ax2 = plt.subplots(figsize=(8, 8))\n", "circle.set_ax(ax2)\n", "circle.draw(ax=ax2)\n", "ax2.axes.xaxis.set_visible(False)\n", "ax2.axes.yaxis.set_visible(False)\n", "group_1_predicted = mpatches.Patch(color=colors[0], label=\"Predicted Group 1\")\n", "group_2_predicted = mpatches.Patch(color=colors[1], label=\"Predicted Group 2\")\n", "group_centroids = mpatches.Patch(color=colors[2], label=\"Cluster centroids\")\n", "\n", "for i in range(n_clusters):\n", " for i_embedding, embedding in enumerate(embeddings):\n", " x = embedding[0]\n", " y = embedding[1]\n", " pt_id = i_embedding\n", " if labels[i_embedding] == 0:\n", " color = colors[0]\n", " else:\n", " color = colors[1]\n", " plt.scatter(x, y, c=color, s=150)\n", " ax2.annotate(pt_id, (x, y))\n", "\n", "for i_centroid, centroid in enumerate(centroids):\n", " x = centroid[0]\n", " y = centroid[1]\n", " plt.scatter(\n", " x,\n", " y,\n", " c=colors[2],\n", " marker=\"*\",\n", " s=150,\n", " )\n", "\n", "plt.title(\"K-means applied to Karate club embedding\")\n", "plt.legend(handles=[group_1_predicted, group_2_predicted, group_centroids])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "By comparing the $K$-means labels and the true labels, notice how $K$-means \n", "accurately finds the two groups of nodes (not perfectly, e.g., nodes 2 and 8). We therefore achieved good performances in\n", "predicting the belonging of each member of the Karate club to one of the two groups." ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## References" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ ".. [ABY2013] Arnaudon, Marc, Frédéric Barbaresco, and Le Yang. \"Riemannian medians and means with applications to radar signal processing.\" IEEE Journal of Selected Topics in Signal Processing 7.4 (2013): 595-604.\n", "\n", ".. [GBH2018] Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. \"Hyperbolic neural networks.\" Advances in neural information processing systems. 2018.\n", "\n", ".. [M2013] Mikolov, Tomas, et al. \"Distributed representations of words and phrases and their compositionality.\" Advances in neural information processing systems. 2013.\n", "\n", ".. [ND2017] Nickel, Maximillian, and Douwe Kiela. \"Poincaré embeddings for learning hierarchical representations.\" Advances in neural information processing systems. 2017.\n" ] } ], "metadata": { "backends": [ "numpy", "autograd", "pytorch" ], "celltoolbar": "Edit Metadata", "cite2c": { "citations": { "7875465/9CMMEH2F": { "author": [ { "family": "Mikolov", "given": "Tomas" }, { "family": "Sutskever", "given": "Ilya" }, { "family": "Chen", "given": "Kai" }, { "family": "Corrado", "given": "Greg S" }, { "family": "Dean", "given": "Jeff" } ], "container-title": "Advances in Neural Information Processing Systems 26 (NIPS)", "id": "7875465/9CMMEH2F", "issued": { "year": 2013 }, "page": "3111–3119", "page-first": "3111", "publisher": "Curran Associates, Inc.", "title": "Distributed Representations of Words and Phrases and their Compositionality", "type": "chapter" }, "7875465/9DR2QEF9": { "author": [ { "family": "Ganea", "given": "Octavian" }, { "family": "Becigneul", "given": "Gary" }, { "family": "Hofmann", "given": "Thomas" } ], "container-title": "Advances in Neural Information Processing Systems 31 (NIPS)", "id": "7875465/9DR2QEF9", "issued": { "year": 2018 }, "page": "5345–5355", "page-first": "5345", "publisher": "Curran Associates, Inc.", "title": "Hyperbolic Neural Networks", "type": "paper-conference" }, "7875465/KRA9K53S": { "author": [ { "family": "Nickel", "given": "Maximillian" }, { "family": "Kiela", "given": "Douwe" } ], "container-title": "Advances in Neural Information Processing Systems 30 (NIPS)", "id": "7875465/KRA9K53S", "issued": { "year": 2017 }, "page": "6338–6347", "page-first": "6338", "publisher": "Curran Associates, Inc.", "title": "Poincaré Embeddings for Learning Hierarchical Representations", "type": "chapter" }, "7875465/TLSACAEQ": { "author": [ { "family": "Arnaudon", "given": "Marc" }, { "family": "Barbaresco", "given": "Frédéric" }, { "family": "Yang", "given": "Le" } ], "container-title": "Journal of Selected Topics in Signal Processing", "id": "7875465/TLSACAEQ", "issue": "4", "issued": { "year": 2013 }, "page": "595–604", "page-first": "595", "title": "Riemannian Medians and Means With Applications to Radar Signal Processing", "type": "article-journal", "volume": "7" } } }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.12" } }, "nbformat": 4, "nbformat_minor": 4 }