{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Knowledge graph link prediction with ComplEx\n" ] }, { "cell_type": "markdown", "metadata": { "nbsphinx": "hidden", "tags": [ "CloudRunner" ] }, "source": [ "
Run the latest release of this notebook:
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "This notebook reproduces the experiments done in the paper that introduced the ComplEx algorithm: Complex Embeddings for Simple Link Prediction, Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier and Guillaume Bouchard, ICML 2016. http://jmlr.org/proceedings/papers/v48/trouillon16.pdf\n", "\n", "In table 2, the paper reports five metrics measured on the WN18 and FB15K datasets: \"raw\" MRR (mean reciprocal rank), \"filtered\" MRR and filtered Hits at {1, 3, 10}. This notebook measures all of these, as well as raw Hits at {1, 3, 10}." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden", "tags": [ "CloudRunner" ] }, "outputs": [], "source": [ "# install StellarGraph if running on Google Colab\n", "import sys\n", "if 'google.colab' in sys.modules:\n", " %pip install -q stellargraph[demos]==1.2.1" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "nbsphinx": "hidden", "tags": [ "VersionCheck" ] }, "outputs": [], "source": [ "# verify that we're using the correct version of StellarGraph for this notebook\n", "import stellargraph as sg\n", "\n", "try:\n", " sg.utils.validate_notebook_version(\"1.2.1\")\n", "except AttributeError:\n", " raise ValueError(\n", " f\"This notebook requires StellarGraph version 1.2.1, but a different version {sg.__version__} is installed. Please see .\"\n", " ) from None" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from stellargraph import datasets, utils\n", "from tensorflow.keras import callbacks, optimizers, losses, metrics, regularizers, Model\n", "import numpy as np\n", "import pandas as pd\n", "\n", "from stellargraph.mapper import KGTripleGenerator\n", "from stellargraph.layer import ComplEx\n", "\n", "from IPython.display import HTML" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialisation\n", "\n", "We need to set up our model parameters, like the number of epochs to train for, and the dimension of the embedding vectors we compute for each node and for each edge type.\n", "\n", "The evaluation is performed in three steps:\n", "\n", "1. Load the data\n", "2. Train a model\n", "3. Evaluate the model\n", "\n", "The paper says that it used:\n", "- the AdaGrad optimiser for 1000 epochs with an early stopping criterion evaluated every 50 epochs, but we've found using the Adam optimiser allows for much fewer epochs\n", "- an embedding dimension of 150 or 200, since they had close results\n", "- 10 negative samples (corrupted edges) per positive edge, which gives noticeably improved performance on FB15k compared to using 1, and but not for WN18 (the paper evaluated 1, 2, 5 and 10 negative samples)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "tags": [ "parameters" ] }, "outputs": [], "source": [ "epochs = 50\n", "embedding_dimension = 200\n", "negative_samples = 10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## WN18\n", "\n", "The paper uses the WN18 and FB15k datasets for validation. These datasets are not good for evaluating algorithms because they contain \"inverse relations\", where `(s, r1, o)` implies `(o, r2, s)` for a pair of relation types `r1` and `r2` (for instance, `_hyponym` (\"is more specific than\") and `_hypernym` (\"is more general than\") in WN18), however, they work fine to demonstrate StellarGraph's functionality, and are appropriate to compare against the published results.\n", "\n", "### Load the data\n", "\n", "The dataset comes with a defined train, test and validation split, each consisting of subject, relation, object triples. We can load a `StellarGraph` object with all of the triples, as well as the individual splits as Pandas DataFrames, using the `load` method of the `WN18` dataset." ] }, { "cell_type": "markdown", "metadata": { "tags": [ "DataLoadingLinks" ] }, "source": [ "(See [the \"Loading from Pandas\" demo](../basics/loading-pandas.ipynb) for details on how data can be loaded.)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "tags": [ "DataLoading" ] }, "outputs": [ { "data": { "text/html": [ "The WN18 dataset consists of triplets from WordNet 3.0 (http://wordnet.princeton.edu). There are 40,943 synsets and 18 relation types among them. The training set contains 141442 triplets, the validation set 5000 and the test set 5000. Antoine Bordes, Xavier Glorot, Jason Weston and Yoshua Bengio “A Semantic Matching Energy Function for Learning with Multi-relational Data” (2014).\n", "\n", "Note: this dataset contains many inverse relations, and so should only be used to compare against published results. Prefer WN18RR. See: Kristina Toutanova and Danqi Chen “Observed versus latent features for knowledge base and text inference” (2015), and Dettmers, Tim, Pasquale Minervini, Pontus Stenetorp and Sebastian Riedel “Convolutional 2D Knowledge Graph Embeddings” (2017)." ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "wn18 = datasets.WN18()\n", "display(HTML(wn18.description))\n", "wn18_graph, wn18_train, wn18_test, wn18_valid = wn18.load()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "StellarDiGraph: Directed multigraph\n", " Nodes: 40943, Edges: 151442\n", "\n", " Node types:\n", " default: [40943]\n", " Features: none\n", " Edge types: default-_also_see->default, default-_derivationally_related_form->default, default-_has_part->default, default-_hypernym->default, default-_hyponym->default, ... (13 more)\n", "\n", " Edge types:\n", " default-_hyponym->default: [37221]\n", " default-_hypernym->default: [37221]\n", " default-_derivationally_related_form->default: [31867]\n", " default-_member_meronym->default: [7928]\n", " default-_member_holonym->default: [7928]\n", " default-_part_of->default: [5148]\n", " default-_has_part->default: [5142]\n", " default-_member_of_domain_topic->default: [3341]\n", " default-_synset_domain_topic_of->default: [3335]\n", " default-_instance_hyponym->default: [3150]\n", " default-_instance_hypernym->default: [3150]\n", " default-_also_see->default: [1396]\n", " default-_verb_group->default: [1220]\n", " default-_member_of_domain_region->default: [983]\n", " default-_synset_domain_region_of->default: [982]\n", " default-_member_of_domain_usage->default: [675]\n", " default-_synset_domain_usage_of->default: [669]\n", " default-_similar_to->default: [86]\n" ] } ], "source": [ "print(wn18_graph.info())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train a model\n", "\n", "The ComplEx algorithm consists of some embedding layers and a scoring layer, but the `ComplEx` object means these details are invisible to us. The `ComplEx` model consumes \"knowledge-graph triples\", which can be produced in the appropriate format using `KGTripleGenerator`." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "wn18_gen = KGTripleGenerator(\n", " wn18_graph, batch_size=len(wn18_train) // 100 # ~100 batches per epoch\n", ")\n", "\n", "wn18_complex = ComplEx(\n", " wn18_gen,\n", " embedding_dimension=embedding_dimension,\n", " embeddings_regularizer=regularizers.l2(1e-7),\n", ")\n", "\n", "wn18_inp, wn18_out = wn18_complex.in_out_tensors()\n", "\n", "wn18_model = Model(inputs=wn18_inp, outputs=wn18_out)\n", "\n", "wn18_model.compile(\n", " optimizer=optimizers.Adam(lr=0.001),\n", " loss=losses.BinaryCrossentropy(from_logits=True),\n", " metrics=[metrics.BinaryAccuracy(threshold=0.0)],\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Inputs for training are produced by calling the `KGTripleGenerator.flow` method, this takes a dataframe with `source`, `label` and `target` columns, where each row is a true edge in the knowledge graph. The `negative_samples` parameter controls how many random edges are created for each positive edge to use as negative examples for training." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "wn18_train_gen = wn18_gen.flow(\n", " wn18_train, negative_samples=negative_samples, shuffle=True\n", ")\n", "wn18_valid_gen = wn18_gen.flow(wn18_valid, negative_samples=negative_samples)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train for 101 steps, validate for 4 steps\n", "Epoch 1/50\n", "101/101 [==============================] - 24s 240ms/step - loss: 0.6971 - binary_accuracy: 0.5005 - val_loss: 0.6970 - val_binary_accuracy: 0.5013\n", "Epoch 2/50\n", "101/101 [==============================] - 23s 230ms/step - loss: 0.6967 - binary_accuracy: 0.5067 - val_loss: 0.6966 - val_binary_accuracy: 0.4989\n", "Epoch 3/50\n", "101/101 [==============================] - 23s 230ms/step - loss: 0.6963 - binary_accuracy: 0.5180 - val_loss: 0.6961 - val_binary_accuracy: 0.5077\n", "Epoch 4/50\n", "101/101 [==============================] - 23s 231ms/step - loss: 0.6958 - binary_accuracy: 0.5308 - val_loss: 0.6957 - val_binary_accuracy: 0.5162\n", "Epoch 5/50\n", "101/101 [==============================] - 24s 242ms/step - loss: 0.6953 - binary_accuracy: 0.5430 - val_loss: 0.6953 - val_binary_accuracy: 0.5318\n", "Epoch 6/50\n", "101/101 [==============================] - 24s 236ms/step - loss: 0.6942 - binary_accuracy: 0.5969 - val_loss: 0.6930 - val_binary_accuracy: 0.7030\n", "Epoch 7/50\n", "101/101 [==============================] - 24s 237ms/step - loss: 0.6784 - binary_accuracy: 0.8850 - val_loss: 0.6488 - val_binary_accuracy: 0.9090\n", "Epoch 8/50\n", "101/101 [==============================] - 24s 242ms/step - loss: 0.5640 - binary_accuracy: 0.9092 - val_loss: 0.4654 - val_binary_accuracy: 0.9091\n", "Epoch 9/50\n", "101/101 [==============================] - 24s 239ms/step - loss: 0.3702 - binary_accuracy: 0.9107 - val_loss: 0.3183 - val_binary_accuracy: 0.9108\n", "Epoch 10/50\n", "101/101 [==============================] - 24s 241ms/step - loss: 0.2636 - binary_accuracy: 0.9190 - val_loss: 0.2561 - val_binary_accuracy: 0.9176\n", "Epoch 11/50\n", "101/101 [==============================] - 24s 237ms/step - loss: 0.1915 - binary_accuracy: 0.9382 - val_loss: 0.2023 - val_binary_accuracy: 0.9316\n", "Epoch 12/50\n", "101/101 [==============================] - 24s 236ms/step - loss: 0.1316 - binary_accuracy: 0.9671 - val_loss: 0.1555 - val_binary_accuracy: 0.9520\n", "Epoch 13/50\n", "101/101 [==============================] - 24s 234ms/step - loss: 0.0928 - binary_accuracy: 0.9854 - val_loss: 0.1226 - val_binary_accuracy: 0.9695\n", "Epoch 14/50\n", "101/101 [==============================] - 24s 235ms/step - loss: 0.0711 - binary_accuracy: 0.9939 - val_loss: 0.1019 - val_binary_accuracy: 0.9817\n", "Epoch 15/50\n", "101/101 [==============================] - 24s 235ms/step - loss: 0.0597 - binary_accuracy: 0.9971 - val_loss: 0.0897 - val_binary_accuracy: 0.9882\n", "Epoch 16/50\n", "101/101 [==============================] - 23s 229ms/step - loss: 0.0535 - binary_accuracy: 0.9982 - val_loss: 0.0825 - val_binary_accuracy: 0.9917\n", "Epoch 17/50\n", "101/101 [==============================] - 24s 236ms/step - loss: 0.0496 - binary_accuracy: 0.9988 - val_loss: 0.0784 - val_binary_accuracy: 0.9927\n", "Epoch 18/50\n", "101/101 [==============================] - 24s 240ms/step - loss: 0.0472 - binary_accuracy: 0.9991 - val_loss: 0.0754 - val_binary_accuracy: 0.9936\n", "Epoch 19/50\n", "101/101 [==============================] - 24s 236ms/step - loss: 0.0454 - binary_accuracy: 0.9993 - val_loss: 0.0741 - val_binary_accuracy: 0.9936\n", "Epoch 20/50\n", "101/101 [==============================] - 24s 235ms/step - loss: 0.0444 - binary_accuracy: 0.9993 - val_loss: 0.0728 - val_binary_accuracy: 0.9938\n", "Epoch 21/50\n", "101/101 [==============================] - 24s 235ms/step - loss: 0.0434 - binary_accuracy: 0.9994 - val_loss: 0.0717 - val_binary_accuracy: 0.9939\n", "Epoch 22/50\n", "101/101 [==============================] - 24s 237ms/step - loss: 0.0427 - binary_accuracy: 0.9994 - val_loss: 0.0702 - val_binary_accuracy: 0.9941\n", "Epoch 23/50\n", "101/101 [==============================] - 24s 238ms/step - loss: 0.0420 - binary_accuracy: 0.9994 - val_loss: 0.0697 - val_binary_accuracy: 0.9940\n", "Epoch 24/50\n", "101/101 [==============================] - 24s 237ms/step - loss: 0.0414 - binary_accuracy: 0.9994 - val_loss: 0.0690 - val_binary_accuracy: 0.9941\n", "Epoch 25/50\n", "101/101 [==============================] - 24s 237ms/step - loss: 0.0408 - binary_accuracy: 0.9994 - val_loss: 0.0693 - val_binary_accuracy: 0.9938\n", "Epoch 26/50\n", "101/101 [==============================] - 24s 238ms/step - loss: 0.0404 - binary_accuracy: 0.9994 - val_loss: 0.0684 - val_binary_accuracy: 0.9938\n", "Epoch 27/50\n", "101/101 [==============================] - 24s 234ms/step - loss: 0.0398 - binary_accuracy: 0.9995 - val_loss: 0.0680 - val_binary_accuracy: 0.9938\n", "Epoch 28/50\n", "101/101 [==============================] - 24s 237ms/step - loss: 0.0394 - binary_accuracy: 0.9994 - val_loss: 0.0667 - val_binary_accuracy: 0.9941\n", "Epoch 29/50\n", "101/101 [==============================] - 24s 236ms/step - loss: 0.0390 - binary_accuracy: 0.9994 - val_loss: 0.0666 - val_binary_accuracy: 0.9940\n", "Epoch 30/50\n", "101/101 [==============================] - 24s 239ms/step - loss: 0.0385 - binary_accuracy: 0.9994 - val_loss: 0.0661 - val_binary_accuracy: 0.9940\n", "Epoch 31/50\n", "101/101 [==============================] - 24s 238ms/step - loss: 0.0381 - binary_accuracy: 0.9994 - val_loss: 0.0659 - val_binary_accuracy: 0.9941\n", "Epoch 32/50\n", "101/101 [==============================] - 24s 235ms/step - loss: 0.0377 - binary_accuracy: 0.9994 - val_loss: 0.0657 - val_binary_accuracy: 0.9939\n", "Epoch 33/50\n", "101/101 [==============================] - 24s 235ms/step - loss: 0.0373 - binary_accuracy: 0.9994 - val_loss: 0.0642 - val_binary_accuracy: 0.9941\n", "Epoch 34/50\n", "101/101 [==============================] - 24s 238ms/step - loss: 0.0369 - binary_accuracy: 0.9994 - val_loss: 0.0640 - val_binary_accuracy: 0.9940\n", "Epoch 35/50\n", "101/101 [==============================] - 24s 234ms/step - loss: 0.0365 - binary_accuracy: 0.9994 - val_loss: 0.0638 - val_binary_accuracy: 0.9940\n", "Epoch 36/50\n", "101/101 [==============================] - 24s 239ms/step - loss: 0.0362 - binary_accuracy: 0.9994 - val_loss: 0.0639 - val_binary_accuracy: 0.9938\n", "Epoch 37/50\n", "101/101 [==============================] - 25s 246ms/step - loss: 0.0359 - binary_accuracy: 0.9994 - val_loss: 0.0629 - val_binary_accuracy: 0.9941\n", "Epoch 38/50\n", "101/101 [==============================] - 24s 237ms/step - loss: 0.0355 - binary_accuracy: 0.9994 - val_loss: 0.0622 - val_binary_accuracy: 0.9940\n", "Epoch 39/50\n", "101/101 [==============================] - 24s 233ms/step - loss: 0.0351 - binary_accuracy: 0.9994 - val_loss: 0.0624 - val_binary_accuracy: 0.9939\n", "Epoch 40/50\n", "101/101 [==============================] - 24s 234ms/step - loss: 0.0347 - binary_accuracy: 0.9994 - val_loss: 0.0610 - val_binary_accuracy: 0.9941\n", "Epoch 41/50\n", "101/101 [==============================] - 23s 231ms/step - loss: 0.0344 - binary_accuracy: 0.9994 - val_loss: 0.0617 - val_binary_accuracy: 0.9941\n", "Epoch 42/50\n", "101/101 [==============================] - 23s 229ms/step - loss: 0.0340 - binary_accuracy: 0.9994 - val_loss: 0.0608 - val_binary_accuracy: 0.9941\n", "Epoch 43/50\n", "101/101 [==============================] - 23s 232ms/step - loss: 0.0335 - binary_accuracy: 0.9994 - val_loss: 0.0599 - val_binary_accuracy: 0.9943\n", "Epoch 44/50\n", "101/101 [==============================] - 23s 231ms/step - loss: 0.0331 - binary_accuracy: 0.9995 - val_loss: 0.0610 - val_binary_accuracy: 0.9938\n", "Epoch 45/50\n", "101/101 [==============================] - 23s 232ms/step - loss: 0.0329 - binary_accuracy: 0.9994 - val_loss: 0.0601 - val_binary_accuracy: 0.9940\n", "Epoch 46/50\n", "101/101 [==============================] - 23s 232ms/step - loss: 0.0326 - binary_accuracy: 0.9994 - val_loss: 0.0596 - val_binary_accuracy: 0.9940\n", "Epoch 47/50\n", "101/101 [==============================] - 23s 230ms/step - loss: 0.0324 - binary_accuracy: 0.9994 - val_loss: 0.0596 - val_binary_accuracy: 0.9939\n", "Epoch 48/50\n", "101/101 [==============================] - 24s 238ms/step - loss: 0.0320 - binary_accuracy: 0.9994 - val_loss: 0.0597 - val_binary_accuracy: 0.9939\n", "Epoch 49/50\n", "101/101 [==============================] - 24s 241ms/step - loss: 0.0317 - binary_accuracy: 0.9994 - val_loss: 0.0595 - val_binary_accuracy: 0.9938\n", "Epoch 50/50\n", "101/101 [==============================] - 24s 241ms/step - loss: 0.0315 - binary_accuracy: 0.9994 - val_loss: 0.0580 - val_binary_accuracy: 0.9942\n" ] } ], "source": [ "wn18_es = callbacks.EarlyStopping(monitor=\"val_loss\", patience=10)\n", "wn18_history = wn18_model.fit(\n", " wn18_train_gen, validation_data=wn18_valid_gen, epochs=epochs, callbacks=[wn18_es]\n", ")" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAI4CAYAAACV/7uiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdeXxU5dn/8c81k4QsZMOENayKKAq4UNy31vq4ULVPq6LWqn0qPK1b1bZqrVRtbe1Ta0uttkVtXepStbZipT+r1tZK1YqioCAKCJKwJQgkBLLMzP3740zCZBggA5OZzJzv+/Wa15xz5p4zVw5hrtz3uRdzziEiIiLZJZDpAERERCR5SuAiIiJZSAlcREQkCymBi4iIZCElcBERkSyUl+kA9lRVVZUbMWJEpsMQERHpEW+++WaDc646/njWJ/ARI0Ywd+7cTIchIiLSI8xsRaLjakIXERHJQkrgIiIiWUgJXEREJAspgYuIiGQhJXAREZEspAQuIiKShdKWwM3st2a2zsze3cHrZma/MLMlZjbfzA5JV2wiIiLZJp018PuBk3fy+inA6OhjKvCrNMQkIiKSldKWwJ1zLwOf7KTIGcCDzvMaUGFmg9ITnYiISHbpTTOxDQFWxuzXRo+tzkw4kktC4QhNLSEaW9q9563tbG4N0RKK0NIeprU9TEu7t90S8rZbQ2HCEUco7AhFoo9whFDEEY442sMRnINwxBFxHQ+854i37XAAOLctlo5t17nviLetzPavJS4Xe2zb3nbvdjvdTRhLBzPznjsPdD2Ri77fkTh+i74hehos0TkTSCbG+Fh3eM64c+z6jN3X8ck7vV7b7+5SKmPsjtj4Yq9nx7XrjCf2dzuuTNzLCX8vdvXPGf9Pmej3KDZG61o40WYXu/o/svPgtj+3mXHrmQdy2Ki9kjjR7ulNCbzbzGwqXjM7w4YNy3A00hu0tIdZsm4z769p4v3Vjby/pom1jS2dCXtLW7jb5woGjMK8AAV5AfKCAfIDRjBo5AcCBANGMGDkB7dtBwwCZuQFAgQC3rb38LY7WIJvlu2/hGJL2HbHEtl2jsTfVvFvj09u27++/Wfs7I8Ox7ZkbNH3xyfn+Pd1TfK71p0Y42ONj2+7c+5hQk342XEx7OiPtIRJoxsBpCLG7nAJdhxu2+/YDpJn12Nsdyy27I4S7Y4DSfx75O1vXzzhHxE7uM67+j+SMLT4P2Ridkr6pCe19qYEXgcMjdmviR7bjnNuJjATYOLEien+w1QyzDnHq0vXM2/lRhZFk/VHDc2EI96vQp+8AGMGlrJ3dV/KivIoK8yntDCfsqI877nQe+7bJ4/C/ACF+UH6RJ+L8oPkB3dwZ8k5CLVCWzO0bYb2LdFvDrftdW9j276LbHtEwtHtcNx+9ByxZbd7uJjyMefo+EwzwBI/d54/0TNgAQgEIZAHFsT7K6RjO+i97hXs+lkxh7q+FrfvHERCEG73niPt3s8Sbt+23RFHbNwW6PqzxIr/gu1yDaPPsftmMT9PzM/VsW8Wvb7hmGsd7npsV9c54fWJuR5mcZ/fEUPA23aRmOvUDuHoterY704Mndcs9loGtsXR+XsXjnuO/l5tZxfXPd6ufocBgvkQ7BN9Log+8iGvj/d751zM73fH/52Y7djf3e3+78XEmfBaWNzPsIN/JwvEPYIx58T7Nwm3QrgNQm3ec8cj1ApFlUD5zq9VCvSmBD4LuMzMHgMOAzY559R8Lp1C4QjPLljNr/6xlPfXNAEwtF8R+w0s49QDB7LfoDLGDCxlxF4lBAM7+aIJtUH9Itj4MbRsij4at223Nm57bmuOeWzewZeciEiMAQdCv5E9/jFpS+Bm9ihwPFBlZrXA94B8AOfcr4HZwKnAEmALcHG6YpPerTUU5qm36vj1P5eyYv0W9unfl5+eNYGTDhhAaWH+Lt7cBGvehTXzYfV8WPMOrHvfq9HE61MGheXbnvsOhD59oaAECvpCfvG27YISyC/0/jKHxDVPiPnrPaam1Vnzi/vLPv4v/47aQ2fZuEcgGP28HdWu3bba5w5rjWxfK+vcDkVrx65rG2Rs7adLzWcHLREAgXwI5nk1rEC+V+MKBL3tQHDb+7rUsGJq0F3Et6s6Eta4YmvwHbW6zp8tsn0NO7523qWGHtj1dU54Tej6sySq9XYcs0DMdeq4RnnR5/xuxhDfChF3XTtaWLZrjej4PbS4uHfDjn5fLeCds7O22h59bo3Zbo+79vHniKkF76jVJ+G1iNnf7ueL+73t0noTV/vveK2jBSGvT0yLQgHkdbQo9Nm9a5ektCVw59y5u3jdAZemKRzJAs2tIR79z8fc869lrG1sZXxNOb/+0qGcNHYAgR3VsCMRWPEKzP8DrHgVPllG53/Q4r1g4Hg44uvec9VoKKyAwjIvaXckEhHpOcE8oDjTUeSE3tSELgJAezjCr/+xlN/O+YgNW9o5YtRe3H7WBI7ep2rHnUvqP4D5j8H8x2HTSigohVHHwfhzYNB4L2GXDd71PTwRkSyhBC69Sigc4crH5jF7wRpO3L8/Xzt+Hw4dXpm4cPN6ePePXuKue9NrWtv703DiTTDmVCjQX/kikruUwKXXCEccVz/+DrMXrOG7p+3PV48Zlbhg4yp47juw6BnvPu2AcXDSrTDui1A6ML1Bi4hkiBK49AqRiONbT77DrHdWce3J+yVO3s55TeR//ZbX2eWw/4UJU2DguPQHLCKSYUrgknGRiOM7f1rAU2/VcfVn9+Vrx++9faHN9fCXb8D7f4Ghh8OZd8NeCcqJiPiEErhklHOO6bPe5bE3VnL5p/fhis+M3r7Qwlle8m5tgs9+H464VD3GRcT3lMAlY5xz3PKXhfz+tY+Zdtworv7svl0LbN0As78NCx6HQRPg87+B/vtnJlgRkV5GCVwywjnHj/76Pr+bs5yvHDWS607er+sQsQ9fgFmXQXM9HH89HHONN2GCiIgASuCSAc45bv/bYma+vIwLDh/OjZP375q8X/s1/L9roXo/OPdRGHxw5oIVEemllMAl7R6fu5K7XlrKuZOGcvPpB3RN3v/+JfztBthvMnzhPm+6UhER2c4Oll0S6RkNm1u59dlFHDayH7eeOa7rlKiv/MxL3mPPhLPuV/IWEdkJ1cAlrX747CK2toe59fMHdk3eL/8E/v4DOPAL8PmZ0fmSRURkR1QDl7T599IGnppXx7Rj92af/qXeQefgH7d5yXv8OUreIiLdpG9KSYvWUJjv/vldhvUr5rJP7+MddA5eutWrfR90Ppx+p8Z3i4h0kxK4pMXMfy5jWX0z91/8KQrzg17yfvFm7773IV+GyTO8dYpFRKRblMClxy1vaObOl5Zw2vhBHD+mv5e8//ZdePWXMPErcOpPlbxFRJKkb03pUc45bnz6XQqCAaZPHusdfOlWL3lPmgqn3aHkLSKyG/TNKT3q2QWr+deHDXzzpH0ZUFYIb97v3fM++AI45f8gdgy4iIh0mxK49JjGlnZueWYh44aUc8ERI+DD5+EvV8M+J8Lknyl5i4jsAd0Dlx5zx98+oH5zK/deOJHgmnfg8QthwAHeJC2a11xEZI+oBi49YkHtJh58dTlfPnw44/s2wiNnQ3E/OP8J6FOa6fBERLKeauCScuGI44Y/L2Cvvn245rgB8PvToL0FvjwLSgdmOjwRkZygBC4pd++/ljG/dhN3nX0AZX+6CDZ8BF96Cvrvl+nQRERyhhK4pNSf59Xxo7++zylj+3Pqsu/Dile8VcVGHpPp0EREcorugUvKvLhoLdc88Q6Hj+rHnQP/gr37JJx4E4z7YqZDExHJOUrgkhKvLVvP1x9+iwMGlfLAvnPI+/fPYeL/wFHfyHRoIiI5SU3ossferdvEVx+Yy+hK44mqe+jzj6fhgM9rohYRkR6kBC57ZMm6zXz5t/9hXJ91PJj/C/I/WOI1mx/1DSVvEZEepAQuu61u41a+fN/rnOD+w0/c3QRa+sAFf4JRx2c6NBGRnKcELrulYXMrF97zby5ufZBL+DMMPBjOfggqhmY6NBERX1ACl6Q1trRz+b3Pc/Pm2zjKFsAhF3r3u/MLMx2aiIhvKIFLt7WHI8yaV8fLL87i9i13MCjYCJPvhEO+nOnQRER8RwlcdmlLW4in57xD/ZwH+a+2F/hCoJaWksEEzn8OhhyS6fBERHxJCVx2aEPTFl6e/Qh9F/2BL7o3ybcwm6on4A6/hsJxX4TCskyHKCLiW0rg0smFQ2zZuJb1q5ax8pVH2XfNs5xhG2kMVLB+/68w8LivUq75zEVEegUl8FzT3gIL/wztW8A5cJGY5wjhSJi3VzTQsrGe/JZ6ClvX0ze0norwBspdIyXmKAEGuwDvlx5B29FfYcinzqBM63eLiPQqSuC5ZvGz8KdpO3w5CBwKtLp8PglU0BSsZFPBINYVjiNUXI0r6U+wdAAjD/kMBw4ZnrawRUQkOWlN4GZ2MjADL4/c65y7Le714cBvgWrgE+BLzrnadMaY9ZrWeM+X/gcKK8AC0Yfx6rINTHv4LT43oYZbzzmcQWYMymy0IiKym9K2mImZBYG7gFOAscC5ZjY2rtjtwIPOufHALcCP0hVfzmiuh0A+VO0LpQOgbzWU7EV9uITL//wR/av7c8N/f0rTnIqIZLl0rkY2CVjinFvmnGsDHgPOiCszFvh7dPulBK/Lrmyuh5LqLgk6EnFc/fjbNLW088vzDqa4QHdORESyXToT+BBgZcx+bfRYrHeA/45ufx4oNbO94k9kZlPNbK6Zza2vr++RYLNWcz2UVHU59JuXl/GvDxuY/rmx7DdQQ79ERHJBb1sP/JvAcWY2DzgOqAPC8YWcczOdcxOdcxOrq6vTHWPv1hytgUe9uWIDt/9tMaeNG8R5k4ZlMDAREUmldLal1gGxK13URI91cs6tIloDN7O+wBeccxvTFmEuaG7w7n8Dm7a0c8Wj8xhcUciPvjAO031vEZGckc4a+BvAaDMbaWYFwBRgVmwBM6sys46YrsfrkS7d5VxnE7pzjm//8R3WNrZw57mHUFaocdwiIrkkbQncORcCLgOeAxYBjzvn3jOzW8zs9Gix44HFZvYBMAC4NV3x5YS2zRDaCiXVPPTaCp57by3fPnkMBw2tyHRkIiKSYmntjuycmw3Mjjs2PWb7SeDJdMaUU5q9Dn21ob784K+LOH5MNV89elSGgxIRkZ7Q2zqxyZ5obgDgl69torIkn5+eNYFAQPe9RURykRJ4LonWwN/dVMBt/z2evfr2yXBAIiLSU5TAc0k0ga935RwwROO9RURymRJ4Lokm8E8oVa9zEZEcpwSeSzbX0xLsi8srpDA/mOloRESkBymB55LmejYHK1T7FhHxAa1qkUua69kUqKCsj/5ZRURynWrguaS5gU+sXDVwEREfUALPJc31rHdllBUpgYuI5LqkEriZ3WNmE3sqGNkD4RBsWc/acBmlhWpCFxHJdcnWwPcGXjezeWb2NTPTYOPeYusngGNNWEPIRET8IKkE7pz7NDAG+H/AjcAqM/udmR3RE8FJEqJjwOva+1JWpBq4iEiuS/oeuHNuiXPuery1vb8EVAP/NLMFZnZZdB1vSbdoAl8bUg1cRMQP9qQTWwQIRZ8NaAeuAz42szNSEJskI7qQSQPqxCYi4gdJJ3AzG25mtwAfA48Aa4AjnHOHACOAnwF3pTJI6YZoDbzBlVOmTmwiIjkv2V7ozwFLgTOAHwFDnHNTnXNzAZxzIeA3wOBUByq7sHkdzvJopFg1cBERH0i2qrYKONo599pOytQDI3c/JNktzfW0FfbDbQ2oBi4i4gNJfdM75y7uRhkHrNjtiGT3NDfQUtAPQJ3YRER8INkm9BlmdmWC41eY2R2pC0uS1lzPlrxKADWhi4j4QLKd2L4AvJrg+KvAF/c8HNltzfU0BSsA1cBFRPwg2QRehXePO956oP+ehyO7rbmejcFK8oNGYb6muBcRyXXJftPXAolmXTsCr4ObZEJbM7Rv4RPKKCvMx8wyHZGIiPSwZLsrPwTcYWZbgeejx04Cfoo3fEwyIXYMuO5/i4j4QrIJ/AfAKOCPgIseM+Bh4JYUxiXJiM7Cti5cqpXIRER8ItlhZGHgQjP7PnBw9PBbzrmlKY9Mui9aA18T6ktZiWrgIiJ+sFvVNefcEmBJimOR3RVN4LXtpVRoJTIREV9I+tvezPYBzgKGAwWxrznnvpKiuCQZm9cBsLKliGEaQiYi4gtJJXAz+y/gaeB9YCzwDt498QDwRsqjk+5pboCCUuq3BtSJTUTEJ5IdRvZ94P+ccwcBrcA5wDDgZeCpFMcm3dVcjyupoqU9onnQRUR8ItkEvj/wYHQ7BBQ555qB7wHfTmVgkoTmekJFewGaRlVExC+STeBb2NbsvgZv/W/wkvmAFMUkyWpuoK1PFaBpVEVE/CLZ9tY3gUl498BfAn5oZjXA+cC8FMcm3dW8jq2VEwA0DlxExCeSrYHfwLalQqfjTa36E6AImJbCuKS7ImHYsp4t+VqJTETET7pdXTOzAF4T+gcAzrkG4LQeiku6a+sGcBGtRCYi4jPJ1MAdsAAY1EOxyO6ITuKy0aIJXBO5iIj4QrcTuHPOAUuByp4LR5IWTeDrKQdUAxcR8Ytk74F/D/g/MxvSE8HIbuhciayUYMAoLghmOCAREUmHZBP4rXi90FeYWZ2ZfRD72NWbzexkM1tsZkvM7LoErw8zs5fMbJ6ZzTezU5OMz382ewl8bbiMssI8rQUuIuITyd4w/f3ufpCZBYG7gM/i9V5/w8xmOecWxhT7LvC4c+5XZjYWmM22seaSSHM9WIC17UWUFrZnOhoREUmTZJcTvXkPPmsSsMQ5twzAzB4DzgBiE7gDyqLb5cCqPfg8f2iuh+IqGlsj6sAmIuIjyTah74khwMqY/drosVg3AV8ys1q82vfliU5kZlPNbK6Zza2vr++JWLNHcwOUVNO4tV0d2EREfCSpBG5mETML7+iRgnjOBe53ztUApwIPRcefd+Gcm+mcm+icm1hdXZ2Cj81izfXQt5rGFiVwERE/SbbN9ct4zdwd8oFD8dYH31Xzeh0wNGa/Jnos1v8AJwM45141s0KgCliXZJz+0bwOKj9FY21ITegiIj6S7D3wRJ3Y7jezd4ATgF/t5O1vAKPNbCRe4p4CnBdX5mPgM9Fz7g8UAj5vI9+FjiZ01cBFRHwlVffA/w58bmcFnHMh4DLgOWARXm/z98zsFjM7PVrsGuCS6B8EjwIXRSeQkUTatkDbZsJFe7GlLax50EVEfCRVba4nA5t2Vcg5Nxuvc1rssekx2wuBo1IUU+7b0gDA1oJ+gFYiExHxk6S+8c3sb/GHgMHAfnhjuCWdorOwbcn3Eria0EVE/CPZKlt8p7MIMBe4wjn3YmpCkm6LzsLWGKgENqsJXUTER5LtxHZxTwUiu6FjJbJAObCZMjWhi4j4RrLjwEea2b4Jjo82sxGpCkq6KZrAP3GlAKqBi4j4SLK90H9L4k5mRwL37Xk4kpTmBsgvYUPIS9xK4CIi/pFsAj8YmJPg+KvAIXsejiSlYxa2rSEANaGLiPhIsgk8H+iT4HgfoGDPw5GkNNd3TuISMCgpUAIXEfGLZBP4m8BFCY5/BXh7j6OR5EQTeFNLiL598ggEtBa4iIhfJFtluwX4f2a2D/B89NhJwCl4i49IOjXXw5BDvJXIdP9bRMRXkqqBO+dewJt1rRL4cfRRAZzqnHt+Z++VFItENA+6iIiPJX3TNJrEX+iBWCQZLRvBhaGkP41btRKZiIjfJDsO/FNmdliC44eZ2cTUhSW7FB0DTkmVauAiIj6UbCe2O4ERCY4PBX6xx9FI922OLpFeUq174CIiPpRsAj8Ab+7zeG9FX5N06ayBV9PYElINXETEZ5JN4BGgLMHxyt04l+yJZm8p0XBxFZtbQ1pKVETEZ5JNuv8Grklw/Bq82dgkXZrrwQJsNu/vKTWhi4j4S7LVtu8C/zSzeUDH8qGfAUYDx6cwLtmV5noo3ovGtgigaVRFRPwm2XHgbwKHAe/iTd5yCrAAONw5l+jeuPSU6Cxsm7a2A6qBi4j4ze6MA38PuKAHYpFkNNd3DiED1IlNRMRndrvd1cwGEreAiXPu4z2OSLqnuR4GH7xtJTJN5CIi4itJfeubWRkwA5hC4tXHgqkISrohZhpVUA1cRMRvku2F/mO8e+DnAi14K5PdCKwCzktpZLJj7S3Q2tg5iQsogYuI+E2y7a6nARc6514yswjwqnPuITOrxbsv/oeURyjb2+KNAaekmqZPvCb0vuqFLiLiK8nWwPcClka3G/EmcAH4F3BcqoKSXYidRrWlndI+eQS1FriIiK8km8BXADXR7SXA5Oj2CcDmVAUlu9C8rQburUSm5nMREb9JNoE/xbYJW2YAN5jZamBm9CHpELcSmaZRFRHxn6S++Z1z343ZfsrMjgSOBhY7555NdXCyAx0JvG9/GreuUQ1cRMSH9qjq5pz7D/Cf+ONm9izwVefc6j05v+xAcz3kF0NBCY0tIYZUFGU6IhERSbOeWkHsWEBZpac0N0BJFYC3Fria0EVEfEdLgGaj5nVQUg1AU0u7mtBFRHxICTwbRRcyiUQcTa0h1cBFRHxICTwbRadR3dwWwjmtRCYi4kdK4NnGuc4auKZRFRHxLyXwbNOyESKhzklcQCuRiYj4UU8l8H8BW3vo3P62uWMSF61EJiLiZ0klcDN73sy+aGY7rfI5507VGPAeEjsLW0cTuu6Bi4j4TrI18JXA74A6M/uxme3TAzHJzjTH1sC9JnRNpSoi4j9JJXDn3FeAQcB04NPAYjN7ycymmNkuq4FmdrKZLTazJWZ2XYLXf2Zmb0cfH5jZxmTi84WYaVSb1IQuIuJbSd8Dd85tds79xjn3KeAQ4D28WvkqM7vdzEYkep+ZBYG7gFOAscC5ZjY27txXOecOcs4dBNyJt3iKxGpuAAyK+nV2YlMNXETEf3a7E5uZlQPH4E2bGsTruHYM8IGZTUvwlknAEufcMudcG/AYcMZOPuJc4NHdjS9nNa32plEN5tHY0k5JQZC8oAYTiIj4TdLf/GZ2jJk9CKwCvgn8ARjmnPtv59xhwFTg1gRvHYJ3D71DbfRYos8YDowE/r6D16ea2Vwzm1tfX5/sj5DdmlZD6SAgOg+6OrCJiPhSsr3Q3wdeBMqBs4CRzrlbnXNrYoo9A/Tbw7imAE8658KJXnTOzXTOTXTOTayurt7Dj8oyTauhbDAAjS3tuv8tIuJTyd48fRS41zlXt6MCzrn1JP7DoA4YGrNfEz2WyBTg0iRj84fG1TDkUG9za0iTuIiI+FS3a+DRXubTgIrd/Kw3gNFmNtLMCvCS9KwEn7MfUAm8upufk7tCrbClAUq31cBLVQMXEfGlbidw51x7dDNhs3Y33h8CLgOeAxYBjzvn3jOzW8zs9JiiU4DHnHNudz4np21e6z2XeffAm1q0EpmIiF8l++1/L3AF8PXd+TDn3Gxgdtyx6XH7N+3OuX2hMTq5XUwNXJ3YRET8KdkEPhg4y8w+DbwJNMe+6JybmqrAJIGmVd5z6UCcc14vdDWhi4j4UrIJfG/grej24LjX1OTd05qinf3LBtPcFibitBKZiIhfJfXt75w7oacCkW5oXAXBPlBUSeOmFkDTqIqI+JWm8MomTauhdCCYbVtKVPfARUR8Ken2VzM7DjgPGA4UxL7mnPt0iuKSRBpjJnHRPOgiIr6W7ExsXwKeBwYAJwCNwEC8RU2Wpjw66SpmGlWtRCYi4m/JNqF/G7jKOXcm0AZcDRwAPIE3t7n0FOe6zoOuJnQREV9LNoHvzbZx3G1ASXTClZ/hLWIiPaVlE7Rv6ZzEpaMJXRO5iIj4U7IJfCNQEt1eDewb3S4BylIVlCTQMYQsZiUyQFOpioj4VLLVt9fw1v9+F/gL8DMzm4i3rvcrKY5NYnVO4rKtCb0oP0hBngYSiIj4UbIJ/Bqgb3T7ZqAUOA14D7gqhXFJvI5pVGOa0DWJi4iIfyU7kcvymO2taMnP9GnqmAd9Ww1czeciIv6l9tds0bQaCisgvwiILmSiDmwiIr6V7DjwajN7wMzqzCxkZuHYR08FKXSZxAWiS4lqCJmIiG/tznKi44AZwCq0gEn6xIwBB68X+oi9SnbyBhERyWXJJvDjgZOcc6/3QCyyM02rYcDYzt3GFnViExHxs2TvgW8AmnoiENmJcAg2r+2sgWstcBERSTaB3wrcYGaq+qVTcz24SGcC39oeJhRxugcuIuJjySbis4FPAbVm9j7edKqdnHMnpSowidExiUvcSmSqgYuI+FeyCbwWLVqSfh2TuJQO9HZbOqZRVUOIiIhfJTuRy8U9FYjsROckLl4NvEkrkYmI+J4mcskGTavBglBSDWglMhER6UYN3Mz+BpzlnNsU3d4h3QPvIY2rvebzgPf3ltYCFxGR7lTh6oBIzLakW9Oq7SZxAXViExHxs10m8Nj73h3bZlYE7B09vDS6sIn0lKY1UDW6c7exxWtCVyc2ERH/SnYu9AIz+ymwHngn+lhvZneYWZ+eCFCINqF3rYH3yQtQmB/MYFAiIpJJyVbh7gTOBK4E5kSPHQXcApQA01IXmgDQ1gytm7omcC0lKiLie8km8CnAec65Z2OOLTSzVcAjKIGnXtMa7zlmJTLNgy4iIskOI2sDliQ4vhRo3/NwZDuN0VnYopO4AJoHXUREkk7g9wJXm5l1HIhuXwHcl8rAJCpuEhfoqIErgYuI+Fl3xoHPjNkNAGcBnzWz/0SPfQrYC3gy9eFJZwIv23YPvGlrO0MrizIUkIiI9AbduZE6Om7/rejzgOjzx9HHqFQFJTEaV0NBKfQp3XaopV01cBERn+vOOPAT0hGI7EDTqi73v721wEO6By4i4nOaC723a1zdpfm8NRShLRzRJC4iIj6nBN7bNa3p2oFtq+ZBFxERJfDeLRLxOrHFDiFr0UpkIiKiBN67bVkPkfa4SVxUAxcRkTQncDM72cwWm9kSM7tuB2XONrOFZvaemT2Szvh6nc4x4FqJTEREukpbO6yZBYG7gM8CtcAbZjbLObcwpsxo4HrgKOfcBjPrn674eqVECWOd5vEAACAASURBVDzahF6uqVRFRHwtnTXwScAS59wy51wb8BhwRlyZS4C7nHMbAJxz69IYX+/TMY1qmWrgIiLSVToT+BBgZcx+bfRYrH2Bfc1sjpm9ZmYnJzqRmU01s7lmNre+vr6Hwu0FmtYABn0HdB7SPXAREYHe14ktD2/mt+OBc4F7zKwivpBzbqZzbqJzbmJ1dXWaQ0yjplVQUg3Bbcl69cYWCvMD9Mnrbf90IiKSTunMAnXA0Jj9muixWLXALOdcu3PuI+ADtp/K1T/iJnGJRBzPL1zLMaOriVlPRkREfCidCfwNYLSZjTSzAry1xWfFlfkzXu0bM6vCa1JflsYYe5e4SVzmrdzAmsYWThs3aCdvEhERP0hbV2bnXMjMLgOeA4LAb51z75nZLcBc59ys6GsnmdlCIAx8yzm3Pl0x9jpNq6BmYufus/PXUJAX4DP7+7tzvohfRCIRamtraW5uznQo0kNKSkqoqakhEEi+Pp3WsUjOudnA7Lhj02O2HXB19OFvoVZvIpfoJC6RiOOv767m2NHVlKoHuogvNDQ0YGaMGTNmt77gpXeLRCLU1dXR0NBA//7JV8z0G9FbxY0Bf7t2I6s3tXDquIE7eZOI5JKNGzcyYMAAJe8cFQgEGDBgAJs2bdq996c4HkmVpjXec7QT2+z5qykIBjhx7ICdvElEckk4HCY/Xy1uuSw/P59QKLRb71UC7606JnEpHYRzjr++u4ZjRldpAhcRn9GIk9y2J/++SuC9VUwT+tsrN1K3cSunqve5iIhEKYH3Vk2rIa8QiiqZvWA1+UFT87mIiHRSAu+tGr11wB0we8Eajt6ninJNnyoiPnT//feTl6cFnOIpgfdWTauhdDDzazep+VxEss6JJ57IRRddlJJznXPOOdTVxU/cKUrgvVWTN41qR/P5SWM1fExEcktbW1u3yhUVFTFggG4hxlMC742cg8bVuL4DeXbBao7ap4ryYjWfi0h2uOiii3jxxRd54IEHMDPMjPvvvx8z4+GHH+bUU0+lpKSEG2+8Eeccl1xyCXvvvTdFRUWMGjWK73znO7S2tnaeL74JvWN/zpw5HHLIIRQXF3PooYfyxhtvZOLHzRjdVOiNWjZCaCurXSW1G7Zyxaf9u56LiGxz8zPvsXBVY9o/d+zgMr73uQO6XX7GjBksW7aMQYMGMWPGDAAaG724r732Wn784x9z1113AeCco3///jzyyCMMGDCA+fPnM23aNPLz87n55pt3+BmRSITrr7+eGTNmUF1dzVVXXcXZZ5/Nhx9+6Jv75f74KbNNdBKX/6wvJC9gnHSAmo5EJHuUl5dTUFBAUVERAwd6t/9aWloAmDZtGueff36X8rfeemvn9ogRI1i6dCl33333ThO4c46f//znHHLIIQDcdNNNHH744SxdupQxY8ak+kfqlZTAe6PoJC4vrDSO3KeKiuKCDAckIr1BMrXg3mrSpEnbHbvnnnu49957Wb58Oc3NzYRCISKRyE7PY2ZMmDChc3/wYG/diLVr1/omgeseeG8UncTlncZiTtPc5yKSQ0pKSrrsP/HEE1x66aWcc845zJ49m3nz5jF9+nTa29t3ep5AIEAwGOzc75jRbFeJP5eoBt4bRRN4g/Xjs+p9LiJZqKCggHA4vMtyL7/8MgcffDBXX71tEcrly5f3YGS5QzXwXsg1rmYTpUzcexD9StR8LiLZZ+TIkbz55pssXbqUhoaGHdaox4wZw4IFC3j66adZunQpM2bM4KmnnkpztNlJCbwXaqr/mFWRCk3eIiJZ65prrqGqqooJEyZQXV3NnDlzEpabNm0aF1xwARdffDEHH3wwr7/+OjfddFN6g81S5pzLdAx7ZOLEiW7u3LmZDiOl1vzkMBY39WHctS+oBi7iY4sWLWL//ffPdBjSw3b172xmbzrnJsYfVw28l3HOkbdlLZG+aj4XEZEdUwLvZRbVbaAyspGqQcMzHYqIiPRiSuC9zItzFxA0x8hR+2Q6FBER6cWUwHuJxpZ2vvnEO7zw+tsA9K0aluGIRESkN9M48F7g30sa+NaT81m9aSt3jOsDHwClGv8tIiI7php4Bm1tC3PTrPc4797X6ZMX4I9fO5Iz947+k5QNzmxwIiLSq6kGniHzPt7ANY+/w7KGZi46cgTXnrwfRQVB+GA1BPKguCrTIYqISC+mBJ5mbaEIv3jxQ+7+xxIGlhXy8FcP46h9YpJ102roOxACahwREZEdUwJPoyXrmrji0bdZuLqRLx5aw/TPjaWsMH9bgQ/+BgtnwbDDMxekiIhkBVXz0sA5x8Ovr2Dyna+wetNWfnPBodx+1oSuyXvu7+DRKVC1D5x5d+aCFRHpBe6//37y8rbVMf/xj39gZtTW1u70fWbG73//+z3+/IsuuogTTzxxj8/Tk1QD72GfNLdx7R/n8/zCtRwzuoqfnjWB/mWF2wpEIvD378Mrd8Dok+CLv4M+fTMXsIhIL3TkkUeyevVq+vfvn9Lz/v73v+eCCy4gflrxGTNm9PqlSZXAe9CcJQ1c/fjbfNLcxndP25+vHDWSQMC2FQi1wp+/Du8+CYdeDKfeDkH9k4iIxCsoKGDgwPQNry0vL0/bZ+0uNaH3gLZQhB/NXsSX7nudvn3y+NPXj+Krx4zqmry3fAIPfd5L3ifeBJN/puQtIjnhnnvuoby8nJaWli7Hf/zjHzNs2DDC4TCXXHIJe++9N0VFRYwaNYrvfOc7tLa27vCciZrQX3rpJcaPH09hYSHjx4/npZde2u59N9xwA/vvvz/FxcUMHTqU//3f/2XTpk2d57zgggsAr+ndzLjooouA7ZvQnXPcfvvtjBo1ioKCAvbee29+/vOfd/msESNGMH36dK688kr69evHgAEDuOqqqwiFQsldwG5SxkixpfWb+cZjb7OgbhPnHTaMG08b6w0Pi7VhOTx8lvf8hftg3BczEaqIZJu/XgdrFqT/cweOg1Nu63bxs88+myuuuIKnn36ac845p/P4gw8+yJe+9CXMjP79+/PII48wYMAA5s+fz7Rp08jPz+fmm2/u1mesWrWKyZMnc/bZZ/PYY49RV1fHlVdeuV25oqIiZs6cydChQ1m6dCmXXnopV1xxBQ888ABHHnkkv/zlL7nssstYvXp1Z/lE7r77bm688UZmzJjBCSecwIsvvsg3vvENSktL+Z//+Z/OcnfeeSfXXnstr7/+OvPmzeP888/nwAMP7FImVZTAUyQUjvDbOR/xs+c/pE9+gN9ccCj/dUCC5p66N+GRKRBuhQv+DCOOSn+wIiI9qLy8nDPOOIMHH3ywM4HPnTuXhQsX8tRTTxEIBLj11ls7y48YMYKlS5dy9913dzuB33333VRVVXHPPfeQl5fH2LFj+eEPf8jnPve5LuW++93vdvmcH/3oR0yZMoXf/e53FBQUdDaV76p5/rbbbuPyyy9n6tSpAIwePZrFixdz6623dknOxxxzDNddd11nmd/97ne88MILSuC91fzajVz/1ALeW9XIifv35wdnjmNgeWHXQmve9TqqvfcnKK+Bi/4C1WMyE7CIZKckasGZduGFF3L66aezbt06+vfvz4MPPsikSZMYM8b73rvnnnu49957Wb58Oc3NzYRCoaQ6jS1cuJBJkyZ16al+9NFHb1fuqaee4uc//zlLliyhsbGRSCRCW1sba9asYfDg7s142djYSG1tLccee2yX48cddxwzZsxgy5YtFBcXA3DQQQd1KTN48GA++uijbv9cydA98D3Q3BrilmcWcuZdc6hvauVX5x/CPV+e2DV5r/wPPHIO/Poo+OA5OPJyuOQfSt4iktNOOukkqqqqeOSRR2hvb+exxx7jwgsvBOCJJ57g0ksv5ZxzzmH27NnMmzeP6dOn097entIYXn/9dc466yyOPfZY/vSnP/HWW2/x61//GoC2traUflaHgoKCLvtm1mO92VUD300vLFzL9KffZXVjC+cfNoxvn7zftnHdzsGyl+Bfd8Dyf0FRPzjhBph0CRRVZjZwEZE0CAaDnH/++Tz00EOMGjWKTZs2MWXKFABefvllDj74YK6++urO8suXL0/q/GPHjuWhhx4iHA4TDHr9jObMmdOlzCuvvEJVVRU/+MEPOo89+eSTXcp0JNzY88QrKyujpqaGl19+mcmTJ3ce/+c//8nIkSM7a9/ppgSepHWNLdz0zHvMXrCGfQf05cnzjuDQ4f28FyMRWPws/OunsGoelA6C//ohHHoRFJRkNG4RkXT78pe/zE9/+lO+973vMXnyZPr1874rx4wZw3333cfTTz/NgQceyF/+8heeeuqppM79ta99jTvuuIOpU6fyzW9+k1WrVnHDDTd0KTNmzBjq6+u57777OOGEE3jllVe4++6uE2WNHDkSgFmzZnH00UdTVFRE377bz8Vx/fXXc8011zB69GiOP/54/v73v/OrX/2Ku+66K6m4U0kJvBsiEcfcFRuY9U4dT89bRWs4wjdP2pepx+5NQV4AImFY+DS8/BNYtxAqR8LnfgETpkBen0yHLyKSEePHj+eggw7i7bff5qabbuo8Pm3aNBYsWMDFF19MKBRi8uTJ3HTTTVx++eXdPveQIUN45pln+MY3vsFBBx3E6NGj+cUvfsFnPvOZzjKTJ0/mhhtu4Dvf+Q6bN2/muOOO4yc/+QnnnXdeZ5lPfepTXHnllUybNo36+nouvPBC7r///u0+72tf+xrNzc388Ic/5Otf/zpDhw7ltttu65HOad1l8bPP9OiHmZ0MzACCwL3OudviXr8I+AlQFz30S+fcvTs758SJE93cuXNTHqtzjnfrGpn1Th1/mb+a1ZtaKMwPcOL+A7jmpDGMrCrxEve7T3mJu2ExVO0Lx34LDvhvjekWkT22aNEi9t9//0yHIT1sV//OZvamc25i/PG0ZRkzCwJ3AZ8FaoE3zGyWc25hXNE/OOcuS1dcXax5lzVL5vHSumKeWBLkrU/yyAsEOHbfaq49eT8+O3YAJX3yIByCtx+Ff90O65dA/7HeFKhjz4BA4nsoIiIiqZTOauIkYIlzbhmAmT0GnAHEJ/CMeW32Axz+8UzOBc4F2kuKCPQbSbBwFKwbAe0jvA5qr93lTcIyYByc/RDsN1nLf4qISFqlM4EPAVbG7NcChyUo9wUzOxb4ALjKObcyvoCZTQWmAgwbNixlAfb99Df54/v/xacHbqGyZRX5G5bDho+8WvaSFyAUnRZw0EEw5VEYcwqY7fScIiIiPaG33ah9BnjUOddqZtOAB4BPxxdyzs0EZoJ3DzxVH37giAEcOGJA4hedg81rYesGqN5PiVtERDIqne2+dcDQmP0atnVWA8A5t9451zGb/b3AoWmKbdfMoHQg9N9fyVtE0iadHY0l/fbk3zedCfwNYLSZjTSzAmAKMCu2gJkNitk9HViUxvhERHqVYDCY8tnJpHdpb2/vMh1sMtKWwJ1zIeAy4Dm8xPy4c+49M7vFzE6PFrvCzN4zs3eAK4CL0hWfiEhvU1FRwdq1a3tsKk7JrEgkwtq1a3d77fG0jgPvCT01DlxEJNMikQi1tbU0NzdnOhTpISUlJdTU1BDYyUimjI8DFxGR5AQCgZSOtJHcosHLIiIiWUgJXEREJAspgYuIiGQhJXAREZEspAQuIiKShbJ+GJmZ1QMrUnjKKqAhhefzK13H1NB1TA1dx9TQdUyNZK/jcOdcdfzBrE/gqWZmcxONt5Pk6Dqmhq5jaug6poauY2qk6jqqCV1ERCQLKYGLiIhkISXw7c3MdAA5QtcxNXQdU0PXMTV0HVMjJddR98BFRESykGrgIiIiWUgJXEREJAspgYuIiGQhJXAREZEspAQuIiKShZTARUREspASuIiISBZSAhcREclCeZkOYE9VVVW5ESNGZDoMERGRHvHmm282JFqNLOsT+IgRI5g7d26mwxAREekRZpZwyWw1oYuIiGQhJXAREZEspAQuIiKShZTARUREspASuIiISBZSAhcREclCaU3gZnaymS02syVmdl2C139mZm9HHx+Y2cZ0xiciIpIt0jYO3MyCwF3AZ4Fa4A0zm+WcW9hRxjl3VUz5y4GD0xWfiIhINklnDXwSsMQ5t8w51wY8Bpyxk/LnAo+mJbKoUDhCeziCcy6dHysiIpK0dM7ENgRYGbNfCxyWqKCZDQdGAn/fwetTgakAw4YNS1mA/3n4Jg5e+itaCBImQJhg9BEgZHlECBI271jEvEeYPMKWRySQR8TycBYkEt2n4zmQh4t5tkAeBPMhkO89B/MIBPOxYAEE86BPGZT0x8oGEiwbRGFJGcUFQYoL8ijuE6SkII9gwFL2c4uISPbprVOpTgGedM6FE73onJsJzASYOHFiyqrLe42exKK2TyAcAheCSBiLeNsWCWMuhEVCmAsTiIQwFyIvEqLAhQi4LQQiYQIuRNCFCLiw90yYoPP+DMgjTJAQeS5MviX80RLa7ApZ5ypYQQX1roIF+eOY8PlrOHXcoFT96CIikmXSmcDrgKEx+zXRY4lMAS7t8YjijDliMhwxOT0f5hxEwoRDbbS3t9Le3ka4rY1QeyvtWxpp37SKSOMaXNNarHktBVvWMWLLOsZuXcHkltc499H+PLvgRL5/xoH0KylIT8wiItJrpDOBvwGMNrOReIl7CnBefCEz2w+oBF5NY2zpZwbBPILBPIJ9iincrsAhid/X3oK78xB+HvkTR793AK8tXc8PzjyQU1QbFxHxlbR1YnPOhYDLgOeARcDjzrn3zOwWMzs9pugU4DGnnmSJ5RdiJ9zAgM0LeenURgaWF/K1h9/i8kfn8UlzW6ajExGRNLFsz5MTJ050vltONBKGXx0FkXbap/2bu1/+mDv//iEVxfn84MxxnHzgwExHKCIiKWJmbzrnJsYf10xs2SgQhBO/B+uXkD//Ya48cTSzLjua/qWF/O/v3+SqP7xNOJLdf5iJiMjOKYFnq31PhmFHwD9ug7Zmxg4u4+nLjuKrR4/kT/PqmLv8k0xHKCIiPUgJPFuZwYk3w+a18NrdAOQHA0w7bm8A5tduymR0IiLSw5TAs9mww2DMaTDnF9C8HoDq0j4MLi/knVpNIy8iksuUwLPdZ6ZD22b41087D42vqVANXEQkxymBZ7v++8FB58Mb98CGFQCMH1rOx59sYeMWDSsTEclVSuC54PjrwQLw0g8BGD+kAtB9cBGRXKYEngvKh8Bh02D+H2DNu4yrKQdgvu6Di4jkLCXwXHH0VVBYBi/eTHlRPiOrSnhHNXARkZylBJ4riirh6Kvhw7/B8lcYX1OuGriISA5TAs8lh02D0sHw/PcYP6SctY2trG1syXRUIiLSA5TAc0l+ERxzNdTN5fC+awB4Z6Vq4SIiuUgJPNcMORSAffIaCAZMPdFFRHKUEniuqRwBQJ/NKxndvy/z65TARURykRJ4rimqhIJS2LCisyNbti8ZKyIi21MCzzVmUDkcNq5gfE0FG7e0s/KTrZmOSkREUkwJPBdVDIcNK5hQ483IpoVNRERyjxJ4LqocARtXMGZAXwqCAY0HFxHJQUrguahyOLRvoaB1PfsPLtOMbCIiOUgJPBdVDPeeN6xgQk0579ZtIhxRRzYRkVyiBJ6LKqMJPNqRbUtbmKX1mzMbk4iIpJQSeC6qGOY9b1jOhM6VydSMLiKSS5TAc1FBCZRUw8YVjKruS3FBUB3ZRERyjBJ4rooOJQsGjAOHlKsjm4hIjlECz1XRyVwAJtSUs2hVI22hSIaDEhGRVElrAjezk81ssZktMbPrdlDmbDNbaGbvmdkj6Ywvp1QMh021EAkzvqaCtnCExWuaMh2ViIikSNoSuJkFgbuAU4CxwLlmNjauzGjgeuAo59wBwDfSFV/OqRwBkRA01mlGNhGRHJTOGvgkYIlzbplzrg14DDgjrswlwF3OuQ0Azrl1aYwvt3QMJduwnKH9iqgszldHNhGRHJLOBD4EWBmzXxs9FmtfYF8zm2Nmr5nZyYlOZGZTzWyumc2tr6/voXCzXMxkLmbGuJoKDSUTEckhva0TWx4wGjgeOBe4x8wq4gs552Y65yY65yZWV1enOcQsUV4DFujSke2DtU1saQtlODAREUmFdCbwOmBozH5N9FisWmCWc67dOfcR8AFeQpdkBfOhrAY2eAl83JByIg4WrmrMcGAiIpIK6UzgbwCjzWykmRUAU4BZcWX+jFf7xsyq8JrUl6UxxtwSO5RsaEdHNjWji4jkgrQlcOdcCLgMeA5YBDzunHvPzG4xs9OjxZ4D1pvZQuAl4FvOufXpijHnRCdzARhQVsiAsj7qyCYikiPy0vlhzrnZwOy4Y9Njth1wdfQhe6pyOGxeA+1bIb+I8erIJiKSM3pbJzZJpcoR3vNGr/P/hJpyPmpoZtPW9szFJCIiKaEEnssqto0FBxgfndBlgWrhIiJZTwk8l8WsCw4wPrq0qGZkExHJfkrguazvAMgr7KyBVxQXMHyvYnVkExHJAUrgucwMKoZ11sDBa0ZXE7qISPZTAs91MUPJAMYPKWfVphbqm1ozGJSIiOwpJfBcFzOZC2y7D65mdBGR7KYEnusqhkPLJtjqJezRA0oBWL5+SyajEhGRPaQEnus6x4J7tfDK4nyK8oPUblACFxHJZkrgua6y61hwM2NIZRF1G7ZmLiYREdljSuC5LmZd8A41lUXUbVQCFxHJZkrgua6oAgrLu3RkG1KhBC4iku2UwP0gbijZkMoiNm5pZ3NrKINBiYjInlAC94O4oWQ1lcUAug8uIpLFlMD9oGI4bPwYnAO8JnSAuo3qiS4ikq2UwP2gcgSEWmDzWsDrxAZQqxq4iEjWUgL3g7ie6NV9+1AQDKgJXUQkiymB+0HHZC7RseCBgDG4opBa9UQXEclaSuB+UDHMe97YtSe6mtBFRLKXErgf5BdC34FdJ3OpKFYTuohIFlMC94u4oWRDKoto2NxKS3s4g0GJiMjuUgL3i/jJXDqHkqkWLiKSjZTA/aJyODTWQrgd2DaUTM3oIiLZSQncLyqGg4vAplrAa0IH1cBFRLKVErhfdCwrGr0PPrCskGDAtC64iEiWUgL3i7ix4HnBAAPLCtWELiKSpdKawM3sZDNbbGZLzOy6BK9fZGb1ZvZ29PHVdMaX08qGQCBvu1XJ1IQuIpKd8tL1QWYWBO4CPgvUAm+Y2Szn3MK4on9wzl2Wrrh8IxCE8pquq5JVFPHqsvUZDEpERHZXOmvgk4Alzrllzrk24DHgjDR+vsQNJaupLGJtYwvt4UgGgxIRkd2RzgQ+BFgZs18bPRbvC2Y238yeNLOhiU5kZlPNbK6Zza2vr++JWHNTgslcIg7WbGrJYFAiIrI7elsntmeAEc658cDzwAOJCjnnZjrnJjrnJlZXV6c1wKxWMRya66GtGYAhFcUArFRPdBGRrJPOBF4HxNaoa6LHOjnn1jvnWqO79wKHpik2f+joib7xY0CTuYiIZLN0JvA3gNFmNtLMCoApwKzYAmY2KGb3dGBRGuPLfXHrgg+qKAQ0mYuISDZKWy9051zIzC4DngOCwG+dc++Z2S3AXOfcLOAKMzsdCAGfABelKz5f6KyBewm8T16Q/qV9VAMXEclCaUvgAM652cDsuGPTY7avB65PZ0y+UlIF+cWdk7mA14yudcFFRLJPb+vEJj3JbPtVySqL1YQuIpKFlMD9Jn4oWUURqzdtJRxxGQxKRESSpQTuNx01cOcl7JrKItrDjnVNGgsuIpJNlMD9pnI4tDXB1g1AzLKiug8uIpJVlMD9pnMo2XLAmw8dNJRMRCTbKIH7Tdyyoh01cPVEFxHJLkrgftNvFGDQ8CEAxQV59CspUAIXEckySuB+U1AMFcOgYXHnoSEVWhdcRCTbKIH7UfUYqN+WwL3JXLSgiYhINlEC96Oqfb0m9EgY8GrgqzZuxTmNBRcRyRZK4H5UvR+EWzsndBlSWURLe4T1zW0ZDkxERLpLCdyPqsd4z9Fm9JpKb11wdWQTEckeSuB+VLWv9xxN4EMqNJmLiEi2UQL3o6IK6DsQGj4AYmZj26iObCIi2UIJ3K+q94X69wEoL8qntDBPTegiIllECdyvqsZA/Qedi5oMqShSE7qISBZRAver6jHeoiZNqwFvLLgmcxERyR5K4H7V2RPda0avqSymdoPGgouIZAslcL+q6kjg0Y5sFUVsbg3RuDWUwaBERKS7lMD9qm9/KKzonBO9c1Uy9UQXEckKSuB+ZdZlTvSaSo0FFxHJJkrgfla173aTuWgomYhIdlAC97Pq/WBLAzSvp19JAYX5AfVEFxHJEkrgftbRE71hMWZGTWWxmtBFRLKEErifJZgTXZ3YRESyQ1oTuJmdbGaLzWyJmV23k3JfMDNnZhPTGZ/vlA+F/OIuc6KrBi4ikh3SlsDNLAjcBZwCjAXONbOxCcqVAlcCr6crNt8KBKBqdMxkLkVs2NJOc6vGgouI9HbprIFPApY455Y559qAx4AzEpT7PvBjoCWNsflXx5zoxCwrqo5sIiK9XjoT+BBgZcx+bfRYJzM7BBjqnHt2Zycys6lmNtfM5tbX16c+Uj+pHgONtdDapLHgIiJZpNd0YjOzAHAHcM2uyjrnZjrnJjrnJlZXV/d8cLmssyf6B9RUFgNQqxq4iEivt0cJ3Mz6mtlpZja6G8XrgKEx+zXRYx1KgQOBf5jZcuBwYJY6svWwmDnRq/v2oSAYoHaDeqKLiPR2SSVwM3vEzK6IbufjdTR7BnjPzCbv4u1vAKPNbKSZFQBTgFkdLzrnNjnnqpxzI5xzI4DXgNOdc3OTiVGS1G8kBPKhYTGBgDGoolBN6CIiWSDZGvjxwJzo9ufwas2DgJuAG3f2RudcCLgM/n979x4nR1nne/zz6+7pmcwkM5PL5EJuBBISEiAXQryxCKxiAA+wgkoEdQFl9QiLZ3ePsurRlVXPUXddLwsoi6jrBRRBREUUEAUFImOAlgAAIABJREFUhBDCJQlJBoRcIMnkMkkmk7l01+/8UdXTNcMEZpJOX2a+79erX/XUUzXVvy7o/J566umn+A2wBviJu68ys2vM7JxBxiGFkqyCsUf3mhNdg9hERMpfapD7jwG2RuW3Are7+1Yz+xFwwN9157j7XcBdfeo+fYB9Tx1kbHKwxh0DW1cB4Uj0+9dqYKCISLkb7BV4CzAjKr8VuD8q1wJBoYKSImuaA7v+AplOJjfW0rK3k47ubKmjEhGRVzHYBH4r8EMzuxeoB+6J6hcA6wsZmBRR02zwAHY09/yU7OXd+hm+iEg5G2wC/xjwVeAZ4K3unhuufATwX4UMTIooNif65NG5x4pqJLqISDkb1D3waCDaV/qp/7eCRSTFN24WYLB9HZNPWApoMhcRkXI32J+RzTezebH1s8zsVjP7FzMb7IA4KRdVI2D0dGh5lkkNNSQTxkZdgYuIlLXBdqF/CzgewMymAD8FRgIfBD5X2NCkqKI50VPJBLPGj+SpTbtLHZGIiLyKwSbw2cATUfkdwGPufibwPuDdhQxMiqxpNuxohmyGhdNGs3JjK0HgpY5KREQOYLAJPE3+KWGnAr+OyuuAiQWKSUqhaTZkO6H1RRZNa2RvR4bmlrZSRyUiIgcw2AS+FrjAzKYR/g783qh+ErCrkIFJkfXMib6WRdNHA7DiRf0nFREpV4NN4J8FvgD8BfhjbJ7yM8h3rUslaop+SrZ9LUeNq6OxtooVG5TARUTK1WB/Rvbz6Op7EvBUbNN9wO2FDEyKrKYBRk2ClrWYGQunNrJiQ2upoxIRkQMY9ONE3X2ru68E0mZWE9U97O6rCx6dFNe4Y3oeanLi9NE0b2tjd3t3iYMSEZH+DDqBm9klZtYMtAFtZrbezP624JFJ8TXNge3rwJ1F08L74E9sVDe6iEg5GuxELlcB1xE+x/v86PVL4Dozu7Lw4UlRNR0DXW2wZzPzpzaSMNSNLiJSpgY7e9qVwFXufkOs7udm9izwv4FvFCwyKb7YSPS6mVOYPbFeI9FFRMrUYLvQpxIOWOvrvmibVLKmOeFy+zoAFk1rZOXGVrKa0EVEpOwMNoFvIpzApa9To21SyerGwYjR0PIsAIumjaatM8P6bXtLHJiIiPQ12C7064Gvm9lM4MGo7hTCrvVPFzIwKQGznjnRgdiELq3MmVhfyshERKSPQV2BR48N/RhwEeHgtV8C7wH+yd3/vfDhSdE1zYbt4U/Jjhxby5i6tCZ0EREpQwfzO/Br3X0a0AA0uPs0d7++8KFJSTTNhvYdsG97bEIXJXARkXLzml3oZvbb19jeU3b3MwoQk5RSbCQ6deNYNH009z27jdb2Lhpr06WNTUREegzkHvjmwx6FlI+mKIFvXwtHvik/ocuGVk6bM76EgYmISNxrJnB3v6QYgUiZaJgSzou+4RFYfCnzpzaQTBgrNuxSAhcRKSODvgcuQ5wZzPsbWH0ndOyhNp1izsRRPK4JXUREyooSuLzSwvdCZj+s+hkQ/h78SU3oIiJSVpTA5ZUmnxgOZnviBwAsmt7Ivq4sa7doQhcRkXJR1ARuZkvNbK2ZNZvZ1f1s/5CZPW1mK83sj2Y2t5jxScQMFl4Mmx6FlrU9A9n0czIRkfJRtARuZkngWuBMYC6wrJ8E/SN3P97dFwBfAr5SrPikj/kXgiVh5Q+ZNqaWsZrQRUSkrBTzCnwJ0Ozuz7t7F3ALcG58B3ffE1utA3TTtVRGjodj3gZP3oIFWRZOG80TerSoiEjZKGYCnwxsjK1viup6MbOPmNlzhFfgf9/fgczscjNbbmbLW1paDkuwQtiN3rYVmu/lxOmj+cv2fezc11XqqEREhDIcxBZN1Xo08HHgUwfY5wZ3X+zui5uamoob4HAy6wyoa4Invs+iaY0APKFudBGRslDMBL6Z3s8Mn8Krz/J2C3DeYY1IXl2yCk54N6y7mxNGd5OKJnQREZHSK2YCfwyYZWYzzCwNXAjcGd/BzGbFVs8G1hcxPunPwoshyDDi2ds4dlI9K17UfXARkXJQtATu7hngCuA3wBrgJ+6+ysyuMbNzot2uMLNVZrYS+Afg/cWKTw5g/LEweTGs+D6LpjawcmMrmWxQ6qhERIa9gTzMpGDc/S7grj51n46VrypmPDJACy+CX/4vTp+7me91Z3l2y16Om9xQ6qhERIa1shvEJmXouPMhVcOJO8O2lwayiYiUnhK4vLaaBph7LnXrfsbkOlih34OLiJScErgMzIKLsM49XDL2GY1EFxEpA0rgMjBH/hU0TmNp9328uKOd7W2dpY5IRGRYUwKXgUkkYMHFTN71KFOsRdOqioiUmBK4DNyCZQC8M/mAutFFREpMCVwGrnEadtSbuTD9Rx5p1hz0IiKlpAQug7PwvUwItlLz0sO81Lq/1NGIiAxbSuAyOHPOJlvdwIXJ+/n1M1tKHY2IyLClBC6DUzWC5IL3cHbyzzyy8plSRyMiMmwpgcvgve5ykgTM33IrW3Z3lDoaEZFhSQlcBm/MUeyb8TYuSt7HPU8+X+poRESGJSVwOSgj3/z3jLY22pffXOpQRESGJSVwOTjT38jWujmc3nob23ZrNLqISLEpgcvBMcNf/2FmJTbz1B9uL3U0IiLDjhK4HLSJb3gPO2wMTc/cWOpQRESGHSVwOXipNGumvpv5XSvY8ZeVpY5GRGRYUQKXQzLxtA/T4VXsvO9rpQ5FRGRYUQKXQ3L0kdO4N3060zf9AvZtL3U4IiLDhhK4HBIzo2XepaTpZt9DN5Q6HBGRYUMJXA7Z61/3Rn6fnU/isRsh01nqcEREhgUlcDlkcyaO4q66v2FE1w545rZShyMiMiwogcshMzOaFixlXTCZzJ/+E9xLHZKIyJCnBC4FcdYJR3BT9kxSLavghQdLHY6IyJCnBC4FMXdSPY83vJU9iQZ4+LpShyMiMuQVNYGb2VIzW2tmzWZ2dT/b/8HMVpvZU2Z2n5lNL2Z8cvDMjLeccCT/3X06vu5u2PFcqUMSERnSipbAzSwJXAucCcwFlpnZ3D67PQEsdvcTgJ8CXypWfHLozj5+Et/rfguBJeGR60sdjojIkFbMK/AlQLO7P+/uXcAtwLnxHdz9fndvj1YfAaYUMT45RPOOqKdmzBE8VHsaPPEDaNtW6pBERIasYibwycDG2PqmqO5ALgN+3d8GM7vczJab2fKWlpYChiiHwsw46/hJXLNrKZ7thD9pelURkcOlLAexmdnFwGLgy/1td/cb3H2xuy9uamoqbnDyqs4+fhLrg0m8OPnt8NiNsHdLqUMSERmSipnANwNTY+tTorpezOwtwCeBc9xd03pVmOMnNzBl9AiuD94B2W7441dLHZKIyJBUzAT+GDDLzGaYWRq4ELgzvoOZLQS+RZi8dQO1ApkZZ58widteqGb/vHfB8ptgz0ulDktEZMgpWgJ39wxwBfAbYA3wE3dfZWbXmNk50W5fBkYCt5rZSjO78wCHkzL27sVTyQTOT+uWgWfhwa+UOiQRkSHHvMKnvVy8eLEvX7681GFIH+/+1sO8vLuDP8z5GfbkzXDlCmic+tp/KCIivZjZ4+6+uG99WQ5ik8r3ntdNY8POdh6fdlk4N/qD/17qkEREhhQlcDks3jZvIo21VXxndRYWvS/8XfiuF0sdlojIkKEELodFTVWS8xdN4bertrDzxCvBDB78t1KHJSIyZCiBy2GzbMlUurPOresCOPESeOKHsPP5UoclIjIkKIHLYTNz/ChOOnI0tzy2ET/5o5Csggd0FS4iUghK4HJYLVsyjb9s38cjLdWw+DJ48mY9qUxEpACUwOWwOuv4SdTXpLj50Q1w8kchWQ1/+GKpwxIRqXhK4HJY1VQleceiKdz9zBZ2WiMs+QA8fSu0rCt1aCIiFU0JXA67C5dMpSsbcPuKTfCmj0JqhK7CRUQOkRK4HHZzJtazaFojNz+6Aa8dC6+7HJ65DV58qNShiYhULCVwKYplS6bxXMs+HnthF/zVP8KYGXDbB2H/rlKHJiJSkZTApSjOPmESo6qjwWzVo+D8G6FtC/ziqnCqVRERGRQlcCmK2nSK8xZO5ldPv0xrexdMPhFO/xSs/jms+O9ShyciUnGUwKVoli2ZRlcm4PYVm8OKN14FM06Bu6/WqHQRkUFSApeimXtEPfOnNnLLYxtwd0gk4G9ugFQN3HYpZDpLHaKISMVQApeiWnbSVNZtbWPFhmjwWv0kOO862PI03PvZ0gYnIlJBlMClqP7H/COoSyf50Z835itnnwknfRAeuRbW31u64EREKogSuBRVXXWKcxdO5pdPvUTL3liX+Rn/CuPnwh0fgrZtpQtQRKRCKIFL0V128gwCd/7vr9fkK6tGwPnfhs69cMeHIQhKF6CISAVQApeiO7ppJJefchS3r9jMI8/vyG+YMBfO+Bw03wt/vr50AYqIVAAlcCmJK06bxZTRI/jUHc/QlYldbZ/0AZh9FtzzGXj+9yWLT0Sk3CmBS0mMSCf57DnzaN7Wxrf/+Jf8BjM45z9h7Ez4wQWw8ubSBSkiUsaUwKVk/vrYCZwxdwJfv289m3a15zfUjYVL74bpbwgHtf3+i5puVUSkDyVwKanPnDMPgM/+YnXvDSMa4aLbYP4y+P0X4OdXQLa7BBGKiJQnJXApqcmNI7jqLbO4Z/VW7l29tffGVBrOux7e/HFY+QP44TuhY09pAhURKTNK4FJyl508g1njR/KZO1exvyvbe6MZnPYJOPdaeOFB+M6ZsHtzaQIVESkjRU3gZrbUzNaaWbOZXd3P9lPMbIWZZczsgmLGJqVTlUzwufOOY3Prfr7xu/X977TwYrjop7DrRbjxLeHUqyIiw1jREriZJYFrgTOBucAyM5vbZ7cNwN8CPypWXFIeXnfUWM5fNIX/evB5mrft7X+no0+Dy34TXpXfdCas+D5kM8UNVESkTBTzCnwJ0Ozuz7t7F3ALcG58B3d/wd2fAjQN1zD0ibPmUJtO8ak7ngmfVtafCfPgA/fC+GPhzivgutfB0z/VzG0iMuwUM4FPBmJPsGBTVDdoZna5mS03s+UtLS0FCU5Kb+zIaj62dDaPPL+TO1a+yn3u+iPgst/ChT+CZDXcdhl8802w5hf6uZmIDBsVOYjN3W9w98XuvripqanU4UgBLTtpGvOnNvL5X61h576uA+9oBnPOhg/9ES64KfyJ2Y8vhhtOhfX3KJGLyJBXzAS+GZgaW58S1Yn0SCSMz593HHs6MrzrWw/zUuv+1/oDOO58+J+PwHnfhP274IcXwLfPgNU/h8624gQuIlJkxUzgjwGzzGyGmaWBC4E7i/j+UiGOm9zAf1+6hK27Ozj/+ocOPKgtLpmCBcvgysfh7V+FPZvhJ++DL82A778DHv0vaN342scREakQdsDBQofjzczOAr4KJIGb3P3zZnYNsNzd7zSzk4CfAaOBDmCLu897tWMuXrzYly9ffrhDlxJY/dIe3v+dR+nOBnz7/Sdx4vTRA//jbAY2PgJrfx2+dj4X1k84HmYvhWPOhCMWhlfwIiJlzMwed/fFr6gvZgI/HJTAh7YNO9p5301/ZsueDq6/6EROmzP+4A60fX2YyNfdDRseBg+guiEczT5hLoyPXhPmwohBNBRERA4zJXCpWC17O7nku4+y5uW9fPmCE3jHoimHdsD2neEzxzc8AttWw9bV0Lk7v33UEWFiH38sNE6HxqnQOA0apkJN/aG9t4jIICmBS0Xb29HN333/cR56bgefPOtYPnjKUYU7uDvseSlK5qtg2xrYtiq8as909N63pjFK6NOhYUq4XlMP1aOguj4q18fKo6CqNhw1LyJyEJTApeJ1ZrL8w4+f5FdPv8zfnXIUV585BzucidEd9rVA64b8a/fGWHkzdA1ggJ0l8wm+elT+VVMP6TqoqoN0bZjo03W9l9WjoKYhfDpbTQOkR+m+vcgwc6AEnipFMCIHozqV5OvLFjKmLs23HnieR1/Yyd+fPotTZzcdnkRuBiPHh68pr/juhIIsdO6Fzj3hsmNPrLw72hbbnivvawkH1nW1Q3c7dO0Dz/b/Hr1iSkRX91FST48KR+AnqiCZjpWrIJEK61LV4SsZLVM14ZPeUjVhnQfheweZ8PN4NlwGUV0ynW9gVNW+spyszr9f/JWsUs+DyGGkBC4VJZkwrjl3HvOOqOcbv2vmku8+xrwj6rny9JmcMXciiUSRE0YiGSbSEY2Hdhx3yHaFiby7PUrs+/INgY7dsL81X+5ozTcQutoh6A5H3gfd4XHi5UwXZDvDcrFZIkzw1SMhPTJcVtfny+mRYW+DRb0KZlHZouQfrecaBMl0tIzKPY2VZOzvEv28cseJ9rNk7/VUddgoqmmM4lHDQ8qfutClYnVlAu54YjPX/b6ZF3a0c8yEkXzktJm8/YQjSBY7kVeCIAgTeSZ65co9iSwZLhOpWDkZznKXa1j0NC72hw2MrvbwOEEmajREDYfc1Xy2OxxH0NUWTqrTs9wbLjv3hsd0Bzy2DPqUizjXfSIVJfMoodc0hLcy+jYGejUWYuX4uYzXuec/R+7zeZD/jImqfno66qBqRFhOjcj3pqRq+vSqVId/nzuH8eP2nEvC4+oWTMXRPXAZsjLZgF89/TLX3t/Muq1tzBhXx4dPPZrzFkwmndI/VkNCEEQ9ClGvQpCJehq6wrqeRBVLjD3lbGwZ5G8TeBAe1wPI7O+np6M1v97V1n/S9QCc/G2Ifo8f3Y7oN/nHehqCzMBvpRw0yw+wrGnoM/ByVPjema6w0ZWNlvEGXyIVNTBGRK/a3stUTVSuCRscuf1y9ZaIGn65V1vvcqYzdozob9N1vd8j3uuSiG4ZJVLR7aNUvjeHqBGfO7+5ck/OizcY40v69Nb07cWpCuMoYi+NErgMeUHg/Hb1Fr7xu2ZWvbSHunSSJTPG8Majx/HGmWM5dmJ98bvYRQbDPWyQdO8LezlyYyRyr9ztkExn7+Sa6QgbAD0NA/KJJ1fnQZgkc+M0epa782M0LJkfIxG/us+NpQiyUe9L1AuT2R+tR3WH0lOSG5OR6Qg/YzlLVPUeXBrvrRnRCPPfA03HFOztNIhNhrxEwlh63CTeNm8iD6zfzj2rt/DQczu4f+0aAEbXVvGGo8eGCf3oscwYV3d4R7GLDJZZNMAwXXkTCuXGcXTvD5Nwdzt0d8SSfEd4hZ+ui14j8+WquvAKOqdXQ6G9d2Mm6M7fngkyvV/ZbnpdSfdX7ntF3mu8RVTfX49OEPWqZLvyDZ/4uJTWDfm6GacUNIEfiBK4DDlmxpuPaeLNx4RPqnt5934efm4Hf2rewUPPbeeup7cAMLYuzawJI5k5fiSzxo9i1viRzJwwkqaR1UrsIoNllr9iP1SJZDTgceShH6vYijk9ubrQZThxd17Y0c6fmrfz1KZW1m9ro3lbG3s7Mj371NekmDVhFEeNq2NiQw3jR1Uzvj5cTqivoWlUNVVJ3VsXkeJQF7oI4dX5jHF1zBhXB0wHwqS+bW8n67e20bxtb09S/8O6Fra3dRL008YdW5emaVQ1Y+rSjK5N01hb1Ws5uq6Kxto0dekUtekkI9JJatNJalJJ3YcXkYJQApdhz8yYUF/DhPoaTp41rte2bODs2NfJtj2dbN3Twba9+eW2PZ20tnexZsseWtu7aW3v6jfZ91VTlaA2nWJEVbInuefKtelUT7LP1adTCapTSapTifBVFZbTufVoW01VgnQySXVVvj6dSugndSJDlBK4yKtIJozxo2oYP6qG4yY3vOq+QeDs7ciwq72LXe1dtLZ3s68rQ3tXlv1dWfZ3Z6Nyf3VZWto6ae9qp6MrS3tU35U59N8/JwyqkgmqkglSSQvLCaMqleipT0f16agunUqQTiaoShqp3DKR//tUIqqPluH+lj9ez3Hyf5dKhI2JVMLCZTJXzh0v2jcqVyXz+2tMgsgrKYGLFEgiYTTUVtFQW8WR1BXkmEHgdGUDOrsDOrPZcJkJ6MoEdGaydHQH0fZsrD7c1pkJ6OjOksk63UFAd8bJBAHd2YDurEfLsNyVCXrW93Vm6Iq2d2UCMtmA7sDJZIOeY2WyTmYg3Q0FkkrkGgf5RkJVyqhKJPKJPhk2DKoSvdfzDYaocZDINUzi5XxjJJmwngZLbluuUZGwfEMkfuxkItoW7ZPbliv3bbgkE/njVEXryWhfNVZkoJTARcpYImHUJJLUVCWBqlKH04t7mMQz2bCRkUv43T3lqBGQDcgGTjYI989GDYDceibWIMjEGgq5cq7xEG94ZLL5Y3dHx8/k3iM6dmcmSzZwuqP17iDo2d6dDcgE+WOFDZvyGNCbMHo1EuKvXCMjlQwbKqlejY9X9nLEezESfZfxhkasLqwP/9/LNSriDZGwwZHvlelp8PQ0kHrHUpXs3VCpijWkUsne+8rgKIGLyEExs+iKGEaQLHU4h8w93qjINRryDYzctsDzjYSsxxok7gQBZIKAwJ1sANkgIBur69VwiRoR8cZHvKGT2z/wMIZcY+SVPSL5cnsm07sx07MMyGZz8cbeI8jXZQIv5i+gXsGM3r0WFjYs8g0ISFrYQxHWETY44mUzEolY2ejp1UjGtsVv3SSTsd6ReH2ssRJfTyQMwzALG1u5cvgZDAPeOncCU8fUHvZzpgQuIkL4j294RVjqSErH3Qk8HLwZeL6REsQaF92Bk+11K6V3j0lun1wDpfc++UZLrjHSa58g9l4eb2DQUw48bGjk4ouXc9uy0ecI4p8jcLqyvRtKPY2bWGOnO2o0xXt2MtlgQANUc2Y01SmBi4hI8YRXqqg7ux/x3gp3cPKNByc2nTpObbo4qVUJXERE5DUkEkaC8JZRudB0UiIiIhVICVxERKQCKYGLiIhUICVwERGRCqQELiIiUoGKmsDNbKmZrTWzZjO7up/t1Wb242j7n83syGLGJyIiUimKlsDNLAlcC5wJzAWWmdncPrtdBuxy95nAfwBfLFZ8IiIilaSYV+BLgGZ3f97du4BbgHP77HMu8L2o/FPgr00z+4uIiLxCMRP4ZGBjbH1TVNfvPu6eAXYDY/seyMwuN7PlZra8paXlMIUrIiJSvipyJjZ3vwG4AcDMWszsxQIefhywvYDHG650HgtD57EwdB4LQ+exMAZ7Hqf3V1nMBL4ZmBpbnxLV9bfPJjNLAQ3Ajlc7qLs3FTJIM1vu7osLeczhSOexMHQeC0PnsTB0HgujUOexmF3ojwGzzGyGmaWBC4E7++xzJ/D+qHwB8Dv3Uj7gTkREpDwV7Qrc3TNmdgXwGyAJ3OTuq8zsGmC5u98JfBv4vpk1AzsJk7yIiIj0UdR74O5+F3BXn7pPx8odwDuLGVM/bijx+w8VOo+FofNYGDqPhaHzWBgFOY+mHmoREZHKo6lURUREKpASuIiISAVSAo95rbnapX9mdpOZbTOzZ2J1Y8zsHjNbHy1HlzLGcmdmU83sfjNbbWarzOyqqF7ncRDMrMbMHjWzJ6Pz+Nmofkb0fIXm6HkL6VLHWgnMLGlmT5jZL6N1ncdBMrMXzOxpM1tpZsujuoJ8r5XAIwOcq136911gaZ+6q4H73H0WcF+0LgeWAf7R3ecCrwc+Ev3/p/M4OJ3A6e4+H1gALDWz1xM+V+E/oucs7CJ87oK8tquANbF1nceDc5q7L4j99rsg32sl8LyBzNUu/XD3Bwh/9hcXn9f+e8B5RQ2qwrj7y+6+IirvJfxHczI6j4PiobZotSp6OXA64fMVQOdxQMxsCnA2cGO0bug8FkpBvtdK4HkDmatdBm6Cu78clbcAE0oZTCWJHqO7EPgzOo+DFnX7rgS2AfcAzwGt0fMVQN/tgfoq8DEgiNbHovN4MBz4rZk9bmaXR3UF+V5X5FzoUlnc3c1Mv1ccADMbCdwGfNTd98QfxqfzODDungUWmFkj8DNgTolDqjhm9nZgm7s/bmanljqeCneyu282s/HAPWb2bHzjoXyvdQWeN5C52mXgtprZJIBoua3E8ZQ9M6siTN4/dPfbo2qdx4Pk7q3A/cAbgMbo+Qqg7/ZAvAk4x8xeILydeDrwNXQeB83dN0fLbYQNyiUU6HutBJ43kLnaZeDi89q/H/h5CWMpe9H9xW8Da9z9K7FNOo+DYGZN0ZU3ZjYCeCvheIL7CZ+vADqPr8nd/9ndp7j7kYT/Fv7O3S9C53FQzKzOzEblysAZwDMU6HutmdhizOwswvs+ubnaP1/ikCqCmd0MnEr4iLytwGeAO4CfANOAF4F3uXvfgW4SMbOTgQeBp8nfc/wE4X1wnccBMrMTCAcFJQkvUH7i7teY2VGEV5JjgCeAi929s3SRVo6oC/2f3P3tOo+DE52vn0WrKeBH7v55MxtLAb7XSuAiIiIVSF3oIiIiFUgJXEREpAIpgYuIiFQgJXAREZEKpAQuIiJSgZTAReSwM7NTzcyj+bVFpACUwEVERCqQEriIiEgFUgIXGQbM7Eoze9bMOsxsvZl9MjentZm9YGafN7MbzWyPmW03sy+YWSL296PM7Ftm1mJmnWa23MzO6PMe483sO2a2NXqftWZ2aZ9QjjWzB8ys3cxWm9mZRfj4IkOSnkYmMsSZ2b8AlwAfBVYCxwLfBGqA/xPtdiXhNMInET5s4ZuE0+J+Ldp+U7TtYmAD8CHgl2Z2grs/G807/gdgP3AR8Dwwk3DKzbh/Az5O+IjPTwA/NrPp7r6rsJ9aZOjTVKoiQ5iZ1QLbgXe4+92x+vcBX3f3xuiJUxvd/a9i278AvNfdp5rZTGA9cLa73xXbZwWw0t0vNbPLgGuBme6+qZ84TiV8EMb5uSetmdkEwmchL3X33xT6s4sMdboCFxna5gEjgNv6PHM4CdSYWVO0/nCfv/sT8M9mVg/Mjeoe6LPPA4SP6gQ4EVjdX/LuY2Wu4O5bzSwLTBjQJxGRXpTARYa23H3sdwLr+tle7CebdfVTp7E4Ige9WVxiAAABHklEQVRBXxyRoW0V0AEc5e7N/byy0X6v7/N3bwQ2u/ue6BgAp/TZ5xTCZxsDPA7M1e+8RYpHCVxkCHP3NuALwBfM7CNmNtvM5pnZhWb2xdiuC8zsX8zsGDN7D3AV8O/RMZ4DbgWuM7O3mdkcM/sacBzw5ejvbyZ8rvGdZvYWM5thZn9tZu8u1mcVGW7UhS4yxLn7v5rZy8AVhEl5P2F3+ndju30DmA4sB7qB/yQ/Ah3gA4TJ+gdAPfA08HZ3fzZ6j3YzezPwJeAWYCTwAvD/DtfnEhnuNApdZJiLRqHf6O6fK3UsIjJw6kIXERGpQErgIiIiFUhd6CIiIhVIV+AiIiIVSAlcRESkAimBi4iIVCAlcBERkQqkBC4iIlKB/j9F7tAN2iVWHAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "utils.plot_history(wn18_history)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate the model\n", "\n", "We've now trained a model, so we can apply the evaluation procedure from the paper to it. This is done by taking each test edge `E = (s, r, o)`, and scoring it against all mutations `(s, r, n)` and `(n, r, o)` for every node `n` in the graph, that is, doing a prediction for every one of these edges similar to `E`. The \"raw\" rank is the number of mutated edges that have a higher predicted score than the true `E`." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "wn18_raw_ranks, wn18_filtered_ranks = wn18_complex.rank_edges_against_all_nodes(\n", " wn18_gen.flow(wn18_test), wn18_graph\n", ")" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "# helper function to compute metrics from a dictionary of name -> array of ranks\n", "def results_as_dataframe(name_to_results):\n", " return pd.DataFrame(\n", " name_to_results.values(),\n", " columns=[\"mrr\", \"hits at 1\", \"hits at 3\", \"hits at 10\"],\n", " index=name_to_results.keys(),\n", " )\n", "\n", "\n", "def summarise(name_to_ranks):\n", " return results_as_dataframe(\n", " {\n", " name: (\n", " np.mean(1 / ranks),\n", " np.mean(ranks <= 1),\n", " np.mean(ranks < 3),\n", " np.mean(ranks <= 10),\n", " )\n", " for name, ranks in name_to_ranks.items()\n", " }\n", " )" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
mrrhits at 1hits at 3hits at 10
raw0.5987310.48140.60650.8192
filtered0.9401280.92990.94510.9546
\n", "
" ], "text/plain": [ " mrr hits at 1 hits at 3 hits at 10\n", "raw 0.598731 0.4814 0.6065 0.8192\n", "filtered 0.940128 0.9299 0.9451 0.9546" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "summarise({\"raw\": wn18_raw_ranks, \"filtered\": wn18_filtered_ranks})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For comparison, Table 2 in the paper gives the following results for WN18 (`NaN` denotes values the paper does not include). All of the numbers are similar:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
mrrhits at 1hits at 3hits at 10
raw0.587NaNNaNNaN
filtered0.9410.9360.9450.947
\n", "
" ], "text/plain": [ " mrr hits at 1 hits at 3 hits at 10\n", "raw 0.587 NaN NaN NaN\n", "filtered 0.941 0.936 0.945 0.947" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results_as_dataframe(\n", " {\"raw\": (0.587, None, None, None), \"filtered\": (0.941, 0.936, 0.945, 0.947)}\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## FB15k\n", "\n", "Now that we know the process, we can apply the model on the FB15k dataset in the same way.\n", "\n", "### Loading the data" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "tags": [ "DataLoading" ] }, "outputs": [ { "data": { "text/html": [ "This FREEBASE FB15k DATA consists of a collection of triplets (synset, relation_type, triplet)extracted from Freebase (http://www.freebase.com). There are 14,951 mids and 1,345 relation types among them. The training set contains 483142 triplets, the validation set 50000 and the test set 59071. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Durán, Jason Weston and Oksana Yakhnenko “Translating Embeddings for Modeling Multi-relational Data” (2013).\n", "\n", "Note: this dataset contains many inverse relations, and so should only be used to compare against published results. Prefer FB15k_237. See: Kristina Toutanova and Danqi Chen “Observed versus latent features for knowledge base and text inference” (2015), and Dettmers, Tim, Pasquale Minervini, Pontus Stenetorp and Sebastian Riedel “Convolutional 2D Knowledge Graph Embeddings” (2017)." ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fb15k = datasets.FB15k()\n", "display(HTML(fb15k.description))\n", "fb15k_graph, fb15k_train, fb15k_test, fb15k_valid = fb15k.load()" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "StellarDiGraph: Directed multigraph\n", " Nodes: 14951, Edges: 592213\n", "\n", " Node types:\n", " default: [14951]\n", " Features: none\n", " Edge types: default-/american_football/football_coach/coaching_history./american_football/football_historical_coach_position/position->default, default-/american_football/football_coach/coaching_history./american_football/football_historical_coach_position/team->default, default-/american_football/football_coach_position/coaches_holding_this_position./american_football/football_historical_coach_position/coach->default, default-/american_football/football_coach_position/coaches_holding_this_position./american_football/football_historical_coach_position/team->default, default-/american_football/football_player/current_team./american_football/football_roster_position/position->default, ... (1340 more)\n", "\n", " Edge types:\n", " default-/award/award_nominee/award_nominations./award/award_nomination/award_nominee->default: [19764]\n", " default-/film/film/release_date_s./film/film_regional_release_date/film_release_region->default: [15837]\n", " default-/award/award_nominee/award_nominations./award/award_nomination/award->default: [14921]\n", " default-/award/award_category/nominees./award/award_nomination/award_nominee->default: [14921]\n", " default-/people/profession/people_with_this_profession->default: [14220]\n", " default-/people/person/profession->default: [14220]\n", " default-/film/film/starring./film/performance/actor->default: [11638]\n", " default-/film/actor/film./film/performance/film->default: [11638]\n", " default-/award/award_nominated_work/award_nominations./award/award_nomination/award->default: [11594]\n", " default-/award/award_category/nominees./award/award_nomination/nominated_for->default: [11594]\n", " default-/award/award_winner/awards_won./award/award_honor/award_winner->default: [10378]\n", " default-/film/film_genre/films_in_this_genre->default: [8946]\n", " default-/film/film/genre->default: [8946]\n", " default-/award/award_nominee/award_nominations./award/award_nomination/nominated_for->default: [7632]\n", " default-/award/award_nominated_work/award_nominations./award/award_nomination/award_nominee->default: [7632]\n", " default-/film/film_job/films_with_this_crew_job./film/film_crew_gig/film->default: [7400]\n", " default-/film/film/other_crew./film/film_crew_gig/film_crew_role->default: [7400]\n", " default-/common/topic/webpage./common/webpage/category->default: [7232]\n", " default-/common/annotation_category/annotations./common/webpage/topic->default: [7232]\n", " default-/music/genre/artists->default: [7229]\n", " ... (1325 more)\n" ] } ], "source": [ "print(fb15k_graph.info())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train a model" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "fb15k_gen = KGTripleGenerator(\n", " fb15k_graph, batch_size=len(fb15k_train) // 100 # ~100 batches per epoch\n", ")\n", "\n", "fb15k_complex = ComplEx(\n", " fb15k_gen,\n", " embedding_dimension=embedding_dimension,\n", " embeddings_regularizer=regularizers.l2(1e-8),\n", ")\n", "\n", "fb15k_inp, fb15k_out = fb15k_complex.in_out_tensors()\n", "\n", "fb15k_model = Model(inputs=fb15k_inp, outputs=fb15k_out)\n", "fb15k_model.compile(\n", " optimizer=optimizers.Adam(lr=0.001),\n", " loss=losses.BinaryCrossentropy(from_logits=True),\n", " metrics=[metrics.BinaryAccuracy(threshold=0.0)],\n", ")" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "fb15k_train_gen = fb15k_gen.flow(\n", " fb15k_train, negative_samples=negative_samples, shuffle=True\n", ")\n", "fb15k_valid_gen = fb15k_gen.flow(fb15k_valid, negative_samples=negative_samples)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train for 101 steps, validate for 11 steps\n", "Epoch 1/50\n", "101/101 [==============================] - 41s 401ms/step - loss: 0.6933 - binary_accuracy: 0.5003 - val_loss: 0.6933 - val_binary_accuracy: 0.5016\n", "Epoch 2/50\n", "101/101 [==============================] - 40s 394ms/step - loss: 0.6933 - binary_accuracy: 0.5035 - val_loss: 0.6933 - val_binary_accuracy: 0.5024\n", "Epoch 3/50\n", "101/101 [==============================] - 41s 402ms/step - loss: 0.6932 - binary_accuracy: 0.5107 - val_loss: 0.6932 - val_binary_accuracy: 0.5129\n", "Epoch 4/50\n", "101/101 [==============================] - 41s 408ms/step - loss: 0.6927 - binary_accuracy: 0.5552 - val_loss: 0.6911 - val_binary_accuracy: 0.6468\n", "Epoch 5/50\n", "101/101 [==============================] - 42s 416ms/step - loss: 0.6597 - binary_accuracy: 0.8393 - val_loss: 0.5786 - val_binary_accuracy: 0.9068\n", "Epoch 6/50\n", "101/101 [==============================] - 42s 417ms/step - loss: 0.4349 - binary_accuracy: 0.9086 - val_loss: 0.3232 - val_binary_accuracy: 0.9097\n", "Epoch 7/50\n", "101/101 [==============================] - 43s 424ms/step - loss: 0.2778 - binary_accuracy: 0.9140 - val_loss: 0.2465 - val_binary_accuracy: 0.9178\n", "Epoch 8/50\n", "101/101 [==============================] - 43s 429ms/step - loss: 0.2200 - binary_accuracy: 0.9232 - val_loss: 0.2014 - val_binary_accuracy: 0.9265\n", "Epoch 9/50\n", "101/101 [==============================] - 43s 423ms/step - loss: 0.1778 - binary_accuracy: 0.9333 - val_loss: 0.1643 - val_binary_accuracy: 0.9367\n", "Epoch 10/50\n", "101/101 [==============================] - 42s 411ms/step - loss: 0.1452 - binary_accuracy: 0.9443 - val_loss: 0.1389 - val_binary_accuracy: 0.9453\n", "Epoch 11/50\n", "101/101 [==============================] - 42s 412ms/step - loss: 0.1220 - binary_accuracy: 0.9530 - val_loss: 0.1215 - val_binary_accuracy: 0.9515\n", "Epoch 12/50\n", "101/101 [==============================] - 41s 407ms/step - loss: 0.1051 - binary_accuracy: 0.9599 - val_loss: 0.1091 - val_binary_accuracy: 0.9559\n", "Epoch 13/50\n", "101/101 [==============================] - 41s 405ms/step - loss: 0.0925 - binary_accuracy: 0.9653 - val_loss: 0.0995 - val_binary_accuracy: 0.9598\n", "Epoch 14/50\n", "101/101 [==============================] - 41s 404ms/step - loss: 0.0823 - binary_accuracy: 0.9699 - val_loss: 0.0922 - val_binary_accuracy: 0.9628\n", "Epoch 15/50\n", "101/101 [==============================] - 41s 408ms/step - loss: 0.0741 - binary_accuracy: 0.9736 - val_loss: 0.0862 - val_binary_accuracy: 0.9657\n", "Epoch 16/50\n", "101/101 [==============================] - 41s 405ms/step - loss: 0.0676 - binary_accuracy: 0.9765 - val_loss: 0.0820 - val_binary_accuracy: 0.9674\n", "Epoch 17/50\n", "101/101 [==============================] - 41s 408ms/step - loss: 0.0622 - binary_accuracy: 0.9788 - val_loss: 0.0789 - val_binary_accuracy: 0.9687\n", "Epoch 18/50\n", "101/101 [==============================] - 41s 406ms/step - loss: 0.0579 - binary_accuracy: 0.9805 - val_loss: 0.0755 - val_binary_accuracy: 0.9704\n", "Epoch 19/50\n", "101/101 [==============================] - 41s 407ms/step - loss: 0.0542 - binary_accuracy: 0.9818 - val_loss: 0.0736 - val_binary_accuracy: 0.9717\n", "Epoch 20/50\n", "101/101 [==============================] - 41s 410ms/step - loss: 0.0513 - binary_accuracy: 0.9829 - val_loss: 0.0718 - val_binary_accuracy: 0.9728\n", "Epoch 21/50\n", "101/101 [==============================] - 41s 411ms/step - loss: 0.0490 - binary_accuracy: 0.9837 - val_loss: 0.0709 - val_binary_accuracy: 0.9732\n", "Epoch 22/50\n", "101/101 [==============================] - 42s 413ms/step - loss: 0.0468 - binary_accuracy: 0.9844 - val_loss: 0.0697 - val_binary_accuracy: 0.9739\n", "Epoch 23/50\n", "101/101 [==============================] - 42s 414ms/step - loss: 0.0451 - binary_accuracy: 0.9850 - val_loss: 0.0694 - val_binary_accuracy: 0.9743\n", "Epoch 24/50\n", "101/101 [==============================] - 42s 412ms/step - loss: 0.0435 - binary_accuracy: 0.9855 - val_loss: 0.0693 - val_binary_accuracy: 0.9744\n", "Epoch 25/50\n", "101/101 [==============================] - 42s 417ms/step - loss: 0.0425 - binary_accuracy: 0.9859 - val_loss: 0.0686 - val_binary_accuracy: 0.9748\n", "Epoch 26/50\n", "101/101 [==============================] - 42s 414ms/step - loss: 0.0412 - binary_accuracy: 0.9863 - val_loss: 0.0683 - val_binary_accuracy: 0.9750\n", "Epoch 27/50\n", "101/101 [==============================] - 42s 413ms/step - loss: 0.0403 - binary_accuracy: 0.9866 - val_loss: 0.0689 - val_binary_accuracy: 0.9752\n", "Epoch 28/50\n", "101/101 [==============================] - 41s 411ms/step - loss: 0.0394 - binary_accuracy: 0.9868 - val_loss: 0.0681 - val_binary_accuracy: 0.9754\n", "Epoch 29/50\n", "101/101 [==============================] - 42s 416ms/step - loss: 0.0385 - binary_accuracy: 0.9871 - val_loss: 0.0690 - val_binary_accuracy: 0.9752\n", "Epoch 30/50\n", "101/101 [==============================] - 42s 415ms/step - loss: 0.0378 - binary_accuracy: 0.9874 - val_loss: 0.0694 - val_binary_accuracy: 0.9757\n", "Epoch 31/50\n", "101/101 [==============================] - 42s 414ms/step - loss: 0.0373 - binary_accuracy: 0.9875 - val_loss: 0.0696 - val_binary_accuracy: 0.9754\n", "Epoch 32/50\n", "101/101 [==============================] - 42s 415ms/step - loss: 0.0367 - binary_accuracy: 0.9877 - val_loss: 0.0696 - val_binary_accuracy: 0.9756\n", "Epoch 33/50\n", "101/101 [==============================] - 42s 416ms/step - loss: 0.0364 - binary_accuracy: 0.9878 - val_loss: 0.0704 - val_binary_accuracy: 0.9755\n", "Epoch 34/50\n", "101/101 [==============================] - 42s 414ms/step - loss: 0.0359 - binary_accuracy: 0.9879 - val_loss: 0.0707 - val_binary_accuracy: 0.9755\n", "Epoch 35/50\n", "101/101 [==============================] - 42s 413ms/step - loss: 0.0354 - binary_accuracy: 0.9881 - val_loss: 0.0713 - val_binary_accuracy: 0.9759\n", "Epoch 36/50\n", "101/101 [==============================] - 42s 415ms/step - loss: 0.0350 - binary_accuracy: 0.9882 - val_loss: 0.0714 - val_binary_accuracy: 0.9756\n", "Epoch 37/50\n", "101/101 [==============================] - 42s 412ms/step - loss: 0.0347 - binary_accuracy: 0.9883 - val_loss: 0.0718 - val_binary_accuracy: 0.9758\n", "Epoch 38/50\n", "101/101 [==============================] - 42s 416ms/step - loss: 0.0343 - binary_accuracy: 0.9883 - val_loss: 0.0724 - val_binary_accuracy: 0.9754\n" ] } ], "source": [ "fb15k_es = callbacks.EarlyStopping(monitor=\"val_loss\", patience=10)\n", "fb15k_history = fb15k_model.fit(\n", " fb15k_train_gen, validation_data=fb15k_valid_gen, epochs=epochs, callbacks=[fb15k_es]\n", ")" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAI4CAYAAACV/7uiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzde3ycZZ338c9vJuekOTVJS9OkaUspFORYykEFOYqI4B44iCKwKl0FQXSfRw6KyLOorLpaFVwB5aAiCosrrnVZQRRFxLacCi2FppQ2PSVN0rSZHCYzcz1/3JPJZDIpHZrMZGa+79drXjP3NffM/CaBfHtd931flznnEBERkeziy3QBIiIikjoFuIiISBZSgIuIiGQhBbiIiEgWUoCLiIhkoYJMF7C/6urqXEtLS6bLEBERmRSrVq3a6ZyrT2zP+gBvaWlh5cqVmS5DRERkUpjZm8naNYQuIiKShdIW4Gb2IzNrN7OXx3nezOw7ZrbezF4ys6PTVZuIiEi2SWcP/F7grL08/z5gQfR2BfD9NNQkIiKSldIW4M65p4CuvexyHnC/8/wVqDazA9JTnYiISHaZSsfAG4HNcdtt0bYxzOwKM1tpZis7OjrSUpyIiMhUMpUCfJ855+50zi12zi2urx9zZr2IiEjOm0oBvgVoitueHW0TERGRBFMpwB8FPho9G/14oMc5ty3TRYmIiExFaZvIxcx+BrwHqDOzNuBLQCGAc+4/gOXA2cB6oA+4PF21iYiIZJu0Bbhz7kNv8bwDrkxTOSIik845RzjiCEVG7kPhCGHncA4izhFxEInEb3ttLnofjnht4Ygj7ByRSPxjkrR59yOvg3Ak4t3H7RtxLlqj97ku7vFw7d52fF3e+7m4elxcjZHotnPgiD6Ovi94bcTaRj4zfpv418Y9jiT9+Xg/g4gb/fMb/Znehov9ThLf33uvsT+LkXqGfyax90v8TnHbAF//xyM4Yf70CfgvaO+yfipVEck9kYgjGI4wOBQhGI4wFI4QDEXvwxGGwm70dshrG4ruOxyUwbB3H4rEPRd2DIUd4UiEoYgjHHYMRbz2cHS/cMR5z0Ui0X2HQ3hkP2+fCOHw2ICO384VZuA3w2eGzwc+M2/bZ/gM/D7DzLDovoZF78HMRr1PsucNIGE7fr/hz/Wb9zm+4Tbznvf7jEKfxfYn7nNH3mukfbiNuDqG32v4c4l+xsi+xN4v8f3jH1SVFk7Uj32vFOAisk+cc/QPhQkMhgkMhggEQ7HHfcEwA0NhBkJhBoYiDA7fD4UZDEW85+IeJ7/3Xjcc2pPB7zMKfEah30eB33tc4It77PdF76Pt0cdFhf7Ya/3R18dvF/i9IIlvL/Ab/uh7eK8Z2fb5ouFnI6ERH4yjAyoalnGv8/ss1ub3xT0fa4t7bIbfPxy2xF5vcZ87HFC+4UCKBpjPiNUSH8IyNSjARfJAfzBMd1+QXX1D7BkYoncwxJ6BEHsGQ/QOhOgdHKJ31PbILTAYom8wTCAYItUOZXGBj+ICHyWFfkoK/d52oY+SAj9lRQXUlPni2v2xfYfviwp83s3vhWOhf3jbF902igp8secK/fHh7O1X4PfCtNDnw+dTCEnuUICLZBnnHD39Q7TvGaR99yDtewboCgTp7gvS3TfErr4g3YGhWGB39wUZDO29R+v3GRXFBVQUFzCtxLuvKSuiqbaM8iI/5dHnyooKqCj2tsuLCygvKqA8ul1S4Kek0EdxgZ/iQi+41WsTmTwKcJEpZGAozJZd/Wzu6mN7z4AX0nsGokE9SEf0lmyI2e8zqksLqS4rpKasiNk1ZbyjsZCa8qJYW01ZIZUlhVREQ7qipIBpxYWUFCpsJcs5By76/4X5Rh+0zlEKcJE0Ckcc23r62dzVz+buPtq6+tjc7QX25u4+duweHPOamrJC6qcV0zCthHl15dRXeo8bphV7t8oSasuLqCwpyP4Qdg7CQxAagNAghAchEoJI2GuPhMbewkPe8y4c90c8ep9sG5K/PtaWsO0ice/l4t4r2ftHIDL8XDiuLe7xqO3w6OeStePA/ODzJ9z7xra7SPTnNuT97MJBCAW9++Hb8PNm3mt8BXG3JNvmj/5u4r7veLfYOdrRM8HG3Cc8N/KLH/n9J7YNf/bwfwejfv8J26PYSP3mS/i5+RK+V9zvk3G+o8X9bPwFCT+n4fZC73POug3mnPA2/yfYdwpwkUmwe2CI1vZeWjsCrG/vpbXDu23u6mMoPPKHyWdwQFUps2tKefeCeppqymiqLaWptoxZ1aXUVRRRXOBP/xeIRGAoAIO9EIzeYo8DMLjHuw/2eo+H+sf5A5ukLT6gQwNxt+j2cC8q0ywuILGRXl2sd5fQFguM6OssftsXFySJ23EhU1A0th3iAj0u2EPBhPaI997+IvAXQ1EFFBR7oeIv9toLiqLPF3nv+1a/q+Ht2HeN/xkku1lcCCbcQ5K2+FO74wI+WdtwOI4Jzri24Z974s9q+Ocz/I+q4edJ/C7jfE8s7h8R8f8tx2/H/cOvsGSi/2tMSgEush86ewdZu20PrR29saBe395L+56RnnSh32iZXs7CGdN476Ezaa4tiwX1AVWlFBVMwoSI4SHo64L+ruh9t/e4vxsGdnuhO7gHgntGHg/2jm7fV4VlUFgKvsLxe3GJvZSSSigoid6KvfvC+O3SaPgURf9wD79PIeP2gob/eMcHarKwxcbvQY26TaWJKkXGUoCL7KPO3kFWb+nh5S09vNTm3W/tGYg9P624gPkNFZx0UD3z6ys4sKGC+fXlNNeWUeBPMQyc83q3A7thoAcGo/cDu2GwZ+Tx8HP93XGB3b33ADY/FFdAcSUUT/NuJdVQ1RTdrvSeL6qI3k+L3pd7bbH26LYvAyMEIqIAF0lmOKxXt/XEQjs+rOfWlXNMSy2XNVZy6KwqFjRUUD+tePxj0AO7oWcz7NkG/buiPeLuuN5xkpsL771If5EXtiWVUFoD5fVQvxBKa6Gs1msrq43bjrYVlefFCT4iuU4BLnnPOUdrRy8rN3azYmM3K9/s4s3Ovtjz8+rKWdxSyzsaqzissYpDGyupLEmYaWlgN7S3wq5Ncbc3Rx73dyf/8OJKKK32grW0BqoaRx6XVHvhXFIVDerodnG0LU3H2URkalKAS94ZDIV5eUuPF9Ybu1n1ZhfdfUMATC8v4pg5NVy8pJnDZ1ePhHUo6PWgu5+H1W9C90YvoLvf9O4TA7qgFGrmQHUzzD7Wu69uhmkHjPSES6u947siIm+DAlxyXjjieG5TN0++2s6KjV282NZDMDqxyby6cs5YNIPFc2pZ3FTBXN8OrGMtdDwBL22Ep6JhvXsroy5r8Rd5gVzTAo1HQ3U0rIfvy+s0TC0ik0oBLjlpz8AQT722kyfW7uDJde109w1R4DMOa6zisuNn8+76Po4o2krlnjXQvhZWvAq/fc27FGTYtFleL3ruSV4w17REe9VzvJ60zlIWkQxSgEvO2NzVx+Nrd/DE2naefaOTobCjqTTIx5p2cWrVdg50b1DU+Sq8sM673nhYdTPUHwILzoCGQ7xb3UHepVEiIlOUAlyylnOOFzbv4ndrdvDEmh10t2/mUN9GTq3YyhcattAy1EpJ72bYFH1BxQyYcSgs/lg0qBdB/UHepVMiIllGAS5ZZ3vPAL9cuYHXVjzBvN6VHO/bwBUFm6gu2eXtEAQq5kHzMXDA5TDzCDjgcKhoyGjdIiITSQEuWWEgGOKZZ59m66rf0tj5Fy71vUqZDRIp9OPqD8E/6/0w83AvqGcc5l1uJSKSwxTgMmW5PTvYtHI5nS89RmP3XzkF71KtrvJmQgs+BIe+F1/LuxTWIpKXFOAytXS8RmDlT+h/5bfU9b7GHKDSTeONysXsXnQm848/h9qa5kxXKSKScQpwyby+Lnj5Pxlc9VOKdzxPsfPxYuRg/qfyUuqOfB8nvutUji4tznSVIiJTigJcMiM8BOufgBcfIPLqb/FFgmyINPNf7hLcO/6RC96zmBMbKjJdpYjIlKUAl/Ta/jK88ACs/gUEOtjtq+bh4Kn82k7hmONO4hMnzWNGpeb4FhF5KwpwmXzBgBfaz90H21cT8RWyqvg4fhD8KM8VHcNHTl7Aj05soaa8KNOViohkDQW4TJ6eLfC3O2HVPTDQw+6aw/j5tE9xe8cRFPjq+Ph75/Kt45qZlriyl4iIvCUFuEy8Lc/BM7fDmv/CuQhtM8/g3/eczi+3NdJYXcrnzpvH+YubKCn0Z7pSEZGspQCXiREJw7rlXnBveoZIUQXPzbiAL+14F6+8Uc3BM6fxjfPncd6Rsyj0axEQEZH9pQCX/TO4B57/KTz7fejeSLCiid/MuIpb2o5i155STj9kBl9451yOn1eLaXlNEZEJowCX1DkH216AF34GLz4Igz10TT+KH1V/mO9vP5jSPcWcf/xsLjuxhTnTyzNdrYhITlKAy77bs8O7/OuFB6B9Dc5fTOv093DbrlP53ZYmmmpLueGcuVyweLZOTBMRmWQKcNm70CCs+60X2usfBxemt/4ofjvzs/xb2yI6NpVx/LxafnDeXE4/ZAZ+n4bJRUTSQQEuYzkHW5+LTrjyMAzsIlIxk5fnXMp3di7m8c3VVBQX8MHFs7h4yRwWzdJiIiIi6aYAlxG7NnmB/dLPoeNVXEEJ3c1n8nD4JL69YRZ9O+GI2VV87e+b+cARsygv1n8+IiKZor/A+S7QCWt+6QX3pmcACDUuYcWiL/KNtkWsWuMoL/LzwaMbuXhJM4c1VmW4YBERAQV4fgoGvOPaL/0CWp+ASIi+qgW8OOdTPDR0PL/dVEz/UJh3NFby1Whvu0K9bRGRKUV/lfNFeAg2/AFe+gXu1f/GhvrYXdjA48Xn8aPdx/Lyjias3TioYRoXLK7lH49p4h2z1dsWEZmqFOC5JBSE3W3esezuN737XZtwuzYxtH0tRUM97KGcX4eO41fhd/FC6BCOmF3LSUfU8LmWWo5urqGqTJd/iYhkAwV4tnIO/noHbHsxFtTs3gq4kX3MD1WNdBXO5Hf9R/HXwiUMzDmFI+fO4P+21HBYYxXFBZqPXEQkG6U1wM3sLGAZ4Afuds59LeH5OcCPgHqgC/iIc64tnTVmjZ42eOwGKG+AuoNg7slQMweqm0du02aBv4CbHniOv3Z18uwNp1GgechFRHJC2gLczPzA7cAZQBuwwswedc6tidvtG8D9zrn7zOxU4KvAJemqMav0tnv3534XFp417m79wTC/X9vO3x3dqPAWEckh6fyLvgRY75zb4JwLAg8C5yXsswj4ffTxk0mel2GBaIBX1O91tyfXtdM/FOacdxyQhqJERCRd0hngjcDmuO22aFu8F4G/jz7+O2CamU1PfCMzu8LMVprZyo6Ojkkpdsob7oGX7z3Af7N6G9PLi1gytzYNRYmISLpMtTHVfwFONrPngZOBLUA4cSfn3J3OucXOucX19XsPsJwViP7Dpbxh3F2Gh8/fe9hMDZ+LiOSYdJ7EtgVoitueHW2Lcc5tJdoDN7MK4B+cc7vSVmE2CXRAcSUUloy7yx+iw+fv1/C5iEjOSWe3bAWwwMzmmlkRcBHwaPwOZlZnZsM1XY93Rrok09sO5XV73eU3q7dRW17EcRo+FxHJOWkLcOdcCLgKeAxYC/zCOfeKmd1iZudGd3sPsM7MXgNmALemq76sE+jY6/D5wFCY37/aznsP1fC5iEguSut14M655cDyhLab4h4/DDyczpqyVqAD6haM+/Qf1rXTF9TwuYhIrlLXLFv1tu/1DPTfrN5ObXkRx8/T8LmISC5SgGejcAj6u8YdQh8YCvPE2h2899AZGj4XEclR+uuejfp2evfjTOIyMnw+K41FiYhIOinAs1FsEpfkPXANn4uI5D4FeDYKjD8Lm4bPRUTyg/7CZ6PA8BD62B74H9Z10BcMc7bOPhcRyWkK8Gy0l3nQl6/eRk1ZISfMGzOFvIiI5BAFeDYKtIO/GIqnjWoeGT7X5C0iIrlOf+WzUWCnN3xuNqr5j691ENDwuYhIXlCAZ6NxJnFZvnob1WWFnDBfw+ciIrlOAZ6NAmMDfGAozONrdvDeRTMp1PC5iEjO01/6bBTYOWYSl+Hh8/cfruFzEZF8oADPNpFI0pXINHwuIpJfUgpwM7vLzBZPVjGyDwZ2QSQ0agjdO/u8XcPnIiJ5JNW/9vOBZ83seTP7pJlVTkZRsheBDu8+bhKXp17roHcwxNkaPhcRyRspBbhz7lRgIfA/wBeBrWZ2j5mdMBnFSRJJJnEZHj4/UcPnIiJ5I+XxVufceufc9UAT8BGgHvijma02s6vMrGKii5Q4w/OgR3vgA0NhHl/bzpmLZmj4XEQkj+zPX/wIEIreGzAEXAdsMrPzJqA2SaY3OoQe7YH/6fWd3vC5Jm8REckrKQe4mc0xs1uATcADwHbgBOfc0UAL8C3g9oksUuIEOsD8UOotFfqbl7ZSVVrIOw+sy3BhIiKSTqmehf4Y0AqcB3wVaHTOXeGcWwngnAsBPwBmTXShEhVoh/I68Pliw+fvPVTD5yIi+aYgxf23Au9yzv11L/t0AHPffkmyV70dGj4XEZHUAtw5d/k+7OOAN992RbJ3gZEAX756m4bPRUTyVKpD6MvM7Jok7Veb2b9PXFkyrkA7VDQQDEV4fM0OnX0uIpKnUv3L/w/AM0nanwH+cf/Lkb1yLjaEvq2nnz2DIZbMrc10VSIikgGpBngd3jHuRJ1AQ5J2mUjBAIT6obyezkAQgLqK4gwXJSIimZBqgLcByWZdOwHvBDeZTHGTuHT1egFeW16UwYJERCRTUj0L/cfAv5tZP/C7aNuZwDfxLh+TyRSbxKWBrl0KcBGRfJZqgP8rMA/4T8BF2wz4KXDLBNYlyQz3wMvr6NqqABcRyWepXkYWBi41s/8HHBVtfs451zrhlclYcSuRdQV6KC7wUVbkz2xNIiKSEan2wAFvQRNg/QTXIm9leAi9rI7O3g6mlxdhZpmtSUREMiLlADezA4HzgTnAqPFb59w/TVBdkkygHUqqoaCIrsAgtRUaPhcRyVcpBbiZvRf4FfAqsAh4Ee+YuA9YMeHVyWiBjtgyol19Q9SUKcBFRPJVqpeR/T/g35xzRwKDwIVAM/AU8MgE1yaJejugPBrggUGm6wQ2EZG8lWqAHwLcH30cAkqdcwHgS8D/ncjCJInhlciArt4gteWaxEVEJF+lGuB9jAy7b8db/xu8MJ8xQTXJeKJD6ANDYQLBMNN1DFxEJG+lehLbKmAJ3jHwJ4GvmNls4MPA8xNcm8QLDcJAjzeJS3QaVR0DFxHJX6n2wG9kZKnQm/CmVv06UAosncC6JNHwNeDldbEA1yQuIiL5a5974GbmwxtCfw3AObcTeP8k1SWJRk3i4gW4htBFRPJXKj1wB6wGDni7H2ZmZ5nZOjNbb2bXJXm+2cyeNLPnzewlMzv77X5WzomfB109cBGRvLfPAe6cc0ArUPN2PsjM/MDtwPvwriH/kJktStjtC8AvnHNHARcBd7ydz8pJsZXIRpYSrdUxcBGRvJXqMfAvAf9mZo1v47OWAOudcxucc0HgQeC8hH0cUBl9XIWWKB3RO7yQST3dgSB+n1FVWpjZmkREJGNSPQv9VmAW8KaZ7QAC8U865w7ay2sbgc1x223AcQn73Az8r5l9GigHTk/2RmZ2BXAFQHNzcwrlZ7HATigsh6JyOgNBasoK8fk0D7qISL5KNcB/MilVjPgQcK9z7ptmdgLwYzM7zDkXid/JOXcncCfA4sWLXZL3yT2BdqioB7xZ2HT8W0Qkv6W6nOiX9+OztgBNcduzo23xPgacFf2sZ8ysBKgD2vfjc3NDbzuUDwd4UAEuIpLnUj0Gvj9WAAvMbK6ZFeGdpPZowj6bgNMAzOwQoAToSGONU1dgZ2we9E4FuIhI3kspwM0sYmbh8W57e61zLgRcBTwGrMU72/wVM7vFzM6N7vY54BNm9iLwM+Cy6NnvEjeE3q0AFxHJe6keA/8o3pniwwqBY/DWB3/L4XXn3HJgeULbTXGP1wDvTLGm3BcJQ18nlNcTjjh29Q9pIRMRkTyX6jHwZCex3RvtMZ8CfH9CqpLR+rrARaC8ge6+IM6hpURFRPLcRB0D/z3wgQl6L0kUN4lLbCETBbiISF6bqAA/C+iZoPeSRHGTuMTmQVeAi4jktZSG0M3sfxOb8CZ2ORhvGlSZDIGd3n15A13bNQ+6iIikfhJb4nXbEWAlcLVz7omJKUnGGDUP+m5APXARkXyX6klsl09WIbIXve3gK4SSarp6vd54tRYyERHJa6leBz7XzMbMd25mC8ysZaKKkgSBDm8WNjO6+4JMKymgqCCdc/CIiMhUk2oK/Ijk12mfCPxw/8uRpAIdsUlcOgNBDZ+LiEjKAX4U8HSS9meAo/e/HEmqtz02jaoWMhEREUg9wAuBZFOAFQNKlckyPIQOdPYGNQubiIikHOCrgMuStP8T8MJ+VyNjOTdqCN1biawww0WJiEimpXoZ2S3A/5jZgcDvom1nAu8Dzp7IwiRqoAfCQShvwDlHd5964CIikmIP3Dn3ON6sazXAbdFbNXC2c+53e3utvE2B6Gqq5fXsGQwxFHY6iU1ERFLugQ+H+OOTUIskMxzgFfV09WoWNhER8aR6HfixZnZckvbjzGzxxJUlMbF50BvoDCjARUTEk+pJbN8FWpK0NwHf2e9qZKxYD7yBbgW4iIhEpRrgh+LNfZ7ouehzMtECHYBBaW1sJTIFuIiIpBrgEaAySXvN23gv2Re97VA2HfwFsSH06RUKcBGRfJdq6P4F+FyS9s/hzcYmEy3QARUjs7AVF/goLfRnuCgREcm0VM9C/wLwRzN7HhhePvQ0YAHwngmsS4b1tkN5HQBdgSGmlxdhZhkuSkREMi3V68BXAccBL+NN3vI+YDVwvHMu2bFx2V+BjtHzoGv4XEREeHvXgb8CXDIJtUgyo4bQNQubiIh4Ug7wYWY2k4QFTJxzm/a7IhkR7INgb2wIvTMQZF59RYaLEhGRqSClADezSmAZcBHJVx/T2VUTKTaN6kgPvKZMQ+giIpL6Wei34R0D/xAwgLcy2ReBrcDFE1qZjJrEZWAoTF8wrEvIREQESH0I/f3Apc65J80sAjzjnPuxmbXhHRf/+YRXmM9i06jWaRIXEREZJdUe+HSgNfp4N94ELgB/Ak6eqKIkKm4IXQEuIiLxUg3wN4HZ0cfrgXOij08BeieqKIkKDPfA67WQiYiIjJJqgD/CyIQty4AbzWwbcGf0JhOptwOKq6CwRAuZiIjIKCkdA3fOfSHu8SNmdiLwLmCdc+43E11c3gt0jLqEDGC6AlxERNiP68ABnHN/A/6W2G5mvwE+7pzbtj/vn/cS5kH3+4zKksIMFyUiIlPBZK0gdhJQOknvnT9626G8Hhi+BrwQn0/zoIuIiJYAndoCowNcx79FRGSYAnyqCg9Bf3fCPOgKcBER8SjAp6rATu8+2gPvDASZroVMREQkSgE+VcVdAw7qgYuIyGgK8Kkqbh70UDjCrr4hahTgIiISNVkB/iegf5LeOz/0Dk+jWs+u/iFA14CLiMiIlALczH5nZv9oZnu9ftw5d3aya8DN7CwzW2dm683suiTPf8vMXojeXjOzXanUl1OGh9ArNA+6iIiMlWoPfDNwD7DFzG4zswP39YVm5gduB94HLAI+ZGaL4vdxzl3rnDvSOXck8F28qVvzU6ADCkqgqILOXs3CJiIio6UU4M65fwIOAG4CTgXWmdmTZnaRmb3VFGFLgPXOuQ3OuSDwIHDeXvb/EPCzVOrLKb0dUN4AZrEeuI6Bi4jIsJSPgTvnep1zP3DOHQscDbyC1yvfambfMLOWcV7aiNeDH9YWbRvDzOYAc4Hfp1pfzgi0Q0X0DPQ+9cBFRGS0t30Sm5lVAe/GmzbVj3fi2ruB18xs6X7WdRHwsHMuPM5nX2FmK81sZUdHx35+1BQV6Bi5hKxXPXARERkt5QA3s3eb2f3AVuBfgJ8Dzc65v3fOHQdcAdya5KVbgKa47dnRtmQuYi/D5865O51zi51zi+vr61P9CtmhNy7AA4NUlhRQ6NdVfyIi4kn1LPRXgSeAKuB8YK5z7lbn3Pa43X4N1CZ5+QpggZnNNbMivJB+NMlnHAzUAM+kUltOiURGrUTWqUlcREQkQarLif4MuNs5N17PGedcJ0n+YeCcC5nZVcBjeEPuP3LOvWJmtwArnXPDYX4R8KBzzqVYW+7o7wYX1ixsIiIyrn0O8OhZ5kuBhxl/6HuvnHPLgeUJbTclbN/8dt47pwRGJnEBL8Bn15RlsCAREZlq9nkI3Tk3FH2Y9MQymUBxk7iAF+A6A11EROKlelbU3cDVk1GIxOkdWcjEOUd3X5DaCgW4iIiMSPUY+CzgfDM7FVgFBOKfdM5dMVGF5bXYUqIN7B4IMRR21JYpwEVEZESqAT4feC76eFbCc/l70tlEC7SD+aG0hu4ub00YncQmIiLxUgpw59wpk1WIxOlt905g8/noHF7IREPoIiISRzODTEWBnaPOQAdNoyoiIqOlOoSOmZ0MXAzMAUalinPu1AmqK7/Fz4MeGASgRsfARUQkTqozsX0E+B0wAzgF2A3MxFvUpHXCq8tXwyuRAV0B7+q96RpCFxGROKkOof9f4Frn3AeBIPBZ4FDgIbzVxWR/ORddyKQO8HrgJYU+yopSHiwREZEclmqAz2dkJrUgUB6d8vRbeIuYyP4K9kKof9Q86NPLizNclIiITDWpBvguoDz6eBtwUPRxOVA5UUXltdgkLiOzsNWUF2awIBERmYpSHZf9K9763y8D/w18y8wWA+cBf57g2vJTknnQa9UDFxGRBKkG+OeAiujjLwPTgPcDrwDXTmBd+Ws4wCtGAnx+fcVeXiAiIvko1YlcNsY97geunOiC8l6SIXTNwiYiIok0kctUExtCr2NgKExfMKwAFxGRMVK9DrzezO4zsy1mFjKzcPxtsorMK4EOKK0Bf+HINKoKcBERSZDqMfC7gXcAy4CtaAGTidfbHhs+71aAi4jIOFIN8PcAZzrnnp2EWgS8HnjcNeCgedBFRGSsVI+BdwN7JqMQiUqYhZNX4o8AACAASURBVA3UAxcRkbFSDfBbgRvNTPN6TgbnYM92qJgBQGevhtBFRCS5VIP4AuBYoM3MXsWbTjXGOXfmRBWWl3Zv8aZSrVsAQHdfEL/PqCzRTGwiIjJaqgHehhYtmTzta737hkVAdBrVsiJ8PstgUSIiMhWlOpHL5ZNViADta7z7hkMAbwhdJ7CJiEgymshlKmlfC9NmedeBo4VMRERkfG/ZAzez/wXOd871RB+PS8fA91P7mljvG7wAP+QALfImIiJj7csQ+hYgEvdYJkMkDB3r4Nh3x5q6+jQPuoiIJPeWAR5/3Hv4sZmVAvOjza3RhU1kf3RvhNBA7AS2UDjCrr4hBbiIiCSV6lzoRWb2TaATeDF66zSzfzczLVq9P3a84t1Hh9C7+4YAmF6hABcRkbFSvYzsu8AHgWuAp6Nt7wRuAcqBpRNXWp5pXwsY1C8EvOPfADVlCnARERkr1QC/CLjYOfebuLY1ZrYVeAAF+NvXvgZqWqCoHBgJcF1GJiIiyaR6GVkQWJ+kvRUY2v9y8lj72tjxbxgJ8FoNoYuISBKpBvjdwGfNLDY1WPTx1cAPJ7KwvBIahM71CZeQaSETEREZ375cB35n3KYPOB84w8z+Fm07FpgOPDzx5eWJna+DC48K8E4dAxcRkb3Yl2PgCxK2n4vez4jeb4re5k1UUXknYQ508IbQK0sKKPRrsjwRERlrX64DPyUdheS19jXgK4DpB8aaugJBplfoyjwREUlO3bupoH0t1B0EBSPD5V0BzcImIiLjU4BPBQlzoMPIUqIiIiLJKMAzbbAXdr05JsA7A1pKVERExpfWADezs8xsnZmtN7PrxtnnAjNbY2avmNkD6awvIzrWefdxJ7A55+gOBHUNuIiIjCvVmdjeNjPzA7cDZwBtwAoze9Q5tyZunwXA9cA7nXPdZtaQrvoypn30HOgAuwdChCJOPXARERlXOnvgS4D1zrkNzrkg8CBwXsI+nwBud851Azjn2tNYX2a0r4WCUqhuiTVpHnQREXkr6QzwRmBz3HZbtC3eQcBBZva0mf3VzM5KW3WZ0r4GGg4G38ivIjYLm4bQRURkHFPtJLYCvIlj3gN8CLjLzKoTdzKzK8xspZmt7OjoSHOJEyxhDnSArkB0KVENoYuIyDjSGeBbgKa47dnRtnhtwKPOuSHn3BvAa4ydCQ7n3J3OucXOucX19fWTVvCkC3RC744kl5BpHnQREdm7dAb4CmCBmc01syK8pUkfTdjnv/B635hZHd6Q+oY01pheHcNTqI69hAxgerlmYhMRkeTSFuDOuRBwFfAYsBb4hXPuFTO7xczOje72GNBpZmuAJ4H/45zrTFeNaZdkDnSArt4gJYU+Sov8GShKRESyQdouIwNwzi0Hlie03RT32AGfjd5yX/saKKmCaQeMau4KBNX7FhEikQhtbW0EAoFMlyKTpLy8nNmzZ+Pzpd6fTmuAS4L2tdBwKIwsrw5AV5/mQRcR2LlzJ2bGwoUL39YfeJnaIpEIW7ZsYefOnTQ0pD7tif6LyBTnks6BDlrIREQ8u3btYsaMGQrvHOXz+ZgxYwY9PT1v7/UTXI/sqz3bYKAnaYB39irARQTC4TCFhYWZLkMmUWFhIaFQ6G29VgGeKe3RGWQTTmAD9cBFZIQlHGKT3LI/v18FeKa0J7+ErD8Ypn8orAAXEZG9UoBnyo41UDETympHNXf1DV8DrgAXEZHxKcAzZbwT2HqjC5kowEVEALj33nspKNBFU4kU4JkQCXvrgCc5/t0ZnUZVPXARyWann346l1122YS814UXXsiWLYkzb4v+SZMJ3Rsh1J+0B94dHULXMXARyXXBYJCiorf+W1daWkppaWkaKsou6oFnwjhTqIJ3CRloHnQRyV6XXXYZTzzxBPfddx9mhplx7733Ymb89Kc/5eyzz6a8vJwvfvGLOOf4xCc+wfz58yktLWXevHnccMMNDA4Oxt4vcQh9ePvpp5/m6KOPpqysjGOOOYYVK1Zk4utmjHrgmTAc4PULxzzVFQji9xmVpfrViMhoX/71K6zZujvtn7toViVf+sCh+7z/smXL2LBhAwcccADLli0DYPdur+7Pf/7z3Hbbbdx+++0AOOdoaGjggQceYMaMGbz00kssXbqUwsJCvvzlL4/7GZFIhOuvv55ly5ZRX1/PtddeywUXXMDrr7+eN8fL8+NbTjXta6B6DhRXjHmqKxCkpqxI136KSNaqqqqiqKiI0tJSZs6cCcDAwAAAS5cu5cMf/vCo/W+99dbY45aWFlpbW7njjjv2GuDOOb797W9z9NFHA3DzzTdz/PHH09raysKFYztHuUgBngnta2FG8n/NdgaCOoFNRJJKpRc8VS1ZsmRM21133cXdd9/Nxo0bCQQChEIhIpHIXt/HzDjiiCNi27NmzQJgx44deRPgOgaebqEgdL6e9AQ2gG7NwiYiOay8vHzU9kMPPcSVV17JhRdeyPLly3n++ee56aabGBoa2uv7+Hw+/P6RJZeHRy3fKvhziXrg6da5HiKhpCewgTeEfsisyjQXJSIysYqKigiHw2+531NPPcVRRx3FZz87sor0xo0bJ7Gy3KEeeLrF5kAf2wPvC4bYvnuAOvXARSTLzZ07l1WrVtHa2srOnTvH7VEvXLiQ1atX86tf/YrW1laWLVvGI488kuZqs5MCPN3a14KvAKYvGPPUvX/ZSF8wzLlHzspAYSIiE+dzn/scdXV1HHHEEdTX1/P0008n3W/p0qVccsklXH755Rx11FE8++yz3HzzzektNkuZcy7TNeyXxYsXu5UrV2a6jH33s4uhqxWufHZUc0//ECf925Mc3VzNPZePPclDRPLP2rVrOeSQ5OfLSO54q9+zma1yzi1ObFcPPN3aX0k6fP7DP79BT/8QnzszP86eFBGR/aMAT6dgwJtGNeEEtq5AkB/+aQPvO2wmhzVWZaY2ERHJKgrwdOp41btP6IH/xx9b6RsK89kzDspAUSIiko0U4OmUZA709t0D3PeXjfzdkY0smDEtQ4WJiEi2UYCnU/taKCiBmpZY0/eeXE844rjm9LFnpYuIiIxHAZ5O7Wu8BUx83uxBm7v6+NnfNnH+4ibmTC9/ixeLiIiMUICnU/taaBiZy/g7T7yOmXH1aQdmsCgREclGCvB06euCPdtiJ7Bt6OjlP59r4yPHzeGAKi1ULyIiqVGAp0vsDHTvBLZvPf46xQV+PnXK/AwWJSIi2UoBni5xc6Cv3babX7+4lcvf2UJdRXFm6xIRmYLuvfdeCgpG1tv6wx/+gJnR1ta219eZGT/5yU/2+/Mvu+wyTj/99P1+n8mkAE+X9rVQXAWVs/jm/77GtJIClp6k3reIyL448cQT2bZtW2zd74nyk5/8JLYUabxly5bx0EMPTehnTTQtJ5ou7Wuh4RBeaOvh8bU7+NwZB1FVVpjpqkREskJRUREzZ85M2+dVVU39WTHVA08H57wh9IZD+Ob/rqO2vIjL3zU301WJiEyKu+66i6qqKgYGBka133bbbTQ3NxMOh/nEJz7B/PnzKS0tZd68edxwww0MDg6O+57JhtCffPJJDj/8cEpKSjj88MN58sknx7zuxhtv5JBDDqGsrIympib++Z//mZ6enth7XnLJJYA39G5mXHbZZcDYIXTnHN/4xjeYN28eRUVFzJ8/n29/+9ujPqulpYWbbrqJa665htraWmbMmMG1115LKBRK7Qe4j9QDT4c926G/mzf8c/jT6zu58exDqCjWj15EUvTb62D76vR/7sx3wPu+ts+7X3DBBVx99dX86le/4sILL4y133///XzkIx/BzGhoaOCBBx5gxowZvPTSSyxdupTCwkK+/OUv79NnbN26lXPOOYcLLriABx98kC1btnDNNdeM2a+0tJQ777yTpqYmWltbufLKK7n66qu57777OPHEE/ne977HVVddxbZt22L7J3PHHXfwxS9+kWXLlnHKKafwxBNP8JnPfIZp06bxsY99LLbfd7/7XT7/+c/z7LPP8vzzz/PhD3+Yww47bNQ+E0Upkg7RE9jufb2UGZXFXHLCnAwXJCIyeaqqqjjvvPO4//77YwG+cuVK1qxZwyOPPILP5+PWW2+N7d/S0kJrayt33HHHPgf4HXfcQV1dHXfddRcFBQUsWrSIr3zlK3zgAx8Ytd8XvvCFUZ/z1a9+lYsuuoh77rmHoqKi2FD5Ww3Pf+1rX+PTn/40V1xxBQALFixg3bp13HrrraPC+d3vfjfXXXddbJ977rmHxx9/XAGetaJzoP96WxXXfnABJYX+DBckIlkphV5wpl166aWce+65tLe309DQwP3338+SJUtYuNBbMvmuu+7i7rvvZuPGjQQCAUKhEJFIZJ/ff82aNSxZsmTUmervete7xuz3yCOP8O1vf5v169eze/duIpEIwWCQ7du37/MJcbt376atrY2TTjppVPvJJ5/MsmXL6Ovro6ysDIAjjzxy1D6zZs3ijTfe2OfvlQodA08D176GbqumrGYGFy5uynQ5IiKT7swzz6Suro4HHniAoaEhHnzwQS699FIAHnroIa688kouvPBCli9fzvPPP89NN93E0NDQhNbw7LPPcv7553PSSSfxy1/+kueee47/+I//ACAYDE7oZw0rKioatW1mKf3DJBXqgU+mHWvgL9/FvfQQz4UO45rTFlBUoH8ziUju8/v9fPjDH+bHP/4x8+bNo6enh4suugiAp556iqOOOorPfvazsf03btyY0vsvWrSIH//4x4TDYfx+b1Tz6aefHrXPn//8Z+rq6vjXf/3XWNvDDz88ap/hwI1/n0SVlZXMnj2bp556inPOOSfW/sc//pG5c+fGet/ppjSZaM7BG0/Rf8/fwfdPYOCl/+T+oVP5fuXV/N1RjZmuTkQkbT760Y/y3HPP8aUvfYlzzjmH2tpaABYuXMjq1av51a9+RWtrK8uWLeORRx5J6b0/+clP0tHRwRVXXMHatWt54oknuPHGG0fts3DhQjo6OvjhD3/Ihg0buP/++7njjjtG7TN3rndF0KOPPkpHRwe9vb1JP+/666/nu9/9LnfddRevv/46P/jBD/j+97/PDTfckFLdE0k98AniwkNsfvrnFP3te8zsXUvAVXJ76Hz+WvtBjj9sAcuOa6bAr38viUj+OPzwwznyyCN54YUXuPnmm2PtS5cuZfXq1Vx++eWEQiHOOeccbr75Zj796U/v83s3Njby61//ms985jMceeSRLFiwgO985zucdtppsX3OOeccbrzxRm644QZ6e3s5+eST+frXv87FF18c2+fYY4/lmmuuYenSpXR0dHDppZdy7733jvm8T37ykwQCAb7yla/wqU99iqamJr72ta9Nyslp+8qcc+n7MLOzgGWAH7jbOfe1hOcvA74ObIk2fc85d/fe3nPx4sVu5cqVk1DtWxsKR1j1ehtdf76HI7b8lEa3gzciM/ld9fkUHn0xp75jjpYJFZG3be3atRxyyCGZLkMm2Vv9ns1slXNucWJ72nrgZuYHbgfOANqAFWb2qHNuTcKuP3fOXZWuuvbKOQgNMjgQYOvObrbu7GZ75y52du9iZ/duprf/hQvd/1BjvawvXsTTh13PwSdfyBWVmTkeIiIi+SOdQ+hLgPXOuQ0AZvYgcB6QGOAZs+uPt1PyzLchNIAvPEiBC+LDUQzMjd4StTeeysBp/8KB89+JVvUWEZF0SWeANwKb47bbgOOS7PcPZnYS8BpwrXNuc5J9JsWfO8oI9C5igEKssJSSsnLKyiuYVlFB1bRpVFdVUlddRXl5BVZYAjVzaajVlKgiIpJ+U+0ktl8DP3PODZrZUuA+4NTEnczsCuAKgObm5gn78KNPv4iuE/6e5ullVJZooREREZm60nla9BYgfhaT2YycrAaAc67TOTc8m/3dwDHJ3sg5d6dzbrFzbnF9ff2EFTirupTDGqsU3iIyZaTzRGNJv/35/aYzwFcAC8xsrpkVARcBj8bvYGYHxG2eC6xNY30iIlOK3++f8NnJZGoZGhoaNR1sKtIW4M65EHAV8BheMP/COfeKmd1iZudGd7vazF4xsxeBq4HL0lWfiMhUU11dzY4dOyZtKk7JrEgkwo4dO9722uNpvQ58MmTyOnARkckUiURoa2sjEAhkuhSZJOXl5cyePRufb/z+dMavAxcRkdT4fL4JPVFXcovm9hQREclCCnAREZEspAAXERHJQgpwERGRLKQAFxERyUJZfxmZmXUAb07gW9YBOyfw/aaSXP1u+l7ZRd8r++Tqd8uW7zXHOTdm2tGsD/CJZmYrk11vlwty9bvpe2UXfa/sk6vfLdu/l4bQRUREspACXEREJAspwMe6M9MFTKJc/W76XtlF3yv75Op3y+rvpWPgIiIiWUg9cBERkSykABcREclCCnAREZEspAAXERHJQgpwERGRLKQAFxERyUIKcBERkSykABcREclCBZkuYH/V1dW5lpaWTJchIiIyKVatWrUz2WpkWR/gLS0trFy5MtNliIiITAozS7pktobQRUREspACXEREJAspwEVERLKQAlxERCQLpTXAzewsM1tnZuvN7Lokz3/LzF6I3l4zs13prE9ERCRbpO0sdDPzA7cDZwBtwAoze9Q5t2Z4H+fctXH7fxo4Kl31iYiIZJN09sCXAOudcxucc0HgQeC8vez/IeBnaalMREQky6QzwBuBzXHbbdG2McxsDjAX+P04z19hZivNbGVHR8eEFyoiIjLVTdWJXC4CHnbOhZM96Zy7E7gTYPHixW6iPjQcChGJJP1IzGycdh+Y4T1r3mN769eJiIjsj3QG+BagKW57drQtmYuAKye9ogR/++lNnPDG7RP2fhFnOMABAxTRQwW7KWe3TWMP5eyxafT6ptFrFQR80+jzT6PPX8lR7zyTC05YOGF1iIhI7klngK8AFpjZXLzgvgi4OHEnMzsYqAGeSWNtANQc/B6ecZGxT7jxO/nmHA4wXNx+0dh2w4+hINxPcWg3xUO7mRHazZxQB6WhVkrDeyh0wVHvec9v/sQvCr7JBcc2ISIikkzaAtw5FzKzq4DHAD/wI+fcK2Z2C7DSOfdodNeLgAed20tqTpKDjzsTjjsz3R8LQ/3Qvwv6u4n89vOcvfklTnjkJcqK/Zxz+Kz01yMiIlNeWo+BO+eWA8sT2m5K2L45nTVNCYWl3q3yAHyLzmXGxn/h/Y39fObBFygvKuCUgxsyXaGIiEwxmoltqpl/KgD/dtRODj5gGv/8k1U809qZ4aJERGSqUYBPNbXzoHoOpW/+gfv/6Tiaa8v4+H0reGGzJqUTEZERCvCpxgwOPA02/onaYvjJx49jekUxl/7ob7y6fXemqxMRkSlCAT4VzT8Ngr3Q9jdmVJbw048fR0mhj4/c/Tfe2BnIdHUiIjIFKMCnorknga8A1j8BQFNtGT/9+HFEnOMjdz/Lll39GS5QREQyTQE+FZVUwuwl0PpErOnAhmnc/09L2D0wxCV3P0vHnsEMFigiIpmmAJ+qDjwVtr0IvSNzvR/WWMU9lx3Ltp4BLvnhs/T0DWWwQBERySQF+FQ1/zTvfsOTo5oXt9Ry50ePYUNHgEvv+RuBwVAGihMRkUxTgE9VBxwBpbWx4+Dx3r2gnu986Che2LyLR55ry0BxIiKSaQrwqcrnh/mnQOvvk87F/t5DZ1BW5Ke1Q2eli4jkIwX4VDb/NAi0w46XxzxlZsyZXs6mrr4MFCYiIpmmAJ/KotOqJhtGB5hTW8abneqBi4jkIwX4VFZ5ADQcOupysnhzppexuaufcCTtC7eJiEiGKcCnugNPhU1/heDYnnbz9DKC4Qjbdw9koDAREckkBfhUN/80CAdh45/HPNUyvRxAw+giInlIAT7VNZ8ABaXe2eiJT9WWAbCpUyeyiYjkGwX4VFdYAi3vTHoi26zqUgr9xkYFuIhI3lGAZ4P5p0Hn67Br06hmv8+YXVPGpi4NoYuI5BsFeDY4MDqtapJe+JzpZbypHriISN5RgGeDuoOgcnbSy8nm1JaxqbMPl2S2NhERyV0K8Gxg5l1OtuEpCI9evKR5ejl7BkN0BYIZKk5ERDJBAZ4t5p8Ggz2wZdWo5jnRM9Hf1JSqIiJ5RQGeLeadDOYbM4zeUqdLyURE8pECPFuU1kDjMWNOZJtdU4YZOpFNRCTPKMCzyfzTYOtz0NcVayop9DOzsoQ3dSmZiEheUYBnkwNPAxeBDX8Y1dxcq0vJRETyTVoD3MzOMrN1ZrbezK4bZ58LzGyNmb1iZg+ks74pb9bRUFI15ji4rgUXEck/Ben6IDPzA7cDZwBtwAoze9Q5tyZunwXA9cA7nXPdZtaQrvqygr8A5r0HWp8E57zLy4A508vZ2dtGYDBEeXHafqUiIpJB6eyBLwHWO+c2OOeCwIPAeQn7fAK43TnXDeCca09jfdlh/mmwewt0rIs1zZkePRNdl5KJiOSNdAZ4I7A5brst2hbvIOAgM3vazP5qZmcleyMzu8LMVprZyo6Ojkkqd4qaf6p3HzeMPqdWy4qKiOSbqXYSWwGwAHgP8CHgLjOrTtzJOXenc26xc25xfX19mkvMsOomb2rVuMvJmqM9cB0HFxHJH+kM8C1AU9z27GhbvDbgUefckHPuDeA1vECXePNPgzefhqF+AKpKC6kpK9RsbCIieSSdAb4CWGBmc82sCLgIeDRhn//C631jZnV4Q+ob0lhjdjjwNAgNwJt/iTU1Ty/XbGwiInkkbQHunAsBVwGPAWuBXzjnXjGzW8zs3OhujwGdZrYGeBL4P865znTVmDXmvBP8xdD6+5Gm2jI26hi4iEjeSOs1R8655cDyhLab4h474LPRm4ynqAzmnDA6wKeX8d8vbSUYilBUMNVObRARkYmmv/TZav5p0L4Gdm8FvGvBIw627OrPcGEiIpIOCvBsdeBp3v2GPwIj14LrUjIRkfygAM9WdQvB/NDVCsStC64T2URE8oICPFv5C6ByFuzy5sapn1ZMaaFfAS4ikicU4Nmsqgl6vAA3M+ZML2OTlhUVEckLCvBsVt0U64GDlhUVEcknCvBsVtXkLWwSDgHRZUW7+ohEXIYLExGRyaYAz2bVTeDCsMe7lKx5ejnBUIQdewYyXJiIiEw2BXg2q2727qPD6C1a1EREJG8owLNZVTTAoyeyDS8rqjnRRURynwI8m1XN9u6jPfBZ1SUU+ExzoouI5AEFeDYrLIHyBujZBECB30djTamWFRURyQMK8GxX3QS7NsU252hZURGRvKAAz3bVzaOuBR9eVtRb2E1ERHKVAjzbVTVBTxtEIoB3LfiegRC7+oYyXJiIiEwmBXi2q26G8CAEOgBvNjZAx8FFRHKcAjzbVTV598OXkk33LiXTsqIiIrlNAZ7tqqMBvutNYKQHrhPZRERymwI82w33wKMnspUW+ZlRWawhdBGRHKcAz3YllVBSHRtCB29GNg2hi4jkNgV4LkhcVnS6lhUVEcl1CvBcUNU8qgfeMr2M9j2D9AfDGSxKREQmkwI8FwzPxhadvKU5eib6Jh0HFxHJWQrwXFDdDMFe6O8GvNnYAC1qIiKSwxTguWDMteC6lExEJNcpwHNB9ehLyarLiqgqLeTNLvXARURyVVoD3MzOMrN1ZrbezK5L8vxlZtZhZi9Ebx9PZ31Zq6rZu4+/lExnoouI5LSCdH2QmfmB24EzgDZghZk96pxbk7Drz51zV6WrrpxQVguFZaOWFW2uLeOltp4MFiUiIpMpnT3wJcB659wG51wQeBA4L42fn7vMvOPgo9YFL2PLrn6GwpEMFiYiIpMlnQHeCGyO226LtiX6BzN7ycweNrOmZG9kZleY2UozW9nR0TEZtWaf6uaEIfRywhHH1l39GSxKREQmy1Q7ie3XQItz7nDgd8B9yXZyzt3pnFvsnFtcX1+f1gKnrITZ2IYvJdNxcBGR3JTOAN8CxPeoZ0fbYpxznc65wejm3cAxaaot+1U1QX8XBL0zz7WsqIhIbktngK8AFpjZXDMrAi4CHo3fwcwOiNs8F1ibxvqyW3X0TPRoL7xhWjHFBT71wEVEclTazkJ3zoXM7CrgMcAP/Mg594qZ3QKsdM49ClxtZucCIaALuCxd9WW92LKim6DhYHw+8y4l03SqIiI5KW0BDuCcWw4sT2i7Ke7x9cD16awpZwz3wHviLyUr12xsIiI5aqqdxCZvV8UM8BeNPpFtehlvdgVw0UVOREQkdyjAc4XPB5WNY2ZjGxiK0L5ncC8vFBGRbKQAzyWJl5LFzkTXMLqISK5RgOeSqubRs7HFrgXXpWQiIrlGAZ5LqpuhdzuEvCHzxppS/D5TD1xEJAcpwHPJ8LKiPW0AFPp9zKou0aVkIiI5SAGeS4avBY8/ka22nE0aQhcRyTkK8Fwy3AMfcymZeuAiIrlGAZ5LKhvBfGOWFd3VN0RP31AGCxMRkYmmAM8l/kKYdsCoIfTm2uilZF0aRhcRySUK8FxT3TxmCB10LbiISK5RgOeaqqZR86EPB/gmHQcXEckpCvBcU90Eu7dCOARAWVEB9dOK2bhTQ+giIrlEAZ5rqpogEoI922JNc2p1JrqISK5RgOea6rHXgjdPL9OyoiIiOUYBnmuq53j3cSeytUwvZ/vuAQaGwhkqSkREJpoCPNdUzfbuk5zItlnD6CIiOUMBnmsKS6G8flQPfF5dBQCv7ejNVFUiIjLBFOC5qKpp1GxsB82soNBvrN7Sk8GiRERkIinAc1F106iT2IoL/Bw0YxovK8BFRHKGAjwXVTd7S4o6F2t6R2MVq7f04OLaREQkeynAc1FVM4QGINARazqssYqe/iHauvszWJiIiEwUBXguSrKs6DsaqwB0HFxEJEcowHNR1fBkLiMnsi2cOY0Cn+k4uIhIjlCA56JYD3wkwEsKvRPZ1AMXEckNCvBcVFIFxVWjhtDBG0Z/WSeyiYjkhLQGDZlVEgAAIABJREFUuJmdZWbrzGy9mV23l/3+wcycmS1OZ305pbp51KVkAIfNrqK7b4gtu3Qim4hItktbgJuZH7id/9/encdHVd/7H399Zib7yhLWgFDABRARo1WsXqvFQq1yq7Z1qXuvtb+6tbe19ra11i6/apdr69LWVruqVG1VLnUr1Vv3BRRk30FACAkISQhJZvneP86ZZBLCEhhmcpL38/E4j/M933PmzOd4JJ+zfOf7hWnAWOBCMxvbyXYlwA3AG5mKrUcqH9bpHTig9+AiIj1AJu/ATwBWOudWO+dagBnA9E62+x5wO9CUwdh6njK/M5eUx+VHDiohHFKPbCIiPUEmE/hQIPWWcINf18rMJgHDnHN/39uOzOxqM5tjZnNqamr2tmnvVT4MmuugaXtrVX5OmDEDilmwsS6LgYmISDp0m0ZsZhYCfgb85762dc7d55yrcs5VVVRUHPrggqhs99+CgxqyiYj0FAeVwM2s2MzOMrMx+7H5RmBYynKlX5dUAowH/tfM1gInAjPVkO0AlQ/35h0ash1dWca2nS28v0NvKEREgqxLCdzMHjKz6/1yDl5Ds/8BFpnZJ/fx8beAMWY20sxygQuAmcmVzrkdzrn+zrkRzrkRwOvAOc65OV2JUXzJBN7hDnx8ske2DXoPLiISZF29Az8NeMUvn4131zwYuBX49t4+6JyLAdcCzwJLgEecc4vM7DYzO6eLcci+FPaDSMFud+BjB5cSVo9sIiKBF+ni9n2Bar88Bfibc67azB4C9vi77iTn3FPAUx3qbtnDtqd1MTZJZeb/lGxdu+q2hmxK4CIiQdbVO/AaYKRfngK84JcLgUS6gpI0Kdv9t+DgPUZXQzYRkWDragJ/FHjQzGYDpcA//PqJwIp0BiZp0ElvbOC1RN+6s4VNasgmIhJYXU3gNwF3AguBKc65Rr9+CPCbdAYmaVA+DBq3QsvOdtXj1SObiEjgdekduN8Q7Wed1P8kbRFJ+pQlf0q2ASqOaK0eO7iUkHkJ/Mxxg7IUnIiIHIyu/ozsGDMbl7L8CTN71MxuNbOuNoiTQ62TYUUBCnLDjBmgoUVFRIKsq4/Qfw0cDWBmlcBjQDHwH8D30xuaHLSyzhM4eI/RF2ysU0M2EZGA6moCPwJ4xy+fC7zlnJsGXAp8Np2BSRqUDIJQZA8N2UqpbWimuq45C4GJiMjB6moCz6VtlLDTgKf98nJAL1O7m1AYyio7/SnZ0ZV+j2x6jC4iEkhdTeDLgPPNbDje78Bn+/WDgQ/SGZikSXJY0Q6O8huyKYGLiARTVxP4d4EfAmuAl1P6KT+Ttkfr0p2UD+/0HXhhboRRFcX6KZmISEB19WdkT/p334OBd1NW/RP4WzoDkzQpGwb1myHWApHcdquOHlrGSytrsxSYiIgcjC4PJ+qcq3bOzQNyzSzfr3vNObc47dHJwSsfBjio27DbqvFDy6ipb6a6Tj2yiYgETZcTuJldYWYrgQagwcxWmNnlaY9M0mMPw4pCSkM2DS0qIhI4Xe3I5QbgXrxxvM/zp1nAvWZ2XfrDk4OW/C14Jw3Zxg4uxdSQTUQkkLrae9p1wA3OuftS6p40s6XA14C70haZpEfpUMA6bchWlKeGbCIiQdXVR+jD8BqsdfRPf510N5FcKBnc6SN08Bqy6Q5cRCR4uprAN+B14NLRaf466Y7KO/8tOHgN2bbUN7NFDdlERAKlq4/Qfwn8wsxGAy/5dafiPVq/JZ2BSRqVD4f1b3a66ujk0KLv7+D00vxMRiUiIgehS3fg/rChNwEX4zVemwVcBHzVOffT9IcnaVE2DOo2QiK+26pxQ/yGbBvqshCYiIgcqC4PAeqcuwe4x8xK/OX6tEcl6VU+DBIxr0OXsqHtVhXlRfhQ/yK9BxcRCZh9JnAze24f61vLzrkz0xCTpFvyt+C1y3ZL4OA9Rn999bYMByUiIgdjf+7ANx7yKOTQGn4S5JXC/Bkw6vTdVo8fWsYT896npr6ZipK8LAQoIiJdtc8E7py7IhOByCGUWwQTPgtv/xGm/ggK+7Zb3dqQbeMOPnrkgGxEKCIiXdTlrlQloKqugHgzzHtot1XjhmpscBGRoFEC7y0GjoNhJ8KcB8C5dquK1ZBNRCRwlMB7k6orYdsqWPPibqvGDy1Tl6oiIgGS0QRuZlPNbJmZrTSzmztZf42ZLTCzeWb2spmNzWR8Pd7Y6VDQx7sL7+DooWVs2tFEbUNzFgITEZGuylgCN7MwcA8wDRgLXNhJgn7IOXe0c24icAfws0zF1yvk5MPEi2HpLGjY0m7VeL0HFxEJlEzegZ8ArHTOrXbOtQAzgOmpGzjnUrsDKwLav6yVg3fc5V6nLu/8qV31uKGlACzU2OAiIoGQyQQ+FEgdUWODX9eOmX3JzFbh3YFf39mOzOxqM5tjZnNqamoOSbA9Vv8xMPJUmPv7dl2rlubnMFIN2UREAqPbNWJzzt3jnBsFfB341h62uc85V+Wcq6qoqMhsgD1B1ZXe+OCrnm9XrYZsIiLBkckEvpH2Y4ZXsvde3mYA/35II+qtjjgLiip2a8x29NBS3t/RxFY1ZBMR6fYymcDfAsaY2UgzywUuAGambmBmY1IWzwJWZDC+3iOSC8deAsufgR1t11DjW4cW1chkIiLdXcYSuHMuBlwLPAssAR5xzi0ys9vM7Bx/s2vNbJGZzQO+AlyWqfh6neMu8zp0efuPrVXjU7pUFRGR7q3Lw4keDOfcU8BTHepuSSnfkMl4erU+I2D0x+DtP8CpX4NwhNL8HEb0K2SBWqKLiHR73a4Rm2RQ1ZVQv8l7lO4bP7RMLdFFRAJACbw3G3MmlAyBub9rrZpQWcbG7bt4b2tjFgMTEZF9UQLvzcIR7134yn/CtjUAnH3MEPIiIe6cvTzLwYmIyN4ogfd2ky4FC3nvwoHBZQVccfJIHp+3kcVqjS4i0m0pgfd2pUPgiGnw9p8g1gLAF/9tFKX5Odzx7NIsByciInuiBC5w3BXQWAtL/weAssIc/t9po/jfZTW8uqo2y8GJiEhnlMAFRp0O5cNhTltjtssmj2BwWT63P7MM5zSmjIhId6MELhAKeXfha1+CGq/xWn5OmC9POZz567fzzMLNWQ5QREQ6UgIXz7Gfg1CON0qZ77xJlRw+sJgfP7uMaDyRvdhERGQ3SuDiKR4AR50N8x6E6C4AwiHjpo8fyeranTwyZ/0+diAiIpmkBC5tqq6Apu2w6InWqjOOGsDxI/pw5+wVNLbEshiciIikUgKXNiNOgX6j2/XMZmbcPO1IauqbeeDlNVkMTkREUimBSxszqLoK1r8By9r6Rz/usL5MGTuQX/1rNdt2tmQxQBERSVICl/aOvwoGjoeZ18HOtt+A3/TxI2hsiXH38yuzGJyIiCQpgUt7kTw49z7vXfisG70xw4ExA0v49HHD+NPra1m/TQOdiIhkmxK47G7gODj9W7Dkf2D+jNbqG6eMIWTGz/6hgU5ERLJNCVw6d9K1MHwyPH0TbPd+QpYc6OQJDXQiIpJ1SuDSuVAYPvVLcAl44ouQ8Dpy0UAnIiLdgxK47FmfETD1/3tdrL7xS8Ab6ORLH9VAJyIi2aYELnt37CVw+DSY/V3Y4t11X3qSP9DJ00s10ImISJYogcvemcE5v4C8Enj8aoi1tA10smEHT2ugExGRrFACl30rHgBn/xw2zYcX7wDaBjq5/Zml6mJVRCQLlMBl/xz1SZh4Mbz0U1j/FuGQ8d1zxvPetka+8+SibEcnItLrKIHL/pv6Iyit9B6lt+zkpFH9uO6jo3l07gaeeGdjtqMTEelVlMBl/+WXwr/fC9vWwHPfBuD6M8Zwwoi+fPPxBayp3ZnlAEVEeo+MJnAzm2pmy8xspZnd3Mn6r5jZYjN718z+aWaHZTI+2Q8jT4GTvgRz7oeVs4mEQ9x5wURyIiGue/htmmPxbEcoItIrZCyBm1kYuAeYBowFLjSzsR02eweocs5NAB4D7shUfNIFp38bKo6CJ74EjdsYUl7Aj88/hoUb67j96WXZjk5EpFfI5B34CcBK59xq51wLMAOYnrqBc+4F51xypIzXgcoMxif7Kycfzv01NG6FWV8G55gydiCXTx7BA6+sYfbi6mxHKCLS42UygQ8F1qcsb/Dr9uQq4OlDGpEcuMHHwOnfhMVPwCt3AvCNTxzJuCGlfPWx+WzasSvLAYqI9GzdshGbmX0OqAJ+vIf1V5vZHDObU1NTk9ngpM3JN8L487xe2pbMIi8S5u6LJhGNJbjh4XnE4olsRygi0mNlMoFvBIalLFf6de2Y2ceAbwLnOOeaO9uRc+4+51yVc66qoqLikAQr+8EMpt8DQyfB366GTe8ysn8R3//UeN5cu41fPL8y2xGKiPRYmUzgbwFjzGykmeUCFwAzUzcws2OBX+Ml7y0ZjE0OVE4BXPAQFJTDwxdCfTWfOraS8yZVctfzKzTgiYjIIZKxBO6ciwHXAs8CS4BHnHOLzOw2MzvH3+zHQDHwqJnNM7OZe9iddCclg+DCh2HXNphxEUR3cdv0cYzsX8SX/zKPrQ2dPkgREZGDYEEfTaqqqsrNmTMn22EIwJJZ8JeLYfz5cN5vWbSpjk/d+yonj+rH/ZcdTyhk2Y5QRCRwzGyuc66qY323bMQmAXXUJ+GM78DCx+DFnzBuSBnfOusoXlhWwwOvrMl2dCIiPUok2wFID/ORL0PNMnjh+9B/DJecOJ2XV9Ry+zNLOX5EX44ZVp7tCEVEegTdgUt6JccPH/ZhePwa7P13uOP8CQwoyeeaP89l/bbGfe9DRET2SQlc0i+SB599EIr6w4yLKI/V8ptLq2hsiXPRb19n846mbEcoIhJ4SuByaBRXwEV/geZ6ePhCxvaP8IcrT+CDnVEu/u3r1KpluojIQVECl0Nn4Dg4737YNB+euIaJQ0t54PLj2bh9F5/77Rtsb2zJdoQiIoGlBC6H1hFT4czvweInYfZ3OGFEH35zaRWra3Zy2QNvUt8UzXaEIiKBpAQuh95J10LVVfDqL+DRyznlsELuuXgSi96v46rfz6GxJZbtCEVEAkcJXA49MzjrpzDle7BkJtx/JlMGN/Hfn53InHXb+MKf5tIUjWc7ShGRQFECl8wwg5Ovh4sehe3r4Tcf5eyy1dx+3gReWlHLtQ+9TVSjl4mI7DclcMmsMR+D/3geCvvBH6fzafcs35s+jtlLtvDlv8wjngh2174iIpmiBC6Z1380fH42jDoD/v6fXFJ7J9+aOopZ727i6399l4SSuIjIPimBS3bkl3kjmH3kKzD3d3x+9Y1849R+PDZ3A9+ZuYigD7IjInKoKYFL9oTC8LHveL8Vf38eVy+9iluOi/Kn19fxX48vVMM2EZG9UAKX7Dv6fLjyGQy4Ytk1/Hz8ah5+8z3OvfdVVtc0ZDs6EZFuSQlcuochE+HqF7AhE5m+8lu8PG4WjdurOfuul3ly3sZsRyci0u0ogUv3UTwALp0JH/4ilatn8Hzul7m57Dm+NuMtvv7Yu+xq0SN1EZEkJXDpXiK5MO1H8MXXCA0/kUvq7+etspvZ+c6jnHPXSyyvrs92hCIi3YISuHRPA46Eix+FS56grLwfd+f8gp/Wf41b7n6AR95ar1bqItLrKYFL9zbqo/CFF+GcuxlXtJ0Z4VsonHkV3/vTUzQ0qw91Eem9lMCl+wuFYdIlhK9/h8SpN/HxnHl8fdWl/P0nV7F49XvZjk5EJCuUwCU48ooJnf5Ncm6cR93o6Xw6+iSD/3ASr9//VRpr1mU7OhGRjFICl+ApHULFJQ9Qd8lsNhaN5cT1vyHvnolsvOds4otnQVyP1kWk57OgNwaqqqpyc+bMyXYYkkULFs5n2dP3ckrDMwy07TTlDyDv+EuxSZdCn8OyHZ6IyEExs7nOuard6pXApSdwzjF74QZe+vuDnLbzaU4Lz8cAG30GTLoMjpgG4Zxshyki0mVK4NIrxOIJHp27gYefe5kzmv7BpXkv0ideC0UD4NiLYdy5MOhob3xyEZEA2FMCz+g7cDObambLzGylmd3cyfpTzextM4uZ2fmZjE16hkg4xIUnDGfGTZ8hfPp/cVrsLj4f/RpLwmNwr/wcfn0K3DkBnv46rP4XxKPZDllE5IBk7A7czMLAcmAKsAF4C7jQObc4ZZsRQCnwVWCmc+6xfe1Xd+CyN1sbmrnr+ZU8+MY6BoTq+cphq5kSmkPp+y9DrMkb1nTMx+HIs2D0GZBXku2QRUTa2dMdeCSDMZwArHTOrfYDmgFMB1oTuHNurb8ukcG4pAfrV5zHreeM44qTR3D38yv55rul/Gd0AicM/SLXj1jPiS2vE1n5HCx4BMK5MPLf4MhPwOHToHRwtsMXEdmjTN6Bnw9Mdc593l++BPiwc+7aTrb9PTBrT3fgZnY1cDXA8OHDj1u3Tr8Blv2zozHK4+9s4ME33mPFlgZK8iKcO3EgVwzfwojaf8HSWfDBWm/jfqNh+IkwfLI37/shvTsXkYzrDnfgaeOcuw+4D7xH6FkORwKkrDCHy08eyWWTRzB33Qc89MZ7PDx3E394I8Gxwz/BRZO/wNmDd5C/9p/w3uuwZBa882fvw8UD/YR+kjcfeDSEA/lPSER6gEz+9dkIDEtZrvTrRDLOzKga0ZeqEX255eyx/PXtjTz0xjq+9tcF3JYf4VPHfoyzJ1/KcZ8tI7R1Obz3Gqx7zUvqi5/0dpJbDJXHw7APw+BjYPAEKB2qu3QRyYhMPkKP4DViOwMvcb8FXOScW9TJtr9nL4/QU6kRm6SLc44312zjoTff4+mFm2mJJagoyWPquEFMGz+IE0b2JRIOwY4NXiJ/z0/o1YsA/99RQV8vkQ86Ggb5Sb3faK8/dxGRA9AtfgduZp8A7gTCwAPOuR+Y2W3AHOfcTDM7Hngc6AM0AZudc+P2tk8lcDkU6puiPL90C88s3MwLy7bQFE3QtyiXKUcNZNrRg5g8qj+5Ef9XmM0NXhLf/C5smg+bF8CWxRBv8dbnFMLAcTBoAgwcC31GQt+RUDZMncuIyD51iwR+KCiBy6G2qyXOv5Zv4akFm3l+6RYammOU5EeYctRApo4fxKmHV5Cf0+EOOx6FmmV+Un/Xm29eAM11bdtYGMqHtSX0jvPcosweqIh0S0rgImnQFI3zyspanlqwmdlLqtmxK0p+TojjR/Rl8qj+TB7Vj/FDywiHOnkP7hzUb4Jta+CDNbvPd33QfvuCvlAyyGs8VzIYSgZC8SCvrrV+EOQUZObgRSQrlMBF0iwaT/Daqq08v3QLr66qZXl1AwAl+RE+PLIfk0f14+TR/Tl8YDG2Pw3bdm1vn9B3bISGai/p11d75UQnPcfll0HJEO9366VDvIZ0JYO9eak/L+ijxnUiAaUELnKI1dQ389rqrby2qpZXV21l3dZGAPoV5XLSqH5MHtWfD3+oLyP7FRHq7A59XxIJ2LUN6jdDw2Zv3jptgrr3vamhmtZGdUmRfD+pD/GSeUE55CenMn+5zJ9S6iL5SvwiWaYELpJhGz5o5NVVW3lt1VZeXVVLdV0z4N2hT6gs45jKciZUljNxWDmDyvLT98XxqJfE6zZB3UYvqdf7yb1uk/eovmkHNG2HaOPe9xWKQF6p18VsXinkp5TzSrwpvxRySyCS6yX8cC5E8rwpnJz765Lb5BRApMBbpwsE6cg5SMTBJbwpEfOePsWT86hXF492shwDnLeP5Dy1nDpPxMHF/f0nUsrJ+nhbObnv1O9pV07GF4OTr/d+iZImSuAiWeScY1XNTt5e9wHzN2xn/obtLN1UTyzh/fsbUJLnJ/MyJlSWc0xlOWWFGWihHmvxGtbt2u4n9WRy3+HVNdd765vrvampzl+ua1vu7LH+fjOvlX5OvjdPJvdkXTjXa+wXCoOF/Hk4ZR5qvx7zLwj2NMebhyIQyvH2H+6s7E+hnLbvslCH7+sQB9YhqaQmmw7LLtF2oRNOfndKOZLbVgcQb/bOVbzZ68O/tdyhLhFtn3QSfkLqmIwSsbbkmExwLoGX3BJty611+5knksk21uz9CiM5xZLlZu/4Y81tMaQmaueXuzsL+f9vRPz/Z1L+vwmF4Zy7YeQp6fs6JXCR7qUpGmfxpjreXb+d+Rt2MH/DdlbX7GxdX9mngMMHlvhTMYcPLGH0gOLdW7xnk3PeH+Pmej+R+H+4Y8myn2RiTSnlXRBt8ucdpo518RY/4STaklDrPLH7cru7LDosp8yTd0y9QchPMMkLjtSLD8xLRpYyb63rsLw/zHa/IEk+gUlOyYuT1osja7tAar1IC7WfQuGUBBlpu7hKveBKLrfuK+XCbbeLO7x5KLzn/zap9RZKuaCL+BdsmdOjulIV6Qnyc8JMGt6HScP7tNbt2BVl4UYvmS/ZVM+K6npeWlFDNO5daIcMDutXxJgBxRwxqIQxfnIf0a8oO4ndzL97TuMrgExJJvLUO+R4S4dHsi1tFwnOdbhgSM5d253jbkklNdmkLFvIvyONtl3wJMupd6nJvgR2ey2R2/nridYEk/pkQHoqJXCRbqSsIIeTR/fn5NH9W+ui8QTrtu5k2eYGllfXt07/XLqFuP8I3gyGlBUwvG8hI/oXcli/Ikb08+aH9SukMFf/1HfTereoznQkmPSvWqSbywmHGD2ghNEDSjiLtiFOm2NxVtfsZHl1PWtqd7JuayNrt+7kuUXVbN3Z0m4fFSV5rQl9aHkBQ8sLGFJewJDyfIaUF3Svx/Iisl+UwEUCKi8S5qjBpRw1uHS3dXVNUd7zE/q6rY2s27qTtVsbeWlFDVvqm3drk9S3KNdL5mVeYh9aXsDg8nwGlORTUZLHgJI8ivL050KkO9G/SJEeqDQ/h/FDyxg/tGy3dS2xBNV1TWzcvov3t+9i04628tqtO3llZS07W+K7fa4wN9yazCtK8qgozvOX8+lfkkt5YS59CnMpL8ihtCCn897oRCRtlMBFepncSIhhfQsZ1rew0/XOOeqaYmzasYua+mZq6pvZ4s+T0/LqBl5eUUtdU6zTfZh57/P7FOb6c69cXphLeWEOZQU5lBd6ib68ILmcS2l+xBvxTUT2SQlcRNoxM8r8pHrkoL1v2xSNU9vQTG1DCx80trC9sYUPdka9eWPUr4tS0+Al/e2NLZ3e3acqzou0fn9ZQQ7F+RFK8iKU5Ee8cn4Oxf5ySX6E4ry25aK8CIW5YfIiof3rvlYkwJTAReSA5eeEqexTSGWfzu/mO9MSS7BjVzRlavHmjVG2p9Y3RqlvirF+WyMNzTEammPUN8VaW97vTSRkFOVFKM6LUJQXpigvQlFuW7kgJ0x+Tpj8nBD5kbZyXrI+EiI/J0xBbrh122S5IMe7QDig7nBF0kgJXEQyKjcS8t6hl+R1+bPOOZqiCeqbotQ3x2ho8pJ6fVOUhuYYO5tj7GyJe/PmGA3NfrnFuwDYUt/EzuY4TVFv2hWNsx/XA53Kzwm1JvT8lESf5yf/5MVBXrKcui4SIicSIiccIjccItcv54SN3HDHdUZuOExuxNsuz59HQqanDL2cEriIBIaZeXfCuWEGpGF/zjmicUdTzEvozdGEn9wTrXVN0QS7onGaWryE3+jPm6Jxdvnl5HrvM94ThuZY276ao966ZIc86WBGa/LPi3gXBzlh8y8E/IuAkLWWc8NGJJS8ODAiISMS9i4EwqG9L0dC1npRkfodueEQkdTvDBsh8z4fDqWUzQiFaC0n16dOkVCIkKGLki5QAheRXsvMvDvcSIjS/EPfoUs84bwLhViCWDxBcyxBNJ4gGndE4wla4glaWusStMRca11LLEFzLN5aTtY3+1Pq55L7S067dsVby7G4ozmWIJ5wxBKOeCJBLOGIxZ1flzjgpxLpEE65gAibEQ63Jf1IyAh1mIdDIcIhvLnR7sIhZN52IcO/iPDL/tOL5H7NX99aH/KWLeViI7lNcp9eGX+f3mdC/vopYwfusZFoOimBi4hkSNh/N1/U9bcHGZVoTe6OaMJL+jH/AqP9xUGHcixB3DkSCUfceZ9POEc8Qbu65JRwbd+TekERT9B6YZFIOKL+vPWzyc/F2+8zlkiQSEDceXF534Efg19OxuTHmXC0xtIaa2u57VgSzntik9zP3oysKFICFxGRzAuFjFy/kV4B6qWvI+cczuEndtd60ZDwLwoy1XWxEriIiEgXmHmP1ENk9329ekwQEREJICVwERGRAFICFxERCSAlcBERkQBSAhcREQkgJXAREZEAUgIXEREJICVwERGRAFICFxERCSBzLou91qeBmdUA69K4y/5AbRr315301GPTcQWLjit4euqxBeW4DnPOVXSsDHwCTzczm+Ocq8p2HIdCTz02HVew6LiCp6ceW9CPS4/QRUREAkgJXEREJICUwHd3X7YDOIR66rHpuIJFxxU8PfXYAn1cegcuIiISQLoDFxERCSAlcBERkQBSAk9hZlPNbJmZrTSzm7MdT7qY2VozW2Bm88xsTrbjORhm9oCZbTGzhSl1fc3sH2a2wp/3yWaMB2IPx3WrmW30z9s8M/tENmM8EGY2zMxeMLPFZrbIzG7w6wN9zvZyXIE+Z2aWb2Zvmtl8/7i+69ePNLM3/L+NfzGz3GzH2hV7Oa7fm9malPM1MduxdoXegfvMLAwsB6YAG4C3gAudc4uzGlgamNlaoMo5F4QOC/bKzE4FGoA/OufG+3V3ANuccz/yL7z6OOe+ns04u2oPx3Ur0OCc+0k2YzsYZjYYGOyce9vMSoC5wL8DlxPgc7aX4/oMAT5nZmZAkXOuwcxygJeBG4CvAH9zzs0ws18B851zv8xmrF2xl+O6BpjlnHssqwEeIN2BtzkBWOmcW+2cawFmANOzHJN04Jx7EdjWoXo68Ae//Ae8P6SBsofjCjzn3Cbn3Nt+uR5YAgwl4OdsL8cVaM7T4C/m+JMDTgeSSS6I52tPxxVoSuBthgLrU5bZrVXkAAAE/ElEQVQ30AP+Qfoc8JyZzTWzq7MdzCEw0Dm3yS9vBgZmM5g0u9bM3vUfsQfqMXNHZjYCOBZ4gx50zjocFwT8nJlZ2MzmAVuAfwCrgO3OuZi/SSD/NnY8Ludc8nz9wD9f/21meVkMscuUwHuHjzjnJgHTgC/5j2t7JOe9Ewr8lbXvl8AoYCKwCfhpdsM5cGZWDPwVuNE5V5e6LsjnrJPjCvw5c87FnXMTgUq8J5NHZjmktOh4XGY2HvgG3vEdD/QFAvMaB5TAU20EhqUsV/p1geec2+jPtwCP4/2j7Emq/XeSyXeTW7IcT1o456r9PzoJ4DcE9Lz57xz/CjzonPubXx34c9bZcfWUcwbgnNsOvACcBJSbWcRfFei/jSnHNdV/FeKcc83A7wjY+VICb/MWMMZvbZkLXADMzHJMB83MivxGNphZEXAmsHDvnwqcmcBlfvky4MksxpI2yQTn+xQBPG9+46H7gSXOuZ+lrAr0OdvTcQX9nJlZhZmV++UCvEa9S/AS3vn+ZkE8X50d19KUi0jDe68frPOlVuht/J983AmEgQeccz/IckgHzcw+hHfXDRABHgrycZnZw8BpeMMAVgPfAZ4AHgGG4w0t+xnnXKAahO3huE7DexTrgLXAF1LeGweCmX0EeAlYACT86v/Ce18c2HO2l+O6kACfMzObgNdILYx3g/eIc+42/+/IDLzHzO8An/PvWgNhL8f1PFABGDAPuCalsVu3pwQuIiISQHqELiIiEkBK4CIiIgGkBC4iIhJASuAiIiIBpAQuIiISQErgInLImdlpZubMrDLbsYj0FErgIiIiAaQELiIiEkBK4CK9gJldZ2ZLzazJzFaY2TeTfVub2Voz+4GZ/dbM6sys1sx+aGahlM+XmNmvzazGzJrNbI6ZndnhOwaY2e/MrNr/nmVmdmWHUI4ysxfNrNHMFpvZtAwcvkiPFNn3JiISZGZ2K3AFcCNed5FHAb8C8oFv+5tdh9eN8PF4Azr8Cq9L15/76x/w130OeA+4BphlZhOcc0v9/qX/BewCLgZWA6Pxut5M9RO8EZ9W4XU9+hczO8w590F6j1qk51NXqiI9mJkVArXAuc65Z1LqLwV+4ZwrN7O1wHrn3Ckp638IXOKcG2Zmo4EVwFnOuadStnkbmOecu9LMrgLuAUY75zZ0EsdpeANinJccucvMBuKNBT7VOfdsuo9dpKfTHbhIzzYOKAD+amapV+thIN/MKvzl1zp87hXgG2ZWCoz1617ssM2LeENNAhwHLO4seXcwL1lwzlWbWRwYuF9HIiLtKIGL9GzJ99ifBpZ3sj7TI4C1dFKntjgiB0D/cER6tkVAE/Ah59zKTqa4v92JHT43GdjonKvz9wFwaodtTqVt/OS5wFj9zlskc5TARXowf2zjHwI/NLMvmdkRZjbOzC4ws9tTNp1oZrea2eFmdhFwA/BTfx+rgEeBe83s42Z2pJn9HBgP/Nj//MN443rPNLOPmdlIMzvDzD6bqWMV6W30CF2kh3POfc/MNgHX4iXlXXiP03+fstldwGHAHCAK3E1bC3SAz+Ml6z8DpcAC4JPOuaX+dzSa2b8BdwAzgGJgLfCjQ3VcIr2dWqGL9HJ+K/TfOue+n+1YRGT/6RG6iIhIACmBi4iIBJAeoYuIiASQ7sBFREQCSAlcREQkgJTARUREAkgJXEREJICUwEVERALo/wBxBcjCMU2z5AAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "utils.plot_history(fb15k_history)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate the model" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "fb15k_raw_ranks, fb15k_filtered_ranks = fb15k_complex.rank_edges_against_all_nodes(\n", " fb15k_gen.flow(fb15k_test), fb15k_graph\n", ")" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
mrrhits at 1hits at 3hits at 10
raw0.2575890.1413300.2271330.513856
filtered0.5913300.4640690.6137780.818049
\n", "
" ], "text/plain": [ " mrr hits at 1 hits at 3 hits at 10\n", "raw 0.257589 0.141330 0.227133 0.513856\n", "filtered 0.591330 0.464069 0.613778 0.818049" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "summarise({\"raw\": fb15k_raw_ranks, \"filtered\": fb15k_filtered_ranks})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For comparison, Table 2 in the paper gives the following results for FB15k:" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
mrrhits at 1hits at 3hits at 10
raw0.242NaNNaNNaN
filtered0.6920.5990.7590.85
\n", "
" ], "text/plain": [ " mrr hits at 1 hits at 3 hits at 10\n", "raw 0.242 NaN NaN NaN\n", "filtered 0.692 0.599 0.759 0.85" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results_as_dataframe(\n", " {\"raw\": (0.242, None, None, None), \"filtered\": (0.692, 0.599, 0.759, 0.850)}\n", ")" ] }, { "cell_type": "markdown", "metadata": { "nbsphinx": "hidden", "tags": [ "CloudRunner" ] }, "source": [ "
Run the latest release of this notebook:
" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 4 }