{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"# Comet.ml Python API\n",
"\n",
"Comet.ml has an extensive interface to all of your data using a [REST API](https://en.wikipedia.org/wiki/Representational_state_transfer) through [Comet.ml endpoints](https://www.comet.ml/docs/rest-api/getting-started/). Now, you can access this information easily through the Comet.ml Python SDK. Requires version comet_ml version 3.0.0 or greater."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To install comet_ml:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install comet_ml --quiet"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To run the following experiments, you'll need to set your COMET_API_KEY. The easiest way to to this is to use the `comet_ml.init()` function:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import comet_ml\n",
"\n",
"comet_ml.init()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Quick Overview\n",
"\n",
"To access the Python API through the Comet.ml SDK, you will need to make an API() instance. \n",
"\n",
"**Note: this is a new interface.**\n",
"\n",
"### What's new?\n",
"\n",
"The new API:\n",
"\n",
"* is faster, having more items cached\n",
"* allows setting and logging more items\n",
"* `APIExperiment` now works similarly to `Experiment`, `ExistingExperiment`, and `OfflineExperiment`\n",
"* has a consistent interface\n",
"\n",
"Let's try it out! \n",
"\n",
"First, we import the API class and other libraries we will need. Note that this is a new interface and comes from `comet_ml.api`:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from comet_ml import API\n",
"import comet_ml\n",
"\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and create the API instance:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"comet_api = API()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the `comet_api` instance, you can get the name of your workspaces:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['sampleworkspace', 'cometpublic']"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"comet_api.get()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you reference your workspace by name using `comet_api.get(WORKSPACE_NAME)`, you'll see your projects:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['caffe2', 'ludwig', 'comet-sagemaker', 'fasttext', 'comet-notebooks', 'shap', 'parameter-space-exploration', 'comet-examples', 'fastai', '3d-histograms', 'pyspark-example', 'general', 'keras-example', 'home-credit']\n"
]
}
],
"source": [
"print(comet_api.get(\"cometpublic\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Or, get the projects from another user or shared workspace:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['mnist']"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"comet_api.get(\"testuser\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the same method, you can refer to a project by name and get all of the experiments in a project:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"comet_api.get(\"cometpublic\", \"fasttext\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Or, using the slash delimiter:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ,\n",
" ]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"comet_api.get(\"cometpublic/fasttext\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And one more level, get an `APIExperiment` object using the Experiment's ID:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"comet_api.get(\"cometpublic\", \"fasttext\", 'e64c5915920f481bab8f4cb4dbd615be')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Or, again using the slash shorthand:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"comet_api.get(\"cometpublic/fasttext/e64c5915920f481bab8f4cb4dbd615be\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's get an experiment and save it to a variable named `exp`:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"exp = comet_api.get(\"cometpublic/fasttext/e64c5915920f481bab8f4cb4dbd615be\")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"exp"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are a number of items you get and set from the APIExperiment instance. For a complete reference, see: https://www.comet.ml/docs/python-sdk/API/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For example, we can explore the `other` property, which shows items saved with Experiment.log_other(NAME, VALUE):"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'name': 'Name',\n",
" 'valueMax': 'last',\n",
" 'valueMin': 'last',\n",
" 'valueCurrent': 'last'},\n",
" {'name': 'storage_size_bytes',\n",
" 'valueMax': '0',\n",
" 'valueMin': '0',\n",
" 'valueCurrent': '0',\n",
" 'timestampMax': 1574708415277,\n",
" 'timestampMin': 1574708415277,\n",
" 'timestampCurrent': 1574708415277},\n",
" {'name': 'trainable_params',\n",
" 'valueMax': '376560',\n",
" 'valueMin': '376560',\n",
" 'valueCurrent': '376560',\n",
" 'timestampMax': 1529089066169,\n",
" 'timestampMin': 1529089066169,\n",
" 'timestampCurrent': 1529089066169}]"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"exp.get_others_summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this example, we see that the experiment has the `Name` \"last\". We can use `Name` to also look up experiments:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"('e64c5915920f481bab8f4cb4dbd615be', 'last')"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"exp = comet_api.get(\"cometpublic/fasttext/last\")\n",
"exp.id, exp.name"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perhaps one of the most useful abilities for the Python API is to access your experiment's data in order to create a variation of a plot. To access the raw metric data, use the `.get_metrics()` method of the APIExperiment:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2804"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(exp.get_metrics())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We see here were over 2800 metrics logged during the training of this experiment. We can get the first using indexing with an integer:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'metricName': 'val_acc',\n",
" 'metricValue': '0.9832226913061168',\n",
" 'timestamp': 1529089066713,\n",
" 'step': 69,\n",
" 'epoch': None,\n",
" 'runContext': None,\n",
" 'offset': 314}"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"exp.get_metrics()[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That shows that the \"acc\" (accuracy) metric had a value of about 0.5 at step 1 of the experiment.\n",
"\n",
"We can also filter on a single metric name, like so:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"acc_metrics = exp.get_metrics(\"acc\")"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"700"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(acc_metrics)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'metricName': 'acc',\n",
" 'metricValue': '0.5003255009651184',\n",
" 'timestamp': 1529089066219,\n",
" 'step': 1,\n",
" 'epoch': None,\n",
" 'runContext': None,\n",
" 'offset': 29}"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"acc_metrics[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Therefore, `exp.get_metrics(\"acc\")` gives us the dictionary for all \"acc\" items. We can then easily use Python's built in zip and matplotlib to plot these values:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"steps_acc = [(m[\"step\"], float(m[\"metricValue\"])) for m in acc_metrics]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This breaks up the data into (step, value) pairs:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1, 0.5003255009651184)"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"steps_acc[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A little Python trick to separate the steps from the accuracies so we can easily use matplotlib:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"steps, acc = zip(*steps_acc[:100]) # just the first 100 for now"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD5CAYAAAA3Os7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAe4UlEQVR4nO3de3Rc5Xnv8e8zMxpdbUm25ZtksA3mYsAGYwgpIRASuoAk0CY5p5C2adoE2pOS9JZzDjk5K01ZTbu6TlaTZh2aU9qkuTUQQpLGSUgot1xIAlg2xsZ3g7El+SLJ1l2a+3P+mLGrCBnJ9oy2Zs/vs5aWZ+95rXn2bOk3r9797r3N3RERkfIXCboAEREpDgW6iEhIKNBFREJCgS4iEhIKdBGRkFCgi4iERGyqBmb2ReAdQLe7XzrJ8wb8A3ArMAq83903T/V9FyxY4MuXLz/tgkVEKtmmTZt63b1lsuemDHTgS8D/Bb5yiudvAVYVvt4AfL7w7+tavnw57e3t03h5ERE5wcwOnOq5KYdc3P2nwPHXaXI78BXPexZoMrMlp1+miIicjWKMobcCHeOWOwvrRERkBs3oQVEzu9vM2s2svaenZyZfWkQk9IoR6F3AsnHLbYV1r+HuD7j7endf39Iy6Zi+iIicoWIE+gbgfZZ3DTDg7oeL8H1FROQ0TGfa4oPADcACM+sE/hKoAnD3/wc8Sn7K4j7y0xZ/v1TFiojIqU0Z6O5+5xTPO/DHRatIRETOyHTmoYuURDKTpW8kzcBYmv7RFEOJDIlMlkQ6RzaXY159NS1zqmmuq+L4SIqjg0l6h5PUVEVprK2isbaKeCxCxCBiRjKTZTiZZSSZIRoxmmqraKyrojoWJZvLkc462ZyTcyfnkMnmGE1lGU1lSWVzxKNGPBYhFomQzuZIZXKkc04sYsSjEWJRYziZoW8kRf9omoaaGK1NtSxtqmV+Q5yG6hj18RiRiL3udnvh9aNTtBM5XQp0KZrhZIbeoSTDyQwjyQwjqczJgO0eTNLRN0rH8VGODiY4NpxiKJkJuuSSiEcjWOFDJhY1qmNRaqry64YSGYYSGbI5Z359nEVza2iur2I0lWUokWE4kcHJ33TGMJrqqmiZU82ChmpS2RyDY/kPQHeoqYpQUxUlFjEiZpgZEct/UETMyOac0XSWsVSGVCZ3sj4HTtzXpqE6xv2/vY65NTH+z3/s5thwimzOyeTyDU58WA4lMhwbSXJsOEXEoLk+zry6ODVV0cIHZL59LJL/4MvmnIGxNINjaYaTGWKRCNFI/v0Y/zEWj+W3oToWJRrJ124Y6WyORCZHIp3FCu1OvK+ZXP6D2QsfilVRwx2ShfaZnFNbFaUuHqUqGmEklWEwkf+ZPHFDHzOjpaGa1uZaljbWMJzMcnQoQfdggmzOqYpGiMciVMciJx+7QyKdJZnJkfNCm2j+MORQMs3gWIZEOjtum/LbfGK/5Dz/YZ7NOX9w7QretnpR0X/2FOhyWkaSGV442E/EoKEmRiwS4bn9x3h8x1Ge33/8ZBBMZtHcapY113FpayMLGqpZ0BCnuT5OU22cxtoq5tTEqI1HqYlFiUTg+EiK7sEk/WNp5tVXsXBODS1zqkmmc/le/ViKzLhedzwWyfeSq2MnA2VgLE0yk80HTcSIRIyoWT70IkZdPEptVT5wUxknnc2RyeWIRfK/xFVRI5PzfG89m6Ohuorm+iqaauMMJdJ09Y/R1TdG32iakWSG4WSGVDb/C+8OqUyOZCZHMp0l586cmirm1ubft+6hJN2DCfpGUzRUx1jSWEN9PHay555zp280Tc9Qkv29I8SjERrrqmiuixMxSKRzDCczZLJ+8vVOhGvOwSC/ffEoTXVxbFySGnB8NM0vXznGyz3DVMci/NNPXmF+fZz66lghWCFb+H4N1VUsaIhzzrw6cg59IykODyRIZXMnQ98dMrkc2ZxjZsytraKxLk5rcy25XP658T8fJ96f4WSG3uEUuZN/PeXDsroQik6+s3DiQykWsZPv0fgPnxMhWhsxEukshwfSpLM56qtjNNZW0dpUgxXehFzO6R5K8szeXo4OJaitirJ4bv7nq6Y6SjKTYyiR4fiJv9Syuf98jaooEYNMNv9z4eT36/yGOLVVUVKZHIlMlmQ6d3JfuPvJYI+YkS3RneIU6DKl3uEkP9x2mMd3dvPsy8dIZXOvabNqYQN3vXkl57c0UF8do6E6RkNNjIbqKPXVMZoLvbnT0dZcV6xNKIn5DdXMb6hmTVtT0KWckWf29vI7X3gOgMMDCQC+9PtXc1lbY5BlzbhszokYJ8O+nCnQZVIDo2me2dfLtzd38uM9PWRzzvL5dfzuG8/lzRe0EI9GGElmGEtnubS1kRUL6oMuWc7C0cF8oC9qrA64kpkXpmMZCvQKls05n//xPp7a1U1TXZzmujjZXI6tnQO80jsC5IdJPnjdCt51RRsXLGoIRS9GXuvwQIJYxFhQX3mBHiYK9ArVN5LiT7+xhZ/s6WFtWyNHBxPsOjxIzuGytkbefWUbV57bzFXL54WqByOTOzqQYNHcmiln6MjspkCvQC91DfCHX91Ez1CST/3mpbz36nPU865wRwYTLJqr3nm5U6BXmG9t6uRj39nGgvo4D//RG7l8WXke0JPiOjKQ4OIlc4MuQ86SAr1CpLM5PvWDnXzpF69yzcp53P/edcxvUI9M8tMHjwwmuOHChUGXImdJgR5iL3b08/OXe9lxaJAtHf109o3xwTet4N5bLiIW1e1kJW8okWY0lWVxBc5wCRsFegiNJDP87Q938rVnDwLQ1lzL6iVz+fitF3PLZbqZlPyqI4Upi4sbawOuRM6WAj1knt9/nI9+80U6+kb54JtWcM+N59NUFw+6LJnFjhZOKlo8tybgSuRsKdBDIp3N8dkn9vCPP36ZZc11fOPuN3L1inlBlyVl4MRZoksaFejlToEeAgeOjfCRh7bwYkc/v7V+GZ9452rqq7VrZXpODLks1LTFsqff+jLm7jyyqZNPbthONGLc/951vH2Nxsjl9BwdTDCvPk517PSutSOzjwK9TPWNpPjYt7fxo+1HeMOKefz9b11Oa5MOasnpOzyQYNksvxCaTI8CvQx1HB/l3Z//BX2jKe695SLuum6lTs+XMzaUyLBY4+ehoEAvQ3/9gx0MJzN850PXcmlrZV3qVEpjkWa4hILOLikzv3i5l8e2H+VDN5ynMJei0QyXcFCgl5FMNsd939tBa1MtH7xuZdDlSIhoDno4KNDLyEMbO9h1ZIiPv/3i0777j8jrWaQeeihoDL0MJNJZtnUN8PeP7+HqFfO45dLFQZckIaMhl3BQoM9i2zoH+KvvbWdr5wCpbI6aqgh/+c7Vuna5FJ0OioaDAn2WGklm+NDXN5FM53j/tcu58txm1p/brEveStHVxaPMrVEUhIH24iz1N4/upLNvjIf/8I1ctVzXZJHSWTy3Rn/1hYQOis5CP93Tw789d5C7rlupMJeS03BLeCjQZ5mBsTT/81tbOX9hA39+0wVBlyMVQAdEw0NDLrPMpx/bTfdQkm//zpWamigzQlMWw0M99Fmk4/goD208yB1XLWOtbt4sM0QnFYWHAn0W+dyTezEzPnzjqqBLkQqiC3OFhwJ9lni5Z5hvbe7kd685V79gMiOWL6hj1cIG1rTpmkBhMa1AN7ObzWy3me0zs3snef5cM3vSzLaa2Y/NrK34pYbbZ5/YS01VlP92w3lBlyIVoq25jsf//HqW6ObQoTFloJtZFLgfuAVYDdxpZqsnNPs08BV3XwPcB/xtsQsNs52HB/nei4f4/WuXs0AnDonIGZpOD/1qYJ+7v+LuKeAh4PYJbVYDTxUePz3J83IKx4aTfPSbLzKnJsbd16l3LiJnbjqB3gp0jFvuLKwb70XgXYXHvwnMMbP5Z19euHX1j/Ff/umX7Ose5nN3XEFjXVXQJYlIGSvWQdGPAteb2QvA9UAXkJ3YyMzuNrN2M2vv6ekp0kuXp33dw7zn87+gZzDJVz/wBt5y0cKgSxKRMjedE4u6gGXjltsK605y90MUeuhm1gC82937J34jd38AeABg/fr1foY1lz13556vbyadzfHQH17DJUs1y0BEzt50eugbgVVmtsLM4sAdwIbxDcxsgZmd+F4fA75Y3DLDZfPBfnYdGeKjv36hwlxEimbKQHf3DHAP8BiwE3jY3beb2X1mdluh2Q3AbjPbAywCPlWiekPhwecPUh+P8s61S4MuRURCZFrXcnH3R4FHJ6z7xLjHjwCPFLe0cBoYS/P9rYd417o26qt1KR0RKR6dKTrDvruli0Q6x3uvPifoUkQkZBToM8jd+fpzB7mstZFLWzV2LiLFpUCfQVs68gdD71TvXERKQIE+gx58/iB18Si3Xa6DoSJSfAr0GdJxfJR/f+EQv3lFKw06GCoiJaBAnyGf/o/dRCJwz43nB12KiISUAn0GbO3s57tbDvGBN63QpUpFpGQU6CXm7nzqBzuZXx/nj67X1RRFpHQU6CX25M5untt/nD952yrm1OhqiiJSOgr0EsrlnL/70S5WLqjXVEURKTkFegn9/OVe9nYP8+G3nk9VVG+1iJSWUqaEvv7cQZrrqrj1siVBlyIiFUCBXiLdQwke33GU91zZRnUsGnQ5IlIBFOgl8s32TjI519i5iMwYBXoJZHPOg88f5NfOm8/KloagyxGRCqFAL4Gf7u2hs2+M975BvXMRmTkK9BL4+nMHWdAQ59dXLw66FBGpIAr0IuseSvDUrm7ec+Uy4jG9vSIyc5Q4RfbDbUfI5px3rWsNuhQRqTAK9CL7/tZDXLCogQsWzQm6FBGpMAr0Ijo8MMbGV/t4xxrdwEJEZp4CvYh+sPUwAO9YozNDRWTmKdCL6PtbD7N6yVzNPReRQCjQi6Tj+ChbOvp5x1r1zkUkGAr0Inl0W2G45TKNn4tIMBToRfL9rYdZ29bIOfPrgi5FRCqUAr0IXu4ZZlvXgGa3iEigFOhF8K8/3088GuH2KxToIhIcBfpZ6htJ8cimTn7jiqUsnFMTdDkiUsEU6Gfpa88eIJHO8cHrVgZdiohUOAX6WUiks3z5l69y/QUtOtVfRAKnQD8L393SRe9wirvUOxeRWWBagW5mN5vZbjPbZ2b3TvL8OWb2tJm9YGZbzezW4pc6u7g7//Kz/Vy0eA7Xnj8/6HJERKYOdDOLAvcDtwCrgTvNbPWEZv8beNjdrwDuAP6x2IXONj/Z08Pe7mHuum4lZhZ0OSIi0+qhXw3sc/dX3D0FPATcPqGNA3MLjxuBQ8UrcXb6wjP7WTinmneu1VRFEZkdphPorUDHuOXOwrrxPgn8jpl1Ao8CHy5KdbPU7iND/GxvL7/3a8t1VyIRmTWKlUZ3Al9y9zbgVuCrZvaa721md5tZu5m19/T0FOmlZ94Xn9lPTVWE916tm0CLyOwxnUDvApaNW24rrBvvA8DDAO7+S6AGWDDxG7n7A+6+3t3Xt7S0nFnFAesdTvKdLV28e10bzfXxoMsRETlpOoG+EVhlZivMLE7+oOeGCW0OAm8FMLOLyQd6+XbBX8fXnj1AKpPjD960IuhSRER+xZSB7u4Z4B7gMWAn+dks283sPjO7rdDsL4C7zOxF4EHg/e7upSo6KIl0lq89e4C3XNjCebqJhYjMMrHpNHL3R8kf7By/7hPjHu8Ari1uabPPj146Qu9wig+8SScSicjsoykap+GJnUdpmVOtE4lEZFZSoE9TNuf8bG8vb17VohOJRGRWUqBP04ud/QyMpbnhwvKcnSMi4adAn6Yf7+4hYnDdqtfMxhQRmRUU6NP0kz09XL6siaY6zT0XkdlJgT4Nx4aTbO3s5/oLFgZdiojIKSnQp+GZfb24w/UaPxeRWUyBPg0/2d3DvPo4a1obgy5FROSUFOhTyOWcn+zp4bpVC4hENF1RRGYvBfoUth8a5NhIStMVRWTWU6BP4end3QBct0qBLiKzmwJ9Co9tP8K6c5pY0FAddCkiIq9Lgf46Dh4bZfuhQW65dEnQpYiITEmB/joe234EgJsvXRxwJSIiU1Ogv44fvnSYS5bOZdm8uqBLERGZkgL9FI4OJth8sJ+bL1HvXETKgwL9FP6jMNxyy2UKdBEpDwr0U/jhS0c4r6We8xfOCboUEZFpUaBP4vhIiuf2H9fBUBEpKwr0STyx8yjZnGu6ooiUFQX6JH68u5sljTVcsnRu0KWIiEybAn0Smw/0c9Xyebp3qIiUFQX6BIf6xzgymGDdOU1BlyIicloU6BNsPtgHwLpzmwOuRETk9CjQJ9h8oJ/qWISLFmv8XETKiwJ9ghc6+ljT1kg8prdGRMqLUmucZCbL9q5B1p2j4RYRKT8K9HFe6hoklc1xhQJdRMqQAn2cF04eENUMFxEpPwr0cTYf7KOtuZaFc2qCLkVE5LQp0Md54WC/xs9FpGwp0AsOD4xxeCDBFTqhSETK1LQC3cxuNrPdZrbPzO6d5PnPmNmWwtceM+svfqmltflAvmT10EWkXMWmamBmUeB+4CagE9hoZhvcfceJNu7+Z+Pafxi4ogS1ltTmg31UxyJcvEQnFIlIeZpOD/1qYJ+7v+LuKeAh4PbXaX8n8GAxipsp/aMpvrvlEFctn6cTikSkbE0nvVqBjnHLnYV1r2Fm5wIrgKdO8fzdZtZuZu09PT2nW2vJ/PUPdtI3muJjt14UdCkiImes2N3RO4BH3D072ZPu/oC7r3f39S0tLUV+6TPz0z09PLKpkz+6fiWXLG0MuhwRkTM2nUDvApaNW24rrJvMHZTRcMtIMsP/+s42VrbU8+EbVwVdjojIWZlOoG8EVpnZCjOLkw/tDRMbmdlFQDPwy+KWWDqfeXwPnX1j/N2711BTFQ26HBGRszJloLt7BrgHeAzYCTzs7tvN7D4zu21c0zuAh9zdS1Nqcbk733mhi3esWcJVy+cFXY6IyFmbctoigLs/Cjw6Yd0nJix/snhlld6hgQTHRlK8YeX8oEsRESmKip2jt60zfyLRmlYdCBWRcKjYQN/aOUBV1LhoyZygSxERKYqKDfRtXQNcuHgO1TEdDBWRcKjIQHd3tnYOcFmrLsQlIuFRkYHecXyMgbE0a9o0fi4i4VGRgf5i4YDoZTogKiIhUpGBvq1rgHgswgWLdEBURMKjIgN9a2c/Fy+ZqysrikioVFyi5XLOS12Dmn8uIqFTcYG+/9gIw8kMl+mAqIiETMUF+rbOAQDNcBGR0Km4QN/aOUBNVYTzWxqCLkVEpKgqLtC3dfVzydJGYtGK23QRCbmKSrVs4YCo5p+LSBhVVKDv7R5iLJ3l8mU65V9EwqeiAv3FjsIlc3VAVERCqKICfUvHAHNrYiyfXx90KSIiRVdRgf5iRz9rlzURiVjQpYiIFF3FBPpYKsvuo0OsbdP4uYiEU8UE+o7DA2RzzlodEBWRkKqYQN/SkT9DdK0OiIpISFVMoL/Y0c/SxhoWzq0JuhQRkZKonEDv7GeNxs9FJMQqItD7RlIcODaq8XMRCbWKCPQTt5xbu0zj5yISXpUR6B0DmOkeoiISbpUR6J39nN/SwJyaqqBLEREpmdAHuruztbNf4+ciEnqhD/Q9R4fpHU5x5bnNQZciIlJSoQ/0J3YeBeDGixYGXImISGmFPtAf33GUtW2NLNIJRSIScqEO9O7BBFs6+rlp9aKgSxERKblpBbqZ3Wxmu81sn5nde4o2/9XMdpjZdjP7enHLPDNP7uoG4G0KdBGpALGpGphZFLgfuAnoBDaa2QZ33zGuzSrgY8C17t5nZrNiwPqJHUdpa67lwkVzgi5FRKTkptNDvxrY5+6vuHsKeAi4fUKbu4D73b0PwN27i1vm6RtNZXhmXy83rV6EmW5oISLhN51AbwU6xi13FtaNdwFwgZn93MyeNbObJ/tGZna3mbWbWXtPT8+ZVTxNz+ztJZnJcdPFGm4RkcpQrIOiMWAVcANwJ/DPZvaaM3nc/QF3X+/u61taWor00pN7fMdR5tbEuGrFvJK+jojIbDGdQO8Clo1bbiusG68T2ODuaXffD+whH/CByOacp3Z185aLFlIVDfVEHhGRk6aTdhuBVWa2wsziwB3Ahglt/p187xwzW0B+COaVItZ5WnYeHuTYSEonE4lIRZky0N09A9wDPAbsBB529+1mdp+Z3VZo9hhwzMx2AE8D/93dj5Wq6KlsOtAHwFXLNdwiIpVjymmLAO7+KPDohHWfGPfYgT8vfAVu04E+ljTWsLSpNuhSRERmTCgHmDcd6GOdLsYlIhUmdIF+ZCBBV/8YV56jQBeRyhK6QN98MD9+rsvlikilCV2gbzrQR01VhNVL5wZdiojIjAploK9pbdL8cxGpOKFKvUQ6y/ZDAzogKiIVKVSBvq1rgHTWNX4uIhUpVIF+4oSidefohtAiUnlCF+grFtQzv6E66FJERGZcaALd3XnhYB/rNP9cRCpUaAL9wLFReodTGj8XkYoVmkB/Zl8vAFevUKCLSGUKTaA/vaubZfNqOa+lIehSREQCEYpAT6Sz/PzlXm68cKHuHyoiFSsUgf7sK8dIpHPcoBtaiEgFC0WgP72rm5qqCG9cOT/oUkREAlP2ge7uPLW7m2vPW0BNVTTockREAlP2gf5yzzAdx8d4i4ZbRKTClX2gP7WrG0CBLiIVr+wD/eldPVy4aA6tun+oiFS4sg70wUSaja8eV+9cRIQyD/Rf7Oslk3PecmFL0KWIiASurAP9uf3HqamKcIUuyCUiUt6B3v5qH5cvayIeK+vNEBEpirJNwuFkhu2HBrhq+bygSxERmRXKNtC3HOwn57BegS4iApRxoD//6nEiptvNiYicULaB3v7qcS5eMpc5NVVBlyIiMiuUZaCnszleONiv8XMRkXHKMtB3HBpkLJ1l/XJNVxQROaEsA33jq8cB1EMXERlnWoFuZjeb2W4z22dm907y/PvNrMfMthS+Plj8Uv9T+6t9nDOvjkVza0r5MiIiZSU2VQMziwL3AzcBncBGM9vg7jsmNP2Gu99Tghp/hbuz8dXjXK/T/UVEfsV0euhXA/vc/RV3TwEPAbeXtqxT2987wrGRlIZbREQmmE6gtwId45Y7C+smereZbTWzR8xsWVGqm0T7q30AXKUDoiIiv6JYB0W/Byx39zXA48CXJ2tkZnebWbuZtff09JzRCzXVVXHT6kWc19Jw5tWKiITQdAK9Cxjf424rrDvJ3Y+5e7Kw+C/AlZN9I3d/wN3Xu/v6lpYzGwP/9UsW88/vW4+ZndH/FxEJq+kE+kZglZmtMLM4cAewYXwDM1sybvE2YGfxShQRkemYcpaLu2fM7B7gMSAKfNHdt5vZfUC7u28APmJmtwEZ4Djw/hLWLCIikzB3D+SF169f7+3t7YG8tohIuTKzTe6+frLnyvJMUREReS0FuohISCjQRURCQoEuIhISCnQRkZAIbJaLmfUAB07jvywAektUzmym7a48lbrt2u7pOdfdJz0zM7BAP11m1n6qqTphpu2uPJW67drus6chFxGRkFCgi4iERDkF+gNBFxAQbXflqdRt13afpbIZQxcRkddXTj10ERF5HWUR6FPdpDoszGyZmT1tZjvMbLuZ/Ulh/Twze9zM9hb+DeXtmswsamYvmNn3C8srzOy5wn7/RuHyzaFiZk2Fu3ztMrOdZvbGStjfZvZnhZ/xl8zsQTOrCeP+NrMvmlm3mb00bt2k+9fyPlfY/q1mtu50X2/WB/q4m1TfAqwG7jSz1cFWVTIZ4C/cfTVwDfDHhW29F3jS3VcBTxaWw+hP+NVr6f8d8Bl3Px/oAz4QSFWl9Q/Aj9z9ImAt+e0P9f42s1bgI8B6d7+U/GW57yCc+/tLwM0T1p1q/94CrCp83Q18/nRfbNYHOrPsJtWl5O6H3X1z4fEQ+V/uVvLbe+K2fl8GfiOYCkvHzNqAt5O/4xWWvyXVjcAjhSah224zawTeDHwBwN1T7t5PBexv8vdiqDWzGFAHHCaE+9vdf0r+HhHjnWr/3g58xfOeBZom3DxoSuUQ6NO9SXWomNly4ArgOWCRux8uPHUEWBRQWaX0WeB/ALnC8nyg390zheUw7vcVQA/wr4Whpn8xs3pCvr/dvQv4NHCQfJAPAJsI//4+4VT796yzrhwCveKYWQPwLeBP3X1w/HOen5YUqqlJZvYOoNvdNwVdywyLAeuAz7v7FcAIE4ZXQrq/m8n3RlcAS4F6XjssURGKvX/LIdCnvEl1mJhZFfkw/zd3/3Zh9dETf3oV/u0Oqr4SuRa4zcxeJT+kdiP5seWmwp/kEM793gl0uvtzheVHyAd82Pf324D97t7j7mng2+R/BsK+v0841f4966wrh0Cf8ibVYVEYN/4CsNPd/37cUxuA3ys8/j3guzNdWym5+8fcvc3dl5Pfv0+5+28DTwPvKTQL43YfATrM7MLCqrcCOwj5/iY/1HKNmdUVfuZPbHeo9/c4p9q/G4D3FWa7XAMMjBuamR53n/VfwK3AHuBl4ONB11PC7XwT+T+/tgJbCl+3kh9PfhLYCzwBzAu61hK+BzcA3y88Xgk8D+wDvglUB11fCbb3cqC9sM//HWiuhP0N/BWwC3gJ+CpQHcb9DTxI/jhBmvxfZB841f4FjPyMvpeBbeRnAZ3W6+lMURGRkCiHIRcREZkGBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIfH/AYDU7d+v19hvAAAAAElFTkSuQmCC\n",
"text/plain": [
"