{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Repeat Copy Task\n",
"### Differentiable Neural Computer (DNC) using a RNN Controller\n",
"\n",
"![\"DNC](\"../static/dnc_schema.png\")
\n",
"\n",
"**Sam Greydanus $\\cdot$ February 2017 $\\cdot$ MIT License.**\n",
"\n",
"Represents the state of the art in differentiable memory. Inspired by this [Nature paper](https://goo.gl/6eiJFc). Some ideas taken from [this Gihub repo](https://github.com/Mostafa-Samir/DNC-tensorflow)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Brain analogy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" However, there are interesting parallels between the memory mechanisms of a DNC and the functional capabilities of the mammalian hippocampus. DNC memory modification is fast and can be one-shot, resembling the associative long-term potentiation of hippocampal CA3 and CA1 synapses. The hippocampal dentate gyrus, a region known to support neurogenesis, has been proposed to increase representational sparsity, thereby enhancing memory capacity: usage- based memory allocation and sparse weightings may provide similar facilities in our model. Human 'free recall' experiments demonstrate the increased probability of item recall in the same order as first presented—a hippocampus-dependent phenomenon accounted for by the temporal context model, bearing some similarity to the formation of temporal links."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import sys\n",
"sys.path.insert(0, '../dnc')\n",
"\n",
"from dnc import DNC\n",
"from rnn_controller import RNNController\n",
"\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Hyperparameters"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"xydim = 6\n",
"tf.app.flags.DEFINE_integer(\"xlen\", xydim, \"Input dimension\")\n",
"tf.app.flags.DEFINE_integer(\"ylen\", xydim, \"output dimension\")\n",
"tf.app.flags.DEFINE_integer(\"length\", 5, \"Sequence length\")\n",
"tf.app.flags.DEFINE_integer(\"batch_size\", 3, \"Size of batch in minibatch gradient descent\")\n",
"\n",
"tf.app.flags.DEFINE_integer(\"R\", 1, \"Number of DNC read heads\")\n",
"tf.app.flags.DEFINE_integer(\"W\", 10, \"Word length for DNC memory\")\n",
"tf.app.flags.DEFINE_integer(\"N\", 7, \"Number of words the DNC memory can store\")\n",
"\n",
"tf.app.flags.DEFINE_integer(\"print_every\", 100, \"Print training info after this number of train steps\")\n",
"tf.app.flags.DEFINE_integer(\"iterations\", 30000, \"Number of training iterations\")\n",
"tf.app.flags.DEFINE_float(\"lr\", 1e-4, \"Learning rate (alpha) for the model\")\n",
"tf.app.flags.DEFINE_float(\"momentum\", .9, \"RMSProp momentum\")\n",
"tf.app.flags.DEFINE_integer(\"save_every\", 1000, \"Save model after this number of train steps\")\n",
"tf.app.flags.DEFINE_string(\"save_path\", \"rnn_models/model.ckpt\", \"Where to save checkpoints\")\n",
"FLAGS = tf.app.flags.FLAGS"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Data functions"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAfMAAAD+CAYAAAAu5uwhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAEZ9JREFUeJzt3X+snXVhx/H359buojghEUe76FSCOntr0BZ1zgBzhbGQ\niVuWIHcu8xcaRDfTLVEYGDeVyWagDpVkcfJDkRLc4oRsii24KTokUsH0glvYUFBaFBg1gq2l97s/\nzmm999Lenufcc/s83/b9Ss4f9+lznvPJt+c8n/N9znnOk1IKkiSpXmNtB5AkSQtjmUuSVDnLXJKk\nylnmkiRVzjKXJKlylrkkSZWzzCVJqpxlLklS5Z6ymBtP8kzgVOB7wPbFfCxJkg4yhwHPA24spTw8\n34qLWub0ivyzi/wYkiQdzN4AXDPfCotd5t8DeN3Vr+WoFx+1oA1tWLuRU9adPIpMByXHZ98cm/k5\nPvvm2MzP8ZnfQsfnobsf4gt/fAP0u3Q+i13m2wGOevFRLF+1bEEbGj9ifMHbOJg5Pvvm2MzP8dk3\nx2Z+js/8Rjg++/2Y2i/ASZJUOctckqTKWeaSJFWumjKfmFzRdoROc3z2zbGZn+Ozb47N/Byf+R3I\n8UkpZfE2nqwCbn/r7W/2SxKSJDWwZdNWPrX6CoDVpZRN861bzcxckiTtnWUuSVLlLHNJkipnmUuS\nVLmhyjzJO5Pcm+RnSW5N8vJRB5MkSYNpXOZJXg9cDLwfeBlwJ3BjkoX9+LokSRrKMDPztcA/lFI+\nXUr5LnA28DjwlpEmkyRJA2lU5kmWAquBm3YvK70T1TcCrxptNEmSNIimM/OjgCXAg3OWPwj4qzCS\nJLXAb7NLklS5ptczfwjYBRw9Z/nRwNZ93WnD2o2MHzE+a9nE5ApWTk40fHhJkg4+m9dPMbX+rlnL\ndmzbMfD9G/82e5JbgW+WUt7d/zvAfcClpZSPzFnX32aXJGkITX6bvenMHOAS4MoktwO30ft2+9OA\nK4fYliRJWqDGZV5Kua5/TvkH6B1evwM4tZTy41GHkyRJ+zfMzJxSymXAZSPOIkmShuC32SVJqpxl\nLklS5SxzSZIqZ5lLklQ5y1ySpMpZ5pIkVc4ylySpcpa5JEmVs8wlSaqcZS5JUuUsc0mSKmeZS5JU\nOctckqTKDXXVtMZK6d06YGzJRW1HmGV6+ry2I3RXR54zu/ncmV/H/ru4cMnStiPsccH0zrYjdFrn\nnjtjXXnuDF7RzswlSaqcZS5JUuUsc0mSKmeZS5JUOctckqTKWeaSJFXOMpckqXKWuSRJlbPMJUmq\nnGUuSVLlLHNJkirXuMyTnJDk+iQ/TDKd5PTFCCZJkgYzzMz8cOAO4BygYz+PL0nSoafxVdNKKV8C\nvgSQJCNPJEmSGvEzc0mSKmeZS5JUucaH2YexYe1NjB85PmvZxJkrWDm54kA8vCRJHbe5f5tp+8D3\nPiBlfsq6NSxftexAPJQkSRVa2b/NtAX45ED39jC7JEmVazwzT3I4cCyw+5vsxyQ5DniklHL/KMNJ\nkqT9G+Yw+/HAV+idY16Ai/vLrwLeMqJckiRpQMOcZ/4feHhekqTOsJQlSaqcZS5JUuUsc0mSKmeZ\nS5JUOctckqTKWeaSJFXOMpckqXKWuSRJlbPMJUmqnGUuSVLlLHNJkipnmUuSVLlhrppWtZS2E8w2\nNvbhtiPMMr3r3LYj7DG25KK2I8zSpbHRAEr2v84B8qGxpW1HmOX8XTvbjjDLhUs6Nj7T3RifLZue\n4PLjB1vXmbkkSZWzzCVJqpxlLklS5SxzSZIqZ5lLklQ5y1ySpMpZ5pIkVc4ylySpcpa5JEmVs8wl\nSaqcZS5JUuUalXmS85LcluQnSR5M8vkkL1yscJIkaf+azsxPAD4GvBI4GVgKfDnJU0cdTJIkDabR\nVdNKKafN/DvJm4AfAauBW0YXS5IkDWqhn5kfCRTgkRFkkSRJQxi6zJME+ChwSynlrtFFkiRJTTQ6\nzD7HZcAK4NUjyiJJkoYwVJkn+ThwGnBCKWXL/tbfsPYmxo8cn7Vs4swVrJxcMczDS5J0UJlaP8XU\ntbMPcm9/dMfA929c5v0ifx1wUinlvkHuc8q6NSxftazpQ0mSdEiYmJxgYnJi1rItm7Zy+fFXDHT/\nRmWe5DJgEjgdeCzJ0f1/2lZK2d5kW5IkaTSafgHubOAZwL8DD8y4nTHaWJIkaVBNzzP3518lSeoY\ny1mSpMpZ5pIkVc4ylySpcpa5JEmVs8wlSaqcZS5JUuUsc0mSKmeZS5JUOctckqTKWeaSJFXOMpck\nqXKWuSRJlbPMJUmqXKOrpg0t6d06YNeuc9uOMMvYkovajtBZKW0nmK1r/1fT0+e1HWGWjrzE9zh/\n18/bjrDHhUuWth2h20q3njwXjnXl/2vwinZmLklS5SxzSZIqZ5lLklQ5y1ySpMpZ5pIkVc4ylySp\ncpa5JEmVs8wlSaqcZS5JUuUsc0mSKmeZS5JUuUZlnuTsJHcm2da/fSPJ7y5WOEmStH9NZ+b3A+8F\nVgGrgZuB65OsGHUwSZI0mEZXTSul/OucRRckeQfwSuCukaWSJEkDG/oSqEnGgDOAceBrI0skSZIa\naVzmSVYC/wkcBjwOnFFKuWfUwSRJ0mCG+Tb7d4HjgFcAHweuTfKykaaSJEkDazwzL6U8Afxv/89v\nJ3kF8A7g7fu6z4a1Gxk/YnzWsonJFaycnGj68JIkHYQ2928zbR/43kN/Zj7DGLBkvhVOWXcyy1ct\nG8FDSZJ0MFrZv820BfjkQPduVOZJ/gb4InAf8MvAG4ATgQ812Y4kSRqdpjPzXwGuApYD24DvAKeW\nUr4y6mCSJGkwTc8zP2uxgkiSpOH42+ySJFXOMpckqXKWuSRJlbPMJUmqnGUuSVLlLHNJkipnmUuS\nVDnLXJKkylnmkiRVzjKXJKlylrkkSZWzzCVJqpxlLklS5ZpeAnU4pfRuHTC25KK2I3Rb0naCPXZN\nn9t2BDXQkZf4HhcuWdp2hM7q0MscgPOnf952hE7asukJLj9+sHWdmUuSVDnLXJKkylnmkiRVzjKX\nJKlylrkkSZWzzCVJqpxlLklS5SxzSZIqZ5lLklQ5y1ySpMpZ5pIkVW5BZZ7k3CTTSS4ZVSBJktTM\n0GWe5OXA24E7RxdHkiQ1NVSZJ3k6cDVwFvDoSBNJkqRGhp2ZfwK4oZRy8yjDSJKk5hpfzzzJmcBL\ngQGvsipJkhZTozJP8mzgo8DJpZSdixNJkiQ10XRmvhp4FrApSfrLlgAnJnkXMF5KKXPvtGHtTYwf\nOT5r2cSZK1g5uWKIyJIkHVym1k8xde1ds5Ztf3THwPdvWuYbgZfMWXYlcDdw0d6KHOCUdWtYvmpZ\nw4eSJOnQMDE5wcTkxKxlWzZt5fLjrxjo/o3KvJTyGDDrrUOSx4CHSyl3N9mWJEkajVH8AtxeZ+OS\nJOnAaPxt9rlKKb89iiCSJGk4/ja7JEmVs8wlSaqcZS5JUuUsc0mSKmeZS5JUOctckqTKWeaSJFXO\nMpckqXKWuSRJlbPMJUmqnGUuSVLlLHNJkipnmUuSVLkFXzVtIEnv1gHT0+e2HUGD6shzZo/Srav9\njo19uO0Is0xPn9d2hFnOn97ZdgQNqGsv9a5oMi7OzCVJqpxlLklS5SxzSZIqZ5lLklQ5y1ySpMpZ\n5pIkVc4ylySpcpa5JEmVs8wlSaqcZS5JUuUsc0mSKteozJO8P8n0nNtdixVOkiTt3zAXWtkMrAF2\n/wT8E6OLI0mSmhqmzJ8opfx45EkkSdJQhvnM/AVJfpjkf5JcneQ5I08lSZIG1rTMbwXeBJwKnA08\nH/hqksNHnEuSJA2o0WH2UsqNM/7cnOQ24PvAGcAVowwmSZIGM8xn5nuUUrYl+W/g2PnW27B2I+NH\njM9aNjG5gpWTEwt5eEmSDgqb108xtX72yWE7tu0Y+P4LKvMkT6dX5J+eb71T1p3M8lXLFvJQkiQd\ntFZOTjxpgrtl01Y+tXqwg95NzzP/SJITkzw3yW8Cnwd2AuubbEeSJI1O05n5s4FrgGcCPwZuAX6j\nlPLwqINJkqTBNP0C3ORiBZEkScPxt9klSaqcZS5JUuUsc0mSKmeZS5JUOctckqTKWeaSJFWumjLf\nvH6q7QidtnnOzwDqF3zuzG9zaTtBd0353JmXr635HcjxqabM5/5mrWabutbx2RefO/Pb3HaADvN1\nNT9fW/M7kONTTZlLkqS9s8wlSaqcZS5JUuUWdAnUARwG8NDdDy14Qzu27WDLpq0L3g7l4Py2z45H\nRzQ+ycK30TEH63NnbERxtgNbRrCt6VGM8QiN4r9r+6heVx0zqpf5yF5bB6mFjs+M7jxsf+umLOIO\nKskfAZ9dtAeQJOng94ZSyjXzrbDYZf5M4FTge/QmAJIkaTCHAc8DbtzfpcYXtcwlSdLi8wtwkiRV\nzjKXJKlylrkkSZWzzCVJqlwVZZ7knUnuTfKzJLcmeXnbmdqW5LwktyX5SZIHk3w+yQvbztVVSc5N\nMp3kkrazdEGSX03ymSQPJXk8yZ1JVrWdqwuSLEny4f4+5/Ek9yS5oO1cbUlyQpLrk/yw/xo6fS/r\nfCDJA/3x2pDk2DayHmjzjU2SpyT52yTfSfLT/jpXJVm+GFk6X+ZJXg9cDLwfeBlwJ3BjkqNaDda+\nE4CPAa8ETgaWAl9O8tRWU3VQ/83f2+k9dw55SY4Evg7soHfq6IuBvwD+r81cHXI+8FbgHcCvA+8B\n3pPkXa2mas/hwB3AOcCTTn9K8l7gXfReY68AHqO3j/6lAxmyJfONzdOAlwJ/Ta+7/gB4EfCFxQjS\n+VPTktwKfLOU8u7+3wHuBy4tpfxdq+E6pP/m5kfAiaWUW9rO0xVJng7cTm/H/D7g26WUP283VbuS\nXAS8qpRyUttZuijJDcDWUsrbZiz7J+DxUsqftJesfUmmgd8vpVw/Y9kDwEdKKev6fz8DeBB4Yynl\nunaSHnh7G5u9rHM88E3guaWUH4zy8Ts9M0+yFFgN3LR7Wem9+9gIvKqtXB11JL13ho+0HaRjPgHc\nUEq5ue0gHfJa4FtJrut/RLMpyVlth+qQLwJrkrwAIMlxwKuBf2s1VQcleT6wjNn76J/QKyz30U+2\nez/96Kg3vNi/zb5QRwFL6L3Lm+lBeocrxJ6jFR8FbimleIHhviRn0jvMdXzbWTrmGHpHKi4GLqR3\naPTSJDtKKZ9pNVkHlFIuS/Ic4L+SPEFv0nN+KeXalqN10TJ65bS3ffSyAx+nu5KMAxcB15RSfjrq\n7Xe9zDWYy4AV9GYPApI8m94bnJNLKTvbztMxY8BtpZT39f++M8lK4GzgkC/zJH8GvBF4PXAXvTeE\nf5/kAd/saBhJngJ8jt4bn3MW4zE6fZgdeAjYBRw9Z/nRgJfqAZJ8HDgN+K1Sypa283TIauBZwKYk\nO5PsBE4C3p3k5/2jGYeqLcDdc5bdDfxaC1m66C+BD5ZSPldKmSqlfBZYB5zXcq4u2goE99H7NKPI\nnwP8zmLMyqHjZd6fUd0OrNm9rL8TXgN8o61cXdEv8tcBryml3Nd2no7ZCLyE3qzquP7tW8DVwHGl\n69/8XFxf58kfU70I+H4LWbpojN4kYqZpOr6/bEMp5V56pT1zH/0MemfZuI/+RZEfA6wppSzaGSM1\nHGa/BLgyye3AbcBael/5v7LNUG1LchkwCZwOPJZk9zvjbaWUQ/4KdaWUx+gdIt0jyWPAw6WUubPS\nQ8064OtJzgOuo7fjPQt427z3OnT8C3BBkh8AU8Aqevudf2w1VUuSHA4cS28GDnBM/0uBj5RS7qf3\ncdYFSe6hd4XMDwI/YJFOweqS+caG3hGwf6Y3ofg9YOmM/fQjo/74r/OnpgEkOYfeuZ5H0zun709L\nKd9qN1W7+qdB7O0/782llE8f6Dw1SHIzcMehfmoaQJLT6H0Z51jgXuDiUsrl7abqhiRPo3du8B/S\n2+c8AFxD79D7E21ma0OSk4Cv8OT9zVWllLf01/kreueZHwl8DXhnKeWeA5mzDfONDb3n0L1z/i39\nv19TSvnqSLPUUOaSJGnf/AxIkqTKWeaSJFXOMpckqXKWuSRJlbPMJUmqnGUuSVLlLHNJkipnmUuS\nVDnLXJKkylnmkiRVzjKXJKlylrkkSZX7f58vyCFn3W1mAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def get_sequence(length, dim):\n",
" X = np.concatenate((np.random.randint(2, size=(length,dim)), np.zeros((length + 3,dim))))\n",
" X = np.vstack(X) ; X[:,dim-1] = 0\n",
" \n",
" X = np.concatenate((X[-1:,:],X[:-1,:]))\n",
" y = np.concatenate((X[-(length + 2):,:],X[:-(length + 2),:]))\n",
" markers = range(length+1, X.shape[0], 2*length+3)\n",
" X[markers,dim-1] = 1\n",
" return X, y\n",
" \n",
"def next_batch(batch_size, length, dim):\n",
" X_batch = []\n",
" y_batch = []\n",
" for _ in range(batch_size):\n",
" X, y = get_sequence(length, dim)\n",
" X_batch.append(X) ; y_batch.append(y)\n",
" return [X_batch, y_batch]\n",
"\n",
"batch = next_batch(1, FLAGS.length, FLAGS.xlen)\n",
"plt.imshow(batch[0][0].T - batch[1][0].T, interpolation='none')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Helper functions"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def binary_cross_entropy(y, y_hat):\n",
" return tf.reduce_mean(-y*tf.log(y_hat) - (1-y)*tf.log(1-y_hat))\n",
"\n",
"def llprint(message):\n",
" sys.stdout.write(message)\n",
" sys.stdout.flush()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def free_recall_loss(y, y_hat, tsteps): \n",
" # sorry this dimension stuff is uuuuugly but we have to because it's batched\n",
" y = tf.expand_dims(y, [1])\n",
" y_hat = tf.expand_dims(y_hat, [1])\n",
" \n",
" y_hat = tf.tile(y_hat,[1,tsteps,1,1])\n",
" y_hat = tf.transpose(y_hat, [0,2,1,3])\n",
" \n",
" y_minus = -y*tf.log(y_hat) - (1-y)*tf.log(1-y_hat) # binary cross entropy loss\n",
" y_minus = tf.reduce_sum(y_minus, axis=-1)\n",
" y_minus = tf.reduce_min(y_minus, axis=1)\n",
" \n",
" return tf.reduce_sum(y_minus) / (FLAGS.batch_size*FLAGS.length)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"permute by: [0 4 3 1 2]\n",
"guessed permutation: [0 4 3 1 2]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAGMAAAB1CAYAAABatF8TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAB65JREFUeJztnWuoXcUVx3//xOC7LXh9gVVbrIIoUeOzmgSjEPFDBBFf\nhaKiEh/9EBAfIApBpFAMFdNAY6lXESv9YMBSzNUY+5AY0l41eqlWfMRnvDURrxAjJnH5YfbJPZ6c\ns8+ZObN3hu75webeM2fPnnX22vNae80amRmZNJixtwXITJOVkRBZGQmRlZEQWRkJkZWREFkZCZGV\nkRD7VHlxSYcAC4FNwNdVlpUw+wHHAmNmtrX0TDOr7ACuBiwfGHB1v/sVVDMk3QLcBhwBbAR+ZWb/\n6nLqJvfnUmCky9ergYs8S4+Zp8wUNIar1N/nRh72KmUL8JT7d1OZhBDQTEm6AngAuBHYACwBxiQd\nb2ZbOk4vmqYR4MguV9uvR3oZMfOUKaN7nrKS+0jWt5kO6cCXAL83s8fM7E1gMfAVcF3AtTJteClD\n0ixgDvB8K81c57AGOCeuaM3Dt2aMADOByY70SVz/kRmCSoe206zGtajtnFQcvqSb54fAnzrSfMbz\nvsrYAuwCDu9IPxz4tHe2i/DvdHtxck15/JVxcZe0zcDKAfN7NVNmtgMYBy5opUlS8Xmdz7UyexLS\nTC0DRiWNMz20PQAYjShXI/FWhpn9WdIIsBTXPL0KLDSzz2IL1zSCOnAzWwGsiCxL48lW24SoaWjb\nspUNxj0srU6UNpZyTy3lDIp3zZA0V9LTkj6W9K2kRVUI1kRCmqkDcZ32zfg87pm+hIymVuOm1K05\nRiYSuQNPiKyMhKhpNDVGPENhurwOTHSkVWkoDGQh8QyF6XIye5okKzMUZqol5B34gcBxQGsk9VNJ\ns4HPzezDmMI1jZBm6nTgBaan1Q8U6Y+S34MPRcg84+/k5q0S8k1NiJpGU2K6i+lPXQa8ugySg+Lr\nqnOXpA2SvpQ0KWmVpOOrEq5p+DZTc4GHgLOAC4FZwLOS9o8tWBPxaqbM7HsOEJKuAf6Hc2x7MZ5Y\nzWTYDvxHuOHt5xFkaTzByijM578FXjSz/8QTqbkMM5paAZwInNv/1F4ehSHOZemyVwyFkpbjHOjm\nmtnm/jliehSmy7CGwhDb1HLgEmC+mX3gmz/TGy9lSFoBXAUsArZJavncTplZU9fsRcO3A18M/AD4\nG/BJ23F5XLGaie88I9uyKiTf3ISoxVB4AyuTHEvVY5DcDCUrZNvxNRQulrRR0lRxrJPkuw440wPf\nZupD4A7gNJw9ai3wtKQTYwvWRHw78L92JN0t6SacFTebRIYkuM+QNAM3pN0X+Gc0iRpMyAz8JOAl\nnLHpK+ByM3s7tmBNJKRmvAnMxq20vQx4UtJ8M3ulV4Zm+BOCMxOGmwpDvEN2Au8WH1+RdCZwEy6W\nSFea4U8I3R+xioa2JdeYGeE6jcfXUHg/8AzwAXAw8AtgHnBffNGah28zdRjOc/BIYAp4Dbfs+IXY\ngjUR33nG9VUJksmGwqSoyaPQj/8nj8LBx1JD1gxJdxbLj5cNc52MYxhXnTNwc4uN8cRpNkHKkHQQ\n8DhwPfBFVIkaTGjN+B3wFzNbG1OYphNiKLwSOAW3gikTEd8Z+FE4l84Li6hsA9EUQ+FwZkL/mjEH\nOBR4uS1UxUxgnqRbgX2ty45aTTEUDmcm9FfGGvb0YBwF3gB+3U0RmcHxNYdso+P1qqRtwFYzeyOm\nYE0khjkk14ZIDG0OMbMFMQTJZENhUqjKPlfSacA43IDfeKqumGJ1tLC7x1NzzOzlsjN9PQrvLQyD\n7Uf2l4pESJ8xgQu33Xp8d8YTp9mEKGNnju5cDSEd+M+KMKrvSHpc0o+jS9VQfJWxHrgGZ+FYDPwE\n+EcRgyozJL4z8LG2jxOSNgDv43xuH+mds5epsP3voLyO/5LlkDwT+Mv2DM5ppp0KPQrbMbMpSW/h\nIrOV0MtU+CT+P3iCsBtbhzKmgCs70mryKCze+B1XlJgZEt95xm8kzZN0jKSfA6uAHey51VAmAN9m\n6ijgCeAQ4DNcJJ2z++5ZmhkI3w78Ks/rF71258aWLb6mewtXZg7plaeMXnnKzCGxytn92ztHMF3E\nyRvtJrPRbtWGwrwFtccW1JUqI+NHfp+REFkZCZGVkRBZGQmRlZEQe0UZkm6R9J6k7ZLWF8sLys73\n2o4uJDJ1jCA1w65XqV0Zkq7AbfNwL3Aqbn3HWLFfbC98t6MLiUw9VJCaKOtVqpyB95iVrwcebPss\n4CPg9gHzfwss8ixzpMh3nme+rcC1A5x3EPBfYAFub5FlIfem1pohaRbuqXu+lVb4564BzqmwaK/I\n1JJmFEsfBg1SE2W9St0LLEdwXuuTHemTwAlVFOgTmTokSE3M9SpJrnaNjEdkar8gNaHrVXpSc38x\nC/cyalFH+iiwKnafASzHvaM/OlDe54CVJd9fAuwCvil+145Cvlaaku0ziqdnHOcEB+xuRi4A1sUs\nqy0y9fkWHpm6X5Ca1nqVU3A1ajbwb9zi09lFfzgwe6OZWgaMShoHNgBLgANwtaMrvtvRhUSmDglS\nY7HXq9TZTLVV75tx7zi24zrM0/ucP5/p6t9+/LGkKes8dxfwy5Iy/oCLo7Ud+BR4FlgQ8NvWEji0\nze8zEiLbphIiKyMhsjISIisjIbIyEiIrIyGyMhIiKyMhsjISIisjIbIyEuI7aIsTsxBe7NYAAAAA\nSUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAGMAAAB1CAYAAABatF8TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAB5BJREFUeJztnWusXFUVx3//YgMCPhKuIAmKGsSEYAqCD4S2oZCU+KHE\nxABiYtBQUkA/NDE+EgOxIYTE0ECsTWyNXg1R4gebYNReLOUhqU21YvVG0CgioOVKS7wmpcS2LD7s\nM73TuTNzZ+/Z53Q3Z/2Sk8nsOfvsNbNmv9Zee22ZGU4ZLDreAjhzuDIKwpVREK6MgnBlFIQroyBc\nGQXhyiiIN9X5cElnACuB54DX6iyrYE4B3gNMmdn+oXeaWW0XcCNgfmHAjQv9Xkk1Q9LtwJeAdwJ7\ngC+a2W/73PpcePkkMNHn4ylCxTmW1WweWHb/HLCZW4ZIvBW4ZsjndebZB/wUjv4Wg4lWhqTrgXuB\nW4BdwFpgStL5Zrav5/aqaZoAzu7ztFP6pve7c3iO9FwN5lmwmU7pwNcC3zGzH5rZM8Aa4FXg8wnP\ncrqIUoakxcAlwCOdNAudwzbgsryitY/YmjEBnATM9KTPEPoPZwxqHdrOMUVoT7u5sLriiM+Rmisl\nz9uAH/ekjT6ij1XGPuAIcFZP+lnAS4OzrSS+M+xPmjI+2FCeT/RJ2wtsGil3VDNlZoeA3cBVnTRJ\nqt7viHmWM5+UZmo9MClpN3ND21OByYxytZJoZZjZTyRNAOsIzdMfgJVm9nJu4dpGUgduZhuBjZll\naT1utS2Ihoa2TRDvcnQH62qQ41j2whBL27FE1wxJSyU9JOlfkl6XtCr2GU5/Upqp0wid9m2k/B2d\ngaSMprYSbMWdOYaTCe/AC8KVURAnnKGwZKarq5uYhf+GlJHPUFgy/f5etQ5tnfpIWQM/DTgP6Iyk\n3idpCfCKmb2QU7i2kdJMXQo8ypwLyr1V+g/wdfCxSJlnPI43b7XgP2pBFGkoTJnWpxj91nFHQkmx\njD6einXV+ZqkXZL+J2lG0hZJ56eI6MwntplaCnwL+ChwNbAYeFjSm3ML1kaimikzO8b9QdJNwH8I\njm1P5hOrnYzbgb+dMLx9JYMsrSdZGZX5/D7gSTP7cz6R2ss4o6mNwAXA5Qvf2g5D4bimwtT9GRsI\n7nNLzWzvwjnaYSgc11SYYpvaAFwLLDez52PzO4OJUoakjcCngVXAAUkdn9tZM2vrnr1sxHbga4C3\nAo8B/+66rssrVjuJnWe4LatG/MctiEYMhavZHDWW+kYjBrwT3KNQ0hpJeyTNVtcOSbH7c50BxDZT\nLwBfAT5EsEdtBx6SdEFuwdpIbAf+856kr0u6lWDFdZPImCT3GZIWEYa0JwO/ziZRi0mZgV8I/IZg\nbHoVuM7M/pZbsDaSUjOeAZYQ9tl+CnhQ0nIze2pQBjcTjkaKd8hh4Nnq7VOSPgLcCoMjqbiZcDRy\nTPoWEaImOGMSayi8G/gl8DzwFuAzwDLgrvyitY/YZupMgufg2cAs8EfCtuNHcwvWRmLnGTfXJYjj\nhsKiaMRQKOK8BJsw4MEJ7lHYi6SvVtuP14/zHCcwjqvOhwlziz35xGk3ScqQdDrwAHAz8N+sErWY\n1JrxbeBnZrY9pzBtJ8VQeANwEWEHk5OR2Bn4OQSXzqurqGwjsZX+hsKUwHNl06xH4SXAO4Dfd4Wq\nOAlYJukLwMnW50Sta2iHobBpj8JtzP9DTwJPA/f0U4QzOrHmkAP0LK9KOgDsN7OncwrWRnKYQ7w2\nZGJsc4iZrcghiOOGwqJoxFC4idWUOZ5qInbZ6GXEehTeWRkGuy/3l8pESs2YJoTb7qj8cD5x2k2K\nMg57dOd6SOnA31+FUf27pAckvSu7VC0lVhk7gZsIrlBrgPcCT1QxqJwxiZ2BT3W9nZa0C/gnwef2\n+4NzDvIp7H4dlemG8vyJeFPmLwhOM93UvPW4g5nNSvorITLbEAb5FD5IucqYJl4Zs4T9p93UdJhJ\nL9WK33lVic6YxM4zvilpmaRzJX0c2AIcYv5BQ04Csc3UOcCPgDOAlwmRdD624JmlzkjEduC9DeJC\nVL1278GWHV4jvoXLmWeYqSJXOUe/e+8IZj5+0G45B+2qzsU5P4IaiDiCulZlOHH4ekZBuDIKwpVR\nEK6MgnBlFMRxUYak2yX9Q9JBSTur7QXD7o86ji4lMnWOIDXj7ldpXBmSricc83AncDFhf8dUdV7s\nIGKPo0uJTD1WkJos+1XqnIEPmJXvBO7vei/gReDLI+Z/HVgVWeZEle+KyHz7gc+NcN/pwF+AFYSz\nRdan/DaN1gxJiwn/ukc6aZV/7jbgshqLjopMLWlRtfVh1CA1WfarNH1kwwTBa32mJ30G+EAdBcZE\npk4JUpNzv0qR52dkJiIydVyQmtT9KgNpuL9YTFiMWtWTPglsyd1nABsIa/TvTpT3V8CmIZ9fCxwB\n/l99r0OVfJ00FdtnVP+e3QQnOOBoM3IVsCNnWV2Rqa+09MjUCwWp6exXuYhQo5YAvyNsPl1S9Ycj\nczyaqfXApKTdwC5gLXAqoXb0JfY4upTI1ClBaiz3fpUmm6mu6n0bYY3jIKHDvHSB+5czV/27r+8N\nacp67z0CfHZIGd8lxNE6CLwEPAysSPhu20kc2vp6RkG4baogXBkF4cooCFdGQbgyCsKVURCujIJw\nZRSEK6MgXBkF4cooiDcAgcEJ8jLx8x4AAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def remove_duplicates(k, l):\n",
" i = 0 ; removed_duplicate = False\n",
" while i < len(k) and not removed_duplicate: \n",
" if k[-i] == k[-i-1]:\n",
" ri = np.random.randint(2)\n",
" k = np.delete(k,-i-ri) ; l = np.delete(l,-i-ri) ; removed_duplicate = True\n",
" k,l = remove_duplicates(k,l)\n",
" i+=1\n",
" return k,l\n",
"\n",
"def guess_recall_order(real_y, pred_y, tsteps):\n",
" # sorry this is uuuuugly but we have to because it's batched\n",
" real_y = np.tile(real_y, [1,1,1,1]) ; real_y = np.transpose(real_y, (1,0,2,3))\n",
" pred_y = np.tile(pred_y,[1,1,1,1]) ; pred_y = np.transpose(pred_y, (1,0,2,3))\n",
" \n",
" pred_y = np.tile(pred_y,[1,tsteps,1,1])\n",
" pred_y = np.transpose(pred_y, (0,2,1,3))\n",
" \n",
" real_y = real_y[0,:,:,:] ; pred_y = pred_y[0,:,:,:]\n",
" y_minus = .5*(real_y - pred_y)**2\n",
" y_minus = np.sum(y_minus, axis=-1)\n",
" y_mins = np.amin(y_minus, axis=1)\n",
" \n",
" k, l = np.where(y_minus == np.tile(y_mins,[tsteps,1]).T)\n",
" k, l = remove_duplicates(k, l)\n",
" return l\n",
"\n",
"# test\n",
"X, real_y = next_batch(FLAGS.batch_size, FLAGS.length, FLAGS.xlen)\n",
"real_y = np.stack(real_y)[:,-FLAGS.length:,:]\n",
"\n",
"p = np.random.permutation(real_y[0].shape[0]) ; print 'permute by: ', p\n",
"pred_y = [y_i[p,:] for y_i in real_y]\n",
"pred_y = np.stack(pred_y)\n",
"\n",
"plt.figure(0, figsize=[1,1]) ; plt.imshow(real_y[0,:,:].T, interpolation='none')\n",
"plt.figure(1, figsize=[1,1]) ; plt.imshow(pred_y[0,:,:].T, interpolation='none')\n",
"\n",
"print \"guessed permutation: \", guess_recall_order(real_y, pred_y, FLAGS.length)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Build graph, initialize everything"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"building graph...\n",
"defining loss...\n",
"computing gradients...\n",
"init variables... \n",
"ready to train..."
]
}
],
"source": [
"sess = tf.InteractiveSession()\n",
"\n",
"llprint(\"building graph...\\n\")\n",
"optimizer = tf.train.RMSPropOptimizer(FLAGS.lr, momentum=FLAGS.momentum)\n",
"dnc = DNC(RNNController, FLAGS)\n",
"\n",
"llprint(\"defining loss...\\n\")\n",
"y_hat, outputs = dnc.get_outputs()\n",
"y_hat = tf.clip_by_value(tf.sigmoid(y_hat), 1e-6, 1. - 1e-6) # avoid infinity\n",
"rlen = (dnc.tsteps-3)/2\n",
"loss = free_recall_loss(dnc.y[:,-rlen:,:], y_hat[:,-rlen:,:], tsteps=rlen)\n",
"\n",
"llprint(\"computing gradients...\\n\")\n",
"gradients = optimizer.compute_gradients(loss)\n",
"for i, (grad, var) in enumerate(gradients):\n",
" if grad is not None:\n",
" gradients[i] = (tf.clip_by_value(grad, -10, 10), var)\n",
" \n",
"grad_op = optimizer.apply_gradients(gradients)\n",
"\n",
"llprint(\"init variables... \\n\")\n",
"sess.run(tf.global_variables_initializer())\n",
"llprint(\"ready to train...\")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"model overview...\n",
"\tvariable \"dnc_scope/basic_lstm_cell/weights:0\" has 73728 parameters\n",
"\tvariable \"dnc_scope/basic_lstm_cell/biases:0\" has 512 parameters\n",
"\tvariable \"W_z:0\" has 6144 parameters\n",
"\tvariable \"W_v:0\" has 768 parameters\n",
"\tvariable \"W_r:0\" has 60 parameters\n",
"total of 81212 parameters\n"
]
}
],
"source": [
"# tf parameter overview\n",
"total_parameters = 0 ; print \"model overview...\"\n",
"for variable in tf.trainable_variables():\n",
" shape = variable.get_shape()\n",
" variable_parameters = 1\n",
" for dim in shape:\n",
" variable_parameters *= dim.value\n",
" print '\\tvariable \"{}\" has {} parameters' \\\n",
" .format(variable.name, variable_parameters)\n",
" total_parameters += variable_parameters\n",
"print \"total of {} parameters\".format(total_parameters)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"loaded model: rnn_models/model.ckpt-30000\n"
]
}
],
"source": [
"global_step = 0\n",
"saver = tf.train.Saver(tf.global_variables())\n",
"load_was_success = True # yes, I'm being optimistic\n",
"try:\n",
" save_dir = '/'.join(FLAGS.save_path.split('/')[:-1])\n",
" ckpt = tf.train.get_checkpoint_state(save_dir)\n",
" load_path = ckpt.model_checkpoint_path\n",
" saver.restore(sess, load_path)\n",
"except:\n",
" print \"no saved model to load.\"\n",
" load_was_success = False\n",
"else:\n",
" print \"loaded model: {}\".format(load_path)\n",
" saver = tf.train.Saver(tf.global_variables())\n",
" global_step = int(load_path.split('-')[-1]) + 1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Train loop"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"loss_history = []\n",
"for i in xrange(global_step, FLAGS.iterations + 1):\n",
" llprint(\"\\rIteration {}/{}\".format(i, FLAGS.iterations))\n",
"\n",
" rlen = np.random.randint(1, FLAGS.length + 1)\n",
" X, y = next_batch(FLAGS.batch_size, rlen, FLAGS.xlen)\n",
" tsteps = 2*rlen+3\n",
"\n",
" fetch = [loss, grad_op]\n",
" feed = {dnc.X: X, dnc.y: y, dnc.tsteps: tsteps}\n",
"\n",
" step_loss, _ = sess.run(fetch, feed_dict=feed)\n",
" loss_history.append(step_loss)\n",
" global_step = i\n",
"\n",
" if i % 100 == 0:\n",
" llprint(\"\\n\\tloss: {:03.4f}\\n\".format(np.mean(loss_history)))\n",
" loss_history = []\n",
" if i % FLAGS.save_every == 0 and i is not 0:\n",
" llprint(\"\\n\\tSAVING MODEL\\n\")\n",
" saver.save(sess, FLAGS.save_path, global_step=global_step)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1AAAAHpCAYAAACMQd2lAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xm4JFV5+PHvOzMwbMKgyOKKBFxgEBkQRQGNoBB3NFFG\nk7gTXPMjC4IoKIlCNAoiEncHFyAYNzQRBJcIoiKMomxRBNxYhmVmkIGBYeb9/VF1oadvL9XL7ep7\n7/fzPPUwXV3nnLerL33OqTp1TmQmkiRJkqTu5tQdgCRJkiRNF3agJEmSJKkiO1CSJEmSVJEdKEmS\nJEmqyA6UJEmSJFVkB0qSJEmSKrIDJUmSJEkV2YGSJEmSpIrsQEmSJElSRXagJEmSJKkiO1CSJEmS\nVNG06EBFxKsjYl1EPKruWAYREUdExJVDyOfvIuK3EbHBMOIaloh4d0SsqzuOcdHP9z2u360kqbVW\ndd+w2y3j0A7qVKdFxLzm2KzPNJNNiw4UkOU2FiJi74g4NiI27yHNg4AjgBOGEMISYEPg74aQ1zBV\n+p76OX91GvH3vYTx/G4lSa21qvt6brd0qWtqbQd1qtMiIoBPAJ+JiIc3vLUE6zPNUNOlA/U5YOPM\n/F3dgZSeBhwDLOghzeuAucCZzW9ExCsj4qLy6tK9EbGk4b3/KPffHhFfi4iHZuY9wGnAPwz2MWrT\nz/mr01C/705mwHcrSeqv3dKprqm7HdSpTvs3ivheBhwfEVuC9ZlmtmnRgcrCvXXH0SD6SPNq4OxW\nnyMzv5iZTwMuA67MzFc3vH0ycDbw6Mx8cWbeUu4/C9g+Ip7ZRyx16+f8dc80YpOpyJchf98VTOfv\nVpKmhSmsM/ptt7Sta8agHfRq2tdpH8vM72fm7cCbKTpaE6zPNCNNiw5U89jfifHGEfFnEbEkIpZH\nxIqI+ExEbNSUduLYx0XEWRGxMiJujYiTImJ+w3FLIuK6FmWvN7Y5Io4F3l++vL7Me22ncckRsT3w\nROD8Lh/1k8CuEbFHme6hwNuAl2bmnxoPzMylwO3Ai7rkSUQ8KiJOjYirI+Ku8vOfFRGPbvVZK57X\nfSLipxFxd0T8OiIO7RZHma7j+esj1idExOkRcTtwQcP7z4yISxrjazNO/WHl57spIlZHxOUR8Zqq\n8bb5jNtT7ftuqZfvVpJmsx7q+G51Rse6oOG4SnVfc7uloYxPR8QfyzKuLeu7eRXqxpbPQEXE7hHx\nrfJz/ykizo+Ip7Q5R13r9jafZXs61GmZeW3Dv/+Umbc2vLY+04w0r+4AKmoe+zvx77OAa4EjgUXA\n64GbgaPaHHtdeexTKTomCyiuqrQqo13ZXwYeCxwC/D1wW7n/Ftp7WpnH0g7HAHwR+ADw+oi4Cngf\ncHhmrm1z/FLg6V3yBHgyxWc+A/gDsD3wJuB7EbFzZq4uj6t0XiNiIXAusIxiuMEGwLvL1910O3+9\nxvol4FdlbFHGtzvwLeAG4F0Uf+fvAm5tSEdEbA38BFhLcafvVuAvgE9HxIMy82TgK13ibaXq991J\n1e9WkmazXup4aF1nVKkLeq371ms7RMR2wE+BzYGPA/8HPBz4S2ATutc1k9ooEbEz8ANgJcWzSfdR\nPG/0/YjYLzN/2uIcdWsztTJonWZ9ppknM8d+A15F8cP2qPL1scA64BNNx30ZWNa0b+LYrzTtP6XM\nc2H5+rPAtS3KPhZY27TvHxvjqRD/ceXxm1Q4dgnFj+HHgW27HPsx4M4Kec5vsW+v8ry8ssW56nhe\nga8Cq4CHN+x7HLCm+Vy1iaft+esj1s+3OP5s4E/ANg37dgDubYwP+BRFJ21BU/rTKa6YzR/29w38\nNUUn+aSGfRtSVOjb9frdurm5uc3mrYc6vlOdUbUuqFz3Mbndclp53O4dPkunuvFVze+V8dxNMcR/\nYt+2ZRviey3OUdc2U5u4WtZp1mdus3mbFkP42kiKTkajC4CHRMRmLY79aNO+j1BcfXru1IS3nocA\n92XmXRWO/STwIODyzLypy7HLgY273YLP4kFO4P6pRh9McRVqBcVVqPUOp8N5jYg5wHOAr2bmHxvK\n+D+KK3MDGTTWMr79ga9l5s0N+V5LcVeq0UuAbwBzI+IhExvwbWCLFuVV1fL7jojHUXTivgK8tuGt\nPYCtgcbvu9J3K0mqXMe3qt+gc12wAFg0SN0XEUExhO3szPxZT5+sfZ5zgGeX8fy2IZ6bKDp++zS1\nhXppMzWbVKdZn2m2m84dKIDm2WiWl//dssWx1zS9/g3FFZnthxzToPaguBK2uMKxEw+cdpzaNCI2\niojjIuJ3wD0UwxOWUXQStmiRpNN5fSiwMZPPJxRDEgbSR6zNz61t3SG++/dF8XzZAuBQiiESjdtn\nGvIapqcDXwOeCVzSsP9pwEWZ2fg9VvpuJUlA9Tp+vTqjQl2QFHXBIHXfQymG7l3R5bhePJRi6N+v\nWrx3FUX77pFN+3tpM3VjfaZZbbo8A9VOu2eDqsya1mrNhlbmttnfi9uAeRGxaWauandQRLyM4qrN\nvwMnRsQumdnpB3dL4K7GuzZtnEJx+/9E4McUt/cT+E9ad6IHOa+D6jXWu/ssZyKvL1AMrWjlF33m\n3fL7zszPAETEiymGjE54GsU49kZVv1tJ0mTt6vTmOqNqXTCMtkDd+q3bJ9Vp1mea7aZ7B6oXOwG/\nbXi9I8UP58TVqOW0Xnth+xb7er2KcnX538cAl7c6ICKeQfHM08kRsYDigdBDKR4mbecxFFeaunkp\nsCQzj2gobz79rcN0C0UFtFOL9x5fMY9O52/QWJcBqym+32aNMd9C8ZzU3Mz8bpc8h/Z9R7E+xh4U\nDw5P2Bv4YFMeVb9bSVL3Or6dSnVBOWSu37rvFuAOYGGX43qpa24B7qJ4BqvZEyjuvv2+h/w6aVmn\nWZ9pNpvuQ/iqCoq1CRq9jeLH6pzy9W+ALcpZdopExaw5L26R38RdhaqN+h+VMezZMrhiJp39s5zp\nJzNXUDzc+deN07C2sAi4qEL5a5n8Xb+NPq6oZeY6ivHeL46IR0zsj4gnUIwPr6LT+Rso1jK+88v4\ntm2Ib0fgoKbjvgy8NCJ2ac4nIraqGG8rnb7vPwNWZObvy3J2oBia+NOm46p+t5I023Wq45uffV1P\n1bpgkLqvHM72NeAFEdHp2drKdU0Zz7eBFzVNlb4NxSMAF2Tmnd3yqahdnWZ9pllrNt2BekxEfJ2i\nw/Q04JXAFzLzl+X7Z1Kspv21iDgZ2BQ4jGJsc/MP3qUUPybvi4gzKWbWOTszWw4ny8zrIuJy4ACK\nWfbuFxF7AUdl5sFNyT5VxvhyihW+aUq3B/Bgih/lbr4J/E1E3AFcSXGFaH+K54v6cSxFZ+TCiDiV\nYirXt1BcmXpihfSdzt8wYn03RYV2UUT8B8Xf+ZvL+HZrOO5IivHbP4mIT5blPZjiitqzgIlO1NC+\nb4ornRtERJSV6huBizNzzcQBPX63kqT2dXzLUR9NqtYFg9R976CY9OEHEfEJijsyD6O4e/P0zLyD\nHusa4J0U9cwPy3jWUoxc2RA4ok2annWo06zPNHvVPQ1glY3W05ivBR7c6bimYx9HsQbCCorG+EnA\nhk3p9wcuo7hNfyXFVZxJ05iXx76D4oHMNc1ltvkM/4/ieZ6J6VBfTDFOeA3F+OLnNBz7VIox12vL\neL/c4rOeAFxX8fxtTtEhu7mM4b8phiFcC3y6xbmqcl73AS4uz9WvgTe0O1dtYmp5/iiuXvUda8P7\nz6R4sHUivtdTTLe6qum4rSjW/bieYujfHymu6r12mN9303v/QvGM15EUdz6P6fe7dXNzc5vNW9U6\nvkKdUbUuqFT3tak3H0HxvNBNFMPvfg18GJjXcEy7unFSfuX+3YD/KeubPwHnAXu1OUdd6/YO57ll\nnWZ95jZbt8ic2ZOiRLG69zHAQzPz9hrj2Jzix+WIzPxst+O75LUhxY/8+zLzlCGENytExFeBnTOz\n1ZjxYZc16fuOiA2ADwEfzsxryiGivwZ2zczrymP8biWponGp42e65jrN+kyz3Wx5Bqp2Wdye/wDw\nz0PI7jUU6y+0Ws9CFNOhN73eiWI9kO+Novw23/eTgL+lmHoWijW/jp6obEp+t5KksdKiTrM+06zm\nHSjNSBFxA8VY7WspZlI8jGK8+qLM/E1NMc2jGOZwL7Ad8OPM/M86YpGkmcA6vh7WZ5rtZtMkEppd\nvgUcAmxLsSDvRcA76uo8AWTmfcC/1lW+JEnDYH2m2W7G34GSJEmSpGHxGShJkiRJqmhKh/BFxEOA\nA3lgWlBJo7URxTNg52bmbTXHIkkjZ1tkyli/aNaa6megDgS+OMVlSOrulcDpdQchSTWwLTK1rF80\n60x1B+p6gBd94QVs9YSt2h503uHn8+wTD5jiUKoZSSwVnjs77/Dv8OwT9+94zJw9lwwcyrpLX1Mh\nlvH4fsYlDugey7AeLfzMnht0OeIc4KC272622UruvPM/ofx/UZJmoesBDqZYLbedcyl6Wu18kkMr\nFNX5N7maccmjWz63Al8B6xfNQlPdgVoNsNUTtmK7Rdu2PWj+FvM7vj9KI4mlQut6/oLucQzjAbZ1\nFT7ruHw/4xIHdI9leHOzbNjl/Y0oZpBtbe7c+RP/dNiKpNmqaIvQ6dey268pXd+tmsv0yaNyPtYv\nmnX6aoNHxJsj4rqIuDsifhwRTx52YJIkSa3YDpFUp547UBHxcuCDwLHA7sBlwLkR0enOuCRJ0sBs\nh0iqWz93oA4HPp6Zn8vMq4HDgLuA1w41MkmSpMlsh0iqVU8dqIjYANgD+M7EvixW4j0f2LvfIHZZ\nvHO/SYduXGLZ5ZDxiAPG6JyMSRwwTrEsrDsASRqZqWqHwLB+TYeRy7jkMcx8pJml1ztQWwFzgZub\n9t8M9P10/8LFu/SbdOjGJZaFY9NAH6dzMh5xwDjFsmvdAUjSKE1JOwSG1VUYxm/yuOQxzHykmWWq\nZ+EDiimf528xf719uyzeeYwaodJM8Evg8vX2rFq1pp5QJGnMnEsxp1yjhXiPpZrJ9YuT72k267UD\ndSuwFtimaf82wE3tEj37xAPGZvppaebalearhZtuejsrV36knnAkafj6aodAscbTMCb2np0m1y9w\nI/CJGmKR6tfTEL7MXANcCty/wmtERPn6ouGGJkmS9ADbIZLGQT9D+D4ELImIS4GLKWbD2QRYMsS4\nJEmSWrEdIqlWPXegMvOscq2F4yhumf8cODAzbxl2cJIkSY1sh0iqW1+TSGTmqcCpQ45FkiSpK9sh\nkurUz0K6kiRJkjQr2YGSJEmSpIrsQEmSJElSRSNZSJfMYhvAnLknDBzGurVHDpzHnDmDxzHTDPjV\n3u+9czcYOI+j1w6+cOx75wwehyRpvES59esY3jOsUMbCcRxbdwjStOUdKEmSJEmqyA6UJEmSJFXU\ncwcqIvaNiLMj4o8RsS4iXjgVgUmSJDWzHSKpbv3cgdqUYtG6NwFDevpFkiSpEtshkmrV8yQSmXkO\ncA5ARAzyPKYkSVJPbIdIqpvPQEmSJElSRXagJEmSJKmikawDdd7h32H+gvnr7dvlkJ1ZuHjnURQv\nzRK/BC5fb8+qVYOviyVJM8E5wEZN+xYCu9YQy/QzuX6B1XUEIo2FkXSgnn3i/my3aNtRFCXNYrvS\n3BTYdNPbWbnyI/WEI0lj5CBgu7qDmLYm1y9wI/CJGmKR6ucQPkmSJEmqqOc7UBGxKbAjMDHzzQ4R\nsRtwe2b+fpjBSZIkNbIdIqlu/Qzh2xP4HsXaCwl8sNx/GvDaIcUlSZLUiu0QSbXqZx2o/8Whf5Ik\nqQa2QyTVzR8gSZIkSarIDpQkSZIkVWQHSpIkSZIqGsk6UEORdQcwPOvWHVl3CONpBn3HR68bjwVs\nly+9j1P3rDsKSarfJ3gDg6wEdSzHDRzDsKq54zhmSDlJ6od3oCRJkiSpIjtQkiRJklRRTx2oiDgq\nIi6OiDsi4uaI+GpEPHaqgpMkSZpgO0TSOOj1DtS+wEeApwAHABsA346IjYcdmCRJUhPbIZJq19Mk\nEpn53MbXEfFqYBmwB3Dh8MKSJElan+0QSeNg0GegFlBMKnP7EGKRJEnqhe0QSSPXdwcqIgI4Cbgw\nM68cXkiSJEmd2Q6RVJdB1oE6FdgZePqQYpEkSarKdoikWvTVgYqIU4DnAvtm5o3djj/v8O8wf8H8\n9fbtcsjOLFy8cz/FS2rhijOu4Ioz178Iu2bFeCzoK0nD1Gs7pHAusFHTvoXlps5+CVzetG91HYFI\nY6HnDlT5o/Ui4BmZ+bsqaZ594v5st2jbXouS1INdFu/CLot3WW/f8qUrOHXP/6gpIkkavn7aIYUD\nge2mKKqZbtdya3Qj8IkaYpHq11MHKiJOBRYDLwRWRcQ25VsrM9NLEZIkacrYDpE0DnqdROIwYHPg\n+8ANDdvLhhuWJEnSJLZDJNWu13WgBp32XJIkqS+2QySNA3+IJEmSJKkiO1CSJEmSVJEdKEmSJEmq\naJCFdKuLKLYBrFt75MBhzJl7wsB5DCWOOYPHAcBgpxSAdeuOGjyMIcQBcPTawdcseu/cDcYjjjmD\nxzEMW2wxmv/FJWncvYFPDjSJ+Xs4ZmixDOoYjqs7BG4EPll3EFJNvAMlSZIkSRXZgZIkSZKkinrq\nQEXEYRFxWUSsLLeLIuKgqQpOkiSpkW0RSXXr9Q7U74G3A4uAPYDvAmdHxM7DDkySJKkF2yKSatXr\nQrr/3bTrnRHxRuApwJVDi0qSJKkF2yKS6tb3FF0RMQd4GTAfuGBoEUmSJFVgW0RSHXruQEXEQuBH\nwEbAXcDLMvOaYQcmSZLUim0RSXXqZxa+q4HdgL2AU4AzI2L3oUYlSZLUnm0RSbXp+Q5UZt4HXFu+\n/FlE7AW8ETi0XZrzDj+f+VvMX2/fLot3ZuHiXXotXlJbl5fbA1atGnxRYEkaN/20Rc6luF3VaGG5\nqbPJtQusriMQaUz0/QxUgznA3E4HPPvEA9hu0bZDKEpSe5ObAptuupyVKz9STziSNDpd2yIHAtuN\nJpYZp1VH80bgkzXEIo2DnjpQEfE+4FvA74AHAa8E9gP+dfihSZIkrc+2iKS69XoHamvgNIqLOCuB\nXwAHZub3hh2YJElSC7ZFJNWq13WgXj9VgUiSJHVjW0RS3fqZhU+SJEmSZiU7UJIkSZJUkR0oSZIk\nSarIDpQkSZIkVTSMdaC6yyy2AcyZd8LAYaxbe+TAeRAxHnEAc+YOfk6GYcCv9n7vnbfBwHkcvXbw\nhWOH8BUPJQ6A987dcCj5SNJsF+XWr2M4buAYjuPYgfMo8jlmKPkMxpWgNHt5B0qSJEmSKrIDJUmS\nJEkVDdSBiogjI2JdRHxoWAFJkiRVYTtEUh367kBFxJOBQ4HLhheOJElSd7ZDJNWlrw5URGwGfAF4\nPbBiqBFJkiR1YDtEUp36vQP1UeAbmfndYQYjSZJUge0QSbXpeRrziDgEeBKw5/DDkSRJas92iKS6\n9dSBiohHACcBB2Rm5YVuzjv8O8xfMH+9fbscsjMLF+/cS/GSOvolcPl6e1atGs56VJI0DvpthwCc\nA2zUtG8hsOuQYpvZLqe5foHVdQQijYVe70DtATwUWBpx/3Kjc4H9IuItwPzMycuqPvvE/dlu0baD\nRSqpi11pbgpsuuntrFz5kXrCkaTh66sdAnAQsN1oYpyBFpZbIxfS1ezVawfqfCZfrFkCXAWc0O5H\nS5IkaQhsh0iqXU8dqMxcBVzZuC8iVgG3ZeZVwwxMkiSpke0QSeNgoIV0S17tkSRJdbEdImmkep6F\nr1lmPmsYgUiSJPXKdoikURvGHShJkiRJmhXsQEmSJElSRXagJEmSJKmigZ+BqiSi2Aawbu2Rw4lj\nHMwZThzDOCdz5hw/eBzrjho4D4Cj1w6+6Ou4fMUxpEsTR6+7d6D0y5fex6l7DicWSZrOPsEbmDkr\nQY1DZTcOMUj18A6UJEmSJFVkB0qSJEmSKuqpAxURx0bEuqbtyu4pJUmSBmdbRFLd+nkG6nJgfx4Y\n/Hrf8MKRJEnqyraIpNr004G6LzNvGXokkiRJ1dgWkVSbfp6B2iki/hgRv4mIL0TEI4celSRJUnu2\nRSTVptcO1I+BVwMHAocBjwF+EBGbDjkuSZKkVmyLSKpVT0P4MvPchpeXR8TFwG+BlwGfbZfuvMPP\nZ/4W89fbt8vinVm4eJdeipfUwRVnXMEVZ67/HPWaFYOvrSVJ46TftgicC2zUtG9huamzX1I8dtZo\ndR2BSGNhoIV0M3NlRPwK2LHTcc8+8QC2W7TtIEVJ6mKXxbuwS9NFieVLV3Dqnv9RU0SSNPWqtkWK\nG1YzZSHdUdu13BrdCHyihlik+g20DlREbEbxg3XjcMKRJEmqzraIpFHrdR2oD0TEfhHx6Ih4GvBV\nYA1wxiBBXH7GFYMkH6pxiWVc4gC4POuOoHDFOJ2TMYllnM6JJI3CVLVFJg9RqyuPX45JHsPMR5pZ\ner0D9QjgdOBq4EzgFuCpmXnbIEFcccb4rH83LrGMSxwwnOpgGJqf76nTuHw/43ROJGlEpqQtMj4d\nqHHJY5j5SDNLr5NILJ6qQCRJkrqxLSKpbgM9AyVJkiRJs4kdKEmSJEmqaKBpzCvYCODWq27teNA9\nK+/hxqU3dc4phzCTQUTXQyrFMgKV4lg3+DmZUyGL1cCNHY5bN6Tz1e0rXr2i+zmp8BUPxaj+TgY9\nJ3+6+k8T/2xe/ESSZovy969zW6Ss7QYsqlseVSqpUcQxjHzuP5/WL5p1IofRMWmXecQrgC9OWQGS\nqnplZp5edxCSNGq2Raac9YtmnanuQD2EYuW663HJaqkOGwHbA+cOPkOVJE0/tkWmjPWLZq0p7UBJ\nkiRJ0kziJBKSJEmSVJEdKEmSJEmqyA6UJEmSJFVkB0qSJEmSKqq9AxURb46I6yLi7oj4cUQ8ecTl\nHxURF0fEHRFxc0R8NSIeO8oY2sR1ZESsi4gP1VT+wyLi8xFxa0TcFRGXRcSiGuKYGxHHl38jd0XE\nNRHxzhGUu29EnB0Rfyy/hxe2OOa4iLihjOu8iNhx1LFExLyI+LeI+EVE3Fkec1pEbDcVsUjSTDNo\nO6RKfVEhj4HbIhFxWFlXryy3iyLioF5jacqz1raINK5q7UBFxMuBDwLHArsDlwHnRsRWIwxjX+Aj\nwFOAA4ANgG9HxMYjjGE95Y/3oRTno47yFwA/BO6hmPr1CcA/AstrCOdo4HXAG4HHA0cAR0TEW6a4\n3E2BnwNvAiZNVRkRbwfeQvE97QWsovjb3XDEsWwCPAl4D8X/QwcDjwO+PgVxSNKMMqR2SMf6oqJh\ntEV+D7wdWATsAXwXODsidu4noLrbItI4q3Ua84j4MfCTzPz78nVQ/ACcnJnvrymmrYBlwH6ZeWEN\n5W8GXErRYXgX8LPM/IcRx3ACsHdmPmOU5baJ5RvATZn5hoZ9/wXclZl/O6IY1gEvzsyzG/bdAHwg\nM08sX28O3Ay8KjPPGmUsLY7ZE/gJ8OjM/MNUxSJJ092w2yFVfqMr5jOUtkhE3Ab8U2Z+tsd0tbdF\npHFW2x2oiNiA4grJdyb2ZdGbOx/Yu664gAUUV5Bur6n8jwLfyMzv1lQ+wAuASyLirHIowdKIeH1N\nsXwL2D8idgKIiN2ApwP/U1M8RMRjgG1Z/2/3DopOS51/uxMm/oZX1B2IJI2rMW6HwIBtkYiYExGH\nAPOBC/rIYhzaItLYmldj2VsBcymu2je6mWII0siVV55OAi7MzCtrKP8QiuFYe4667CY7UFx1+iDw\nXoohaidHxD2Z+flRBpKZp0bEI4H/i4j7KDr9R2fmmaOMo8m2FBVbq7/dbUcfzgMiYj5wAnB6Zt5Z\nZyySNObGrh0Cg7VFImIh8CNgI+Au4GWZeU2PeYxLW0QaW3V2oMbRqcDOFHc4RioiHkHxg3lAZq4Z\ndflN5gAXZ+a7yteXlT/KhwEj7UBFxNuAVwEvB66k+FH/cETcMOrO3LiLiHnAlyg6d2+qORxJUn8G\naYtcDewGbAH8JXBmRDwjM39WJfGYtUWksVVnB+pWYC2wTdP+bYCbRh1MRJwCPBfYNzNvHHX5FMMI\nHgosLa8+QXFlbL9ywoT5OboH1m4ErmradxXwkhGV3+gdwHsy80vl6ysiYnvgKEbcmWtwExAUf6uN\nVy63ASpVUsPW0Hl6JPAs7z5JUldj1Q6BwdsimXkfcG358mcRsRfFiJJDK2YxTm0RaWzV9gxUeWXj\nUmD/iX3l/6z7AxeNMpbyB+tFwJ9n5u9GWXaD84FdKe6w7FZulwBfAHYb8Q/WD5k8fOFxwG9HGMOE\nORQVXKN11Pu3ex1F5dr4t7s5xexJI/3bLcue6DztAOyfmXXMlihJ08o4tUPKsqeiLTKHogNU1Ti1\nRaSxVfcQvg8BSyLiUuBi4HCKaZmXjCqAiDgVWAy8EFgVERNXolZm5upRxZGZqyiGqDXGtgq4LTOb\n7wZNtROBH0bEUcBZFB2D1wNv6JhqanwNeGdE/AG4gmJ61sOBT01loRGxKbAjxZ0mgB3KCSxuz8zf\nUwxxeGdEXANcD/wL8AemYPrwTrFQ3C38MkVl93xgg4a/4dsdgiFJHQ3cDqlQX1TJY+C2SES8j2Li\npd8BDwJeCewH/GvVzzJmbRFpbNU6jTlARLyJYm2fbSjWUXhrZl4ywvLX0Xrdhtdk5udGFUcrEfFd\n4Od1TB0aEc+lmIxgR+A64IOZ+Zka4tiEYo2jl1L8jdwAnA78SzlUYarKfQbwPSb/bZyWma8tj3k3\nxbCIBRSzHL2514d1B42F4txc1/RelK//PDN/MOx4JGkmGbQdUqW+qJDHwG2RiPgU8CxgO2Al8Avg\nhEFn0quzLSKNq9o7UJIkSZI0XdT2HIkkSZIkTTd2oCRJkiSpIjtQkiRJklSRHShJkiRJqsgOlCRJ\nkiRVZAdKkiRJkiqyAyVJkiRJFdmBkiRJkqSK7EBJkiRJUkV2oCRJkiSpIjtQkiRJklSRHShJkiRJ\nqsgOlCRJkiRVZAdKkiRJkiqyAyVJkiRJFdmBkiRJkqSK7EBJkiRJUkV2oCRJkiSpIjtQkiRJklSR\nHShJkiRH0JQoAAAgAElEQVRJqsgOVEUR8eqIWBcRj6o7lkEM+jki4t1l+gfXFcMwRMQREXHlFJfx\ndxHx24jYYCrLkSQ9ICL2jIgfRsSdEbE2Ip5Yd0zTUa/1pHWeZhM7UNVlud0vIvaOiGMjYvOaYurH\npM8xFem7nJtBYxhIRDwIOAI4YYqLWgJsCPzdFJcjSQIiYh7wX8CWwP8D/gb4ba1BTZGpbIP0WU8u\nwTpPs4QdqOo+B2ycmb9r2Pc04BhgQT0h9aXV55gKnc7NqGJo53XAXODMqSwkM+8BTgP+YSrLkSTd\n78+ARwEfyMxPZebpmbmy7qCmyFS2QSrVkxFxZEScHBHfBOZjnadZwg5UFxGxCUAW7m1+u4aQBtLm\nc0yFtudmhDG082rg7BHFcBawfUQ8cwRlSdJst035346dpom6fZqbyjbIq+lST0bEAcCrgK8AewCb\nYZ2nWWJGdqAiYtfyGZvnN+xbVO67pOnYb0XEj8p/Tzzf84SIOD0ibgcuKN9b77mdiDgWeH+ZzfXl\ne2sb3n9YRHwmIm6KiNURcXlEvGaqYq9aZrvnjyLimRFxSUTcHRG/johDJ85Hm1C3jIglEbE8IlaU\n5W5U8dw0n8uJ8/5n7fIcMNbGtNsDTwTO73bsMGTmUuB24EWjKE+SZquI+CzwfYoh4v9V1ivfLYe5\ntazby3SV6ut+6/WG9F3rroh4VEScGhFXR8RdEXFrRJwVEY9uymtK2iBl2u2pVk++FLggM7+fmdtl\n5g3WeZot5tUdwBS5HFgB7Ad8s9y3L7AO2C0iNsvMOyMigL2Bj5XHTDyX8yXgV8BRPHCFp/m5nS8D\njwUOAf4euK3cf0tEbA38BFgLnAzcCvwF8OmIeFBmnjzs2Hsos9WzXLsD3wJuAN5F8XfxrjKPVs8q\nBcVVpmuBI4FFwBuAm8tz9pV256ZNDBP/bs7z9Q159htrs6eVxy2tcOywLAWePsLyJGk2+hjwB+Bo\n4MPATynqkH3K9yfV7VXrzgHr9V7qricDTwXOKD/L9sCbgO9FxM6Zubo8bqraIFC9nnwacFKL/dZ5\nmvkyc0ZuwDeAHzW8/i+KH897geeU+3an6Jg8r3x9bPn68y3yexXFj9GjGvb9Y/O+cv+nKH74FjTt\nP53iysz8KYi9UpltPsfZwJ+AbRr27VCWt7Ypv4lz9Imm/V8GlnU7N61iqJpnr7G2ObfHlWVv0rT/\nr4EPACc17NsHOK/898bAb4Dn9vG3+DHgzrr/n3Bzc3Ob6RvwjLI+eUnDvk51e9W6c9B6vVLd1Sof\nYK8y/lc27Z+qNkjLerLh/VdSPBu1juKC6WnAFg3vW+e5zfhtRg7hK10ALIqIjcvX+wD/A1xGcUcH\nHriz88OGdAl8fMCyX0LRCZobEQ+Z2IBvA1tQ3F0ZVuwXDlJmRMwB9ge+lpk3T+zPzGsprpa10uoc\nXQA8JCI26/LZ2umaZ5+xNnsIcF9m3jWxIyIeB9xNURE0DnF4HnBT+e97KK7w7Vr1AzVYDmzcajii\nJGkk2tXtnerOBTxQd/Zdr/dSd2Ux+dBEunlRLBlyLcXIlG5thyqfqUobZFI92RTjF4HjgTXAyzLz\nVbn+RB3WeZrxZuoQPiga3xsAe0fEH4CHlvsW8kAnZB/gysxc0ZT2un4LjYiHUvzoHkrrqTwT2LpL\nNlVi35ci9pUDlrk1xd2Va1q812rfhOYZ9JaX/90SuLNDuk665dlvrN08Hfg88M8Uwx4m7E05A1Fm\nrouID9Df/zONw0AlSfVYr26vWncOoV6vXHeVnY53UEzi8HDWrz+26FDGRPphtEGq2A24OjPvaxVG\nQ1nSjDSTO1CXAKspniX6PcVQsGsi4gLgjRGxIUUn5Cst0t49QLkTd/W+QHFbu5VfdMmjSuz78EDs\nwyizV2vb7B9kVqCpyLPZbcC8iNg0M1cBZOZnACLiJcCp5b/nAnsCb2lIuwENDx73YEvgrsYri5Kk\nkWuu26vWnXMrHjcMp1AMcz8R+DHFbIIJ/CfVJv4aRntgUj3Zwm7Az9u8Z52nGW/GdqAyc01EXEzR\nCfkdDzR8L6BYq+CVFNOd/mCQYlrsu4VinPPczPxuX5n2HvsgZS6j6Kzt2OK9nXrMq9FUXHkaRqxX\nl/99DMWEHQCUwxt2pxgqCcVQvfsy8/KGtI/MzN/3FPEDZV3VRzpJ0tSpVHeWQ/AGqdd7qbteCizJ\nzCMayp9P67WepqQNQpt6sskTaT903jpPM95MfgYKig7HU4Bnlv8mM2+j+HF4O8WPTz93FCZMXJm5\n/4ctM9dRTH7w0ojYpTlBRGxVMe/KsQ9SZpn2fODFEbFtQ5odgYMqxtrKpHMzqCHF+iOKO1p7Nu3f\nEViemRPPPO1Ew9CK8nm0e8t/PyIi/ioilpSvnxER74yIuRFxZVnZNloEXFQxPknSCFStOwet13us\nu9YyuW32Nh64C9Zoqtog7erJRk+i/R0o6zzNeDP2DlTpAorpTB/J+h2lH1CMDb4uM28YIP9LKX5k\n3hcRZ1I8UHk2xTTczwR+EhGfBK4EHkyx0NyzgCo/YL3GPkiZ7waeA1wUEf9B8XfxZoorT7tViLWV\nlucmMwcZHjlwrJl5XURcDhwALGl4648UD9zOK8d0PxlofAD2UOCL5b93oqg4XlG+fj7w3cxcGxFn\nlxUYABGxB8X38LUeP6ckaepVrTsHrdffTbW665vA30TEHWUZe1NMQHFrizynpA3SoZ4EICIeWeZ3\nWYv3rPM0K8z0DtRFFFdz7mT9/9EvoGgQDzJ8j8y8JCLeCRwGHEhx1egxmfm7iNgLOAY4GHgjxZji\nK4Aj2uU3SOyZuazfMjNzaUQcBPw7xfSlf6D4sX9cufWs3blh8kQRveY7jFg/A7wnIuZPjNHOzD9E\nxDHAJyPitxSV0JyI+DDFefxBZi4rj/1eRPwrD8zotG8ZA0y+IvdXwG8z8/u9flZJUl8qDyGvWncO\nUseW6avWXW8D7qO4QLcRxUy7BwDnNn+uKW6DTKonGywCLi9HxTSzztOsEJlOkqLWIuKrwM6Z2Vcn\napR6iTUiNqdY0+mIzPxsn+V9E3g5xfTmP8/MhRGxBbB3Zp5THrMhcD3wvsw8pZ9yJEkz17jWs63q\nyYh4D8VzT8+jeMbqHU1prPM0a8z0Z6BUUfN6DRGxE/Bc4Hv1RNTeoLFm5h0Ui+b+8wBhnE4xU9Jr\ngfPKGfwOnug8lV5D8dzUoOuKSZKmuelUzzbXk+WzU0dRDM97OvDJFsms8zRreAdKAETEDRRjna8F\ntqcYErABsCgzf1NfZJNNp1glSYLpX3eVz27dDlybmZ+uOx6pTnagBEBEfBr4c2BbimFpFwHvyMxJ\nD4nWbTrFKkkSWHdJM4kdKEmSJEmqaKxn4SsXNz2Q4qHE1fVGI9ViI4qhHue2mfFIGip/dyVJs1il\ndtdYd6AoKvEvdj1KmvleSTFxhTTV/N2VJM12Hdtd496Buh6KxRC2bnPA2cAL27x3EocOUPSGHd77\nJsXaqe2s6/DeIP6HYsKeVlotUl5Vt2GcnT7vvQOU2ynmTp8ViiWy+tHpe4Wp+6ybdnjv68CLWr6z\n5ZYrWb78M1D+vyCNwPVQzNHf7ne3im6/klW86gUDZjBEf3j3QwbO472H38HRJ24+UB4fyn8YOI5h\nufjupwycx71HvosNT/iXgfJYu3Z8JhReM+D3C8BPDoennDhYHl//xOBxDMm2lzxn4DyWH34CW554\n5EB5nLDnXw4cx7A0LxjZj07t36ruGkIcwzKMDkm3lmMVmz372bz/vPOgS7urlg5URLwZ+CeKBykv\nA96amT9tcehqKCrxR7TJa6MO78F2A0S5UYf3NgYe3uH9fhv33WwEPKzNe4N8ld06UJ0+7yAjfDrF\n3OmzQrHOYD86fa8wdZ+1U6Xa/q94gw02GUbhUi/u/93t9CvXzUYDpgdYNHifZWgWLNpg4DwetMUc\nFg6Yz4J8zMBxDMvcVU8cOI/YfHPmPmmwfPK+QS4gDtlWWw6ex4YLYKtFA2YySPtnuOYv2nngPOYs\neNDA+YzTQlu3DiGPzu3fav40hDiGZfBf2O4txyq22PL+/4c7trtGftkmIl4OfBA4FtidogN1brnG\ngCRJkiSNrTruex8OfDwzP5eZV1Osg3AXxYKkkiRJkjS2RtqBiogNgD2A70zsy2Ie9fOBvUcZiyRN\nJxGxZUR8MSJWRsTyiPhURHR6uK45/cciYl1EvG0q45QkaaYb9R2orShmDri5af/NFM9D9Wz3QSPq\ny+Djvvuza03l1vF56zrHdZRbz1+xpp3TgScA+wPPA/YDPl4lYUQcDDwF+OOURddkt1EVNI28YHG3\nZzBnn3l/dXDdIYyfHQ6pO4Kxs+khg04NMPPYcphslK3k8Zm6pk/1/AE9qZZS62uS1PF56+pA1fFZ\nB31YWDNdRDyeYnrx12XmJZl5EfBW4JCI6HjxKSIeDnyYYkLTfmdf6Vldv5Lj7AWLN647hLEz769e\nUncI4+fPFtcdwdjZdPHz6g5h7NiBmmyUreRRz8J3K8UUdds07d8GuKldorOZPHfa7vjHo5lmKfCz\n9fasWLGmnlA0bvYGlmdm4x/I+RRTaD6FYi78SSIigM8B78/Mq4qXkiRpECPtQGXmmoi4lGIIytlw\nfwW/P3Byu3QvZPCpGqXxt4jmu1ELFtzOsmXvrSccjZNtgWWNOzJzbUTcTufhz0cC92bmKVMZnCRJ\ns0kdQ/g+BLwhIv62HJbyMWATYEkNsUhSbSLi+HJih3bb2oh4bJ957wG8DXjNcKOWJGl2G/lCupl5\nVrnm03EUQ/d+DhyYmbeMOhZJqtm/A5/tcsy1FEOct27cGRFzgQfTfvjzPsBDgd83DN2bC3woIv5f\nZu7QqdBvMnno9G74bJMkaWa4DPhl0755F15YKe3IO1AAmXkqcGodZUvSuMjM24Dbuh0XET8CFkTE\n7g3PQe0PBPCTNsk+B5zXtO/b5f5unTaeDzy820GSJE1TuzF54okt9tmHt551Vte0tXSgJEnVZebV\nEXEu8MmIeCOwIfAR4IzMvP8OVERcDbw9M7+emcuB5Y35RMQa4KbM/PUIw5ckaUaZ9tOYS9Is8Qrg\naorZ974J/AD4u6ZjdgK26JBHTk1okiTNHtPiDtRJvAHYbsSlrh4g7SCntb9pq6/hXX2XuCOn9Z0W\nfjNA2v6t3vy4vtJtdEd/6aS6ZeYK4K+7HDO3y/sdn3uSJEndeQdKkiRJkioaeQcqIvaNiLMj4o/l\nNL0vHHUMkiRJktSPOu5AbUoxdfmbcDy+JEmSpGmkjnWgzgHOAYiGxUkkSZIkadz5DJQkSZIkVWQH\nSpIkSZIqmhbTmMO5wEZN+xaWmzRTLAV+tt6eFSv6m9ZekiRJU2OadKAOZPTrQEmjtqjcHrBgwe0s\nW/beesKRJEnSJA7hkyRJkqSKRn4HKiI2BXYEJmbg2yEidgNuz8zfjzoeSZIkSaqqjjtQe1I86HEp\nxTpQH6R4+OM9NcQiSdNCRGwZEV+MiJURsTwiPlVekGp3/LyI+LeI+EVE3FkuXn5aRDgeWpKkAdSx\nDtT/4tBBSerV6cA2wP7AhsAS4OPAX7c5fhPgSRQXp34BbAmcDHwd2GuKY5UkacaaJpNISNLsFRGP\np5hNZ4/M/Fm5763Af0fEP2XmTc1pMvOOMk1jPm8BfhIRj8jMP4wgdEmSZhzvBEnS+NsbWD7ReSqd\nTzEM+ik95LOgTLNiiLFJkjSrzOg7UMcM8FjVcRw7QMn3DZA2uh/Swo68a4Ay69L/edr4jv4+79qv\nHtN3mXMP7j8tbNZnursGKFMzyLbAssYdmbk2Im4v3+sqIuYDJwCnZ+adww9RkqTZYUZ3oCRpnEXE\n8cDbOxySwBOGUM484Etlfm+qkuabTF6+fDeKh6okSZruLgN+2bRv3oUXVko70g5URBwFHAw8Hrgb\nuAh4e2b+apRxSNKY+Hfgs12OuRa4Cdi6cWdEzAUeXL7XVkPn6ZHAs6refXo+8PAqB0qSNA3tVm6N\ntthnH9561lld0476DtS+wEeAS8qyjwe+HRFPyMy7RxyLJNUqM28Dbut2XET8CFgQEbs3PAe1P8WY\n3590SDfRedoB+PPMXD541JIkzW4j7UBl5nMbX0fEqynG9e8BVLtnJkmzTGZeHRHnAp+MiDdSTGP+\nEeCMxhn4IuJqirv6Xy87T1+mGHX3fGCDiNimPPT2zFwz2k8hSdLMUPczUBMzQt1ecxySNO5eAZxC\nMfveOuC/gL9vOmYnYIvy3w+n6DgB/Lz8b1D85v458IOpDFaSpJmqtg5URARwEnBhZl5ZVxySNB1k\n5graL5o7cczchn//Fpjb4XBJktSHOu9AnQrsDDy9+6HnMnk+qIXlJs0UFwM/XW/PihX31BOKJEmS\nWqqlAxURpwDPBfbNzBu7pzgQ2G6Ko5Lqtle5PWDBgmUsWzYd1/iSJEmamUbegSo7Ty8CnpGZvxt1\n+ZIkSZLUr1GvA3UqsBh4IbCqYUaolZm5epSxSJIkSVKv5oy4vMOAzYHvAzc0bC8bcRySJEmS1LNR\nrwM16g6bJEmSJA2NHRpJkiRJqsgOlCRJkiRVVOc6UD2IcuvNcRwz/FAqyb5TfpXj+kp3MMf2XSYs\nHyBtPbKPvweAuQfXMyX4Yevu7CvdPUtX89k9hxyMJEmS+jbSO1ARcVhEXBYRK8vtoog4aJQxSNJ0\nFhFvjojrIuLuiPhxRDy5y/HPjIhLI2J1RPwqIl41qlglSZqJRj2E7/fA24FFwB7Ad4GzI2LnEcch\nSdNORLwc+CBwLLA7cBlwbkRs1eb47YFvAt8BdgM+DHwqIp49inglSZqJRtqBysz/zsxzMvM3mXlN\nZr4T+BPwlFHGIUnT1OHAxzPzc5l5NcXSEHcBr21z/BuBazPziMz8v8z8KPBfZT6SJKkPtU0iERFz\nIuIQYD5wQV1xSNJ0EBEbUNy5/87EvsxM4Hxg7zbJnlq+3+jcDsdLkqQuRj6JREQsBH4EbERx5fRl\nmXnNqOOQpGlmK2AucHPT/puBx7VJs22b4zePiPmZec9wQ5Qkaear4w7U1RRj8fcCTgHOjIjda4hD\nkiRJknoy8jtQmXkfcG358mcRsRfFOP1D26c6h+KGVaOFwK5TEKFUj1+fsZRrzvzZevvWrVhTUzQa\nQ7cCa4FtmvZvA9zUJs1NbY6/o9vdp28y+Vd3N+BJlUKVJGm8XQb8smnfvAsvrJR2HNaBmkMxLKWD\ng4DtRhGLVJudFi9ip8WL1tt3z9Lb+eye760pIo2TzFwTEZcC+wNnA0RElK9PbpPsR8BfNO17Trm/\no+cDD+87WkmSxttu5dZoi3324a1nndU17ajXgXpfROwbEY+OiIURcTywH/CFUcYhSdPUh4A3RMTf\nRsTjgY8BmwBLACLi+Ig4reH4jwE7RMS/RcTjIuJNwF+W+UiSpD6M+g7U1sBpFLeTVgK/AA7MzO+N\nOA5JmnYy86xyzafjKIbi/ZziN/SW8pBtgUc2HH99RDwPOBF4G/AH4HWZ2TwznyRJqmikHajMfP0o\ny5OkmSYzTwVObfPea1rs+wHF9OeSJGkIalsHSpIkSZKmGztQkiRJklTROMzCN4VigLTZd8qPcVzf\naQ/mmL7T9q95suKZbMe+U36MV/Wd9rA5x/eVbuut7+u7TEmSJA2fd6AkSZIkqaJaO1ARcWRErIsI\np9SVJEmSNPZq60BFxJOBQykWApYkSZKksVdLByoiNqNYPPf1wIo6YpAkSZKkXtV1B+qjwDcy87s1\nlS9JkiRJPRv5LHwRcQjwJGDPUZctSZIkSYMYaQcqIh4BnAQckJlrRlm2JEmSJA1q1Heg9gAeCiyN\niIlFmuYC+0XEW4D5mdliAaZzmLxW0UJg16mLVBq5n9M8p8qKFV5n0Poi4s3APwHbUvzBvDUzf9rm\n2IOBN1Lc9Z8PXAG8OzO/PaJwJUmacUbdgTqfyb2eJcBVwAmtO08ABwHbTWVc0hh4Urk9YMGC5Sxb\n9m/1hKOxExEvBz5IMYPpxcDhwLkR8djMvLVFkv2AbwNHUUzY81rgGxGxV2Y6A6okSX0YaQcqM1cB\nVzbui4hVwG2ZedUoY5Gkaehw4OOZ+TmAiDgMeB5Fx+j9zQdn5uFNu46OiBcBL8AlJCRJ6kutC+mW\n2tx1kiRNiIgNKIZBf2diX3nX/nxg74p5BPAg4PapiFGSpNlg5LPwNcvMZ9UdgyRNA1tRPDN6c9P+\nm4HHVczjn4FNgbOGGJckSbNK7R0oSdLUi4hXAO8CXtjmeSlJklSBHShJmh5uBdYC2zTt3wa4qVPC\ncv29TwB/mZnfq1LYN5k89+luNE9zIknS9HQZ8MumffMuvLBSWjtQbUX3Q9o4jGMGKHfjPtNt2H+R\nx72h/7THvK//tOw3QNob+kv2jJf1XeJh//vbvtPyd0f0l+6epbDEWfgEmbkmIi4F9gfOhvufadof\nOLlduohYDHwKeHlmnlO1vOcDDx8oYkmSxtdu5dZoi3324a1ndR/lbgdKkqaPDwFLyo7UxDTmm1As\nB0FEHA88LDNfVb5+Rfne24CfRsTE3au7M/OO0YYuSdLMMNJZ+CLi2IhY17Rd2T2lJCkzz6JYRPc4\n4GfAE4EDM/OW8pBtgUc2JHkDxcQTH6W4bTuxnTSqmCVJmmnquAN1OcWQk4kxcvfVEIMkTUuZeSpw\napv3XtP0+s9HEpQkSbNIHR2o+xqulkqSJEnStFHHQro7RcQfI+I3EfGFiHhk9ySSJEmSVL9Rd6B+\nDLwaOBA4DHgM8IOI2HTEcUiSJElSz0Y6hC8zz214eXlEXAz8FngZ8NlRxiJJkiRJvap1GvPMXBkR\nvwJ27HzkOUxe0nEhsOvUBCbV4Zoz4Joz19u1Yt2KmoKRJElSK7V2oCJiM4rO0+c6H3kQsN0IIpJq\ntOPiYmuw4J6lLFuyZ00BSZIkqdmo14H6QETsFxGPjoinAV8F1gBnjDIOSZIkSerHqO9APQI4HXgI\ncAtwIfDUzLxtxHFIkiRJUs9GPYnE4u5HSZIkSdJ4qmMdKEmSJEmaluxASdI0EhFvjojrIuLuiPhx\nRDy5YrqnR8SaiFg61TFKkjST1ToL39TLvlN+muP6Tvs6ju07LawecTrgmPf3n5Y1A6T9zgBp+/S/\n7xl9mQAfv7u/dFvfM9w4NK1FxMuBDwKHAhcDhwPnRsRjM/PWDum2AE4Dzge2GUWskiTNVN6BkqTp\n43Dg45n5ucy8GjgMuAt4bZd0HwO+CPx4iuOTJGnGG3kHKiIeFhGfj4hbI+KuiLgsIhaNOg5Jmk4i\nYgNgDxpu3WZmUtxV2rtDutcAjwFquv0qSdLMMtIhfBGxAPghRQPgQOBWYCdg+SjjkKRpaCtgLnBz\n0/6bgce1ShAROwHvA/bJzHURMbURSpI0C4z6Gagjgd9l5usb9v12xDFI0owXEXMohu0dm5m/mdhd\nNf03gY2a9u0GPGk44UmSVKvLgF827Zt34YWV0o66A/UC4JyIOAt4BvBH4NTM/NSI45Ck6eZWYC2T\nJ4HYBripxfEPAvYEnhQRHy33zQEiIu4FnpOZ329X2POBhw8asSRJY2q3cmu0xT778NazzuqadtTP\nQO0AvBH4P+A5wH8AJ0fE34w4DkmaVjJzDXApsP/EvijG5O0PXNQiyR3AQoqbRhP1xMeAq8t//2SK\nQ5YkaUYa9R2oOcDFmfmu8vVlEbGQYiapz7dPdg6TB5MsBHadghClupwFfGm9PStWrKwnFI2rDwFL\nIuJSHpjGfBNgCUBEHA88LDNfVU4wcWVj4ohYBqzOzKtGGrUkSTPIqDtQNwLNFfdVwEs6JzsI2G5q\nIpLGxsvK7QELFvycZcueXk84GjuZeVZEbAUcRzF07+fAgZl5S3nItsAj64pPkqTZYNRD+H7I5Nmi\nHsdAE0k0P/419eob9zL6z1r4eQ1l1vVZ6yi3+1hbaUJmnpqZ22fmxpm5d2Ze0vDeazLzWR3Svicz\nR7JsRB2/GuPuG2f0uaD2DHbfl75Sdwjj5zdn1B3B2Fl1xn/XHcLY+VndAYyhy0ZY1qg7UCcCT42I\noyLizyLiFcDrgVP6z/LyIYVWXX0dqNF/1sIo/yQn1PVZ6yj3S90PkaaZOn41xt03zlhddwhj574v\nfbXuEMbPtWfWHcHYWXXm/9QdwtixAzXZKC+Bj7QDVV4pPRhYTPE5jwb+PjP9tZAkSZI09kb9DBSZ\n+T+AlxIkSZIkTTujHsInSZIkSdPWyO9A9WgjgB13XMvGG69tecD11yfbb9/6Pci+C76jwxTp911/\nPXdsv33b93elXTyD6fxZB3Fvl3LXsf327Y6Zbp+1rnLbPxFy/fV3sP32rd+fN+/XLFsGTJ7HX5oq\nGwGs2XFH7t14474zWXf99dzb4XeyiqX9Fz90y5ZuMXAeq1Zeya+W7jhQHgvy9oHjGJYnrL5i4Dx+\nfcef2OmywfJZuy4GjmNY7pu/2cB5XD9nBdvPXzpYJruOvv5s5yFLfzNwHneuWMVjBsznd7uOz9I3\nw3gact3117N6wN/Y8fkrGY5111/PmgHPyW3z7u8adWx3RbFUyHgqJ5n4Yt1xSGPglZl5et1BaObz\nd1eSpM7trnHvQD0EOBC4nuF02KXpZiNge+DczLyt5lg0C/i7K0maxSq1u8a6AyVJkiRJ48RJJCRJ\nkiSpIjtQkiRJklSRHShJkiRJqsgOlCRJkiRVNG07UBHx5oi4LiLujogfR8STp7i8oyLi4oi4IyJu\njoivRsRjp7LMFjEcGRHrIuJDIyjrYRHx+Yi4NSLuiojLImLRFJc5NyKOL7/XuyLimoh45xSUs29E\nnB0RfyzP5wtbHHNcRNxQxnFeRAy0eEunMiNiXkT8W0T8IiLuLI85LSK2G6RMqQ6j/m0eZ+NQb4y7\nUdZr466OenecjapNMM7qaK+Mu3FpT03LDlREvBz4IHAssDvFKqXnRsRWU1jsvsBHgKcABwAbAN+O\niO5r+mcAACAASURBVJEs9Vg2Qg6l04qswytrAfBD4B6K6YyfAPwjsHyKiz4aeB3wRuDxwBHAERHx\nliGXs+n/b+/Ow+Wo6nWPf19CmI1BgUQQQQRBLxAIiEYCqFFBjAp6roh6xQGVQeGgHhAFQvAqKkIU\nMVePHI0D6I1yUEQkiuiFMErCKODEIFMShoRIEiDD7/6xakPR6b137R6qqvd+P89TT7pX16r1q07v\nWmtVrVoF3AQcRZOnLUs6AfgE6fveC1hG+n2t16UyNwJ2A6aTfs8HAzsCv2yjPLPSVXRsrrNK6426\nK7Neq7sK6906K6tNUGdVtFfqrhbtqZ6cxlzStcB1EXFs9l7AfcDZEfHVkmLYDFgE7BsRc7tc1ibA\nPNJB5GTgxoj4VBfL+zIwKSL261YZ/ZT7K2BBRHw0l/ZzYHlEfKBLZa4BDoqIi3JpDwJnRMSM7P0Y\nYCFwWETM7kaZTdbZE7gO2CYi7m+3TLMy1OHYXGdl1ht1V3a9VndV1bt1VkWboM6qaK/UXZXtqZ67\nAiVpNLAH8Pu+tEi9wMuASSWGMpbU832shLK+BfwqIi4voSyAtwE3SJqdDTuZL+nwEsr9DTBF0g4A\nkiYAewOXlFA2WZkvBcbz3N/XUtIfXxW/ryUllmnWshodm+uszHqj7squ1+quqnq3zipvE9RZjdor\nddeV9tS6ndxYSTYDRpF62HkLSZfpui47q/p1YG5E3N7lst5Duhy5ZzfLabAd6azgmcAXSZeFz5b0\nVET8qFuFRsRMSVsDf5G0itTB/3xE/LRbZTYxnvSH1uz3Nb6MACStD3wZOD8iniijTLMOqPzYXGdl\n1ht1V1G9VneV1Lt1VpM2QZ1V3l6pu262p3qxA1UHM4FXks6EdI2kF5Mq3DdGxMpultVgHeD6iDg5\ne3+zpJ2BI4CuHcglHQMcBhwC3E6qYL8h6cGRUoFIWhf4GemgeFTF4ZhZ55RSb9RdhfVa3VVS79aZ\n2wTWjm63p3puCB/wCLAaGNeQPg5Y0O3CJZ0DHAi8LiIe6nJxewCbA/MlrZS0EtgPOFbS09kZzW54\nCLijIe0O4CVdKq/P54AvRMTPIuLPEXEeMAM4scvl5i0ARAW/r9wf+9bAm331yXpMpcfmOiu53qi7\nquq1uquq3q2zOrQJ6qyy9krdldGe6rkOVHbGah4wpS8tO+BOAa7uZtlZJfgO4PUR8c9ulpW5DNiF\ndNZlQrbcAPwYmBDdmwHkKtYecrMjcG+XyuuzDqkBlreGEn+nEXE36cCT/32NIc2i1bXfV+6PfTtg\nSkSM5JmXrAdVeWyuswrqjbqrql6ru6rq3TqrvE1QZ1W1V+qurPZUrw7hOwuYJWkecD1wHGnqwlnd\nKlDSTOBQ4O3AMkl9Pf7HI+LJbpQZEctIl63zcSwDHo2IxjNVnTQDuErSicBs0h/j4cBHB8zVvl8A\nJ0m6H/gzMJH0f3tuJwuRtDGwPenMDcB22c2pj0XEfaThJSdJ+jtwD/AF4H7amAZzoDJJZx4vIDUo\npgKjc7+vxzzMxXpI6cfmOqui3qi7Cuu1uquq3q2zUtoEdVZFe6XuatOeioieXEjjGe8BVgDXAHt2\nubw1pDMhjcsHSt7vy4GzSijnQOAWYDnpwPXhEsrcCDgDuIv0LIO/kebyX7fD5ezXz//n93LrnAo8\nmO3/HGD7bpUJbNPks773+5b5+/Lipd2l7GNznZe61Bt1X8qq1+q+VFHv1nkpq01Q56WK9krdl7q0\np3ryOVBmZmZmZmZV8DhSMzMzMzOzgtyBMjMzMzMzK8gdKDMzMzMzs4LcgTIzMzMzMyvIHSgzMzMz\nM7OC3IEyMzMzMzMryB0oMzMzMzOzgtyBMjMzMzMzK8gdKDMzMzMzs4LcgTIzMzMzMyvIHSgzMzMz\nM7OC3IEyMzMzMzMryB0oMzMzMzOzgtyBMjMzMzMzK8gdKDMzMzMzs4LcgTIzMzMzMyvIHSgzMzMz\nM7OC3IEyMzMzMzMryB0oMzMzMzOzgtyBKpGkUyWtqTiGD0paI+klLeY/Ncv/gqpi6ARJx0u6vQvb\n/bikeyWN7vS2zcyGq8Z6oQ71xGB6Ica66HSd67rWquYOVLkiW3o5hkL5JU2SNE3SmC7E0BZJzwOO\nB77chc3PAtYDPt6FbZuZDVeN9UId6sta12WdNMh+trvtbtS5s3BdaxVyB2rk+SGwYUT8s8vlvBY4\nBRhbYQz9+QgwCvhppzccEU8BPwA+1eltm5lZ6epcl3XSQPvZro7Xua5rrWojugMlaaOqYyhbJE+X\nUJRqEEN/Pghc1MUYZgPbSnpdl7ZvZmblqHNd1kn97mcHfJDu1Lmua60yI6YDlbt35xWSzpf0GHBl\n9tmWkr4naYGkJyXdJulDTbbxEkkzJd0pabmkRyTNlrRNk3UnS/qTpBWS/ibpYwXj3CWLc2oubWKW\ndkPDur+RdE3u/aD70d+YbUmvk3RDPt5B7tnaVNIsSYslLcnK3SDb1jTgq9l692Tlre5vbHvu/+Zl\n/W2zzVjzebcFdgUuG2zdVkXEfOAx4B3dKsPMrBcMpd4suL3ds7rvcUn/knSZpFc3WW9LSf8l6YGs\nPrwri2PdonENtS4bSoxDrff6+S4GrQs7sZ+573PQdlKTGLelS3Wu61qr0rpVB1CivnHKPwP+CpwI\nSNIWwHXAauBs4BHgLcB/SXpeRJyd28argNcAPwHuB7YFjgL+IOmVEfEkaaM7A3OARaRL4qOBU7P3\ng7kNWALsC1ycpe0DrAEmSNokIp6QJGAS8O2szKL7sdaYbUm7A78BHgROJv0uTs620Wx8t0hnfu4C\nPgtMBD4KLCR9r/8NvBx4D3As8GiW7+F+Yuh73bjNw3PbbDXWRq/N1ptfYN12zAf27nIZZmZ1V6je\nLELSK4ErgMdJ99OsIt0D80dJ+0bEn7L1XgT8CRgDfAf4C7AV8G/ARsDSgnFdwNDqssIxMoR6r5/v\nomhd2PZ+DrGd1Kjbda7rWqtGRIyIBZhG6oT8qCH9XNJBZWxD+vmkMxvr59LWb7LdvbLtvi+XdiGw\nDNgql7YjsBJYXSDWXwHX5N7/nNTxexp4c5a2e1buW4eyH8BhpIPgS3LrXAT8CxiXS9suK291w/b6\nvsf/bEi/AFiUe//pxnJynz0nhqLbHGqs/Xy3p2Vlb9SQ/n7gDODrubTJwO+y1xsC/wAOLPh7+zbw\nRNW/ey9evHipchlCvdlYLzSrqy4EVgDb5NLGkzorf8il/SCrb3fvQFyF67Ihxli43usn/kJ1YYf2\ns3A7qUnepnVuwzrvyerMk0jD/e4cwu/Lda2XSpYRM4QvE6SzUXnvJHVYRkl6Yd8C/BZ4PumMUMqc\nbloEQNK6SlN530W6YjQxS18HeDNwYUQ8kMv7F9JVqSKuBCZK2jB7Pxm4BLiZdDUKnr0qNXeo+5GX\nxTsF+EVELMzFexfp7FYzzb7HK4EXStqk4D4OeZstxtrohcCqiFjelyBpR1KF999AfkjCW4EF2eun\nSGfldilYzmJgw6JDMczMhqMi9WYR2fH/TaS69d7c9heQGvKTJW2Sjc54B+memxu7HVcrMebDoIW6\ndCh1YYf2s6X2RWatOrdhXz5O6rwdFRH/G3gD6QpXUa5rrRIjaQhfn7v7XkjanDTjzMdoPhVmAFvk\n1t8A+BzpDMlWPHvTZZAOIgCbk65W/L3J9v5Cuuw9mCtJw/4mSbo/2+aVwM48twN1e0Q8PtT9aLDF\nAPE2S+vTOOvQ4uzfTYEnBsg3kMG22Wqsg9kb+BHwH6RhCn0mkc0aFBFrJJ1B8b+Z/G/DzGxEKlhv\nFrE5afjdX5t8dgfpnu6tSSe6xgB/LimuVmK8I5feSl1auC5sdz/bbF8Mtu0XAWcB+0dE331bD5Ou\ndhXeTC4Os9KMxA7UitzrvitwPyZd8m/mltzrc0iX7GcA15IuyQfwf+nshBw3AE+S7oO6j3Q5/++S\nrgSOlLQe6arUf7e4H52wup/0dmby6cY2Gz0KrCtp44hYBhAR3wOQ9E5gZvZ6FLAn8Ilc3tFkE48U\nsCmwPH/2z8xsBCqr3uzVuLpd77W7n+22L9aqc3M+AvwrIubm0vYGpheIq4/rWqvESOxA5T1MGkM8\nKiIuL7D+u4BZEXF8X4Kk9XnucxMeJnXSdmiSf6ciQUXESknXkzpQ/+TZRvuVwPrA+4BxpBtVW9mP\nvEWkztr2TT5rtg9FdeNsUCdivTP796WkCTsAyIYj7E4aKglpqN6qiLgtl3friLivYDkv5blnGc3M\nRqIi9WYRDwPLSfcTN3oFaUj7faT7j5eSRmx0Iq6h1GVFY2zXUOrCdveznfYF9FPnZl4LXJWLawNg\nQj6tANe1VomRdg/Uc2SXjC8A3iXpfzR+LmmzhqTVrP2dHUN6QFx+m3OAgyS9OLetV5DujSrqSuDV\nwOuy10TEo6SD0Qmkg11f+lD34xlZ3suyeMfn8mwPHDCEeBv1nWnq2EP5OhTrNaQze3s2pG8PLM7G\nqkOqhJ4ZCpHdj/Z09voNko6T9GtJR0tqNrvQRODqgjGZmQ1Xg9abRWTH/98C72iYXnsccChwZUQ8\nEREB/AJ4m6SB7s0pGlfhuqxojINtp2A5RevCtvaznfZFpr86F9IJ4fxwx71IE0gszba9k6TTJL1N\n0gWSJjTZhutaq8RIvwIFaerQ1wHXSfoucDvwAmAP0s2M+YPDxcD/krQ0W28S6UbOxhsep5EOYnMl\nzSQN/foE6ezLrgXjuhL4PGm8dH7Y2BWkcch3R8SDLe5Ho1NJnburJf0f0u/i6CzeZgesIuaRDppf\nkvRT0oxIF0XEioGzDaqtWCPibkm3AW8EZuU+eoB0g+y6EbGKNPVr/qbUjwHnZa8fIt0Y+9eI+Jak\nc/NlSNqD9N3/Ysh7Z2Y2vBStN4s4iXTsviqrW1eTjs3rAcfn1vscaTKHKyT9J+kKxZakacz3zhro\nReMaal1WNMZ2nUqxurAT+9ly+2KAOhdSe2ZzgOzWhC/w7PM5NyJNvLFPRCyTdFJW7jNc11qVRnwH\nKiIWSdqL9Lymg4EjSWN2/8zaB7tjSM90eC+pcT2XdFCYQ+7yd0TcKunNpJsjp5NuiDyFdAAv2oG6\nmnTgfYI0+16fK0kH4yvyKw9xP2jIO1/SAcDXSFOO3k86OO9I86EIg4qIG7ID3hHA/qQzYC9l7Rtm\nh7rdTsT6PWC6pPX7xk1HxP2STgG+K+le0hSx60j6Bul7vCIiFmXr3iHpGLIOVZOx1/8TuDci/tj6\nnpqZDQuF6s0iIuJ2SfsAp5Ma9euQ7ut5b0TckFvvQaUH134hK3cM6STZJaQhdoXjGmpdVjTGdg2h\nLmx7P9tpX2TWqnMzXwPOlfTZLJbn8ewJ43cCN2WdpxcCSyJiZcN2XddaZZSudputTdKFwCsjoqVO\nVJmGEqukMaRnOh0fEd9vsbz5wKsiYnVD+nrAPcCXIuKcVrZtZmbWijrW20Xq3Gza9gXAyyJioaTj\ngDUR8Q1J7yc9APi6iPhdtr7rWqvUiL4Hyp7V+AwFSTsABwJ/qCai/rUbazZ84wzStOWtlP9C4C+N\nnafMh0j3SjU+28PMzKxjeqXeblbnZvcP/1dutc8A38891+qnwLaS3gJski35+7Zc11qlfAXKAJD0\nIGl88l2kMz1HkO7dmhgR/6gusrX1UqxmZmbd0Mt1oaTjSRNLXEe6vWFRRHyl2qjMinMHygDIzgS9\nHhgPPEW6B+tzEXHzgBkr0EuxmpmZdYPrQrPquANlZmZmZmZWkO+BMjMzMzMzK6jW05hnN+vvT5pp\n5clqozGrxAakse1zsgcpm3WVj7tmZjaCFWp31boDRarEzxt0LbPh732khwqadZuPu2ZmNtIN2O6q\newfqnvTPOcAO/awyjfSs2mYeaqPoOwb47ALgXQN83vhc1aF4eoDPLiHNUFq2gcot+lzgZuYP8Nml\nwAFtbLtVVZTbf5mbbvoEixefD8/8LZh13T0A7wa2aGMjFwNT2wxkTZv5O2l8B7ZxHqlGbkc7tUun\nbTD4KoP6MfD+NrexXgfi6JQTPtWBZ+X+4jg4aEZ72zirPrN7/4Tvtr2Nlp87klOrud07oBOtlY07\nEUiH7NGBbZwJfLrNbTzypjfx77/7HQzS7qqkAyXpaNKc/+OBm4FPRsSfmqyaDR/Zgf4b6mMG+axV\nywb4bEPgJQN8vqKNcgcaMbMBabbPsg1U7svb2O5AHdwNgBe1se1WVVFu/2WOHr2076WHUllZnoTU\nedqqjY1s2GZ+gGYPWqvKQEf8ojYijQtpR50OBBt2YBud+E460ZHrmK0ntr+NDcd2YDtV1J/NvaID\n23heB7ZzZwfiqJNOtFbaaSV32k4d2MYmHdjOgk037Xs54OG29EkkJB1C6iROA3YndaDmSNqs7FjM\nzMzMzMyGoopZ+I4DvhMRP4yIO0kPflsOfLiCWMzMzMzMzAortQMlaTRpmOPv+9IiPYjqMmBSmbGY\nmfUSSZtKOk/S45IWSzpXUuEh7JK+LWmNpGO6GaeZmdlwV/YVqM2AUcDChvSFtHyP7kHtRdSSPSso\nE2CXEVTuzhWUWVW5Ve2r9ZjzSbcBTAHeCuwLFLpTXNLBwKuBB7oWXYN2ppcZrl5TdQA15DOnTUx8\nT9UR1E4VU0rVnVsOa9u/xLLqPgtfZhpr3+p2EHBwtpStqg7UhBFU7kjqLPaVeStw23M+WbJkVenR\nWP1I2olUN+wRETdmaZ8Efi3pMxGxYIC8WwHfyPJfUka8ALuVVVAPcWdhbf5Omph4aNUR1M5bqg6g\nhqpqJdVZmR3tsjtQj5AmVhrXkD4O6LcBkKYp9/lMG+52ofGQOHbsUhYtanM6WxsOJgGL+zpPmcuA\nIF1Z+mWzTJIE/BD4akTckd6amZlZO0odwhcRK4F5pCEowDMV/BTg6jJjMTPrIeOBRfmEiFgNPMbA\nw58/CzwdEed0MTYzM7MRpYohfGcBsyTNA64nzcq3ETCrgljMzCoj6XTghAFWCVp8/ImkPYBjSI+L\nGLKLWfs5P7vioXlmZjY8XArMaUh7au7cQnlL70BFxOzsmU+nkYbu3QTsHxEPlx2LmVnFvgZ8f5B1\n7iINcd4inyhpFPAC+h/+PBnYHLgvN3RvFHCWpH+PiO0GKnQq7T8I18zMrK4OYO37phZMnszU2bMH\nzVvJJBIRMROYWUXZZmZ1ERGPAo8Otp6ka4CxknbP3Qc1BRBwXT/Zfgj8riHtt1n6YJ02MzMz60eP\nzMJnZjZyRcSdkuYA35V0JLAe8E3gJ/kZ+CTdCZwQEb+MiMXA4vx2JK0EFkTE30oM38zMbFgp+zlQ\nZmbWmvcCd5Jm37sYuAL4eMM6OwDPH2Ab0Z3QzMzMRo5hfgXq0qoDKNHoNvK28zPYpuWcpzC95byn\nMa3lvGa9KCKWAO8fZJ1Rg3w+4H1PZmZmNrjSr0BJ2kfSRZIekLRG0tvLjsHMzMzMzKwVVQzh25g0\n895ReDiJmZmZmZn1kCqmMb+UbGydcnPrmpmZmZmZ1Z0nkTAzMzMzMyvIHSgzMzMzM7OC3IEyMzMz\nMzMrqEemMZ8GjGlIOwg4uIJYzLrlVuC256QsWbKqmlDMzMzMrKke6UBNB3atOgizLtslW541duxS\nFi2aUU04ZmZmZraW0jtQkjYGtgf6ZuDbTtIE4LGIuK/seMzMzMzMzIqq4h6oPYEbgXmk50CdCcwn\nXWYyM7MmJG0q6TxJj0taLOnc7IRUf+uvK+krkm6R9ET28PIfSHpRmXGbmZkNN1U8B+r/4ckrzMyG\n6nxgHDAFWA+YBXwHeH8/628E7EY6OXULsClwNvBLYK8ux2pmZjZs9cg9UGZmI5eknYD9gT0i4sYs\n7ZPAryV9JiIWNOaJiKVZnvx2PgFcJ+nFEXF/CaGbmZkNO74SZGZWf5OAxX2dp8xlpGHQrx7CdsZm\neZZ0MDYzM7MRZZhfgXpZReU+VkGZK9rIG23k/UnLOb/AKS3nXX15a7fMjXrDtJbLNKvQeGBRPiEi\nVkt6LPtsUJLWB74MnB8RT3Q+RDMzs5HBV6DMzCoi6XRJawZYVkt6eQfKWRf4GelsyVFtB25mZjaC\nlXoFStKJpKff7kS6ZHI1cEJE/LXMOMzMauJrwPcHWecuYAGwRT5R0ijgBdln/cp1nrYG3lD06tPF\nwIYNabuSZqUwMzPrdZcCcxrSnpo7t1Desofw7QN8E7ghK/t04LeSXhER7YxBMzPrORHxKPDoYOtJ\nugYYK2n33H1QU0jP07tugHx9naftgNdHxOKisU0Ftiq6spmZWY85IFvyFkyezNTZswfNW2oHKiIO\nzL+X9EHSuP49gGJdPjOzESYi7pQ0B/iupCNJ05h/E/hJfgY+SXeSrur/Mus8XUC6aDQVGC1pXLbq\nYxGxsty9MDMzGx6qnkSib0aoKmZdMDPrJe8FziHNvrcG+DlwbMM6OwDPz15vReo4AdyU/SvSMff1\nwBXdDNbMzGy4qqwDJUnA14G5EXF7VXGYmfWCiFhC/w/N7VtnVO71vcCoAVY3MzOzFlR5BWom8Epg\n7wpjMDMzMzMzK6ySDpSkc4ADgX0i4qHBc0wDxjSkHUSa0M9suLgVuO05KUuWrKomFDMzMzNrqvQO\nVNZ5egewX0T8s1iu6aQJdM2Gs12y5Vljxy5l0aIZ1YRjZmZmZmsp+zlQM4FDgbcDy3IzQj0eEU+W\nGYuZmZmZmdlQrVNyeUeQxuL9EXgwt7y75DjMzMzMzMyGrOznQJXdYTMzMzMzM+sYd2jMzMzMzMwK\ncgfKzMzMzMysoCqfA1WCf1QdQAtGt5ivqv/K1p/TGaj1Ut9wSsultmra6tbnOZk+asOW85qZmZlZ\nffgKlJmZmZmZWUGldqAkHSHpZkmPZ8vVkg4oMwYzs14m6WhJd0taIelaSa8aZP3XSZon6UlJf5V0\nWFmxmpmZDUdlX4G6DzgBmAjsAVwOXCTplSXHYWbWcyQdApwJTAN2B24G5kjarJ/1twUuBn4PTAC+\nAZwr6U1lxGtmZjYcldqBiohfR8SlEfGPiPh7RJwE/At4dZlxmJn1qOOA70TEDyPiTtKz9ZYDH+5n\n/SOBuyLi+Ij4S0R8C/h5th0zMzNrQWX3QElaR9J7gPWBK6uKw8ysF0gaTbpy//u+tIgI4DJgUj/Z\nXpN9njdngPXNzMxsEKVP3SZpZ+AaYAPSmdN3R8Tfy47DzKzHbEaa9nJhQ/pCYMd+8ozvZ/0xktaP\niKc6G6KZmdnwV8UVqDtJY/H3As4Bfipp9wriMDMzMzMzG5LSr0BFxCrgruztjZL2Io3T/1j/uaYB\nYxrSDgIO7kKEZlW5FbjtOSlLlqyqJhSro0eA1cC4hvRxwIJ+8izoZ/2lg119uhhofHrZrsBuhUI1\nMzOrt0tJY9rznpo7t1DeOjxIdx0GfRrrdFLVbTac7ZItzxo7dimLFs2oJhyrlYhYKWkeMAW4CECS\nsvdn95PtGuAtDWlvztIHNBXYquVozczM6u2AbMlbMHkyU2fPHjRv2c+B+pKkfSRtI2lnSacD+wI/\nLjMOM7MedRbwUUkfkLQT8G1gI2AWgKTTJf0gt/63ge0kfUXSjpKOAv4t246ZmZm1oOwrUFsAPwBe\nBDwO3ALsHxF/KDkOM7OeExGzs2c+nUYaincT6Rj6cLbKeGDr3Pr3SHorMAM4Brgf+EhENM7MZ2Zm\nZgWV2oGKiMPLLM/MbLiJiJnAzH4++1CTtCtI05+bmZlZB1T2HCgzMzMzM7Ne4w6UmZmZmZlZQXWY\nhW8Y2qCNvKM7FkU5VlZT7OxpLWU75d2DTPg4gOmjTmk5r5mZmZkND74CZWZmZmZmVlClHShJn5W0\nRpKn1DUzMzMzs9qrrAMl6VXAx4Cbq4rBzMzMzMxsKCrpQEnahPTw3MOBJVXEYGZmZmZmNlRVXYH6\nFvCriLi8ovLNzMzMzMyGrPRZ+CS9B9gN2LPsss3MzMzMzNpRagdK0ouBrwNvjIghzH89DRjTkHYQ\ncHDHYjOr3q3Abc9JWbJkVTWhmJmZmVlTZV+B2gPYHJgvSVnaKGBfSZ8A1o+IWDvbdGDXsmI0q8gu\n2fKssWOXsmjRjGrCsVqSdDTwGWA8aRKeT0bEn/pZ92DgSNJV//WBPwOnRsRvSwrXzMxs2Cn7HqjL\nSC3E3YAJ2XIDaUKJCc07T2ZmBiDpEOBM0mX53UkdqDmSNusny77Ab4G3ABOBPwC/kjShhHDNzMyG\npVKvQEXEMuD2fJqkZcCjEXFHmbGYmfWg44DvRMQPASQdAbwV+DDw1caVI+K4hqTPS3oH8Db8CAkz\nM7OWVPog3YyvOpmZDULSaNIw6N/3pWVX7S8DJhXchoDnAY91I0YzM7ORoPRZ+BpFxBuqjsHMrAds\nRrpndGFD+kJgx4Lb+A9gY2B2B+MyMzMbUSrvQJmZWfdJei9wMvD2iHhksPUvBjZsSNuVdAOrmZlZ\nr7sUmNOQ9tTcuYXyugNlZtYbHgFWA+Ma0scBCwbKmD1/7z+Bf4uIPxQpbCqwVQtBmpmZ9YIDsiVv\nweTJTJ09+CANd6C64smK8rZqdBt5p7aR98LWs777tJayncYprZeJBl/FrEsiYqWkecAU4CJ45p6m\nKcDZ/eWTdChwLnBIRFxaRqxmZmbDmTtQZma94yxgVtaRup40K99GwCwASacDW0bEYdn792afHQP8\nSVLf1asVEbG03NDNzMyGh1Jn4ZM0TdKahuX2wXOamVlEzCY9RPc04EbSbUn7R8TD2Srjga1zWT5K\nmnjiW8CDueXrZcVsZmY23FRxBeo20pCTvvFQqyqIwcysJ0XETGBmP599qOH960sJyszMbASpogO1\nKne21MzMzMzMrGdU8SDdHSQ9IOkfkn4saevBs5iZmZmZmVWv7A7UtcAHgf2BI4CXAldI2rjkOMzM\nzMzMzIas1CF8EZF/XtVtkq4H7gXeDXy//5zTgDENaQcBB3c4QrMq3Uq6RfBZS5b4FkEzMzOz1wqq\nOwAACuVJREFUOql0GvOIeFzSX4HtB15zOmmyKbPhbJdsedbYsUtZtGhGNeGYmZmZ2VqquAfqGZI2\nIXWeHqoyDjMzMzMzsyLKfg7UGZL2lbSNpNcCFwIrgZ+UGYeZmZmZmVkryh7C92LgfOCFwMPAXOA1\nEfFoyXGYmZmZmZkNWdmTSBxaZnlmZmZmZmadVOk9UGZmZmZmZr3EHSgzsx4i6WhJd0taIelaSa8q\nmG9vSSslze92jGZmZsNZpdOY19umbeQd3UbeFW3kbdXKNvJe2rEoyqGWc/7H6tZLfcGo6S3lW8IW\nfKX1Ym2YkXQIcCbwMeB64DhgjqSXR8QjA+R7PvAD4DJgXBmxmpmZDVelX4GStKWkH0l6RNJySTdL\nmlh2HGZmPeg44DsR8cOIuBM4AlgOfHiQfN8GzgOu7XJ8ZmZmw17Z05iPBa4CngL2B14BfBpYXGYc\nZma9RtJoYA/g931pERGkq0qTBsj3IeClpCeSm5mZWZvKHsL3WeCfEXF4Lu3ekmMwM+tFmwGjgIUN\n6QuBHZtlkLQD8CVgckSskVofwmpmZmZJ2R2otwGXSpoN7Ac8AMyMiHNLjsPMbFiTtA5p2N60iPhH\nX3LR/BcDGzak7Qrs1pnwzMzMKnUpMKch7am5cwvlLbsDtR1wJOkm6C8CewFnS3oqIn5UcixmZr3k\nEWA1a08CMQ5Y0GT95wF7ArtJ+laWtg4gSU8Db46IP/ZX2FRgq3YjNjMzq6kDsiVvweTJTJ09e9C8\nZXeg1gGuj4iTs/c3S9qZdCO0O1BmZv2IiJWS5gFTgIsg9YSy92c3ybIU2Lkh7Wjg9cC7gHu6FqyZ\nmdkwVnYH6iHgjoa0O4B3DpxtGjCmIe0g4OBOxWVWuZuAWxrSVi5ZUkUoVl9nAbOyjlTfNOYbAbMA\nJJ0ObBkRh2UTTNyezyxpEfBkRDQeh83MzKygsqcxv4q1b3bekUEnkphOeoRJfunrPF3Y0QCLmVdB\nmQA3VlTuTRWUeWsFZVZTbt+3uxvwgYblXWPHlh6P1VdEzAY+A5xGOiDsCuwfEQ9nq4wHtq4ovOeo\n4qhRd9dUHUAN+TtpYv5Pqo6gdn5TdQA1VFUrqc7KfDJp2R2oGcBrJJ0o6WWS3gscDpzT+iZ/0aHQ\nhmJ+BWVCdR2oxusiZbitgjKrKbeKb9d6V0TMjIhtI2LDiJgUETfkPvtQRLxhgLzTI6KU5+75d702\nP4Rrbe5ANTH/p1VHUDtlNox7RVWtpDprnBCim0rtQGUV/cHAoaTO8+eBYyPCRwszMzMzM6u9su+B\nIiIuAS4pu1wzMzMzM7N2lT2Ez8zMzMzMrGeVfgVqiDZI//xtgFWW0v9o+4faKPqJAT5bAdw3wOft\nfK1PDfDZk8D9bWy7P6sG+XwF6ZnHzRR+LmcTg+1rO/9/req/3IVt3Pq2YpDP+vt2/7VyZd/LDVov\n3WxINgBY1OZGBvpdF7Wmzfyd1IlYltP+3PEDHTXL1omDUie+k/U6EEfH3NeBe6RXLOnAdqqoP5vr\nxJSf/+rAdurzjXRGJ1pJSzsRSIfc2YFtPNGB7TyyeHHfywEPcUoz3dZTNsnEeVXHYVYD74uI86sO\nwoY/H3fNzMwGbnfVvQP1QmB/0gmqJ6uNxqwSGwDbAnMi4tGKY7ERwMddMzMbwQq1u2rdgTIzMzMz\nM6sTTyJhZmZmZmZWkDtQZmZmZmZmBbkDZWZmZmZmVpA7UGZmZmZmZgX1bAdK0tGS7pa0QtK1kl7V\n5fJOlHS9pKWSFkq6UNLLu1lmkxg+K2mNpLNKKGtLST+S9Iik5ZJuljSxy2WOknR69v+6XNLfJZ3U\nhXL2kXSRpAey7/PtTdY5TdKDWRy/k7R9t8qUtK6kr0i6RdIT2To/kPSidso0q0LZx+Y6q0O9UXdl\n1mt1V0W9W2dltQnqrIr2St3VpT3Vkx0oSYcAZwLTgN2Bm4E5kjbrYrH7AN8EXg28ERgN/FbShl0s\n8xlZI+RjpH3tdlljgatIz2zcH3gF8Glg8UD5OuDzwEeAI4GdgOOB4yV9osPlbAzcBBwFrDUNpaQT\ngE+Qvu+9gGWk31c7z2scqMyNgN2A6aTf88HAjsAv2yjPrHQVHZvrrNJ6o+7KrNfqrsJ6t87KahPU\nWRXtlbqrRXuqJ6cxl3QtcF1EHJu9F3AfcHZEfLWkGDYDFgH7RsTcLpe1CTCPdBA5GbgxIj7VxfK+\nDEyKiP26VUY/5f4KWBARH82l/RxYHhEf6FKZa4CDIuKiXNqDwBkRMSN7PwZYCBwWEbO7UWaTdfYE\nrgO2iYj72y3TrAx1ODbXWZn1Rt2VXa/VXVX1bp1V0SaosyraK3VXZXuq565ASRoN7AH8vi8tUi/w\nMmBSiaGMJfV8HyuhrG8Bv4qIy0soC+BtwA2SZmfDTuZLOryEcn8DTJG0A4CkCcDewCUllE1W5kuB\n8Tz397WU9MdXxe9rSYllmrWsRsfmOiuz3qi7suu1uquq3q2zytsEdVaj9krddaU9tW4nN1aSzYBR\npB523kLSZbquy86qfh2YGxG3d7ms95AuR+7ZzXIabEc6K3gm8EXSZeGzJT0VET/qVqERMVPS1sBf\nJK0idfA/HxE/7VaZTYwn/aE1+32NLyMASesDXwbOj4gnyijTrAMqPzbXWZn1Rt1VVK/VXSX1bp3V\npE1QZ5W3V+qum+2pXuxA1cFM4JWkMyFdI+nFpAr3jRGxsptlNVgHuD4iTs7e3yxpZ+AIoGsHcknH\nAIcBhwC3kyrYb0h6cKRUIJLWBX5GOigeVXE4ZtY5pdQbdVdhvVZ3ldS7deY2gbWj2+2pnhvCBzwC\nrAbGNaSPAxZ0u3BJ5wAHAq+LiIe6XNwewObAfEkrJa0E9gOOlfR0dkazGx4C7mhIuwN4SZfK6/M5\n4AsR8bOI+HNEnAfMAE7scrl5CwBRwe8r98e+NfBmX32yHlPpsbnOSq436q6qeq3uqqp366wObYI6\nq6y9UndltKd6rgOVnbGaB0zpS8sOuFOAq7tZdlYJvgN4fUT8s5tlZS4DdiGddZmQLTcAPwYmRPdm\nALmKtYfc7Ajc26Xy+qxDaoDlraHE32lE3E068OR/X2NIs2h17feV+2PfDpgSESN55iXrQVUem+us\ngnqj7qqq1+quqnq3zipvE9RZVe2VuiurPdWrQ/jOAmZJmgdcDxxHmrpwVrcKlDQTOBR4O7BMUl+P\n//GIeLIbZUbEMtJl63wcy4BHI6LxTFUnzQCuknQiMJv0x3g48NEBc7XvF8BJku4H/gxMJP3fntvJ\nQiRtDGxPOnMDsF12c+pjEXEfaXjJSZL+DtwDfAG4nzamwRyoTNKZxwtIDYqpwOjc7+sxD3OxHlL6\nsbnOqqg36q7Ceq3uqqp366yUNkGdVdFeqbvatKcioicX0njGe4AVwDXAnl0ubw3pTEjj8oGS9/ty\n4KwSyjkQuAVYTjpwfbiEMjcCzgDuIj3L4G+kufzX7XA5+/Xz//m93DqnAg9m+z8H2L5bZQLbNPms\n7/2+Zf6+vHhpdyn72FznpS71Rt2Xsuq1ui9V1Lt1XspqE9R5qaK9UvelLu2pnnwOlJmZmZmZWRU8\njtTMzMzMzKwgd6DMzMzMzMwKcgfKzMzMzMysIHegzMzMzMzMCnIHyszMzMzMrCB3oMzMzMzMzApy\nB8rMzMzMzKwgd6DMzMzMzMwKcgfKzMzMzMysIHegzMzMzMzMCnIHyszMzMzMrCB3oMzMzMzMzAr6\n/7BND7tVMJKMAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"X, y = next_batch(FLAGS.batch_size, FLAGS.length, FLAGS.xlen)\n",
"tsteps = 2*FLAGS.length+3\n",
"\n",
"feed = {dnc.X: X, dnc.y: y, dnc.tsteps: tsteps}\n",
"fetch = [outputs['y_hat'], outputs['w_w'], outputs['w_r'], outputs['f'], outputs['g_a']]\n",
"[_y_hat, _w_w, _w_r, _f, _g_a] = sess.run(fetch, feed)\n",
"_y_hat = np.clip(_y_hat, 1e-6, 1-1e-6)\n",
"_y = y[0] ; _X = X[0]\n",
"\n",
"fig, ((ax1,ax2),(ax3,ax5),(ax4,ax6),) = plt.subplots(nrows=3, ncols=2)\n",
"plt.rcParams['savefig.facecolor'] = \"0.8\"\n",
"fs = 12 # font size\n",
"fig.set_figwidth(10)\n",
"fig.set_figheight(5)\n",
"\n",
"ax1.imshow(_X.T - _y.T, interpolation='none') ; ax1.set_title('input ($X$) and target ($y$)')\n",
"ax2.imshow(_y_hat[0,-FLAGS.length:,:].T, interpolation='none') ; ax2.set_title('prediction ($\\hat y$)')\n",
"\n",
"ax3.imshow(_w_w[0,:,:].T, interpolation='none') ; ax3.set_title('write weighting ($w_w$)')\n",
"ax4.imshow(_w_r[0,:,:,0].T, interpolation='none') ; ax4.set_title('read weighting ($w_r$)')\n",
"\n",
"ax5.imshow(_f[0,:,:].T, interpolation='none') ; ax5.set_title('free gate ($f$)') ; ax5.set_aspect(3)\n",
"ax6.imshow(_g_a[0,:,:].T, interpolation='none') ; ax6.set_title('allocation gate ($g_a$)') ; ax6.set_aspect(3)\n",
"\n",
"plt.tight_layout()"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"FLAGS.length = 12"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"starting free recall trials: \n",
"\ttrial #0\n",
"\ttrial #100\n",
"\ttrial #200\n",
"\ttrial #300\n",
"\ttrial #400\n",
"\ttrial #500\n",
"\ttrial #600\n",
"\ttrial #700\n",
"\ttrial #800\n",
"\ttrial #900\n",
"\ttrial #1000\n",
"\ttrial #1100\n",
"\ttrial #1200\n",
"\ttrial #1300\n",
"\ttrial #1400\n",
"\ttrial #1500\n",
"\ttrial #1600\n",
"\ttrial #1700\n",
"\ttrial #1800\n",
"\ttrial #1900\n",
"\ttrial #2000\n",
"\ttrial #2100\n",
"\ttrial #2200\n",
"\ttrial #2300\n",
"\ttrial #2400\n",
"\ttrial #2500\n",
"\ttrial #2600\n",
"\ttrial #2700\n",
"\ttrial #2800\n",
"\ttrial #2900\n",
"\ttrial #3000\n",
"\ttrial #3100\n",
"\ttrial #3200\n",
"\ttrial #3300\n",
"\ttrial #3400\n",
"\ttrial #3500\n",
"\ttrial #3600\n",
"\ttrial #3700\n",
"\ttrial #3800\n",
"\ttrial #3900\n",
"\ttrial #4000\n",
"\ttrial #4100\n",
"\ttrial #4200\n",
"\ttrial #4300\n",
"\ttrial #4400\n",
"\ttrial #4500\n",
"\ttrial #4600\n",
"\ttrial #4700\n",
"\ttrial #4800\n",
"\ttrial #4900\n"
]
}
],
"source": [
"recall_orders = []\n",
"trials = 5000 ; print \"starting free recall trials: \"\n",
"for i in range(trials):\n",
" X, y = next_batch(FLAGS.batch_size, FLAGS.length, FLAGS.xlen)\n",
" tsteps = 2*FLAGS.length+3\n",
"\n",
" feed = {dnc.X: X, dnc.tsteps: tsteps}\n",
" _y_hat = outputs['y_hat'].eval(feed)\n",
" _y_hat = np.clip(_y_hat, 1e-6, 1-1e-6)\n",
" _y = y[0] ; _X = X[0]\n",
" \n",
" real_y = np.tile(_y[-FLAGS.length:,:], [1,1,1])\n",
" pred_y = _y_hat[0:1,-FLAGS.length:,:]\n",
" order = guess_recall_order(real_y, pred_y, FLAGS.length)\n",
" \n",
" recall_orders.append(order)\n",
" if (i%100 == 0): print(\"\\ttrial #{}\".format(i))\n",
"recall_orders = np.stack(recall_orders)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAArsAAAHUCAYAAAA6KeCQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XmYXGWd9//3Jwm7EEGQyIiCO25ooo77Mqi4/NRxGTUu\nOOjzOO4anVHncRTUcRdwRXFFR42DjjrgqCjuGy6JwKggyuaObIYdQvL9/XFOQ6WoTndVV6c7p9+v\n6zpXqk7dp863Th+aT991n/ukqpAkSZK6aNFcFyBJkiTNFsOuJEmSOsuwK0mSpM4y7EqSJKmzDLuS\nJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOupHkpya2SfDXJX5NsSPLoua5pPkryj0k2\nJrlZz7pvJfnGNLffKcm5SVbOXpXTquPm7ec4aJrtNyZ5zWzXNZuSHJjkkiQ3mutapC4z7EoLSJJn\ntCFhYrkiyR+SfCXJC5PcYMA2h7Rt/5Rk+wGvn53k2AHrt0uyKsmJbWC9Ismvkrw7ya2nUe7HgTsA\n/w94OvDTET7yQlDt0r9uul4CXAx8emwVjW6TupM8PMkhm2k7b+93n+TeSb6X5LL2v513Jtmpt01V\nHQ/8BvjXualSWhiWzHUBkra4Al4NnA1sAywDHgi8A3hpkkdX1f8O2O7GwHOBIwa83ybanqrjgbsC\nXwQ+CVwK3BZ4MvB/gesF557ttwfuCby+qo6c/kfTMJIsAV4EHFZVcxocq+qcJDsA63tWPwJ4HvDa\nAZvsAFyzJWobVpK7ACcAvwRWATcF/gW4FfDIvuZHAW9LckhVXbZFC5UWCMOutDB9parW9jx/S5IH\nAv8D/HeS/arqqr5tTgL+JcmRA17r9zFgf+DxVfWF3heSvBp4wxTb37j9d90U7UiyY1VdPlW72ZRk\n+6q6ci5rGNGjgN2Bz8x1IQBVdXXfqgzRdj55I3Ah8ICJAJvkHOADSR5cVSf0tP0v4N3APwBHb+lC\npYXAYQySAKiqbwGvB24OPK3/ZeB1NL3Az93c+yS5B02P3If6g267n/VV9fLNbH8ITa9zAW9vh1Cc\n2b52aPt8vySfSnIh8N2ebW+b5LNJLmiHTfwkyaMG7GNpknck+W2SK5P8OsnLk0warnq2PTvJsUke\n2r7/FcCze15/WpKfJrm8rWN1kpsOeJ+/TfKlJBcmuTTJyUle1PP6nZJ8NMkZ7Wf5U5IPJ9ltqhqH\n8Bjg7Ko6q6+2o9uxpPsmOb6t7w/tHyr9n2PHJIf1HMvTkrxsQLuHJPlukova9z4tyRt6Xt9kzG6S\nj9L06k6Mz92YZENP++uN2U1y1yRfTrKu3ccJSf62r83EUJ57Jzk8yV/az/e5jGHsbJKdgQcD/9HX\nU/tx4DLgib3tq+o84BSan4WkWWDPrqRe/0HTK/VQ4MN9r30X+Abw8iTv20zv7qNpguonRqzhv4CL\naIZVfAr4Es0QCLhuyMRngNNpxjoGIMkdgO8BvwfexHXB4gtJHldV/9222wH4DnAT4P3A74B7t9ss\nA146RX0F3K6t7SjgA8Cv2vd+Fc0fBZ8GPgjsQTNM4NtJ7lpVF7ftHgIcB/yx/Zx/Bvaj+Yr7Xe1+\nHgLsC3ykff0OwD8BtwfuNUWN03VvYO2A9UXTGfIV4Ic0X8E/DHhtksVVdWhP2+OABwAfAk4GDqT5\nWn6vqnoZQJLbt+1OohlCcxXNV/r33kxt7wf2ogmOT2Uzvbw9+/gOzbcBb6YZ4vBPwLeS3L+qftK3\nybtpel8PBfahGW7wHuDaC/XaMbaTDrfpsX7iZwvcieb/rWt6G1TV+iQn0Qzt6bcGw640e6rKxcVl\ngSzAM4ANwPLNtLkI+GnP80PabXYD7gdsBF7c8/pZwLE9z/+rbb/LDOq8ebufl/atP6Rd/x8DtjkB\n+BmwpG/994DTep7/G80FWbfoa/dG4Grgb6ao7az28z24b/3NaMabvqJv/e3b931l+3wRcCZwBrDz\nZvaz3YB1T2r3fZ8BP9Ob9az7JvCNKT7H4na7tw547aPta0f0rT8OuALYrX3+mPbn8cq+dsfQhM19\n2+cvbt9v12n8zA/qWfduYMMk7TcCr+l5/vm2tpv3rFtGE36/2Xe8NtIM5el9v8Pan9POtelx2DiN\n5Rs92zy+/2fU89p/An8YsP6V7Ta7j/rfjIuLy+SLwxgk9bsU2HnQC1X1XZog9fIk202y/S7tv5fM\nQm3Q9Doe1bsiya7Ag2h6fJcmudHEAnwVuHWSm7TNn0DTS72ur93XaXrk7j+NGs6qTcddQhNyAnym\n733/Avy6rQ9gOU1P4juqatJjVD0952lmtrgR8KN2H8unUeNUdmvf66LNtHlv3/P3ANvR9LZCM1zl\nGppQ2uswmlD/8Pb5X9t/HzudoSLDSrKIpif881V1zsT6qvozTQ/8fbPpTCNF0yPf67s0fwDcvGfd\nW2g+61RL77CNHdp/B33zcWXP670mfga7D/6EkmbCYQyS+t0AOHczrx8KfBt4DvDOAa9PfJ27c8/j\ncTur7/mtaILb64F/H9C+aC56+xNwa5qvms/bTLth9z9RwyKaqaQGve/EBVW3aJ//YnM7aAP8oTS9\nub01FbB0GjVO12ThcyNND3Sv09t/92n/vRnwx7r+LAKntv9OBMf/BJ5FM7TjzUm+DnwO+GxVjWMW\niD2AHXvq669lEbB3T13QDF/pNRE4d51YUVWnAacNWcsV7b+D/hjcvuf1XhM/g3k7lZq0NTPsSrpW\nkr+hCVKDAhvQ9O4m+RZN7+5RA5pMhIM7Ad8fe5GN/sAw8S3V22mmPBvkNz1tv0bTazco6A0KTFPt\nf+J9N9KMbd044PVLB6zbnM/QTL/2VpqxsJe2+zie8VxcfCFNuNp1qoYzVc1MFfdP8iCacckPownx\nX0/y0DEF3mFtmGT9tedEkl0Y3BPb7+qqmgjLf2rf4yYD2t2EZpx2v4mfwfnT2JekIRl2JfU6iCYA\nfWWKdofSDGf4pwGvHUdz4djTmL2w22+iB3J9VU1157AzgBtU1TfHXMMZNCHn7Kqa9I+FnnZ3pLng\n73qS3BD4O+DVVdU7Y8GtxlVsVW1IcgbNRXCDLKLphe79LLdt/53o2T4HOCDJTn29u/v1vN67z2/S\nnDf/nORfaXrhH8Qkx4Hp93SeB1zeU1+v/Wj++OjvyZ2Od9KM8Z3Kt2h+XgA/pxnacTfgsxMNkmwD\n3IWml7vfvsD5VXXBCDVKmoJjdiUBkOTvaC7eOpNmnOOkquo7NEMZXkHf1epVdSJNWP4/Sa53hXmS\nbZO8bVx1t/s8jyZw/FOSZQP22TsW8hjgXkkeOqDd0iSLRyzjczShauAdv3qmDFtLExZfkmSy4QgT\nvY79v6NXMd6vun9IE8om84IBz6/munD6JZpOk/52q2iOxZfh2iEZ/U6mCf2Tjf2GZkaNiR7WSVXV\nRpqx2Y/JprdN3pNmdoXvVtWwPeswwpjdamZlOAF4Wja9Y9pBwE4051+/FTQ/C0mzwJ5daeEJ8Igk\n+9H8DtiTplfqITQh7NE1vQn7X0vTSzfIQTRft/9Xki/SXPx1Gc142SfTXCX/LzP5EAM8n+Yio/9N\n8kGa0L4nzTRdf8N1Uz69jWZ6tC8mOZpm2qedgDsDj6MZj3rhsDuvqjOT/BvwxiT7Al+guUjvFsDf\n01xUd3hVVZLnAscCJ7Xzyf6JZjqz21fVw6vqkiTfoRkqsi3wB5rp4PZhiim4hvTfNKHsVgN6o68C\nHtYeox/RXIz2cOANPT2Qx9GcA29oP/PE1GOPopnJYaIH+DVJ7k9z05JzaH4uzwV+SzNbxmTW0Hze\ndyc5nmZmhkE9o9D8ofZg4PtJjqT5g+HZwLZA/7zOkx3DTdaPOGYX4FU032p8J8kHaMYLvxQ4vqq+\ntskOkz1ozr3+i/wkjYlhV1p4iutuv3o1TbD7X5r5YI8ecLHR4Dep+naSb9PMXlB9r52f5N40NwV4\nEs3X1dvSfJX8RZq5ZadT57R7Mavq1CR3o+lZfQYwMRPCz+i53WxVXdEGr/9Hc9eqp9NcSHc68Bqm\nvmvbpHVV1VuS/IqmZ3Pihge/o+npPran3Vfb8auH0ISgRTTDG3pnCFhJE4CeRxPCjqcJm3+cbP8D\n6pzKccAFNPMRv7HvtWtoxta+n2bc8CXAoVX1+p7PUWlu2vE6mp/zP9LcEOSfq6r3ttL/TXOx2sE0\nMw6cT9MTf2jfjBT9NX+OZt7hJ3PdXLv/2dP22vZV9csk96OZL/mVNMf0ROApVfXTvved7NiMpde8\nqn6W5ME0PcOH0xy7D9Kcc/0eTzNLw7y4i53URZmb6wIkSfNB2xt9MHCriQvF2t7mx1fVZocPaOaS\nrKWZp/ef57oWqavmxZjdJPdLc/vNP7S3cXz0NLZ5YJI1aW5PeXqS6VxEIEna1BE0wziePNeFLDRJ\nDqSZsu7Nc12L1GXzIuzS/KI9iebruim7mpPsQ/NV6NeB/WmumP1QewtOSdI0VdVlVbWsqlbPdS0L\nTVUdX1W7VJVTjkmzaF6M2a2qr9BOdTTNu+s8FzizqiYuOvhVkvvSjJP72uSbSZKmyTFukjphvvTs\nDuueNFO79Dqe5qprSdIMVNXBVTXOu7RJ0pyZFz27I1jG9W9nei6wS5Lteu8pP6G9r/yBNFcKXznr\nFUqSJGlY29NMs3j8uG60srWG3VEcCHxyrouQJEnSlJ7KFDc4mq6tNez+mWZS8l57AhcP6tVtnQ3w\niU98gv3222+SJuq3atUqjjjiiKkbahMet+F5zEbjcRuex2w0HrfhecyGd+qpp/K0pz0N2tw2Dltr\n2P0hzeTqvR7K5m+3eCXAfvvtx/Lly2errs5ZunSpx2sEHrfhecxG43EbnsdsNB634XnMZmRsQ07n\nxQVqSXZKsn+Su7SrbtE+37t9/U1JPtazyfvbNm9JctskzwOeQHOnGkmSJAmYJ2EXuBvNLT3X0Ex3\ncxiwlutu8bmM5t7iAFTV2cAjae6DfhLNlGPPqqr+GRokSZK0gM2LYQxV9W02E7yr6uAB674DrJjN\nuiRJkrR1my89u5qnVq5cOdclbJU8bsPzmI3G4zY8j9loPG7D85jND6laGDfJSbIcWLNmzRoHi0uS\nJM1Da9euZcWKFQArqmrtON7Tnl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJ\nnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXY\nlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJ\nUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZ\ndiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJ\nktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZ\nhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1J\nkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZ8ybsJnl+\nkrOSXJHkxCR3n6L9QUlOTnJZkj8m+XCS3bZUvZIkSZr/5kXYTfIk4DDgEOCuwMnA8Ul2n6T9A4CP\nAB8Abg88AbhH+1ySJEkC5knYBVYBR1XVx6vqNOA5wOXAMydpfzfgrKp6b1WdU1U/AI6iCbySJEkS\nMA/CbpJtgBXA1yfWVVUBJwD3mmSzE4BlSR7evseewD8A/zO71UqSJGlrMudhF9gdWAyc27f+XGDZ\noA2q6mTgIOAzSa4G/gRcBLxgFuuUJEnSVmY+hN2hJbkncDTwGmA5cCCwL81QBkmSJAmAJXNdAHA+\nsAHYs2/9nsCfJ9nmJcDxVXV4+/znSZ4HfDfJq6qqv5f4WqtWrWLp0qWbrFu5ciUrV64cqXhJkiQN\nb/Xq1axevXqTdevWrRv7ftIMj51bSU4EflRVL26fB/gt8K6qetuA9scA11TVU3rW3Qv4HvA3VXW9\nkJxkObBmzZo1LF++fJY+iSRJkka1du1aVqxYAbCiqtaO4z3nyzCGw4H/286dezvg/cCONEMVSPKm\nJB/raf8F4PFJnpNk3yT3Ad5JE5gn6w2WJEnSAjMfhjFQVce0c+q+jmb4wknAgVV1XttkGbB3T/tP\nJdkFeD7wduCvNLM5vHKLFi5JkqR5bV6EXYCqOhI4cpLXDh6w7v00PcCSJEnSQPNlGIMkSZI0doZd\nSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIk\ndZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZh\nV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5Ik\nSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l\n2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUk\nSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJn\nDR12k3w7yUFJdpiNgiRJkqRxGaVn92fA24E/J/lgknuOuSZJkiRpLIYOu1X1EmAv4GDgxsB3kvwy\nyT8n2XPcBUqSJEmjGmnMblVdU1Wfq6rHADcFPgW8Hvhdki8k+btxFilJkiSNYkYXqCW5B/Ba4GXA\nX4A3AecDX0zy9pmXJ0mSJI1uybAbJLkx8HSaYQy3Bo4DVgLHV1W1bY4GvgL889gqlSRJkoY0dNgF\nfg+cAXwEOLqqzhvQ5hTgJzMpTJIkSZqpUcLuAVX13c01qKqLgQeNVpIkSZI0HqOM2X1tkhv2r0yy\nS5JvjKEmSZIkaSxGCbsPALYdsH574H4zK0eSJEkan2kPY0hy54mHwO2TLOt5eTHwMOAPY6xNkiRJ\nmpFhxuyeBFS7DBqucAXwwnEUJUmSJI3DMGF3X5pe3TOBewC9szBcDfylqjaMsTZJkiRpRqYddqvq\nnPbhjG5EIUmSJG0p0wq7SR4NfLmq1rePJ1VVx46lMkmSJGmGptuz+wVgGc0tgb+wmXZFc7Ha0JI8\nn+aOa8uAk4EXVtWkN6ZIsi1wCPDUdps/Aq+rqqNH2b8kSZK6Z1pht6oWDXo8LkmeBBwGPBv4MbAK\nOD7Jbarq/Ek2+wywB81ti88AboJDLCRJktRjlDuozYZVwFFV9XGAJM8BHgk8E3hrf+MkD6OZ0/cW\nVfXXdvVvt1CtkiRJ2kpMd8zui6b7hlX1rmEKSLINsAJ4Y897VJITgHtNstmjgJ8Cr0jydOAy4Fjg\n1VV15TD7lyRJUndNt2d31TTbFTBU2AV2pxnne27f+nOB206yzS1oenavBP6+fY/3AbsBzxpy/5Ik\nSeqo6Y7Z3Xe2CxnSImAj8JSquhQgyUuBzyR5XlVdNdmGq1atYunSpZusW7lyJStXrpzNeiVJktRj\n9erVrF69epN169atG/t+UlVjf9OhCmiGMVwOPL532rIkRwNLq+qxA7Y5Grh3Vd2mZ93tgF8At6mq\nMwZssxxYs2bNGpYvXz72zyFJkqSZWbt2LStWrABYUVVrx/Ge0x2zezjNeNjL2seTqqqXDlNAO3fv\nGuAAmnG3JEn7fLIhEd8HnpBkx6q6vF13W5re3t8Ps39JkiR113TH7N4V2Kbn8WRG7SY+HDi6Db0T\nU4/tCBwNkORNwF5V9Yy2/aeAfwM+muRQminI3gp8eHNDGCRJkrSwTHfM7oMGPR6Xqjomye7A64A9\ngZOAA6vqvLbJMmDvnvaXJXkI8G7gJ8AFwH8Crx53bZIkSdp6zWie3SR7A1TV72ZaSFUdCRw5yWsH\nD1h3OnDgTPcrSZKk7hr6jmNJliR5fZJ1wNnA2UnWJfn39mIzSZIkaV4YpWf33cDjgJcDP2zX3Qs4\nFLgR8NyxVCZJkiTN0Chh9ynAk6vqyz3rTknyO2A1hl1JkiTNE0MPYwCuohm+0O8s4OoZVSNJkiSN\n0Shh9z3Aq5NsN7Giffyq9jVJkiRpXpjuTSU+17fqwcDvk5zcPt8f2Bb4+hhrkyRJkmZkumN2+29U\n/F99z2c89ZgkSZI0btO9qcT15rmVJEmS5rtRxuxKkiRJW4WR7qCW5AnAE4Gb0YzVvVZVLR9DXZIk\nSdKMjXIHtRcBHwXOBe4K/Bi4ALgF8OXNbCpJkiRtUaMMY3ge8OyqeiHNvLpvraqHAO8Clo6zOEmS\nJGkmRgm7NwN+0D6+Ati5ffwfwMpxFCVJkiSNwyhh98/Abu3j3wL3bB/vC2QcRUmSJEnjMErY/Qbw\n6PbxR4EjknwN+E/g8+MqTJIkSZqpUWZjeDZtSK6q9ya5ALg3cCxw1BhrkyRJkmZk6LBbVRuBjT3P\nPw18epxFSZIkSeMw6jy7uwLPAvZrV/0S+GhVXTiuwiRJkqSZGmWe3fsDZwEvAnZtlxcBZ7WvSZIk\nSfPCKD277wWOAZ5bVRsAkiwGjmxfu9P4ypMkSZJGN8psDLcCDpsIugDt48Pb1yRJkqR5YZSwu5br\nxur22g84eWblSJIkSeMzrWEMSe7c8/RdwDuT3Ao4sV13T+D5wCvHW54kSZI0uumO2T0JKDa9Q9pb\nB7T7FM3NJSRJkqQ5N92wu++sViFJkiTNgmmF3ao6Z7YLkSRJksZt1JtK3BJ4CZveVOKdVXXGuAqT\nJEmSZmqUm0ocSBNu7wGc0i5/C/wiyUPGW54kSZI0ulF6dt8MHFFVm8y8kOTNwFuAr42jMEmSJGmm\nRplndz/gwwPWfwS4/czKkSRJksZnlLB7HnCXAevvAvxlZuVIkiRJ4zPKMIYPAh9IcgvgB+26+wCv\noLllsCRJkjQvjBJ2Xw9cArwMeFO77o/AoTR3V5MkSZLmhaHCbpIAewPvq6ojkuwMUFWXzEZxkiRJ\n0kwMO2Y3wG9oAi9VdYlBV5IkSfPVUGG3qjYCvwZuNDvlSJIkSeMzymwMrwTeluSO4y5GkiRJGqdR\nLlD7OLAjcHKSq4Erel+sqt3GUZgkSZI0U6OE3ZeMvQpJkiRpFgwddqvqY7NRiCRJkjRuo/TskmQx\n8FiaWwcD/BL476q6ZlyFSZIkSTM1dNhNcgfgWGAZ8Kt29SuA85I8qqp+Psb6JEmSpJGNMhvDh4Bf\nADetquVVtZxm3t1TgA+MszhJkiRpJkYZxnAX4G5VddHEiqq6KMmrgJ+MrTJJkiRphkbp2T0d2HPA\n+hvT3F1NkiRJmhdGCbv/CrwryROS3LRdngC8A3hFkl0mlvGWKkmSJA1nlGEMX2z/PQao9nHaf4/r\neV7A4tFLkyRJkmZmlLD7oLFXIUmSJM2CUW4q8e3ZKESSJEkat1HG7EqSJElbBcOuJEmSOsuwK0mS\npM4y7EqSJKmzDLuSJEnqrGnNxpDkZ1w3p+5mVdXyGVUkSZIkjcl0px77wqxWIUmSJM2CaYXdqnrt\nbBciSZIkjZtjdiVJktRZ0x2zexHTH7O724wqkiRJksZkumN2XzKrVUiSJEmzYLpjdj8224VIkiRJ\n4zbdnt2BkmwPbNu7rqounlFFkiRJ0pgMfYFakp2SvCfJX4DLgIv6FkmSJGleGGU2hrcCfwc8F7gK\n+D/AIcAfgYPGV5okSZI0M6MMY3gUcFBVfSvJR4HvVtVvkpwDPBX45FgrlCRJkkY0Ss/ubsCZ7eOL\n2+cA3wPuP46iJEmSpHEYJeyeCezbPj4NeGL7+FHAX0ctJMnzk5yV5IokJya5+zS3u0+S9UnWjrpv\nSZIkddMoYfejwP7t4zcDz09yJXAE8LZRikjyJOAwmrG/dwVOBo5PsvsU2y0FPgacMMp+JUmS1G1D\nj9mtqiN6Hp+Q5HbACuA3VXXKiHWsAo6qqo8DJHkO8EjgmTQXxE3m/TRjhDcCjxlx35IkSeqoUXp2\nN1FV51TV50YNukm2oQnLX+95z6Lprb3XZrY7mGY4xWtH2a8kSZK6b5R5dt+V5AUD1r8gyTtGqGF3\nYDFwbt/6c4Flk9Rwa+CNwFOrauMI+5QkSdICMErP7uNpZl7o9wPgCTMrZ2pJFtEMXTikqs6YWD3b\n+5UkSdLWZ5R5dm8EXDJg/cU0vbTDOh/YAOzZt35P4M8D2u8M3A24S5L3tusWAUlyNfDQqvrWZDtb\ntWoVS5cu3WTdypUrWbly5QilS5IkaRSrV69m9erVm6xbt27d2PeTZnjsEBskPwfeX1Xv6Vv/QuC5\nVXX7oYtITgR+VFUvbp8H+C3wrqp6W1/bAPv1vcXzgQfR9DqfXVVXDNjHcmDNmjVrWL58+bAlSpIk\naZatXbuWFStWAKyoqrFMKztKz+7hwHuS7AF8o113APAy4CUj1nE4cHSSNcCPaWZn2BE4GiDJm4C9\nquoZ7cVrv+zdOMlfgCur6tQR9y9JkqQOGmXqsY8k2Q54FfDqdvXZNL26Hx+liKo6pp1T93U0wxdO\nAg6sqvPaJsuAvUd5b0mSJC1co/TsUlXvA97X9u5eUVWXzrSQqjoSOHKS1w6eYtvX4hRkkiRJ6jPS\nPLtJliR5MPA42pkQkuyV5AbjLE6SJEmaiaF7dpPcHPgKcDNgO+BrNLMzvKJ9/pxxFihJkiSNapSe\n3XcCPwV2BXpnPfg8zYVqkiRJ0rwwypjd+wH3rqqrm1nArnU28DfjKEqSJEkah1F6dhfR3N63300Z\nfLMJSZIkaU6MEna/yqbz6VZ7YdprgS+NpSpJkiRpDEYZxvAy4PgkvwS2Bz4F3Jrmtr/ec1eSJEnz\nxig3lfgUNbcoAAAWHklEQVR9kv2BJwH7AzcAPgx8ctBteiVJkqS5MupNJa4BPtku10qyU1VdNo7C\nJEmSpJka6aYS/ZIsS/Jm4HfjeD9JkiRpHKYddpPsnOSDSU5J8tEkOyTZI8lRwFnAU4G3zFqlkiRJ\n0pCGGcbwFuAfaO6YdiDwPuDBwIXAwcBnqmrD2CuUJEmSRjRM2P3/gIOq6tgkdwROAY4GnlVVNRvF\nSZIkSTMxzJjdZcAagKr6OXAVcIRBV5IkSfPVMGF3MXB1z/NrgEvHW44kSZI0PsMMYwhwdJKr2ufb\nA+9PsslUY1X1uHEVJ0mSJM3EMGH3Y33PPzHOQiRJkqRxm3bYraqDZ7MQSZIkadzGclMJSZIkaT4y\n7EqSJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqS\nJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmz\nDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuS\nJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnq\nLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOu\nJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOuJEmSOsuwK0mSpM6aN2E3yfOTnJXkiiQnJrn7Zto+\nNslXk/wlybokP0jy0C1ZryRJkua/eRF2kzwJOAw4BLgrcDJwfJLdJ9nk/sBXgYcDy4FvAscl2X8L\nlCtJkqStxLwIu8Aq4Kiq+nhVnQY8B7gceOagxlW1qqreXlVrquqMqnoV8GvgUVuuZEmSJM13cx52\nk2wDrAC+PrGuqgo4AbjXNN8jwM7AhbNRoyRJkrZOcx52gd2BxcC5fevPBZZN8z3+BdgJOGaMdUmS\nJGkrt2SuC5ipJE8BXg08uqrOn6r9qlWrWLp06SbrVq5cycqVK2epQkmSJPVbvXo1q1ev3mTdunXr\nxr6fNCMG5k47jOFy4PFVdWzP+qOBpVX12M1s+2TgQ8ATquorU+xnObBmzZo1LF++fCy1S5IkaXzW\nrl3LihUrAFZU1dpxvOecD2OoqvXAGuCAiXXtGNwDgB9Mtl2SlcCHgSdPFXQlSZK0MM2XYQyHA0cn\nWQP8mGZ2hh2BowGSvAnYq6qe0T5/Svvai4CfJNmzfZ8rquriLVu6JEmS5qt5EXar6ph2Tt3XAXsC\nJwEHVtV5bZNlwN49m/xfmova3tsuEz7GJNOVSZIkaeGZF2EXoKqOBI6c5LWD+54/aIsUJUmSpK3a\nnI/ZlSRJkmaLYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmG\nXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmS\nJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHXWkrkuYEs77jg47TTYZpvrL9tuO3j9ZG0WL57r\nTyNJkqTNWXBh99BDx/deycwD86htxrkfQ7skSeqqBRd2f/QjuOMdYf36wcvVV0/+2rhfv/TS0ba/\n5prxHpNBoX1Lh/dxBPzFi5vPIkkL0caNzf8jZmO55hpYuhT22AN23735d+LxbrvBIgdFah5bcGF3\nyRLYcce5rmJmqppfPPMhsA8K7dPdftyhHeZXr/mSJcMvixcPXm+Il+bGxO/b2QqRM1muumrT5xs2\nzPzzLl7c/H7rXxYvhnXr4IILmlDda9GiJvD2BuBBobj33+23n3mt0nQtuLDbBb09sVuzzYX2LR3Y\n16+Hyy4bbfvZCO39Fi0aLTyPI2hviWWqfS9aZODvmg0brvtvaaoQN9fLOAwKkFMtu+wydZvtthvt\nvQct07kWZcMGuOgiOP98OO+86/7tfXz++XD22detv/LK67/PDW4wvVA88fiGN/R3gEZn2NWc6XJo\nv+aamS8bNoznfQYtl18+nn2vX7/ljvNCDfrD9O5XTR4g59syjl7IJUuGD3Q77tgEp3EFxOksXfp2\nZvHiJnzuvjvc7nbT2+ayyyYPxRPrzjgDTjyxWXfhhdd/jyVL4EY3mn7v8e67N8deAsOuNGNdCe2j\n2rhxfCF8NgN+73L11dcP/KPuexyhbToG9e4vXrzpV+zj+uNjlN7CHXecnaA4WS3bbOM40a3FTjs1\ny81vPr3211zTDJeYqvf49NOvWz/o3F+6dHq9xhP/7rxzd/4o0aYMu5JmZNGi6wLIQlQ185A+k+0n\nxpCPa/FCT821JUtgzz2bZTqq4JJLpu49PvVU+M53mscXX3z999l22+mPOd5jj2ac8hJT1FbBH5Mk\nzUByXU+rpC0vacY377IL3OIW09vm6qubILy5gPznP8PPf37duv5vcRLYddfheo932mn8n19T89ez\nJElaULbdFvbaq1mmowr++tfJe40nHp988nXrLrvs+u+zww7D9R7vuqvDdcbBsCtJkrQZE724u+4K\nt7nN9La54oqpe49/+1tYu7Z5fMEFTajutWjRdRfmTTcgb7fd+D//1s6wK0mSNGY77AB7790s0zEx\nrdtUvcdnnnnd48mmdZvulG577NFcyNf1cfqGXUmSpDnWO63bdFQ1s8ps7qK888+HX/8afvjD5vlF\nF13/fZYsuX4Y3lxQ3n33rW/2IcOuJEnSVia5blq3ffaZ3jbr1zfzGG9uSrfzzoPTTrvu8WTTuk0W\nigcF5BvcYG57jw27kiRJC8A224w2rVt/KO4PyL/4xXWPB03rtt120x9zfMEF4/3MYNiVJEnSAL3T\nut3yltPb5qqrmsC6ueEVf/oTnHLKdRfwzfbNeQy7kiRJGovtthtuWreNG5tp3SbC8I9/DC996Xhr\nMuxKkiRpTixa1NyNbrfdmmnddthhFvYx/reUJEmS5gfDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ\n6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izD\nriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJ\nkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOmjdhN8nzk5yV5IokJya5+xTtH5hk\nTZIrk5ye5BlbqtaFZPXq1XNdwlbJ4zY8j9loPG7D85iNxuM2PI/Z/DAvwm6SJwGHAYcAdwVOBo5P\nsvsk7fcBvgh8HdgfeCfwoSQP2RL1LiT+hzoaj9vwPGaj8bgNz2M2Go/b8Dxm88O8CLvAKuCoqvp4\nVZ0GPAe4HHjmJO2fC5xZVS+vql9V1XuBz7bvI0mSJAHzIOwm2QZYQdNLC0BVFXACcK9JNrtn+3qv\n4zfTXpIkSQvQnIddYHdgMXBu3/pzgWWTbLNskva7JNluvOVJkiRpa7VkrgvYgrYHOPXUU+e6jq3K\nunXrWLt27VyXsdXxuA3PYzYaj9vwPGaj8bgNz2M2vJ6ctv243jPNiIG50w5juBx4fFUd27P+aGBp\nVT12wDbfBtZU1Ut71v0jcERV7TrJfp4CfHK81UuSJGkWPLWqPjWON5rznt2qWp9kDXAAcCxAkrTP\n3zXJZj8EHt637qHt+skcDzwVOBu4cgYlS5IkaXZsD+xDk9vGYs57dgGSPBE4mmYWhh/TzKrwBOB2\nVXVekjcBe1XVM9r2+wD/CxwJfIQmGL8DeERV9V+4JkmSpAVqznt2AarqmHZO3dcBewInAQdW1Xlt\nk2XA3j3tz07ySOAI4EXA74FnGXQlSZLUa1707EqSJEmzYT5MPSZJkiTNCsOuJEmSOqszYTfJ85Oc\nleSKJCcmufsU7R+YZE2SK5OcnuQZW6rW+WSY45bkAUk29i0bktx4S9Y8l5LcL8mxSf7Qfv5HT2Ob\nBX+uDXvcPNcgyb8m+XGSi5Ocm+TzSW4zje0W7Pk2yjHzXIMkz0lycpJ17fKDJA+bYpsFe57B8MfM\n8+z6kryyPQ6HT9FuxudaJ8JukicBhwGHAHcFTgaOby96G9R+H+CLNLco3h94J/ChJA/ZEvXOF8Me\nt1YBt6a5aHAZcJOq+sts1zqP7ERzAeXzaI7FZnmuXWuo49Za6Ofa/YB3A38LPBjYBvhqkh0m28Dz\nbfhj1lro59rvgFcAy4EVwDeAY5PcflBjzzNgyGPWWujn2bXajrVn0+SOzbXbh3Gca1W11S/AicA7\ne56HZoaGl0/S/i3AKX3rVgNfmuvPMs+P2wOADcAuc137fFiAjcCjp2jjuTbacfNcu/4x2b09dvfd\nTBvPt+GPmefa4ONyAXDwJK95ng1/zDzPrjsWNwB+Bfwd8E3g8M20Hcu5ttX37Ka5A9sKmtQPQDVH\n4wTgXpNsds/29V7Hb6Z954x43KAJxCcl+WOSrya59+xWutVb8OfaDHiubeqGND1DF26mjefbpqZz\nzMBz7VpJFiV5MrAd8N1Jmnme9ZjmMQPPswnvBY6rqm9Mo+1YzrWtPuzS/OW+GDi3b/25NF8TDLJs\nkva7JNluvOXNW6Mctz8B/wQ8Hngczdc430pyl9kqsgM810bjudYjSWhunPO9qvrlZpp6vrWGOGae\na0CSOya5BLgKOAp4YlX9ZpLmnmcMfcw8z4D2j4K7AP86zU3Gcq7Ni5tKaOtQVacDp/esOjHJLWnu\neLegLk7Q7PJcu54jgdsD95nrQrYi0zpmnmvXOo1mTORSmjuYfjrJA6rqZ3Nb1rw27WPmeQZJbkrz\nB+iDq2r9ltx3F3p2z6cZB7Nn3/o9gT9Pss2fJ2l/cVVdNd7y5q1RjtsgPwZuNa6iOshzbXwW5LmW\n5D3AI4AHVtWfpmju+cbQx2yQBXeuVdU1VXVmVf2sql4F/Ah47iTNPc8Y+pgNstDOsxXAHsDaJOuT\nrKcZy/ziJFe338b0G8u5ttWH3favgzXAARPr2gN2APCDSTb7YW/71kPb9QvCiMdtkLvQfD2jwRb8\nuTZGC+5ca0PbY4AHVdVvp7HJgj/fRjhmgyy4c22ARTRD3QZZ8OfZJDZ3zAZZaOfZCcCdaD73/u3y\nU+ATwP7tdUP9xnOuzfVVeWO6su+JwOXAQcDtaMbOXADs0b7+JuBjPe33AS6hucrvtjTTIV1N07U+\n559nHh+3FwOPBm4J3IHm64j1NL0nc/55ttAx26n9D/QuNFd5v6R9vrfn2liPm+da8zX8RTTTae3Z\ns2zf0+aNnm8zPmaea80xuR9wc+CO7X+P62n+YPD32niO2YI/zyY5jpvMxjBbv9M6MWa3qo5p54Z9\nHc0vtpOAA6vqvLbJMmDvnvZnJ3kkcATwIprptp5VVf1X/HXasMcN2JZmXt69aELyKcABVfWdLVf1\nnLsbzX+c1S6Htes/BjwTz7XJDHXc8FwDeA7NsfpW3/qDgY+3j2+C51uvoY8ZnmsAN6b5b/EmwDqa\nY3BgVX2zfd3fa9c31DHD82wy/b25s/I7LW1yliRJkjpnqx+zK0mSJE3GsCtJkqTOMuxKkiSpswy7\nkiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJsyDJN5McPkT7ByTZmGSX\n2axrihoOSbJ2ijY3b+u885aqS5JmwtsFS1qwkuwOvB54BLAncBFwEvC6qvrhDN/7hsD6qrpsmu0f\nAHwD2LWqLp7JvkeVZEdgu6q6qH3+UWBpVT2up02APYDzq2rjXNQpScNYMtcFSNIc+hzN78GnA2fR\nBN4DgBuN+oZJtqmq9VX11/GUuOVU1eXA5VO0KeAvW6YiSZo5hzFIWpCSLAXuC7yiqr5TVb+rqp9W\n1Vuq6ou97ZJ8KMlfkqxLckLvV/jtV/8/S/KsJGcCV7Trv9U7jCHJ05L8JMnFSf6U5JNJ9hiy5o1J\nnpPkS0kuT3JGksf3tbljkq+3r5+f5KgkO/W8/sAkP0pyaZKLknw3yd69n2XiMfAM4DHtfjckuf+g\nYQztEIwfJbkyyR+TvCnJop7Xv5nknUnekuSC9vMfMsxnl6RRGXYlLVSXtsvfJ9l2M+0+S9PTeyCw\nHFgLnNAOU5hwK+BxwGOBu7Tr+seILQH+Dbgz8Bjg5sBHR6j7dcBn2vf5JPDpJLeFa4chHA9cAKwA\nngA8GHh3+/pi4PPAN4E7AvcEPtBX68TjtwPHAF+h6fG+CfCD/s+WZC/gf4AftTU9B3hW+1l7HURz\nvO8BvBx4TZIDRvj8kjQUhzFIWpCqakOSZwAfBJ7bXpj1beDTVfW/AEnuA9wNuHFVrW83fXmSx9IE\nyQ+167YBnl5VF25mf0f3PD07yUuAHyXZsR0+MF3HVNVESH5NkocALwReADwV2A44qKquBE5N8gLg\nuCSvAK4BdgH+p6rObt/jV5PUe1mSK4Btq+q8ifXNkF3S0/T5wG+r6kXt89PbXts30wTzCadU1evb\nx2e0dR0AfH2Izy5JQ7NnV9KCVVWfB/YCHgV8GXgAsDbJQW2T/YGdgQuTXDKxAPsAt+x5q3M2F3QB\nkqxIcmySc5JcDHyrfelmQ5Z9Yt/zHwL7tY9vB5zcBt0J36f5XX/b9sKzjwFfbWt5UZJlQ+6/3+3a\nGnp9H7hBkpv2rDulr82fgBvPcN+SNCXDrqQFraqurqqvV9Ubquq+wNHAa9uXbwD8kebr+f17ltsC\nb+t5m83OuNAOL/gK8FfgKTS9xY9tX97cEIqxq6pn0gxf+D7wJJqe2HtsgV2v73te+P8gSVuAv2gk\naVOnAhMXdK0FlgEbqurMvmWzPbl9bgfsBvxrVX2/qk6nGQc7insOeH5qT+37J9mh5/X7AhvoGa5Q\nVSe3F+LdB/g5TQAf5Gpg8RT1nArcq2/dfYFLqur3U2wrSbPOsCtpQUqyWztrwVOT3CnJPkn+AfgX\n4AsAVXUCzVf0X0jykHYmgnsn+fcky4fY3W9pguOLkuyb5NFc/wIu2HQs7GT+IcnBSW6d5LXA3YH3\ntK99ErgS+FiSOyR5EPAu4ONVdV77Gd+Y5J5JbpbkocCtgV9Osq+zgTsnuU2SGyUZdJ3HkcDeSd6d\n5LZJHgMcChw2jc8iSbPOC9QkLVSX0ox/fQnN+NttgN8BRwFv6mn3COANwEdobqbwZ+A7wLlTvP+1\nMxZU1flJ/hF4I83FZGuBlwHHTrbNZhwCPBl4L8241ydX1Wntfq5IciDwTuDHNHPmfrbdF+3z29HM\njHCjdvt3V9UHJtnXB2nGMf+Uprf7QcA5fZ/tj0keQTOs4yTgwna7Nwz5uSRpVngHNUnaSiTZCPx9\nVfWHZEnSJBzGIEmSpM4y7ErS1sOv4iRpSA5jkCRJUmfZsytJkqTOMuxKkiSpswy7kiRJ6izDriRJ\nkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrr/wdtwH+odHa/8AAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAArsAAAHUCAYAAAA6KeCQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xe8nGWd///XO6FJMaIo0RXFtooNTdTFXhCw/AS7Rl1c\ndNcVK9Fdy9d1saxdwIpiA101LroWZFUU7AVLIuiu2GgWFJASEBJaPr8/7vvAZJhzcmbOnJyT+7ye\nj8f9ODP3fd0zn7lzn+Sda677ulNVSJIkSV20aK4LkCRJkmaLYVeSJEmdZdiVJElSZxl2JUmS1FmG\nXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVfSnEty+yRfTXJxkmuS7D/XNc1HSf4h\nyYYkt+pZ980kX5/m/jskOTfJitmrclp13Lr9HAdOs/2GJP8+23XNF0lWJfmvua5D6grDrtRRSZ7Z\nhoSJZV2SPyb5SpIXJtlxwD6Htm3/lGS7AdvPSnLcgPXbJlmZ5OQ2sK5L8qsk705yh2mU+zHgLsD/\nA/4e+MkIH3khqHbpXzddhwCXAJ8aW0Wj26juJI9McugUbeflve2T7JPkw0l+nuTqJGdM0u6OSd6a\n5KdJLklyTpLjkywf0PwtwBOS3G12q5cWhq3mugBJs6qAVwNnAVsDS4GHAO8AXpJk/6r6+YD9bgYc\nDBwx4PU2kuQmwAnAPYHjgU8AfwXuCDwV+CfgesG5Z//tgL2A11fVkdP/aBpGkq2AFwGHVdWcBseq\nOjvJDYCrelY/Cnge8NoBu9wAuHpz1DaCpwFPBtYAf5yi3T8CzwL+G3gvsAT4Z+DkJPtV1bW981V1\nSpKfAC8F/mGW6pYWDMOu1H1fqao1Pc/fkuQhwP8AX0iyR1Vd0bfPKcC/JjlywLZ+HwX2BJ5QVZ/v\n3ZDk1cAbNrH/zdqfazfRjiTbV9Xlm2o3m5JsV1Xr57KGET0G2AX49FwXAlBVV/atyhBt55NXAv9Y\nVdck+SLNNxSDfBI4tPf8TXI0cBrwGqB/KMqxwGuSPG+uz3lpS+cwBmkBqqpvAq8Hbg08o38z8Dqa\nXuCDp3qdJPeh6ZH7UH/Qbd/nqqp62RT7H0rT61zA29shFGe0217TPt8jySeTXAh8p2ffOyb5TJIL\n2mETP07ymAHvsSTJO5L8Lsn6JL9J8rIkk4arnn3PSnJckn3b118HPKdn+zOS/CTJ5W0dq5LccsDr\n/F2SLyW5MMlfk5ya5EU92++W5Ogkp7ef5U/tV+M33lSNQzgAOKuqzuyr7Zgklya5TZIT2vr+2P5H\npf9zbJ/ksJ5j+cskLx3Qbp8k30lyUfvav0zyhp7tG43ZbUPf89rHE8Nurulpf70xu0numeTLSda2\n73Fikr/razMxlOd+SQ5Pcl77+T7bfiMxY1X156q6ZhrtftofWqtq4pzeY8AuXwN2BPYZR53SQmbP\nrrRw/SfwRmBf4MN9275D09P0siTvm6J3d3+aoPrxEWv4b+AimmEVnwS+RDMEAq4bMvFp4Nc0PWgB\nSHIX4LvAH4A3AZfRfJX8+SSPr6ovtO1uAHwbuDnwfuD3wP3afZYCL9lEfQXcqa3tKOADwK/a134V\nzX8KPgV8ELgpzTCBbyW5Z1Vd0rbbB/gicE77Of9ME24eDbyrfZ99gNsAH2m334XmK+47A/fdRI3T\ndT+ar9oHfcZFwFeAHwD/CjwCeG2SxVX1mp62XwQeDHwIOBXYD3hbkltU1UsBkty5bXcKzRCaK4Db\nt+8/mfcDtwAeDjydKXp5e97j2zTfBryZZojDPwPfTPKgqvpx3y7vBi6k6UHdHVgJvAe49kK9JDsw\nxXCbHldN/NmOyVLgLwPW/wJYB9wf+MIY309aeKrKxcWlgwvwTOAaYNkUbS4CftLz/NB2nxsDDwQ2\nAC/u2X4mcFzP8/9u299wBnXeun2fl/StP7Rd/58D9jkR+CmwVd/67wK/7Hn+bzQXZN22r90bgSuB\nv9lEbWe2n+/hfetvRTPe9OV96+/cvu4r2ueLgDOA04GdpnifbQese0r73vcf8Gd6q5513wC+vonP\nsbjd760Dth3dbjuib/0XacLWjdvnB7R/Hq/oa3csTdi8Tfv8xe3r7TyNP/MDe9a9G7hmkvYbgH/v\nef65trZb96xbShN+v9F3vDbQDOXpfb3D2j+nnWrj47BhGsukx7o9ZmcMce4/sD1Wh06y/ZfA8aP+\nbrm4uDSLwxikhe2vwE6DNlTVd2iC1MuSbDvJ/jdsf146C7VB0+t4VO+KJDsDD6Xp8V2S5CYTC/BV\n4A5Jbt42fyJNL/XavnYn0Xyz9aBp1HBmVZ3Yt+4JNL2Pn+573fOA37T1ASyj6Ul8R1VNeoyqp+c8\nzcwWNwF+2L7HsmnUuCk3bl/roinavLfv+XuAbWl6W6EZrnI1TSjtdRhNqH9k+/zi9ufjpjNUZFhJ\nFtH0hH+uqs6eWF9Vf6bpgX9ANp5ppGh65Ht9h+Y/ALfuWfcWms+6qeV6wzZG/Bw3bes9HXjbJM0u\nohlnLWkGHMYgLWw7AudOsf01wLeA5wLvHLB94uvcnXoej9uZfc9vTxPcXg/8x4D2RXPR25+AOwB3\nA86fot2w7z9RwyLgt5O87sQFVbdtn//fVG/QBvjX0PTm9tZUNFftj8tk4XMDTQ90r1+3P3dvf94K\nOKeqLutrd1r7cyI4/hfwbJqhHW9OchLwWeAzVTWOWSBuCmzfU19/LYuA3Xrqgmb4Sq+J0L/zxIqq\n+iVNT+qsS7I9zQWiOwD71uQXoIV5OuWatCUx7EoLVJK/oQlSgwIb0PTuJvkmTe/uUQOaTISDuwHf\nG3uRjXV9zye+kXo7zZRng/y2p+3XaHrtBgW9QYFpU+8/8bobaMa2bhiw/a8D1k3l0zTTr72VZizs\nX9v3OIHxXEh8IU1o2nlTDWeqmpkqHpTkoTTjkh9BE+JPSrLvmALvsCa7gOzacyLJDWmmONuUK6tq\nqh7yKSXZmmYYxl1pgu5pUzTfmemdo5KmYNiVFq4DaQLQVzbR7jU0wxn+ecC2L9JcOPYMZi/s9pvo\ngbyqeuYmncTpwI5V9Y0x13A6TVA6q6om/c9CT7u7cv2ppQBIciPgYcCrq6p3xoLbj6vYaqbFOp3m\nIrhBFtH0Qvd+lju2Pyd6ts8G9k6yQ1/v7h4923vf8xs0582/JHklTS/8Q5nkODD9Hszzgct76uu1\nB81/Pvp7cqfjnTRjfDflmzR/XkNrh3X8J81xeFJVfXeKtotpeqi9OE2aIcfsSgtQkofRXLx1Bs24\nwUlV1bdphjK8nL6r1avqZJqw/I9JDhjwPtskmWw84kiq6nyawPHPSZYOeM/eMY7HAvdNsu+Adkva\nQDGKz9KEqoF3/OqZMmwNTVg8JMlkwxEmeh37/z5eyXi/wv4BcK8ptr9gwPMruS6cfommg6S/3Uqa\nY/FluHZIRr9TaUL/ZGO/oZlRY6KHdVJVtYFmbPYB2fi2ybvSzK7wnaoatmcdNs+Y3fcATwIOrnbG\nkCncmeb3bXP9J1LqLHt2pW4L8Kgke9D8vu9K0yu1D00I27+mN2H/a2l66QY5kObr9v9OcjzNxV+X\n0YyXfSrNVfL/OpMPMcDzaS4y+nmSD9KE9l1ppun6G5q7uUFz4c/+wPFJjgFW04yTvDvweJrxqBcO\n++ZVdUaSfwPemOQ2wOdpLtK7LfBYmovqDq+qSnIwcBxwSjuf7J9opjO7c1U9sqouTfJtmqEi29Dc\nhWvftrZxXuD1BeAZSW4/oDf6CuAR7TH6Ic3FaI8E3lBVF7RtvkhzDryh/cwTU489hmYmh4ke4H9P\n8iCaMaln0/y5HAz8jma2jMmspvm8705yAs3MDP81Sdt/owme30tyJM1/GJ4DbAP0z+s82THcaP2o\nY3bT3NJ3//bp7WkumnxV+/zUqjq+bXcIzXH4PrA+ydP7XuqzVdU7ZGZfmt+j/osjJQ1rrqeDcHFx\nmZ2F66apmljW0QSpr9CExR0G7HPt1GMDtn2j3faFAdu2penhO5lm+qd1NGMN3w3cbhN13rp93ZXT\nraXdvjvNdFF/BNbThKkvAI/ta7c9zVfov2rrOpcmKB8CLN5EbWcM+rw92x9L0+t9Sbv8H83X4bfv\na3ff9rhf3Lb7KU3v3sT2mwOfAS6gCd+raELiNTTDG/r/TPunHjtpGufD1jSzRfy/vvVHtzXt3tZ4\nKc2cwK8e8Brb04yV/n17zH854M/tITQ9379vj/fvab66v11Pm4k/896pxxZx3TzEV9MzDVn/cWjX\n7UnT27y2rflrwH0m+R1Y1rf+we36B83C71nv8pG+4zxZu43+TNv2PwCOma2/H1xcFtKSKi/0lKSF\noO2NPogmjFe77miaWz1POXxAm0+SewA/Ae5ZVT+f63qkLd28GLOb5IFpbsn5x/bWjvtPY5+HJFmd\n5paVv04ynQsLJGkhO4JmGMdT57oQTenlwKcNutJ4zJcxuzvQ3FrywzRff00pye7A8cCRwNNoxm59\nKMk5VfW12StTkrZc1cyicL2L+jS/VNWKTbeSNF3zIuxW1Vdopz+a5h13Dqa5JePEhQi/SvIAmjGD\nhl1JGo7j2SR11rwYxjCCvbj+Faon0FwEIkmapqo6qKrGeZc2SZpX5kXP7giWcv1bnJ4L3DDJttVz\nn/kJ7b3m9wPOormKWJIkSfPLdjSzw5xQ1019OCNbatgdxX7AJ+a6CEmSJG3S09nETY+ma0sNu3+m\nmYOy167AJYN6dVtnAXz84x9njz32mKSJ+q1cuZIjjjhirsvY4njchucxG43HbXges9F43IbnMRve\naaedxjOe8Qxoc9s4bKlh9wc0d/fptW+7fjLrAfbYYw+WLVs2W3V1zpIlSzxeI/C4Dc9jNhqP2/A8\nZqPxuA3PYzYjYxtyOi8uUEuyQ5I924m0AW7bPt+t3f6mJB/t2eX9bZu3JLljkucBTwQO38ylS5Ik\naR6bF2EXuBfN7TNX00yBcxiwBnhtu30psNtE46o6C3g0zfy6p9BMOfbsqvIe4pIkSbrWvBjGUFXf\nYorgXVUHDVj3bWD5bNYlSZKkLdt86dnVPLVihTfyGYXHbXges9F43IbnMRuNx214HrP5IVUL48Y5\nSZYBq1evXu1gcUmSpHlozZo1LF++HGB5Va0Zx2vasytJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrL\nsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJ\nkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTO\nMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxK\nkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSp\nswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7\nkiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ\n6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izD\nriRJkjpr3oTdJM9PcmaSdUlOTnLvTbQ/MMmpSS5Lck6SDye58eaqV5IkSfPfvAi7SZ4CHAYcCtwT\nOBU4Ickuk7R/MPAR4APAnYEnAvdpn0uSJEnAPAm7wErgqKr6WFX9EngucDnwrEna3ws4s6reW1Vn\nV9X3gaNoAq8kSZIEzIOwm2RrYDlw0sS6qirgROC+k+x2IrA0ySPb19gVeBLwP7NbrSRJkrYkcx52\ngV2AxcC5fevPBZYO2qGqTgUOBD6d5ErgT8BFwAtmsU5JkiRtYeZD2B1akr2AY4B/B5YB+wG3oRnK\nIEmSJAGw1VwXAPwFuAbYtW/9rsCfJ9nnEOCEqjq8ff6/SZ4HfCfJq6qqv5f4WitXrmTJkiUbrVux\nYgUrVqwYqXhJkiQNb9WqVaxatWqjdWvXrh37+6QZHju3kpwM/LCqXtw+D/A74F1V9bYB7Y8Frq6q\np/Wsuy/wXeBvqup6ITnJMmD16tWrWbZs2Sx9EkmSJI1qzZo1LF++HGB5Va0Zx2vOl2EMhwP/1M6d\neyfg/cD2NEMVSPKmJB/taf954AlJnpvkNknuD7yTJjBP1hssSZKkBWY+DGOgqo5t59R9Hc3whVOA\n/arq/LbJUmC3nvafTHJD4PnA24GLaWZzeMVmLVySJEnz2rwIuwBVdSRw5CTbDhqw7v00PcCSJEnS\nQPNlGIMkSZI0doZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJn\nGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYl\nSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLU\nWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZd\nSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIk\ndZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZhV5IkSZ1l2JUkSVJnGXYlSZLUWYZdSZIkdZZh\nV5IkSZ1l2JUkSVJnDR12k3wryYFJbjAbBUmSJEnjMkrP7k+BtwN/TvLBJHuNuSZJkiRpLIYOu1V1\nCHAL4CDgZsC3k/wiyb8k2XXcBUqSJEmjGmnMblVdXVWfraoDgFsCnwReD/w+yeeTPGycRUqSJEmj\nmNEFaknuA7wWeClwHvAm4C/A8UnePvPyJEmSpNFtNewOSW4G/D3NMIY7AF8EVgAnVFW1bY4BvgL8\ny9gqlSRJkoY0dNgF/gCcDnwEOKaqzh/Q5mfAj2dSmCRJkjRTo4TdvavqO1M1qKpLgIeOVpIkSZI0\nHqOM2X1tkhv1r0xywyRfH0NNkiRJ0liMEnYfDGwzYP12wANnVo4kSZI0PtMexpDk7hMPgTsnWdqz\neTHwCOCPY6xNkiRJmpFhxuyeAlS7DBqusA544TiKkiRJksZhmLB7G5pe3TOA+wC9szBcCZxXVdeM\nsTZJkiRpRqYddqvq7PbhjG5EIUmSJG0u0wq7SfYHvlxVV7WPJ1VVx42lMkmSJGmGptuz+3lgKc0t\ngT8/RbuiuVhtaEmeT3PHtaXAqcALq2rSG1Mk2QY4FHh6u885wOuq6phR3l+SJEndM62wW1WLBj0e\nlyRPAQ4DngP8CFgJnJDkb6vqL5Ps9mngpjS3LT4duDkOsZAkSVKPUe6gNhtWAkdV1ccAkjwXeDTw\nLOCt/Y2TPIJmTt/bVtXF7erfbaZaJUmStIWY7pjdF033BavqXcMUkGRrYDnwxp7XqCQnAvedZLfH\nAD8BXp7k74HLgOOAV1fV+mHeX5IkSd013Z7dldNsV8BQYRfYhWac77l9688F7jjJPrel6dldDzy2\nfY33ATcGnj3k+0uSJKmjpjtm9zazXciQFgEbgKdV1V8BkrwE+HSS51XVFZPtuHLlSpYsWbLRuhUr\nVrBixYrZrFeSJEk9Vq1axapVqzZat3bt2rG/T6pq7C86VAHNMIbLgSf0TluW5BhgSVU9bsA+xwD3\nq6q/7Vl3J+D/gL+tqtMH7LMMWL169WqWLVs29s8hSZKkmVmzZg3Lly8HWF5Va8bxmtMds3s4zXjY\ny9rHk6qqlwxTQDt372pgb5pxtyRJ+3yyIRHfA56YZPuqurxdd0ea3t4/DPP+kiRJ6q7pjtm9J7B1\nz+PJjNpNfDhwTBt6J6Ye2x44BiDJm4BbVNUz2/afBP4NODrJa2imIHsr8OGphjBIkiRpYZnumN2H\nDno8LlV1bJJdgNcBuwKnAPtV1fltk6XAbj3tL0uyD/Bu4MfABcB/Aa8ed22SJEnacs1ont0kuwFU\n1e9nWkhVHQkcOcm2gwas+zWw30zfV5IkSd019B3HkmyV5PVJ1gJnAWclWZvkP9qLzSRJkqR5YZSe\n3XcDjwdeBvygXXdf4DXATYCDx1KZJEmSNEOjhN2nAU+tqi/3rPtZkt8DqzDsSpIkaZ4YehgDcAXN\n8IV+ZwJXzqgaSZIkaYxGCbvvAV6dZNuJFe3jV7XbJEmSpHlhujeV+GzfqocDf0hyavt8T2Ab4KQx\n1iZJkiTNyHTH7PbfqPi/+57PeOoxSZIkadyme1OJ681zK0mSJM13o4zZlSRJkrYII91BLckTgScD\nt6IZq3utqlo2hrokSZKkGRvlDmovAo4GzgXuCfwIuAC4LfDlKXaVJEmSNqtRhjE8D3hOVb2QZl7d\nt1bVPsC7gCXjLE6SJEmaiVHC7q2A77eP1wE7tY//E1gxjqIkSZKkcRgl7P4ZuHH7+HfAXu3j2wAZ\nR1GSJEnSOIwSdr8O7N8+Pho4IsnXgP8CPjeuwiRJkqSZGmU2hufQhuSqem+SC4D7AccBR42xNkmS\nJGlGhg67VbUB2NDz/FPAp8ZZlCRJkjQOo86zuzPwbGCPdtUvgKOr6sJxFSZJkiTN1Cjz7D4IOBN4\nEbBzu7wIOLPdJkmSJM0Lo/Tsvhc4Fji4qq4BSLIYOLLddrfxlSdJkiSNbpTZGG4PHDYRdAHax4e3\n2yRJkqR5YZSwu4brxur22gM4dWblSJIkSeMzrWEMSe7e8/RdwDuT3B44uV23F/B84BXjLU+SJEka\n3XTH7J4CFBvfIe2tA9p9kubmEpIkSdKcm27Yvc2sViFJkiTNgmmF3ao6e7YLkSRJksZt1JtK3A44\nhI1vKvHOqjp9XIVJkiRJMzXKTSX2owm39wF+1i5/B/xfkn3GW54kSZI0ulF6dt8MHFFVG828kOTN\nwFuAr42jMEmSJGmmRplndw/gwwPWfwS488zKkSRJksZnlLB7PnCPAevvAZw3s3IkSZKk8RllGMMH\ngQ8kuS3w/Xbd/YGX09wyWJIkSZoXRgm7rwcuBV4KvKlddw7wGpq7q0mSJEnzwlBhN0mA3YD3VdUR\nSXYCqKpLZ6M4SZIkaSaGHbMb4Lc0gZequtSgK0mSpPlqqLBbVRuA3wA3mZ1yJEmSpPEZZTaGVwBv\nS3LXcRcjSZIkjdMoF6h9DNgeODXJlcC63o1VdeNxFCZJkiTN1Chh95CxVyFJkiTNgqHDblV9dDYK\nkSRJksZtlJ5dkiwGHkdz62CAXwBfqKqrx1WYJEmSNFNDh90kdwGOA5YCv2pXvxw4P8ljqup/x1if\nJEmSNLJRZmP4EPB/wC2rallVLaOZd/dnwAfGWZwkSZI0E6MMY7gHcK+qumhiRVVdlORVwI/HVpkk\nSZI0Q6P07P4a2HXA+pvR3F1NkiRJmhdGCbuvBN6V5IlJbtkuTwTeAbw8yQ0nlvGWKkmSJA1nlGEM\nx7c/jwWqfZz25xd7nhewePTSJEmSpJkZJew+dOxVSJIkSbNglJtKfGs2CpEkSZLGbZQxu5IkSdIW\nwbArSZKkzjLsSpIkqbMMu5IkSeosw64kSZI6a1qzMST5KdfNqTulqlo2o4okSZKkMZnu1GOfn9Uq\nJEmSpFkwrbBbVa+d7UIkSZKkcXPMriRJkjprumN2L2L6Y3ZvPKOKJEmSpDGZ7pjdQ2a1CkmSJGkW\nTHfM7kdnuxBJkiRp3KbbsztQku2AbXrXVdUlM6pIkiRJGpOhL1BLskOS9yQ5D7gMuKhvkSRJkuaF\nUWZjeCvwMOBg4ArgH4FDgXOAA8dXmiRJkjQzowxjeAxwYFV9M8nRwHeq6rdJzgaeDnxirBVKkiRJ\nIxqlZ/fGwBnt40va5wDfBR40jqIkSZKkcRgl7J4B3KZ9/Evgye3jxwAXj1pIkucnOTPJuiQnJ7n3\nNPe7f5KrkqwZ9b0lSZLUTaOE3aOBPdvHbwaen2Q9cATwtlGKSPIU4DCasb/3BE4FTkiyyyb2WwJ8\nFDhxlPeVJElStw09Zreqjuh5fGKSOwHLgd9W1c9GrGMlcFRVfQwgyXOBRwPPorkgbjLvpxkjvAE4\nYMT3liRJUkeN0rO7kao6u6o+O2rQTbI1TVg+qec1i6a39r5T7HcQzXCK147yvpIkSeq+UebZfVeS\nFwxY/4Ik7xihhl2AxcC5fevPBZZOUsMdgDcCT6+qDSO8pyRJkhaAUXp2n0Az80K/7wNPnFk5m5Zk\nEc3QhUOr6vSJ1bP9vpIkSdryjDLP7k2ASwesv4Sml3ZYfwGuAXbtW78r8OcB7XcC7gXcI8l723WL\ngCS5Eti3qr452ZutXLmSJUuWbLRuxYoVrFixYoTSJUmSNIpVq1axatWqjdatXbt27O+TZnjsEDsk\n/wu8v6re07f+hcDBVXXnoYtITgZ+WFUvbp8H+B3wrqp6W1/bAHv0vcTzgYfS9DqfVVXrBrzHMmD1\n6tWrWbZs2bAlSpIkaZatWbOG5cuXAyyvqrFMKztKz+7hwHuS3BT4ertub+ClwCEj1nE4cEyS1cCP\naGZn2B44BiDJm4BbVNUz24vXftG7c5LzgPVVddqI7y9JkqQOGmXqsY8k2RZ4FfDqdvVZNL26Hxul\niKo6tp1T93U0wxdOAfarqvPbJkuB3UZ5bUmSJC1co/TsUlXvA97X9u6uq6q/zrSQqjoSOHKSbQdt\nYt/X4hRkkiRJ6jPSPLtJtkrycODxtDMhJLlFkh3HWZwkSZI0E0P37Ca5NfAV4FbAtsDXaGZneHn7\n/LnjLFCSJEka1Sg9u+8EfgLsDPTOevA5mgvVJEmSpHlhlDG7DwTuV1VXNrOAXess4G/GUZQkSZI0\nDqP07C6iub1vv1sy+GYTkiRJ0pwYJex+lY3n0632wrTXAl8aS1WSJEnSGIwyjOGlwAlJfgFsB3wS\nuAPNbX/9p6SsAAAVRUlEQVS9564kSZLmjVFuKvGHJHsCTwH2BHYEPgx8YtBteiVJkqS5MupNJa4G\nPtEu10qyQ1VdNo7CJEmSpJka6aYS/ZIsTfJm4PfjeD1JkiRpHKYddpPslOSDSX6W5OgkN0hy0yRH\nAWcCTwfeMmuVSpIkSUMaZhjDW4An0dwxbT/gfcDDgQuBg4BPV9U1Y69QkiRJGtEwYff/Aw6squOS\n3BX4GXAM8OyqqtkoTpIkSZqJYcbsLgVWA1TV/wJXAEcYdCVJkjRfDRN2FwNX9jy/GvjreMuRJEmS\nxmeYYQwBjklyRft8O+D9STaaaqyqHj+u4iRJkqSZGCbsfrTv+cfHWYgkSZI0btMOu1V10GwWIkmS\nJI3bWG4qIUmSJM1Hhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJ\nUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZ\ndiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJ\nktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZ\nhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1J\nkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZhl1JkiR1lmFXkiRJnWXYlSRJUmcZdiVJktRZ8ybsJnl+\nkjOTrEtycpJ7T9H2cUm+muS8JGuTfD/JvpuzXkmSJM1/8yLsJnkKcBhwKHBP4FTghCS7TLLLg4Cv\nAo8ElgHfAL6YZM/NUK4kSZK2EPMi7AIrgaOq6mNV9UvgucDlwLMGNa6qlVX19qpaXVWnV9WrgN8A\nj9l8JUuSJGm+m/Owm2RrYDlw0sS6qirgROC+03yNADsBF85GjZIkSdoyzXnYBXYBFgPn9q0/F1g6\nzdf4V2AH4Ngx1iVJkqQt3FZzXcBMJXka8Gpg/6r6y6bar1y5kiVLlmy0bsWKFaxYsWKWKpQkSVK/\nVatWsWrVqo3WrV27duzvk2bEwNxphzFcDjyhqo7rWX8MsKSqHjfFvk8FPgQ8saq+son3WQasXr16\nNcuWLRtL7ZIkSRqfNWvWsHz5coDlVbVmHK8558MYquoqYDWw98S6dgzu3sD3J9svyQrgw8BTNxV0\nJUmStDDNl2EMhwPHJFkN/IhmdobtgWMAkrwJuEVVPbN9/rR224uAHyfZtX2ddVV1yeYtXZIkSfPV\nvAi7VXVsO6fu64BdgVOA/arq/LbJUmC3nl3+ieaitve2y4SPMsl0ZZIkSVp45kXYBaiqI4EjJ9l2\nUN/zh26WoiRJkrRFm/Mxu5IkSdJsMexKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7\nkiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ\n6izDriRJkjrLsCtJkqTOMuxKkiSpswy7kiRJ6qyt5rqAze2AA2CnnWDrrbeMZfFiSOb6qEmSJG2Z\nFlzYfdjD4CY3gauumnxZvx4uvXTqNpMt11wz/prnOnCPa9lqK4O7JEnavBZc2H3xi2HZstl7/Q0b\n4OqrRwvKs7FceSVcdtlo+1599fiPz1ZbzY/QPY7XWeQgIEmS5r0FF3Zn26JFsM02zbKlq5pfwf2q\nq2DdutH3HbdFiwaH4G22gW23he22u27pfz7Tdf3rt/I3WZKkgfwnUpNKrgtwW7qqZojJOIP3ZP8R\nuPJKuOKKZlm/fuPlgguuv279+o3bjhLMFy+enVA9naA98Xzx4vH/uUmauaqN/34a5vHE86uvhh13\nhJ13hhvdqPm5886w/fYOT9P8Z9jVgpA0vZ9bbQU3uMFcVzO1a66ZPCwPWjfZ+snWrV07ddt165rh\nOMPaaqvZD9VTrdt2W4eWaHZs2DB1EBzH49l8rdkYkjZhq602DsDD/LzRjfxWSpuHp5k0zyxe3PSW\nbL/93NVw9dXDBehhAvjll8OFF256/6rh695mm7kJ2hOPt9nGXq5BNmyYv0FwOo/HeeHxxDdm22yz\n8dCnTT3eeuvmHJuYTWjY/ftfa9h9Fi9urv+46KJmufjiyX+efz785jfXPb/44sl/n/t7izf1s/ex\nvcqaLsOupOuZ6AXfYYe5ef+J8eKzEbbXr4dLLoHzztv0/qPY3EH7mmvmf+/iKN8UTKZ3rPywgW37\n7ccX/kYNklvycJ8dd4Rddx1+vw0bmhmONhWSJ37+5jcbP1+3bvDr2qus6fKPWtK80ztefKed5qaG\nqiaszUbYXr/+ut7tqfa/8srxf67Fi0cPbDvssPl7FPv335LD4kK1aBEsWdIso1i//roe4rnsVe79\nucMO9ipvSQy7kjRAct1Y4FH/kZ6pDRs2DsP9wfiKK6YXXnsfO65ZW5rttoOlS5tlWKP0KveuW79+\n8Ovaq7xl8XBL0jy1aFFzQeV8v6hSmq/sVRYYdiVJkgayV7kbFuBHliRJml32Ks8fhl1JkqR5ZnP1\nKl900XC9ylNNBzdfe5UNu5IkSR2yJfcqX3zx6J97MoZdSZIkXWu2e5V7H0+3V3kmDLuSJEkai5n2\nKv/wh7DXXmOuabwvJ0mSJI1m663H/5qGXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWW\nYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeS\nJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeSJEmd\nZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHXWvAm7SZ6f5Mwk\n65KcnOTem2j/kCSrk6xP8uskz9xctS4kq1atmusStkget+F5zEbjcRuex2w0Hrfheczmh3kRdpM8\nBTgMOBS4J3AqcEKSXSZpvztwPHASsCfwTuBDSfbZHPUuJP6ijsbjNjyP2Wg8bsPzmI3G4zY8j9n8\nMC/CLrASOKqqPlZVvwSeC1wOPGuS9gcDZ1TVy6rqV1X1XuAz7etIkiRJwDwIu0m2BpbT9NICUFUF\nnAjcd5Ld9mq39zphivaSJElagOY87AK7AIuBc/vWnwssnWSfpZO0v2GSbcdbniRJkrZUW811AZvR\ndgCnnXbaXNexRVm7di1r1qyZ6zK2OB634XnMRuNxG57HbDQet+F5zIbXk9O2G9drphkxMHfaYQyX\nA0+oquN61h8DLKmqxw3Y51vA6qp6Sc+6fwCOqKqdJ3mfpwGfGG/1kiRJmgVPr6pPjuOF5rxnt6qu\nSrIa2Bs4DiBJ2ufvmmS3HwCP7Fu3b7t+MicATwfOAtbPoGRJkiTNju2A3Wly21jMec8uQJInA8fQ\nzMLwI5pZFZ4I3Kmqzk/yJuAWVfXMtv3uwM+BI4GP0ATjdwCPqqr+C9ckSZK0QM15zy5AVR3bzqn7\nOmBX4BRgv6o6v22yFNitp/1ZSR4NHAG8CPgD8GyDriRJknrNi55dSZIkaTbMh6nHJEmSpFlh2JUk\nSVJndSbsJnl+kjOTrEtycpJ7b6L9Q5KsTrI+ya+TPHNz1TqfDHPckjw4yYa+5ZokN9ucNc+lJA9M\nclySP7aff/9p7LPgz7Vhj5vnGiR5ZZIfJbkkyblJPpfkb6ex34I930Y5Zp5rkOS5SU5NsrZdvp/k\nEZvYZ8GeZzD8MfM8u74kr2iPw+GbaDfjc60TYTfJU4DDgEOBewKnAie0F70Nar87cDzNLYr3BN4J\nfCjJPpuj3vli2OPWKuAONBcNLgVuXlXnzXat88gONBdQPo/mWEzJc+1aQx231kI/1x4IvBv4O+Dh\nwNbAV5PcYLIdPN+GP2athX6u/R54ObAMWA58HTguyZ0HNfY8A4Y8Zq2Ffp5dq+1Yew5N7piq3e6M\n41yrqi1+AU4G3tnzPDQzNLxskvZvAX7Wt24V8KW5/izz/Lg9GLgGuOFc1z4fFmADsP8m2niujXbc\nPNeuf0x2aY/dA6Zo4/k2/DHzXBt8XC4ADppkm+fZ8MfM8+y6Y7Ej8CvgYcA3gMOnaDuWc22L79lN\ncwe25TSpH4BqjsaJwH0n2W2vdnuvE6Zo3zkjHjdoAvEpSc5J8tUk95vdSrd4C/5cmwHPtY3diKZn\n6MIp2ni+bWw6xww8166VZFGSpwLbAt+ZpJnnWY9pHjPwPJvwXuCLVfX1abQdy7m2xYddmv+5LwbO\n7Vt/Ls3XBIMsnaT9DZNsO97y5q1RjtufgH8GngA8nuZrnG8mucdsFdkBnmuj8VzrkSQ0N875blX9\nYoqmnm+tIY6Z5xqQ5K5JLgWuAI4CnlxVv52kuecZQx8zzzOg/U/BPYBXTnOXsZxr8+KmEtoyVNWv\ngV/3rDo5ye1o7ni3oC5O0OzyXLueI4E7A/ef60K2INM6Zp5r1/olzZjIJTR3MP1UkgdX1U/ntqx5\nbdrHzPMMktyS5j+gD6+qqzbne3ehZ/cvNONgdu1bvyvw50n2+fMk7S+pqivGW968NcpxG+RHwO3H\nVVQHea6Nz4I815K8B3gU8JCq+tMmmnu+MfQxG2TBnWtVdXVVnVFVP62qVwE/BA6epLnnGUMfs0EW\n2nm2HLgpsCbJVUmuohnL/OIkV7bfxvQby7m2xYfd9n8Hq4G9J9a1B2xv4PuT7PaD3vatfdv1C8KI\nx22Qe9B8PaPBFvy5NkYL7lxrQ9sBwEOr6nfT2GXBn28jHLNBFty5NsAimqFugyz482wSUx2zQRba\neXYicDeaz71nu/wE+DiwZ3vdUL/xnGtzfVXemK7sezJwOXAgcCeasTMXADdtt78J+GhP+92BS2mu\n8rsjzXRIV9J0rc/555nHx+3FwP7A7YC70HwdcRVN78mcf57NdMx2aH9B70Fzlfch7fPdPNfGetw8\n15qv4S+imU5r155lu542b/R8m/Ex81xrjskDgVsDd21/H6+i+Q+Df6+N55gt+PNskuO40WwMs/V3\nWifG7FbVse3csK+j+YvtFGC/qjq/bbIU2K2n/VlJHg0cAbyIZrqtZ1dV/xV/nTbscQO2oZmX9xY0\nIflnwN5V9e3NV/WcuxfNL2e1y2Ht+o8Cz8JzbTJDHTc81wCeS3Osvtm3/iDgY+3jm+P51mvoY4bn\nGsDNaH4Xbw6spTkG+1XVN9rt/r12fUMdMzzPJtPfmzsrf6elTc6SJElS52zxY3YlSZKkyRh2JUmS\n1FmGXUmSJHWWYVeSJEmdZdiVJElSZxl2JUmS1FmGXUmSJHWWYVeSJEmdZdiVJElSZxl2JWkWJPlG\nksOHaP/gJBuS3HA269pEDYcmWbOJNrdu67z75qpLkmbC2wVLWrCS7AK8HngUsCtwEXAK8Lqq+sEM\nX/tGwFVVddk02z8Y+Dqwc1VdMpP3HlWS7YFtq+qi9vnRwJKqenxPmwA3Bf5SVRvmok5JGsZWc12A\nJM2hz9L8Pfj3wJk0gXdv4CajvmCSravqqqq6eDwlbj5VdTlw+SbaFHDe5qlIkmbOYQySFqQkS4AH\nAC+vqm9X1e+r6idV9ZaqOr63XZIPJTkvydokJ/Z+hd9+9f/TJM9Ocgawrl3/zd5hDEmekeTHSS5J\n8qckn0hy0yFr3pDkuUm+lOTyJKcneUJfm7smOand/pckRyXZoWf7Q5L8MMlfk1yU5DtJduv9LBOP\ngWcCB7Tve02SBw0axtAOwfhhkvVJzknypiSLerZ/I8k7k7wlyQXt5z90mM8uSaMy7EpaqP7aLo9N\nss0U7T5D09O7H7AMWAOc2A5TmHB74PHA44B7tOv6x4htBfwbcHfgAODWwNEj1P064NPt63wC+FSS\nO8K1wxBOAC4AlgNPBB4OvLvdvhj4HPAN4K7AXsAH+mqdePx24FjgKzQ93jcHvt//2ZLcAvgf4Idt\nTc8Fnt1+1l4H0hzv+wAvA/49yd4jfH5JGorDGCQtSFV1TZJnAh8EDm4vzPoW8Kmq+jlAkvsD9wJu\nVlVXtbu+LMnjaILkh9p1WwN/X1UXTvF+x/Q8PSvJIcAPk2zfDh+YrmOraiIk/3uSfYAXAi8Ang5s\nCxxYVeuB05K8APhikpcDVwM3BP6nqs5qX+NXk9R7WZJ1wDZVdf7E+mbILulp+nzgd1X1ovb5r9te\n2zfTBPMJP6uq17ePT2/r2hs4aYjPLklDs2dX0oJVVZ8DbgE8Bvgy8GBgTZID2yZ7AjsBFya5dGIB\ndgdu1/NSZ08VdAGSLE9yXJKzk1wCfLPddKshyz657/kPgD3ax3cCTm2D7oTv0fxdf8f2wrOPAl9t\na3lRkqVDvn+/O7U19PoesGOSW/as+1lfmz8BN5vhe0vSJhl2JS1oVXVlVZ1UVW+oqgcAxwCvbTfv\nCJxD8/X8nj3LHYG39bzMlDMutMMLvgJcDDyNprf4ce3mqYZQjF1VPYtm+ML3gKfQ9MTeZzO89VV9\nzwv/DZK0GfgXjSRt7DRg4oKuNcBS4JqqOqNvmbInt8+dgBsDr6yq71XVr2nGwY5irwHPT+upfc8k\nN+jZ/gDgGnqGK1TVqe2FePcH/pcmgA9yJbB4E/WcBty3b90DgEur6g+b2FeSZp1hV9KClOTG7awF\nT09ytyS7J3kS8K/A5wGq6kSar+g/n2SfdiaC+yX5jyTLhni739EExxcluU2S/bn+BVyw8VjYyTwp\nyUFJ7pDktcC9gfe02z4BrAc+muQuSR4KvAv4WFWd337GNybZK8mtkuwL3AH4xSTvdRZw9yR/m+Qm\nSQZd53EksFuSdye5Y5IDgNcAh03js0jSrPMCNUkL1V9pxr8eQjP+dmvg98BRwJt62j0KeAPwEZqb\nKfwZ+DZw7iZe/9oZC6rqL0n+AXgjzcVka4CXAsdNts8UDgWeCryXZtzrU6vql+37rEuyH/BO4Ec0\nc+Z+pn0v2ud3opkZ4Sbt/u+uqg9M8l4fpBnH/BOa3u6HAmf3fbZzkjyKZljHKcCF7X5vGPJzSdKs\n8A5qkrSFSLIBeGxV9YdkSdIkHMYgSZKkzjLsStKWw6/iJGlIDmOQJElSZ9mzK0mSpM4y7EqSJKmz\nDLuSJEnqLMOuJEmSOsuwK0mSpM4y7EqSJKmzDLuSJEnqLMOuJEmSOuv/B9qjCHN5E8faAAAAAElF\nTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def plot_recall(probs, pos):\n",
" # print \"Is normalized: \", np.abs(1-np.sum(position_probs)) < 1e-6\n",
" plt.figure(pos, figsize=[8,5])\n",
" plt.axis((0,4,0.0,1.0))\n",
" plt.title(\"DNC free recall (position={})\".format(pos))\n",
" plt.xlabel(\"Serial position\") ; plt.ylabel(\"Recall probability\")\n",
" plt.plot(range(len(probs)), probs) ; plt.show()\n",
"\n",
"for pos in range(1):\n",
" position_probs = [np.sum(i == recall_orders[:,pos])/float(trials) for i in range(FLAGS.length)]\n",
" plot_recall(position_probs, pos=pos)\n",
" \n",
"p0_probs = np.asarray([np.sum(i == recall_orders[:,0])/float(trials) for i in range(FLAGS.length)])\n",
"p1_probs = np.asarray([np.sum(i == recall_orders[:,1])/float(trials) for i in range(FLAGS.length)])\n",
"plot_recall(.5*(p0_probs + p1_probs), pos=12)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# # X, y = next_batch(FLAGS.batch_size, FLAGS.length, FLAGS.xlen)\n",
"# # y = np.stack(y)\n",
"# # y = y[:,FLAGS.length + 3:,:]\n",
"# # # p = np.random.permutation(FLAGS.length)\n",
"# # y_shuffle = y #y[:,p,:] ; print 'permute by: ', p\n",
"# # # y_shuffle[:,-1,-1] = 4\n",
"\n",
"# # plt.figure(0, figsize=[1,1])\n",
"# # plt.imshow(y[0,:,:].T, interpolation='none')\n",
"# # plt.figure(1, figsize=[1,1])\n",
"# # plt.imshow(y_shuffle[0,:,:].T, interpolation='none')\n",
"# # plt.show()\n",
"\n",
"# def guess_recall_order(real_y, pred_y, FLAGS):\n",
"# # sorry this is uuuuugly but we have to because it's batched\n",
"# real_y = np.tile(real_y, [1,1,1,1]) ; real_y = np.transpose(real_y, (1,0,2,3))\n",
"# pred_y = np.tile(pred_y,[1,1,1,1]) ; pred_y = np.transpose(pred_y, (1,0,2,3))\n",
" \n",
"# pred_y = np.tile(pred_y,[1,FLAGS.length,1,1])\n",
"# pred_y = np.transpose(pred_y, (0,2,1,3))\n",
" \n",
"# # real_y = real_y[0,:,:,:] ; pred_y = pred_y[0,:,:,:]\n",
"# y_minus = .5*(real_y - pred_y)**2\n",
"# y_minus = np.sum(y_minus, axis=-1)\n",
"# y_mins = np.amin(y_minus, axis=1)\n",
" \n",
"# k, l = np.where(y_minus == y_mins)\n",
"# i = 0\n",
"# while i < len(k): \n",
"# if k[-i] == k[-i-1]:\n",
"# k = np.delete(k,-i) ; l = np.delete(l,-i)\n",
"# i+=1\n",
"# return l\n",
"\n",
"# X, real_y = next_batch(FLAGS.batch_size, FLAGS.length, FLAGS.xlen)\n",
"# real_y = np.stack(real_y)[:,-FLAGS.length:,:]\n",
"\n",
"# p = np.random.permutation(y_i.shape[0]) ; print 'permute by: ', p\n",
"# pred_y = [y_i[p,:] for y_i in real_y]\n",
"# pred_y = np.stack(pred_y)\n",
"# # pred_y[:,-2:,-1] = 4\n",
"\n",
"# plt.figure(0, figsize=[1,1])\n",
"# plt.imshow(real_y[0,:,:].T, interpolation='none')\n",
"# plt.figure(1, figsize=[1,1])\n",
"# plt.imshow(pred_y[0,:,:].T, interpolation='none')\n",
"# plt.show()\n",
"\n",
"# np_loss(real_y, pred_y, FLAGS)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def scatter(idx, vals, target):\n",
" \"\"\"target[idx] += vals, but allowing for repeats in idx\"\"\"\n",
" np.add.at(target, idx.ravel(), vals.ravel())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 1
}