{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Train a ConvNet!\n", "We now have a generic solver and a bunch of modularized layers. It's time to put it all together, and train a ConvNet to recognize the classes in CIFAR-10. In this notebook we will walk you through training a simple two-layer ConvNet and then set you free to build the best net that you can to perform well on CIFAR-10.\n", "\n", "Open up the file `cs231n/classifiers/convnet.py`; you will see that the `two_layer_convnet` function computes the loss and gradients for a two-layer ConvNet. Note that this function uses the \"sandwich\" layers defined in `cs231n/layer_utils.py`. " ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# As usual, a bit of setup\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from cs231n.classifier_trainer import ClassifierTrainer\n", "from cs231n.gradient_check import eval_numerical_gradient\n", "\n", "%matplotlib inline\n", "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n", "plt.rcParams['image.interpolation'] = 'nearest'\n", "plt.rcParams['image.cmap'] = 'gray'\n", "\n", "# for auto-reloading external modules\n", "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n", "%load_ext autoreload\n", "%autoreload 2\n", "\n", "def rel_error(x, y):\n", " \"\"\" returns relative error \"\"\"\n", " return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from cs231n.layers import (\n", " affine_forward, affine_backward,\n", " softmax_loss\n", ")\n", "\n", "from cs231n.layer_utils import (\n", " conv_relu_pool_forward, conv_relu_pool_backward,\n", ")\n", "\n", "def two_layer_convnet(X, model, y=None, reg=0.0):\n", " \"\"\"Compute the loss and gradient for a simple two-layer ConvNet.\n", "\n", " The architecture is conv-relu-pool-affine-softmax, where the conv layer\n", " uses stride-1 \"same\" convolutions to preserve the input size;\n", " the pool layer uses non-overlapping 2x2 pooling regions.\n", " We use L2 regularization on both the convolutional layer weights and the\n", " affine layer weights.\n", "\n", " Inputs:\n", " - X: Input data, of shape (N, C, H, W)\n", " - model: Dictionary mapping parameter names to parameters.\n", " A two-layer Convnet expects the model to have the following parameters:\n", " - W1, b1: Weights and biases for the convolutional layer\n", " - W2, b2: Weights and biases for the affine layer\n", " - y: Vector of labels of shape (N,).\n", " y[i] gives the label for the point X[i].\n", " - reg: Regularization strength.\n", "\n", " Returns:\n", " If y is None, then returns:\n", " - scores: Matrix of scores, where scores[i, c] is the classification score\n", " for the ith input and class c.\n", "\n", " If y is not None, then returns a tuple of:\n", " - loss: Scalar value giving the loss.\n", " - grads: Dictionary with the same keys as model, mapping parameter names to\n", " their gradients.\n", " \"\"\"\n", "\n", " # Unpack weights\n", " W1, b1, W2, b2 = model['W1'], model['b1'], model['W2'], model['b2']\n", " N, C, H, W = X.shape\n", "\n", " # We assume that the convolution is \"same\", so that the data has the same\n", " # height and width after performing the convolution. We can then use the\n", " # size of the filter to figure out the padding.\n", " conv_filter_height, conv_filter_width = W1.shape[2:]\n", " assert conv_filter_height == conv_filter_width, (\n", " 'Conv filter must be square'\n", " )\n", " assert conv_filter_height % 2 == 1, 'Conv filter height must be odd'\n", " assert conv_filter_width % 2 == 1, 'Conv filter width must be odd'\n", " conv_param = {'stride': 1, 'pad': (conv_filter_height - 1) / 2}\n", " pool_param = {'pool_height': 2, 'pool_width': 2, 'stride': 2}\n", "\n", " # Compute the forward pass\n", " a1, cache1 = conv_relu_pool_forward(X, W1, b1, conv_param, pool_param)\n", " scores, cache2 = affine_forward(a1, W2, b2)\n", "\n", " if y is None:\n", " return scores\n", "\n", " # Compute the backward pass\n", " data_loss, dscores = softmax_loss(scores, y)\n", "\n", " # Compute the gradients using a backward pass\n", " da1, dW2, db2 = affine_backward(dscores, cache2)\n", " dX, dW1, db1 = conv_relu_pool_backward(da1, cache1)\n", "\n", " # Add regularization\n", " dW1 += reg * W1\n", " dW2 += reg * W2\n", " reg_loss = 0.5 * reg * sum(np.sum(W * W) for W in [W1, W2])\n", "\n", " loss = data_loss + reg_loss\n", " grads = {'W1': dW1, 'b1': db1, 'W2': dW2, 'b2': db2}\n", "\n", " return loss, grads\n", "\n", "\n", "def init_two_layer_convnet(\n", " weight_scale=1e-3, bias_scale=0, input_shape=(3, 32, 32),\n", " num_classes=10, num_filters=32, filter_size=5\n", "):\n", " \"\"\"\n", " Initialize the weights for a two-layer ConvNet.\n", "\n", " Inputs:\n", " - weight_scale: Scale at which weights are initialized. Default 1e-3.\n", " - bias_scale: Scale at which biases are initialized. Default is 0.\n", " - input_shape: Tuple giving the input shape to the network; default is\n", " (3, 32, 32) for CIFAR-10.\n", " - num_classes: The number of classes for this network. Default is 10\n", " (for CIFAR-10)\n", " - num_filters: The number of filters to use in the convolutional layer.\n", " - filter_size: The width and height for convolutional filters. We assume\n", " that all convolutions are \"same\", so we pick padding to ensure that data\n", " has the same height and width after convolution. This means that the\n", " filter size must be odd.\n", "\n", " Returns:\n", " A dictionary mapping parameter names to numpy arrays containing:\n", " - W1, b1: Weights and biases for the convolutional layer\n", " - W2, b2: Weights and biases for the fully-connected layer.\n", " \"\"\"\n", " C, H, W = input_shape\n", " assert filter_size % 2 == 1, (\n", " 'Filter size must be odd; got %d' % filter_size\n", " )\n", "\n", " model = {}\n", " randn = np.random.randn\n", " model['W1'] = weight_scale * randn(\n", " num_filters, C, filter_size, filter_size\n", " )\n", " model['b1'] = bias_scale * randn(num_filters)\n", " model['W2'] = weight_scale * randn(num_filters * H * W / 4, num_classes)\n", " model['b2'] = bias_scale * randn(num_classes)\n", " return model\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train data shape: (49000, 3, 32, 32)\n", "Train labels shape: (49000,)\n", "Validation data shape: (1000, 3, 32, 32)\n", "Validation labels shape: (1000,)\n", "Test data shape: (1000, 32, 32, 3)\n", "Test labels shape: (1000,)\n" ] } ], "source": [ "from cs231n.data_utils import load_CIFAR10\n", "\n", "def get_CIFAR10_data(num_training=49000, num_validation=1000, num_test=1000):\n", " \"\"\"\n", " Load the CIFAR-10 dataset from disk and perform preprocessing to prepare\n", " it for the two-layer neural net classifier. These are the same steps as\n", " we used for the SVM, but condensed to a single function. \n", " \"\"\"\n", " # Load the raw CIFAR-10 data\n", " cifar10_dir = 'cs231n/datasets/cifar-10-batches-py'\n", " X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir)\n", " \n", " # Subsample the data\n", " mask = range(num_training, num_training + num_validation)\n", " X_val = X_train[mask]\n", " y_val = y_train[mask]\n", " mask = range(num_training)\n", " X_train = X_train[mask]\n", " y_train = y_train[mask]\n", " mask = range(num_test)\n", " X_test = X_test[mask]\n", " y_test = y_test[mask]\n", "\n", " # Normalize the data: subtract the mean image\n", " mean_image = np.mean(X_train, axis=0)\n", " X_train -= mean_image\n", " X_val -= mean_image\n", " X_test -= mean_image\n", " \n", " # Transpose so that channels come first\n", " X_train = X_train.transpose(0, 3, 1, 2).copy()\n", " X_val = X_val.transpose(0, 3, 1, 2).copy()\n", " x_test = X_test.transpose(0, 3, 1, 2).copy()\n", "\n", " return X_train, y_train, X_val, y_val, X_test, y_test\n", "\n", "\n", "# Invoke the above function to get our data.\n", "X_train, y_train, X_val, y_val, X_test, y_test = get_CIFAR10_data()\n", "print('Train data shape: ', X_train.shape)\n", "print('Train labels shape: ', y_train.shape)\n", "print('Validation data shape: ', X_val.shape)\n", "print('Validation labels shape: ', y_val.shape)\n", "print('Test data shape: ', X_test.shape)\n", "print('Test labels shape: ', y_test.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Sanity check loss\n", "After you build a new network, one of the first things you should do is sanity check the loss. When we use the softmax loss, we expect the loss for random weights (and no regularization) to be about `log(C)` for `C` classes. When we add regularization this should go up." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sanity check loss (no regularization): 2.30259705088\n", "Sanity check loss (with regularization): 2.34509025355\n" ] } ], "source": [ "model = init_two_layer_convnet()\n", "\n", "X = np.random.randn(100, 3, 32, 32)\n", "y = np.random.randint(10, size=100)\n", "\n", "loss, _ = two_layer_convnet(X, model, y, reg=0)\n", "\n", "# Sanity check: Loss should be about log(10) = 2.3026\n", "print('Sanity check loss (no regularization): ', loss)\n", "\n", "# Sanity check: Loss should go up when you add regularization\n", "loss, _ = two_layer_convnet(X, model, y, reg=1)\n", "print('Sanity check loss (with regularization): ', loss)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Gradient check\n", "After the loss looks reasonable, you should always use numeric gradient checking to make sure that your backward pass is correct. When you use numeric gradient checking you should use a small amount of artifical data and a small number of neurons at each layer." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "W1 max relative error: 2.308053e-07\n", "W2 max relative error: 8.429969e-06\n", "b1 max relative error: 5.879161e-09\n", "b2 max relative error: 1.017082e-09\n" ] } ], "source": [ "num_inputs = 2\n", "input_shape = (3, 16, 16)\n", "reg = 0.0\n", "num_classes = 10\n", "X = np.random.randn(num_inputs, *input_shape)\n", "y = np.random.randint(num_classes, size=num_inputs)\n", "\n", "model = init_two_layer_convnet(num_filters=3, filter_size=3, input_shape=input_shape)\n", "loss, grads = two_layer_convnet(X, model, y)\n", "for param_name in sorted(grads):\n", " f = lambda _: two_layer_convnet(X, model, y)[0]\n", " param_grad_num = eval_numerical_gradient(f, model[param_name], verbose=False, h=1e-6)\n", " e = rel_error(param_grad_num, grads[param_name])\n", " print('%s max relative error: %e' % (param_name, rel_error(param_grad_num, grads[param_name])))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Overfit small data\n", "A nice trick is to train your model with just a few training samples. You should be able to overfit small datasets, which will result in very high training accuracy and comparatively low validation accuracy." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "starting iteration 0\n", "Finished epoch 0 / 10: cost 2.280086, train: 0.180000, val 0.092000, lr 1.000000e-04\n", "Finished epoch 1 / 10: cost 2.193264, train: 0.220000, val 0.126000, lr 9.500000e-05\n", "Finished epoch 2 / 10: cost 1.557828, train: 0.260000, val 0.124000, lr 9.025000e-05\n", "Finished epoch 3 / 10: cost 1.340603, train: 0.320000, val 0.136000, lr 8.573750e-05\n", "Finished epoch 4 / 10: cost 1.672992, train: 0.580000, val 0.149000, lr 8.145062e-05\n", "Finished epoch 5 / 10: cost 1.418175, train: 0.680000, val 0.162000, lr 7.737809e-05\n", "Finished epoch 6 / 10: cost 0.575956, train: 0.780000, val 0.161000, lr 7.350919e-05\n", "Finished epoch 7 / 10: cost 0.244489, train: 0.880000, val 0.179000, lr 6.983373e-05\n", "Finished epoch 8 / 10: cost 0.694723, train: 0.860000, val 0.187000, lr 6.634204e-05\n", "Finished epoch 9 / 10: cost 0.045149, train: 0.960000, val 0.188000, lr 6.302494e-05\n", "Finished epoch 10 / 10: cost 0.235708, train: 0.880000, val 0.144000, lr 5.987369e-05\n", "finished optimization. best validation accuracy: 0.188000\n" ] } ], "source": [ "# Use a two-layer ConvNet to overfit 50 training examples.\n", "model = init_two_layer_convnet()\n", "trainer = ClassifierTrainer()\n", "best_model, loss_history, train_acc_history, val_acc_history = trainer.train(\n", " X_train[:50], y_train[:50], X_val, y_val, model, two_layer_convnet,\n", " reg=0.001, momentum=0.9, learning_rate=1e-4, batch_size=10, num_epochs=10,\n", " verbose=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Plotting the loss, training accuracy, and validation accuracy should show clear overfitting:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": [ "iVBORw0KGgoAAAANSUhEUgAAAmMAAAHuCAYAAADTDgvmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XeYXVXZ/vHvTe9NECEBUYqACkQQ6YQOoRcpKigooMhL\n", "9LXgyw8FKygWRBSRpgIigtIhlEBCEQlI6IReQgsd6UWe3x9rDTk5OTNzyt6nzNyf65orc/bZe+01\n", "s5OZJ2s961mKCMzMzMysM2bpdAfMzMzMhjMHY2ZmZmYd5GDMzMzMrIMcjJmZmZl1kIMxMzMzsw5y\n", "MGZmZmbWQaUFY5LmknSDpFsk3SXpiH7OO0bSfZJulTSqrP6YmZmZdaPZymo4It6QtFFEvCZpNuBa\n", "SetFxLV950gaAywXEctL+hRwHLBWWX0yMzMz6zalTlNGxGv50zmAWYHnq07ZDvhTPvcGYCFJi5fZ\n", "JzMzM7NuUmowJmkWSbcA04CrIuKuqlNGAFMrXj8GjCyzT2ZmZmbdpLRpSoCIeBdYTdKCwKWSRkfE\n", "hKrTVH1ZdTuSvGeTmZmZ9YyIqI5v+lVqMNYnIl6SdBGwBjCh4q3HgaUqXo/Mx2q1UfcXZd1F0uER\n", "cXin+2GN87PrbX5+vcvPrrc1OohU5mrKRSUtlD+fG9gMmFx12vnAXvmctYAXI2JaWX0yMzMz6zZl\n", "jowtAfxJ0iykoO/UiBgvaX+AiDg+Ii6WNEbS/cCrwN4l9sfMzMys6yii+9OxJIWnKXtXP7mC1gP8\n", "7Hqbn1/v8rPrbY3GLQ7GzMzMzArUaNzi7ZDMzMzMOsjBmJmZmVkHORgzMzMz6yAHY2ZmZmYd5GDM\n", "zMzMrIMcjJmZmZl1kIMxMzMzsw5yMGZmZmbWQQ7GzMzMzDrIwZiZmZlZBzkYMzMzM+sgB2NmZmZm\n", "HeRgzMzMzKyDHIyZmZmZdZCDMTMzM7MOcjBmPUni1xK7drofZmZmrXIwZr1qXWCrTnfCzMysVbN1\n", "ugNmjZKYBVgRWLjTfTEzM2uVR8asF40EXgYWlBjR6c6YmZm1wsGY9aKVgLuAa4H1O9wXMzOzljgY\n", "s160InA3cDWwQYf7YmZm1hIHY9aLVgKmANfgkTEzM+txDsasF61EGhmbDHxQ4n0d7o+ZmVnTHIxZ\n", "L1oRuDuCd4DrgfU63B8zM7OmlRqMSVpK0lWS7pR0h6SDapwzWtJLkibnj0PL7JP1NolFgLmAJ/Mh\n", "T1WamVlPK7vO2NvA1yPiFknzAf+WdHlE3F113sSI2K7kvtjQsBIwJYLIr68Gft7B/piZmbWk1JGx\n", "iHgqIm7Jn79CyvNZssapKrMfNqT0raTsMwlYWWK+DvXHzMysJW3LGZO0DDAKuKHqrQDWkXSrpIsl\n", "rdyuPllP6ltJCUAEb5AS+dfuWI/MzMxa0JbtkPIU5dnA2DxCVulmYKmIeE3SVsC5wAo12ji84uWE\n", "iJhQUnetu60E/KHqWF+9scvb3x0zMxvuJI0GRjd9fUQMflYLJM0OXAhcEhFH13H+Q8DqEfF8xbGI\n", "CE9lGhIPAFtFcG/FsS2A/4to/h+CmZlZURqNW8peTSngJOCu/gIxSYvn85C0JilAfL7WuTa8ScxN\n", "yjl8sOqt64E1JOZsf6/MzMxaU/Y05brA54DbJE3Oxw4BlgaIiOOBXYCvSHoHeA3YveQ+We9aAXgw\n", "1xd7TwT/kZgCfJK0X6WZmVnPKH2asgiepjQAid2BXSLYpcZ7vwKejuCI9vfMzMxsuq6apjQrWHVZ\n", "i0reNNzMzHqSgzHrJTOUtahyLbCOxKxt7I+ZmVnLHIxZL+nbIHwmETwDPA6s2tYemZmZtcjBmPWE\n", "POK1PHDPAKd5qtLMzHqOgzHrFcuQEvRfHeAcB2NmZtZzHIxZr+h3irLCNcD6kvc6NTOz3uFgzHrF\n", "QCspAYhgKvBKPtfMzKwnDJlgTEIS75dYrNN9sVIMtJKy0tXA+iX3xczMrDBt2Si8SDmRexnSL+cV\n", "q/4EmEXiTuA84LyIARO+rXesBPypjvOuIW3WWr2ZuJmZWVfqmQr8EGeRgq7lgKdJoyR3V/x5N/As\n", "MAfpl/H2+eM/5MAMuCGCd9vdf2tNzgF7Hlghl7AY6NwVgCsi0pZbZmZm7dZoBf5eCsY+Swq47h1k\n", "RV3FdcwCrE4KynYAFgXOJwVm4yN4o6Qu9xwJRdCVfxkkFgfuAhYdrI85cHsKWDOCR9rRPzMzs0pD\n", "NhgrYm9KiWWZHpitCtwAPAe8VMfHcxG83GofupHEVsDXI9i8032pRWI08KMI1qvz/LOA8yM4tdSO\n", "mZmZ1dBo3NJzOWOtiOAB4JfAL3Oi/yeBhYAFKz5GVr1eIP+5qMTNwDnAORE83PYvoDy7kEpCzBnB\n", "m0U1mkepTgEOiOC1Fpqqp6xFpb56Yw7GzMys6w2rkbHW+sBcwCbAjsB2pK13zskfd3TrFN9g8lTu\n", "E8C7wM4RXF9g233lKMZEcEkL7fwaeCSCX9Z5/mrAXyNc4sLMzNqv0bhlyJS2KFsEb0RwUQRfApYA\n", "xgILAxcC90n8TGLtHNz0kjVIyfFnQX3TgA3YAHgH2LLFduota9HnduADOdfMzMysq3lkrEV5Km4U\n", "acRsR2AR4AJgMnAvaS/FJ7p15Ezih8CcwI3A5yLYvsC2TyUVYd2olVEqianABhE81MA1FwJ/jODs\n", "Zu9rZmbWDCfwd5jE8sA2wMrAR4AVgPlIgVnfxz19n0fwUoe6CoDEZOAg4AHSiNJiRZX/kHiENCo2\n", "AfhUM3l2EvMD04D5GumXxMHAkhGMbfSeZmZmrXACf4dFcB/wq8pjEgsBy5MCs48A2+bPV5B4BZhE\n", "SjqfCEyO4J129FViKWBp4PoI3pF4kVTL7a4C2v4gMBdpevEyYAvg+Caa+ggpaG00QLwa+F0T9zMz\n", "M2srB2NtEMGLpGnAGyuP5ynOkcDapPyqzwNLS1xPCsyuBm6M4K2SurYNcElF8HctKW+s5WCM9PVc\n", "HUFIXArsRHPBWKMrKfv8G1hOYqH8/TczM+tKvZZsPqREEBFMjeBvERwYwceBZYHfA+8HjgGek7hS\n", "4jCJjfJ2UEXZhpTf1qcvGCvCBqRgEtLI2MYSszfRTlPBWA5gbwTWaeKeZmZmbeNgrMtE8GwE50bw\n", "9QhWJ42c/RyYBzgOOKSI+0jMS9pQ+9KKw0UGY+uTg7EIniblpK3VRDsr0thKykp99cbMzMy6loOx\n", "LhfBSxFcHMHBwG7AvgWNjm0K3FQ1hTcFmF9iRCsN55ISiwN3VBy+lOZKXDQ7TQlp0/D1m7zWzMys\n", "LRyM9ZAIbiXtu7hFAc1VT1GSy29cR+ujY+sD10Xw34pj42iw33la80PAfU3241/AahLzNHm9mZlZ\n", "6RyM9Z4TgS+10kAuTLsNqWBttSKmKivzxfpcT0qof38D7SwLPNbshu55Q/nbgE81c72ZmVk7OBjr\n", "PWcAG0l8oIU2PgG8lMtwVCslGIvgbeAqYLMG2mllirKPpyrNzKyrlRaMSVpK0lWS7pR0h6SD+jnv\n", "GEn3SbpV0qiy+jNURPAy8HfgCy00sy1VU5QVbgaWl1iwmYZzTbVlczvVLqWxqcoigjEn8ZuZWVcr\n", "c2TsbeDrEfFR0iq6r0paqfIESWOA5SJieWA/0mpBG9wJwJdynbJmbEvtKcq+khA30dzKR4B1gUn9\n", "1Ea7FNiigf07+zYab8V1wKeaLKthZmZWutKCsYh4KiJuyZ+/QvqlumTVadsBf8rn3AAsJMmbOw9u\n", "EvA6MLrRC/NKyQ+SgpT+tDJVWStfDIC8t+SLwKp1ttXoBuG17vkC8CBpatbMzKzrtCVnTNIypM20\n", "b6h6awQwteL1Y6S6WjaAvOrxBGDfJi7fBhg3yJZLreRZ9RuMZXWVuMijfkWMjJH7s2EB7ZiZmRWu\n", "9O2QJM0HnA2MzSNkM51S9brmzuWSDq94OSEiJhTSwd51GvADifdF8FwD120LnD7IOdcDa0jM0chW\n", "TLmQ7CrMHHRXGgd8GzhikOZGAq8UtJXRRFKO3c8KaMvMzGwGkkbTxGzVe9dH1Ix9CiFpdlJu0iUR\n", "cXSN939PCqz+ml9PATaMiGlV5zW0+/lwIXEaae/KX9d5/jzANGDpPH030LmTga9E8K8G+rMx8MMI\n", "1h3gnHlJtdJGRPCfAc7bDDgkgo3qvf8Abb0fuBdYtF2bsJuZ2fDVaNxS5mpKAScBd9UKxLLzgb3y\n", "+WsBL1YHYjagE0kV+et94JsA/x4sEMuayRsbbIqyr/bXv2DQIKuIlZR993waeBxYrYj2zMzMilRm\n", "zti6wOeAjSRNzh9bSdpf0v4AEXEx8KCk+4HjgQNK7M9QNBGYg/pXPg5U0qJas8HYNXWcV0+Ji8KC\n", "sWwCLQwhm5mZlaXUacqieJqyfxLfBj4SwRcHOU+k0aHREdxbR7sjgVuAxfKCgcHOnwN4njT9+NIg\n", "536MNCq6bH9tS1wF/CSCywe7dz0kPg3sFcG2RbSX25wdWCeCiUW1aWZmva9rpimtbf4E7CSxwCDn\n", "fYKUED9oIAYQwWPAy6QVjfVYA7h3sEAsu5M0orfcAOcUPTI2EVi/oE3W+2wGXJQDUTMzs6Y4GOtx\n", "EUwDrgT2GOTUmTYGr8M11D9VuT6D5Iv1yaNhl9FPiQuJhYF5SSN5hSgpb2xLUj/XLrBNMzMbZhyM\n", "DQ311BxrJF+sTyN5Y4Mm71cZR/95YysCU+qZHm3QRIrNG9sSuITG9ts0MzObgYOxoeFyYDGJmnt7\n", "SiwJfJiBq+7XUlcwlqf+1s3n1+sKYAOJOWu8V/QUZZ8JFBSMSSwLzA8chYMxMzNrgYOxISCC/5LK\n", "iHypn1O2Bi6N4O0Gm54CLJSDuYGsAjyZpwLrEsHzpNyxWsFeWcHYRGC9gvLGtiStCv0nsJLEIgW0\n", "aWZmw5CDsaHjFGD3XNi1Wr8bgw8kgndJo2mDjY7VnS9Wpb8SF6UEYzm/7knq3xtzIFuStpV6kzQi\n", "uHEBbZqZ2TDkYGyIiGAqqZjqpyuPS8xNmpq7pMmm65mqbDRfrM84aifxr0iLG4QPYAItTlXmqdUN\n", "4b2yG5fjqUozM2uSg7Gh5QRmnqrcGJicpwWbMWAwluuXNRuM3QiMqJwGlZiLtIH8A020V48ikvjX\n", "A+6s2BPUwZiZmTXNwdjQchGwnMRKFceamqKs8G9ghQHqmH0EeDWPzDUk57qNZ8apyhWAh5rIb6tX\n", "EfXGtiKN6vW5E5grJ/WbmZk1xMHYEJIDmD+SR8fyqFUz9cUq23yTFJD1t+VSvVsg9ae6xEWZU5RE\n", "8BRpo/JVWmhmSyqCsVyCw6NjZmbWFAdjQ8+JwJ45r2k14HXgnhbbHGiqstnk/T6XAZtVjFSVtZKy\n", "0gSanKqUWAr4AHBT1VsOxszMrCkOxoaYCB4Abge2J09RFlA89VpS0FVLs/liwHvbLj1B2k4JujwY\n", "I43iXZanWCtdAWwsMVsL/TIzs2HIwdjQ1FeRv5mq+7VcD3yyeg9GiQ8CcwL3tdj+pUxfVVnqNGU2\n", "kVRwtpm8sep8MeC96c+pTA8qzczM6uJgbGg6lzRFuRyNVcWvKYIXSasbqyv8bwBcXcDI2zhgixwc\n", "rUDJwVgOnKbRYN6YxOzAJqTgsRZPVZqZWcMcjA1BEbwB/JlUlPStgpqtlTfWavJ+ZdsfIwWQz0bw\n", "SgFtDmYCqVZYI9YCHsjFY2txMGZmZg1zMDZ0/R+wX4HtXcPMwViryfvAe8HjtcCBlD9F2WcCjeeN\n", "zbCKsoargVES8zfZJzMzG4YcjA1REbwVwcsFNnktaV9HAUgsDiwO3FFQ++OAz1B+8n6fvryxRv4N\n", "1MwX6xPBa8AkGh9xaxuJJfKzMzOzLuFgzOqSVz2+SiryCmlU7NoaqwqbdSkwB20KxiJ4EniGOvPG\n", "JD4AfIi05dRALgc2b613pToJOLXTnTAzs+kcjFkjKvPGWippUcO9wIMUN9JWjwnUP1W5OTC+jp0B\n", "ujZvTOIjpNWey0v9lioxM7M2czBmjSgtGMsrMtcGriuqzTpMoP4pxcHyxfpMBhaTGNlsp0r0P6Sy\n", "Jz/IH2Zm1gUU0WpVgvJJiohQp/sx3El8jFQ2Yw1STa1FStxDsnR5g/LbgcUieHeA82YllcIYVc8e\n", "nBJnklaynlJYZ1sksRDwEGnV6jTSdPB+EVzV0Y6ZmQ1BjcYtHhmzRtwFLALsAtzQy4EYQARPAM8C\n", "Hx/k1DWApxrYDP0yum+qch9SgPh4BO8A3wd+0Lcgw8zMOmfQYEzS1yQtqOQkSZMlbTHYdTb05NGj\n", "64CDKTZfrJMmMnje2JbAJQ20eTmwaYMrNWdQZJCUR/b+Bzi64vAZwKJ0X9BoZjbs1PPLYp+IeImU\n", "wLwIsCdwZKm9sm52Lamy/1AJxiYweN5YvfliAETwKPACsGozHZJYALhXYtNmrq9hW2BaBDf0Hcir\n", "YA/Ho2NmZh1XTzDW94N6a+DUiGjnajfrPtcCb8P0X+w9biKwYX+jWBLvAz5K49tKtbKq8kfAK8DR\n", "BW08Phb4dY3jZwHzAmMKuIeZmTWpnmDs35IuI/3AvlTSAtB/snMlSSdLmibp9n7eHy3ppTz1OVnS\n", "ofV33TrkBmD7CF7vdEeKEMHjwHOkxPZaNgMmRvBmg003FYxJfBLYFdgUeArYv9E2qtpblbTf59nV\n", "7+Vp58Pw6JiZWUfVNU1J2lpnjYh4FZgd2LvO9k8hTfEMZGJEjMofP6qzXeuQCN6JaCh/qhdMoP+8\n", "sUbzxSrbXEti7novyKNgxwPfiuA54OvA9yQWaeL+fcYCvxtgscW5pJ8D27dwDzMza0E9wdjawD0R\n", "8aKkPYFDgZfqaTwiriHlzgzE/yO3TquZxJ+nLhvKF+sTwUvAbcy8n+dADiT9ezktt3E78HdSblfD\n", "JBYDdgT+MEA/3wW+B3y/lQUHZmbWvHp++P4eeFXSqsD/AvcDfy7o/gGsI+lWSRdLWrmgds0a0d8+\n", "lasC/4ngwSbbrbvERS4SeyhwQC6A2+d7wB4Szfzb2B/4RwTPDHLehcBbwE5N3MPMzFpUT3LwOxER\n", "knYAfhsRJ0r6YkH3vxlYKiJek7QVacpkhVonSjq84uWEiJhQUB9smIvgMYkXSIn6lfmNTY2KVbgc\n", "+G2d5x4DHBvBPVV9e1bix8AvJbaqCtT6JTEH8BXS5uYDiiAkvgf8XOKcAvcbNTMbFiSNpv7t9Wa+\n", "frAK/JKuJv1C2pu0OfQzwC0RMVihzL7rlwEuqOd8SQ8Bq0fE81XHXYHfSiVxAnBbBL+pODYROLLZ\n", "HLmcA/Ys8JEIpg1w3nbAz4FVInijxvuzk6Y8vxnBRXXe+zPAvhFsVOf5ItWQ+00EZ9RzjZmZ1VZG\n", "Bf7dgDdJ9caeAkYARzXZvxlIWlyS8udrkoLD5we5zKwME6j4X43EgsAnSFOYTcmV7icAm/R3jsR8\n", "wG+AL9cKxHI7b5NSBH6ZR7zqMZYZi7wO1tcgTYkeXlA5DTMzq9OgwVhEPAmcDiwkaRvgjYioK2dM\n", "0hnAP4GPSJoqaR9J+0vqW66/C3C7pFtIvzh2b+qrMGtddb2xTYB/RvBai+0OVuLicFLpjCsHaiSP\n", "zj0AfHWwG0qsBSxGygVrxHhSOY3PNHidmZm1oJ5pyl1JI2F9IwQbAN+KiLNK7ltlHzxNaaWTuB/Y\n", "IYI7JP4A3B3Br1pscwXgSmCp6nyvXAPscuBjETxdR1srAtcAKw+UlC9xBjCpmb5LbAicDKzYi3uP\n", "SswRwVud7oeZDW+Nxi31BGO3AZtGxNP59WLA+IhYpaWeNsDBmLWDxInALaSk+0eAzSOY0mKbAh4C\n", "torg7orjs5JytE6M4MQG2jsamDOCr/Tz/gjSIoQP5fIazfT5CuCMCE5q5vpOynl+twBfq3exg5lZ\n", "0crIGRPM8L/w53BtMBuaJpDyxlYm7TJxz0An1yMHBLWmKvcD3iGNQjXi+8BOEv39Z+gA4PRmA7Hs\n", "MOC7DeSndYVcHuRjwEakQtVmZj2hnmBsHGkbpC9I2hu4mOYqkpt1u4mkTcPHAOMKHFm5HNi874XE\n", "EsAPgP1z0dW6RfACKc/s6OotjHK1/31h+orQZkRwHTCFtPtGL9kJOJ9UkmRfiS91uD9mZnWpZ5pS\n", "pB9y65GKtF4TEee0oW+VffA0pbWFxAOk+ntjIzi3oDbfBzwILBbBWxJ/BR6M4JAm25sNmAx8L4Jz\n", "Ko5/Cdgxgq0L6POapOr/y/e3yrPb5CnKn0VwUc7Vm0hapXpeh7tmZsNM4Tlj3cDBmLWLxEnAXsAi\n", "EbxcYLs3kcpTzA38Dvh4Kys1JTYhbXO0cgRv5lGy24BvRHBZQX2+ALissvZat5L4AHAXsETfpu55\n", "0/WLgZ0iuKaT/TOz4aWwnDFJr0h6uZ+P/xTTXbOucwWp1ERhgVh2ObAdKRD7aqslMyIYT0rU/1o+\n", "tBEwa75PUb4LHCIxb4FtlmVH4OK+QAwgghtJZTr+LlFXkWozs07wyJhZhVxnbO4IXi243Y1JgdLZ\n", "EexWUJvLAf8iJa0fD1wSwe+LaLviHn8l7UzwkyLbLZrEeNLuATNNLUvsQSrPs14ED7e7b2Y2/Hia\n", "0qwLScxJmjL7XARPFtjuz4BVgDWAD5YQRC5PLtwcQVfujiGxGHAfaYry9X7OOYhUMHe9OjZONzNr\n", "iYMxs2FEYgFSIPLHCA4u6R5/AJ6P4DtltN8qiX2BTSIG3sFD4ifApsDGEbzSls6Z2bDkYMxsmJFY\n", "GZhaQp5bX/sjgVtJOwUUNqpXFIlxpOK5Zw9ynoATgZHAtq7Ub2ZlcTBmZoWT+DkwTwQHFNDWrMAW\n", "EVxcQFuLkHY4WLKeKdpcFuTvwCvAno3WeTMzq0cZFfjNzI4EdpVYtoC2DgYuklivgLa2A66oN1cu\n", "gneA3YGlgV9UF841M+sEB2NmNqgIngWOIW3H1DSJdYGxwA8pZsuiXWDg6clqOcl/O9IWVZ8toA9m\n", "Zi3xNKWZ1UViftJigc0juK2J6xch7RxwIHAZaVeCrSO4pcn+LAg8CiwVQcO1DyU+C+wcwU7N3N/M\n", "rD+epjSzUuQFAkcCP2702jwdeDLwjwguyMVZf0Fro2PbAhOaCcSyq4DROYfNzKxjHIyZWSN+D6yS\n", "pxsbcSBpFWNleYw/ABvlfSSbsTMpGb8pETwBPAWs1mwbZmZFcDBmZnXLm4YfDvyk3uR3iU8A3wN2\n", "r9qu6BXgWGi8PlqeMt0YuKDRa6tcmdsxM+sYB2Nm1qhTgfcDWwx2Yg6a/gocFMH9NU45FthBYukG\n", "+zAGuC6CFxq8rtp4YJMW2zAza4mDMTNrSC4PcShpdKzfnyF55Ow44OoIzuinreeBk4BvNNiNhldR\n", "9mMisI7EHAW0ZWbWFAdjZtaMfwD/JQVF/fk8MAo4aJC2fgXsKfH+em4sMS+wOXBePecPJAeD9wKf\n", "arWtPhILSWxbVHtmNvQ5GDOzhkUQwCHAD3NV+xlIrAQcBewWwWuDtPUkaSpzbJ233xKYFMFzjfW6\n", "X1dS7FTl54AzJRYusE0zG8IcjJlZs64AHge+UHlQYm5ScPX/IrijzraOAvbPtcMGszPFTFH2GU+x\n", "Sfw7AM8DexbYppkNYS76amZNk1gLOAtYPq+0ROJ3wPtIqyfr/gEj8Wfg7giOGOCcuUjlKD4SwbSW\n", "Oj+9zXmBacDi9W6rNEBbCwOPALsCvwQ+2sj3wMyGBhd9NbO2ieBfwL8hbSAusTNpleV+TQQhRwJj\n", "JeYZ4JzNgclFBWIAOQC7GQrZK3Nr0rTnpfn1+gW0aWZDXKnBmKSTJU2TdPsA5xwj6T5Jt0oaVWZ/\n", "zKwUhwLfkViFtHpy9whearSRCO4C/gl8cYDTilpFWa2ovLEdgHNzIHo88OUC2jSzIa7skbFTSMm2\n", "NUkaAywXEcsD+5F+kJtZD8l5YeOA64GfRnBjC80dAXyrVqkJiTmBbYBzWmi/Py3njeVcuc2AC/Oh\n", "PwNj6l0lambDV6nBWERcAwMWZdwO+FM+9wZgIUmLl9knMyvFocDRpDIVTcuB3BTgszXe3gS4K29j\n", "VLQbgBXyZubN2pQ0hfosQC5Iew5VCxzMzKp1OmdsBDC14vVjpP3rzKyHRPBoBP8vgncLaO4npGnP\n", "6g28i15F+Z4I3iJNkW7YQjM7AOdWHfs9aZVop3/WmlkX64YfENWrDbzyyGx4mwg8B+zUd0BidmB7\n", "UrHZsjSdN5YDx22ZORibBPyn2XbNbHiYqVhjmz0OLFXxemQ+NhNJh1e8nBARE8rrlpl1SgQh8RNS\n", "QdmzczL8aOD+CB4t8dbjSftuNmMd4PEIHq48mL+W35MS+S9vrXtm1q0kjSb9nGru+rLrjElaBrgg\n", "Ij5e470xwIERMUbSWsDREbFWjfNcZ8xsGMn7Wt4CfCeCSySOJwVjR5V4z1mBZ4CPNZqXJvEL4D8R\n", "fL/Ge/OTao813K6Z9aauqjMm6QxSHsZHJE2VtI+k/SXtDxARFwMPSrqftAz8gDL7Y2a9IY+GHQEc\n", "krdb2hH4e8n3/C8wAdiokety4FgrX6yv3ZeBMxm4ZIeZDWOuwG9mXSmPVE0hTR1uH8HqbbjngcAn\n", "ItingWs+DpwPfLi/QrcSqwIX5HPeKaSzZta1umpkzMysWXmk6qfA9ylpFWUN44FN8mhXvSoLvdYU\n", "wa3AE8BWLfbPzIYgB2Nm1s1OBf4F/K1N95sCzA58uIFr+p2irNKXyG9mNgNPU5qZVZA4DZgYwQl1\n", "nPtB4CZgicGmH/Oem1OB1atXXZrZ0OJpSjOz1oyn/rpg2wMX1pMHFsFrpJG+fVvom5kNQQ7GzMxm\n", "dCWwcZ1V83egsb0yjwf2yUVszcwAB2NmZjOI4BFS1fyPDnSexPuA1WmgmGsEdwP3kEbUzMwAB2Nm\n", "ZrXUszXS1sD4CF5vsG0n8lvHSOwssXCn+2EzcjBmZjaz8cDGg5xT7yrKaucAH5dYoYlrzZomsSjw\n", "V+CMXMfPuoSDMTOzmV0FbJCr/88kr4zcBLio0YYjeBM4BdivpR4WSOIIiT063Q8r3W6k/wzMDvyo\n", "w32xCg7GzMyqRPA08Cj0W/V/U+DfETzX5C3+AHxeYq4mry+MxCjgq6SN2T1aMrTtSfqPwG7A7hKf\n", "7nB/LHMlaQYMAAAgAElEQVQwZmZW20B5Y81OUQIQwYPAzcAuzbZRhLzTwK+BbwJPk/YAtSEoT4sv\n", "A1wewbPATsDv8nZe1mEOxszMaquZN5anLrcFzmux/W5I5N8FWAA4CfgZ8O0Gt4Ky3vE54Iy+mngR\n", "TAbGAudILNLRnpmDMTOzflwNfKrGVOK6wKO5BEYrLgQ+1KmRCYm5gaOAsXkf0POBBYENO9EfK0+u\n", "mbcnqejweyL4C+k/FU7o7zAHY2ZmNUTwEnAnsHbVWy1NUVa0/zZwIrB/q2016RvATRFMzP15lxSc\n", "fbtD/bHyrAu8Bkyu8d7BwKzAT9raI5uBgzEzs/7NkDeWp/AKCcayE0mJ1FsW1F5dJEYAXwO+VfXW\n", "acBqEqu0sz9Wuj2BUyOYaTPqPG25O7CrxG5t75kBDsbMzAZSnTe2CvAucEcRjUcwlVSN/48SXyii\n", "zTodCRwfwUNV/XkDOIaU0G8FkpDEfB2471zAzsDp/Z2TE/p3BI6VWLVdfbPpHIyZmfXvn8AqEgvk\n", "1zsA59YaYWhWBNcBo4HDJL5bdgK9xFqkAPOIfk75PbCNxNJl9mMY+gJwaQfuuw1waw78+xXBLcBB\n", "pIT+97WlZ/YeB2NmZv3IWx1NAtbPh4qcoqy8zxRSbtqOwPH9FZttVU7kPho4JIJX+unLi8DJpGlM\n", "K85uwDoSy7T5vnsCf67nxAjOAP5BSugv5e+g1eZgzMxsYFcCm+RfoiNIo2WFi+Ap0krGD5JGJ+Yt\n", "4TafJf3cP3WQ844GvuA9DIuRv4/rAGcAu7bxvouS/k79vYHLvgMIJ/S3lYMxM7OB9eWNbQ+cn8tA\n", "lCKCl0nTSs8BV0q8v6i2c77SEaRSFu8O0o/HSKUuvlLU/Ye5bUlB/UnQ1iT53YCL89+rulQk9O/i\n", "LbLax8GYmdnAbgI+BOxDCVOU1XLJi72By4DrJJYrqOmDgYkRXF/n+T8HDuqGLZuGgJ1Io1MTgZEF\n", "PtPBzFRbrB55m68dgWMkViy8VzYTB2NmZgPIwdE1wLKkUbJ23DMi+C4pILpGYs1W2stTrF8hBWT1\n", "9uEOUiC6Vyv3Hu7yiOTGwIV51Ols2jA6Vrn9UTPXR3AraSRvzwK7Zf1wMGZmNrhLgYtyQn/bRHA8\n", "sB9wkcQ2LTT1M+CYPP3Y6HXfdHX2lowB/hnBC/n1mbRnqnKG7Y+adDG0twbecKWIwlZol0ZSRIT3\n", "SzOzjsjByOy5Dlcn7r8maduaw4E/NFJaQ2ID0lTVShG81uB9BVwP/CyCfzRyrSUSZwJXRHBCfj0L\n", "MBXYNIK7S7qngAeAXSK4uYV2ZgeeAVbMC0ysTo3GLR4ZMzMbRAT/7VQglu/fV17jAOAhiV9IrJ1/\n", "sfcrB5FHAwc3Gojl+wZpdOxgbyDeuJxvtwUVuYZ58cRZlDs6ti7wOrW3P6pbnqIfT/oarESlBmOS\n", "tpQ0RdJ9kmbKVZA0WtJLkibnj0PL7I+ZWa+K4H5gNdLKvFdIWyk9KvFriQ36mUrcm7Qn4Zkt3Po8\n", "YGGm11qz+m0OTI7gmarjZwK7lRjg9rv9URMuwVOVpSttmlLSrMA9wKbA48CNwB4RcXfFOaOB/42I\n", "7QZpy9OUZmZVJFYibXWzC/ABUsHOvlV785B+Bm8Twb9bvM9+wHYRLeWtDTsSfyJtxv6bquMCHga2\n", "jeC2gu85F+l37mqDVd2vs72RwK3A+8ss6zLUdNM05ZrA/RHxcES8DfyVVKenmoMsM7MmRHB3BD+K\n", "YDXSyNWjpH0nnyRNL13SaiCW/RlYXeJjBbQ1LEjMQaoZN1OuXR6x+hvlTFVuTR3bH9UrL/p4Avhk\n", "Ee1ZbWUGYyNghr8Mj+VjlQJYR9Ktki6WtHKJ/TEzG7IiuC+CIyP4JOkX559ooJTFIG2/AfwGbyDe\n", "iI2AeyN4vJ/3y5qqbKq22CAuAbYquE2rUObeU/XMf94MLBURr0naipTkuEKtEyUdXvFyQkRMaLmH\n", "ZmZDUAQPA8cW3OxxwAMSI5sokTEc9RV67c+/STNDn8iftyxv8D2a4mvDjSONuB5WcLtDRk67Gt30\n", "9SXmjK0FHB4RW+bX/we8GxE/HeCah4DVI+L5quPOGTMz6zCJXwLvRniEbCB5McUTwNoRPDjAeT8B\n", "Zovg2wXd9wBg/YhitzHKU67PAMvVWIxgNXRTzthNwPKSlpE0B2lu/PzKEyQtLkn58zVJweHzMzdl\n", "ZmZd4Ghgb4mFOt2RekksInFeq7sYNGg94ImBArHsTGDXAqcqy5iiJIK3gAmk1aFWgtKCsYh4BziQ\n", "VLn6LuDMiLhb0v6S9s+n7QLcLukW0j/y3cvqj5mZtSaCR4GLgK92ui/1kFiMtEH3vMDJeYSnHXZm\n", "4CnKPrcBbwCfavWGEsuT9lC9rNW2+uESFyVyBX4zM6tb3uT6X8DqETzS6f70R2IJ4ArSasbvARcA\n", "N0Tww5LvOwtpVetm9VTYl/g+sEAEX2/xvj/I7XytlXYGaH8ZYBLwgVy41gbQTdOUZmY2xOTis78C\n", "ju3WqvwSS5Fqrf0lgu/mUhIHAGNzbbYyfRL4TwNbHZ0JfHqw3RQGkp/D50glSEqRF4U8B6xe1j2G\n", "MwdjZmbWqKOAZYEdO92RahIfIgVix0fw477jeYr1cOCEVgKfOtQ7RQlABHcBLwDrtHDPQrY/qoOn\n", "KkviYMzMzBqSE7q/DBwjsUCn+9Mn501NAH4RwS9qnHIc6ffe/jXeK+L+IgVjjW6qfiatFYAtcvuj\n", "gYzD9cZK4ZwxMzNrisRJwKsRHNQFfVmZlLx+eAQnDnDeR0kB26ii66VJrAqcAyzbSGCUg8irgZGN\n", "bjmURwJvoqDtjwa511zA08AyETRd+UBiR9LuBF9qQwDZEc4ZMzOzdvk2qTRDR7fKyUHQeOA7AwVi\n", "ABHcSSqI+7sSct52Bv7eaIARwX2kLaw2aOQ6iaVJq0UPLTsQg/d2Yrga2KzZNnINtiNJG97vVFDX\n", "ep6DMTMza0oEz5ECsuOlUnd06ZfEGqQRsYMiOK3Oy44k5bztUnB3GsoXq9LQVKXECFIgdnQExzV5\n", "z2a0OlW5K6mA7I6kae6FC+lVj/M0pZmZNS2PLo0Hzo/g6Dbfex3SNnr7RnBeg9euTQqcPtbKlFtF\n", "eyuSSmks3UzphzzdeAOwZATvDHLu4qRFCidH8LNm+tssiWWB60j9bOjrzAsn7gC+FsFlEr8F5ohg\n", "3xK62lGepjQzs7bJU3JfBg7NJSXaQmJD4Dxgr0YDMYAIricFY0cV1KWdgHOarcEVwUPAQ8DGA50n\n", "sSgp6Du93YEYQAQPAP8BVm3i8p3ztZfn1/8HbCGxUUHd61kOxszMrCUR3Av8BjimzPtIzCaxrcS5\n", "pEBqtwjGtdDkIcBm0sABUJ1amaLsM+BUpcQipEDmfOBHLd6rFeNosMRFHhX7LvCDvpy6CP5Dqv92\n", "gsTchfeyhzgYMzOzIhwJrCyxQ9ENSyyXN9V+lDSacgHwoQiubKXdCF4mBQN/aCUYyFOMI4FrWukP\n", "cBawQ61tmyQWJAVB40kJ+53MMbqExvPGdgDezNe+J4ILSatBDyuma73JwZiZmbUsgjeZXnts/lbb\n", "k5hb4nMSE4B/AnMCm0awTgQn5UCqZTkYuJHWgoGdgPMaLUtRoy9TgSlUrVbM38+LSTll3+qCchAT\n", "gVH1bhif8wq/R8WoWJWxwD4SowrsY09xMGZmZoWI4CrSCr/vN9uGxCckfgc8BnyWNP05MoJv5Gr1\n", "ZRgL7N1CMLATrU9R9plhqlJiHtJI4J3A2C4IxIjgNVIS/yZ1XrItEMCF/bQ3jbQq98ROrcrtNK+m\n", "NDOzwuQE8zuBrSK4uc5rlgR2J1WSXwg4GfhjO2pnVfTh88BBwKcGW81Ydd2SpBWCH8g7E7TajyVJ\n", "378l8qELgMeBfbppg26JscDHI/jSIOeJNPL44wjOGeS8y4BLI/h5oZ0tgMRI4Ml6Rz+9mtLMzDom\n", "gmeB75Bqj83a33kSC0rsI3EFKZj5GPBNUvX6H7YzEMv+DDwPfK3B63YELiwiEAOI4AngVmA70mjb\n", "M8AXuykQy8YBW9ZROHcrYA4YeMVrHvHbH/hOLp/RNSRWAybR2v6hA3IwZmZmRfsj8BopOf49EnNK\n", "7ChxNikZf1vg98CICPaJYHyngo6qYOCzEqvk7X8GU8QqympnkoLD10mlO1rKRSvJvcBbpCC6popc\n", "sR/W81wjeBA4grSgoitmwyTWZ3pR4VYXaPR/H09TmplZ0SRWIq0uHEWqdv9ZUm7V7cDpwNkRvNC5\n", "HtaW9038HLAS8GHSFOHdpMT69/6M4HmJxYD7gCUieL3APiwMfB34UVEjbmXIuX0P91fvTGJz4Fek\n", "6cy6guycM/Yv4LcRnFJYZ5sgsQ1pyvwzEVzR2LWNxS0OxszMrBQSPwAOJgUwpwNndGD6sWkSs5MC\n", "shVJwdlKFZ+/ATwL3BHBrh3rZAdJbAt8PWLmOm15ZOta4NgIzmiw3dVIo1GrRPBUIZ1tkMTngJ8D\n", "20UwqfHrHYyZmVkXyKMcH8obYQ8ZOdBYghSU3dtLAWaRJOYjbXC+ZHWpkVxI9zhg5WamWSWOAD4c\n", "Uf9+nUWROIiUv7hlsyt4HYyZmZlZW+QFGMdGcG7V8QnASRGc2mS7c5MWMnwzgvNb7mh99xRwOLAH\n", "sFkEjzTflldTmpmZWXtcQtXWSHnf0JHQ2PRkpZyDty/wW4kFWuphHfJ2Tb8hLSpZr5VArBkOxszM\n", "zKxZ44CtqlY/fpdUV6zuem21RDCRFOydLPHJgUqltCJvP3Ua8HFgowieLuM+A3EwZmZmZs26CxBp\n", "YQMS65JWz55WUPvfAqaSyqU8I3G2xJfzfqUtpy/lHQ7OBeYl5Yi91GqbTfXDOWNmZmbWLInjgXsi\n", "+KXEOODvEZxQwn2WJG3BtClp/863gMuBK4ArI3imwfYWJm3RdD+psG5LI3kztu0EfjMzM2uTXJvt\n", "AOBQ4G/A8mXXR8ujYiuRArNNgQ2BB0nlNN4hbSw/R/6z8vPKY315bd8sutiwgzEzMzNrm5xg/zhp\n", "y6CzIziuA32YHfgksFY+9CZp5OzNio+3qj5/GZhSxubrXRWMSdoSOBqYFTgxIn5a45xjSHtXvQZ8\n", "ISIm1zjHwVgPkzQ6IiZ0uh/WOD+73ubn17t67dlJXAUsT9pb9M1O96fTuqa0haRZgWNJS15XBvaQ\n", "tFLVOWOA5SJieWA/aH80bW0xutMdsKaN7nQHrCWjO90Ba9roTnegQceQ9m8c9oFYM2Yrse01gfsj\n", "4mEASX8Ftift7dVnO+BPABFxg6SFJC0eEdNK7JeZmZkVKIJzOt2HXlZmaYsRMMMWEY/lY4OdM7LE\n", "PpmZmZl1lTJHxupNRqueU615naTuX2lg/ZJ0WKf7YM3xs+ttfn69y89u+CgzGHscWKri9VKkka+B\n", "zhmZj83AyftmZmY2VJU5TXkTsLykZSTNAewGM232eT6wF4CktYAXnS9mZmZmw0lpI2MR8Y6kA4FL\n", "SaUtToqIuyXtn98/PiIuljRG0v3Aq8DeZfXHzMzMrBv1RNFXMzMzs6GqqzcKl7SlpCmS7pN0cKf7\n", "YwOTdLKkaZJurzi2iKTLJd0r6TJJC3Wyj9Y/SUtJukrSnZLukHRQPu5n2OUkzSXpBkm3SLpL0hH5\n", "uJ9dD5E0q6TJki7Ir/38eoCkhyXdlp/dpHysoWfXtcFYPUVjreucQnpelb4DXB4RKwDj82vrTm8D\n", "X4+Ij5K2FPlq/jfnZ9jlIuINYKOIWA1YBdhI0nr42fWascBdTK8q4OfXGwIYHRGjImLNfKyhZ9e1\n", "wRgVRWMj4m2gr2isdamIuAZ4oerwe4V98587tLVTVreIeCoibsmfv0Iq0DwCP8OeEBGv5U/nIOXp\n", "voCfXc+QNBIYA5zI9JJPfn69o7rqQ0PPrpuDsXqKxlr3q9xRYRqweCc7Y/WRtAwwCrgBP8OeIGkW\n", "SbeQntFVEXEnfna95FfAt4B3K475+fWGAK6QdJOkffOxhp5dmXXGWuWVBUNMRISL93Y/SfMBfwfG\n", "RsTL0vT/8PkZdq+IeBdYTdKCwKWSNqp638+uS0naBng6IiZLGl3rHD+/rrZuRDwpaTHgcklTKt+s\n", "59l188hYPUVjrftNk/QBAElLAE93uD82AEmzkwKxUyPi3HzYz7CHRMRLwEXA6vjZ9Yp1gO0kPQSc\n", "AWws6VT8/HpCRDyZ/3wGOIeUZtXQs+vmYKyeorHW/c4HPp8//zxw7gDnWgcpDYGdBNwVEUdXvOVn\n", "2OUkLdq3WkvS3MBmwGT87HpCRBwSEUtFxIeA3YErI2JP/Py6nqR5JM2fP58X2By4nQafXVfXGZO0\n", "FXA004vGHtHhLtkAJJ0BbAgsSpoj/x5wHvA3YGngYWDXiHixU320/uXVd1cDtzE9TeD/gEn4GXY1\n", "SR8nJQnPkj9OjYijJC2Cn11PkbQh8I2I2M7Pr/tJ+hBpNAxS6tfpEXFEo8+uq4MxMzMzs6Gum6cp\n", "zczMzIY8B2NmZmZmHeRgzMzMzKyDHIyZmZmZdZCDMTMzM7MOcjBmZmZm1kEOxsysZ0i6Lv/5QUl7\n", "FNz2IbXuZWZWNtcZM7Oek/fv+0ZEbNvANbNFxDsDvP9yRMxfRP/MzBrhkTEz6xmSXsmfHgmsL2my\n", "pLGSZpF0lKRJkm6VtF8+f7SkaySdB9yRj50r6SZJd0jaNx87Epg7t3dq5b2UHCXpdkm3Sdq1ou0J\n", "ks6SdLek09r73TCzoWK2TnfAzKwBfUP5BwPf7BsZy8HXixGxpqQ5gWslXZbPHQV8NCIeya/3jogX\n", "8h6OkySdHRHfkfTViBhV4147AasCqwCLATdKujq/txqwMvAkcJ2kdSPC05tm1hCPjJlZL1LV682B\n", "vSRNBv4FLAIsl9+bVBGIAYyVdAtwPbAUsPwg91oP+EskTwMTgU+SgrVJEfFEpHyPW4BlWviazGyY\n", "8siYmQ0VB0bE5ZUHcm7Zq1WvNwHWiog3JF0FzDVIu8HMwV/fqNmbFcf+i3+mmlkTPDJmZr3oZaAy\n", "2f5S4ABJswFIWkHSPDWuWwB4IQdiKwJrVbz3dt/1Va4Bdst5aYsBGwCTmDlAMzNriv8XZ2a9pG9E\n", "6lbgv3m68RTgGNIU4c2SBDwN7JjPr1wyPg74sqS7gHtIU5V9/gDcJunfEbFn33URcY6ktfM9A/hW\n", "RDwtaaWqtqnx2sxsUC5tYWZmZtZBnqY0MzMz6yAHY2ZmZmYd5GDMzMzMrIMcjJmZmZl1kIMxMzMz\n", "sw5yMGZmZmbWQQ7GzMzMzDrIwZiZmZlZB5UajEk6WdI0SbcPcM4xku6TdKukUWX2x8zMzKzblD0y\n", "dgqwZX9vShoDLBcRywP7AceV3B8zMzOzrlJqMBYR1wAvDHDKdsCf8rk3AAtJWrzMPpmZmZl1k07n\n", "jI0Apla8fgwY2aG+mJmZmbXdbJ3uAKCq1zPtXC7Ju5mbmZlZz4iI6vimX50Oxh4Hlqp4PTIfm0kj\n", "X5R1F0mHR8Thne6HNc7Prrf5+fUuP7ve1uggUqenKc8H9gKQtBbwYkRM62yXzMzMzNqn1JExSWcA\n", "GwKLSpoKHAbMDhARx0fExZLGSLofeBXYu8z+mJmZmXWbUoOxiNijjnMOLLMP1hUmdLoD1rQJne6A\n", "tWRCpztgTZvQ6Q5Y+yii+3PjJYVzxszMzKwXNBq3dDqBvyVeZdk/B69mZma9oaeDMXDQUYuDVDMz\n", "60YSswGbk6onXBrBIx3uUlfo6WlKT1/W5u+LmZl1CwkBnwD2BHYHHswfmwPPABcBFwPXRfB2p/pZ\n", "pGE1TWlmZmbdSWJp4LOkIGxO4FRgvQjuz+/PAqwBjAGOApaTuIIUnI2L4KmOdLwDPDI2BPn7YmZm\n", "nSCxALALKQD7OHA2KQj7Z8TMO+xUXbs4sCWwNbAZcD9pxOxi4MYI3i2x64Vq9Pewg7EuJek44PGI\n", "+FET1w7Z74uZmXWXijywPYGtgKtIAdhFEbzZZJuzA+uQRs22Bt4PjCONml0WwQsFdL00Dsa6hKSH\n", "gX0i4soO3Ltrvy9mZtb7+skDOxX4WwTPlXC/D5ICszGkYvK3kEbMLgLuGGzUrd2cM9Y9gpk3QQdA\n", "0mwR8U6b+2NmZtaSwfLAypJXXR4HHCcxNykg2xo4D5hNem86c3wEr5bZlzJ0em/KIUnSqcDSwAWS\n", "Xpb0LUnvStpH0iPAFfm8syQ9KelFSRMlrVzRxh8l/TB/PlrSY5L+V9I0SU9I+kInvjYzMxteJBaQ\n", "2EfiKuBm4IPAvsByERxediBWLYLXIxgXwf8Ay5KmSO8FxgJPSVwqcZDEcu3sVyscjJUgIvYEHgW2\n", "iYj5gb/ltzYAVgS2yK8vApYDFiP9BT+9spn80WdxYAFgSeCLwG8lLVjW12BmZsOXxGwSYyTOIP0+\n", "2xb4DTAigi9HcF03TA1GEBFMieCXEWwCjAB+D6wCXC1xj8SvJDaTmLOzve3fkM4Zk4r5ixJRe7px\n", "IJIeAr4YEVdKWoY0n/7hiHi4n/MXAp4HFoyIlyWdAjwWEd+VNJo0/DpfRLybz58GbBsRk2q05Zwx\n", "MzNrSLvzwMqWv57VmL4I4KOkxQUXAZdE8Fh593bO2HuaCaJKNrXvE0mzAD8hLQFeDN5bsrso8HKN\n", "a5/rC8Sy14D5SuqnmZkNE53KAytbHrmbnD9+LLEoaWZqDHCExGNMXwRwQwQdy+Ue0sFYh9Ualas8\n", "9llgO2CTiHikYmRM/ZxvZmZWiH7qge1LHfXAelUEz5LSgU6XmBX4FGnE7FhgaYlLScHZuHxu2zhn\n", "rDzTSImF/ZkPeBN4XtK8pFGySqKf1ZhmZmaN6pU8sHaI4L8R/DOC/xfBKFKO2ZXAzsADEtdLHCrx\n", "iTzdWSoHY+U5AjhU0vOkh1v9F/zPwCPA48AdwPVV51Qn8A+LfyBmZlYcCUmsLnE08BhwKHA1sGwE\n", "O0bwj2YLsw4lETwewYkR7EgqMPtdYBHgL8DjEidJ7JRHFAs3pBP4hyt/X8zMhrcaeWCnAadFcF9H\n", "O9aDcomMrUhTmusCNzJ9c/MptUYTXYHf/H0xMxuGqvLAVgHOos59Ia0+EvMCGzN9heY7TF8EMCGC\n", "19N5DsaGPX9fzGwoy3shrg68DTxHWvz0ynAMOMrYF9Lqk3PJPkoKysYAo0hTwBeDfutgbJjz98XM\n", "hpr8i29VUtDxGeBZ0qjE+0i5PbOTgrLnSQHacxWf1zr2HPB8BG+09QspQI16YA+R8pB7sh7YUCGx\n", "MLAZsDVoLwdjw5y/L2Y2VEiMYHru0/xMz32aUnXeXKSgbBFSgPa+is9rHev7eJsGAzhSENf2mlTO\n", "A+sdnqY0f1/MrKdJzA/sRAo6PgH8gzT1dk0E7w50bYP3ETAvjQVwiwALA6/Sf7DW37GXGu2/88B6\n", "k4Mx8/fFzHpOzn3ahBR0bANcQwo6LuhLiu4WErOQ9gruL1jr79h8wIvUF8DNA+yB88B6koMx8/fF\n", "zHpCVR7YHqQ6WKcCf43gmU72rQw54FyY+gI4SFXxnQfWg7w3pZmZdbV+8sA2rs4DG2pyntkz+cPs\n", "Pa7A30UkjZY0dfAzzcx6i8R8EntJXA7cDqwAHAh8OIJDh3ogZjYQj4yZmVkp+skDOwHYrtvywMw6\n", "ycGYmZkVZoA8sK8PxTwwsyI4GCuBpIOBNSLi0xXHfp0/nQx8GxhJyhv4aUT8of29NDMrznDNAzMr\n", "gldTlkDS0sDdwOIR8YqkWYGpwA7AosDdEfGQpA2AS4D1ImKypNHAqRGxVIv378rvi5kNLRLzMb0e\n", "2OqUVA/MrNd4NWUFfV+FRJpxWGOBTUQ8KulmYEfSD6aNgdciYlLVeVdLugxYnzRiZmbW1ZwHZla8\n", "UoMxSVsCRwOzAidGxE+r3l+UNJT9gdyXn0fEH4u6f6NBVMH+QsqXOJW0j9rpAJK2Ag4DlietZp0H\n", "uK1DfTQzG5TzwMzKVVppizw1dyywJbAysIeklapOOxCYHBGrAaOBX0gaKqN1ZwOjJY0gTU/+RdKc\n", "wN+BnwHvj4iFgYsBTymaWdeRGCHxbdJ/GM8FXiflga0ZwW8ciJkVo8w6Y2sC90fEwxHxNvBXYPuq\n", "c54kbSlB/vO5iGj75qtliIhngAnAH4EHI+IeYI788Szwbh4l27xTfTQzq+Z6YGbtV+Yo1AhS0nqf\n", "x4BPVZ1zAvD/27v3KEnr+s7j788M1+GOuKgwgsoQ0UQEFPESMiBZifGIuvEC3lZNZJPgLSdqNHFn\n", "ZnOicXfjNauiIkdFxRsGzCoo6hCPCgIC3mYQ0NEZUJAV5S637/7x1DjVNX2p7unqp6r6/TqnznQ9\n", "9TzV356Gnk//ft/n9/tqkuto7r55zgDracPHgY8ArwWoqluSvBL4FLAj8Hng7J5rhv+OCkljxT4w\n", "qV2DDGP9hIo3ApdX1cokDwO+nOTQqrql98Qkq7uerq2qtfNT5uBU1Rk0PXHdx94DvGeK89cCDx58\n", "ZZIWO/vApPnTWQ1h5VyvH2QYuxboXqJhOc3/7N2eAPwTQFVdk+QnwO8Bl/S+WVWtHkyZkrR4uB6Y\n", "NP86gylrNz9Psmo21w+yZ+wSYEWSA5PsADwXOKfnnPXAcQBJ9qUJYj8eYE2StOjYByYNt4GNjFXV\n", "PUlOAc6jWdritKpal+TkzuunAm8GTk9yBU0wfF1V/WpQNUnSYmEfmDQ6XIF/DPn3Ii1O0/SBnWkf\n", "mLRwXIFfkhYZ+8Ck0WYYk6QRNMW+kKfgvpDSyBn5MJbMz/6TkjTs7AOTxtNI94xJ0rizD0waPfaM\n", "SdIYsA9MWjwMY5I0JOwDkxYnw5gktcg+MEn2jEnSArMPTBpv9oxJ0pCyD0zSZAxjkjRA9oFJmolh\n", "TJLmmX1gkmbDnjFJmgf2gUnazJ4xSVpA9oFJ2laGMUmapYTd2NIHdjj2gUnaBoYxSepDVx/Yi4A/\n", "pekDez/wefvAJG0Le8YkaQo9fWAnARuxD0zSDOwZk6Rt1NMHtjtNADvGPjBJg2AYkyTsA5PUHsOY\n", "pEs91dEAABeMSURBVEXLPjBJw8CeMUmLyiR9YJuAj2AfmKR5Ys+YJE1ikj6wM7APTNIQMIxJGlv2\n", "gUkaBYYxSWPFPjBJo8aeMUkjzz4wScPEnjFJi4Z9YJLGgWFM0khJ2JUtfWBHYB+YpBFnGJM09Lr6\n", "wF4IPI2mD+wDwNPtA5M06uwZkzSUevrATqTpA3NfSElDz54xSSOtpw9sN5o+sGPtA5M0rgxjklpn\n", "H5ikxcwwJqkV9oFJUsOeMUkLxj4wSYvBbHPLkgEXc3yS9UmuSvL6Kc5ZmeSyJN9PsnaQ9UhqR8J+\n", "Ca8Fvgv8G3AHTR/YkVW82yAmaTEb2MhYkqXAlcBxwLXAxcCJVbWu65w9gW8AT6mqTUn2qaobJ3kv\n", "R8akETNFH9hHsQ9M0pgbprspjwSurqoNAEnOBE4A1nWdcxLw2araBDBZEJM0OhKW0vwCZh+YJPVp\n", "kGFsP2Bj1/NNwON6zlkBbJ/kazS3sL+zqj46wJokDUDCoTQbc3f3gb3G6UdJmtmMYSzJWcBpwBer\n", "ajZTC/3Mf24PHE5zR9Uy4FtJLqyqqyapY3XX07VVtXYWtUiaZ531wE5i4r6QrgcmadFJshJYOdfr\n", "+xkZey/wEuDdST4FnF5VV/Zx3bXA8q7ny2l+Y+62Ebixqu4A7kjyHzR3Wm0VxqpqdR+fU9IATdEH\n", "9grsA5O0iHUGiNZufp5k1Wyun/Fuyqr6clWdRDOCtQH4SpJvJnlJku2nufQSYEWSA5PsADwXOKfn\n", "nLOBJyVZmmQZzTTmD2fzBUgarISlCU9JOIPmF6pn0/SB7VfFn1dxgUFMkuaur56xJPej+U34BcB3\n", "gI8DTwJezBTDclV1T5JTgPOApcBpVbUuycmd10+tqvVJzqW53f0+4ANVZRiThkDCA4C/pZmKtA9M\n", "kgZkxqUtknwOeDjND+LTq+rnXa9dWlVHDLZEl7aQFlrCQ4DzgX8H3msfmCT1b7a5pZ8wdkxVfW2b\n", "K9sGhjFp4SQcAnwJeEsV72m7HkkaNYNYgf+RSfbq+gR7JfmrOVUnaaglHAZ8Ffh7g5gkLYx+Rsau\n", "qKpDe45dXlWPHmhlEz+fI2PSgCU8Efgc8N+qOKvteiRpVA1iZGxJkt+d19nmaLq7KCWNmIQ/ptkz\n", "8oUGMUlaWP3cTXkecGaSU4EAJwPnDrQqSQsm4RnA+4FnVfH1tuuRpMWmn2nKpcDLaVbJB/gy8MGq\n", "unfAtXXX4DSlNAAJzwf+BfjTKi5tux5JGgfzfjflMDCMSfMv4WTgTcB/rnKxZUmaL7PNLf3sTXkw\n", "8GbgEcDOncNVVQ+dW4mS2pbwWuAvgaOr+HHb9UjSYtZPA//pwPuAe4BjgA8DHxtkUZIGIyEJ/wi8\n", "FIOYJA2FfnrGvlNVhyf5XlX9QfexBakQpyml+ZCwBHg7cDTwlCpuaLkkSRpL8z5NCdzZaeK/urPX\n", "5HXALnMtUNLCS1hKc8fkw4Fjqvh1yyVJkjr6GRl7LLAe2BP4R2B34H9W1YWDL+93NTgyJs1Rwg7A\n", "GcDewAlV3NZySZI01ub1bsrOiNhbq+pv56O4uTKMSXOTsDPwWeAu4HlV3NlySZI09uZ1Bf7OWmJP\n", "SmIQkkZMwu7AF4GbgGcbxCRpOPXTM3Y5cHaSTwO3d45VVbllijSkEvam2SnjO8BfVXFfyyVJkqbQ\n", "TxjbCfgVcGzPccOYNIQSHkCzU8a5wOuqGP6VnSVpEXMFfmmMJBwAnE+zHuA/GcQkaeENYgX+03sO\n", "FUBVvXSWtUkaoISDaUbE3lbFO9uuR5LUn36mKf8v/O63652BZ9KsNSZpSCQ8imZa8h+q+FDb9UiS\n", "+jfracokS4BvVNXjB1PSpJ/TaUppCgmPA84BXlHFp9quR5IWu0GswN/rYOD+c7hO0jxLOAb4JPBf\n", "q/hC2/VIkmavn56xW9kyTVnA9cDrB1mUpJklPA34EPCcKta2XI4kaY5mDGNVtetCFCKpfwnPBd4F\n", "PK2Kb7ddjyRp7qZdgR8gyTOT7Nn1fM8kzxhsWZKmkvAy4G3AcQYxSRp9/WwUfkVVHdpz7PKqevRA\n", "K5v4+Wzgl4CEVwOvBv64iqvarkeStLVBNPBP9mZL+y9J0rZKCPAm4AXA0VX8rOWSJEnzZMZpSuDS\n", "JG9L8rAkByV5O3DpoAuT1OgEsf8F/BnwhwYxSRov/YSxVwB309w+fyZwJ/DXgyxKUiNhKfA+4EnA\n", "yiqub7kkSdI8c29KaUglbE+zx+QDgadXcUvLJUmS+jDb3NLP3ZTn99xNuXeS8+ZaoKSZJewEfBbY\n", "HXiqQUySxlc/05T7VNWvNz+pql8B+w6uJGlxS9gV+HfgDuBZVdzRckmSpAHqJ4zdm+SAzU+SHAjc\n", "18+bJzk+yfokVyWZctX+JI9Nck+SZ/XzvtK4StgT+BKwATipirvarUiSNGj9LG3x98DXk1xAs8zF\n", "0cDLZ7ooyVLgX4HjgGuBi5OcU1XrJjnvrcC5TL6MhrQoJPwnmiD2NeBvqhj+hk5J0jabcWSsqs4F\n", "HgNcSXM35d8At/fx3kcCV1fVhqq6u3PtCZOc9wrgM8Av+y1aGjcJy4H/AM7GICZJi0o/G4X/BfBK\n", "YDlwGXAU8C3g2Bku3Q/Y2PV8E/C4nvfejyagHQs8FvwHSItPwsOA84H/U8X/brseSdLC6qdn7FU0\n", "o1wbquoY4DDgN31c10+wegfwd9WsrxGcptQik/BI4ALgLQYxSVqc+ukZu7Oq7khCkp2qan2S3+vj\n", "umtpRtM2W04zOtbtCODMJAD7AH+S5O6qOqf3zZKs7nq6tqrW9lGDNLQSHkNz1+RrqvhE2/VIkuYm\n", "yUpg5Zyv72Oj8M8BL6UZIXsycBOwXVU9dYbrtqPpM3sycB3wbeDE3gb+rvNPBz5fVWdN8pqLvmqs\n", "JBxN0yv551Vs9cuHJGl0zftG4VX1zM6Hq5OspVmE8tw+rrsnySnAeTQbi59WVeuSnNx5/dR+i5TG\n", "ScLxwEeBE6s4v+16JEntcjskaQEl/BfgPcAzq/hm2/VIkubfvG+HJGl+JLwYeDfwFIOYJGmzfhr4\n", "JW2jhFOA1wHHVrG+7XokScPDMCYNWMIbgJcBR1exoeVyJElDxjAmDUhCgDcDT6cJYte1XJIkaQgZ\n", "xqQBSFhC0x/2OOCPqrix5ZIkSUPKMCbNs4TtgA8BB9L0iN3cbkWSpGFmGJPmUcKOwMeBXYDjq7i9\n", "5ZIkSUPOpS2keZKwDDibZl/WEwxikqR+GMakeZCwB81uE78AnlfFb1suSZI0Igxj0jZK2Af4KnA5\n", "8NIq7mm5JEnSCDGMSdsg4UHABTSjYq+s4r6WS5IkjRgb+KVZStgFOAI4CjgZ+EAV/9xuVZKkUeVG\n", "4dI0Ogu3HkyzXthRncfDge8BFwLnVvHF9iqUJA2b2eYWw5jUJWEv4Ei2BK/HATfTBK8LgYuAy6q4\n", "s7UiJUlDzTAm9amzOOvvMzF47Q9cQhO6LgQuquLnrRUpSRo5hjFpCgkPZOJ04xHAJraMel0I/MC7\n", "ISVJ28IwJgEJOwGHsSV4HQXsxsTgdXEVN7VWpCRpLBnGtOh0muwfwsTg9UhgPRN7va6qYvj/g5ck\n", "jTTDmMZewu7AY5k45Xg3E0e9LnU7IklSGwxjGisJS4BDmDjq9VDgMrrCVxWbWitSkqQuhjGNtIT7\n", "M3HE67HADUwc9fpuFXe3VqQkSdMwjGlkJOwAHMqWZSWOAvYBvs3EpSVubK1ISZJmyTCmodRpst+f\n", "idONjwauYeKo13r3d5QkjTLDmFrXWVbiYcAKmq2DjqQZ+dqOicHrkipuaatOSZIGwTCmBZGwPc1y\n", "Eito9m5c0fV4APBT4EedxyU04WuDS0tIksadYUzzJmEpcAATg9bm4LWcZvX6q7oeP+r8+VNXsZck\n", "LVaGMc1KZ+mI/dg6bK2gGfm6gYlBa/Pjx1Xc1UbNkiQNM8OYttJpnt+XracTVwAHAb9h67D1I+Ca\n", "Ku5oo2ZJkkaVYWyR6gSu+zH5lOIK4E4mn1K82iZ6SZLmj2FszCXsweRN8wcDYeuwdRXNnoxuiC1J\n", "0gIwjI2BhF3Yejpxc/haBlzN5NOKN3q3oiRJ7TKMjYietbh6R7r2plkMdbJpxZ8buCRJGl5DF8aS\n", "HA+8A1gKfLCq3trz+vOB19FMsd0C/GVVfbfnnJEMY11rcU02pbgvzVpck00rbnQVekmSRtNQhbEk\n", "S4ErgeOAa4GLgROral3XOY8HflhVv+kEt9VVdVTP+wxtGOusxfVgJgau7rW4rmXyKUXX4pIkaQzN\n", "NrdsN8hiaLbBubqqNgAkORM4AfhdGKuqb3WdfxHN/oVDZRZrcW0OWud3Pv5JFb9to2ZJkjQaBh3G\n", "9gM2dj3fRLNH4VReBnxhoBVNoWstrsnuVJxsLa5vdP68porb26hZkiSNvkGHsb7nQJMcA7wUeOIU\n", "r6/uerq2qtbOpaCE+zF5D9dBwG+ZOJ34SVyLS5IkTSPJSmDlnK8fcM/YUTQ9YMd3nr8BuG+SJv5H\n", "AWcBx1fV1ZO8z+wa4basxTXZtOISJt/ex7W4JEnSNhu2Bv7taBr4nwxcB3ybrRv4Hwx8FXhBVV04\n", "xfts9UV11uI6iMmnFXehJ2ixJXy5FpckaV5kTZYCOwI79flnv+fsQDN4kGkezPD6sF3bfd0twHqa\n", "HvLNj6trVd3dx1/70BuqMAaQ5E/YsrTFaVX1liQnA1TVqUk+CDwT+Fnnkrur6sie9yio1zFxpGsv\n", "4MdMPsrlWlySNMayJtvTfwCaTRCa7blLaLab++0kf052rN9z7wbuo2n3me5BH+cM47V7Ag8HDul6\n", "LAd+wsSAtg5YX6vqNkbI0IWx+dAJY29n4ijXJtfikqTRlDUJsDPNL9Z70fzjvNcsnu9IE1rmMwDN\n", "5dx7atUI/EM6ArImO9EMuBzS81gB/JJOMKMrqNWq+mU71U5vbMPYsK4zJkmLVSdQ7c7WQanfcHUf\n", "8Gvgpq5Hv89vNQQtDp2p4ANpglnvaNq9bD2Stg7YWKuqtQEbw5gkqW9Zk+2APZh9kNqrc90dzC1M\n", "3VSr6s6F+Bo1njq/DOzL1iNph9D8t3klW4e0BelLM4xJWpQ6P5h3BHbtPHbr+niqx/ady0ep56bf\n", "6+h8jTONWi0Dbmby4DRTuPp1rSp3EtHQyZrswdajaNP1pV1Zq+rWefv8hjFJw66rX2g2wamfc+8B\n", "bp3mcUvXx7cBd9HuHWWDvvZWZg5WN7c5nSMtpCn60h5Oc2PgjUwy5TmXvjTDmKR5lTVZQjN6Mtvg\n", "NN15u9A0QPcbnPo577ZxuS1e0sLq9KUdwORTnrPuSzOMSYtY5wfKLsxvcFoG3E7/oaif826rVXXv\n", "oP4eJGk+zLUvjdXcZRiTRkCncXo2oaif83akmX6bzYjSTI/bnMaSpImm7UtbzY6GMWkedU3T7dLz\n", "2JWJo1CzDU7bM/upuJkedxicJKk9WZOdWM0dhjEtOj0N4VOFppmOTXV8Z5rFHTc3ffc+JgtK/YSn\n", "O10nSZLGz2xzy3aDLEZT60xRPYimYXAHmsUPN299cd8Uj+le25Zre1+rQYSETmDaifkJSL3HltE0\n", "hG+eousNTJMd+2Wf597uaJMkaVAcGRuQzr5p+9OsGnxA58/ujx9EEwZ+RrNo4pKuR3qez8drs7k2\n", "TAxo2xLyii1TfMto9lubKhzNJkj1Hr/dhnBJ0jDwbsoFkjXZkWbxuAPZErC6Q9e+wC+AnwIbOo/u\n", "jzfWqrpr4SruX2cEqzecbUvIu50tgckFIiVJY80wNl+fc012Bh7M5KNaBwL7ANcyddi61jWPJEla\n", "fAxj/b7nmuxCE64mC1sH0GwTspEtAas3dF3nKI8kSeplGNt8zZrsxtRTiAfQLDfwMyYf1fop8HOb\n", "tiVJ0mwtmjCWNdmTqacQD6C5a2+yKcTNf95g2JIkSfNtbMMYq3kXE0PXUqYe1doA3OgaTpIkaaGN\n", "8zpjG4AL2BK6bjJsSZKkUTcyI2PDtrSFJEnSZGabW5YMshhJkiRNzzAmSZLUIsOYJElSiwxjkiRJ\n", "LTKMSZIktcgwJkmS1CLDmCRJUosMY5IkSS0yjEmSJLXIMCZJktQiw5gkSVKLDGOSJEktMoxJkiS1\n", "aKBhLMnxSdYnuSrJ66c4512d169Ictgg61E7kqxsuwbNjd+70eb3b3T5vVtcBhbGkiwF/hU4HngE\n", "cGKSQ3rOeSpwUFWtAF4OvHdQ9ahVK9suQHO2su0CtE1Wtl2A5mxl2wVo4QxyZOxI4Oqq2lBVdwNn\n", "Aif0nPN04MMAVXURsGeSfQdYkyRJ0lAZZBjbD9jY9XxT59hM5+w/wJokSZKGynYDfO/q87z0c12S\n", "ft9PQyjJqrZr0Nz4vRttfv9Gl9+7xWOQYexaYHnX8+U0I1/TnbN/59gEVdUb2CRJksbCIKcpLwFW\n", "JDkwyQ7Ac4Fzes45B3gRQJKjgF9X1fUDrEmSJGmoDGxkrKruSXIKcB6wFDitqtYlObnz+qlV9YUk\n", "T01yNXAb8JJB1SNJkjSMUmUrliRJUluGegX+fhaN1XBKsjzJ15L8IMn3k7yy7Zo0e0mWJrksyefb\n", "rkX9S7Jnks8kWZfkh502EI2IJG/o/Oz8XpKPJ9mx7Zo0tSQfSnJ9ku91Hds7yZeT/CjJl5LsOd17\n", "DG0Y62fRWA21u4HXVNUjgaOAv/b7N5JeBfyQ/u+O1nB4J/CFqjoEeBSwruV61KckBwJ/ARxeVX9A\n", "0+bzvDZr0oxOp8kq3f4O+HJVHQx8pfN8SkMbxuhv0VgNqar6RVVd3vn4Vpp/DB7UblWajST7A08F\n", "PsjWS9BoSCXZA/jDqvoQNP27VfWblstS/26m+WV2WZLtgGVMssqAhkdVfR24qefw7xa17/z5jOne\n", "Y5jDWD+LxmoEdH7TOwy4qN1KNEtvB14L3Nd2IZqVhwC/THJ6ku8k+UCSZW0Xpf5U1a+AfwF+BlxH\n", "s8rA+e1WpTnYt2t1iOuBaXcXGuYw5rTIGEiyK/AZ4FWdETKNgCRPA26oqstwVGzUbAccDrynqg6n\n", "uVN92ikSDY8kDwNeDRxIM5uwa5Lnt1qUtkk1d0pOm2mGOYz1s2ishliS7YHPAmdU1b+1XY9m5QnA\n", "05P8BPgEcGySj7Rck/qzCdhUVRd3nn+GJpxpNDwG+GZV/b+qugc4i+b/R42W65M8ACDJA4Ebpjt5\n", "mMNYP4vGakglCXAa8MOqekfb9Wh2quqNVbW8qh5C0zz81ap6Udt1aWZV9QtgY5KDO4eOA37QYkma\n", "nfXAUUl27vwcPY7mJhqNlnOAF3c+fjEw7YDEILdD2iZTLRrbclnq3xOBFwDfTXJZ59gbqurcFmvS\n", "3Nk2MFpeAXys84vsNbig9sioqis6o9CX0PRrfgd4f7tVaTpJPgH8EbBPko3Afwf+GfhUkpcBG4Dn\n", "TPseLvoqSZLUnmGeppQkSRp7hjFJkqQWGcYkSZJaZBiTJElqkWFMkiSpRYYxSZKkFhnGJGkaSVYm\n", "+XzbdUgaX4YxSZKkFhnGJI2FJC9IclGSy5K8L8nSJLcmeVuS7yc5P8k+nXMfneTCJFckOSvJnp3j\n", "B3XOuzzJpUkeSrP7wK5JPp1kXZIz2vw6JY0fw5ikkZfkEJrtRp5QVYcB9wLPB5YBF1fV7wMXAKs6\n", "l3wEeG1VHQp8r+v4x4B3V9WjgccDPwcCHAa8CngE8NAkT1yQL0zSojC0e1NK0iw8GTgCuKTZW5md\n", "gBto9vb7ZOecM4CzkuwO7FFVX+8c/zDw6SS7Ag+qqrMBquougM77fbuqrus8vxw4EPjG4L8sSYuB\n", "YUzSuPhwVb2x+0CSN3U/ZfINz9PHe/+26+N78WenpHnkNKWkcfAV4M+S3B8gyd5JDqD5Gffszjkn\n", "AV+vqpuBm5I8qXP8hcDaqroV2JTkhM577Jhk5wX9KiQtSv52J2nkVdW6JP8AfCnJEuAu4BTgNuDI\n", "zmvXA8/tXPJi4H1JlgHXAC/pHH8hcGqS/9F5j+fQjKb1jqhNNsImSXOSKn+mSBpPSW6pqt3arkOS\n", "puM0paRx5m+bkoaeI2OSJEktcmRMkiSpRYYxSZKkFhnGJEmSWmQYkyRJapFhTJIkqUX/HwF6+I8n\n", "/s90AAAAAElFTkSuQmCC\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.subplot(2, 1, 1)\n", "plt.plot(loss_history)\n", "plt.xlabel('iteration')\n", "plt.ylabel('loss')\n", "\n", "plt.subplot(2, 1, 2)\n", "plt.plot(train_acc_history)\n", "plt.plot(val_acc_history)\n", "plt.legend(['train', 'val'], loc='upper left')\n", "plt.xlabel('epoch')\n", "plt.ylabel('accuracy')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Train the net\n", "Once the above works, training the net is the next thing to try. You can set the `acc_frequency` parameter to change the frequency at which the training and validation set accuracies are tested. If your parameters are set properly, you should see the training and validation accuracy start to improve within a hundred iterations, and you should be able to train a reasonable model with just one epoch.\n", "\n", "Using the parameters below you should be able to get around 50% accuracy on the validation set." ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "starting iteration 0\n", "Finished epoch 0 / 1: cost 2.299955, train: 0.109000, val 0.125000, lr 1.000000e-04\n", "starting iteration 50\n", "Finished epoch 0 / 1: cost 1.777689, train: 0.341000, val 0.368000, lr 1.000000e-04\n", "starting iteration 100\n", "Finished epoch 0 / 1: cost 1.487571, train: 0.398000, val 0.435000, lr 1.000000e-04\n", "starting iteration 150\n", "Finished epoch 0 / 1: cost 1.648848, train: 0.450000, val 0.449000, lr 1.000000e-04\n", "starting iteration 200\n", "Finished epoch 0 / 1: cost 1.486470, train: 0.455000, val 0.462000, lr 1.000000e-04\n", "starting iteration 250\n", "Finished epoch 0 / 1: cost 1.494823, train: 0.462000, val 0.463000, lr 1.000000e-04\n", "starting iteration 300\n", "Finished epoch 0 / 1: cost 1.405308, train: 0.513000, val 0.496000, lr 1.000000e-04\n", "starting iteration 350\n", "Finished epoch 0 / 1: cost 1.296015, train: 0.554000, val 0.540000, lr 1.000000e-04\n", "starting iteration 400\n", "Finished epoch 0 / 1: cost 1.322547, train: 0.537000, val 0.522000, lr 1.000000e-04\n", "starting iteration 450\n", "Finished epoch 0 / 1: cost 1.326853, train: 0.564000, val 0.508000, lr 1.000000e-04\n", "Finished epoch 1 / 1: cost 1.443458, train: 0.558000, val 0.542000, lr 9.500000e-05\n", "finished optimization. best validation accuracy: 0.542000\n" ] } ], "source": [ "model = init_two_layer_convnet(filter_size=7)\n", "trainer = ClassifierTrainer()\n", "best_model, loss_history, train_acc_history, val_acc_history = trainer.train(\n", " X_train, y_train, X_val, y_val, model, two_layer_convnet,\n", " reg=0.001, momentum=0.9, learning_rate=1e-4, batch_size=100, num_epochs=1,\n", " acc_frequency=50, verbose=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Visualize weights\n", "We can visualize the convolutional weights from the first layer. If everything worked properly, these will usually be edges and blobs of various colors and orientations." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": [ "iVBORw0KGgoAAAANSUhEUgAAAdwAAAHbCAYAAABhpitNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl83WWZ9/HrTpqtTZM0+540bdMtdKF0Y+8Csi8qIA6I\n", "jI7j9ojO6AjOo6Ojo+K44OMoMyBoxQWQHUSgZSm0pS1d0n1v0zRpszX72iz380cDFmhyXUPJDbSf\n", "9+vFiyTn2+85+eXkXP2l51xx3nsBAABDK+K9vgEAAJwKGLgAAATAwAUAIAAGLgAAATBwAQAIgIEL\n", "AEAAw07kDzvnLhKRO0QkUkR+7b2//S2X85ojAMApxXvvjvdx905fh+ucixSRHSKyUEQqReQ1Ebne\n", "e7/tmIz/1Uf/5U1/7qkty+SyyWe/8X5+1FLT9V26usuUey0yW81MqrV9zpvG96mZuiN6RkTk8nWL\n", "1cxNC6abug4czFQz05Mr3/T+ivIqOTP/7X+udnWU2tV7eYzpdiVsLlQzZyRcYur61Job1Mx/f+m/\n", "TF2H9q9TM+nF40xdM08brWa2btr3pvcfXb5Erj5r4dtyGzdWqF25E8eYbtc/3fFPaub5+79h6qp9\n", "ZoaaOZjbY+qqm/FNNTN9Q4Gpa9U0/XadWRn3pvfv/8uL8rFL570t9+Ev/Jva9cyNnzPdrryCsWpm\n", "U8RxH3/fpnNljZo5Mkb//hcRaerWM5MTMkxdl/z0ejXzx0e+8Kb3H7p/tXz0Y7PelksfcZ7aVTDt\n", "GtPt+ua3/13NjGvpNHX1fKhEzaRUZ6mZr351/oAD90R+pDxLRHZ778u8990icr+IXHkCfQAAnLRO\n", "ZODmiMiBY96v6P8YAAB4ixMZuO/oZ9HFafkncJV4p/IS49/rm3DKmpBX9F7fhFNWybjC9/omnLIm\n", "lXD+9VYn8qSpShHJO+b9PDl6lvsmT21Z9sbbxWn5UpzOwH0vMHDfOxPzGbjvlZJi/d/cMTROlYG7\n", "e3ep7NlTasqeyMBdIyLjnHOFInJQRK4Tkbf9y/qxT5ACAOBkMnbsNBk7dtob7y9e/LsBs+944Hrv\n", "e5xzXxSRZ+Xoy4LuOfYZygAA4G9O6HW43vu/ishf36XbAgDASeuEBq7Fzs6mQS+/aHuvqWd1rf76\n", "WhGRjIitaqalMdXU1VR+WM2MjLW9TtJi9Cb99XwiIpuLpqmZ0tVrbNeZqv+b+qEXdpq6jiTqr9dd\n", "s8b22lkLb3s5oux5Vs+MyLC9zjs+V/83wdZ9+utrRUQODtulZrLajZ+kgdtZbspltg9XMw1n236Y\n", "lX6n/jrJpov+bOqKe2Sqmtn78ZWmLovi4jg9JCLZpRvVzP5k/bFERGR4Y7WaOavmLFPXvQXfUjNl\n", "Cf9q6rJ49unJplxW1atqZv79tuecfO/W+Wpm08t1pq7n2p5TMy2V401dA2G1IwAAATBwAQAIgIEL\n", "AEAADFwAAAJg4AIAEAADFwCAABi4AAAEwMAFACCAIV98cU7F4L+g/Xei/9JfEZHZvbaburtFXxSQ\n", "HaX/kmcRkZhDb//lyW/18HjbQg6RJWoi/dz/MDXNLUpXM8Nn1Jq6mlr1F4Xv3Wb4TdYics64VWqm\n", "bqptuYc8oS8wKG6w/cKqX3XpCx9SWhJNXWPH6r+wu3T9EVNXxd4ONZOSY+uy2H3oWlMuYkGrmjl4\n", "MNfU1VGo31enetsvQe+a26xmujedb+oS+bma6N3eZmo6sOwxNRNzsW1ZRfNE/Zezdw0fZ+q6IfaP\n", "auaJjdbHL13Eecv0kIisb9Bv1/O3bjZ13fn9H6iZSZMnmbpebNqkZly7vlhpMJzhAgAQAAMXAIAA\n", "GLgAAATAwAUAIAAGLgAAATBwAQAIgIELAEAADFwAAAJg4AIAEMCQb5raHD/4TB/Zats0tTW13pTr\n", "7ht8s5WISEu8vv1GRKTg4Go1E58XbeqS7XqkL7HaVFUxYriaGXXYtjmpanyWmikqjDF17SpPVTPt\n", "MfoGKatD4/XrExGpqz2sZhoTW0xdbTH6fWdEqu3+1dnapXeN0jNWEdlrTLnmbUVqpnO6bQNWUpq+\n", "remJSeNNXZOX6rfr4DTb19Gi7OA+Uy7lw/p5S9euJFPXtSn68XKzbA/by+t2q5nmR2zbtCw+fvZF\n", "ptxrf0hQM8u7/mTqevpXT6iZMydeaeoaXac/Zk4onKxmfi53DXgZZ7gAAATAwAUAIAAGLgAAATBw\n", "AQAIgIELAEAADFwAAAJg4AIAEAADFwCAAIZ88cWGSwoHvTx6se3F5YWjbUsO2vZGqZnuSNt11oye\n", "qmb29NkWAFj8fqftyzFqh76QY+Ohq0xdvq5UzRTU2xZfJObpCybaEiNNXRZN7bbjldimL0Ppre00\n", "dQ1P92rGt9uWjrRnj1QzvfVNpi6L3CW7TLnKKfoyl6qI5aauYUemqBm/R1++IiKSU7NIzdQcnGnq\n", "suicOtqUyxxdqGbG584wde3JOV3NJL+6wtS1slFfKLLz9HJTl6zTI+v/mGmq+rtv6I9NlT/RvzdE\n", "RF4p26JmlibZlnuc3ztHzfTM2GnqGghnuAAABMDABQAgAAYuAAABMHABAAiAgQsAQAAMXAAAAmDg\n", "AgAQAAMXAIAAGLgAAAQw5Jum2tYNvlmofFKBqWd/RYIpV56brWYm9OjbgkREtubrXZNeqDV1Wezq\n", "3GvKjavVNwHFlDxj6krOyFAzaXHtpq64hMlqZnZHnKnrcUOmvde20WnE2Fw9E68fBxGRxl2b1Uxr\n", "qu3+VZicpGZ6E1JMXRbVn9A3lImIDKvZo2a6bMuh5NLeu9RMyv4rTF3XF21SMzH79O1wIiK/MmR6\n", "ay83de1OfVnNpEfYvo77nl2iZtb1dpi6ql45qGYS5+iPcUevVI/Ejde31omIPPK7hWrmwxOvNHX9\n", "Z+R31cyel1aaum7Jna1mhh28yNB024CXcIYLAEAADFwAAAJg4AIAEAADFwCAABi4AAAEwMAFACAA\n", "Bi4AAAEwcAEACGDIF1/4cdsHvbxp2/mmnoTsQ6bcrPI6NVOVd8TUld7SomZqi1NNXaLvEpDZvbbF\n", "FxsNr8efsnSfqautVF86EN/yoqkrblSPmlmebVsKYdF3YIcpN7pIX5qSmqV/rUVE1jY5NdOX3Wvq\n", "Sp8Qo2YOxraZuizOqak35epyD6uZT9zbZ+raY1ges+DyRaauJ3dep3cV3GfqsvB/12rKpdfpjznP\n", "rbUthXhy1Etq5qqUc0xd5ZPHqJmCKZNMXfKoHkkZGWWqerzhp2pmd9MZpq6SS0rUzG3zbF2b1+tf\n", "7+1P32/qGghnuAAABMDABQAgAAYuAAABMHABAAiAgQsAQAAMXAAAAmDgAgAQAAMXAIAAGLgAAATg\n", "vH/3Nv+8rdy5oSsHAOB9yHt/3JV0nOECABAAAxcAgAAYuAAABMDABQAgAAYuAAABMHABAAiAgQsA\n", "QAAMXAAAAhg21FfwqVsfHPTyhPYGU0/vnimm3JxxZWpmUsJxX5P8Np1FU9VMwaO3mLqyHn9Ozdz5\n", "21+ZurZH71YzZ+yzfY77z/i1mskp/Yipa2N3vprZdv4sU9czZ1+iZs7+w49NXTNa9GOx46UyU1ds\n", "TrmamdvUauo6qz1TD03vNXWd/bX71cx1HznH1JWd0adf32WTTF2Hlz2tZnbH55q60obp348tMbZj\n", "/++3/FHN/NeiX5i6ml7ZpGYqfIqpKzOpVs0kjJtt6pqW36Fmho/KMXXNnqs/BnzuF780dS3763fU\n", "zLjr55q6zorT7xMr9u83dUVNblEzcUkJauY3c3474GWc4QIAEAADFwCAABi4AAAEwMAFACAABi4A\n", "AAEwcAEACICBCwBAAAxcAAACYOACABDAkG+a+tDiyYNeXjn8WVtR9n2m2PjHF6qZ9vl/MXX1bNAz\n", "tV/5tqlLDJumyir1TSciIiXpzWqmrW6vqWtt9fVqpkPGmrriRzWpmev+yXa7njFk2jbbtvccmZaq\n", "Zrrm7zR1Ra+OVzOldTGmrpLIbWpm/OhCU5fFuoY2U657aoaaOcPbPsfaKXPUzJaV1aau05IOqZmG\n", "jjhTl8W4xBJTLn3hKDVzxxJ9s5WISF3EaDWzfo2+vUtEZO2GKDXzL1/8O1OXRc4X15pyrf+nUc0k\n", "THnB1DXyYn2EtezaZ+pqaNmqZs4uucrUNRDOcAEACICBCwBAAAxcAAACYOACABAAAxcAgAAYuAAA\n", "BMDABQAgAAYuAAABDPniiyXVvx/08rQpsaaexr4cUy6+5G41M/qQrWv7qE1q5tWVWaYui+R821KI\n", "8p0XqJml2SNMXQtWvqpmDud2mroeWaDf/nmjbcsX5BI9MiHVtnwh3heomfTtxaaubP+8mmlosS0m\n", "eDQtW83cnWZbOmJxxYzLTbmi3P1qprux3dS1vV5fKDIp/rCpK7J3jJoZmxdt6rJobztgy+Xoyyq+\n", "c87nTV2v9JSpGf9Unalr9Tb92D+Q97Kpy2L0k0mmXFJxt5qpT7XdV1On9KmZng0zTF1nH9Dv09E+\n", "3dQ1EPUM1zl3r3Ou2jm36ZiPJTvnFjvndjrnnnPO2Y40AACnKMuPlH8jIhe95WO3ishi732xiDzf\n", "/z4AABiAOnC996+ISMNbPnyFiCzqf3uRiJzYgkkAAE5y7/RJUxne+9c3jleLiL7tHACAU9gJP0vZ\n", "e+9FxL8LtwUAgJPWO32WcrVzLtN7X+WcyxKRmoGCa5peeePt7Jh8yY7VnzEKAMAHQfmWCinfUmHK\n", "vtOB+4SI3CQit/f//7GBgmcknvMOrwIAgPe3/Mm5kj859433lz+0esCs5WVBfxKRFSIy3jl3wDl3\n", "s4j8UEQucM7tFJH5/e8DAIABqGe43vvrB7ho4bt8WwAAOGkN+aap9P9ZMOjlFZfPMvWck7LYlPt2\n", "j775ac78LaauC68pUjNFT9k2ZVlM3zrclFuX/5Kame1sz2NrXrBLzexwo0xdX/7pmWrmJ6d3mLos\n", "trRV6yERGdGlP4l+9oRCU9d+w5KfkdttT9of3lulZmr69E06ViNm5OohEWnYsFnN3HNohamrJKJW\n", "zcRPnW7qSijUt0jtrNS3GFk9+eAzpty1C/VNRrHXnG3quvDIpWomIce2fWztnbermf/47ztNXRaH\n", "Ym0bsLon6xviShIeNnWtXXexmpkTN+BTjN7kQMdbX/36dk2dE01dA2GXMgAAATBwAQAIgIELAEAA\n", "DFwAAAJg4AIAEAADFwCAABi4AAAEwMAFACCAIV98seDLg79Y/fD8+0w9G8ttL/a+YJy+POKW2v2m\n", "rl9tjlIzt8pzpq7vGjK7G223yyXPVDPTDm01dbVuilMz3W05pq6XoyvVzFfW25ZV/Ishk1gWb+rq\n", "G5+nZnZstt2u3NT5aqYpepOpq6leXyiyq7TF1GXRF7vOlHu01LB0IEVf9iAi8pmv6IsJ6pvTTV1/\n", "OLhTzcTHv3u/uKysVb8/i4j8/Df6wofrGjtNXXMX1quZGadlm7puvulGNZOVlWbqeuS/H9S7am3j\n", "ZN4+/THap9k+x/pl+mP07Cbbsp2pE/WFNcuqbI+rA+EMFwCAABi4AAAEwMAFACAABi4AAAEwcAEA\n", "CICBCwBAAAxcAAACYOACABAAAxcAgACGfNPUzZfPHfTyj8fbNvzc3JZiyjV/TN/Mk/XgPlNXfnmi\n", "min89WWmLpFvqom4a21//xm1VT9mPVN6TF1rT2tUMzfdecDUVTG2Ss2sntpt6pL/0SOHMm2bpiYs\n", "1Y9XwkTbRqf4qBo182ruRaauyT2r1cyB7mRTl8WWV/pMuW3b9ev86j9PM3Wdlf4lNfPvW39u6ipf\n", "3qVmRkutqcti4Xx9S5aIyLZd+kaq9eVrTV0rfqbnRhfYtnxd98Vr1Ex2Xpapy7JpavJYfUuWiEji\n", "pblqZvqS80xdLekVaua12foGKRGRh6tK1cysnu2mroFwhgsAQAAMXAAAAmDgAgAQAAMXAIAAGLgA\n", "AATAwAUAIAAGLgAAATBwAQAIYMgXX/z41eWDXr5jzjmmnrKJvzXlnl0ZrWb6xsWauopH6S9yvv+u\n", "ZaYu+YweaWtvN1V9sulJNfOqTDF1nWHYO/KHz9uWCZwZs0LNnPdiuqnrJ4ZMWr43dR0ocWqmqqHJ\n", "1BVTe5aamTZGfwG9iEjjDsPtKttj6rKI6So05a68/ky9K2asqWvDshfUzJ4dtmUokd2RaqYj63RT\n", "l4i+dGRSQompqTlLv+/sbyg2dXVV60tmKt0qU1fR+glqJirp3RsBl3+205QrKtYXZCxsftbU9ekz\n", "WtVMwjLbUpvzNs9XM9X/eKGhafGAl3CGCwBAAAxcAAACYOACABAAAxcAgAAYuAAABMDABQAgAAYu\n", "AAABMHABAAiAgQsAQADOe9u2nndU7tzQlQMA8D7kvT/uGjnOcAEACICBCwBAAAxcAAACYOACABAA\n", "AxcAgAAYuAAABMDABQAgAAYuAAABMHABAAhg2FBfwYrf/N9BL18XddyFHG9TUxVnyg0flaVmWjOr\n", "TF1z90ermUlx601do2/+vZopuu0qU9eImJlqJv6aWaauudW9aqbtpWRTV17mV9VM7Iq9pq6v/r5C\n", "zTz1yD2mrj8vfkHN7N+8xdQ1P/E0NXPB175g6sqaO0PNdLbrXx8RkQlJsWrmN/96t6lrX1m7monI\n", "2mXqGh6hf3/vf2K8qati3v1qpj1hjqlr8e0/VjO/fubrpq7H4/TzlriXbOc2Z350gZqJL40xdT2z\n", "9XeGK+w2dT106b1qZulj/2jqWtWVqGaerNhs6qot1h/nDi8rMnWdPWqSmvlrc6ma6fz+pwe8jDNc\n", "AAACYOACABAAAxcAgAAYuAAABMDABQAgAAYuAAABMHABAAiAgQsAQABDvvhid8eUQS+PjOkw9ewc\n", "UWLKpSeVq5m4VlOVbG7tVDPRbTm2MoPGVNsCgHmRaWom4dARU1dvj/4i+oyqMlNXY7l+vKZU6Qsa\n", "rIYtmGjKfWK4fp2/7CwzdT207T41s/mb+tIOEZHLb/usmrnqoitNXRaPPu1NuXFnnKtm+mbNM3Vt\n", "f+kZNZM7rtHUFX3np9RM7ZQkU5eIvviiJTrb1OR2pauZSZH7TF2pmfqD056sBlNXzG79WJw2M8XU\n", "9ZAh07hGPw4iIikV+nKiktY8U9czCfrj/Yh2233CN+gLkW68eZyaufv7A1/GGS4AAAEwcAEACICB\n", "CwBAAAxcAAACYOACABAAAxcAgAAYuAAABMDABQAgAAYuAAABDPmmqZaDg29FWX3YdhM2ZekbkURE\n", "0rqcmknpjDd1TYlJVDMVLaYqkxtjS0258sSpambkiDhTV2vn02qmt6jZ1LVlnb6FpWaBbeOOLNmt\n", "RjKqR5qq4medp2b+PjrT1PXAj76sZp6vtH2OLXc+qGamjTvL1GVx5cKdptyiES+rmfMfnGTqyp40\n", "U81UNo0wdU3+zFo1M315n6lrhSHTVjH4lrzXTRivf3/8VdabuhYatrWdc8C23a5jzCY1E1mrP5ZY\n", "7djWbspF90WrmeU9tgfW5JVdamZj9UFT18hEfXvaxt/YHlcHwhkuAAABMHABAAiAgQsAQAAMXAAA\n", "AmDgAgAQAAMXAIAAGLgAAATAwAUAIIAhX3yxvSR70MsbC0839VSXZZlyCTmj9a7u7aauvsOtaqam\n", "PsrUZTF10sdMuXo/+DEVEYmrrDF13fFcpZqZuL7W1HXNRfrf3xZm5Zm6fmfIvPjcclPXZZddo2bG\n", "jJ1u6rrxG/+oZup++aKpa9nWZWrmrqcfMXVZ3N07ypS78a4FamZz4URTV9f4H6mZlN5Pmbqm1V2i\n", "ZtZ864CpS67TI0U3ppmq6l7ZqmamrjvH1NU5PVfNNAy3PWyP/FmsmtnwNdv3tsXk1iJT7t65+sKa\n", "pl3Xm7pqLtWXe5z+yl5T19qIHjVzvj+sZga7Ns5wAQAIgIELAEAADFwAAAJg4AIAEAADFwCAABi4\n", "AAAEwMAFACAABi4AAAEwcAEACEBdWeKcy5Oji3/SRcSLyF3e+//nnEsWkQdEpEBEykTkWu9941v/\n", "/KEDSYP2d9XWmW5o3VTbRpS6tvFqJqku09QVUxutZiambjZ1WWwp02+7iMjqyD+qmRefWW3qGr1N\n", "PxY3TNG3d4mIdI4tVjPtLXGmrqN3rcFtftW2VSi/YKmaKc6fZOqafPq1auamz9i2ovU93KZmVj66\n", "xNRlMX3TDFOufewqNbNvrL6hTERkUpy+VWh3t23b3J6kn6qZyV/Ut2RZxdyjbx4SEbmnYoeaOffq\n", "aaau89r3q5n2vAxT1yOT9M1PT49INXX93pBZeqjF1JXQpW/TGuOeNnWlPqlv3VvlbMdrlte3p1Vu\n", "sW2tGojlDLdbRL7ivZ8sInNE5AvOuYkicquILPbeF4vI8/3vAwCA41AHrve+yntf2v92q4hsE5Ec\n", "EblCRBb1xxaJyFVDdSMBAPig+1/9G65zrlBEpovIKhHJ8N5X919ULSK283YAAE5B5oHrnIsXkYdF\n", "5Bbv/Zt+WO+993L033cBAMBxmH7Pk3MuSo4O2/u894/1f7jaOZfpva9yzmWJyHF/H9yW5+594+20\n", "MdMlfYzt16ABAPB+19a8Utqb9ScaitiepexE5B4R2eq9v+OYi54QkZtE5Pb+/z92nD8uky/8e9MN\n", "AQDgg2ZEwhwZkTDnjffrDv5iwKzlDPcsEblBRDY659b3f+w2EfmhiDzonPuU9L8s6B3eXgAATnrq\n", "wPXeL5OB/6134bt7cwAAODmZ/g33RHRkVA96eV/q4IsxXpccnWbKJaTHqpnINH2hhYhIQubb9ni8\n", "TVrVEVOXRdPK75lyc2P0RRTby2zLKtImJKuZopmmKpmyYZ+amZSfbyszOHSk3pRbva5VzeREbzR1\n", "ReRMVjOzp59m6mpvvl7N/KjzPlOXvKhHTou3Ha/1ifoClht22BZfLI/sVTMT235s6mqdlq5mNn9J\n", "XxwhIiLf1CNd6beZqiKW6I9Ne7MHfxx83aiUC9RM0cJDpq7IQ/pxffw522OOxecT9UUuIiKbn3Vq\n", "pn1hjKnrYKz+4PTZpHZT1/5nD6uZyEx9dnxnkMtY7QgAQAAMXAAAAmDgAgAQAAMXAIAAGLgAAATA\n", "wAUAIAAGLgAAATBwAQAIgIELAEAAQ75pqiRu8K0or+0bZ+rJaLBtyRmb0KJmDvfZtluN792rZqJH\n", "6puarPZt/agp13bOD9TMGWMmmLrOuPxyNRM/ss/U1awvdJIX1ujH1KqnR98qJiJSXq9vmlmyd5Op\n", "a1aPfiwKzp1l6ipKL1IzcybZtqdulyVqJrLQtu0oJ3qimtmUYvseamiYo2aK0vXvWRGRunGvqpnC\n", "H+eYuiw25PyzKTdzaqmauaLbdl9NLS5TM9tv0Te6iYick5arZq4uHWPq+owh0x2pbwITETlrznNq\n", "ZvVB22bBlpn67U9utj3mpF0xT83U1rygFz0x8EWc4QIAEAADFwCAABi4AAAEwMAFACAABi4AAAEw\n", "cAEACICBCwBAAAxcAAACcN77oSt3bujKAQB4H/Leu+N9nDNcAAACYOACABAAAxcAgAAYuAAABMDA\n", "BQAgAAYuAAABMHABAAiAgQsAQAAMXAAAAhg21Fdw/89XDHr5IxF7TT2N9z5sys2dmqRmJl11q6mr\n", "JOrPaubhNba/s3zr376hZv7ts98zdUUNK1Qz/pd/NXVVfyJTzWzpaDV1jZg5R81curTd1PW5p76g\n", "Zh6692lTV3TyDjUzMcb2dayJTVczcdX1pq4tlZVq5vCwRFPXl2/5upr51x/91tRl+Ryj4g6aulrX\n", "Hnfhzptsbthq6sqqTlAzr21sNnXV1P9EzXzoxw+YujaN0L8/UmsqTF2jh49WMyPzik1dqdEdaiat\n", "2Hb/+teS09XMdV/9nKlr7PA+NfPhT19v6ko4oj/e//CH95i6Djc0qJmFV3xEzXzxpqsHvIwzXAAA\n", "AmDgAgAQAAMXAIAAGLgAAATAwAUAIAAGLgAAATBwAQAIgIELAEAAQ774Ii1/8Bd8fzJKf5G9iMi3\n", "oqtMucq6AjWzMG7wZRyva+ieoWZS0pabuixOGz3ZlPv1tjVqxs0b+MXXx1q+6lI18+Hp+0xdF6zV\n", "Xzg+NfcVU5dFvv5afBERGb6rRc0MSzjD1HUoTV9qMaouxdQ1OTNZzdTvt3VZ7HFRptzK1Q+pmepy\n", "fWmHiEhXQ5yaSTmsL3sQEenLiFYzY88rNHXVPKpn4qPqTF1nVR5WM5vbmkxdK1bsVzO1I14zdZ2d\n", "mqZm0rvHm7osIppsS0dmnX+5msnMyzZ1Pf+ofiz+8JRtQc4Nc/TFPQtnn9jx4gwXAIAAGLgAAATA\n", "wAUAIAAGLgAAATBwAQAIgIELAEAADFwAAAJg4AIAEAADFwCAAIZ801SNst0mJUHfDCUi0rve9neD\n", "ptxH1ExO2sWmrvpX9W06STX6NherJx/fbcq1dkxQM21Ntu1QH0rXN+AU9bWauh6u0Y/XoaIeU5fF\n", "X5+pMeUyR+p384O7201dEzrb1MySPfGmroI8r2YOv7zB1GWxZ/HjplzZoWo1M7rcdrzKZunbtDLO\n", "STR1ze3UHytijYuAVhg2Tf1Dru12bRqVqWbGxurHVERkR2aZmmneY3ss3F+l3+97Ntu+ty3iR9k2\n", "mc0epz9mlpUfNHW98ueX1UxGlf49KyIy8eoPqZn8ohM7XpzhAgAQAAMXAIAAGLgAAATAwAUAIAAG\n", "LgAAATBwAQAIgIELAEAADFwAAAIY8sUXHW2Dv/B9pm3vhUQesb14eVPCKDWT2niaqatm7Fo1k9Rs\n", "WzBh0Z6ZY8pF9urLIyInjzN1te3JUDOrjqwzdX0+VX9B+7PDbAsALP7Y3mnKZa4vVTNLd20ydZVE\n", "6p9jbPdIU1dWbb6aKcjQlypYrbPtVZGYrA41UzHDttzjwuQRaqY42nbsO4bpxzWqXb8+q7Z62/d2\n", "SVSSmkmo7zV1TUjXv94bq20LX+LT9qqZ6v2bTV0WublZptyuPv2+c+hl/baLiBzaX6Fmxp47x9Q1\n", "b9w0NbPn1fWmroFwhgsAQAAMXAAAAmDgAgAQAAMXAIAAGLgAAATAwAUAIAAGLgAAATBwAQAIgIEL\n", "AEAAQ77ZooDKAAAUNUlEQVRpatvIwbe1zIsYbupJnmXbINNbtV0PZb9m6qo7PEPNrMp59zbbJG7I\n", "M+WyssvVzI6Xu01djafr27Qy26NNXU+tzFUzEycnmrosxqbpW59ERA70fl/NzM5/0tSVG9eqZraO\n", "1TeBiYhMODxazaTVtJi6LD6UtdqUu+4afTNP4vAiU1fMhAQ109bYbOqq3a5vKfMNVaYui9idlaZc\n", "zTD9OnsybZu5ckTfppWdFmPq2uH177VNFbZtWq8aMhnt2aau4e2H1MzOA1tMXY3xkWrm/DmzTV15\n", "xfrj3D13n9hmLs5wAQAIgIELAEAADFwAAAJg4AIAEAADFwCAABi4AAAEwMAFACAABi4AAAEM+eKL\n", "yOopg19+pm3ZQ9SwhaZcbbT+gunlu7tMXXHJo9TM+TW2Q/hdQ2bmafqSABGRMU7P7d6vv7hcRGTZ\n", "S5epmSMjbV05+QVqZk7Nw6Yui5svnmvKndbSp2Yyiq80dTUNb1MzEbX6i/FFROoPlKqZA+XJpi6L\n", "2z873ZQ7NE4/rgsnppi6mhv15RGHWvWvj4hIS2KNmllT3WDqsmhsOmDK5WUVqpm0WH2hhYjIsMIo\n", "NTMhK8vUVTBRf5xL3WNbHvM/f35ezURn276OOzfpjyfN1bbFPRPzx6uZ6VMKTV1VO/QFLG2ttu/t\n", "gXCGCwBAAAxcAAACYOACABAAAxcAgAAYuAAABMDABQAgAAYuAAABMHABAAiAgQsAQACDrklyzsWK\n", "yFIRiRGRaBF53Ht/m3MuWUQeEJECESkTkWu9943H61j/8OCbheI+/VnTDf30jdNMuSd/kapm1i+q\n", "NnVlXKZvFUk8vcXUZZEZHW/KvfbgHDXzwBXlpq6S4UvUTETUTFNXp9SpmYKaMlOXxYGHnzbl2o/o\n", "G3ByCptMXTuj9E1T6Z2xpq6u9fo2ndcyd5m6LGoTzzXlopduUDOPbqmwXem+WjWyzbi8Z1RDh5rZ\n", "lWU79hYRzT2mXJnXN1K9tmKjqWt6ygQ1s7Vwh6kr1fB4kpn/7m0yi2i1bd073HREzURG2rpmjc9Q\n", "M/Wdtg1Y7S36fbVxp+1xYiCDnuF67ztFZJ73fpqITBGRec65s0XkVhFZ7L0vFpHn+98HAAADUH+k\n", "7L1v738zWkQiRaRBRK4QkUX9H18kIlcNya0DAOAkoQ5c51yEc65URKpF5EXv/RYRyfDev/5z2WoR\n", "0c/rAQA4hak/KPfe94nINOdcoog865yb95bLvXPOD9UNBADgZGD+9Xze+ybn3F9EZIaIVDvnMr33\n", "Vc65LBEZ8Pdm7d718htvJycXSHKK/ivcAAD4IKioqZTK2oOmrPYs5VQR6fHeNzrn4kTkAhH5jog8\n", "ISI3icjt/f9/bKCOseNsz4wEAOCDJjc9R3LTc954f/XWNQNmtTPcLBFZ5JyLkKP/3nuf9/5559x6\n", "EXnQOfcp6X9Z0IneaAAATmaDDlzv/SYROf04H68XkYVDdaMAADjZsGkKAIAAnPdD9wRjnr0MADjV\n", "eO/d8T7OGS4AAAEwcAEACICBCwBAAAxcAAACYOACABAAAxcAgAAYuAAABMDABQAgAPNvC3qnbln7\n", "2qCX9zXXmXqmu0OmXENZkpqpysgydWV1RaqZ+ld7TF3fu/1MNXP13baV1Mn5Y9TMho27TF2trTvV\n", "zEdnX2TqWrthr5qplwOmrlW3rVIzy558wdQ1oq9ZzQyL6DV1tcfvUDNdB2z3iZ3VRWqmI2ekqev/\n", "fPxKNfONb99o6jpQW61metOOmLpqmuPUjPNdpq6W5iY1kxMfbep6+Oevqpmfffnzpq6Wjlo1Uz9q\n", "iqmrYIL+OcYdTDN1dY7LUzNJeemmrk/O0Tf5/u7qAX+HzZvMnanvQ4o+km/qGtWtH6+Djbbvxw3t\n", "+iza0DBCzXz/sYG/FznDBQAgAAYuAAABMHABAAiAgQsAQAAMXAAAAmDgAgAQAAMXAIAAGLgAAATA\n", "wAUAIIAh3zTV8+iiQS/ffsC2seYXm3abchm1fXqmx5m6Urr07UOlcxaYuixqum1bmHqS9Q0yZ1x2\n", "uqnrsXuXqZm1MQ2mrsIZ+mau5l2Jpi6LqJFrTbmuDfrXe2RGvakrol0/FoU1saau8pQ2NRO1rdzU\n", "ZVHyyatNuav7ktVM9Gj9+0xEpLWjVc1EDes0dXW06t+PHd22Lsumqej4w6authr9+zY+xfZQe7B5\n", "pppJybJ9D7lh+ramun22LUwWuTX61joRke7F8WpmeKR+vxERGTmxUM1k9Yw2dWUW68crpVffWvf9\n", "QRZucYYLAEAADFwAAAJg4AIAEAADFwCAABi4AAAEwMAFACAABi4AAAEwcAEACGDIF18Mq84c9PKC\n", "xCZTz82Z40256JIMNRPba3vR/hyXrma2nn3I1PXdp/XMmOLhpq7evho1s69ef2G/iEhTaoua6Ry2\n", "3NTVl69/jUoivanLco3lP1tq6oodqS+YqCnXl3aIiLQX1unXd0hfTCIicqSwTM30ZYw0dVn85dk9\n", "plxecqmaKWwea+rqiItTM+29tr/3d9brS0f6htnuXxYNkbYFJs0N+lKeuLRRpq6kOv1zTO/LNXUN\n", "n6DfD3e22hZMWLQm2boa1m7TQ222hRxtS/T7V2/B2aauzlklaub08balSQPhDBcAgAAYuAAABMDA\n", "BQAgAAYuAAABMHABAAiAgQsAQAAMXAAAAmDgAgAQAAMXAIAAhnzT1Jjs8we9fNKN+mYoEZGS5gRT\n", "LmNMt5rp6Rh8+9Xrqrv0jSg52/RNJyIi35X/UDOxYttQVJWvb06qWmXbgJUZr3ctGH2Bqau+slrN\n", "LN26z9RlEfuhi025rCZ9A07+jBxTV1pql5ppHJFq6irr1rfpxLSMMXWJ/E5N7Nmkb0QSEVnco2/T\n", "6Vu6xtRVF5WsZkam27YwRXbqx2tcp/71sUqKsz08lnbp28ei6vXjICLSbjgHSp9u287X1KFvyiuL\n", "rjV1WUycYPt+7Khdq2aqaypMXUUjjqiZjY22x5yGcn2DV1l6vqlrIJzhAgAQAAMXAIAAGLgAAATA\n", "wAUAIAAGLgAAATBwAQAIgIELAEAADFwAAAIY8sUXCX7LoJe339Vi6jmQlGTKvTRmj5rp3dFg6tpX\n", "qy9DqPcvmrosNq9Zbsp5Ga9mPnLlQlPXc4+sVjMb6vaaupojK9VMZ7S+HMNqQ1WUKbepaLaaSY3U\n", "l2OIiIw4UqNmaiobTV3dLfp9uj19hanLoniufr8REUko15fHNPSmmLrGNkWrmdqWelPXyKI0NZNe\n", "F2PqskhvOWzKVbfFq5lx0YM/Dr4u17AEpGLnBFNXwcRYNZPX1WHqsogZZVvuMXxOsZrpaygydbVU\n", "RaqZrhbb59iYrC98iei0LeQY8M+f0J8GAAAmDFwAAAJg4AIAEAADFwCAABi4AAAEwMAFACAABi4A\n", "AAEwcAEACICBCwBAAEO+aaq3r3fQyzuqD5h6St19plz54olqJnePbfPIyInlamZMdp6py2LaeTNN\n", "udeWvKBmhsfOM3VdmKofr2c327ZpTUwp1DMx55q6tsgDaubeLX2mrubajWqmbr/tfljQWKdmukYk\n", "mLoyO/QtTEcyc01dFtM6RppyI1L07T0HRmWburqa9M1cSaJvahIRaa/TNycVZ+tbskRE/mLINMXZ\n", "th0VJesbw9I78k1d7d1ZaqZzZJOpa33SbjVzsK7K1GXxfJK+aU5EJGtYjx7qtm2Raxurb4iL2KPf\n", "b0RE6rP1rq6oE9tkxhkuAAABMHABAAiAgQsAQAAMXAAAAmDgAgAQAAMXAIAAGLgAAATAwAUAIIAh\n", "X3wRVxA36OUdstfU0yBnmHJFO3eqmYpMU5XEO/3w7K5PtJUZZMsIU27meR9TM08tW2LqiozTX/ge\n", "lzfG1NXi9UUB+ekn9sLxY/WdW2DKJXTpCx+KT59su9KKFjUycqTtczxc1K5mciq8qWvrQ3omq05f\n", "ACIiMqxjuN7Vqy/tEBEZPkxfYFDbm2HqSkvcpmb2700zdVnsPqIvORER8ZH68pi+/EJTV2/HETXT\n", "ElFv6hpVfljNpCW8e4t72tL0+7OIyGbv1ExMj76EQkQkulA/Z6xOsi1DaY/Wb39HxXZT10A4wwUA\n", "IAAGLgAAATBwAQAIgIELAEAADFwAAAJg4AIAEAADFwCAABi4AAAEwMAFACAA571tk807Kndu6MoB\n", "AHgf8v7467Q4wwUAIAAGLgAAATBwAQAIgIELAEAADFwAAAJg4AIAEAADFwCAABi4AAAEMMwScs5F\n", "isgaEanw3l/unEsWkQdEpEBEykTkWu994/H+7C//9Mig3UnNGaYbOmL0KFNu4vAmNdOwZ6Wp69nH\n", "lqqZgoJkU9cn77hXzfzDpz9r6qraVaNmDh4+ZOqaOTpbzVx85hRTV/y0i/TQ4WZT14IbLjDlAOCD\n", "wnqGe4uIbBWR1zdH3Soii733xSLyfP/7AABgAOrAdc7lisglIvJrEXl9XdUVIrKo/+1FInLVkNw6\n", "AABOEpYz3J+JyNdEpO+Yj2V476v7364WEdvPhQEAOEUNOnCdc5eJSI33fr387ez2TfzR337ALykA\n", "AGAQ2pOmzhSRK5xzl4hIrIgkOOfuE5Fq51ym977KOZclIgM+i+cvD93/xtvjJpVI8aSSd+FmAwDw\n", "wTLowPXef0NEviEi4pw7T0S+6r2/0Tn3IxG5SURu7///YwN1XPrRj717txYAgA+o/+3rcF//0fEP\n", "ReQC59xOEZnf/z4AABiA6XW4IiLe+6UisrT/7XoRWThUNwoAgJMNm6YAAAjAfIb7Tq2srR/08ouy\n", "J5p6IjptfzeozNe3IlWseMHUVbdrl5opvvB6U5fFvoqNplxOzgI1c+2155m6PnLmNDUTlxRp6tpX\n", "rT9Z/e5Va01dAHCy4QwXAIAAGLgAAATAwAUAIAAGLgAAATBwAQAIgIELAEAADFwAAAJg4AIAEMCQ\n", "L77o8ymDXt49Y/DLX5fZ0GLKpTRWqZnXGspNXVWbK9VMWpvtdlkUJqaZchMuTlIz89Onm7pcm34X\n", "WFO639R1x9J71czSB3aaugDgZMMZLgAAATBwAQAIgIELAEAADFwAAAJg4AIAEAADFwCAABi4AAAE\n", "wMAFACAABi4AAAEM+aaplK71g16eHHGVqaenyrbRaXhbnJrp6+gxdXUlNKqZtBmJpi6LeRd+yZT7\n", "yFn6dq6WA22mrlVlr6qZB+9bYepa80KXmkmZOs3UVbHBtt0KAD4oOMMFACAABi4AAAEwcAEACICB\n", "CwBAAAxcAAACYOACABAAAxcAgAAYuAAABDDkiy+iMqYOenl6TKmpp6Yj0pQ7GK0vTDgYoS/HEBFJ\n", "7M1UMwea3r1DeN78IlOupqFSzZTuaDV1vfDwOjXzZPlhU1fS5JFq5szZZ5q6Nmx43JQDgA8KznAB\n", "AAiAgQsAQAAMXAAAAmDgAgAQAAMXAIAAGLgAAATAwAUAIAAGLgAAATBwAQAIYOg3TaUfGfTy3Vv1\n", "7UQiItFdL5pyvYfb1MzkEbZPe9uMEjWTXZtn6rIor9xmyu35o74d6uEDS0xdK0pHqJn8gkmmrpk3\n", "zFUz182/zNR1511fN+UA4IOCM1wAAAJg4AIAEAADFwCAABi4AAAEwMAFACAABi4AAAEwcAEACICB\n", "CwBAAEO++GJs0cxBL/d1faaebeW2mzp8mNczvsfU5VLPUDNN+52py+L7P19vyh1sWKNmYo5MNnVd\n", "dt1oNTPx4nNMXSnFg3+tRUQONOpLOwDgZMQZLgAAATBwAQAIgIELAEAADFwAAAJg4AIAEEDwgbt9\n", "1crQVwkRaWqsfK9vAgCc0oIP3B2rV4W+SohIMwMXAN5T/EgZAIAAGLgAAATgvNc3M73jcueGrhwA\n", "gPch7/1xVxAO6cAFAABH8SNlAAACYOACABBAsIHrnLvIObfdObfLOff1UNd7KnLO3eucq3bObTrm\n", "Y8nOucXOuZ3Oueecc0nv5W08WTnn8pxzLzrntjjnNjvnvtT/cY7/EHPOxTrnVjnnSp1zW51zP+j/\n", "OMc+EOdcpHNuvXPuyf73OfbHCDJwnXORIvJfInKRiEwSkeudcxNDXPcp6jdy9Fgf61YRWey9LxaR\n", "5/vfx7uvW0S+4r2fLCJzROQL/fd1jv8Q8953isg87/00EZkiIvOcc2cLxz6kW0Rkq4i8/uQgjv0x\n", "Qp3hzhKR3d77Mu99t4jcLyJXBrruU473/hURaXjLh68QkUX9by8SkauC3qhThPe+yntf2v92q4hs\n", "E5Ec4fgH4b1v738zWkQi5ej3Acc+AOdcrohcIiK/FpHXn6XLsT9GqIGbIyIHjnm/ov9jCCfDe1/d\n", "/3a1iGS8lzfmVOCcKxSR6SKySjj+QTjnIpxzpXL0GL/ovd8iHPtQfiYiXxORvmM+xrE/RqiBy2uP\n", "3kf80deC8TUZQs65eBF5WERu8d63HHsZx3/oeO/7+n+knCsi5zrn5r3lco79EHDOXSYiNd779fK3\n", "s9s34diHG7iVIpJ3zPt5cvQsF+FUO+cyRUScc1kiUvMe356TlnMuSo4O2/u894/1f5jjH5D3vklE\n", "/iIiM4RjH8KZInKFc26fiPxJROY75+4Tjv2bhBq4a0RknHOu0DkXLSLXicgTga4bRz0hIjf1v32T\n", "iDw2SBbvkHPOicg9IrLVe3/HMRdx/IeYcy719WfBOufiROQCEVkvHPsh573/hvc+z3s/WkQ+JiIv\n", "eO9vFI79mwTbNOWcu1hE7pCjT2S4x3v/gyBXfApyzv1JRM4TkVQ5+u8m3xKRx0XkQRHJF5EyEbnW\n", "e9/4Xt3Gk1X/s2JfFpGN8rcfn90mIquF4z+knHOnydEn5kT0/3ef9/4/nXPJwrEPxjl3noj8s/f+\n", "Co79m7HaEQCAANg0BQBAAAxcAAACYOACABAAAxcAgAAYuAAABMDABQAgAAYuAAABMHABAAjg/wOB\n", "cnRbWgIPWQAAAABJRU5ErkJggg==\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from cs231n.vis_utils import visualize_grid\n", "\n", "grid = visualize_grid(best_model['W1'].transpose(0, 2, 3, 1))\n", "plt.imshow(grid.astype('uint8'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Testing" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Test accuracy: 0.54\n" ] } ], "source": [ "scores_test = two_layer_convnet(X_test.transpose(0, 3, 1, 2), best_model)\n", "print('Test accuracy: ', np.mean(np.argmax(scores_test, axis=1) == y_test))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.4.3" } }, "nbformat": 4, "nbformat_minor": 0 }