{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Keras\n",
"\n",
"Keras es una biblioteca para manipular redes neuronales. Es una capa de alto nivel por arriba de Theano (o de TensorFlow)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Ejemplo simple\n",
"\n",
"Construcción de una red neuronal:"
]
},
{
"cell_type": "code",
"execution_count": 79,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"from keras.layers import Dense\n",
"from keras.models import Sequential\n",
"\n",
"model = Sequential()\n",
"model.add(Dense(output_dim=2, input_dim=5, activation=\"sigmoid\"))\n",
"model.add(Dense(output_dim=1, activation=\"sigmoid\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Podemos ver una descripción del modelo:"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"____________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"====================================================================================================\n",
"dense_13 (Dense) (None, 2) 12 dense_input_5[0][0] \n",
"____________________________________________________________________________________________________\n",
"dense_14 (Dense) (None, 1) 3 dense_13[0][0] \n",
"====================================================================================================\n",
"Total params: 15\n",
"____________________________________________________________________________________________________\n"
]
}
],
"source": [
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Así como también podemos visualizarlo:"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 81,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import SVG\n",
"from keras.utils.visualize_util import model_to_dot\n",
"\n",
"SVG(model_to_dot(model).create(prog='dot', format='svg'))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Luego hay que compilar:"
]
},
{
"cell_type": "code",
"execution_count": 82,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"model.compile(loss='binary_crossentropy', optimizer='sgd', metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Usamos un conjunto de datos de ejemplo:"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"def dataset(n_train, n_test):\n",
" n = n_train + n_test\n",
" points = np.random.uniform(-3, 3, [n, 2])\n",
" features = np.c_[points, points[:, 0]**2, points[:, 1]**2, points[:, 0] * points[:, 1]]\n",
" labels = (np.linalg.norm(points, axis=1) > 2).astype(int)\n",
" return (features[:n_train], labels[:n_train]), (features[n_train:], labels[n_train:])\n",
"\n",
"(X_train, y_train), (X_test, y_test) = dataset(8000, 2000)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Entrenamos la red:"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/50\n",
"8000/8000 [==============================] - 0s - loss: 0.7362 - acc: 0.4261 \n",
"Epoch 2/50\n",
"8000/8000 [==============================] - 0s - loss: 0.6680 - acc: 0.5556 \n",
"Epoch 3/50\n",
"8000/8000 [==============================] - 0s - loss: 0.6313 - acc: 0.6415 \n",
"Epoch 4/50\n",
"8000/8000 [==============================] - 0s - loss: 0.6026 - acc: 0.6461 \n",
"Epoch 5/50\n",
"8000/8000 [==============================] - 0s - loss: 0.5858 - acc: 0.6461 \n",
"Epoch 6/50\n",
"8000/8000 [==============================] - 0s - loss: 0.5725 - acc: 0.6461 \n",
"Epoch 7/50\n",
"8000/8000 [==============================] - 0s - loss: 0.5593 - acc: 0.6461 \n",
"Epoch 8/50\n",
"8000/8000 [==============================] - 0s - loss: 0.5454 - acc: 0.6461 \n",
"Epoch 9/50\n",
"8000/8000 [==============================] - 0s - loss: 0.5304 - acc: 0.6616 \n",
"Epoch 10/50\n",
"8000/8000 [==============================] - 0s - loss: 0.5143 - acc: 0.7059 \n",
"Epoch 11/50\n",
"8000/8000 [==============================] - 0s - loss: 0.4968 - acc: 0.7360 \n",
"Epoch 12/50\n",
"8000/8000 [==============================] - 0s - loss: 0.4780 - acc: 0.7723 \n",
"Epoch 13/50\n",
"8000/8000 [==============================] - 0s - loss: 0.4580 - acc: 0.7986 \n",
"Epoch 14/50\n",
"8000/8000 [==============================] - 0s - loss: 0.4374 - acc: 0.8245 \n",
"Epoch 15/50\n",
"8000/8000 [==============================] - 0s - loss: 0.4168 - acc: 0.8471 \n",
"Epoch 16/50\n",
"8000/8000 [==============================] - 0s - loss: 0.3967 - acc: 0.8641 \n",
"Epoch 17/50\n",
"8000/8000 [==============================] - 0s - loss: 0.3775 - acc: 0.8802 \n",
"Epoch 18/50\n",
"8000/8000 [==============================] - 0s - loss: 0.3593 - acc: 0.8908 \n",
"Epoch 19/50\n",
"8000/8000 [==============================] - 0s - loss: 0.3423 - acc: 0.9059 \n",
"Epoch 20/50\n",
"8000/8000 [==============================] - 0s - loss: 0.3262 - acc: 0.9139 \n",
"Epoch 21/50\n",
"8000/8000 [==============================] - 0s - loss: 0.3110 - acc: 0.9226 \n",
"Epoch 22/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2966 - acc: 0.9289 \n",
"Epoch 23/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2831 - acc: 0.9366 \n",
"Epoch 24/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2704 - acc: 0.9405 \n",
"Epoch 25/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2585 - acc: 0.9467 \n",
"Epoch 26/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2475 - acc: 0.9506 \n",
"Epoch 27/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2373 - acc: 0.9545 \n",
"Epoch 28/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2280 - acc: 0.9591 \n",
"Epoch 29/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2194 - acc: 0.9626 \n",
"Epoch 30/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2114 - acc: 0.9631 \n",
"Epoch 31/50\n",
"8000/8000 [==============================] - 0s - loss: 0.2040 - acc: 0.9667 \n",
"Epoch 32/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1970 - acc: 0.9683 \n",
"Epoch 33/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1905 - acc: 0.9717 \n",
"Epoch 34/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1844 - acc: 0.9744 \n",
"Epoch 35/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1787 - acc: 0.9745 \n",
"Epoch 36/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1733 - acc: 0.9754 \n",
"Epoch 37/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1682 - acc: 0.9780 \n",
"Epoch 38/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1633 - acc: 0.9779 \n",
"Epoch 39/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1587 - acc: 0.9793 \n",
"Epoch 40/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1543 - acc: 0.9786 \n",
"Epoch 41/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1501 - acc: 0.9803 \n",
"Epoch 42/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1460 - acc: 0.9815 \n",
"Epoch 43/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1420 - acc: 0.9809 \n",
"Epoch 44/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1381 - acc: 0.9848 \n",
"Epoch 45/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1342 - acc: 0.9824 \n",
"Epoch 46/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1302 - acc: 0.9853 \n",
"Epoch 47/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1262 - acc: 0.9848 \n",
"Epoch 48/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1224 - acc: 0.9860 \n",
"Epoch 49/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1188 - acc: 0.9846 \n",
"Epoch 50/50\n",
"8000/8000 [==============================] - 0s - loss: 0.1156 - acc: 0.9880 \n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 84,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.fit(X_train, y_train, nb_epoch=50, batch_size=32)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Podemos evaluar y ver métricas:"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2000/2000 [==============================] - 0s \n",
"\n",
"\n",
"Valor de la función de costo: 0.111558535993\n",
"Acierto: 0.983\n"
]
}
],
"source": [
"loss_and_metrics = model.evaluate(X_test, y_test, batch_size=32)\n",
"print()\n",
"print()\n",
"print('Valor de la función de costo:', loss_and_metrics[0])\n",
"print('Acierto:', loss_and_metrics[1])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Y también podemos ver las clases predecidas y sus probablidades:"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1760/2000 [=========================>....] - ETA: 0s\n",
"[[1]\n",
" [0]\n",
" [1]\n",
" ..., \n",
" [0]\n",
" [1]\n",
" [1]]\n",
"[[ 0.90671945]\n",
" [ 0.0555241 ]\n",
" [ 0.98214972]\n",
" ..., \n",
" [ 0.11941245]\n",
" [ 0.97730196]\n",
" [ 0.90484691]]\n"
]
}
],
"source": [
"y_pred = model.predict_classes(X_test)\n",
"probability = model.predict_proba(X_test)\n",
"print()\n",
"print(y_pred)\n",
"print(probability)"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQMAAAEACAYAAAC3RRNlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXt4VNd1Nv7OSOg6gEB0BhASg+UmduILOHGR7xIaDInF\njCRjbDQIEBjsBCREGJzGCDEubuo0JHY+J704seskbkhc0sT9fenXWs5Fjdo6aVMnji0ncRgG21D7\n+zmNLwKDwVrfH/vsOfvss/Y5ZzAl6vNoP888oJlz9tl7n7XXXpd3rRUiIky2yTbZJlv4dz2AyTbZ\nJtvEaJPMYLJNtskGYJIZTLbJNtmsNskMJttkm2wAJpnBZJtsk81qk8xgsk22yQYAKH23HYRCoXIA\n/wSgzOrvABHd9W77nWyTbbKd2xY6GziDUChURUTHQ6FQCYB/BtBHRD9+1x1Ptsk22c5ZOytqAhEd\nt/5bDiEdTCKZJttk+x/WzgozCIVC4VAo9BSAlwEMEdG/nY1+J9tkm2znrp0tyWCciBYBmAdgcSgU\net/Z6HeyTbbJdu7auzYgqo2I3giFQt8HsBzAqPpbKBSaVB0m22T7HTUiCvld864lg1AoNCsUCk23\n/l8JYCmAX3DXjgHIWv+S8hkD0FZVhU319YXfxgDsaGxEPpcDETk++VwO2XQag83NyKbT7DWmTzad\nLjxjj/KsAeX/2wFsqq/37VftS52LnOOKSIT9XX1WorIS2wF0Wd9vs77viUQCzSufy7HPGQXQGY8H\nXiPjXNJpEBH27NlTeF6P8rwxADsADFljDvL+uE93NOp4tvx0x2LGe9Qx7WhsDPzsweZm9lmDyr0j\nw8OBaEztKw/gCoW+i10Dz7G1tHi+J693HbgFHaTpA+BiAP8B4KcAngawy3AdEUCD1r/6ZxdAA9p3\nYwBlkknKptM02NxM2XSaRoaHaUdjI40p1+xobKR8LkdBWj6XK9y/x7p/O0B57bntAHXG4579qn0V\nxqL0NQpQTyTi+F19Vh6g9db38vcrrO/HAMqm05TP5QrzzyST1J9KFdZC/rZLW7e8NQ7TGql9yn4G\nm5vZ9zLY0kJERHv27CEiomw6XehXXa/m0lL2+2w6Hei9dESj7P0dsRg7XjmmfC5HnfE47QIoq6yt\n17NNc+i3+tgKUJcyH7l+I8PDrnGofWUB+sS7WIMgc+Fobl1pKY167AexzQPs5SAXnY0PlAVjicbA\nKLoqKhwT74lEChMvZsFVgupPpSiTTNK18Ti1zZjhYASFTRCQ0ch+u2tqHC9Qfm5buFA8t6WF2mpr\nqd/qOwtQhlmLT1i/EUDbmppcL/5W675dAK2IRGjDpZcW1jRv3dthWuN0mg7s30/LSksLBDdqzTGT\nTHpuZskMTEyja9o0T2bi1/pTKdoOJwPbDtDGRMLI/Lf19XkyY9Oz87mci0lvAqhPoUVuLVZo9/RY\n6y/7GoQ4YM5kDUwHi3w/Kg0WaLmlhTrjcd/9MGGZAXtqWZPmJAPuO7npJAPJWhunqIVubKT9X/sa\nLamuNjKnYji76bRZEYlQPpejfC5HqyoracAa8wBAKYZwvg+bEXXG444+89YGUefRVVpKQxYxy99M\n0lffokXUFQqxa78xkXBtEJUIv//973vOUx8rWf12xuOuE930jjbV1zvWZ1N9vYNJ5SEY4RqAlpaX\n0+WzZrEbod+6vzsaNT53ZHiYVkQitMa6VmXMXtIrRydDAF0fClE7QH/vszGLpR8/6dRPoiOaoMxA\nTnYUoARAtwDUaS1mX0MDbaqvd3Leigr21N4KN0PpKi2lkeHhohY6m05TZyRCPRZBFIgQzhO+s7aW\ntiQS1B2NUkc0Sv2plOsFcaeN3GjZdJo9+dZbv3MMcEdjI93R1OT4zevEUvs2XWdifBmAeiorxVhh\nSx3cepoYq66+jUKIr8Woc/lcjvpTKeqIxag7GqVMMllYgzzsk1uVHPR3xalepufK5/VUVjo2upf0\nqtPiTouGpWTmpZ6Z5pxNpyk9darvpi6WtmWbkMwgm07TzqYmWqGI+lLcGhkedog/cgNxE70iHC5s\nXiklqKewvtgd0Wjh2hHYEkVHLEbN0Sj1ai+wD069fx1HhPX1BR1y2+LF1BmPU6qqijotAlFVhsGW\nFqNOvEzru6eyssBs9BdtOrF2NjVRdzTq2BAcUXZGIuz9azTiz8P7ZNXfk1yHO5qaqDMep23Wv35E\nym0MndFI0dy0QQe0TTpguK4zHrfHt3ixY15SV5cbuh+gHm391lVVsYx7BZwSgyqxSntHkPkaGVAA\n9dfPhjYhmQFRME7mNdFN9fW0tqyM1RN3af2wxhbAwYg+XFlpPAXkyzYRYSIcLpxO3HjUuXXV1rIb\ncZXceLGYa+Pp4zcRejaddq1rnunXtEFbtft0RtLX0FBQdXQDmokYdakmyEnH0YY0wuoiusogdynP\nXl1ezl6nXsPp4lJNkRKWXL9bystpSyJBq2fPdkl2kpZMGzmTTBqNn/p8uXXvYQ43runMWb9nwjKD\nIDqO10RNhi55Qmxraiosvon4O2FLFTsMRNZt/X6HBxEOWsSwwrpOlVIkgUi991qFgaljaQswfynZ\nbAZorUYwkqC5Tdmjifojw8Mu0b0rFKK1V17pq2JsMRjyTO/jTCQDE214SRoD1rMGW1ook0zS8qoq\nXxFffT/qePzmop76AxCGWrmRORVm9ezZ1NfQYDy19fnK/iXtednBHHtDYzT69xOWGRQjGRRDMF0Q\ntgdVbzedJuop4XXye22OAQhxkjOG5gHqrqmh/lSK+hoaaBSg5XCrG30AbQswf/0EkXq9blySRjHd\nU6Bf0xmPU3dNDXXG4wX1TG50kyrSbmBmqnqib+BiXcBetJHP5VwbS6prss9sOi3mrK1zD9xenm44\nRfl8LmecS3dNDfv9UsXTJY2WqurqJcl5zjcATXjZbvR1mrDMIIiO49VMC7gdTteP10ZWT4lRuH3K\nqjEzD7hsClI1yBj6H4AtuqvEKQmky7pXelDWeRg/87kcZZJJ6qqooAHYkge3ZsUyWs7dunTKFF9V\nQv10xGLG01TaUnZa0pqf/uw3T3lNdyxGHbGYw5Crbmbd07TV8I7k/1dVVtJH5871tDdw3/enUp5M\n1MvGI20s0n6m0obO5IrZB8m6Otf3E5YZOIiQ0XG8dCz5u0sctoxuup5qcmN6YQFUXVh+t3zmTNoC\ncZostZhD3uNld1VUFIA8JobRC2E8XG29/P5Uit0gprlyhFKMCmZiyptbW13Mby2cDNG0IeR3fiCY\ndztP0/0DsHV4yQik50pn5ioNSCYg6UX1qiwLhym1aBF1l5S45nhg/37h/YhGHVKC3q++ZjpeYW1V\nFa1UmLA8kDiQk9+7Xjlliuu7Cc0MiiEKjpBMzITT+UYBSkJw3Hbwrjw/FcVk6PEyHBEJIM1q5oXl\n4bTgS+LXmaLXicQRSTGSgenajYkErVS8NVI3HgEjfltjLhYEE3QsujHYzxA3Arcqtg6gP4dQB3dA\nMHP9MFCZ+gjcnoTtFg0lIQ6EToAegVMl5Vypm+rrXSK7CTSn42k49Kq6F0xr1lJe/j9DMvATFc/E\nniCJ5I6mJkpUVroNbBD6nMr5OT2rGGlkFIK737ZwoecLyyST7OngpUuq4vIa7RrJSHoUD4j6zGJU\nMNPJYhL7peiuGtA4aaZYA3GQe7zmpRpYpb/fta6wxXC/E9yI5dDoRtqo9A0sjZmchJlNp+n2hQvZ\nd9oBp73BNI4Oy91rguVvTCRcXo8JyQw44lQ5fkc0Snm4dT6TVZUVLyGMdRJZpovzat/J2trAsQ4m\nacRL5RlsbmYZ0EpmLFkIdcXPlehnlPJzM8lmYrwmI5oOCzcxmTM55U14Ej9Dm66imAzGO5X/b2tq\nYqHIUg0qBn24AoyU0dLiKcUEQZXugNnLpcLkC4eYxnx0JOeEZAYc4XLYe91NY/K39qdSLPgoAydw\nxMjtIxFPTL6f/cJE3Loop5+oH4rFWKt3l6Jrmwilq6KCJ5KAMQDqmDlX5JZEwqj6BGEyftIJ9zuH\nPvVywcmPLsV4neoFl691qupeF+kCNklGWeb5u7TvOQalz2XDpZc61BATczcGfjH7SKdDFfw1cV2L\nGuFyHN/v5FMn3qOfVnBKAlJs29bU5PIY9EBYmTsMIJWdAV1jfiKsyf2jez5MRCdFSOlK9Aso4jan\niaGZNoWXbzxI85JOTKe8F7MJKsWYDMbSa8NFbkoMwx3WxuFcc+thsDVBAzP5BHzlc7mCzUDe287Q\nHsEtJar0zR0AXnQ4YZmB+uK7p093LYJJTNNPPhOBtEPojrqPfdvixZSFEBlXKC/XxHw4Qwy36Uyo\nORmgUwg71ojchNDjxNF22K6mYuwCXriDQAZKHyngTNqZ2BRMczYZjFVQmdw83Qw02ItZqy7MjYkE\nrVPATCqT0W0EXvMr5vCTGJBsOk3dsVhB5TXRopd6FpQZnNVMR37tGIA7Gxrw1lNPIfvii9hnfVet\nXDPOfHcMwPGpUx19jR854rgGAF4FEAJwIYDtpaXYdPfdmL9gAQBgxvnnI/OjH2EfgP1K/7cC2A1g\nr/XdMevvS06edPVfDWD86FHPcRwG8CCAr+TzqM7nC3Pe8IMfAAAe3r0b40eO4Ncvv8zO8+lIBMfG\nxgpj2QNgC4DP/eY3+LOuLlQ1NqLjoYew74EHMH70KMJz56J3717MX7AAhw8dKvT/xvTp+O3QEPYf\nP+7oa+PBg/h8fz/o2WdxYT7PznHaG29gz7e/jf+OFq6rY+cdnjvXeM/8BQvQOzSEfbt3O+YMAHue\nfRZ3HTxYmOOnIxHcPTaGC7X+GxOJAi3I9vDu3YV7YY3proMHse+BB/Dpxx5zXPvP//RPWH3DDbhk\nbAxTAGwE8BcNDWi49FLg9dfZ+R0G8DCAUwCeO3QI9da7UNutALaUl+MLFr3J93R3Po8HN2xA79AQ\nsHcv7l+6FLMOHizMZ09jY2ENAH4/VAM4Zt0TqAXhGGfjA9jRe5KD5eEW6zYB9FHtOw6EoXPCPPjw\n3g2XXkqd8Tj1LVpEKyIRFwClIIrHYg4OnDVwbD/JwHSfDuc1RfVJhKA8zTmXXhB1xahuWaekly0l\nKBr0TNq7BZ1x/alSzIH9+13rqoK6OIN1ECkln8sVMAXdsRhtSSRYO4c0SLNISINbsT+VcrzzAxDS\nzRqISFMuiC8o4GxFJDJB1QSyxXU9ilDCY0fBQzsd4qslfqsvwwjwYBiEyQ+uingcowqyCU0W7Q5G\n7RgFqG3ePCGORqO0ubWV+lMp6p4+PVCiEi9i8LKKdyhIvWJCboMaVP1aUI/HmTSJ+lRprGA7WrzY\nM/OUaW05BqbbfPKwIz0zySQtZ5CA+n3qekvaY7ESHghVdYymEPoJyQy8BqxaedfAqe/Jxdb9630N\nDWLzxGKsT57bnF4vRNdB5Qvuqq0twHW5jaASt8nWcE1JiYNA84Y5bQdoI/wTleinl66rmk79RDhM\nrbEYbbf6zkAwXy7WwW9D+KVSO5P2bvuR7lx9rXd5rMmAYU6ycaeupKs8+JBnUy4OCUXWGaF8hgkr\n0RmP+85dP2jl8yckM/ASZUaGh92ZbpQJ+QF1glrn9ReibvK2efPoVu2l+qXe0tvm1laXunIrQKvD\nYRcTVK9Tx5xUvg8iyudzOVpeV0ftEPDmJEAb4E7ysR3i5OHSi6n5GfzCbbn1Pxuiv8ndqecf8Lq/\nbd48Wg0bYyLnJ9PNcYy1OxbzlFI4o6CUOHZ40CabpcvD67OjsdF4qHXX1Piun+kdTUhmYLKg9y1a\nVMgE5Mi2A3FKjsHfvz4yPOySOnrgDT9mfd4K4UjuavQ9My+2P5WiWyFALB0Qet+1hnEkDS++A86k\nqX7JTdU4fJXZDAHUVlVFq2trfW0hphBlP1+/yUqur3OQ097YDzMedRNl02lWBZCuuDEICShjmLsf\nwpVLUjoK0DLLNuEVo1IMg8zncsZsVF6SgYov0BMH7cAEZQbG/ALxOHVEo6zRZTVAy+vqqK22NtAJ\nmUkmqau2lhLhMK2HG8AUBN+d1f42ofI4Q9NgczOLb+cCpLzCp/UsyqYEKNl02ngySeLvUMbvdTp6\nra/XhvdiFKbTntOBjf14vG8pTXbDlga4d7kTzjyRBeZvJW4xbTKvJKV9ixYRwcxg2+bNKxgcZaIT\nv8blnPCLapVjlHSyEqAbIA61PCYoM9i2eDEbzttWW0udM2caF7XdmpgeTcdBm/WXt6q8nNrmzWPD\naP18/fIZJiBJB5MWzGtz6kzGKx+CKmZ6nSqDzc3mk8nqS93opjXu8GF4XqqAF6Mw/bastJS2LV7s\nUNOMCVGY8RDxGZVdwByFhiTdSb1aja/gpBfT2KVtRUWY6u8xDTu61U8q0J99YP9+V84JU/Mag1yL\nCckMVG8Bl2POD4+9CUL/XlpRUbDaBtVruRdgsjPoQBKOW/fCNrypyUPzuZxRpVGZzJpwmHohdHg9\nSKWwmWtrCxl8+lOpQn6AOxTG5sV8BqxPJpl0nB6uDdTY6Bsf4CBaJj7DxChMp/0aiCxPqo2Gdbcq\na6KPx5RXMgvnGvQ1NFDnzJlGZmeSXkw5IzkGKdf1JjjtFSpzN9Ehl9rPZKzWm1xfo20JE5QZyJdt\nIt7rlJhx7uWOghG/A+q1epMuKE4t2dzaWkDpZdPpggqThRBHubyHUvTNptPUNmMGO49kXZ0recd2\nmHXZpeXllKytpURlJQ0xY93R2Eh/ft99lAKTEdh6hsytoG7kwmmsBbi8GyOgiVH42QF01550A+60\nolCl10OmTlfHY1Tf5DuprLTdtdGoEcVnlABMG4xhkN2xWEH6MNGviQ7Ve0zM2vQepNRq9DphgjID\ngjBqtYTD7EnIhgQr1/hZ1oNIBoWXZ0GhdTeUTEP20blzCzgGdaEHPcbRZkFWuRcq3aCcqNo8fbrR\n/y3/v9zwzGUWbmI5eGyGzK0QpP13+P9Hhofd1YmgnfY6AVuntR4joOv3JvUtWVtbcFXrHqpb4Rbd\nTYfITvjjMFR66vbYkPJd6KpIUJewSaqQxmPTAduOCcoMOL2mx9qAUn9TOa3OyY0hqlaIcxBfeFCD\nkxQx9ReU9RjHamUzjlj3y+fIsmjcGqyyMvB2Wxlzeplx+eEo/LwOXo3Tl88GbkC+jyEIYymHH5Gb\nRZ2r3DT6iamnb/djGJybdztEhilVtfOSXiTj7q6pYfElQVCfAxDZjDjEoq6eBcWVqOOWWAc9Yc6E\nthl46TX6i+Q29jJDWGdbVRVtbm2l7miUls+cWTAYbm5tFQi/6dOpMx6nriuuYIlD1+/0F6NutDz8\nk6iqfck+pO6vptjKQmRVVgOnuHEQPJCI2obJWkThV4nHRNBjAK0qK6O1emDOGeAG1E02AtCSAOsm\nMQ+qcZc9RAy1NlT6SSjYDvV5ci29XMxetgpufnKcXDr1hKWucGNR7TleDIV7vi5VmJC7E5IZmE5U\nL9eR+qJHhoeN0WM6QaVmzaK1mni61LDQLeClBNlnHnZZr0RZGd185ZV0i/bSOVirTN8u5yXToHlZ\nfvVNLk/EWyBONL34DFvYI2C+fY6giyVItenSxLbFix2bmbXRlJRQL0PAqmfBdIh4zTObTpsBPJLu\ntBDgbJov8lNMtqg8RM7FFeGwqK1pSbt+7lfVnhM0hDwIM+qpqAjMDN511GIoFJoH4CsAYhBBh18k\nov/FXftcPI5jWqTcMdh14fWowPkLFmDPI48U/j586BBeO30a+6wHhQH0Apiv9bEXwNJXX8XFAP7U\n+m09gD8A2MiuqlAILxFhljKmTwD4z/JyPHfyJB4EcJd17bG338adL72EX02bhn1vvIFxAD8HcK81\nDrXfHESZdRlh9tCGDSgF8CWlP3ntXQD2QUSsVUNEuh0G8Dk4Iyo3l5Zi6kUXYc7734+NmzfjwQ0b\nHFF7vZEIPv6d77gi9EyNi3YLG9ZJj9jUoyTHn3oKn3zhBVQDeA4icjQD4BCAuyGiSXuteZ6CoIf6\nWAz3/ehHrnHVz56NPSUluOvgQYwbxjN9bAz3bd+Oe5kIy/EjRzAbfARsDO5ISZXWCvPSokL1Fq6r\nw3MAHoVNj6sAXA1g/LrrcNf3vue41hStydG5jNB8Y9o0lBLhoQ0bEK6rw3plLOv37sWdP/xhYc1n\nAXhtzhx8/P3vx29HRnDeiRPYc+IE/so1ckMLwjG8PgBmA1ho/T8C4JcALmCuO2NxTOWEQXz4eQg/\nr/6cfsO9MgLNAYWur6eNiYTRQqympM4arrmurMzhApUejJsMJ5YqIS2FkFhkv3nY9gpVBWC9BEXo\n+WcqGXjpy3l4Sz7qqegHc86m0555GfVEsuq8RuEGna0FX0/Cr3E2lJHhYerWCtmugzCS63QcxFuj\nP8MvJZ+0galo1+VVVXTzlVc61AUElAzeNTNwdQh8G0Ar831hAipKMCgwg8icU9CU+lonnu0QYql6\nr6t0mLKpvCzEO5uaCuJcHm7xrDsUcoma8uXeYNps2nyCGAc94bgBCJ71czObSE8cqwOEVMOXiTmq\nDFu6ECWEdggKDD0SoQP79xee1Z9KuW0YCt1wHhM5L0fa89JS2mClxS+WEXCb0oTNMKkvXt4a7hnG\neBvFe8apXmvgVCd/J8wAQBxAHkCE+c046a5QiK6sqKC+RYsCYdflKTlobZyVSqTgGMCmJyfr+1W/\n93suH7vXi/cLkNIZyM6mJmr2CJPO53K0MZGgbu0FdoVCtMzCYEjGJjeVaXOpxifjBvTR8+Wc9QIm\nMgP0TiUdmLouW5V3kIVT6vJLKqqDi0YB9wmr1V7omTOHWi2XdAbO+JFUWZnn5ttpKLgatJmkFxNq\nc6chgW+xz/Dzng02NwdivOecGVgqwr8DSBl+91zYINBbjtvLUySTTFLHlCnUAYFsYzdPgA0SxCjj\nhWfPptNmQ6lisOLKnJmKwJj6U2HGxbikZHv79Nv0wmsv0I9e+hF9+7lv095v/RF1tF5Eqy6eQ4lr\n5tOH719O7V9vp7avtVHLVfMcaoAO/uqDjSY0EahEdupSRRCCHgNoaTRK2yFg1ipwaz34tO0qzRQr\nMcl7VUyK1/oXy4D1ZoqM9JI8+lMpagePXlXpISgzOCtpz0KhUCmAAwC+SkSPma7LZrP4/sgICECz\n9QGEUUU1AN518CD27d7tMKoAwsjT8dBD+PQNN+B+mRpsbAx7BgbQOzSEh6dOReav/xqvQhjiCkY/\n6+9e8IYwtekGtfkAtgHomDIFs2fOxLSLL8aU6mrWoCPvnwLecKUarK669lpcdeiQ49lPNDbi2JNP\nFu6bD5Fea2t1NY4dO+bo7zkAL732WsFA+obHM4kIL77xIn5y9Cf4yX/+BD/82T/jx1/9DU78tgaV\nM1/HeZ2E886LY05kDi4dWIk5U+dgatlUlIZLURouRUm4BP9waheq8RIAkcrrfjgNoJ8EsBIiZdzP\nqkvwy9MhfPHk6cL639nQgM9aBrXdTU14FSgYgn8OkbJOHXu19ZtsrwKY8/rrDmOqfKefB9A1MgJA\nGN8+39+PV558EmMATpWV4esvveRObcbQl9oOHzqEP77uOsRefBFjhrWNLV7sSrumpyML2jgD4yoI\ng/EDp0875vwpy3A6/tRTeIRZj1EIo+brAGqKGUQQjuH3gfAmfNbnGiIKFinodaL5ZZ9VseLtgCv0\ntFjJwNS/esroujQHHQ5SXturf9XA2Qu4yn31wQ2RXj93Bl33mWto1p/OotinY9R8bwu958o2mlJ2\nMwEDBOQJGKPGxh2Uy+UdY8nl8pROZ6m5eZDS6SxlkrZ+bKwhaP27+sLz6IbyCodBNjmznNZ8+hb6\nSMMc6gWfUyEPM034GY7bw2E2s7FJZfRLLa8GQeWZ8eo2G7ZwShGGXNO7v33hQjZhCWdUHbVovlCv\nEec40xGAqwC8A+CnAJ4C8B8AljPXGScdJPWUvNcUBNQdixXSoW1MJKgjFqPVtbXUVlVVdN2/njlz\nHITcM2dO4QVzL0A3bnWXlLgMYlwOPo5QTFWSVSL3MpAurSilloYSuvzyGtr61Y/SY794jF56/SU6\nePAQNTbuIGDMumWMgD4C+gnYStXVS6ip6Q5Kp7M0PDziurZuzjq6sWSat33C+vcPSqexvy+JTXeo\na/rvqqq4BigEcg0AtDIU4je1MneuQMkNVl8dsEN6XTYf5l3oQVB5axwdU6a4wE3FeAD8GILOWEyH\nU5A08RLGfs6YQdCPZAZy0tJgxQX9eIEs/LLKcJFvPZFIoaCE30sZGR52nbrdJSU0MjzM6nUmwm6b\nN6+o6D6v33VpyHQy3xCZQpdck6Smq8WmVk/7dDqrbG75GSOglwDnxo9EVhiu3UrnI02Xo4lWgk9f\n11U1na7G5ez4OqeUe44/VTbFkaVoFCgYWk0MaABCKsoD1Dl1auEU7QfoNo2ueiHsGjKrk9e7MAVB\ndVpxD1weTqnPny07QjE0YVqfjonODNSJZtOikEXbvHnUVVtLHdGoZ5VhjgOuR/BgJr/mlYCF49JG\nlFss5urbL5Aq6ClgmuOlVe91bOqGhr4CQ2huHuSGSUCHdU+egCwBgwSsMVyr9pGj85Gmq1FHixCn\ny9FE5+NmmjWznc4HP49FaPDc2B+YFjISOPfee2Cf9qOAIyDKlFJuO/jYB4Kz3kWyrs7lERoFXAhY\nXarVjb15mGMbgjZVYlDD2VVXstHIjAnKDEwiWVCLL+daHLAIQk4+qFXdJCKaLMcdU6awPvFrwKc0\n62CYgV+IddAyYpwO21U1nYBnlNvyBAxQbW0XpdNZSqX6iT/tu61rVelgwHDtAPNd1vFdbW2SgGco\nBef7TGEGAcOUAp9GfEdjI225/APOdTFsrK5p01yQYf1E7mbuG4QAc21ranKtNcds1mruzTaFEajv\nWrVt+DGwd5sWXk+3L92/SUM25gwmKDOQG3kAIs5eReZxE9FLj48MD7tKqul5Dv0kA86nrr4kTjKQ\nhhlpvNPRjevgBHlsh0jVZmJmprGZfl92xfl0S6nzmZusF901bRpl02maNbNdYwRO0b+hoY/q6zeR\n02awnYBtOrCmAAAgAElEQVQMs/nz1m/2teFwFwHLrO/zyv15B3OorExY3z9jSQ7X0nvDCwj4oXVN\nrqBqXFkx3ZGBSjVS+r1LXb/W3bIyGImVKCIRF2BI38SS2S+prqbbFi6kZF0ddZSWuhi1zrTUUuqm\n8UuQVLGGRj/jtm48XVtVRduamiYmMzBxShnQonNqrvT4xkTCAbXUs/2aipNIK68JDi0XVc9qNApn\nHnsddiuJpgV2KLa04gZJy6Zew2VU6gqDknd/mNZcewUbkXZpVSPlcnmKRqW4TyROa/fJnkxmKJ3O\nUkvLICWTGYs5jBLQxdB4nmprk1RevoqAFdZ1op/y8rXU2rrZxVxKS7sI2ErCKJkhoVYMUF3dcnY8\n6XTWQeztqT5KYYFj7bvgXI/t5y1gIby6wa/fogtjfkgtWtAL7SklhIyhLx0jUyiLVlPDMo+uigrW\nZtFTUWFUkYnMkqO0h8lMWHqKvwnJDEyckjuNTZsuWVfnMtoU0kRZSECZG6AjFnMsrp9bU820LC36\n12hhsFxYs+xnHWwdVuXaavOCpEp4qepKkjUl9I0iRO8FBGxV1AB5mvP2gZaWQcdYpPuwtraN3azV\n1UvIKSE4N7K8v6lpp2V0tBmGkEyE1NDUtM3lnWhs3EHDwyMO92VT0x1k2yJa6HykCRimptoL6Y5r\nrqBl155HtR+fQfv+eR+9deotlytZPxRWhsNGm44eLShp0MtT0g+3etYLIQVyrkVTWrYBA83L34JG\nKcp5+sHQJyQzMPqnmYrH0oU4AhHvL/ECo7CzBhVrrffKvsttXCK3VVkSi1EEZIguSBsfH6fbL3+/\nkXCFAfCHdD4upqsRp/NxMQnRe5BaWgYpl8tbJ/UA2UZB9wbm2vDwCEUiPeRUIXaQWWogisW6qblZ\nSBji5N9FQiLJK31kXYyjpWXQ6L40eTEikRUFQ+gzrzxDqf0pmnvnbFo6p9aBI5EqqMwiLZm6l2om\nm6QdrzD7bRZD6ICwSWSsZ5qqIedhTkdnkho4WvSKP/HM42mpHxOSGZhENhmwotaUzySTfM5D2Kel\n3s5UJ/fixlyVpe3wtt56ER3XXnjtBVr+yHK6/PIa4/idJ7/cbNsJ6C1scvuk3uba3BywiEgyghXW\n5l9CQsxXN7WfMVG1HTglAmCX8bm8q3PUUjXcTEllZPlcjrbMr2M3GcFpuS8GjjwyPEzNhgQ6SfDJ\nTnVa1GnMlHDEZPDTpVQvoyFnK5GfXUqfE5IZ9KdSLjuAHpCi6vdeVZJMNQtYbuuR7ntNSYkrAaqe\n/ks3zGyCOdtRewCik218fJy+9JMv0aw/nUV/9IM/ouef/6WRcJPJDLspq6qWs5tNP4lNjIDffKpR\nME/Aeu2aXtINh7ZXwZYI4vFO9rlEZlfn1KmdZLs4baakqjjqhpMq5FaITEo7IBKZ6uXnTaqZ2kxR\ngGnwmajkyaz2p9MgJ9kaq3TBab/S56rS2cZEgjrjcVo6ZYonU6GJygyInJFkS6qrC9FvnJ5t4npb\n4RSDfG0CCueW9RzVl6OGF3MbUXoZ5D0jAG2GO+ec9CIESSgqpYFFf7GIfvbyzwrfmwjXtHmamnYW\n7tUhxKaNKK81g4uyjr/r6pZTLNZBsVi3ZV/Ia/fkSagmcgNvNUoEsplAUPF4J/u9KhnIDSftNqyr\nEmYJ0tTUfqXdRjIW02bTIxR1RsVhI0Zg2yy8PFvqmHQGI8FYnJtZzx0xYZkBkTmGPmO9gI5YzBP+\nu1qfvLV4QarRmBiGn26pEoqM9Zc66moI2Ofq2bN93UO6NPD26beD0Klx86gqAmekM21I0d8ulsHY\n349Raek6Gh4eYcYhQUrbSJccSku7HPdwzTRezpYg5yGZ3VXRixx2Gy+jX1CbDVFxYcTSc7S6ttYR\nHq0eKl6oSc4mYDIq633oCXckHXaUlxdqk6jXT2hmECRM2HRaLzO9+HTa0xovm0mVMBp0LGKSY/Z6\nwX5pyY+/fZySf5aimssvpw9e+THf01u2XC5PqVQ/VVT0kDDqiY0ciawobDo/ZqE3IWmYIMrNBOy0\nfnfq67lcnubM6SHbfmF+rp+kYlJluO+dzCNHKSwobFSv/AmZJB/azLV8LufKeJVQ1Fr1QOrS6RW2\nZf/2hQupMx6nzqlT2XHJWhZBx6TvAVOcRndNDXv9hGQGpvjwIOASmTr9DsOLH2xp8bUZEPH6pgSX\neEkGcpG9qj55nUIvvf4SXfTHl1AklmZPPVNzboIR0k9heZoKnIFTzwbc7kTZBPMYJR2cBPRYzyFj\nH/Pmqa5Is/pSjKTi14QBdUCZ4w994c0tqKa5dWsDP5OzD6n1M+R3psS6uqRiSlhbTC0LOS5VcjAZ\nH2VxVv36CckMTOJTEAixbxmptLkyji6SyQQpqj7nBVaSbWR42JiuXRf91PavL/4rzf3MXLrgqg9r\nBC0s8KbTm0jdtFkS0GEZeiwfr7vkxgjYRAL0s8toxLOZjC1plJY2k40VsPuPRjscJ/SUKSuV389c\n9/drUkJYtKiPgAS5cQw/pFsqZ7A2g5WIEDBMwBi1rNgc6Hkm+lELqLbNm0dbPQ4E9d8xOCHMQYzK\nQVqxxVknJDNQDXDqywuSgFONS9BfvMQdBM0D6AU7Xl5XRx2xGHVHo7QlkSgUP8mmRapzLsnmeghx\nknsZDz/1MP3en/4effGJB11ivkD2jRhPbyKixYu3kfv01i3+qu7vhhKbTmRVHE+l+qm1dbM1Rt19\nKZiWDWluV67hoc9i3OqYhIcgFusIrBq5Q67VeQv3Znuqj7LpNN22cBF9MFxJV6NOwWCI509pXEKf\n+ZfP0Pj4uOczTZKl6qbrqaigpIlelX/l96vKyz3T7J1p00Pd1ZyR+nMmJDMobFCLIWQh9JwgueJ1\nMIeMb5CZjdUTXvXDFgPt5CDQapGVNeFwAdyyBYzfWRnzO+PvUOYfM9T4uUb6x38bsk7KrSTEcJXA\n11Mq1W986aYT1u3Kk78VZzsg0jeeCHAKh9vJlkLkZt5lMYIDBKxTnjNqMbadBAxQKtWvGRqLVxfM\nIdf2XCsquhh7gs44xih1005a+BcLad231nkabP0QqvLvDuZAUr0aqmfMS2IM0oLEL5iwCLJA74Rl\nBi4uqujkfr7gYpI/+L0A7j6/bDry5RLM6kpnPE4Hf/08bXxsI1354JX0k2efUgjVHDNgagKi6+Jb\nJFQNwUycon0wKLLa+I0npQ33ZhZ/HyBgKQlvQicBd5BUfSQiUsybBy3pQCndyGgOuR4kfd3MjGOg\nwHjGTo5R29faKLk/SSdOnWDXwSt2Re18ibXhshDZnToh3N3N4PEIxXg0fMfjEc1L4CXnoMzgrORA\nLLbJIiFqvji9kATXuGu4IiBqnkO10Ieas3D93r3Y8+STjvx1uYoKVJ844e4LoqDJwxC5B3cDeBN8\nYY8L83nsuWoRRtf/Ps4//GF86BN/hP/7f7+q9OS+6803q4xzbmyswpNPchn4fg6RRfBWlJd/CidP\nfkF5hvv6uXPDMLUjR7hxyUyOD8Nc8uX9AEIQWe/kKu7GtGkEALjoonfwwgvP4tQp95yPHh3HoUOH\nsXTp/Th40M5W+eSTezA01Iu6ujA7D5Hx8RjC4c0IhaI4dOiwYfzViMUOY2hoLxYsmA8A+Oaqb6Lr\nm11o/0Y7/nbV36JySqXjjvkLFqB3aKhQwOSZQ4dwdz7vKI5zDMB5AD4NOw+kzD94N0TuwVMQdPJZ\niMImau5LEz1y3z+8e3eBPgsrz+RvVPcA97YCtyAc42x8wJyg71Z/yudyrGV1FALu6WdD0CUNU37F\nfobbroYZjDIG0MLIBeQOGnq3Irx6Mtu6c0XFlZRMZgq6f0NDHxUjlpugwQLSbMIiDJIpiOkDH0hT\nVdVyEklSWoxz9gIeLV7shlQLFWsrSVuLnJsJnckZT0+9c4q6vtlFrV9upeNvH/elL1OKvrz1rncA\nlAiFnGnuINRgjt5M2b05MX+1lbkpry2+iqjNptOOcvOcMR4BJYNzzgzOlkXVyysgkYFZmEX5oElW\ntsMcuqpCknWR8mpcyzCAYPqzLjoPD49QMpmhUKid9HBiid1X+xkeHqF4vJNqaropHu88IwBQRUUP\nJRIbKRb7ELvRamvbqLy8w8AoblTuGSWnfcGes1kVkAxoiIQqsoOEKqIyArmeA1Rbu9oQNTnKRkc+\n/+uDdMuBW+hDj3zIqDKoNJFNi2xc14RCdAfg2qDLq6pch05PZaUrHNkEd+5SIPl55neu6piJVrmq\nYROSGZxNi6oRL6AshhcYJQ93khX15e9saqJlVnl0Y1UlCH2Ry8Aswm85BjBKkcgKamraaQTicP55\n288+am0Md5Sg7E+/v75+EyWTGU+Yci6Xp2QyQxUVXaRmTraTlTiNnonERi2HAim/6xmRRkkYHtcE\nAkrZRkKv392M1c6noK7LKBu09cvnn6eOr3dQ+9fbfVGgHP5AjSPww6jIPrqjUd8aEV6/qwdpfyrF\nBkBdW1ZGa5Sw+2JsBueUGegLXEyWF72Z3ECdcOLKg4r9urSiMhvTC8pYfemRlWsiM4hLQRaLdfui\nDk0bRGw8uQF40V0i9pz3q65GMY6Kii5KJLZQKtXvYBBmQ+IIiRNZMqBRqqwUkoObUaTJHbtAJNQK\ngZOQ3hN/FWgbcUFL9t9+cRVmhpJOZ+nk6ZO07KvLaMt3tnjSmle0a1coRH2LFrG0qAfIDcAMb5bf\nm0B1eiSmnvFLMifHQYfi0p6dUwOiNJIcP3gQzz3zDD41NoYLYRlgnnwSvUNDgasHq0Un/hnCWFMG\nUUxkFUTF3+cAbAEgTWvHIIpM/P8Avg64DDNrlyzBRfE4wnV1OK4YbtbDXZTlTgBvQVRffhXAPRAG\nyLnXX4/bduzEv274K8UwNguNjW85jFmmZjKGhUIRCHNUL4CPwWQkdN//MEQd51chTF534cSJajzx\nhDD2AdcD+Ad885t3orr6DbjLmUwB8A8A9ju+f+ut+1FVlUV9/et48cXdAF6BKNtx2hqn2o5BGDYb\nAWTw+OO9OHToMBYsmI+hoV7s3r0PR4+O41e/+imOHLkHonzMYYi3+cdwmuk2wi6vwplwT2nfnWKv\nO3p0HGUlZfjGym+g6cEm/OW//yVu++Bt4JrJSH0QwGs1NXjryBHW1Hl86lQAKBgCX4V4e9y1T0PQ\n63OG3xtvuKFgNLxrzRrcf+KEy6R7D8Tbmg/gVohq34fZGRlaEI5xNj4APN02Xu5AU2jxjsZGGoIz\nLZne7yhAVwJ0+/veR8ss3czEfVVwiR4+nbe4bTIUoqUQGW68RMMgIcTFSAbJZEbDAvDAIidiUZ7G\n5hNSnPgcyEheM0rh8E3cchVciAILMUrCwMdBnNeTQEU6VRq9OXM28OMtKbmeWls3G7MzCcOlP6pS\nYiGamwdpxY0fo5l/OIuG8zyCzyQZSJtRXqMHqb9vqq+nfC7nkGJHwOfoGIESeq39rhfgMUXzroJt\n3FT7wESUDFxuEggH1R6Yy54dPnQI9y9d6ixhZUkRvUND2HDxxfg7pfSY3u+FAFoAHH3hBdx7+jSq\nYOa+U5Q+PjU2ht5IpFDGbRaANxoaEKmrw7F//3fkT51CFsBWoOB6UuewYMF8PPLIHtd8Dh06jN27\nH8aRI+OYPv0NEJXijTeqUFcXxt6967F373o8+eQeh7utsXEP7ruvFwAKp+i0aQSiLN58swpTpx5H\nKPQONmx4CKWlR1Baeg9On/4z6/7dsE9m7ny7BE4ZaS/EGbPXevaDWLAgakkSzhWbOvU4du9+GL/9\nbTWEU00623qtN3AKwLMATgD4C8dKHT2qFk8T7fXXpwHYYN17kB3v5Zdfii9+cQuuu+6P8Zvf7LbG\nKSnjE6iungbgHhw/LuZ/+vQxlJZuwenTHwdQBeBLKCv7FYaGjuP48T8FLNl0TsNbuDG0Ev92x48R\nr4kXnnj40CGMvfkmNpWXo/HkSdwKQQtbAXzKenI1gBJr1cLWZxuAWS++iH27dzuk2HkApgNYA+Cd\nUAjvJUKvtTKPW6PpALBWmVXN+ec7JOZfv/wyS79vWf3cBad7MXA72xKA6QPDaaziuM8ke5Eptbmj\nX+XfrIn7wu3CkVmXZL761bNnu4uwwindmKLknJGH/OkeifTQ8PCIQ6pIJjMu3V7v16l3mzIdmzIW\n6Tq2SGlmjhgU95WXr6XKylVkGzTXkB4kBZAlVbhPZl0ysCUMaZfg3YVOl6QNcxbzk0ZW933z5rVR\nZSWX2s2WVi67/ha65M8voTdPvklEhorhALVOn06dM2ey9OaiQytvAef5GoMo6pKBbeMaCiAZbFu8\nmPU4pKz/62NBQMngnDKDIFZSvflFIhqLnsDp8yXrbzWhaRa2NVYPPtGZUzZtruYk55DCAppff6vH\nhlUJlReDpbU9nc6yvvaGhj4Hc3DXQ+DcdXmqrV3ObAYdvchvVDkHp7ehn3iVwLnBEoktvtGLvCFx\nE4mMSsW4JAcN8xcMzo8ZNjfvpp5v99DKR1fS+Pi4WT2IRIJXNIpGC3kO2ubNc1yThzt03xgRqdEi\nF6rfBgGV1vuYkMzAlRraUPZMtRF4VTgabG6mza2trnJoXJoqCeQwbegVPgFOg83N5rJmqLFciTl2\nM9knmUqoJoLeoTAAk57fTtL1JyQN9TTm72lt3Ux1dctpypSlVF7eQVddtYESiY0uBuGfEGVMGb+X\nRd+dlMRkPzHZSerqlhfAVOp9JpCReC4vGQhvDLfe0mMhIjx/8atf0sV/djF95adf8Qxc6k+lXEAh\nPZJQjWvZVF9Pq8vLHf1wDMSYkNWQdFU+ax1QAD7tAOh6ZQ8EZQbn1GagQj3Dc+dij1bOHHDbCJ4D\nsKW0FF9QylJvKS3F3fk8LsznhYdgzhzcUlKCGWNjOHLqFD5/7BgutPqTNoTVkQg+/p3v4MBnP4ve\nxx/H/W+95Sij/fGHHsK+Bx4ojK1XG1u4rg6nwdsafokb8GvY8FBdH7Yt/CrE1gS3PYSxMQnv1fV8\nCYqOQOjlH8eJE/fD1vEB4ftw6tJz5vTiBz84jXfeOVD47kc/2oLvfncTAGDdurV47bVq1NQcw0MP\nfYz1eBw6dBhDQz8DHEXgpcastmrU1BzEtddmC3YMaQ8xeVJMHpT3vOcP8Nhjd7nG8dRTr7vmKPw7\nt6O+/l6EQnfihRc+CeEZ+RIqKnIoK5Mg4QuV3p6D6rHI54/hhg/twb1//SfY+HgPNkSvNtqWpr3x\nBjZo9Lxp82bse+ABHHziCcx/5RVsg20lib34IiqsUUu7wm+ZWUsQuAuErUCaddj0yI9/jE8cO4Z/\ngNPjdSuAt2trgd/8hl13VzuXkkGQZkp11hmPF/LbszBgLT2Z/lFz1QVNkKlKKJlkkpbXVLjEuhTq\nLYnAPoXMkoEKlHHbDMRvWw2nPBcwtIqADIVCK60TcYSAASorS1FdXbIAbHImI3GevEESkLjF+FEC\nPkwiI5Kf56NYicO8ju61tO0F06cvLcCYZTi2LvWUlq4jFaVYWrrM+Nzd39tNVw9cQV2hkFNitOhR\nrfYl6x9KbxdXFGgb+NB3rp5jkDB8td3R1ORZvQkBJYOztdEfhHA0P+1xjXEyajvTeoR6ejITswja\nOFHsxpoSWtfSLPIdxGL00cRSml9/qy/R2+nId5HQtbcTcJO1mZaRCuZxJipRwT7SfSenxTGTbusa\nG1yUTGYoEulklixPodA1ZEIyqs25Wb0Zmo2WDB6DUUz+RpO9QNgyVGMsn+w1Hu8sqB2miNCWlkE6\nefokffDy6TQEd3bjVZWV9NG5cx200QtbJeAyHOl5C+V912sqbl9DA21MJArJVFR0rKll02lP9SIo\nMzhbasJfQfiVvvJuO1LdMLKpYpLf71w0ohodaWp61NjYm2+6XKFffu0d7Jtbhz3f+z4AIbIe2X4f\n3jrZjVAogsWLY7jvvl6HOHzo0GFs2PAtjI1J0M4xCMfUzQB+CuBXEBGIDYjHB/DlL38cGzbswcGD\nGwF8CzbYR4JupCPqYdhishzhnwMYgHB0CXDR3/3dMZSWboZTRD4M4HMg+j9s32Y1B3DGxVVDONHu\nAZBHPH4cQ0OfxYYND4ET+zl3IgAX+Gju3DD27u1l1QpTNOOJE+c51mJs7BJwY1iw4CJ873tC9Viz\n5i42InTu3DDKSspwzZTzkcBP8PvWrMchHKivV1TgoaNHHSv/JwCysKIZx8awOhLBfsstfQxAWVkZ\nqt9+WxsNcMEll2Df+96H8aNHcXzqVLz11FP43BNP2G/l2WfZNVPb+r17cfu3voVjx48bgU9B2llh\nBkQ0EgqF3G/Op3Fhm+v37sX2f/onxF58saBXPV9VhUsOHsRda9bgorY2bPnGN1w2hE2bNwNw61Oc\n/s+NQ8cybK6oYL3yEkdgh9/eDbmhnn12j6vv3bsfVjADspc7oAfBlpZuwZe//DHU18/DRRe9g5de\n2oKTJ/8/7T4VQWHCDRwG8FXHfadPP4BQqBNEf2t9/yW4GYnsO4OpU49jzZq7cOTIOOrqwpg27Tjs\nDag/d77V1x4sWCA2tmnDeoVRyyaESHPjcBgVFb04cUJfe177VsdgwnTs3SswHdMXXIBj//ITzIdY\ncVg9dmubDtbdryj/v+Cii7CvsbFAg7/3yis4Zm1yKH2V1dc7kIXZF190vhUmZFlv8xcsQMMVV2D3\nd7/rsKLsBkChkPE+VwsiPgT5QFCFr5og9XBZ3lwvoDIyPFwICsnD7XrpUUqimzIgF9uKSnRiPSeo\nnsuLtfy9icQWRZ3oZu4jsuMSTLgB/r73ve92qq5eQiUlKykcXmHsm6vWPHv2aqqqaiNeXZHPHTjj\ntO1cPkY1oIm7XvVO8GrJKFVVrXOMoaGhL3CGZkmrHzvvPJf+nqyt5d2IDJ3I1p9KuWh5O4TtQbYg\nCX1NbbC52eUuzxepJpxTZrCtr4+umDGDPgHQ92EbZPLKAqmuxKxhQ2bOcMG8FlLvLw++CrRfURM9\no5B3FiHnRwB0TOAhQdBS5+XyFggDGZ9jwKlD833X1SWZjeW2C4RCa8gZLryd6us3ubADQeHYpkzN\npaVdtHjxtkAFYXTm09DQR7NnryY7Ae2Aa4xBWj6Xo54PX0Ufem+E9nR1FSIG2Y3N0IlKY6bNKlux\n9i7OBZ+HcDNeC9A1AC1+73snJjMIkmNOrV9g8uuvhhMteCYGQrWZxtWduIouvayaBpqvc3kdvGII\n9FwE7iQdvHXfGfprjj+QjStmmkr1u6zo4vl+xsftVFm5ihYuvF0blwkDkKRYrJtisQ5KpfrPOAU6\nkV8Nh2xhc6dS/bR48TaKxzupqekOz1oLxRoxvdo74+/QZX95Gf3Ns39DRHx9hVWVlSxmRrZiMncH\n8SRweQ9XhsNuhGxDw++EGcQB/Nzjd1F1RuOM6qYPKhkMwM5FGMT14te4l/Cxxkb64D2X0cNPPcze\nw4m2VVVt1mnk3MCXXrqBbDdYhoDVrs1YXr6W+LJl7YFCn/WxqXDmmTN5b4IAL6nhwWNWYI8fotFZ\n1s1rDEFKvQnG6pVRSTKGXuIyMXNw7SCSGzdG07j/8df/SO+5/z106p1TROTOTmxKUy6bX9JSU/Yt\nr7qQ+t5QGYG6X84pMwDwNQBHAZwE8AKAHuYa6tLrEsBOLKnaDAKVtI7F2AU70zwJ+kv40tAD9P4v\nvJ9Ov3OavT6Xy1Nr62YKh9e4Tli9roEzwzHvJxdlzXnx/kxPXb+kpFxcghMBScZ7vcZlEttNMRbB\n6z76JVPRozfNkoFpjLq9RPY3Pj5OS768hP7y3//S9wQ30aCaNakYLAHXH6famiTpcy4Z+D7IwLWW\nlZYWxKuR4eGCcVGWZ19eV8dmdOlPpag/lRL+2GiUMsmkZ/FUv8VV2+l3TtNFf3YRPfaLxxwEK0+M\nRGKjZZwKtsmc1YXMJ61OnDJw6UwbD3ay++YNgaKku5QsUql+lyGOK5POP9d/08rGq1N6nQSTUXXQ\n8RypMgkbit1fSUk3tbZupubmQY8U9HqWJpuB/OilH9Hcz8ylgdU3G0V+E6OQtO0FsedUXVN//alU\nYKP3hGQG3JuUyEBu0murqqh52jRXGqdN9fXUM2eOSz9aW1XliU70WlyVIRx49gAt/uLiQtENd10B\neYp5Bcw4iUkyk1iMP93Ua4rNgWBqTlHZlkRqa5OWVHMTCdE7Q3ZRlyFXGvOpU1eT8CJsIwGayhAw\nSBUVS1lmZT9XlX6y1v3uTSabWkEpFLqedCOlKZLRyXzzFIt1UDTaTcBtBCy3mEiSgFuU+/3UEvuj\nqhbtX2+nDZf9PkvLUqrkELRSEshD5Og03a83k60hk0y6ixfX17P1RyYkM5AnfAbC8roLdpZkr9RS\neevfW6zqNJlk0rfGgWmRgxhyWh5uoa///OsF4hSEJfMCqvpt8SdgsW63d9Pc9QmFXcAWyd2GRFl1\n2RxJ6PZg6AzB5B0QyU9sFcq/DqRqZ7nV0KeqlvGGUTlv57V+745nWj849AP6wAenGWmIE9+lGpyH\nUHODVBCTzcvdyNkXOFvGhGQGuitGivw7GhtZPDdBJCPV1YPuaNQz2anXIntVYc6m0zT0b4/T7H2z\n6RfP/8qQn08WC+HFb6kbe53uZ1sC4Foul3e5HoHtVF6+kuwTl98Q0WiHQYw2uzv1Z/vbAPwqROuv\nSJz4TU3Sm7CTksmMpuP7qW3q//l3Z7IZyDY+Pk7vuet8+kjDXFa65A4bSZM6U3DgZyoqXNmUiYpz\nN44MD9OKSMQBnZ6wkoFrQsr/TZVlC8ZFWCmjKisL3gQW660HlngkOuWes3bONOp9ZKuH3itBN3o5\nspsomcy4XF1BrOnFtKD9msYfDrcof3upOZwYzV9fU9Pter5fJaizFbikMlYhwZmeKRmAWq16yBGr\nECTcmojoCz/+An34/uWs1Z9TQ2V4vHqAqZiDdrhL9BWMjUXUD9Wv64OImZjwzIC0xVlVXu6yAei5\nDMz/w5gAACAASURBVFsguGwGIt8bl3VoCHaEI+dpyCST1FVR4ayRCCfwaedN7R4JNDaQ0K3tXP46\nWu6/SxU4GwE9IiOR7tnQGV7W8FswyYDIvKFjsQ5f5nim62dm4FJFcmaJBm6iRGJL0e/l9ROv04x7\nZtCRN46wv+viuzRs+6m2Uh3uqq11oHOD1A/1UrMnPDPQJYPtsJNCrrT+HVEWSReruiHcjh0QBpnl\nAB2AXcw1SIGUNaEQ9cMJYCIIfYwnrFEqKdGj49wW/2JOtmLamYX66hsjo8yB07F7yDYU6mpGDwlj\nnLfNgOjdM8QzUaV4O4e0Ewwocy6+roTePvK/P0LZ7wd/nyPDw9QcjVICzghIeRBxNK4fUplk0ti/\n0bYwUZmByWawHaCNHothAh+pHLUXdiWlwv2KOOXFOV39ptOsa0pYuP03Y1CocrGtmH6F3s656kYt\nEJI0ztneASBBTgv+rTRvXpuVMkwaUEdI5jysrl7iGT+QSvVTNNpBsVi3Q4VSrzGpPGeqZsn7amq6\nrXn1k3eWaDdDDOLSffrlp2nuZ+b6FmAhshGLuiTbBSHJUgAaJ4iCP8WAkOT9E5IZZJJJujEUKngT\nVOyAKd47azEFE9cLaqE1cc4urRjFxxrPo3wux1i0s9Zm8t+MplM5Hu98VzYEr365/px5FMR8vGoT\ncpZ0uYnPLPDIfD13TUVFD6VS/bR//wF23MWsmXut5N86Q+XXNAjY6/IHLqcnDj7hOxav/Jky3box\noapyrVd5d6+6kBOSGXgtSns4zC5Ge0kJJRSjoM4oJEf1yk4rX4jJX5tNp2ndovPohub3+AQiBRPT\nOULXs+yciQ2BF4PdtRbVxtVd5PtZT1wlJMnoig888kb/OTMhq8Ci7RQKdbFzVGsdFG93GLXegW73\nMNlWdvky77uH76a+v+/zfW9e+TO3Wgyh1bAvVAO6HtikN9VWkUkmaVN9/cQFHcnILZdbpbKSNlx1\nlZFzssUqre/lIhvFLIuT+oGNrvjSFfT4rx8vLKzJZsDV7fNzH9pFRpybI5nM+BIS16+9kVQx2C5b\npl5rOqFzOTXbcbvVjzejCyq6e6kzZoYmGYIpi3GmqOSt+juQqER3IJeXO3IX6Vmj1Oc9/fLTFL8v\nXgCnmZrXIbgMwjY2Cjdcfz1EujTVtV5M9XLJHCYkM5DFIrdBGAh3Qog+ElrMhYWmrL/18M/U9OnU\nEYvRUkvMD+K7NQWBvPzmyzT9T6bTydMnHYTEbSSZxrypaSfF4520cOHthdx7pg3ilarrTNQF0Z/b\nEFZZ2ePoL4jB0WYubkBPJGL3V4yq4PVcb4v/GAnXn3utnF4Qfi5BWy6Xt7IlDxKwkYC1GnNaR6IC\ndMa1Juqcx8fHKX5fnJ5++Wki8o5J2FRfz8bZyNgcAui2hQtFifVYTFRk0ukZNnagmMC8CckMVpeX\nO1x6agmqbYsXUwbCS9ABFKz8CQNH5U586Zq5CW5Qk9fiPfgfD9JNj97EEg0nGptx9Ly47oXVPxNi\nFv3xJ5ran98JLU95OzeCOblIsb5/E+PwLsO+g0wSSjicMM5FfVdBbTK23YSzDY2SCDNfTn45Ivv+\nvo/uHr47UPBS58yZhQNNtZt1wJmgR9q39ANwG0P/QdqEZAYqR5QbVSZ8dAErrAXqhTtyUa8wo574\nXtmTTZw7tT9FX/3ZVz0XVBJbU9MdTJgvkXq6cTYEu5KSyjzyRXkY5BgWL97mWf9QNtMG5rIXS/ea\nDE7SXW3FekhMjDSR2GJYuxUWI9jqshlEIj3U2rrZyIz81CGOSdh1HU0xCqvY9wU4w7f/+gdfpabF\ntdQ5cya1Q9TxzEKI/gOAKMOu0J5Jil1XWloIgw6S96OYZD4TkhlITrcLQk2QBhGvwpZ5CFUhY923\nIhLxjB03eQ22NTWxnPsXvxylqZ+cSr85/htjn25i8w5y4TaIKU5AEqjfqeYeA5/RSGVEtjfBCZAS\npdSL21jFJAvxmo8I1daxDR8hEUhkG/sikRWFVO+5XJ6VxvzClTmmJ92Gtqplil7sJbe0IJ4jQVYy\nLZpu0xqFuxjwpvp62tzaSj0W4I2j92RdHXVHo5SsraVEZaWrEJCKOSjGdjAhmYHODdcDtDGRMNc6\nsK67trLSN9mDbCbGYgob/WhqKV3+wOWefZrdVDoBiQ0fjbqTkeRyeQv37kzDNTw8YnSxeev/bv+4\nqqJ4eR7cVZhEAJb0OHDGzhkzPmQxFm+PCGfFFxtbZCYSadv1aEZvRmP3yasxJqnFVFLNZogDJABW\nutSWIh1cJa7pIGAFLVy4wUFruvFa/zsPGy0roxZ1FSAPZxXw7RDZk25buNCdKxTF2Q4mJDPgNmNb\nVZWrbp38bQBO8SlIM+lupjLWPZedTx/53x/x7NNNbG7jnRAr+dqARFzg0ChVVbVRba2sXahuTmFP\nUO/3Ct7h3H3+hroBMs9lB7ndjF0kvSky3wHHCJwuwxFX36WlXQyz4SWt2trVSu4Bc01I01zNJdXW\nUEnJdUqfQwQsIZGBaoDModJibqWly2h4eKRwiOluQ/1vnTn0ww2l3w7QZob+5QG4vK6O2iFsahnY\nanYQ28GEZAbcm1kDUHM0SuuqqhyLkwboyooKOrB/PxEpaaamT/dNM8V5DUwSw9Jr4vSln3zJczFN\nbkaVOGbMMFfncfcRZAMOBiJ4kwHSO1BILTrixTTUubaTOC1tFyZvhFTnxEcuOpGcZmSnuF+GH7sZ\nlFTHnCqE7QoUlaS4BC5SZeuyNn6GnAZZLyOn6CMS6aFMMhVIMtCZQwaGgxFOiaEDQr3N53K0trzc\nsT9k9e8gtoMJyQxY0I/FFUet/68GnEFEjY10YP9+V1HLYiWGkeFhl5GyJxKheF89PfWfTxnvk7Ba\n3QCon5B+Bjbn5vTbgPb/VWt5UNeefwixnbhVwHa9CH+UhKvN6cJ0qzecDYM/8Rct6nMAoQTikPfO\nuNdEqhi7CiAqW4XoJwGeUhmNnsnZnZNBSATq+wvCIMeoPdXHllrXi7DqNgKvRL+6Kr22qoq2JBLs\n3tmO/8GSwVrt9O+DnV6aYAYONSspy9XvpRElk0xSdzRKHdEoGxMu/by9cAY2DQHUPjNEzz//SyMj\ncGY44sEnRP4ntzM/gOnkGVQ2Qt5xvxxPEBSgDaX2Ryua1QmZLJU3sDnnkydxyurzCS7NLF4sqyGr\nBlZ9rba65uSsl8g/r6TkWhI4hSwJKUB3JfaSUzIIpjq1tAwWpM6rGkppRX1dIbKwkOaspYX6UylH\nBiJT4tIWw/ftZWUs82gPh//n2gx65sxxpJfeBNt3Sh4cs9PESadNc6V5UrELsvnluTdx17PlW7eJ\nXRoQ1cAfu1/hS5ff++P/TRZ7d9qxnQR00tSpnaxh0zsxKM+4nBKFCfcwatkI/KUZb1VMGFwrKq5k\nrtmqbGxeyikra6fy8hbr3m3MRu8jgStQpRuJNegjURPTrW5Eo3Y4dnJ/kh595lH2XRE5k6GmQiEX\nPaYBaoI7czgBdJ0Bjv+hGTOMz1PbhGQG7OkOf8ngasP3S6qrjYZHdYN3RKPsdbICjknvOlu+dSLp\nWnQaGNWUXBUVPZRIbKRkMqNk87nDaKjzYjxBmZhUgWbOXE7l5UupttaWeuRcamuTbF9uSYdHRErE\npp80w3khdPUkHO4iYZSU48iT0xNggha3k+19aTdc00YCjdhCwEoCriZhWCQChph4CfvdNTbuoA33\nbaDUkvexSXZNxU6kbWA7RNi+jrORqvIfgDc4Lq+rY9dSbxOSGXA7ayecbhd90t0Q4jyXz+D2hQvZ\nPgchUqnLF9IdjbLXdePsSQZ+zRQpGA63kioN1Ndvchni9NPUb1xBowbF6e90T+olyEzRhSJDtITx\nyvGoLkM+VsILT6Ey07o6ngnZRkUit1rgdreKv/utv0dIuA05cugjDo69cOEGisc7C3aO2tpOckt1\nz9AtVVNZ9KHu3eKqJZsOwQEI9/tW8K7IzpIStkqzDq6bkMyA1YcgxPWMNdFegD5k1URQ0YQqYKkF\nAp/glaNgQHkhJtfldoDWlk9h7QxBN1XQ5p15SNWR3x3MmNtY3MbzgzTrnoJEYovlgbAZV1nZKhI6\nuOqr520rxa6lN2xZukVVFUGuYZ7sjMjtJHI0jJAtuXhJD241RTdsujEaROfDTYdDEJLryilTqB12\noh5u45vKqd8CUEs4TN2wjeoq/cq++hoaPNOuTUhmoA+yr6GBeubMYXX+keFh44kukzyMDA9TW1WV\nI3OMDPBQ/bD5XM5lW+iyGI9f/IINQ/Yu63WmOQnt000aqPw3+tmQWMRm45/F13AweSdUvILbmh+0\nqElx69VtMSAu1HmInCnR5fcS2ORWZxoa+gxVp8wxJep3V+Max40jcCMQ11nfc1DkZYrngQzfy70h\n6bUHTkSilG65A3JCMgPO/y8jGVXxZwginHMAdlSXWnFZikY6c+mCyJikclBpDwgav8A1rwhGv+rB\najyBf5EQ82ktGVE83kkXXLCKSkpaSUgVHQT0FtCMQYN1vCQDPjMy7ya08Qpk7E+OyY/J6Wvu7W40\nbdRrDd8vIaf0ICSKWKzDg1kFmfMYLYpc4NiAqh1MpS9pCB+FM08nV/ynR0Edqn20Q8D0ZYq/QhCT\nVX+EQ/NOSGbANX3wedjZX1gOa+ELggRzmDZ4saWvTaeUX+gvn2BjGYkTTSL0VDF3K2sz0BOjiJBb\nZ4GR0tJOqqx0Btf4eSNMNgPh+dCXh4+tkHgFr+zEUlXhxHCvBCL79x+w1svOeGQb7lTchmqrSBCX\npMXttrUZlZn58NKQnLOUFi9buJnWRGYU6NFUJEXaqDyrfFkMwlQ6oAO8DU0G73Eq8YRkBlxsgb6p\npV2AIGwJXGk1tViFbljZqiyQSfQ3MZKrohexRGk61YR7zcwonMVX7N/c6dblpu8qZCKSG8wExdUB\nMOI53gVAOIaQSvVTbW0blZd3UG2tKMnOl2V3e0J0acRUskyuqROz0UvOHAWm2AopCewg4BoCblTW\nkBf7uXqXbvCSm1kKJugNp1ajIZ2M/hl6b3kDffSyS4xerhbwAUZcNK2JRpfDPizl9zJ0vzsapbZ5\n8+hWjVFMSGbAbVDO2irVgR5tUmrqp2w6zWdAYmo3cskmts6f57gvhQUE5Fgi8ZYMTBGMNnTVDVjZ\nSu5TRxjeamtXO5KluEuky89O7W/VBuAWg70Ygn8BUl78b23dbEgtxm/w4eERmjevjUKhm0gY7ILG\nG6gbPm+NZzl5GwQHyLT+NTV8VWvxTP19irXU7zHRRPOKTcLepdHlOghJV5c+vWoz6t+vKy+nEQik\nosoI9D3QZzENuZfOKTMAsBzALwD8CsDHTcxADlYX3VVAxpLqahoCqFnjfvJeNXhjhQIvVq+RsQte\nySYe/t5f0TVXzKYbY3E6HzdbjMBNlPyGEZF4ixb1+eQ24P4eo6oqScgqwfFowXA4ReKkc4rn4gTT\niX+A7csrtt9E1EHEf75u5CjV1SWLSAjjRvUR6dIYN0bJTMxAIyF5uCUzk7TkBePW7zFJixd8cBMR\nEW1MJGgFBH6gBcKWpSYwkc2rYpKe07A5GhUSAGzvgsktmVGYxDljBgDCAH4NYD6AKQB+CuACEzMg\nhjtKhrCjsZGGLC5qcreo6aJNkYi7IKIhTYtMRPTt575Nyf3JwIYt1avgJOpRCoW6STUiilN/ROtT\n6qzbqbV1s6FMu85QMuT2m68n4Su/wSLyUet5N5E4LXvZvlKpftbYecEFq4iDAQfxXpiYhJ7OzT9W\nwr3pnM+UoCZ1nLeTl2RQV5ek0tLrCXDaHHQchd688iaozbQmly8X9DUyPBwoniaI/UqF0+vehTsM\n+2QXbPXhXDKDJgD/R/n7DznpwEsyIBKc9AbY+GwTx0vW1RXEflMAR8aDmchFPvDsAer8RudZcnkN\nKaG5HEMYI4Fs6yeZ3cgpbZh86qbkoHIM7QTcTE5m0UWcAU2c4ly8wmpyGiPd5dZN+QRM2Yf0dG68\n+C0/tjplzscgsxKp415GdnCSc/MKVWXINdfKyrWUSGz09bYEiQHh1Ktps9fSnr+9i4iC10jUr1P1\nf1U6MCVEaS4rM3ouJOz/XDKDGwE8oPy9BsD/Yq4zGvVGhocpbf0+qCyKrgutARxJHq4uL6cVgAtn\n0O/BTOTL2P/z/bTqb1YZk44EB8PkyWQQFBtTEqX8fahgQU8mM5RK9RvLtJuTg8ox+OnL9nfiFPfy\n3dt/V1YmXGnPuBOzoaHPAh65RX9VshBrZnp2uzH4y4ZEtzH3jpJgZLYdIRy+icrKlpBdZt7blvBu\nS96pTKM91UfLms6jnsvOp2w6bZRavWwGHAJXFiU2xe1sa2qiHq32hwwAlEb1oMygFOew3XDxxVjY\n0oK/+vKX0dzcjObmZgDAZ9etw1cAVEPoHM8BeBRACYC1AOYCGAXweQAXWn29CuD8kyfxeeu+YwB6\nAWwE8DiAVQA2Afii+nskgo2bNwMQTDCEEAAgFKqEEGjElaHQncY51NWFrd6qARwGcL81qkcB3GV9\nD+vfByB45W7rmo0Ihe5GPv8V5PPiWY2Ne/DoozuwYcMeHDwo7z8GYA+AecqzZDtmrRKsf9Xf5HOf\nATAAobWtQmPjg4jHK/Hd7/7acP248verGB+vw9/93QZrTqfw2GO9WLx4DsbG7nfM74UXPol5827B\nSy/dY40lDPEWZmHu3HChR7Fmq6w52XMsKbkdbW1x3HtvPxYsmO9a6wUL5uORR/agpWUPfvADfdwX\nAqhFXV0X6uvfi2ee+QXGxj6Ft9++0FqjzYa5hgv/P3jwLuzevQ+PPLLH9WyuHTp0GLt3P4wjR8ZR\nVxfG3r3r8cgje3D40CHcv3Qp7jqYEzP7j19jdSTCv7m5cx19zl+wAL1DQ9i3ezd+9sQT+Oorrzgo\n6K6DB7H2nXcwFTwlzGhsRCgWwz2PPYbDAGYBOAjgEgAhACcDzcxqQTiG1wdCTfgH5W+zmmBo3dOn\nFzgdhy3ohtt3mzWc/BKKvB0ChKRLDlIyefSZR2nloyuLVhOc4mFW+ddLDJYnr/lZXL7CysoEzZ37\nUfbkDSYZCMzD/v0HLOyCX8ly2adbygmH1xCnfnBoRXP1JDM4y3TqerksgawHhsE01wyptoempm3G\nMZjfu3OenEowCgSqnqw2k/2gb9EiSlRWuhIDrysvp/5UqmAsH4Xb7UhFSAZngxmUwDYglkEYEC9k\nrjNmJ1bzE5o2ebP2vclwchMEzHgTnLkS8lBiG8rL6eaWD9L19yWMBkQ1Ay5HGMlkhqZMWWldnycT\nQMUmPMkY3M8yE/RYoYpQU9NOqqpSM/eMkYiu0/3/vaRbz6url5AtTuuqzDqtz5soKAxXMrKgOnaQ\n6EV+440aqyyZ0Y15cuc27CURRm5/JxOk+iE3vQ4NryS8XJ0OUzMmBraYyghANwB0I/gEQJ3xON3I\nVCY7Z8yAxEZfDuCXAJ4H8IeGa2hFJMKe0iPDw9RdUuKwGeifHQB1WXHdeQMHHAPoqnCYOi1m0QFz\nhdvtAKVmllN7qo99yVytPTe0WD19RhjiU09DM74/KFzXmWFZ2ilUK/sA2RF66meNtknk9R0kUZB2\nMlRz+vDycifm388yH6QFd3cOudCI8mQ2ZZ4uK7uG6uqS1NS0k5LJDFVWJph+g1XJ8no/gdyDTGiz\n3jjMgYQlSxrmDImqFMIdpueUGQR6kDJIKc7sgI3IGhkeprZ586jFo66irDjTEYuxgKN1cOLC5abv\n1xYoby3qGoCSdfOounIlu4nNWANJrPpJO2rl89tRIFg1PRqXCbmYQB4nQXpF9Tn7MKcDzzqeY6cP\n55mWwEcEM7QGaSbRm4dDU2EdVekil8szqtR2EhLAUCEzczKZoZqam5l+g6mJXu+nGOCQGtpsAsRx\nsGS5ybnDUmUA3ME3IZmBabB65RlTQRXpCZCWWin6F0quMVxxDKKenbwuA3fZqpvKppJwVXVbG0e4\nBM2+dnUjOtF+fsk8TOJykBBf5xhMonwzqXYHsbEPkBmyyzEkt5QjTk8zYvBMmn/Mh4orGGWfZa4o\nvYWZM8fkeCkoFut2vBu7LiWfiSqfy9EVi2dR3xWXeQbhSfe4LIoqabDHUA9ESh1eNUV1N7rcF92Y\nwCXZTZMZg4j9vsPSsQ7s329UKbyQh8vgjFiUC9MFt7SQ1+4VKERJNG7JwB8R9+42hiQ6P0bixPfr\nvvd1xGEGxCl5K5WXL6VYrJsSiS2USvX7MCThrpNuP9NpbQJnBYmcNIneCxfe5oI1l5auYw2OJuAT\nj9EYdTE5YYfh7SONjTvY8vCVle66FkRE8++dTwf/6yARWYea5vKTcPpdBhpcoVUKk/3saGwsqAfc\nYWraDwPW/ycsMzDZBNQCElLEkhDlzni8wCgyyaQxbmFUWQD5MYE1stp3V6PFQQzSZiCJ2xl0NEIi\nOm4FCb37BqqsTHhax8+kcRtLZRitrZst0X0X2TYEnagzLqbBWfvlc2RpNZ1RBFFjOOlGlm3jmIPo\n0y0BmLwHEiKt9iVqIxSD0dhmrUk3AUuotvYqJsW7nZDWmXCVPyiIRBHW8r3ldPzt40RkNgYOwD4Q\ndRrcBR6QJ6UMmS5dqrm3lJdTJplk1ZG+hoaJXZLdC1mY1f+2RC1XzoKKCiLYpar1BJI3aUxCXq9/\nBrXnnY+045JFi/pY4gZuJXG66Kdyr0uHlsataLSDotFuFlhjakEzA8mNbE55zqMYJTEX85zi1Bgi\nTnqpqlpHM2d2UjTaQVdeeTOVlq51/F5aus4QnJW3EsY6DYiiMpIp1ZmUcFQD62bS1Yf6+k1Wrkcn\nJFt8OBtM1iURvXrsVaq5p6bwt8nD0AXbqN2h0O8QhL2ru6aGNTSODA/T8qoqWmPd1wtn4l8uV8iE\nLsneDhFNtU7bsDpiKg8UJIJdcG52edKbmMoGgFoqKqi7poY643EjZHlA+b8dsWi/8HjcBFWWAUG8\naKlusiA5Bk2tGPxDLpf38MXrJ6QzCu9Maiia1Bi32O+FeBwjEWfhlmbcc+FUInFCJ5MZa517rbmu\nppKSa+jCC9NWfgf9vrXsM233KzdWfU/vokRii2PuT7/8NL3vC+8r/G1KtZeBcBHqmAEdXVusHc2r\nTUhmICeyCaAkrIAiwBV/fSvcNRakvpWHAFtw3oQ+pq9N9fWulGd9EJx15ZQwXTq1jsRp7yS0pqZt\nBp12kLzqHsgTQ2xm/3yGplZMAJUN6OEiH1UcghtnIIqM9JMperCY5mZg5nXy2mxuIJMZLNXSMshC\npSsqeigaXcpufH6Du2sy2MzE6bIEVlBlpYCyy9N3y+JL6eorYoUTeVN9vQta3GsxApNbnJOOibyr\nMp/NikrnFI4MCDjlvRBg2RMAfh/AXjhBvLMB/OHx405YJoB9ADIAXpk1C482N2Ps2WfR9swz+MDp\n05gKgADcp/V174svYiCVwr5rrsGxgwdx8OWXUVNTg9deew1llacw5b9CwJurrN7HIeCqG9HY+KjV\niw4CHTd8fwzAOObOFUt65Ijsyw2JPXp0HH7Nhj2/CuBh67njmDZtzHHd7t0PKzDmXmsep1Bd/a+4\n+OKLEYuV4Kmn7sQLL3zS6scJmSb6CwD3QMCqe63vP4d/+Zc8YrFOXHHFfCNcWDYJ0x0d/S+EQp0g\nug8CLjxuWCcbEgyc0no7hsbGanzta+vR35/F448fxYkTJtj1KcydOwUPPPCECyp94sT9OHHiHgAP\nWvOa77hPf6YA8q4vrF9d3U9AdB7eektS5zEIWPkRAB/HW28txMe234nGZ76Duw4etEHkS5ci9P73\n494XX8SrsKlqHAKEsw8CKuwHCq8GMH70KABg/MgRw+yBKRq8WbbDhw7h4d27MX7kCPs724JwjLPx\ngcbZZIgllyLKy8i4A3a+NyKnrmRKoKqHg+p2iJWl0wl4pnAimLPZCP1yzhzeZjB79uqCsUyIuhLa\n6+8i05tJzaiqclrVi8mUbLYrDJJtbOQzGnlF+LntKj0kDHVcrQg976O5cIwtafDqRjicsOpMeNWV\n1CWBMRJuZNOYxMccPJYp/H117CL2xPZKzS/dfWdDMkiEwwUDoopZ0A2KCCgZ/E6YwRgE2EgmM9En\navIAdMJbTwoSNmoswBq/mNWFdT15eHikUHikrCxBZWXtVFvbRonERleVZeDDpBcCKS1dG9jrYPKh\nq+hIL1+9bsE3XSs3y5Qppg0wYGRgfn0CeaqoaLE2nLOku7AZCCZQUtJFM2d2OoyszspQZpyE8Kh4\njWGH8l0PAY9YYKSdxrRyZpelreKsjMbZTZ+srTVu9qxFw64MXaEQazOQngTdTSldk3pdxzGIbF9q\nMtUJywx0dBbnJtwEodc7QBmwEkF6BHuYkGDq9cUmQ1Xdbraxyh1ww28KvRip8N1Ho3zaLb39v/be\nPrqu6roX/e2j7w9bxofoGEm25chNAzHBTtJKYGNsdIxVsM+RFGOwLPkL7FCMbDmWyQuWkbjO7YU+\np6GP0t7BTVLy0pTbvmQ0yb3N7YtIiIgzBn2375GEINIEywcobqB5IxRkPoLxfH+svc5ee+05196y\nHVl5Q2sMDR3p7I+1115zrrnm/M3fdNUOcEUDbAJV09JxhdGiJcy1J74vRJ9mjkkSoQmOMT37I6Ss\nB1t4JVSmPvcwBRWS9Ln9xDsY9R4/S0GNhfFi8pbK+bjbxxGEx0sGM6lrlJaup7b05azQdzQ2shWQ\nCghwApr1+zAUTuCrjz3GRgPM9OYhqPqK/QjqjEhMzKaVMSOVgVQinSM+HQdoTWlpmBNRQGnZCsGV\nHJKUdEISNCnZhTdVbaTi1AqyyKtud4iTTyPkMpk+v6YgjxQMbz2iaLpwZIHvbxRS7coOVKAlVR0p\niSMvUCDZ7F4L9XecgCGfP3HQUgZ3+38Pkkq0Cp4rWs1ZvzMTjRitqi1vf3ZRAFyaoDyWRBafZFLa\naQAAIABJREFUXVddRQNQIcA+gPYijHod9xWALtIaCQ/65v5APh+Zq3chbFVIJD7m/2ekMjCbuUIX\nEDWbJN54U2inkgRinmNbDzsaLomc6w7XRbP3+GNNQZk6alHRhdnJT9vJXsXCAir7EKSVtrm5m7Ec\n+P5Gy55xWIJtVFFhk55oajj9t4vBmHyhHqXAGriWgE9QgBfYTUENBFOwk1RzJuLwA/a7CObAYeNa\n9rhM0FLcQuvqLinu17kw4CjCtRIkpGFIBiorI4ha2xKQ/A9mxGJGKgOuAKV+AG0G9fpaU6yjaBRF\nidsSSO342Bh1NzdT37x51NHwPrr6cGtkEijhkjgKogLX1rafdTYGwpUsVGi3gOdARhkm4VPcunVE\nLHtuOxtzuUEqKbmRPdbzNjHPUaBMpquocHjswjgp6rc+AnIUsBRpRWA7FvspXLJNAhbZ/Asjxjkm\n2MjM5jSrO/PjEJ0L+t78nPjAR28jIncasmtuuhCLWj5GoEqumcdwC+l2gP7KUD4zUhmYwms7PnQm\n493+AHQ0NkYGp1iJxlcmdh3GuxDOceAGn1Mi3fNT9PzzPyseE+fFju5Zx4v05o2NHZRO54qIQ+1P\nkLzT2ix1IRXjogHK8af7EzXv9R5ZqlDErYhyNSO+irGcx8H3CeinqqosrVixj1IpzWdo3svcs/Pv\noaxsHeOP2E8ABzb6Q18h3EW8IpKjPKYTWbIW19ywjUa2bqW+urpISXUC6JARAeOaiFisrAw5HDlL\nYBwB56HeUq8vLS3O/xmpDLiHWFtRQfsRdSJu9TzKVlUVBX4UAZ+BPuYgFIhDO2Rc2ZBaEXCoxkmA\n7sxni8cF+39uEmuh0GauWSsguoK5wpTaxE+KVJR9CJ2k2JI1AetxUqZzAN2VuBS0AzSXG6T6+j6q\nr++ipiY+eUcJbbSwSElJTyhCEvQzcEBylOW53CAR2TwNeqU2V2Deqspk+hI6bvU4DZGkWExHsCvJ\nKniPP6Gl2EqrcC29v3QB3dnQwILk9N9xSEHJMhjM5UJWNGcJmPfSP1vmzi1ee0Yqg+O+Buvzfx+H\nKp0uhRKHANpeXU0ddXWUFY7Reygxduu/BHZbYQxiz1ULiy87LDgFClfZCUzZTKYrsYmur23DeZMg\nFbXpnk5vYVbRIMSm/AkF4Xq8ebtixT4mwrCFPTad1itqVNnYzMacgrPj+dos5xSl4oVwj6u2oKIl\n43vY/rvQoytW7EtcJfr7Y98PlVNzJcNJ21fb3+XiPuBKEI5A5TFcX1PD+tY6GhuL15+RymA7onub\nnquvdiYTTULlNEhe017jWPYavo/BBemcBOgjH5tDRGYmnTkxDnKXpra2Q4kIR1wUaq5qyJpSPSqs\nO0hZAlEnXCbTxYT7tEKLClS84zMsfEpg4mHW7tBc9HhbUYa3NVGLy7Sc7HPd95ZxGXHPpJs9l6S5\nJyUdcQvTvkWL6LZslrrq66kvkykWF9b34woQayVi4ww2pVIhSyWpMphWOPLDCEOFHwbQ8+KL+MAN\nN+D0N7/JglZrAMyB4vnlgK2namrw3OnTIT7gHVDgU5ON1gXpPLBwId5961UMrmpF4edvQnEya2jv\nWQBPs3f/xS9OYOXKDxvfpcTjpKZgx2fY8xoaUjhy5FEfSmyO3J9BQYhLEcBs1XdXXHElGhpS+MpX\nbAbnz8BmJ25pGcally70mZrNdrv//A+Fjn3wQQVXbms7gldfdcOsX31VghCfBnAaixbdg6NHP1n8\nRjMhm6219WM4cuQYTp06i7lzCUQjeOONajQ0pHD06CeLEGnz3JMnX8CBAw+isrIfb7/9EMJQ4v3+\nlZOOAw8dt+cS/9aBlptuwvBf/VXk/EePHClCmPWo/NGLL+L+F1/E5/1zh599tnh8ds8ePPA3f4OH\nz5wpPs3e0lLs3rMHK1evBr7zHWzbvh01r72G0/Pm4bKlS/HA449HRj+2TadlMIJoynHfvHliOKbg\nf94A3iewvbqabrnmmkhdu6IPwTDPJMtgQ1NTJJEpmsUYTWLRyUzxhCMHnQy8ExMFWrBgC0lIRRl4\ndC8FQJow1DncJ3M/Puj/HYQTZT9Ev18G/lDROaq5DqSCsuYqKodl19L5UKbZHA/mPj8Ahel30Ulq\nKzNIYWzIeBGBGN6unZtlUABf70CKHog1Fay5GZuoJPgh7l2zJsQChoSWwbQqA05gc42NxbJqOSjm\n104of8IkVBbiTn+wTdTWjVVVtGXBAhm6bFW7lUKRHLBjEmHmowChFhU8PUHV5O+loICHnItgT+hs\ndq94fVlYD5DtFdcKREcnSkquJbvSsV3khN+GBBWW3YrOhARvKxaLnZgo+GFMzvm6v/h3khwNe9zi\n0JbhrZMN+lLjyxWjTcrrIM2l3QgqeXGVls1zJWaiEVs5+FtcF2qWw9oM5PMh5TQjlYH98OsAuuWa\nayKr/jhA6zyPbpk3j1aXlBTZYbWmGwLo2rIyGoJMmS7Vc7TRidJAfzzTHMpHSFIbIC4qIHHpSQ4v\n2WdwgBTE1rWvnySZCHUosl/XCMZMpqtI6RVWRJJS6vKf5fbi8wRw3qiCM4FFUlyf8+i7QWAjwt9T\nA3pxDl6pPz9+7ke0staLWLrc3NMC2lVfTx1lZZRF1H+2A2oBNOf4QD5PRLJlMJDPswuczeExI5WB\nPWg9AF1dVkYbEBQ60aHCuNCJdi6KueEJSB9cA22fH0fsoY/R5dIymTBegIe3au96fDRBhft6feEb\nIAXBjSqQTMZkNuKPqai4NZGJnoyNOYAQm87BoNCrFE3gEX9S+HVqIDBdrl4O9SZpLmvhoX98iNpX\nLYqdOya3gbnoFXyB74GyKG4oLY1sNfYtWhTJUTCFXiJQsbMmZ6QysH0BA4jutSTh7jKUiWlSsXyI\nCZGI+mXZA729uppunN9IK+uXUWf+/GsDEMVl9xWoqsrN3Z+URSicbMQfk83upfb2PVRRsY7KyhSu\ngMukTGYZmKty0Edt1ahkoENsMlBSHEVgESTpwySpsKeyUjQHY1zxFs4CkPqTyw3S5X92Of3XJ/86\nFgVrFk0dEeb2ENR2OS40ntSq7cpkZr5lYD78QV8j2gMgVk9GkP21G2FLYb9/zZ7KylBIJmnTA32o\nrY3ay1OhVNI8ltDihbeft0JwOwKDyknSpI1OTH7Vi0s2WrRoH73vfVGEnudtoeXL74hhS3bhBuSw\nob5WnGUVTlkOthdz5mwRn0cxNUUZoTOZrkSZodHnDMZS4kkor7iFlt73O3T27FkqTEzQYC5HfZkM\nddXX00A+H8mS1Y5BKQTZU1kpOhV75s6NcBq6Epm47cOMVQYEBToqCIMjac8R4/Og9V1XJkMD+TwN\n5nJTSlqym7RlWIqbpuzsspsLPZjEsy4RrdirXvS4sOc8QPtJK2y0NLoWYh1N4FZ605lYWclTiScb\no3ECwj6SkhITgKQVhUplLi29gcJ+CTnPINm7Ca4v8yIO0cfW9xBRfI5MEstgMJcTuTr1ObsXLoyg\nHO9saIjUX4gUaZmpuQm6wzr5ghuccShyBpfPgKNV58pS7W9tnZJikMyuTlTS1W394nmSk8k+ht9D\nj0fYi+Luk9Tk5Y5zgZzM/X+c8jPvkcsNUnv7HmeRkSRjNTFREIhK1Bidi78lSZNJVMZJisZce909\nRBTvc5J8BrbwchwI5ryXomZ7s9nYeo4zUhlop4jWZlJ8VmcV9iIoMGkOgJ0KGocuTMKD4HqxQ1BM\nSFybSkiKT4lV53C1HS9UMwUwqKfo3ntnMn2JCqHoliROn2Ss0mk+spJOd/v+kHutsXP7W+IUdThK\nwT1DP3F1HPVzxRZdXbOG9maz1J7JUFdZGV1bXk5/kMlEuAy6a2tpEMpqXgflTwtFKHjtTX2ZTOh5\nuFDjjFQGkb3P2rWBeW9pNolhVntYzSa9kEPGeRuqq2kgn3duIwoTE7TdL1Rha+dDbVdzMjAlsApR\nMvai822mAISBOKpvqdTHSWXxufb/QVn3JB74JFyMScZKKoqifQDcd9rfsmLFPqqpuZ5qa7upubmb\nHnvsq07lEygnDT/nohUFUvDv4BqXNf1h8RocAKkfoN5UKjSPbkeAQ9hoLU6FiYmINWxXXJIsgy5D\nGUhblmlRBgA2AfgJgPcAfCTm2EQT2WzHx8aoo7GR1pWVUVdFBe3NZiOsMPtbW+n6mho6iGgm4kbj\n78MI10qQIg5SfTwpVJmUkFTzJqZSa0mZnoNkO9+mSk8eeOvvDqEEo4VIo2XaKyqu9gX+EEW5CaM5\nD3GKKomgJ680HQU3aR+EJNxjY8cjJdk8r48UQQrfp6ivQMIx7CGTGyGfHyj216YmO8AILmcB7zT4\nDeK4DCYB2lxVRf3WNW6HQtDGORSnSxn8LhTb+XcvhDIwTZzBXI4tTrm/tZUG8nnat2gRvwdDsDfT\niEW9ZZDgnnYfpkKaIglBU9MGqq/vonR6C1VUtPuriz3J91GwKrlZkzkYLu+D+IQwofcaE3qASkuv\np8D0PV78rqxsnaUIogIr9e/cKi6FOSED8BZf7Vnyh8iApHVkV0nSz8LzLnAAr93s+fa81SE926QX\nneL+/JMs23xZWXE7cXxsjHYvXFhcqPoB6ispCcsHw4xE06UMihcBnjhfZcAJIVecchAKruzyznYj\nbBGYikFr6hG4s8oG8nlaf0ktbZxb7gxXckJQUtJH4cIsripMir3IZYpz95CqIss1BjUpiStEOLXs\nPa6fLgdnXKhSRyHsStZJeAbq6uJo4INn5C0D/cP7CFxhU920UNvz8pwzaq3FygyBc6zipjVh/u+3\nThm4nIDm/3oQRBPEQbYUwzii7LSulV8rJpvBVnJCmkKghMmsFaAnJG8im8SmUnMDluxrrhOO7fQ/\nS9caEghPp4bai2t6rOS6BEMhQph8foAqK92ALCKXZTDCXlv3ZWpcCO6x0HPYnmMi34GLa0OwRvWx\nEh5HMyOZ8/aCKQMAowB+bPw84//eaByTSBkMDw8Xf5544onQQ4rJGIzmM01/ZxQBCrmogUrOc6za\nCtwWZEtpGV3d2u8UXj50N0JxXmlXczkdo5O/g2S+QGL6pn4ymT6nKZ4kfGo2zp9hnhcHwpoKdwIR\nsT4Dm0zFfEa7nzwXwkSRzeiD5Qti0agctXlPZSVtu+aaSLnAnT7PZ66xkbrnz6eO+fNpQ1NTJMpg\nN61wpHm8N5uljspK+jRAwwB9Gv8/sgxMJ8pOBKv7TvBx21s9j+5C4PzTyqS4NRA06iEjFNRXXy8O\ntqrWLK8QahW3J/BxCldDVgJ62WU7E624PC2Yht2a19xJwCilUpvI3HMrpmG9pZh68g6XXDXVbY32\ni/A1EUxlplbygFUp3umo75nN7qWysg4CVhFwK9lO2qRbHU1rlkcUu3KuVP2mib/RYP42/VwHEK6s\nzF1XcyyyOTxCFu7FUAYfjTkmdhBtXoE7GxpoIJ+nLek0bQRC3lRdwPIuqK3AQSiTaNdVV4UGQgu1\nqQw47IKZVjoEGRa9Cmudk4vPXpSZg+KanLW4m0zHn4Ltdof22CppSdcckPfqLlyEnGwkC5esvEaK\n58VdO+Bb5HEZPH5BKj4b75OxnbPrmq9kF4P1qVQEcjyVFoeJGQLvK+Ao+3SWo5k2zVnY0xVN6ATw\nEoC3APwrgP/hOFYcII3v3lxeTp0Icg20ltSYg36Ek5J0MYp9K1aocm2trSHWZPKF/nZEQzvah6A1\nvn2OlDClLAN5ddq6dSTEkpzJ9IlAmiShRHkF3egUECLbFDfx/morUVnZU8ysnEqijqYZT6dzrNlt\n7/EDIb839Nza6kiluMIno2TH9yXBDvrJKyFN5MI1KQpySMgVOOzPnY21tU4WbqlxwlqA2s7e6//e\nbzEpi2SpCCwCV5hyRoKOuOYiKrWdLCNbt9L+tjbqbm4OhVwiWYdGrblJgDqM/Zo5mF2ZTBGrYL+g\n4wD1WKGbPFooYEBKjq6Li8G79uNxe2tOQPT1JEYiDeDRFkRr634jOqFw+bW1G2nFin3CvQ8Xr2Wz\nOMsKJGAmtsdt/vxuMi0cvliJuk5jY1QBqTEqUFBnIayEXEpX6u+qpt8RV/AeRM1zqSpSXEWvOAwC\nkexPu9nzaCCfDxVe1WF383q/NcogLooQV39eyuk2IcsizVRMaGcgn/f3eVfTitoPElep2T2pgnoK\ndh2CuErP4WrOrtJk4ZWPN7+DJCL+vjpByGYRWs88kx3FSAYsUuXcwkorKBDTS1GlxXMXmFZFGGIt\nhW7lorHR/iqH4VW4ipZzQg9ljXLhOy34cVEBm+IvLtLgmp9DUElOEdalhQtDqN4ZrQxM7SmWUWcG\nhaOXFpmVDSWSJJkk7iXqybfoqh5auOr6yCqs4txhzLyd9KJX26amDTR/fgfV1/f5pCWmsEf39FFq\nL55ynEhe6Sor11F9vcRipFfkqDLji6mEQUlJ7p9Ob4ikR1dXbyNJaUl1HnRSVCqVpQDJOUrKSRqd\nCpWVPQnDthOUxyIaRcBCpCMCN0PVSxyHYuPmgD1a8JLMMx36OwjQTbzmDBVdKUxMRKowa+u5z+Au\n0H4xm3ptxioDW/BcNRNs84urSNspnG8rkThhl7zAdnvr3bfoioevoM//35+PcYJxQjFKqVQ+IuwK\n9aYFjBem8vLrKJXaSOEVNIrgiw9DKutAcRSScU9+JdbFSJUzspOkoq66JU3ckunMc5TJdNHY2PGI\n47S8fBtVVKyhMKBLjx+PvIxz0ur+LsVNRVwKN5/W+oLWL3yvFyluEF0WqDR/u5ubQ/2UYPJd/mLK\n4Wd09GPGKoNEe6aqqqLHthhSqa+nIURDKuOI8sntrKqSi1bECHuS9tN/+ym974/fR+vye4UJzQlX\ngZTTzzZndRqutix4Qg1tZsdFBfg6h5OkVtAAX9/YaKYK677xwqT9ClVVUVIUrvJTklRruZR7b/Ec\nvmL0uN/XuykcqRiIZYvi+qiTuTaWVxEhGnrWq+0mBLR8vYLQxVkG+1tbIwzh+xkhPoioE9Fe0LTz\nfEs6rbYLglJZn0rNXGUgelMzGTY2G+Ep8AfCPN+sNWcSSZ5Pi3MEfetn36KylnZ2Qs+b18cg4rSC\nsD38tmXBC2UgDIO+4lgrCm80FLmP7DLyFRXbrWzGUfK8noig28ekUldTaWmWKiq6KJvdmwgnwTlI\npezEVCpLY2PHhbRibry0JXY4li3K7E9gvagqymuQilgG3Gp7AMqXsBEBb6em5XdVRSpMTLDlAPoR\nQOXtAinctlg70e1r2QVZ9c9hzGAHoqs6jN0k56ANUSYEPoap8B9KLSk8dMU6s+x4MKH5OLqOlceV\naeeIPHZStMahZNYf8uswdBpK43bmPsGqb+cA8NWUo4KYBKYsbRuy2duIQ0pec802i/zUVJ5ypCKO\nDyLqcNR+gnDGoekzkIBnG4T/hwTYWtgkq6Gjujri/Y9TLNy1XCXeZqwy4MpBbS8tjSC7ChMTonPQ\nBgSZYcLzVQRERLdls9QJtUpcD9A+8BbHiRMnqTYTZsKRKMOCqIAJSOL39+F9ui4nbm8v5GzJqJDx\nzrU4nEPY/8DfzxXDJ5IdigHPQoCUvOyynf72RZefty0DfrxSqU1OpiheMU/QCjRHtqxDAK0BaBVU\nzJ+7oYRidUW+RM6NtjZWgUjKo6u+njak0zSA8HajgOj2RUc/ZqwySJqhZXLH2cdutB/aYQ3Emft2\nOz42Rn3goc69flzXvMZzP/sppVvbaMmKbQlN03FfuDtJMvVN/EFwjg1c4lfq+fM52rB4KnYO5xCN\nOHDz+XAoFJoUJ6HZkzW1fDrd41OeaQflOKkqU6afRPfHBFANUXv7Huc7jSqkfZTHEhZlWvDfs8sy\nSOK0tlucRWzPU1fVJb1d0U5ELfS3I7p92TmTlUGcx1U3XSIq4iFFAMOUUpB1c5n7kpLQ5a+liaCj\nHCbQ444bb6APL6ui2z76AWd/bMda0uIsKnTZTlF0ncIxmNerqOBSmAtUUsJbMC6cQ1vb3VRdrUOf\nkol+G7kqMrsAV+5ojFYI3QTcRTU119Py5Z9gnZgNDXdSPj8gJlHZCmkp8uI7Nhcgbv5tq6ig27JZ\n1rR3LTQui5hzDq4pLS0KdcE4Z0T4vKG6mnYvXBjOWPRlZcYqAxFamcuxNNDam1t0DibUxHH3kpRE\nX11dUQO7NLPOZZAQZP/HY3/jzNrjFIRmH7brCLa27mdj8jbphyvLr719D+tciwprNEqhrJLNFIUH\nbya7xJtN1CJxMehnk6Mx+u/DIeUihyRlmjb7PquwRhR2e2uq51+vL1xbFiyg42Njih69vr7IzB1n\ncbrmYndzMx2E2pbuAlMHBIGlamIczDn6ieXLQ33a096uaP5mLDsyyQlJNquRSZyq/xcyjRI4CsUi\nExUVbE37ka1bYy0DrY21iSkddw1KaCluolVoo6VYRkAuIsCmQuDy9pV3f5SkVdmMoQcTPrp9cLEv\nJy3Oolb/46S2Nwf9v2ViVbOIylVX7fLRjGHrQapLYLI02z4JGUdxiGxqMjNT0ESQLsUtxRX0bl8I\n74KK7buc1pNQ3v9IVCABHFmaiz0WmEjKiem0FIEZQTsARYsm9SmpMpjWkuy6vUWE+6FKWZ8F8LNf\n/Qpff+utSInqkVwOx1avxtlTp/DmnDkgz8MXX38dqYYG9B89isVLljjv8+bcuWyp7A+98w6+AKAL\nwON+H1IAfnXiBD75pS9hb3s7PnXmjFW4WxXy7vc/l/nXO4to4fFfAvgA3sO38ffFc7fiLXwDf4SX\nXvoCDhx4EF//+ucAqBLi69Y9hBMnaqHKpgejQPQIgA0AGpm71OCNN6qLf738su5JDcLl5J/Bm2/e\niZtuegDLll2OTObXICrF669Xo7Exhbq61xEuKM49UQ2AD/uj9SBKSj6B9977GoA/Fo59F3PmvOM/\n131+X/630LEnTtyH997bBr6YuSpy3tIyjNHRPymWXgd0CXvunJ8CeAz6bf2f/7ADf/L0GvzRiy8W\n38Edtdfin95/I06efgr3F0rw5++9Fypx/skvfQlNCxdi+NlniyXTzfdeA+BfAHx5cjI0V+87cQLH\njhzBjqNH8dC6deFzn3oK/aOjSDU2sr1+/9tvh671YWFEKysqcOk77wAAngPwAID/HUbB+bfewi8R\nzADdJ64kvNim2zLgzCWxipLhR3A5AqXv7Gq02uQqgC/LphNENFV7vqaGrk6laJPnFdOe9XHashhh\nNLnk+NRcCLoKcDiZSFol9WrqdgBKFOzKWSkDlaJYAleBFZUcpbgGZCuitnajFZaUoybcFkJHU5Kn\nVe8gGxlprv6mw27Zigrq7+pwhgYH8nlaV1FBvVBgnoJxTK8wVw/5CXTcdXVGrW1RbK+oiECbufmk\nQ5AagbhOOGaE6dNULINpVwacuSQNQJLcAdd32gmp00NHjBc7lXsWfAHPl5dTrrGR9q1YUSSoKEBV\n0DWdP1nmfoSACyGd7mYmtAQ20sLNOxpdRCLqt7nHlrcb2qs/f343lZT0CNcbMios81sS7Q9IEpY0\nU6XjgEK2QtDn5HKDVF6ej8jn76GVRfbd/pFlzjCfvYU1t6YHIGfAbqytFRc1/X+NGjzU1kaDuRxl\nDdOejGPsRao3laJ+BCHEm4X73Gtd5/qaGrp3zZqZqwxMnrhBqJjtRoCuS4VrHMblaGuNK+7xtm51\nUkTFWSNJYNPbq6upO52mNXPn0g0lJXQX8yLNdOylyNJS3ELt5WlailsoSIcmX9jspCAzZ0GF0+bO\n7SkKrzursZNMDgH1w6/Qc+ZssRKSRsnzbiBzj19VFZRMCysgBac2uRGIOBry3xyvolJOP/EpytbQ\nUmyl5Whg58W6igpxBV9TVcVaE9ny8qKj8PjYWERh9Pk05l3MPLNXbHNuSuzeowC1l5TQLeXlIYtU\n1yd15fNoRbDdOGbGKgOTqMTWwJurqoqVaJLkcx9G1PurBbcrk6F9K1bQDZ5Ho8ygx5Vyt+8pWRLm\nC1hjhI7sY/oBuhFhZ1GYH4EI2E8BTdlNBGRJOe0CIY8vda6ET2X1afCOm/KMrxcQDVuaWITHHvsq\nNTd3F6HXtoMyyC3Q4dADVFWVLdZ8vFCKgIjo+2Pfp02ldaGx7UaKnTPdAK1HND15e2kpbReEszud\nLt5Lz19tsh8wlAC3YNgRAIJacPT8Kvhzax8U2ElT9pnXMeeSrlMaCXkC1OH3yU62mrHKgEiGGXOU\nT0RuzoNEBSugQjbX+4M4AplDzrZG9MsSuRMRrCTSfrIPKiQq+xG0UG6kaEGTHWRyEURDa3ai03GK\nVlju86/JhQ0P+koo2nWTO8BWQHZaNYePsHMkuKSmC9GSwnPHEWzp9Nbv1ooK2tDUVPTOc+9obWWl\neC97kdDX7YUy06WoVdx1pNC2qXj0vBsC6Fooa2DEn9fmOTNaGbjQVRykkwVs+AJdgMpS1N+5MNr2\nABcFnQEv2eXdJMvAtDBc93ZzKk76YUU+47CsbF1x9Y1SmcU703SIzsQyzJtn8i+4HZTJ8ADhVGYe\nvxBOt75QTUp+21ZRlsga1GnAdwvvaIthGdj3cpVZj0tcMn1SnQj7mURHYjpNt/oIyeI1ES4YtN6y\nUJMqg2kNLd7X24s3T5zA//VP/8SGWc4CGD95EsNr1yLV2Igdfvjw8UcewafOnCkGy1IAPgXgbwEM\nAii75hocW7AAZ0+dwgvPPouaV18N3bfGOM+872L//GM33RQJwSxesgR1K1Zg5KWXUANgBxAJNd4F\nFeLR17sdwBEARxEOR94G4FO1tTg9OYlfAnjU789ZAO+k/wVbO45hcrIO3/jGGSD0lDsALMa77y5H\noTCMXbuG8aEPecZTPGr0SD/pn/nXGA6NwJIly/Dd795X/E9v7334ylcG/XOiT9fSMoyjR/sBmGFL\nblSDv0+dCv4On/MCgIcA3IdXX63BV75yGk89NYzR0f5Q2PBcGxe2uxTA/I4bcay2FicTXWDrAAAg\nAElEQVQefxxvvPIKLheeouq993AaQDX4oGXjypXivaRAZ8MNN2Dl6tVoGh3FsSNHcPbUqUhIvH90\nFPv37MFbTzyBvzLCnMNQYe/IXGppwX8eHcUnP/xhHJucLM6Sfqi5rEf8c2fOYG91NR5+883I8zpb\nEo1xIX5gaLpxKNPZ9hlsEpyILp+BTS/tIo/sB5PM4QAv2fctILAkBnM5ypeXsytSl19lt7u5uegD\n0eWxIluYzCU0ceJ5Ghs7ThKaT+3p1f/CPoM4IhN+1SbiTH/FxGTv6YOwpekM1OnY3SQRsoYtA7fl\nYfZJ4oI0mx1K/v73vkd7murF91qYmKCNtbWi5bahqUksmW4X+uWgw7bVmjRzVveL61M/VOSiJ52O\noBwlB+iIMQfXplLFc5HQMphWZWB2fhwqFbQPah90dXl54v2V/k6XUTMdf1yocVt1NW32qysXEBS3\ncJVNI4pPqnJFMri2N5tlj29bnqY1Cz9Iq3Ct70PQTkXtRwj29CayT6pKFKYMkz33mjfA5QQMU5Dr\nrEuzJsNu4hiLw8pGTlbij5f7zb3fnkw1tQwspv6uDpG85vjYGG2uqooo4zsbGqhj/vwiEnGX//uQ\nr9RdNQz0vYp5KlMkzhnZulVOlrLIeE0Fc3xsLFJnUTsqC4j6zGa8MiCEy6CJXIhr1zoZlM3jpJdl\nFpbQq7vNE8e1OF6DpLwH+lgpJXtzeWlIUXWhkpYi5yuFg2SSn2oCjzVr1P7fdtLpEmlxsfskwhcN\nD/LVmhobOyJp221td/t08RuospIv+WZCjV0JTbYAcQr1yJZbxfdovoOBfJ66Mhnqy2RobzYbgbyb\noLTu5uYIsYgr+5UDv7nAcveuWTMl0Jq54O287DIa8pXWRgSEP9y5M14ZmE65gy0tsatskf4skwkV\nQdEC1FdfL74kM4wTV2dRfMGG1jdf8J72dtrQ1ER98+ZRd3OzWHHHlZI9JPQtjyUEXE86mrBw4e6I\n8OsMw6mAdoimUkLdTBfmhTqT6SIiV9GVUaH0WWBRRHMOxmgFmqmzpCI0rkcSZr0maZJiOYCw6R+3\nFdBKZptvfepjuJwbO2LFbU1uTvFhURcGZgjKyuaiXjNSGejY7BCimIKkq6zthY3EdZlz9OANCsI4\nmMupFaO+nvrq653bh3O5P5Gckt3reU7v8apMi1AHkBfgpM3FM6CbghSblkAve04m00dEcVGHcaqp\nuZ6k6kjhc8eoB9F037/42z+ja9rel2hrloTHwpXIZt5Dejf6ugdbWkRcwAFHX/W5dqFUaTupLRXJ\nij4szPEZqQxcjpnQC4zZe+njugyaaG7QTEWze+FC2sLNZIBuZISa6x9RWCvrSaKFmeNgLPa1vp49\ntqOxkSYhhz07q0rors71VJiYEAS4QJlMV2yqtN2SWAZR5cMrI408VLyGJt+CPkb1W4UzeQU0MVGg\nxQtvp6W4ha5FBftef++SEvpf//sD9Mn3v9+pgG1gkFmdS3qX5n26LGFzlVTX15BwKNz/XVtaaWE0\nCwNJFmZHYyNtrqj47fMZnKs2N5srymBPlMFcThzEqTDXmPe8F7x531tSEiGtkKwIHYuOI9W4tb6K\nfn/tzZYwSiAiubZgXFqxO124QAoiHQYScSQtZt1EHU1obMyRDRsGfkK53GAotV2K9/fOqxMFyGxc\ngtoBRGnruPyTnsrKIghJ39dlGej5IEGRO42/k2xpbVk41NZGa6qqQsWEpbmkt9r2gjPjlQGnIZM6\n43QzNXvIMQiLHcZ/aazgOiYfB0YyB7zboUjWlJZSLp3m93YWX6O955QmX+vyNJWkP24InSvD0BVO\njFZP0glGZuOthzBMeWKiEEM4EiinVddspjzCsOE86uiG7M4Q4lMCB2mHb9yCoa0w+/yuTIYVuoF8\nPgRcs1dijo5fW4663wOMgB4AislIU9lSmn2LMCoZCsFmFJcWxxmvDOyVNyk3oj1Yes/FYcq1QtDO\nPU5rrocbOWi+NG1+moVcJRPysOM7ztllmraSybllzhzas34tNf/+dVTy/jVUMTfHHUbaLL+6rT8k\nOJ15vaonj/snoWVT5dr1dSaKK//vpproI8t3FJXGyvpl7DivrF9G+6/+CBEM6nHrffZ4Hn31scdi\nozsjW7fSprIyXrkzysA197qbm4sOa021Z28FzTm4G4hsTXQUQtrSmvPbVnRceXVzXtqyIT3HtCgD\nKHaL5wD8EMDXAMx1HOvckyflRuSEyAXC0L/HAdpmeYQPALQFoM0IpyFPQiWO2NYF54jkVnGNc5fK\nv8flX0iWgQ4hHWxpoWd++iP62Hoz1TgQaiXsP6He2kvCW43KS0iFK+Odh7rFpRgr60FbKAH1OCes\nm+qb2Xe8uraGPvqxuSHlairtEShaL9eCYVqW0vgN5nLsHHLNvbh5WQxX1tdTdzpNucbGYlHgJPO7\nL5MRC6burKxkS7kdhlwoiFOW06UMsgBS/uf7Afwnx7GiZjUdbC7NKTWX72CnIYzjAOUaG6knnaZs\nKkW3A3Qb1FbB9OjeCLUSSBPDXPHtrQdnUnJ0bfYqYFaCZkOgCOPPR7ZuNVZuvQ+/lpbigwR8m5ZV\ntbDjqVKnRyi8d89RM26iTfXNUwLNEJnhx4O0FLc43+FdN65nv7+jo704kV0xdpdgmtuMAURTySWH\nMJHbKk2qgDjlZ87vHr/ykb0w6KzXuLRk83+d4J3UWqmYRVcLExPTowxCFwI6AXzZ8X3kBZ5rmC7p\ny+xEQKBqa3RdDVcKCQ06Joa98mhfQC9UZiR3vXWVlUXwykA+HymkabInmdfsQ5gkRSsi/RzfH/t+\nxAK4pXoO5Zvns4KTrWwi4NuRvftU+SV1M7kXV6G52HdzVV+/tI4aP9tIdYfm0pZLq0UhlfbvruIh\npqIwlagev5s9LxZp6hJq13dxW1vu3H5/Th6GIsHZ649TFxCxAgoAZVOpEPW5Db9PopAuhjL4JoAe\nx/eicEUEYIoFUThHywEoHnnJRI8LCfUyg+vyUeiXLJW50sUyXKvfRosWSwtoqP/Cc9jXkrZOV1xR\nTr97SR0/ia1xsq0Xvf89suY6+tTmbvrvT/03+sJ3/pLe16Ro05qRV4Sh1tj0VVfSk09/j06eOBHg\nOTIZUUilaIFr4ruiRUmsS1eEQvruXAqt6tWeW/zM981931NSQrdls6G+JfG1XTBlAGAUwI+Nn2f8\n3xuNYw4D+FrMdSIv8Fz9BNLL3NDURL1QWnYPlENH0pj63jlhAmXLy4Nko9bWCPpwv/+dbf5J+1WT\nlUlyLOq95r1r19JgLsdCZceF57B/9re1sYJz8sQJunvV1fy4G5/bmkBd84NU2XGAelLh8bz50nK6\n7rPX0qb/vJk+cPU62lRWUxRI21G7p719ytEi6T0XIcU+QEwngUk4EheXZhKYsdTiBFGc3455Erdt\nsBWb6IswImHTZhlA5b/+AEBFzHG0urmZrrvyStq/bx898cQT5xRBcDVXlqH9ovW9O8Br6LWXXppo\n8tqVdVnSFC1MvkNIItGQcBeH2tqoo7GRNqTT1FdfT3uzWcWJv2YNdTc3iwle0oomjrvxuXNxYGkl\n6bO+pgt7oTkj9XgdgALKSILIhRE560BvsQZj+mifmyTj0BXKjDPRXeMs8iD434kgJt/haL5f+x7f\nAug6gD4N0NWXXDI9ygBAB4BnAaQTHBsRXhuOeRAqPn+H7zm2Q0Zx2ntka/Kirvred4EJCQHUUVYW\nK7DHx8ZovVH5Rm8fRqFYbuyKOP2+AIxDRSvMSbStujpkgXB9lfb4Zkw8yYrLXe92X5g0JPaO5cuL\niuAg4jkjtSIegbyqDTJKQvJVSIImhdu6oLZp9ri6hNNlxUkeflFZ2BmMwvk7EIQpXQo5znLQfWEJ\nVIw5N4npCy3+HIq54v/xf/7ccWxkQo5s3Ur7Vqyg9fZkRmAOaxMwDlqqhdPW9JtSKdqbzYY9rYaJ\nuNYvqGIrkHVCrNoMKXEltjUD7v7W1ogjbQ8CE9pWQNq/YSoG3V8x0mJPYGaP7VrV9CTe095O26rD\njr2dtbU0CmURaCCXZIGYglaAWt24sTO5JOznKPqLfHSelLi2IZ0ujukAlILRDrhRKKdczr/Xhqam\n0FhoTH/RYhT6qRXfVH0Q9uJ2GKDrysvpeiDiBDwOhmDXBjoxoXDOB6bfZd+8eRFGbsJMBR1Zg+bS\nZlozauIJe1BsaKk5IfUgmE4YziGzs7KS2jMZut36/w6ArqmsdE5+V+WdQ21ttKe9PXK/HcZEjBPu\nXYgHNpn/t/0snFM1Sb6F/hmHAvqEJisgWiDmO5XM9WuZiUoAfQLRgqFSuvfNqZT4Pu3+6ec1Hbem\n8zfuPUyFJk+PI+dc5oRYWzLaGlvr06SxVsbatdRVX8+OnfnebctYoyJntDKI27OaL6JLMNdNaGlR\nM9bVhTSj+bJdZldfSQndDrWimdTU20tLaRThjDKdc+AqFz+Yy4n8+toaiBPuDQn6bnv/zSYpKw54\nwzmhJIEuWgoMF4TpXOWsplEoob8bgTUwjihT8UEgxJ5tj1+SMTHHxbRaTKgz59vQhX3te3DH2lsG\niZ/A7henaCR0ZKzMWNtWbU2YyjKpMrgo5dXOvvyyk1EvKLAFTKZS7LG1/ucXTp6MlrSC4oU7C1Xq\n7BiAE+D571IA/uK995CrqcE3T58OHfPwmTP4eGkpvnbmjLr25CSGd+3Ce8uW4f1vv81y3/24thYt\nnoePMvxzNQAmKivFc1PG53eM73cgyr94BMB+BNx4/UePhu71ylNPsWXffvTtb0c4Jm1evxcAvCyM\n1+UA3m5pwZ+MjkbK2y1esqTIJfnCyZPYdv31uLxQQBkUp9/fwSyApp7hGQBfR5jF8T4AIwD2VlTg\n4XfeKR7fX1mJ4bffBiAXgTtr/33qFEBULDtm8iAu9vu1DUGpvz/0+9kENe6ah/BRRNkm7RJmqcZG\nvJugX+a71n9nWltD57xw8iQePXIEZ19+GanGRmT37MHwU0/hthMn8LcA3gXwT6Wl2L1hQ/Gcxx95\nBA/7c/UYAv7EpO2iKAOp7pxWAMNQJKLDLS1Y0tyM09/5DmqgJumjUAPx76WlxQHTigAIJpOu4vcG\n1KAcg1zVrwZAY1kZ+xI/5g9u8donTqDvjTfwOUQFdE9JCT7193+Pbw8PO4kyJz0P/d/+Nh7y60va\nwn0EQIlx/mIo5XY/gBcyGWRaW2PrTk5a938BwJ8C+Lu330bN974XqgO44+hRDD/1VHEcPw+gRej/\nc83NrCKw2+IlS7CsuRn3FQqAP062MB0F0Flejppf/zoy7hUAXk6lMJLLofqNN5BqaEDd5CQu/cY3\nAMhEpCmE58lzJ09i0VVXFY81SU9fgBJ8s2ahnnuPQpHlTi5ciJEVK/DKk0+i5rXXIv08e+pUcR7+\n6vnn8dPSUpw25ozul7nQ3QHgHuPvexYtwicffLB4PLvAPfUUlvb34zMHDuARX7mdPnMGe/v6sKCh\nAStXrw4tspyyjG0XY5sghYc+sXx5iES0MKHqHm6rruaTkVpaRNp1m+ihwJ2PYEsQRzJp7hk3lZU5\nk1ekvWNPaWkxYqD3g4fa2qi9pIT6EY5mjCLK8Z+Et1E3O403CZWW3p/2+fvTiAnN4OGTmrZiKM3K\n7NT36kA0EhSHWO0TCuaYFb0LCLYgIw6T3vYLSGb6YC4XmsujiPpa9kHt3/X7XVlWFoEMS+Nm3muV\nwXsY2ro1N0fOM5/Nl714GZ2KQJ/Pj6kM7MlnA3pMb7/2zkpx7pxPDhJ5ScwELPj/vxlh38DOqio2\nRNPjeUWHFCccZlhs36JFxdi/BgyZXmVd2Ynbbx4fGyvW6hvxJ9T6khLK19XRuvp62m7Rabl4FvX4\nDeZytGXBgqKPQirwwgG8zD22qez2tLdPCaCTJHlIchDvEfpnzhsbh68L5kpC293cTHdBJaa5skMP\ng8d8JA11av7EQ21tYqTMhaWRgERikZ5580J91JiLLZjJfAYJJ44eOBOeK60s+fJytlTbbshOsAHw\nK/rxsTHa0NREXWVldG0qFVplpMnc5cNqbbRgUTmsXRsBBumVvnv+fOpubqa7/UKcA/k87W9ro+2G\n8zFpeIsbP7sPSa7jupaLz8/1XkWnYksLG4It+J+l/tnXNpWTC9VamAhTk0vzoz2VClGTR+6XgENA\nKzEblFawvueaCDEX+qstAz2H7XH+rVMG3ACYQBdJGId84dYroB5wLfT2ar4dfHisMBGuoGSn0kqr\niF6huPtzsFRtYUjbnj3t7aHnTMqJEOdtTpLQYjZ74ou59VNAi3LCJFkh/VBbR4lg1ramNC4ljljX\n3FZKhCR7EoxP0nE/F5StFHrn0K3bS0tDY8Td77dOGcSVamf3sD681xWqG4cKJ+maiKPgATojW8Ps\nxXqC6QkqgW7Wz5tHvX7s23xxBUNgub2cpNzWVVSIY+CaTEnyPFzJOHHtQuaRmM1Wwvr5tvnvSiLF\nlQBft2WzrAXC7f01dHoQQf2OAYD2JxRa3ZcLRadvX1e/K9Oy1ErzMBTK1VaW3HuascpAQsVJwBfz\nxZr17c3VShKYXDpNG43UYNeL2N/aGlr9jyPKS6B5D8xVZLOgJMwisuZeTlfR7ULUdCREcRWco8zF\nAH0+K7ercYCupHx+cc0F4OKeQXrWvVD+H5tt2BQYE4xVtCis8TU5MJIouzgley5K2JSTgXye7mxo\nCPVxKgCyGakMXFpS+s5VrcYUMs7kPj42RoO5HPXV11N7XR2trqqirXPmROob6JXGtgy4CWcXyNST\nytbGPZWVob7qvZzOT+AsiUmA/iCTiXx/O0DXVlQ4azOc6wqUtMV58s/nXq7sPk4gpeOTENtqS2QI\nygJYL50jnDsVB+q5Ni6halMqRf1Qi0gvVOm1pPNgRioDSaN3+auLJv4w01NFLWuloOqU4kNGbUPX\n5DX3Wua+VSe6JEEJ6h87iWcSUaSfvsegMPmGoLT9bdks3Y5w3sImyDBgc0z2t7ZSd3NzMUR7N0O/\ndT5N38fF58cJTJwQuRCpnGUgWRKdEN6ZBdk1z5XIcA/jwpj759LsPooWTG2tO6rkL6AzUhmIuff+\npN9cXk4bfEwBF1ufyguRYq7m5NFeWLPikk4g0tTXtmNrPaKWgQlxnYRKpNqbzVJffT111derKIFP\nayYpmV7/PNu7HpcgxK0iUykEei6rneQhP8RwKOxeuJDNkTDDsLdlsxHmJ467Qfd398KFEc7Kg1D7\n/TjLwJ6DrpCned/f9DbMbFzZ9yS+I+ldzkhlIPoFbGECn4rKrQg6pnu3RURieo0lAewqKwt5tG2H\n5W5ELYp+QwnoEOYoAl8Gt787ANDHKyqKZp5rFbRDgHHRBG4VSTppz2W1cznvuPBlHLefHh+dxdeZ\nSlG2vJw60mk2vKefl4sg2T4m7nk43wcbUWhvdwooZ3XECWXcd1Iftf/DdX8WyFdZSQP5/MxUBuYe\nXz/gGmaySBPILivFRRjMNOI4y0Cb5l997DHa6YepzGMkk15PviGAOktLi/v4woRiaj6M8Iqp7yXR\nXZkZm/bqGocz4FaRpJM2yWpnT14pxKjTtiP3lfpjnc+FgTnFFFc3M64iMic0GpNiKhbb55PUMjgX\nv1hcH8ehQGuu+0v9m9GgIxsUwWk8cSW39qriKugP4OaqKhqC2healWpNAZyEcsZwKEepH/uZiciC\nchAmMjWxC50AqzRC+24fw+Ai2DgfyyAJDTi32hSYc7SvJqlit52uosXkiCSYYTadRZlk5ZXCdrYQ\n2UoxiSC7lMZUthr2vl8vWJKSEauYYwYrgySTV5pAA/l86IWIppM/UeyquLdCCbIdztMwT3u1kfoh\nIcE4WusR4ztTAAqAyAJsT4jiFohxCJ6Pz+BcATPsc27dylp+2aoqdttUsM53gbpsIeH8EoO5HN3d\n1pY4lGxej/NXFIR7uyJbI1ujafTmc5wvVoO7vx4Pl9JNqgymPWvRTl/egWj238+qqnDn2bP4cyN9\ndbilBQOf+xwA4NiRIzh76hSeO3kSpwsFNnPt80DxfPjX/jxU5t+wdfxp//NiqOzAYwBOzJuH9HXX\n4Z6nn8YfvfhiKLuwCXya6gv+z2Ljf88D2A3gVwCGjOMvBVB3ww04VluLs6dORbIP2cy1khJkH3gg\nlNq64+hR9I+OFsck1dCA3Xv24Ngjj7DXNZudrWinQ0up5hOVlTj99tvFc+5ZtAipyUl8cdcuvLZk\nCR7413/Fw34K93NvvYV9AHrSadSVlmLulVfi1//8z7j0pZeK438EQAZCFmJDQ3E89HN7H/oQhpYt\nw9zXX8frc+ei6umnMfLNb0ZS2BcjSDPecfRoZNwWL1mCxUuWoG79etz/jW8gBeBNAO8B+C9QGY8/\nePJJPP7II5HzzGa+q19CzbN7ADQAuMt/1z85eRKXLV2K01Cp5I9CZRaeBTA5d27k3XDNTBHX7b7e\n3uJ9bTnS4zCS6OrAtFoGEoGndj6xSUsOLby/tVV0ZknkI1sMDToJ5RDkIgSmt34wl6O+TIZy6TR1\nNDbSdeXl4oppMzZ1Iti2JCmm4gJh6b15klVfXFXskKwfjuSqAEl92NDUpKIl/phkfaCPbU0VEL+v\nl3Iy9NYrbq8t9dGstykxRdthQzt8l9TKMkPTXDKbdjJv8gv3JK34HfcOCxPh3IgClO/DJulBQstg\nWpWBfvikBJ5SzNo2izfW1tK+FStCprQUi+6ori56oQ8A9HELSpykSEW+vNyZDj1iXKtgfD8O5fcw\nMRURE7XFTSOvt0YFRPfLrkl0LlsJVgiN8zkfjOln0eMgbUPse0lcl67tjGucXFEOux/a+Xsu/heT\nDJY7ftD4bnAKYxL7LmIibDMaZxDS3Ex+QJKHl7zZtkBI55urkivdVTfJKTYORbzJQYo7EVRzMq2E\nEYQrSbkiBS4gDrcCmbH7uBV+qkIqRUgi1a6ta0sOWBd1vd2ngXyeetJp8TpxdTb1QsGdH4dsnGqS\nmCtycm/MNW0KdLu5xifOsTmjlQE3oEkfvkvwmtqIMT2RXVuNrvp6VpjNvumyXSMI10bYVFYmlg4P\n0Vkb19Z58nGTpy+TYfkVdiBcb9F5X2MsbAKYqYQf9RiIY248n3bcajNYVDqIKjCO8LMA5QR2OcdY\nK8fo0x1XXEHrje9dyu9clabOoOyFUGjXuFbcu9OWrokejZurrnlONMOVQRKzyFW11jXBJEis2STz\n1/YXEEUZg7T5f00qxUJEdwiTYRKg9aWloS2ANDEOIKhdYFZt0imsrigKN8ZJzV9dLyDpqm1PcP1b\ne/c/sXw59TgE1VZgdpTBrM7kSiYaB1+nYhKglQL7UY+V+svNi6RFVuy5ZG4P9WdzayWR5JjPWfCf\nP19eHvGlSHNVakmVwbRHEyQCT7tJPImZ1lYMP/ssS4BaA+D0iRMsf1y/wdv34IEDqD1xAn8MFXnY\nAeWFvR/AW1bf3j19Gg8gyt23++xZXI4g+nDWv1YdgmiCPv5dAHs8D7u//OUQAekORD3A9wB4C8Bj\nk5Oo+eEPFbdiZSV0j5oAfBIy/59537OnTgEAFmYyGC4UivfZDGAvgIeN++4tLcVnCgVcXigUx6zr\ni1/E1z/7Wbz8gx/g46kUPuc/sz3m7wLor63F3GXLMJTJoI4IqVdfxannn8e+M2dwDCqqshSBl1/3\nMWV8Puq/g6P+PSYqK1HjRy3McX4GwCLjOpcDuPLKK/H2v/0bPm3Ni/9ChC9AEZ+a72n+mTP4u127\n0GTMi8VLlkw5MsNxcB4F0JfJoMXnqvzcK6/gpV/8AvXz5mHotdfwvnnzsO2119CyYAGen5jA5159\nVUU+/J9fAnjI/3zs17/G4K9/HeH4vB/A634UxyS41X0yox+J27mu9FP9AcBWnZH2SFxBFJ1c5NzH\nxjiLXHHlvkwmovWlqIQEkuFi8Hbeuem9HoTCOWShSDU4fL1tJhcQ9UjvQBhDYT7zyNZopSmT10GK\n8NiFVXoAuosZ847GRhrM5egOhyUwIozXiPW/vkymOE8kJ7CN2TAtQmleRI6fwsrqarY3X4/xuspK\n2t/aGqGfs4sAmVaXtuzMsXJxR9ogNy4P5GBLywzdJpDs2OPCNa5SadJ1OEgsAcUoQ199PbuvsxFn\nug/SfrUfUXNvG6LFXntLStiyaRw91UEExBrmTwFRgJKG0B4E6Aa/jqFpdtoREVdVqjhiGfO5uaQs\nmx/AfmdS6TOpQpA5Rn0GCag+p9vgprTnT1wURm8xdG0EewGYanOFFXWtiO3Mc2uqPXMe6zFPEpHZ\n0NQUu2joY2esMpC0/UA+H7IWJNbjOOYebn/LJrBYE9HGomunDQdl3lZRESE8Xe8XXCkgiPWug6oi\nNIBoBp4rPi6Nz2AuRz2VlaEYsok7MI83IyKFCb66krbQtII02Z47rfHRP4egwqPc6s3BtLWCm4Ti\nZbjJv/4g3FWy9Rhp60kzEfUD1O7zTnIOM2lcVyNcsUmHHYeY+06lxUWGpPfJFQHSPJFxWI0DAF03\nZ04kc1SyImakMjg+NkbZVCriFeVWPmmSn0s8VryWQ3gimHwEqdY6m85URCaTM7cC7kbYspFWsC1z\n5xYVVwF8Grd5X0lpmuE7ycraaLFIcStYnAVlPodI2GmOs/GdNqulcmXOKIYDm2LyWGg+CO5ZNmJq\njjjXnJNyAySY9ZZ0mt0qFyZUyfleIzHJpO7r8vucQ1Tp6qK+9nuekcpAWp05rZokHdX1cpIIzL3C\ndV2Y/DhuAMlnofe5Uuqxqez0hIjLXXBexzjnjuXL2RX7LuMcyRy1EZR20Vvz/hJRyCEEws8yRQmC\naJrgeoJrAlzXeeL4IawQDhmfz5fLcaqW3gam0K3pVxJJdo33Yl9zVUUFbWP8bNOiDAD8BwA/AvA0\ngH8AsMBxrCgkN3tecbKYWm2XX5r9XAg8k7yonK+dk+IPkuwvOVyCNuE0zkDfx4GWNRwAAAaXSURB\nVOU7sftctBLS6RBvA4dH2IkA8DQJ5cB0rdhaMboU5q1QZKOuMK2kULrBVx3mlIt9bY4sdZ8e03Og\nGx+xPutxPV8uR8kHNIpw9elJgLYYPp6QpWIwF7nQnwf892u/q5xQkGa6lEGt8bkfwF84jmUnWy/U\nHrAfUZITidYprnE4fHvPvAOgj5SXs15+bjWZhByHNxtXffkA1F53o/U8nM/jiSeeICKeXp2bGCbX\n482eF/In6L5vmTOHHfuDxuc4b/+3oLZSkmk7snWriCvQZi53fa4QrNlcEYXt2ax4notXUb8TTkFN\n1X9QmJgo+nIOAPR7/jN3I4ju6MjNIT+r0rTIjvvH9gG0Fkrh2tfeXFFBnTAYwYxcEHNMzK2KuSBN\n+zYBwP8C4GHH9+xL7aiupjsbGmJJPKbycrgV97Zslg4gnMTxaUvhuLYHSfMpOC/vpP+ipRoAZhse\nHo70xSWoeoxcfZe2LhGfgQMgNAxlwcUJjhTeE4FSMea5JNQ9lZW0f98+8TwRweo7HzXX5vnMOW6u\nLUfYmjSfU/dJvzeOgbuPmSeSf8p+F1px2ovHtCkDAJ8B8CKAHwNIO46LmrO1tcWiIVOFyE55EtTX\nR4Rq2Hr50sTrqqhw8hCara+ujr3GlrlzE/VfKwNzosWxBbly5XsqK8XJY7MCaQujM5WKhF8/DZnH\nwG5TCVXGCZ70PgdzueJYcS1JCPt8+QW4vrnGyWRp2gmZzdmskOR6PtuqlCIbSZVBLALR87xRqHTz\n4r+gbnCYiP4bEQ0BGPI871OISZ+20V3DR4/ii7t2FZFornz2pE3Kwa/1PLwLnodAI/Uk1CPq6nD5\nq6+K55nt9CWX4PS//3vkGu/Mnz+l5zDRcM88/jhOv/KKWLVaj5FU9Xnl6tVossZeI+lWrl4duu/K\n1avxgyefxBduugmXTk4Wr/M/oEqnJxkDbhw3Q6EUH5qcZLkTpCZyLjz4IP7yS18Sz+PQhDZ6UKwG\nnnDOcXOtHMDPEbwL8zkfPXIEp6GQk7cB+Cz4+WhXe5aez+Y2AJSMHWlrQ401X5M0j9Sqfd7N87yF\nAL5FRFcK31+YG8222TbbptyIyIs75rxyEzzPW0pEz/t/dgJ47nw6M9tm22y7eO28LAPP874K4ANQ\n+R8vALiDiP71AvVtts222TaN7YJtE2bbbJttv90tFX/IhWue5/0Hz/N+5Hne057n/YPneQum8/5C\nn/7Y87znPM/7oed5X/M8Lxk75W+2T5s8z/uJ53nveZ73kYvclw7P837qed7PfCfxRW+e533B87xX\nPM/78cXui26e5zV5nvddz/Oe9TzvGc/z9s2APlV4nvePvrw943nesPOEJCGHC/WDKYCUprFPWQAp\n//P9AP7TDOjT7wL4HQDfBfCRi9iPFBQVwWIAZQB+COCDM2B8VgFYDuDHF7svRp8WAFjuf64F8M8z\nZKyq/d8lAJ4C8PvSsdNqGRDRpPFnDZSv4aI2InqciHQ/noLiD7mojYj+mYh+DhXGvZjt9wH8nIhe\nIKJ3AfxXAPmL3CcQ0XEo9vkZ04joF0T0Q//zJJQzvfHi9gogojf9jxVQAQPRLzCtygAAPM/7jOd5\nLwLoAXDvdN8/pu2CCqnPNtUaAbxk/P0vmAETfKY3z/OaoSyXf7y4PQE8z0t5nvc0gF8AGCWi/ykd\ne8GVged5o57n/dj4ecb/vREAiGiIiBYB+ArUVuE33uL65B9zGMC7RPTXM6VPs+23r3meVwvgqwD2\nW5bwRWlEdJaIVkBZvK2e510hHXvBORCJaF3CQ/8awLcwhYIv59ri+uR53g4ANwK4/jfdF92mME4X\ns70MRTeoW5P/v9nGNM/zSqEUwZeJ6BsXuz9mI6LXPc97AkAHgHHumOmOJiw1/nSClKareZ7XAeAQ\ngBwRvXOx+8O0i+k3+J8Alnqet9jzvHIAtwL45kXsj9k8XHyfit2+CGCciP70YncEADzPu9TzvDr/\ncxWAdQB+Kh7vexqnpc1EkJLneT+HgpT/v/6/niKiOy9il+B5XicUQe6lAF4D8EMi+oOL1JcOAH8K\ntXB8gYjuvxj9MJvneX8NYA2ANIBXAAwT0V9e5D6tBPAkFHkz+T/3ENE/XMQ+XQngS1DvLgXgb4jo\nP4rHT6cymG2zbbbN3Dbt0YTZNttm28xss8pgts222QZgVhnMttk22/w2qwxm22ybbQBmlcFsm22z\nzW+zymC2zbbZBmBWGcy22Tbb/DarDGbbbJttAID/D4QDMBjp7+0KAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"def plot(inside, outside):\n",
" if inside.any():\n",
" plt.plot(inside[:, 0], inside[:, 1], 'bo')\n",
" if outside.any():\n",
" plt.plot(outside[:, 0], outside[:, 1], 'ro')\n",
" circle = plt.Circle((0, 0), radius=2, color='g', fill=False)\n",
" ax = plt.gca()\n",
" ax.set_aspect(1)\n",
" ax.add_patch(circle)\n",
" plt.show()\n",
"\n",
"points_test = X_test[:, :2]\n",
"\n",
"inside = np.array([x for x, y in zip(points_test, y_pred) if y == 0])\n",
"outside = np.array([x for x, y in zip(points_test, y_pred) if y == 1])\n",
"\n",
"plot(inside, outside)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Funciones de costo disponibles\n",
"\n",
"* mean_squared_error\n",
"* mean_absolute_error\n",
"* mean_absolute_percentage_error\n",
"* mean_squared_logarithmic_error\n",
"* squared_hinge\n",
"* hinge\n",
"* binary_crossentropy\n",
"* categorical_crossentropy\n",
"* sparse_categorical_crossentropy\n",
"* kullback_leibler_divergence\n",
"* poisson\n",
"* cosine_proximity"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Optimizadores disponibles\n",
"\n",
"* SGD\n",
"* RMSprop\n",
"* Adagrad\n",
"* Adadelta\n",
"* Adam\n",
"* Adamax\n",
"* Nadam"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"También se puede instanciar \"manualmente\" el optimizador para ajustar parámetros como la constante de aprendizaje, el momento y la desaceleración de la constante de aprendizaje:"
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [],
"source": [
"from keras.optimizers import SGD\n",
"\n",
"model.compile(loss='binary_crossentropy', optimizer=SGD(lr=0.01, momentum=0.9, decay=0.001), metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Funciones de activación disponibles\n",
"\n",
"* softmax\n",
"* softplus\n",
"* softsign\n",
"* relu\n",
"* tanh\n",
"* sigmoid\n",
"* hard_sigmoid\n",
"* linear"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Compatibilidad con scikit-learn\n",
"\n",
"Keras provee una API para usar los modelos como si fueran parte de scikit-learn."
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"8000/8000 [==============================] - 0s - loss: 0.1041 - acc: 0.9882 \n",
"Epoch 2/5\n",
"8000/8000 [==============================] - 0s - loss: 0.0892 - acc: 0.9924 \n",
"Epoch 3/5\n",
"8000/8000 [==============================] - 0s - loss: 0.0808 - acc: 0.9923 \n",
"Epoch 4/5\n",
"8000/8000 [==============================] - 0s - loss: 0.0752 - acc: 0.9923 \n",
"Epoch 5/5\n",
"8000/8000 [==============================] - 0s - loss: 0.0709 - acc: 0.9946 \n",
"Epoch 1/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0679 - acc: 0.9944 \n",
"Epoch 2/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0663 - acc: 0.9946 \n",
"Epoch 3/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0647 - acc: 0.9949 \n",
"Epoch 4/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0632 - acc: 0.9936 \n",
"Epoch 5/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0620 - acc: 0.9946 \n",
"1984/2667 [=====================>........] - ETA: 0sEpoch 1/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0605 - acc: 0.9953 \n",
"Epoch 2/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0594 - acc: 0.9961 \n",
"Epoch 3/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0586 - acc: 0.9949 \n",
"Epoch 4/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0577 - acc: 0.9957 \n",
"Epoch 5/5\n",
"5333/5333 [==============================] - 0s - loss: 0.0571 - acc: 0.9953 \n",
"1472/2667 [===============>..............] - ETA: 0sEpoch 1/5\n",
"5334/5334 [==============================] - 0s - loss: 0.0572 - acc: 0.9961 \n",
"Epoch 2/5\n",
"5334/5334 [==============================] - 0s - loss: 0.0563 - acc: 0.9963 \n",
"Epoch 3/5\n",
"5334/5334 [==============================] - 0s - loss: 0.0558 - acc: 0.9983 \n",
"Epoch 4/5\n",
"5334/5334 [==============================] - 0s - loss: 0.0553 - acc: 0.9966 \n",
"Epoch 5/5\n",
"5334/5334 [==============================] - 0s - loss: 0.0547 - acc: 0.9972 \n",
"1376/2666 [==============>...............] - ETA: 0s"
]
}
],
"source": [
"from keras.wrappers import scikit_learn\n",
"from sklearn import cross_validation\n",
"\n",
"classifier = scikit_learn.KerasClassifier(build_fn=lambda: model, nb_epoch=5)\n",
"# build_fn debe devolver un modelo compilado.\n",
"\n",
"classifier.fit(X_train, y_train)\n",
"\n",
"kfold = cross_validation.StratifiedKFold(y=y_train, n_folds=3, shuffle=True)\n",
"results = cross_validation.cross_val_score(classifier, X_train, y_train, cv=kfold)"
]
},
{
"cell_type": "code",
"execution_count": 90,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Acierto: 0.996625140537\n"
]
}
],
"source": [
"print('Acierto:', results.mean())"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"También está disponible la clase `KerasRegressor`. Con esto se pueden hacer cosas como usar `GridSearchCV` para buscar la combinación de parámetros que mejoran los resultados (valores de la constante de aprendizaje, momento, etc), así como también construir un pipeline en donde el modelo puede tomar datos a partir de otros modelos de scikit-learn."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Ejemplo de análisis de sentimientos de comentarios de IMDb\n",
"\n",
"Este es un ejemplo que aprovecha a mostrar más funcionalidades."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Se comienza estableciendo algunos parámetros:"
]
},
{
"cell_type": "code",
"execution_count": 91,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from keras.preprocessing import sequence\n",
"from keras.layers import Dropout\n",
"from keras.layers import Embedding\n",
"from keras.layers import LSTM, GRU, SimpleRNN\n",
"from keras.layers import Convolution1D, MaxPooling1D\n",
"from keras.datasets import imdb\n",
"\n",
"\n",
"# Embedding\n",
"max_features = 20000\n",
"maxlen = 100\n",
"embedding_size = 128\n",
"\n",
"# Convolution\n",
"filter_length = 5\n",
"nb_filter = 64\n",
"pool_length = 4\n",
"\n",
"# LSTM\n",
"lstm_output_size = 70\n",
"\n",
"# Training\n",
"batch_size = 30\n",
"nb_epoch = 2"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Se cargan los datos:"
]
},
{
"cell_type": "code",
"execution_count": 92,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cargando datos...\n",
"20000 ejemplos de entrenamiento\n",
"5000 ejemplos de evaluación\n"
]
}
],
"source": [
"print('Cargando datos...')\n",
"(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features)\n",
"print(len(X_train), 'ejemplos de entrenamiento')\n",
"print(len(X_test), 'ejemplos de evaluación')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Veamos dos de los ejemplos:"
]
},
{
"cell_type": "code",
"execution_count": 93,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1, 621, 6, 135, 101, 84, 392, 27, 20, 133, 1522, 63, 6401, 6843, 896, 11, 213, 149, 9, 417, 180, 1748, 32, 63, 31, 525, 7, 78, 42, 147, 66, 644, 113, 89, 8, 21, 147, 89, 13, 1851, 3994, 43, 170, 6, 60, 21, 296, 35, 1310, 214, 6, 789, 7146, 7, 15, 16, 12, 14, 15, 16, 12, 14, 5, 927, 10, 5, 6843, 13, 62, 23, 652, 25, 2046, 927, 7, 73, 574, 49, 5, 6843, 1264, 56, 135, 46, 38, 6, 5, 191, 342, 10, 14274, 10, 6843, 16849, 2572, 13, 627, 7, 412, 18, 361, 6, 20, 11393, 342, 17222, 45, 241, 382, 5, 28, 7, 15, 16, 12, 14, 15, 16, 12, 14, 5, 132, 18, 5, 28, 24, 3773, 209, 6, 2380, 61, 6, 2082, 146, 10885, 6, 2962, 146, 1003, 6, 523, 146, 910, 6, 99, 7, 19, 165, 266, 53, 23, 460, 6, 29, 33, 199, 190, 11, 41, 286, 8436, 11, 186, 17, 7, 5, 78, 1522, 24, 89, 33, 4317, 17, 551, 1851, 3994, 43, 37, 240, 40, 635, 9, 189, 331, 4183, 45, 5, 2, 6, 102, 37, 24, 5, 137, 18, 5, 757, 7, 15, 16, 12, 14, 15, 16, 12, 14, 25, 26, 212, 63, 20, 30, 13, 36, 11, 41, 635, 636, 7, 53, 230, 35, 212, 43, 46, 199, 6843, 26, 2539, 61, 1401, 7, 5, 453, 4693, 231, 112, 40, 93, 4232, 27, 9, 5926, 2, 56, 127, 7, 78, 124, 20, 30, 18, 9, 3245, 617, 2806, 6, 911, 19, 66, 82, 64, 681, 4058, 11, 5, 1324, 7]\n",
"\n",
"[1, 19, 101, 186, 17, 9, 243, 3850, 7, 295, 17, 23, 57, 102, 19, 158, 131, 129, 9, 9201, 328, 32, 3071, 4423, 43, 31, 6, 29, 19, 57, 107, 35, 94, 100, 17, 52, 9, 192, 10, 97, 95, 101, 7, 5, 1969, 13, 80, 103, 171, 25, 2175, 19621, 9, 4563, 6, 9, 154, 21, 313, 18, 72, 447, 193, 76, 19, 160, 5864, 43]\n"
]
}
],
"source": [
"print(X_train[1])\n",
"print()\n",
"print(X_train[16])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Cada ejemplo es una secuencia de índices. Se transforman a vectores de largo fijo, rellenando con ceros:"
]
},
{
"cell_type": "code",
"execution_count": 94,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Tamaño de X_train: (20000, 100)\n",
"Tamaño de X_test: (5000, 100)\n"
]
}
],
"source": [
"X_train = sequence.pad_sequences(X_train, maxlen=maxlen)\n",
"X_test = sequence.pad_sequences(X_test, maxlen=maxlen)\n",
"print('Tamaño de X_train:', X_train.shape)\n",
"print('Tamaño de X_test:', X_test.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Los mismos dos ejemplos ahora:"
]
},
{
"cell_type": "code",
"execution_count": 95,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 24 89 33 4317 17 551 1851 3994 43 37 240 40 635 9 189\n",
" 331 4183 45 5 2 6 102 37 24 5 137 18 5 757 7\n",
" 15 16 12 14 15 16 12 14 25 26 212 63 20 30 13\n",
" 36 11 41 635 636 7 53 230 35 212 43 46 199 6843 26\n",
" 2539 61 1401 7 5 453 4693 231 112 40 93 4232 27 9 5926\n",
" 2 56 127 7 78 124 20 30 18 9 3245 617 2806 6 911\n",
" 19 66 82 64 681 4058 11 5 1324 7]\n",
"\n",
"[ 0 0 0 0 0 0 0 0 0 0 0 0\n",
" 0 0 0 0 0 0 0 0 0 0 0 0\n",
" 0 0 0 0 0 0 0 0 1 19 101 186\n",
" 17 9 243 3850 7 295 17 23 57 102 19 158\n",
" 131 129 9 9201 328 32 3071 4423 43 31 6 29\n",
" 19 57 107 35 94 100 17 52 9 192 10 97\n",
" 95 101 7 5 1969 13 80 103 171 25 2175 19621\n",
" 9 4563 6 9 154 21 313 18 72 447 193 76\n",
" 19 160 5864 43]\n"
]
}
],
"source": [
"print(X_train[1])\n",
"print()\n",
"print(X_train[16])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Ahora construimos un modelo:"
]
},
{
"cell_type": "code",
"execution_count": 96,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"model = Sequential()\n",
"model.add(Embedding(max_features, embedding_size, input_length=maxlen))\n",
"model.add(Dropout(0.25))\n",
"model.add(Convolution1D(nb_filter=nb_filter,\n",
" filter_length=filter_length,\n",
" border_mode='valid',\n",
" activation='relu',\n",
" subsample_length=1))\n",
"model.add(MaxPooling1D(pool_length=pool_length))\n",
"model.add(LSTM(lstm_output_size))\n",
"model.add(Dense(1))\n",
"model.add(Activation('sigmoid'))\n",
"\n",
"model.compile(loss='binary_crossentropy',\n",
" optimizer='adam',\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Visualización"
]
},
{
"cell_type": "code",
"execution_count": 97,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 97,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"SVG(model_to_dot(model).create(prog='dot', format='svg'))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Lo entrenamos y evaluamos:"
]
},
{
"cell_type": "code",
"execution_count": 98,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Entrenando...\n",
"Train on 20000 samples, validate on 5000 samples\n",
"Epoch 1/2\n",
"20000/20000 [==============================] - 137s - loss: 0.4298 - acc: 0.7888 - val_loss: 0.3346 - val_acc: 0.8492\n",
"Epoch 2/2\n",
"20000/20000 [==============================] - 138s - loss: 0.2130 - acc: 0.9182 - val_loss: 0.3381 - val_acc: 0.8504\n",
"5000/5000 [==============================] - 9s \n",
"Valor de la función de costo: 0.338069551766\n",
"Acierto: 0.850399994493\n"
]
}
],
"source": [
"print('Entrenando...')\n",
"model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=nb_epoch,\n",
" validation_data=(X_test, y_test))\n",
"score, acc = model.evaluate(X_test, y_test, batch_size=batch_size)\n",
"print('Valor de la función de costo:', score)\n",
"print('Acierto:', acc)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Desafortunadamente los datos de este ejemplo tienen solamente los índices de las palabras, así que no podemos probar el modelo con texto."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Capas disponibles\n",
"\n",
"Hay muchas capas disponibles. Se destacan:\n",
"\n",
"* `Dense`\n",
"* `Activation`\n",
"* `Merge`: permite combinar varias capas ya sea concatenándolas, sumándolas, etc.\n",
"* `Reshape`\n",
"* `Permute`\n",
"* `RepeatVector`\n",
"* Convolutivas (incluídas las de Pooling)\n",
"* Recurrentes (incluyendo `LSTM`)\n",
"* `Embedding`: permite representar con vectores de dimensión fija a números naturales de un rango dado (índices). Los pesos que se entrenan en esta capa son las entradas de la matriz que hace corresponder vectores a índices (que oficia de tabla de look-up).\n",
"* `BatchNormalization`: para dejar la norma de los vectores cercana a 0 y con desviación estándar cercana a 1.\n",
"* Aquellas que agregan ruido con el objetivo de no sobreajustar:\n",
" * `Dropout`: Inhibe aleatoria una fracción `p` de las entradas.\n",
" * `GaussianNoise`: Agrega un ruido gaussiano a la entrada."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Otro ejemplo\n",
"\n",
"Clasificación de imágenes."
]
},
{
"cell_type": "code",
"execution_count": 99,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"from keras.datasets import cifar10\n",
"from keras.preprocessing.image import ImageDataGenerator\n",
"from keras.models import Sequential\n",
"from keras.layers import Dense, Dropout, Activation, Flatten\n",
"from keras.layers import Convolution2D, MaxPooling2D\n",
"from keras.optimizers import SGD\n",
"from keras.utils import np_utils\n",
"\n",
"batch_size = 32\n",
"nb_classes = 10\n",
"nb_epoch = 1\n",
"data_augmentation = False\n",
"\n",
"# input image dimensions\n",
"img_rows, img_cols = 32, 32\n",
"# the CIFAR10 images are RGB\n",
"img_channels = 3"
]
},
{
"cell_type": "code",
"execution_count": 100,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Dimensión de X_train: (50000, 3, 32, 32)\n",
"50000 ejemplos de entrenamiento\n",
"10000 ejemplos de evaluación\n",
"\n",
"X_train[0]: [[[ 59 43 50 ..., 158 152 148]\n",
" [ 16 0 18 ..., 123 119 122]\n",
" [ 25 16 49 ..., 118 120 109]\n",
" ..., \n",
" [208 201 198 ..., 160 56 53]\n",
" [180 173 186 ..., 184 97 83]\n",
" [177 168 179 ..., 216 151 123]]\n",
"\n",
" [[ 62 46 48 ..., 132 125 124]\n",
" [ 20 0 8 ..., 88 83 87]\n",
" [ 24 7 27 ..., 84 84 73]\n",
" ..., \n",
" [170 153 161 ..., 133 31 34]\n",
" [139 123 144 ..., 148 62 53]\n",
" [144 129 142 ..., 184 118 92]]\n",
"\n",
" [[ 63 45 43 ..., 108 102 103]\n",
" [ 20 0 0 ..., 55 50 57]\n",
" [ 21 0 8 ..., 50 50 42]\n",
" ..., \n",
" [ 96 34 26 ..., 70 7 20]\n",
" [ 96 42 30 ..., 94 34 34]\n",
" [116 94 87 ..., 140 84 72]]]\n"
]
}
],
"source": [
"(X_train, y_train), (X_test, y_test) = cifar10.load_data()\n",
"print('Dimensión de X_train:', X_train.shape)\n",
"print(X_train.shape[0], 'ejemplos de entrenamiento')\n",
"print(X_test.shape[0], 'ejemplos de evaluación')\n",
"print()\n",
"print('X_train[0]:', X_train[0])"
]
},
{
"cell_type": "code",
"execution_count": 101,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"y_train[0]: [6]\n",
"y_train[1]: [9]\n"
]
}
],
"source": [
"print('y_train[0]:', y_train[0])\n",
"print('y_train[1]:', y_train[1])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Transformamos las etiquetas a one-hot encoding:"
]
},
{
"cell_type": "code",
"execution_count": 102,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Y_train[0]: [ 0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]\n",
"Y_train[1]: [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]\n"
]
}
],
"source": [
"Y_train = np_utils.to_categorical(y_train, nb_classes)\n",
"Y_test = np_utils.to_categorical(y_test, nb_classes)\n",
"\n",
"print('Y_train[0]:', Y_train[0])\n",
"print('Y_train[1]:', Y_train[1])"
]
},
{
"cell_type": "code",
"execution_count": 103,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"model = Sequential()\n",
"\n",
"model.add(Convolution2D(32, 3, 3, border_mode='same', input_shape=(img_channels, img_rows, img_cols)))\n",
"model.add(Activation('relu'))\n",
"model.add(Convolution2D(32, 3, 3))\n",
"model.add(Activation('relu'))\n",
"model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"model.add(Dropout(0.25))\n",
"\n",
"model.add(Convolution2D(64, 3, 3, border_mode='same'))\n",
"model.add(Activation('relu'))\n",
"model.add(Convolution2D(64, 3, 3))\n",
"model.add(Activation('relu'))\n",
"model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"model.add(Dropout(0.25))\n",
"\n",
"model.add(Flatten())\n",
"model.add(Dense(512))\n",
"model.add(Activation('relu'))\n",
"model.add(Dropout(0.5))\n",
"model.add(Dense(nb_classes))\n",
"model.add(Activation('softmax'))\n",
"\n",
"sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)\n",
"model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])"
]
},
{
"cell_type": "code",
"execution_count": 104,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 104,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"SVG(model_to_dot(model).create(prog='dot', format='svg'))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Cambiamos el intervalo de los datos a [0, 1]:"
]
},
{
"cell_type": "code",
"execution_count": 105,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"X_train = X_train.astype('float32')\n",
"X_test = X_test.astype('float32')\n",
"X_train /= 255\n",
"X_test /= 255"
]
},
{
"cell_type": "code",
"execution_count": 106,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 50000 samples, validate on 10000 samples\n",
"Epoch 1/1\n",
"50000/50000 [==============================] - 898s - loss: 1.7448 - acc: 0.3547 - val_loss: 1.2809 - val_acc: 0.5377\n"
]
}
],
"source": [
"if not data_augmentation:\n",
" model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,\n",
" validation_data=(X_test, Y_test), shuffle=True)\n",
"else:\n",
" datagen = ImageDataGenerator(\n",
" featurewise_center=False, # set input mean to 0 over the dataset\n",
" samplewise_center=False, # set each sample mean to 0\n",
" featurewise_std_normalization=False, # divide inputs by std of the dataset\n",
" samplewise_std_normalization=False, # divide each input by its std\n",
" zca_whitening=False, # apply ZCA whitening\n",
" rotation_range=0, # randomly rotate images in the range (degrees, 0 to 180)\n",
" width_shift_range=0.1, # randomly shift images horizontally (fraction of total width)\n",
" height_shift_range=0.1, # randomly shift images vertically (fraction of total height)\n",
" horizontal_flip=True, # randomly flip images\n",
" vertical_flip=False) # randomly flip images\n",
"\n",
" # compute quantities required for featurewise normalization\n",
" # (std, mean, and principal components if ZCA whitening is applied)\n",
" datagen.fit(X_train)\n",
"\n",
" model.fit_generator(datagen.flow(X_train, Y_train, batch_size=batch_size),\n",
" samples_per_epoch=X_train.shape[0], nb_epoch=nb_epoch,\n",
" validation_data=(X_test, Y_test))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Otra arquitectura de red para el mismo ejemplo"
]
},
{
"cell_type": "code",
"execution_count": 107,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 107,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X_train = X_train.reshape(50000, 3072)\n",
"X_test = X_test.reshape(10000, 3072)\n",
"\n",
"model2 = Sequential()\n",
"model2.add(Dense(100, input_dim=3072, activation='relu'))\n",
"model2.add(Dense(10, activation='relu'))\n",
"\n",
"sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)\n",
"model2.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])\n",
"\n",
"SVG(model_to_dot(model2).create(prog='dot', format='svg'))"
]
},
{
"cell_type": "code",
"execution_count": 108,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 50000 samples, validate on 10000 samples\n",
"Epoch 1/1\n",
"50000/50000 [==============================] - 18s - loss: 7.6269 - acc: 0.1000 - val_loss: 7.5525 - val_acc: 0.1000\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 108,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model2.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, validation_data=(X_test, Y_test), shuffle=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Functional API\n",
"\n",
"Para tener más flexibilidad a la hora de construir una red neuronal que con `Sequential`, se puede usar esta parte de la biblioteca. Se pueden hacer cosas como:\n",
"\n",
"* Tener varias capas de salida.\n",
"* Modelar una arquitectura de grafo dirigido acíclico.\n",
"* Modelos que compartan capas."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Un ejemplo simple:"
]
},
{
"cell_type": "code",
"execution_count": 109,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [],
"source": [
"from keras.layers import Input, Embedding, LSTM, Dense, merge\n",
"from keras.models import Model\n",
"\n",
"main_input = Input(shape=(100,), dtype='int32', name='main_input')\n",
"\n",
"x = Embedding(output_dim=512, input_dim=10000, input_length=100)(main_input)\n",
"\n",
"lstm_out = LSTM(32)(x)\n",
"\n",
"auxiliary_loss = Dense(1, activation='sigmoid', name='aux_output')(lstm_out)\n",
"\n",
"auxiliary_input = Input(shape=(5,), name='aux_input')\n",
"x = merge([lstm_out, auxiliary_input], mode='concat')\n",
"\n",
"x = Dense(64, activation='relu')(x)\n",
"x = Dense(64, activation='relu')(x)\n",
"x = Dense(64, activation='relu')(x)\n",
"\n",
"main_loss = Dense(1, activation='sigmoid', name='main_output')(x)\n",
"\n",
"model = Model(input=[main_input, auxiliary_input], output=[main_loss, auxiliary_loss])\n",
"\n",
"model.compile(optimizer='rmsprop', loss='binary_crossentropy',\n",
" loss_weights=[1., 0.2])"
]
},
{
"cell_type": "code",
"execution_count": 110,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 110,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"SVG(model_to_dot(model).create(prog='dot', format='svg'))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Callbacks\n",
"\n",
"Permiten realizar acciones luego de ciertos eventos como al terminar de entrenar un batch o un epoch. Casos de usos de ejemplo:\n",
"\n",
"* Mostrar una barra de progreso mientras se entrena.\n",
"* Guardar un histórico de medidas.\n",
"* Definir un criterio de parada personalizado. Por ejemplo, dejar de entrenar el modelo si no mejora luego de cierta cantidad de iteraciones.\n",
"* Persistir el modelo cada cierto tiempo, por si se corta la energía.\n",
"* Monitorear remotamente el avance de un entrenamiento."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Hay más\n",
"\n",
"Hay más cosas que se vieron poco acá pero que vale la pena mencionar que existen:\n",
"\n",
"* Usar TensorFlow en lugar de Theano como backend.\n",
"* Poder controlar más el entrenamiento. Por ejemplo para el caso que se precisen manejar datos que no entren en memoria.\n",
"* Cambiar los valores de inicialización de los pesos, con varias distribuciones de probabilidad para asignación aleatoria de valores de los pesos ya disponibles.\n",
"* **Regulzarizadores**: permiten penalizar la función de costo si los pesos cumplen determinadas condiciones. Para combatir el sobreajuste.\n",
"* **Restricciones**: se pueden poner restricciones en los pesos, como por ejemplo que su norma sea menor a un número dado o que sean no negativos.\n",
"* Aceleración de cálculos utiilizando la GPU."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Referencias\n",
"\n",
"* [Documentación de Keras](https://keras.io/)\n",
"* [Ejemplos del repositorio de Keras](https://github.com/fchollet/keras/tree/master/examples)\n",
"* [keras-resources](https://github.com/fchollet/keras-resources)"
]
}
],
"metadata": {
"celltoolbar": "Slideshow",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}