"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So far all the tensors that we created were all constants where you can do operations that generate new tensors but you can never change the value of any tensor after creating it. To start doing \"stateful\" programming which keeps and updates values (or state) or tesnors you need to wrap your tensors in an instance of `ft.Varbiable()`.\n",
"\n",
"As usaul before we start, let's import TensorFlow and start an interactive session."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Python Version: 3.5.2\n",
"TensorFlow Version: 1.7.0\n"
]
}
],
"source": [
"import tensorflow as tf\n",
"import sys\n",
"\n",
"print(\"Python Version:\",sys.version.split(\" \")[0])\n",
"print(\"TensorFlow Version:\",tf.VERSION)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Creating Variables\n",
"\n",
"Let start a new interactive session and create some variables. To create a variables, your this function:\n",
"\n",
"`tf.Variable(initial_value=None, trainable=True, collections=None, validate_shape=True, caching_device=None, name=None, variable_def=None, dtype=None, expected_shape=None, import_scope=None, constraint=None)`"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sess = tf.InteractiveSession()\n",
"\n",
"a = tf.Variable(tf.ones((2,2)), name=\"a\")\n",
"a"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The first parameter that we passed to when creating an instanse of `tf.Variable()` is `initial_value`. This can accept a tensor that has values or a tensor initilizer method. We will discuss some of these later in this tutorial.\n",
"\n",
"You can also use `ft.get_variable()` function to create a variable.\n",
"\n",
"`tf.get_variable(name, shape=None, dtype=None, initializer=None, regularizer=None, trainable=True, collections=None, caching_device=None, partitioner=None, validate_shape=True, use_resource=None, custom_getter=None, constraint=None)`"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"b = tf.get_variable(\"b\", [2,2])\n",
"b"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This creates a variable named `b` with shape `(2,2)`.\n",
"\n",
"To initlize the value of your variable, you could use one of the many inilization method available in TensorFlow.\n",
"\n",
"`tf.zeros_initializer`"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"c = tf.get_variable(\"c\", [2,2], dtype=tf.float32, initializer=tf.zeros_initializer)\n",
"c"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Similar to `tf.zeros_initializer` there is `tf.ones_initializer` which initializes your tensor with ones.\n",
"\n",
"There are also `tf.random_normal_initializer` and `random_uniform_initializer` that inialize your variables with a normal or uniform distribution. For truncated normal distribution, you can use `tf.truncated_normal_initializer` which will limit your normal distribution to 2 standard diviations from the mean.\n",
"\n",
"You can also initialize your variables with a constant."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"d = tf.get_variable(\"d\", initializer=tf.constant([1,2,3]))\n",
"d"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Initialize Variables\n",
"\n",
"Before you can use any of your variables, you should first run an operation that initializes them. To initialize all the variabes that that created already, you can use `tf.global_variables_initializer` to create the operation this you have to run that operation using your session."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"init_op = tf.global_variables_initializer()\n",
"sess.run(init_op)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that you initlized your variables, you can start executing this and getting their values. To do that, you can just call `eval()` method of the variable."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([1, 2, 3], dtype=int32)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"d.eval()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Manually Initializing Variables\n",
"\n",
"Sometimes specially in an interactive environment, you would want to initialize some extra variables after you initialized all your variables using `tf.global_variables_initializer`. To do that, you can run one variable initializer which is an operation that can be retrieved for a single variable from the `initializer`."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"e = tf.get_variable(\"e\", initializer=tf.constant([2,2,2]))\n",
"sess.run(e.initializer)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you try to redecalre your variable with the same name, you will get an error message because TensorFlow does't know if you want to resue the same variable or you want a new one.\n",
"\n",
"To avoid that clarify that you mean to reuse the same variable and you just to reinitize the variable using "
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "Trying to share variable e/e, but specified dtype float32 and found dtype int32_ref.",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariable_scope\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreuse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mAUTO_REUSE\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname_or_scope\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"e\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0me\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_variable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"e\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minitializer\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconstant\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m 1295\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1296\u001b[0m \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcustom_getter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcustom_getter\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1297\u001b[0;31m constraint=constraint)\n\u001b[0m\u001b[1;32m 1298\u001b[0m get_variable_or_local_docstring = (\n\u001b[1;32m 1299\u001b[0m \"\"\"%s\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m 1091\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1092\u001b[0m \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcustom_getter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcustom_getter\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1093\u001b[0;31m constraint=constraint)\n\u001b[0m\u001b[1;32m 1094\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1095\u001b[0m def _get_partitioned_variable(self,\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m 437\u001b[0m \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 438\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 439\u001b[0;31m constraint=constraint)\n\u001b[0m\u001b[1;32m 440\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 441\u001b[0m def _get_partitioned_variable(\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36m_true_getter\u001b[0;34m(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, constraint)\u001b[0m\n\u001b[1;32m 406\u001b[0m \u001b[0mtrainable\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrainable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcollections\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcollections\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 407\u001b[0m \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 408\u001b[0;31m use_resource=use_resource, constraint=constraint)\n\u001b[0m\u001b[1;32m 409\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 410\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcustom_getter\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36m_get_single_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource, constraint)\u001b[0m\n\u001b[1;32m 756\u001b[0m raise ValueError(\"Trying to share variable %s, but specified dtype %s\"\n\u001b[1;32m 757\u001b[0m \" and found dtype %s.\" % (name, dtype_str,\n\u001b[0;32m--> 758\u001b[0;31m found_type_str))\n\u001b[0m\u001b[1;32m 759\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mfound_var\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 760\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mValueError\u001b[0m: Trying to share variable e/e, but specified dtype float32 and found dtype int32_ref."
]
}
],
"source": [
"with tf.variable_scope(reuse=tf.AUTO_REUSE, name_or_scope=\"e\"):\n",
" e = tf.get_variable(\"e\", initializer=tf.constant([2,2,2]))"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Assigning Value to Variables\n",
"\n",
"So far variables are not much different that any constant tensor. Variables get interesting once you can change their values. To do that you can use `tf.assign()` function or the `assign()` method of a variable. These are operation and should be run using a your session for them to perform their assignment."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([2, 3, 4], dtype=int32)"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sess.run(d.assign([2,3,4]))\n",
"d.eval()"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([4, 5, 6], dtype=int32)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sess.run(tf.assign_add(d, [2,2,2]))\n",
"d.eval()"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([1, 2, 3], dtype=int32)"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sess.run(tf.assign_sub(d, [3,3,3]))\n",
"d.eval()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Variable Scrope\n",
"\n",
"You can group variables in a few way in TensorFlow and one these methods is Variable Scope"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"with tf.name_scope(\"dense1\"):\n",
" a = tf.get_variable(\"a\", (3,))\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"with tf.name_scope(\"dense1\"):\n",
" b = tf.Variable(tf.ones((2,2)), name=\"b\")\n",
"b"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"with tf.name_scope(\"dense1\"):\n",
" c = tf.Variable(tf.ones((2,2)), name=\"c\")\n",
" w = c+1\n",
"w"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "Variable dense1/d already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:\n\n File \"\", line 2, in \n d = tf.get_variable(\"d\", (3,))\n File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2963, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2903, in run_ast_nodes\n if self.run_code(code, result):\n",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariable_scope\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"dense1\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0md\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_variable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"d\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3\u001b[0m \u001b[0me\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0md\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m 1295\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1296\u001b[0m \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcustom_getter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcustom_getter\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1297\u001b[0;31m constraint=constraint)\n\u001b[0m\u001b[1;32m 1298\u001b[0m get_variable_or_local_docstring = (\n\u001b[1;32m 1299\u001b[0m \"\"\"%s\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m 1091\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1092\u001b[0m \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcustom_getter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcustom_getter\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1093\u001b[0;31m constraint=constraint)\n\u001b[0m\u001b[1;32m 1094\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1095\u001b[0m def _get_partitioned_variable(self,\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)\u001b[0m\n\u001b[1;32m 437\u001b[0m \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 438\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muse_resource\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_resource\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 439\u001b[0;31m constraint=constraint)\n\u001b[0m\u001b[1;32m 440\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 441\u001b[0m def _get_partitioned_variable(\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36m_true_getter\u001b[0;34m(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, constraint)\u001b[0m\n\u001b[1;32m 406\u001b[0m \u001b[0mtrainable\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrainable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcollections\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcollections\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 407\u001b[0m \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 408\u001b[0;31m use_resource=use_resource, constraint=constraint)\n\u001b[0m\u001b[1;32m 409\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 410\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcustom_getter\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py\u001b[0m in \u001b[0;36m_get_single_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource, constraint)\u001b[0m\n\u001b[1;32m 745\u001b[0m \u001b[0;34m\"reuse=tf.AUTO_REUSE in VarScope? \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 746\u001b[0m \"Originally defined at:\\n\\n%s\" % (\n\u001b[0;32m--> 747\u001b[0;31m name, \"\".join(traceback.format_list(tb))))\n\u001b[0m\u001b[1;32m 748\u001b[0m \u001b[0mfound_var\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_vars\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 749\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_compatible_with\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfound_var\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_shape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mValueError\u001b[0m: Variable dense1/d already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:\n\n File \"\", line 2, in \n d = tf.get_variable(\"d\", (3,))\n File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2963, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py\", line 2903, in run_ast_nodes\n if self.run_code(code, result):\n"
]
}
],
"source": [
"with tf.variable_scope(\"dense1\"):\n",
" d = tf.get_variable(\"d\", (3,))\n",
" e = d + c"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"with tf.variable_scope(\"dense1\", ):\n",
" f = tf.get_variable(\"f\", (3,))\n",
" g = tf.get_variable(\"g\", (3,))\n",
" h = f + g\n",
"h"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# The Match Behind It\n",
"\n",
"Since we are talking about variables we cannot escape the fact that we will use them in the next tutorial for creating a predective model. This means we will have to get a head start start with some basic concepts about calculus. Calculus is branch of math that studies change. It can study the change of a variable as it relates to another variable. So basically it studies the realtionship between two or more variables. There are two main studies in calculus:\n",
"\n",
"- Diffrentiation\n",
"- Integration\n",
"\n",
"## Diffrentiation\n",
"\n",
"We will focus for now on diffrentiation because the we will use that to train neural network with an algorithm called \"Back Probagation\". Diffrentiation is the study of the rate of change or the slope.\n",
"\n",
"We will use numpy and matplotlib for illustration so let's import them"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Slope\n",
"\n",
"Slope is the mesure of change of a variable as another variables changes. In here we will see the change in $y$ as $x$ changes. The mathematical way of saying that is:\n",
"\n",
"$$\\frac{dy}{dx} = \\frac{\\Delta y}{\\Delta x}$$\n",
"\n",
"$\\Delta$ is the capital letter delta and it means the change of. So the change of $y$ as $x$ changes.\n",
"\n",
"- Slope is __positive__ if numbers are __increasing__.\n",
"- Slope is __nagative__ if numbers are __decreasing__.\n",
"- Slope is __zero__ if numbers are __not changing__.\n",
"\n",
"Let's look at three lines to show how slope works."
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAArwAAADSCAYAAACo2xNAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XeYVOX5xvHvs422VOlFQKqAFFnqYo0xWMKCoIIURRSBxRI1MWoSkxijP1tisnRRFAWUTowNu+zSkd5Bekf6Ura8vz9miOu6wAI7c3Zm7s91zeXMnDNn7sF59jxz5n3PmHMOEREREZFwFeV1ABERERGRQFLDKyIiIiJhTQ2viIiIiIQ1NbwiIiIiEtbU8IqIiIhIWFPDKyIiIiJhTQ1vIWFml5rZUTOLPss6R83ssiDn+srM7gvmc4qEIzP7yMzuDvJz/tnM3gnmc4qI9p2FkRreC2Rmm8zsuL8J3W1mY8ws/kK355zb4pyLd85l+bf/s2LxL994sdlzM7OnzOx7/2vZZmbvFfRziASbv0b3mFmJHPfdZ2ZfBeG5f9ZoOuducs69FYDnSjKzxWZ22Mz2mdkXZla7oJ9HpDAxs57+fVbuizOzPwUpg/adIUQN78X5tXMuHrgSSAD+4HGe8+Y/4tQbuMH/WhKAz71NJVJgooGHvQ4RKGZWF3gbeAwoDdQGhgBZXuYSCTTn3Lv+g0D/uwCPALuBUee7PTOLOc/1te8MMWp4C4BzbjvwEdAEwMyqmtkMM/vBzNab2f2n1zWz1ma2wH80ZreZveq/v5b/k2mMmT0HXAWk+D85pvjXcWZW18zamNmunMMfzKyLmS31X48ys9+b2QYz229m75tZuTPEbwV84pzb4H8tu5xzI/Na0b/dP5jZZv+Rs7fNrHSu/P3NbIeZ7TSzx3M9Nr+ZRArKS8DjZlYmr4Vm1tDMZvprdY2Z3ZFj2SVm9h9/rc43s7+Z2awcy18zs63+5QvN7Cr//R2Bp4A7/fW7xH//V/4jzEXM7KCZNcmxrQr+b4wq+m/f6j9qe9DM0sys6RleX3Pge+fc587niHNusnNuyxlebyczW+Hf7ldmdnmOZZvM7EkzW2lmB8zsTTMrmmN5fjOJBJ2ZtQD+CXR3zu3031fazEb790fb/TUc7V92j5mlmtk/zGw/8Oez7ePyoH1niFHDWwDMrAZwM/Cd/64JwDagKtAN+LuZXe9f9hrwmnOuFFAHeD/39pxzTwPfAoP9n1wH51o+FzgGXJ/j7ruAcf7rDwKdgWv8GQ7gO+qTlzlAHzP7rZkl2FnGEAP3+C/XAZcB8UBKrnWuA+oBNwJPmNkNF5BJpKAsAL4CHs+9wHxDHWbiq5uKQHdgqJk18q8yBF+dVQbu9l9ymo+v4Szn38ZEMyvqnPsY+Dvwnr9+m+V8kHPuJDAF6JHj7juAr51ze/w77jeAB4BLgBHADDMrksfrWwQ09O+0r7OzDKsys/rAeHxHwSoAHwL/MbO4HKv1BH6F729TffzfWp1nJpGg8n+gnQQ865z7KseiMUAmUBdogW+/lHOoYBtgI1AJeI787eNO074z1DjndLmAC7AJOAocBDYDQ4FiQA18XyeWzLHu88AY//VvgL8A5XNtrxbggBj/7a+A+3Kt44C6/ut/A97wXy+Jb8dc0397FfCLHI+rAmSc3nYer6Un8Jl/G/uBJ3Is+18OfF/XDMqxrMHp7ebI3zDH8heB0ReSSRddLvbir9Eb8H3zcghfk3cf8JV/+Z3At7keMwJ4Bt9QiAygQY5lfwNmneX5DgDN/Nf/DLyTa3nOWroB2JBjWSrQx399GL4dd87HrgGuOcPztsX3wXkvcALfTj4+dw7gj8D7OR4XBWwHrs3x7zUgx/KbT2c830y66BKsC2DADGA6YDnurwScBIrluK8H8KX/+j3AllzbOuM+7gzPrX1nCF10hPfidHbOlXHO1XTODXLOHcf3CewH59yRHOttBqr5r/fDd+Rktf9r0lsv8LnHAbf5j7DcBixyzm32L6sJTPV/9XgQX8Fk4fsD8DPONxbqBqAMMAB41sx+lceqVf2vJefrism13a25lle9kEwiBcU5txz4APh9rkU1gTan35P+92VPfEd0K+B7b+d8P+e8jpk9bmarzOyQ/7GlgfL5jPUlUNx8w5Nq4TtSPDVHrsdy5arBj7WU+/XNcc7d4ZyrgG8o1NXA03ms+pP6dc5l+19TtRzrnK1+851JJIieABoDdzt/R+hXE4gFduZ4z47A923OaT+pafK3j/sf7TtDixregrcDKGdmJXPcdym+Iyk459Y553rgK7r/AyZZjlnkObg87vtxoXMr8RXFTfx0OAP4CucmfzN++lLU+cYan22bGc65icBS/OOR83htNXO9rkx8kwROq5Fr+Y6LySRSQJ4B7ufnzd3Xud6T8c65gfiOlmYC1XOs/7/3tn+87u/wDUUo65wrg+8osvlXOVf9ZuE7KtvDf/kgx4fkrcBzuXIVd86NP9eLdM7Nxzdc4pz1a2bmf005a/Bs9XtBmUQCxcyuxffhrptz7mCuxVvxHeEtn+M9W8o51zjHOrnrND/7uJ/RvjM0qOEtYM65rUAa8LyZFfVP7OgHvANgZr3MrIL/6MrpAs3OY1O78Y31OZtx+GagXw1MzHH/cOA5M6vpf84KZpaU1wb8A/dvMbOS/sHxN+H7tDw3j9XHA78xs9r+sYKnxylm5ljnj2ZW3MwaA32B06dpyXcmkYLmnFuP7734UI67PwDqm1lvM4v1X1qZ2eX+hnQKvoksxc2sIdAnx2NL4tth7QVizHcapFI5lu8GapnZ2f7GjsM3rKInP/3AOgoY4D/6a2ZW4nSN5t6AmXUws/vtx8luDYFO+MYX5vY+cIuZ/cLMYvGd2eEkvr9XpyWbWXX/pJin+bF+851JJBjMrAq++TKPOOe+y73c+SaufQq8Ymal/Pu3OmZ2zVk2m5993Onn174zxKjhDYwe+Mbl7MD3NeUzzrnP/Ms6AivM7Ci+CWzd/UMhcnsN6Ga+2dL/OsPzjMc3kP0L59y+XI+dAXxqZkfw7fzanGEbh/HNKN+CrwF/ERjonJuVx7pvAGPxjUP+Ht94wQdzrfM1sB7fmKWXnXOfXkAmkUD4K/C/b1P8R1RvxDdZbQewC9+3LqcnYg3GN0xhF773/Xh8DSLAJ8DHwFp837Sc4KdfSZ7+ALrfzBblFcb9OPm0Kr6zvJy+fwG+o9Ep+MYFr8c33jAvB/E1uMv8f1M+xvc358U8nm8N0Av4N7AP+DW+UyueyrHaOHxNwkZgA75xy+ebSSQY7sf3tf5r9vNz8Q73r9MHiANW4nvfTsI3BvZM8rOPO037zhBjPx3yInJh/OMQvwdi8/o0LBLqzOz/gMrOuaD+WlqwmNkmfJNsPjvXuiJSMLTvDB4d4RURyYP5ztHb1P8Vfmt8Q5OmnutxIiJS+JzXL4uIiESQkviGMVTFNyb3FXynPhIRkRCjIQ0iIiIiEtY0pEFEREREwpoaXhEREREJawEZw1u+fHlXq1atQGxaJCQtXLhwn/+XsAod1avIT6leRUJHfus1IA1vrVq1WLBgQSA2LRKSzGzzudfyhupV5KdUryKhI7/1qiENIiIiIhLW1PCKiIiISFjLV8NrZr8xsxVmttzMxptZ0UAHE5ELY2YdzWyNma03s997nUcknJlZDTP70sxW+veTD+exTkMzm21mJ83s8VzLVK8iQXDOhtfMqgEPAQnOuSZANL7fnheRQsbMooEhwE1AI6CHmTXyNpVIWMsEHnPONQLaAsl51NwP+PajL+e8U/UqEjz5HdIQAxQzsxigOLAjcJFEQsO+oycZMHYhOw4e9zpKTq2B9c65jc65U8AEIOlCN+ac49kPVvLx8l0FFlAknDjndjrnFvmvHwFWAdVyrbPHOTcfyMj18AKtVxE5s3M2vM657fg+lW4BdgKHnHOf5l7PzPqb2QIzW7B3796CTypSiKzccZiklFS+WruHtbuPeB0np2rA1hy3t5Fr5wv5r9fjGVks3HyAAe8sJOWLdeiXGUXOzMxqAS2Aufl8SIHWq4icWX6GNJTF94mzNr7flC9hZr1yr+ecG+mcS3DOJVSoUChPXyhSID5ZsYtuw9PIynZMfKA91zao6HWk85bfei0eF8OE/m3p0qIaL3+6locmLOZERlYQk4qEBjOLByYDjzjnDhfktrV/Fbl4+RnScAPwvXNur3MuA5gCtA9sLJHCxznHkC/X88DYhdSvVJIZgxO5onppr2Plth2okeN2df99F6xobDSv3tGMJzo25IOlO7hjxGx2Hz5xUSFFwomZxeJrdt91zk05j4cWeL2KSN7y0/BuAdqaWXEzM+AX+MYoiUSMExlZPDxhMS99sobOzasyoX9bKpYqlCcrmQ/UM7PaZhaHb4LpjIvdqJkx8No6jOydwIY9R+mUMoslWw9edFiRUOffL44GVjnnXj3PhwekXkXk5/IzhncuMAlYBCzzP2ZkgHOJFBq7D5/gzhGz+c/SHfyuYwP+cWdzisZGex0rT865TGAw8Am+D6bvO+dWFNT2f9moEpMHtSc2Ooo7RsxmxhLNX5WIlwj0Bq43s8X+y81mNsDMBgCYWWUz2wY8CvzBzLaZWalA16uI/ChfPy3snHsGeCbAWUQKnaXbDnL/2ws4ciKTEb1acmPjyl5HOifn3IfAh4HafsPKpZienMjAdxbx0PjvWLf7CL+5oT5RURaopxQptJxzs4Czvvmdc7vwDVfIa1lA61VEfPRLayJnMGPJDm4fPpuYqCgmD2wfEs1usFwSX4R37mvDnQk1+PcX6xn47kKOncz0OpaIiEie1PCK5JKd7Xjl0zU8NP47mlYvzYzBiVxepZTXsQqduJgoXuh6BX+6tREzV+6m2/DZbDuQ7nUsERGRn1HDK5JD+qlMBr27iH9/sZ47E2rw7n1tuSS+iNexCi0z494OtXmzb2u2HUin85BUFm7+wetYIiIiP6GGV8Rv+8HjdB02m09X7uJPtzbiha5XEBejEsmPa+pXYOqgROKLxNBj5FwmLth67geJiIgEifbmIsDCzT+QlDKLbQfSebNva+7tUBvf2YYkv+pWjGdaciKtapflt5OW8vcPV5GVrV9mExER76nhlYg3aeE2eoycS3yRGKYOSuSa+volowtVpngcY/q25u52NRn5zUbue2s+R05keB1LREQinBpeiVhZ2Y6/f7iKxycuoVXtskxLTqRuxXivY4W82Ogo/pLUhL91bsK36/bRZWgam/cf8zqWiIhEMDW8EpGOnMjgvrfmM/KbjfRpV5MxfVtTpnic17HCSq+2NXm7X2v2HT1J0pBU0jbs8zqSiIhEKDW8EnE27z/GbUPT+GbdPp7t3IS/JjUhNlqlEAjt65RnenIi5eOL0Gf0PN6du9nrSCIiEoG0l5eIMnvDfpKGpLLnyEnG3tua3m1reh0p7NW8pARTBrXnqnrleXrqcp6ZvpzMrGyvY4mISARRwysRY9zcLfQePZfy8UWYnpxI+7rlvY4UMUoVjeX1u1vR/+rLeGv2Zu55cz6H0jWZTUREgkMNr4S9zKxsnpm+nKemLuOqeuWZMqg9tcqX8DpWxImOMp66+XJe6taUed//QOehqazfc9TrWCIiEgHU8EpYO5SewT1vzuet2Zu5/6ravH53K0oVjfU6VkS7PaEG4+5vw5ETGXQZmspXa/Z4HUlERMKcGl4JW+v3HKXz0FTmfr+fF7s15elbGhEdpR+TKAwSapVjWnIi1csW594x8xk963uc049UiIhIYKjhlbD09dq9dBmayuHjGYy/vy13JNTwOpLkUr1scSYNaMcvG1Xi2Q9W8uSUZZzK1GQ2EREpeGp4Jaw453hj1vf0fXMe1coUY/rgRBJqlfM6lpxBiSIxDOvZkgevr8uE+Vvp9fpc9h896XUsEREJM2p4JWycyszmySnL+OsHK7nh8kpMHtie6mWLex1LziEqynjsxga81r05S7YdJGlIKqt3HfY6loiIhBE1vBIW9h89Sa/Rc5kwfyuDr6vL8F4tKVEkxutYch6Smlfj/QfacSozm65D05i5crfXkUREJEyo4ZWQt3rXYZKGpLJk60Fe696cx3/VgChNTgtJzWqUYcbgDtSpGE//sQsY+tV6TWYTEZGLpoZXQtrMlbvpOjSNU5nZvP9AO5KaV/M6klykyqWL8v4D7bi1aVVe/HgNv3lvMScysryOJSIiIUzf+UpIcs4x/OuNvPjJaq6oVpqRvROoXLqo17GkgBSNjeZf3ZvToFI8L3+6lk370xnZuyUVS+n/sYiInD8d4ZWQcyIji0ffX8L/fbyaW66ownv926nZDUNmxuDr6zG8V0vW7DpC0pBUlm8/5HUsEREJQWp4JaTsOXKC7iPnMPW77Tz2y/r8u0cLisVFex1LAqhjk8pMGtgOA7oNT+O/S3d6HUlEREKMGl4JGcu3HyIpJZU1u44wvNeVPPiLephpclokaFy1NNMHd6Bx1dIkj1vEP2auJTtbk9lERCR/1PBKSPjv0p10G56GAZMGtqNjkypeR5Igq1CyCOPub0O3ltV57fN1DB6/iPRTmV7HEhGREKBJa1KoZWc7/vXFOv752Tpa1izL8F4tqVCyiNexxCNFYqJ5qVtTGlQqyd8/WsXm/emM6pNA1TLFvI4mIiKFmI7wSqF1/FQWD47/jn9+to6uV1Zn3P1t1OwKZsb9V1/GG3e3YvP+dDqlpLJoywGvY4mISCGmhlcKpZ2HjnP7iDQ+XL6Tp25uyMu3N6VIjCanyY+ua1iRqYPaUzwumu4j5zBl0TavI4mISCGVr4bXzMqY2SQzW21mq8ysXaCDSeRatOUAnVJS2bQvndF3J9D/6jqanJYPZvaSv0aXmtlUMyvjdaZAq1epJNOTE7ny0jI8+v4SXvhoNVmazCZBZGY1zOxLM1tpZivM7OE81jEz+5eZrffX55U5lmWZ2WL/ZUZw04tEjvwe4X0N+Ng51xBoBqwKXCSJZFMWbaP7yDkUi41myqD2XN+wkteRQslMoIlzrimwFnjS4zxBUbZEHGP7teGuNpcy/OsNPDB2AUdPajKbBE0m8JhzrhHQFkg2s0a51rkJqOe/9AeG5Vh23DnX3H/pFJTEIhHonA2vmZUGrgZGAzjnTjnnDgY6mESW7GzHCx+t5tH3l9CiRhmmJSdSv1JJr2OFFOfcp865053eHKC6l3mCKTY6iuc6N+GvSY35cs1eug5NY+sP6V7HkgjgnNvpnFvkv34E3wGh3L9xngS87XzmAGXMTKeaEQmi/BzhrQ3sBd40s+/M7HUzK5F7JTPrb2YLzGzB3r17CzyohK+jJzPpP3YBw7/eQI/WlzK2XxvKlYjzOlaouxf46EwLw7FezYw+7WrxVt/W7Dx0nE4ps5i7cb/XsSSCmFktoAUwN9eiasDWHLe38WNTXNRfi3PMrPMZtht29SoSbPlpeGOAK4FhzrkWwDHg97lXcs6NdM4lOOcSKlSoUMAxJVxt/SGdrkPT+HLNXv6a1Ji/d2lCXIzmUp6JmX1mZsvzuCTlWOdpfF+zvnum7YRzvXaoV57pgztQtkQcPV+fy4R5W7yOJBHAzOKBycAjzrnD5/HQms65BOAu4J9mVif3CuFcryLBkp/z8G4DtjnnTn9inUQeDa/I+Zq7cT8D311EZlY2b/VtTYd65b2OVOg5524423Izuwe4FfiFcy5iZ2/VLl+CqYMSeXD8d/x+yjLW7D7C0zdfTky0PkxJwTOzWHzN7rvOuSl5rLIdqJHjdnX/fTjnTv93o5l9he8I8YaABhaJQOf86++c2wVsNbMG/rt+AawMaCoJe+/N30Kv0XMpUzyWacmJanYLgJl1BH4HdHLORfwA1tLFYnnj7gTuTazNm6mb6DtmPoeOZ3gdS8KM+U4hMxpY5Zx79QyrzQD6+M/W0BY45JzbaWZlzayIfzvlgUS0fxUJiPz+0tqDwLtmFgdsBPoGLpKEs8ysbJ77cBVvpm7iqnrlSbnrSkoXi/U6VrhIAYoAM/2ncZvjnBvgbSRvxURH8adfN6J+pXj+OH05XYam8nqfBC6rEO91NAkfiUBvYJmZLfbf9xRwKYBzbjjwIXAzsB5I58d96OXACDPLxncA6gXnnBpekQDIV8PrnFsMJAQ4i4S5Q8czGDxuEd+u20ffxFr6irmAOefqep2hsOre+lJqly/BwHcX0XlIKkN6XslV9TQWUi6ec24WcNYThfuHFyXncX8acEWAoolIDuo2JCg27j1Kl6GpzNm4nxduu4Jnft1Yza4EVZvLLmF6ciJVShfjnjfn81baJiJ4mLOISERRxyEBN2vdPjoPSeVgegbv9GtD99aXeh1JIlSNcsWZPKg91zWoyDMzVvD0tOVkZGV7HUtERAJMDa8EjHOOt2dv4u4351GldDGmJyfS5rJLvI4lES6+SAwje7dk4LV1GDd3C71Hz+XAsVNexxIRkQBSwysBkZGVzR+mLedP01dwXYMKTB7UnhrlinsdSwSAqCjjiY4N+cedzVi05SBJQ1JZu/uI17FERCRA1PBKgTtw7BS9R8/l3blbGHhtHUb0TiC+SH5PCCISPF1aVGdC/7akn8ritqFpfLF6t9eRREQkANTwSoFat/sISUNSWbTlIP+4sxlPdGxIdNRZJzCLeOrKS8syY3AitcoXp99bCxj5zQZNZhMRCTNqeKXAfLF6N12GppF+KosJ/dvSpUV1ryOJ5EvVMsWY+EB7bm5Shb9/uJrHJy7lZGaW17FERKSA6HtmuWjOOUZ9u5HnP1pNoyqlGNUngaplinkdS+S8FIuLJuWuFtT/vCT/+Gwtm/YfY3ivllQoWcTraCIicpF0hFcuysnMLB6fuJS/f7iam5pUZuKAdmp2JWSZGQ/fUI+hPa9kxY5DJKXMYsWOQ17HEhGRi6SGVy7Y3iMnuWvUXCYv2sYjN9QjpceVFI/TlwYS+m6+ogqTBrTHAd2Gzebj5Tu9jiQiIhdBDa9ckJU7DtN5SCordhxiyF1X8sgN9YnS5DQJI02qlWZ6ciINKpdkwDuL+Pfn6zSZTUQkRKnhlfP28fJddB2WRrZzTBrQnluaVvE6kkhAVCxVlAn923Jbi2q8MnMtD01YzIkMTWYTEQk1+v5Z8s05R8oX63ll5lqa1yjDyN4tqViqqNexRAKqaGw0r9zRjPqVS/J/H69m8/5jjOydQOXSeu+LiIQKHeGVfDmRkcVDExbzysy1dGlRjQn926rZlYhhZgy4pg6jeiewYc9ROqXMYsnWg17HEhGRfFLDK+e069AJ7hgxmw+W7uCJjg159Y5mFI2N9jqWSNDd0KgSUwYlEhcTxR0jZjN98XavI4mISD6o4ZWzWrL1IJ1SZrFhz1FG9k5g4LV1MNPkNIlcDSqXZHpyIs1qlOHhCYt55dM1ZGdrMpuISGGmhlfOaPri7dwxYjZxMVFMHtSeXzaq5HUkkULhkvgivNOvDd1b1eDfX6xn4LsLOXYy0+tYIiJyBmp45Weysx2vfLqGhycsplmNMkxPTqRh5VJexxIpVOJionj+tit45teNmLlyN12HpbHtQLrXsUREJA9qeOUnjp3MZOC7C/n3F+vp3qoG7/RrwyXx+mlVkbyYGX0TazOmb2u2HzxOUkoqCzb94HUsERHJRQ2v/M+2A+l0HZbGzJW7+dOtjXj+tiuIi9FbRORcrq5fgWnJiZQqFkuPUXOYuGCr15FERCQHdTMCwIJNP5CUksr2g8d5s29r7u1QW5PTRM5DnQrxTBuUSJval/DbSUt57r8rydJkNhGRQkENrzBxwVZ6jJpDyaIxTB2UyDX1K3gdSSQklS4ey5i+rbinfS1Gffs99701n8MnMryOJSIS8dTwRrCsbMdz/13JbyctpU3tS5iWnEjdivFexxIJaTHRUfy5U2Oe69KEb9ft47ahaWzef8zrWCIiEU0Nb4Q6fCKD+96az6hvv+ee9rUY07cVZYrHeR1LJGz0bFOTsf3asO/oSZKGpJK2YZ/XkUREIpYa3gi0ef8xbhuaxrfr9vFclyb8uVNjYqL1VhApaO3qXMKM5A5UiC9Cn9HzeGfOZq8jiYhEJHU5ESZtwz6ShqSy7+hJxvZrQ882Nb2OJBLWLr2kOFMGtefq+hX4w7Tl/Gn6cjKysr2OJSISUdTwRpB35mymz+h5VIgvwvTkRNrVucTrSCIRoWTRWEb1SeCBqy/j7dmbuefNeRxMP+V1LBGRiKGGNwJkZGXzp+nL+cO05VxdvwJTBrWn5iUlvI4lElGio4wnb76cl29vxvzvD9B5SCrr9xz1OpaISETId8NrZtFm9p2ZfRDIQFKwDqaf4p435/H27M08cPVljOqTQMmisV7HkgAys8fMzJlZea+zyM91a1md8f3bcvRkJl2GpvLVmj1eR5KLYGY1zOxLM1tpZivM7OE81jEz+5eZrTezpWZ2ZY5ld5vZOv/l7uCmF4kc53OE92FgVaCCSMFbv+conYekMv/7A7x8ezOevPlyoqP0YxLhzMxqADcCW7zOImfWsmZZpg/uQI2yxbl3zHxGz/oe5/QjFSEqE3jMOdcIaAskm1mjXOvcBNTzX/oDwwDMrBzwDNAGaA08Y2ZlgxVcJJLE5GclM6sO3AI8Bzwa0ERSIL5eu5fB4xZRJCaK8f3b0LJmOa8jSXD8A/gdMN3rIHJ21coUY9LAdjz63hKe/WAla3cd4dnOTfRz3iHGObcT2Om/fsTMVgHVgJU5VksC3na+TzVzzKyMmVUBrgVmOud+ADCzmUBHYPyF5vnLf1awcsfhC324SKHSqGopnvl14wLZVn7/sv4T3070jFOLzay/mS0wswV79+4tkHBy/pxzjJ71PX3fnEf1ssWZPriDmt0IYWZJwHbn3JJ8rKt6LQSKx8UwtOeVPHR9Xd5bsJVer89l/9GTXseSC2RmtYAWwNxci6oBW3Pc3ua/70z3596u6lXkIp3zCK+Z3Qrscc4tNLNrz7Sec24kMBIgISFB38154FRmNn+ctpz3FmzlV40r8eodzSlRJF8H8SVEmNlnQOU8Fj0NPIVvOMM5qV4Lj6go49EbG1CvUkken7iETimpjL4ngYbvV4vsAAAYmklEQVSVS3kdTc6DmcUDk4FHnHMFeoj1fOq1oI6GiYSb/BzhTQQ6mdkmYAJwvZm9E9BUct72Hz1Jr9fn8t6CrTx0fV2G9WypZjcMOeducM41yX0BNgK1gSX+Wq0OLDKzvJpjKYR+3awqEwe0IzM7m65D05i5crfXkSSfzCwWX7P7rnNuSh6rbAdq5Lhd3X/fme4XkQJ2zobXOfekc666c64W0B34wjnXK+DJJN9W7zpMp5RUlmw7yL96tODRGxsQpclpEcU5t8w5V9E5V8tfq9uAK51zuzyOJuehafUyzBjcgboV4+k/dgFDv1qvyWyFnJkZMBpY5Zx79QyrzQD6+M/W0BY45B/7+wlwo5mV9U9Wu9F/n4gUMB0CDHEzV+7mkQnfEV80hokD2tG0ehmvI4nIRahUqijvPdCOJyYv5cWP17B21xFe6NqUorHRXkeTvCUCvYFlZrbYf99TwKUAzrnhwIfAzcB6IB3o61/2g5k9C8z3P+6vpyewiUjBOq+G1zn3FfBVQJLIeXHOMezrDbz0yRqaVivNyD4JVCpV1OtYUkj4j/JKiCoaG80/72xO/UoleemTNWzan87I3i2pqBovdJxzs4CzfqXmPztD8hmWvQG8EYBoIpKDzn8Tgk5kZPGb9xbz4sdr+HXTqrz3QDs1uyJhxsxIvq4uI3q3ZO3uI3RKSWXZtkNexxIRCUlqeEPMnsMn6D5yDtMW7+C3v2rAa92b66tOkTD2q8aVmTSgPdFRxu0j0vjv0p1eRxIRCTlqeEPIsm2H6JSSytrdRxjeqyXJ19XFN19CRMJZo6qlmD44kSZVS5M8bhH/mLmW7GxNZhMRyS81vCHiv0t3cvuINKKjjEkD2tOxic42JRJJyscX4d3723B7y+q89vk6Bo9fRPqpTK9jiYiEBJ2loZDLzna89vk6Xvt8HQk1yzK8d0vKxxfxOpaIeKBITDQvdmtKg8ol+fuHq9i8P51RfRKoWqaY19FERAo1HeEtxNJPZTJ4/CJe+3wdt7eszrv3t1GzKxLhzIz7rrqM0fe0Ysv+dDqlpLJoywGvY4mIFGpqeAupHQePc/vw2Xy8fBd/uOVyXuzWlCIxmpwmIj7XNajI1OT2lCgSTfeRc5iyaJvXkURECi01vIXQoi0H6JSSypb96Yy+uxX3XXWZJqeJyM/UrViSaYMSSahZlkffX8ILH60mS5PZRER+Rg1vITNl0Ta6j5xDiSLRTBnUnusaVvQ6kogUYmVLxPHWva3p1fZShn+9gQfGLuDoSU1mExHJSQ1vIZGV7Xjho9U8+v4SWl5almmDEqlXqaTXsUQkBMRGR/G3zlfwbFJjvlyzl65D09j6Q7rXsURECg01vIXA0ZOZPDB2AcO/3kDPNpfydr/WlC0R53UsEQkxvdvV4u17W7Pr8Ak6pcxi7sb9XkcSESkU1PB6bOsP6XQdmsaXa/bybFJjnutyBbHR+t8iIhcmsW55piUnUrZEHD1fn8v4eVu8jiQi4jl1Vh6au3E/nVJmsfPQcd7q25re7Wp5HUlEwkDt8iWYOiiR9nXL8+SUZfx5xgoys7K9jiUi4hk1vB6ZMG8LPV+fS9kScUwf3IEO9cp7HUlEwkjpYrG8cXcC/TrUZkzaJvqOmc+h9AyvY4mIeEINb5BlZmXzl/+s4PdTltG+bnmmDkqkdvkSXscSkTAUEx3FH29txItdmzJn4366DE1l496jXscSEQk6NbxBdCg9g75j5vNm6ib6dajNG3cnULpYrNexRCTM3dGqBuPub8vB4xl0HpLKt+v2eh1JRCSo1PAGyca9R+kyNJU5G/fzf12v4I+3NiJGk9NEJEha1SrH9OREqpYpxj1vzuettE04px+pEJHIoI4rCL5dt5fOQ1I5eDyDd+9ry52tLvU6kohEoBrlijNpYHuua1CRZ2as4KmpyzmVqclsIhL+1PAGkHOOt9I2cc+b86lSuhjTkxNpXbuc17FEJILFF4lhZO+WDLq2DuPnbaH36Ln8cOyU17FERAJKDW+AZGRl8/S05TwzYwXXNajI5EHtqVGuuNexRESIijJ+17Eh/7yzOd9tPUjSkFms3X3E61giIgGjhjcADhw7Re/Rcxk3dwsDr63DyN4tiS8S43UsEZGf6NyiGu/1b8uJjGxuG5rG56t2ex1JRCQg1PAWsLW7j5A0JJVFWw7yjzub8UTHhkRFmdexRETy1OLSsswYnEit8sW57+0FjPh6gyaziUjYUcNbgD5ftZvbhqZxPCOL9/q3pUuL6l5HEhE5pyqlizHxgfbcfEUVnv9oNY9NXMKJjCyvY4mIFBh9z14AnHOM/GYjL3y8msZVSzGqTwJVShfzOpaISL4Vi4smpUcLGlQqyasz17Jp3zGG925JxZJFvY4mInLRdIT3Ip3MzOLxiUt5/qPV3NykChMfaK9mV0RCkpnx0C/qMbTnlazceZjOKaks337I61giIhdNDe9F2HvkJD1GzmHyom385ob6pNzVgmJx0V7HEhG5KDdfUYVJA9rjgNuHz+ajZTu9jiQiclHU8F6gFTsOkZQyi5U7DzO055U8fEM9zDQ5TUTCQ5NqpZk+OJGGVUoy8N1F/OvzdZrMJiIhSw3vBfh4+U66DZuNAyYN8E30EBEJNxVLFmX8/W25rUU1Xp25lgfHf8fxU5rMJiKh55wNr5nVMLMvzWylma0ws4eDEawwcs7xr8/XMeCdRTSoXJLpyYk0qVba61gi/2NmD5rZan+tvuh1Hgl9RWOjeeWOZvz+pob8d9lO7hgxm12HTngdq9AwszfMbI+ZLT/D8rJmNtXMlprZPDNrkmPZJjNbZmaLzWxB8FKLRJ78HOHNBB5zzjUC2gLJZtYosLEKn+Onsnhw/He8OnMtXVpUY0L/tlQspdnLUniY2XVAEtDMOdcYeNnjSBImzIwB19RhVO8ENu49SqeUWSzeetDrWIXFGKDjWZY/BSx2zjUF+gCv5Vp+nXOuuXMuIUD5RIR8NLzOuZ3OuUX+60eAVUC1QAcrTHYdOsGdI2fz32U7eaJjQ169oxlFYzU5TQqdgcALzrmTAM65PR7nkTBzQ6NKTBmUSFxMFHeMmM30xdu9juQ559w3wA9nWaUR8IV/3dVALTOrFIxsIvKj8xrDa2a1gBbA3DyW9TezBWa2YO/evQWTrhBYvPUgnVJmsWHPUUb2TmDgtXU0OU0Kq/rAVWY218y+NrNWZ1oxXOtVAq9B5ZLMGNyB5jXK8PCExbz0yWqyszWZ7SyWALcBmFlroCZw+leJHPCpmS00s/5n2oDqVeTi5bvhNbN4YDLwiHPucO7lzrmRzrkE51xChQoVCjKjZ6Yv3s6dI2YTFxPFlEGJ/LKRPpSLt8zsMzNbnsclCd8PyZTDN/Tot8D7doZPZ+FYrxI85UrE8U6/NnRvVYMhX27ggXcWcuxkptexCqsXgDJmthh4EPgOOD3zr4Nz7krgJnzDBa/OawOqV5GLl69fWjOzWHzN7rvOuSmBjeS97GzHqzPXkvLlelrXLsewnldySXwRr2OJ4Jy74UzLzGwgMMX5zh01z8yygfKADglJgYuLieL5266gQeWSPPvBSroOS+P1uxOoXra419EKFf8Bor4A/g+g3wMb/cu2+/+7x8ymAq2BbzyKKhLW8nOWBgNGA6ucc68GPpK3jp3MZMA7C0n5cj3dW9XgnX5t1OxKqJgGXAdgZvWBOGCfp4kkrJkZfRNrM6Zva7YfPE5SSirzN51tOGvkMbMyZhbnv3kf8I1z7rCZlTCzkv51SgA3Anme6UFELl5+hjQkAr2B6/2nTllsZjcHOJcnth1Ip+uwND5btZs/3dqI52+7grgYnapYQsYbwGX+0yNNAO52+qUACYKr61dgWnIipYrFcteoObw/f6vXkYLGzMYDs4EGZrbNzPqZ2QAzG+Bf5XJguZmtwTd04fSpPSsBs8xsCTAP+K9z7uNg5xeJFOcc0uCcmwWE/Syt+Zt+YMDYhZzKyubNvq25pr7GSUlocc6dAnp5nUMiU50K8UwblEjyuEX8bvJS1u4+wpM3X050VHjvPpxzPc6xfDa+CaW5798INAtULhH5KR2+BN5fsJW7Rs2hVLFYpiUnqtkVEbkApYvHMqZvK+5pX4vXZ33PvWPmc/hEhtexREQiu+HNynb87YOV/G7SUtrUvoRpgxKpUyHe61giIiErJjqKP3dqzN+7XEHq+n10GZLKpn3HvI4lIhEuYhvewycy6PfWfF6f9T33tK/FmL6tKF081utYIiJh4a42lzK2Xxv2HztF0pBU0tZr/qSIeCciG95N+45x29A0Zq3bx3NdmvDnTo2JiY7IfwoRkYBpV+cSZiR3oGLJIvR+Yx5j52z2OpKIRKiI6/LS1u+j89BU9h09ydh+bejZpqbXkUREwtallxRnyqD2XFO/An+ctpw/TltORla217FEJMJEVMM7ds5mer8xjwrxRZiR3IF2dS7xOpKISNgrWTSWUX0SeODqyxg7ZzN3vzGPg+mnvI4lIhEkIhrejKzs/x1ZuKZ+BaYMas+ll+jXgEREgiU6ynjy5st55fZmLNh0gKQhqazfc8TrWCISIcK+4T2Yfoq7/WPHHrj6Mkb1SaBkUU1OExHxQteW1Rnfvy3HTmbSZUgaX67Z43UkEYkAYd3wrt9zhM5DUlmw6QAv394sIk6CLiJS2LWsWZbpgztQo1xx+o2Zz+vfbkQ/CigigRS2De9Xa/bQZUgaR09mMr5/G7q1rO51JBER8atWphiTBrbjxkaV+dt/V/HE5KWczMzyOpaIhKmwa3idc4z2/8JPjXLFmT64Ay1rlvM6loiI5FI8LoahPa/koevr8v6CbfR6fS77jp70OpaIhKGwanhPZWbz+8nLePaDldzYqDKTBrajWpliXscSEZEziIoyHr2xAf/u0YKl2w6RlJLKqp2HvY4lImEmbBrefUdP0vP1Oby3YCsPXV+XoT2vpHhcjNexREQkH37drCoTB7QjMzubrsPS+HTFLq8jiUgYCYuGd9XOwySlpLJ02yH+3aMFj97YgChNThMRCSlNq5dhxuAO1KsYT/+xCxny5XpNZhORAhHyDe+nK3bRdVgamdnZTBzQjl83q+p1JBERuUCVShXlvQfakdS8Ki99soZH3lvMiQxNZhORixOy3/k75xj61QZe/nQNTauVZmSfBCqVKup1LBERuUhFY6P5553NqV+pJC99soZN+9MZ1bslFfU3XkQuUEge4T2RkcVv3lvMS5+soVOzqrz3QDs1uyIiYcTMSL6uLiN6t2Td7iN0Skll2bZDXscSkRAVcg3vnsMnuHPkHKYt3sFvf9WAf97ZnKKx0V7HEhGRAPhV48pMHtie6Cjj9hFpfLB0h9eRRCQEhVTDu2zbITqlpLJu9xFG9G5J8nV1MdPkNBGRcHZ5lVJMH5zIFdVKM3jcd7w6cy3Z2ZrMJiL5FzIN7wdLd3D7iDSio4zJA9vzq8aVvY4kIiJBUj6+CO/c14bbW1bnX5+vI3ncItJPZXodS0RCRKFveLOzHa/OXMvgcd9xRbXSTB+cyOVVSnkdS0REgqxITDQvdmvKH265nE9W7KLbsNlsP3jc61giEgIKdcObfiqT5HGL+Nfn67i9ZXXeua8N5eOLeB1LREQ8Ymbcd9VljL6nFVt/SCcpJZWFmw94HUtECrlC2/DuOHicbsNm88mKXfzhlst5sVtTisRocpqIiMB1DSoyNbk9JYpE02PkHCYv3OZ1JBEpxAplw7tw8wE6paSy9Yd0Rt/TivuuukyT00RE5CfqVizJtEGJJNQqy2MTl/D8R6vI0mQ2EclDoWt4pyzaRo+RcyhRJJqpye25rkFFryOJiEghVbZEHG/d25rebWsy4uuN9H97AUdOZHgdS0QKmULT8GZlO174aDWPvr+EhFplmZ6cSN2KJb2OJSIihVxsdBTPdm7Cs0mN+WrtXroOS2PL/nSvY4lIIVIoGt4jJzLo//YChn+9gd5ta/LWva0pUzzO61giIhJCererxdh7W7P78EmShsxizsb9XkcSkULC84Z3y/50ug5L46u1e3k2qTHPdm5CbLTnsUREJAS1r1ue6cmJlCsRR6/X5zJu7havI4lIIZCvztLMOprZGjNbb2a/L6gnn7NxP0lDZrH78EnG3tua3u1qFdSmRSKOmTU3szlmttjMFphZa68ziXihVvkSTE1OJLFueZ6auow/z1hBZlZ2QJ7LzN4wsz1mtvwMy8ua2VQzW2pm88ysSY5lAdm3isjPnbPhNbNoYAhwE9AI6GFmjS72icfP20Kv1+dSrkQc05MTaV+3/MVuUiTSvQj8xTnXHPiT/7ZIRCpVNJY37mnF/VfVZkzaJvqOmc+h9IBMZhsDdDzL8qeAxc65pkAf4DUI3L5VRPKWnyO8rYH1zrmNzrlTwAQg6UKfMDMrmz/PWMGTU5bRoV55piYnUqt8iQvdnIj8yAGnf4awNLDDwywinouOMp6+pREvdmvKnI376TI0lY17jxboczjnvgF+OMsqjYAv/OuuBmqZWSUKeN8qImeXn4a3GrA1x+1t/vt+wsz6+79GXbB3794zbixtw37GpG3i/qtqM/ruVpQqGnveoUUkT48AL5nZVuBl4MkzrZjfehUJB3ck1GDc/W05dDyD5z9aHeynXwLcBuAfZlQTqE4+963+x6leRS5STEFtyDk3EhgJkJCQcMYzf19dvwL/GdyBK6qXLqinFokYZvYZUDmPRU8DvwB+45ybbGZ3AKOBG/LaTn7rVSRctKpVjmnJiZQoUmC7vfx6AXjNzBYDy4DvgKzz2YDqVeTi5afytwM1ctyu7r/vgqnZFbkwzrk8G1gAM3sbeNh/cyLwelBCiYSIGuWKB/05nXOHgb4A5vvJ0O+BjUAxCnjfKiJnlp8hDfOBemZW28zigO7AjMDGEpELsAO4xn/9emCdh1lEBDCzMv59J8B9wDf+Jlj7VpEgOucRXudcppkNBj4BooE3nHMrAp5MRM7X/fi+Oo0BTgD9Pc4jEvbMbDxwLVDezLYBzwCxAM654cDlwFtm5oAVQD//Mu1bRYIoX4OZnHMfAh8GOIuIXATn3Cygpdc5RCKJc67HOZbPBuqfYZn2rSJBop80ExEREZGwpoZXRERERMKaOVfwZzgxs73A5nOsVh7YV+BPHjihlheUOVjyk7mmc65CMMKcrzCtVwi9zKGWF8I3s+o1+JQ58EItLxRgvQak4c0PM1vgnEvw5MkvQKjlBWUOllDMfL5C8TWGWuZQywvKXFiF4mtU5sALtbxQsJk1pEFEREREwpoaXhEREREJa142vCM9fO4LEWp5QZmDJRQzn69QfI2hljnU8oIyF1ah+BqVOfBCLS8UYGbPxvCKiIiIiASDhjSIiIiISFgLesNrZh3NbI2ZrTez3wf7+c+Xmb1hZnvMbLnXWfLLzGqY2ZdmttLMVpjZw15nOhszK2pm88xsiT/vX7zOlF9mFm1m35nZB15nCYRQq1cIvZoNtXqF0K3ZcK9XCL2aVb0GXqjWKxRszQa14TWzaGAIcBPQCOhhZo2CmeECjAE6eh3iPGUCjznnGgFtgeRC/u98ErjeOdcMaA50NLO2HmfKr4eBVV6HCIQQrVcIvZoNtXqF0K3ZsK1XCNmaHYPqNdBCtV6hAGs22Ed4WwPrnXMbnXOngAlAUpAznBfn3DfAD17nOB/OuZ3OuUX+60fwvVmqeZvqzJzPUf/NWP+l0A8uN7PqwC3A615nCZCQq1cIvZoNtXqF0KzZCKhXCMGaVb0GXijWKxR8zQa74a0GbM1xexuF/I0S6sysFtACmOttkrPzf22xGNgDzHTOFeq8fv8Efgdkex0kQFSvQRYq9QohWbPhXq+gmg0q1WvAFWjNatJaGDOzeGAy8Ihz7rDXec7GOZflnGsOVAdam1kTrzOdjZndCuxxzi30OouEh1CqVwitmlW9SkFTvQZWIGo22A3vdqBGjtvV/fdJATOzWHzF+K5zborXefLLOXcQ+JLCP6YrEehkZpvwfW14vZm9422kAqd6DZJQrVcImZqNhHoF1WxQqF6DosBrNtgN73ygnpnVNrM4oDswI8gZwp6ZGTAaWOWce9XrPOdiZhXMrIz/ejHgl8Bqb1OdnXPuSedcdedcLXzv4y+cc708jlXQVK9BEGr1CqFXsxFSr6CaDTjVa3AEomaD2vA65zKBwcAn+AZ6v++cWxHMDOfLzMYDs4EGZrbNzPp5nSkfEoHe+D4RLfZfbvY61FlUAb40s6X4/mDPdM6F7WmDQkUo1iuEZM2GWr2CarZQCsWaVb0GheoV/dKaiIiIiIQ5TVoTERERkbCmhldEREREwpoaXhEREREJa2p4RURERCSsqeEVERERkbCmhldEREREwpoaXhEREREJa2p4RURERCSs/T/+XBl4dF7cQgAAAABJRU5ErkJggg==\n",
"text/plain": [
"