{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Loss Functions\n",
    "\n",
    "This python script illustrates the different loss functions for regression and classification.\n",
    "\n",
    "We start by loading the ncessary libraries and resetting the computational graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.framework import ops\n",
    "ops.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Create a Graph Session"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Numerical Predictions\n",
    "\n",
    "---------------------------------\n",
    "\n",
    "To start with our investigation of loss functions, we begin by looking at numerical loss functions.  To do so, we must create a sequence of predictions around a target.  For this exercise, we consider the target to be zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Various Predicted X-values\n",
    "x_vals = tf.linspace(-1., 1., 500)\n",
    "\n",
    "# Create our target of zero\n",
    "target = tf.constant(0.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### L2 Loss\n",
    "\n",
    "The L2 loss is one of the most common regression loss functions.  Here we show how to create it in TensorFlow and we evaluate it for plotting later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# L2 loss\n",
    "# L = (pred - actual)^2\n",
    "l2_y_vals = tf.square(target - x_vals)\n",
    "l2_y_out = sess.run(l2_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### L1 Loss\n",
    "\n",
    "An alternative loss function to consider is the L1 loss. This is very similar to L2 except that we take the `absolute value` of the difference instead of squaring it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# L1 loss\n",
    "# L = abs(pred - actual)\n",
    "l1_y_vals = tf.abs(target - x_vals)\n",
    "l1_y_out = sess.run(l1_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Pseudo-Huber Loss\n",
    "\n",
    "The psuedo-huber loss function is a smooth approximation to the L1 loss as the (predicted - target) values get larger.  When the predicted values are close to the target, the pseudo-huber loss behaves similar to the L2 loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# L = delta^2 * (sqrt(1 + ((pred - actual)/delta)^2) - 1)\n",
    "\n",
    "# Pseudo-Huber with delta = 0.25\n",
    "delta1 = tf.constant(0.25)\n",
    "phuber1_y_vals = tf.multiply(tf.square(delta1), tf.sqrt(1. + tf.square((target - x_vals)/delta1)) - 1.)\n",
    "phuber1_y_out = sess.run(phuber1_y_vals)\n",
    "\n",
    "# Pseudo-Huber with delta = 5\n",
    "delta2 = tf.constant(5.)\n",
    "phuber2_y_vals = tf.multiply(tf.square(delta2), tf.sqrt(1. + tf.square((target - x_vals)/delta2)) - 1.)\n",
    "phuber2_y_out = sess.run(phuber2_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Plot the Regression Losses\n",
    "\n",
    "Here we use Matplotlib to plot the L1, L2, and Pseudo-Huber Losses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXd8zdf/x5/nhkgiRI2qHSSE2vGlaqtZihYtFaOl1KoqSkvVaFWrtJTae6+2trYIVdRWIxHSUHuvhETGPb8/TuQXFTLu+Hwi5/l45JHcez+fc143uXl/zud93kNIKdFoNBpNxsJitACNRqPROB9t/DUajSYDoo2/RqPRZEC08ddoNJoMiDb+Go1GkwHRxl+j0WgyIHYx/kKIxkKIECFEqBBi8FOOayWEkEKIyvaYV6PRaDRpw2bjL4RwASYDTYDSQDshROkkjssG9AX22DqnRqPRaGzDHiv/KkColDJMShkNLAVaJHHcKOBrIMoOc2o0Go3GBjLZYYwCwLlEj88DVRMfIISoBBSSUq4XQgx80kBCiG5ANwB3d3f/QoUKpVmU1WrFYkn9te3ag2vcirlF8azFcREuaZ7f3roeEhNj4fTprOTJ84Dnnos2XJfH2bOIuDjuFS1qNy2JsfX35Sgymi5LbCwiOpo4D480nW9PXefOeRAXJ/D2vmfzWI76fUXERuAiXHB3cU/T+bboOnny5HUpZZ5kD5RS2vQFtAZmJnrcAZiU6LEF2AZ4xz/eBlROblx/f39pC4GBgWk67/yd83LfhX02zf000qorMZUqSfnSS7ZrSUyada1bJ+WkSVJGR9tVz0Ps8ftyBFpX6rCXrkuXpBRCys8/t8twz+TvC9gvU2C77XHJuwAkXqIXjH/uIdmAMsA2IcQZ4CVgjVk3fQtkL0Dl/KaUlkDr1vDXX3D2rNFKgKZNoVcvyJzZaCUaRxAeDm+8AQcPGq0EgJ9+AimhVSujlSTNLyd+YfLeycRaY42Wkiz2MP77AF8hRFEhhCvQFljz8EUp5R0pZW4ppbeU0hv4C2gupdxvh7kdgpSSIVuGMHzbcKOlJEmbNur7ihXG6kjg3j2YNs0kVyONXVm4EH7+GaLt52K0hWXLoFQpKFPGaCVJ8/OJn5lxcIZDXMb2xmbjL6WMBXoDvwLBwHIp5XEhxEghRHNbxzcCIQQXwi9wMfyi0VKSxMcHKleGpUuNVhLPjRvQsydMmWK0Eo09kRImTQJ/f6haNfnjHcz587BjB7RtC0IYrSZp5raYy9ZOWxFmFZgIe2z4IqXcAGz4z3PDnnBsHXvM6Whmt5iNRZhvQ+8hbdvCgAEQGqouBoZSuDC0aAEzZsDnn4Obm8GCNHZh+3YICoLZs01hbVesUNejt94yWsnjWKWV+zH38XT1JKd7TqPlpAjzWjeDeWj4z905x52oOwareZw331Tfly0zVkcCvXqpO4Dly41WorEXkyZBzpxqpWECli6FihWhZEmjlTzO2pC1FPm+CMeuHjNaSorRxv8pXAy/iM8PPkzYM8FoKY9RqBDUqGEi41+vHvj5KYOheTaoVg0+/RTc0xauaE9On4a9e01zHXqMIjmK8IbfG/jl9jNaSorRxv8p5M+WnwmNJ9C5QmejpSRJ27Zw9CgcP260EpRboFcvsFjgjvnulDRpoH9/9WUCHi5yHt7xmo0KL1RgRvMZZLLYxZPuFLTxT4b3K79PYa/CRstIktatla01zeq/Z08Vg+rlZbQSjS08eKB8LCaJ8AElp1o18PY2WsmjSCmZsm8K1+5dM1pKqtHGPwWE3gyl+9ru3Iu2PaPQnuTNC3Xrqn8MU7RifpiReOOGXv2nZ1atgnbtYNs2o5UAEBwMf/9tTpdP0LUgem3oxbLjZlmBpRxt/FPAlYgrLDq6iAOXDhgt5THatoVTp+DQIaOVxHPlitqQ0GGf6ZdJk8DXF+rXN1oJoO5shfj//BYz8eLzL3K853G6VOxitJRUo41/CqheuDoXPrpArSK1jJbyGG+8AZkymSjmP29eeOklZfzj4oxWo0ktBw/C7t3KhWeC2kVSqs92nTqQL5/Rah7FKq0AlMpTCvfMxm+Kpxbj/7rpBC835ce+GXnTYCWPkjMnNGqkVkdWq9Fq4undW2X7rltntBJNapk8GTw8oHNno5UAyt0TEmJOl89rS15jWGCS6UzpAm38U8FnWz+j7JSyRMZEGi3lEdq2Vbb2r7+MVhJP8+ZQsKAyJJr0g9WqwscCAiBHDqPVAGrVnymTusM1EzFxMRTKXog8HskXzzQr6ScuyQQ08mmEe2Z3JGbYXf1/mjdXSbVLl8LLLxutBvXf+v778NlncO6c2gPQmB+LBfbsgfv3jVYC/L/Lp0EDyJ3baDWPktklM1ObTTVahk3olX8qqFG4Bp/W/BSPzGmrae4osmdXxTWXLzeRm/3991WYhjb86QOrFSIi1M5q1qxGqwHUdejff83n8jly5Qgh10OMlmEz2vinga2nt7L8uLnKGLRrpwJttmwxWkk8uXKZMw9fkzSbNkGBAiYKG1MFRd3cVNkoM/Hx7x/TYEGDdFG2+Wlo458Gvt75NWP+HPOwWY0paNZMuWkXLDBaSSLu3VOF12fMMFqJJjkmTVIr/hdfNFoJADExyuXTooX5cgbnvz6fJa2WpKts3qTQxj8NzG4+m11ddpmqbGuWLCr1/aef1N27KfDwgDNnYMIEk2ShaZIkNFSt/Lt1A1dXo9UASs6NG2rv2Ww8n/V5qheubrQMm9HGPw0UyF4At0xuWKWV6DjzpMAHBKi9ul9+MVpJPA/r/Rw/Dn/8YbQazZOYMgVcXJTxNwkLF6pN3kaNjFby//z2z280X9KcKxFXjJZiF7TxTyP3ou/hP92fb3Z+Y7SUBKpXV7VPTOX6addOJSPoap/mJCpK1et/4w3In99oNYCqDLJ6tdroNVN30Kv3rnLu7jmec3/OaCl2QRv/NJLVNSu1CtcyVQlXi0Wt/jdvhkuXjFYTj7s7vPuuagV44ULyx2uci5ubihL4/HOjlSSwapWqLWc2l09AuQAOdjuIq4s5XGO2oo2/DUxoMoHWpVsbLeMROnRQUXuLFxutJBE9eqi68FmyGK1EkxSVKkHp0karSGDBAlVaqEoVo5UorNLKrnO7AEy1z2cr2vjbSKw1lrmH53L9/nWjpQBQooT6pzGV66dYMRg50nyZOhmd3buhUye4fNloJQmcPauKiQYEmKJzJACrT6ym+uzqbArdZLQUu6KNv438c/MfuqzpwrzD84yWkkBAgKqJcvSo0UoSYbWqnejAQKOVaB4ycaJyrmfLZrSSBB7esZrJ5dPEtwmzms+iQbEGRkuxK9r420jJ3CXZ23UvH1X7yGgpCbRtqyosLFxotJL/MHCgKvmgMZ5Ll2DlSnjnHdNk9Eqp7lhfflndLJoFt0xuvFvxXVwsLkZLsSva+NsB//z+CCFMk/GXJw80bgyLFpmo3IPFosoE79wJhw8brUYzYwbExqq/iUk4fBiCgtS+lRmIio2i6eKm/Hn2T6OlOARt/O3ErnO7KDqhKMHXgo2WAqh/oAsXTNOMSdG5s4r+0dU+jSUmBqZNU0H0vr5Gq0lgwQIV2mmWpi2nb50m5HqIaRZ19kYbfztRIlcJyjxfxjQflNdeUwXfTLXx+9xzypm7aBHcumW0moxLZKTyDX5kHldlTIxyU772mioLZQZK5SnFid4nqONdx2gpDkEbfzuR2yM3G9tvpGzeskZLAdQCu00b5dY1TbkHUBm/+fLBP/8YrSTjkj07jBsHDRsarSSB9evh2jW1BWEGDl06RKw1Nt3X73ka2vjbmbsP7jL70GxTFH175x1VW23lSqOVJKJ8edV0uHJlo5VkTEJDVVKXCT6fiZkzB154Qe1VGc2N+zeoNbcW/X/tb7QUh6KNv51ZeGQhXdZ04e8rfxsthZdfVnH/s2cbreQ/WCyqrMD580YryXiMG6dKwN40TzvSK1fUyr9DBxWlZjQ53XMyv+V8elXpZbQUh2IX4y+EaCyECBFChAohBifx+vtCiKNCiMNCiD+FEOZJJ7QzXSp2Yd97+6jwQgWjpSCEWv3v2AEnTxqtJhFSqky07t2NVpKxuHNHbQK1bWsexzrK1x8XZx6XjxCC10u9TolcJYyW4lBsNv5CCBdgMtAEKA20S8K4L5ZSlpVSVgC+AcbbOq9ZyZIpC5XzK5dGnNX4OMuOHdVCe+5co5UkQgh4/XXYuFH7/p3JvHnKD9i7t9FKEpBS3Zm+9BKUKmW0Gvhw04fMPmS2W2XHYI+VfxUgVEoZJqWMBpYCj/TekVLeTfQwK5isCa4D+HHfj1SaXsnwks/580OTJur/PtYcgUiK7t1VGeEpU4xWkjGwWlWIbdWq4O9vtJoE9u1Tsf1mWPVHx0Vz+PJh/rmZMRYk9jD+BYBziR6fj3/uEYQQvYQQ/6BW/h/YYV5T453Dm7LPl+Ve9D2jpfDuu3DxIvz2m9FKEpE/v1r9z5plmobhzzRnz8Ldu6Za9YPa6HV3h7feMloJuLq4EtgpkOF1hhstxSkIW6NShBCtgcZSyq7xjzsAVaWUSX7KhBBvA42klJ2SeK0b0A0gb968/kuXLk2zroiICDw9PdN8vqMwQldMjKBNm2qUL3+HESOOm0aX199/U/HDDzk+bBjX6tY1ja6UkB51iZgYEAJpwK5qUroePLDQqtXLVKt2gyFDjEmOfKjr9L3T5MmSB89M5vib2vL5qlu37gEpZfLhdFJKm76AasCviR5/AnzylOMtwJ3kxvX395e2EBgYaNP59uLMrTNyydElCY+N0tWvn5SZM0t57VrSrxuiy2qVcu9e9f0JmOXv+F/Sla7wcCljYpyuJTFJ6Vq8WEqQcvNm5+t5SGBgoIyzxsnSk0vLGrNrGCfkP9jy+QL2yxTYbnu4ffYBvkKIokIIV6AtsCbxAUKIxDnkTYFTdpg3XfDFH1/w/rr3CX8QbqiOd95RWZSLFhkq41GEgP/9zzy1e59VvvhClXGIjDRaySPMmQNFisATbvqchkVYmN9yPqPrjTZWiJOx2fhLKWOB3sCvQDCwXEp5XAgxUgjRPP6w3kKI40KIw8BHwGMun2eVL+p9wdEeR8mWxdiyuWXLqryqWbNMl98DgwfrsE9HERUFM2dCxYrKuW4STp+G339XixKLCbKN/PP7U7NITaNlOBW7/NqllBuklCWklMWllF/GPzdMSrkm/ue+UsoXpZQVpJR1pZRJO56fQfJ65qWQVyEAHsQ+MFTLu++qGv/79xsq43GiotQy8Mqz0RjbVCxbBjdumG6jd8YMZfS7dDFWx8J/FzLo90FYpdVYIQZggmtuxqDfpn68Mv8VQ8s+vP02eHiogo6momdP5ZOaMcNoJc8ekyerAHqjfSuJiIlRsf1Nm0LBgsZquRZ9jXN3z2ERGc8UZrx3bBCV8lWirnddYqVxwfZeXiq5c8kSlexpGkqUUEXGpk41WTJCOufAARVI36uXqfZV1q5VN3nduhmtBPr59mPB62Yqfes8tPF3Eh3Kd2BUvVFktmQ2VMf776uwelNt/IIyUBcuqLaCGvtQsaLKojZLd5R4pk1TK/4mTYzTEHQtiLBbYQDPXIeulKKNv5MJuhvEsmPLDJu/cmVlE6ZNM9nGb9Om8PHHUKaM0UqeHSwWVSYze3ajlSRw+rRKNuzaVSV4G0XfTX2pN68ecdL4EixGoY2/k1n470I+3/a5YXV/hFCBNUeOwJ49hkhIGhcX+PprKFnSaCXPBj/+CIMGqbIOJmLmTHNs9M5tMZd5LefhIjLmqh+08Xc6/Uv0Z997+wy91Xz7bfD0NOHGL8Bff5msCl06JC5OXUj37zdHHGU8ZtjofRhwUSB7AWp71zZGhEkwzycjg5ArSy6yZcmGlJLr968boiFbNmjfXkUB3r5tiIQnM22aCks01Y50OmPdOlXLp5e56tGvXQuXLxu70TsscBhvrXzLNO1WjUQbf4Nos6INry561bD44u7dVcKnqXr8gjJY9+7B/PlGK0m/TJqkltbNmyd/rBOZPl3JMrJbV7Ys2fDK4vVMt2dMKdr4G0S7Mu14v/L7hs1fsaKqrGC6jd/KlVXZ4cmTTSYsfeB+9ixs3qzCuszQFiueixfd+O035es3UtbH1T9m+mvTjRNgIszz6chgtCrdymgJdO+uoi527TJayX/o1Ut1odmyxVQGLF1gsUBAgPrDmog1a/JjscB77xkz/57ze4ixxlCjcA1jBJgQvfI3mBXHV/DNzm8MmbttW5X4NXmyIdM/mTZtwM8PLl0yWkm6I7JgQeXLy5vXaCkJ3L8PGzbk4403oMBjnT6cw6g/RtHh5w7ExMUYI8CEaONvML/98xurglcZsgGVNauq97NiBdy44er0+Z+Im5tq72Sy5CTTs307WU3YFnPxYggPz0yfPsZpWNZ6Gavbriazi7FJlmZCG3+D+a7xd+x6d5dhG1A9e6rIwLVr8xsy/xMRAqxW3C9cMFpJ+kBK6NULv6+/NlrJI0ip9p+LFYughgEel4joCKzSSlbXrJTLW875AkyMNv4G4+nqiYvFhciYSI5dPeb0+X18VJr9mjX5iTa23fDj9OpFxV69VNVPzdPZvh2OH+fC668breQRdu6Ev/+G11+/YEh5oe7rulN3Xt0MWbUzObTxNwntf2pP44WNiYp1vqHr0wdu3XJlxQqnT/10WrXC9c4dzCfMhEyeDDlzcrVePaOVPMKkSZAjB7zyijHlul/1eZXX/V7PkFU7k0OHUpiEITWHcOfBHdwyuTl97oYNoWDB+/zwgwft2zt9+ifzyivcL1QIj0mTtP//aZw/Dz//DB99hDVLFqPVJHDxIqxaBR98AO7uxqy825cz0wfaXOjLoUnwz+9PvaJq1ebsmv8WC7RseYE9e1QFYNMgBBdatoS9e00mzGQcPqx273v0MFrJI0yfrvaTevZ0/tw/7vuRRUcWGdo/w+xo428ypu2fRoMFDZzuo2zc+DKenvDDD06dNlkuN2qkChGZLhXZRDRrpuomFC1qtJIEoqNVAuGrr0Lx4s6dW0rJiqAVrApehTBRHwOzod0+JsMjswdZXbMS/iAcLzcvp82bNWscnTqpZlrffgvPP++0qZ9KXNassGOHLvX8JG7dUk51E/XnBVi5Ul2PjOgeKYRgc4fNRERHOH/ydIRe+ZuMgHIBrG672qmG/yG9e6sV23SzZb9XqKAzfZ9Es2bQurXRKh5BShg/XlXnbtjQuXMfv3qce9H3cLG4GPI/lJ7Qxt9kPLxNvRJxhekHnGuF/fzUP+vkyfDA2F7zj7NwIbz8snIiaxSHDqnaHEYE0D+FHTtUB8l+/ZxbUTomLoami5vSZkUb502ajtHG36RM2T+FPhv7cPbOWafO27+/ul1fvNip0yaPuzvs3g3r1xutxDxMngweHtC5s9FKHmHcOMiVS5VnciaZXTKz4PUFjKgzwrkTp1O08TcpH1f/mCPvH6GwV2GnztugAZQtq/6BTRUo0aKFqgc8aZLRSszBzZvqCt2+PTz3nNFqEjh5UtXt79nTudsQDzvj1SxSk/8V+J/zJk7HaONvUjwye1Ayt2ppeO7OOafNKwQMGADHj8Ovvzpt2uTJlEmVIf39dwgJMVqN8SxYoBoymKxhy/ffQ+bMzpUVHRdN1ZlVmbbfjK3pzIs2/iZn2bFlFJ9YnAMXDzhtzrZtIX9+tfo3Fe+9pyzLjz8arcR4undXHbvKlzdaSQI3bqgOnAEBzi0qej/mPkWfK0rB7Ab1hkyn6BAKk9PIpxEDXh6QcBfgDFxdVVbm4MEqf6hCBadN/XTy5oWxY00kyEDc3FQzXBMxdaq6GfnoI+fOm8MtByva6BIgqUWv/E1ODrccjH5lNJ6unk6dt3t3lVs1frxTp02evn2hdsZuvE2PHjBrltEqHuHBA7Ud06gRvPiic+a0Siufbf2Mi+EXnTPhM4Y2/umE07dO88r8Vwi+FuyU+XLkUC33lixRpWNMRVgYfPWVyXakncQ//6jU2XPO2wdKCYsXqyix/v2dN+ffl//m293fsjlss/MmfYawi/EXQjQWQoQIIUKFEIOTeP0jIUSQEOKIEGKLEKKIPebNSGR1zcq5O+c4c/uM0+b88EOwWs1X8oFt2+DTT1VAeUZjyhRwcYFu3YxWkoDVqrLCy5aF+vWdN2/FfBUJ6R1Ch3K66F9asNn4CyFcgMlAE6A00E4IUfo/hx0CKkspywErAWP6FqZjns/6PMG9gmni28Rpc3p7q+TRqVPhzh2nTZs8bduq8MaMFvZ5/75y97zxhtqRNwlr1qjGa598glNq9lullf0X9wNQ2Kuwrt+TRuyx8q8ChEopw6SU0cBSoEXiA6SUgVLK+/EP/wL0tnwacLG4ACoCKPRmqFPmHDwY7t41WZ9fDw/lk/r5Z8hInb6WLIHbt00V3ikljB6tire1cVJi7YK/F/C/Gf9j59mdzpnwGUXYWvJUCNEaaCyl7Br/uANQVUqZZEknIcQk4LKU8oskXusGdAPImzev/9KlS9Os6+7du2TPnj3N5zuKiIgIPD3Tvnl7J+YO7fe0p37e+nzo+6FTdA0eXJYTJ7KxZMlfTq/L/iRdbhcvUjUggH87dODMO+84VdPTdDmSHIcP8/yWLZz86KMnLrGdrevAgecYMKA8H30UwmuvXXricfbUFRUXxabLm2iRv4XNq34j/o7JYbVauX//fpp11a1b94CUsnKyB0opbfoCWgMzEz3uAEx6wrEBqJV/luTG9ff3l2ll/fr10tfXV/7xxx9pHsNRBAYG2jzGkctHZExcjO1iEvE0XX/+KSVI+d13dp0yRTz199WqlZQDBjhNS2Ls8Xd0BM7WVbeulPnySRkV9fTj7KErzhrn1M+9s4mJiZFTpkyRfn5+cs2aNWkeB9gvU2C77eH2uQAUSvS4YPxzjyCEqA8MAZpLKR1eNuz27dvUqlWLN998kzNnzjh6OqdSNm9ZMlkycS/6Hv/e/tfh81WvrqIrx441WcG3FSuUqIzAihWmc3Ht3g2BgSoj3BkNxCbumcjLs17mdtRtx0/mZLZu3UqlSpXo0aMHefPmJSLC8eWo7WH89wG+QoiiQghXoC2wJvEBQoiKwDSU4b9qhzmfyquvvsr8+fMZPnw469atw8/Pj6FDhzrlF+ospJQ0XtSY15e97pTGL0OGqLZ88+Y5fKqU8/CW/8QJY3U4msuXVQ0fk13ovvoKcuZ0XuBRYa/CvPj8i3hlebZKNVutVgYMGEB4eDgrV64kMDCQfPnyOXxem42/lDIW6A38CgQDy6WUx4UQI4UQzeMPGwt4AiuEEIeFEGueMJzdcHNz4/PPPyckJIRWrVrx5ZdfUqJECebNm4fVakw/UXsihOCzWp8xruE4pzSnrl8f/vc/+PpriI11+HQpZ+FCKFVKpSI/q8yYATExxvRDfAJHjqgCbn37qmRAZ/BGqTeY02LOMxHdc/fuXT777DNu3ryJxWLhp59+Ijg4mFatWjnt/dnFakgpN0gpS0gpi0spv4x/bpiUck38z/WllHmllBXiv5o/fUT7UahQIRYtWsSuXbsoVKgQnTt35qWXXnomXEENizekbtG6AMRaHWuRhVCr/7AwsGEf3v40barKR5oqHMmOxMSopK6GDaFECaPVJDBmjDL6zujUNXbnWBYdWeT4iZzIv//+y5gxY9i4cSMA3t7euLm5OVVDhsnwrVatGrt372b+/PlkypSJvPGVp2JNtYxNG3MOzaHStErci77n0Hlee011U/zqK5XYYwqee065RBYtUi0NnzVWr1a+fiP6IT6BEydg2TJVZSJnTsfOFWeNY92pdfz6j5lKzKaNP/74g+HDhwNQtmxZwsLCaN++vWF6MozxB7BYLHTo0IGdO3fi7u5OVFQUFSpUYLrp+hamjqLPFaXoc0WJjot26DwWi1r9BwXBqlUOnSp19OqlKorNmWO0EvsTFAQ+PqoTukkYOVLVlRswwPFzuVhc2NJxC1ObTXX8ZA7izJkztGnThtq1azNnzhzuxGdMFipUKJkzHUuGMv4PeehTu3fvHmXKlKFYsWIAPHjw4GFIarqijncdVrddzXPujm/q0aYNlC4Nn39uoo6KFSqokKRFz5ZrAIBhw+DoUVXSwQQEBSm3X58+8Pzzjp1r9qHZRERHkMmSCY/MHo6dzAFEREQwZMgQ/Pz82LBhAyNGjCA4OBgvL3NsWGdI4/+QXLlysXTpUurHFyQZMmQIL7/8Mnv27DFYWdq4fv86HX7u4NDWjy4uMGIEBAebzPc/Z46q+fMsceOG+u5kX/DTGDECsmZ1/Kr/6JWjdF3TlRkHZjh2IgdgtVqZO3cuvr6+jB49mtatWxMSEsKwYcPw8DDPRSxDG///UqFCBc6cOcNLL71Ehw4duGCyuOrkuB11m42nNrLvwj6HzvPGG6qHyPDhaj/SFPj6QrZsRquwH3fuqOJK335rtJIEjh5V6QYffAC5czt2rrJ5y/JX17/oU7WPYyeyM7t27aJq1aq88847FC5cmN27d7Nw4UIKFjRfRRtt/BMREBDAyZMn+eSTT1ixYgUlSpRg1KhRREZGGi0tRfjk9OF039O0Kt3KofNYLDBqFISGwvz5Dp0qdezdC+XKqZCk9M78+RARAXXqGK0kgREjVISPI8s2x1pjCboWBECVAlXIZElf/aZWrlzJxYsXWbBgAbt37+all14yWtIT0cb/P2TLlo3Ro0cTHBxMkyZNGDZsGCVLlmTp0qXpYj8gWxa1+t12Zhu7z+122DzNmkGVKmrzzzRZvwUKKKf0lClGK7ENKVXoatWqUDn5Ei3O4PBhtcn/4YeOjfAZu3MsFadVdFrhQluJjIxk+PDhBAYGAjBixAhCQkIICAjAYjG3eTW3OgMpWrQoK1euZNu2beTMmZN27dpRs2ZNoqKijJaWLLHWWLqv687n2z532BxCqNX/2bMmaipVoIDySc2apcofp1e2bFFN6k1UvXPYMPDygn79HDvPe/7vMb7heHxy+jh2IjshhGD+/Pls2bIFUItHsxWKexLa+CdD7dq1OXDgADNmzKBixYoJiRj37jk2pt4WMlkysbbdWn566yeHztO3UvWLAAAgAElEQVSgAdSoAV9+qSItTUGvXire31S70alk6lTIk8d5NZKT4c8/VTbvxx+rtApHcPfBXaSU5PbITa8q5rnoJcXevXt58803iYyMxM3NjUOHDvHFF48VKTY92vinABcXF7p27coP8S2tjh07RoECBfjtt98MVvZkSuQqgaerJ7HWWLaf2e6QOYSAL75QNX9+/NEhU6SeWrVUJtqkSem3zeO0abBypSmifKSEQYMgXz5VysERxFpjaba4GQE/BzhmAjtx4cIFOnbsSNWqVdmxYwchISEApgndTC3a+KcBT09PXnvtNfz9/QG4ceOGafcDxvw5hnrz6xFyPcQh49eurSoPfPmlSRJshVBiBgxIv8Y/Vy51ETMBa9fCrl0qsitrVsfM4SJcaFWqFU19mzpmAhu5f/8+X3zxBSVLlmTZsmUMHjyYkydPUqFCBaOl2YQ2/mnA29ubBQsWkCtXLqxWK40aNaJu3bocOnTIaGmP0bdqX5a2WkrJ3CUdNsc336gGU1995bApUkfz5vD22yosKT0RFaUyeU2SrxAXp1ozligB777rmDms0ooQgr4v9eXtsm87ZpI0YrVaWbRoESVLluSzzz6jUaNGBAcH89VXX5HtGQgrTmf/HeZDSsl7773H8ePH8ff3p2vXrly+fNloWQlky5KNNi8q3/H5u+cdUgCufHno2BEmToR/Hd9eIGXcuqWqj125YrSSlLN8OWzcaJrU6fnzVfDU6NGQyQERl2G3wig3pZzD81LSwr59+6hWrRoBAQHkzZuX7du3s2rVqoRqAM8C2vjbiIuLC927d+fUqVP069eP+fPn4+Pjw5dffmmq/IBL4ZcoN6UcI7ePdMj4o0Ypj8vQoQ4ZPvVcvaqWrTPSUYbo5Mng5wf16hmthKgoVcKjShUVQOWQOWKjcM/sTk53B1eHSwNXrlzh/PnzzJs3j71791LLJG44e6KNv53IkSMH48aNIygoiEaNGjF06FBKlizJokWLTNE/IF+2fAyuMZhO5Ts5ZPxChVQM+MKFYArvV8mSKhxp6lSTNSB4Avv2qSS1Xr2e2J/XmXz3HZw7p/o3OEpO6Tyl2dt1L8VzFnfMBKlASsngwYMZNWoUAE2bNiU0NJSOHTuaPl4/rTyb78pAfHx8WLVqFdu3b+f5558nICCAFi1aGC0LgI+rf5zwj+aIVniDB6u9yoEDTbLX2ru3Koe8erXRSpJn8mSVPtuxo9FKuHRJ7Zm3bOmYBOMJf03gs62fJfj7jeRhoIYQgvPnzyeUdBFC4O7ubqQ0h6ONv4OoVasWe/fuZd68ebz9ttrIio2N5dKlSwYrg5HbR1JxWkVuRt6067heXvDZZypHaf16uw6dNpo2hSJF0kejl9q1VSZV9uxGK2HIEIiOdkxZISklwdeDOX7tuP0HTyW//vorXbt25ejRowDMmzePqVPTb+no1JK+CmekMywWCx0TreRmzJhB3759KV++PKVLlzZMV6PijQh/EE72LPY3ND16KE9Lv37K6+KMxt5PxMVFtT7ctk05sU0QN/9E3nnHaAUAHDgAc+eqSNniDvDGCCGY0nQKMdYYp7QfTYqgoCAGDBjAxo0byZ8/P7fiY5RdTFI221nolb8TadGiBe+88w6lSpUCVERBjAFlMasWrMrYhmPJZMnEg1j7FuZxdYUJE1TRt++/t+vQaWPgQNiwwbyGPy4OZs6E8HCjlSClSuTKk8f+G/eRcZF0+qUTF+5eQAiBq4urfSdIAdeuXaNXr16UK1eOXbt28e233zJnzpxncjM3JWjj70Ty589Pu3btEEJw9epV6tSpQ7ly5Vi/fr0hSWKXIy5TaXolZh+abddxGzZUofYPs38N5aFP+exZVSXTbKxfD++9BybIFl++HHbuVH83e3ufTt87zZqQNYa4ex48eMC3336Lr68v06ZNo0ePHoSGhtK/f39cXZ1/ETIL2vgbRJ48eViyZAlxcXE0a9aMhg0bJvgenUVuj9yUy1sO7xzedh97/HjlNx482O5Dp57QUChaVPkzzMakSVCwIBgcFHDvnqrdU768YxK6SmcvzZm+Z2hYvKH9B38K+/fvp3Tp0gwcOJDq1atz9OhRfvjhB3I7uiFBOkAbf4MQQtC8eXOOHTvGhAkTOHDgABUqVKBbt25OSxLLZMnEklZLqFdUxZXbswdw8eLKb7xggSoPYCg+Pqo08uTJJglDiickBH7/Hbp3d0wWVSp4WKH1hx/s2zFy9qHZLD++HAAvN+fVwImOVp/lggUL8sILL/Drr7+yfv36BJerRht/w3F1deWDDz4gNDSUvn37PtL+zZlJYluubqHclHJcvXfVbmN+8omqsty7twlC7Xv1ghMnYOtWg4Uk4scfIXNm5fYxkKAgGDcOOneGmjXtN65VWllwZAFzD891qltz4MCB1KtXDyklL7zwAjt37qRhQ+fecaQHtPE3CTlz5mT8+PEcP36cBg0aMGTIEHo5sZ57frf8lMxdkiwu9gvP8fRUyUKHDinvhqG8+abqPWiWsE8p4Z9/VNnmvHkNldGzp+qA+c039h3bIixsar+JZa2XOTye/+7duwnBE2XLlqVGjRqGBFOkJ7TxNxm+vr789NNPbNu2jcHxDvPTp0/z559/OnTeUtlLsbrtarzcvIizxmGV9slKbt1a1SobOlS5FQzDzQ26dlWRPzftm9+QJoSAdetgtn0321PLggWwfbvK5M2Txz5jXgy/SJ8NfYiKjSJLpiwJ3eUcQUxMDFOmTMHHx4dp06YB0LFjR8aMGZOhN3NTgjb+JqV27dqUKFECgDFjxtC4cWNu37Z/Vu5/iYmLoeWylgz8baBdxhPi/13tvXsb7HL/6CM4fdqxfQhTgpRw/br62cBEiJs31b7MSy9Bly72G3fr6a3MPzKff27+Y79B/4OUktWrV1O2bFl69uxJqVKlTN0v14xo458OGD9+POvXrydHjhxIKRk3bhxXr9rPN5+YTJZMFH+uuF3b6Hl7q+bfa9fCzz/bbdjUkyeP6koCxl6F/vhDbYZsd0yTnZQyeDDcuKFaHtuzfE1AuQBC+4Ty4vMv2m/QROzbt486derQsmVLAFavXs22bduobJJ+x+kFu/zJhRCNhRAhQohQIcRjwX1CiFpCiINCiFghRGt7zJmRyJo1K7Vr1wbg6NGjDBo0KKFy6H0796oVQvB94+/p8b8eAERE2yc2/sMPVRhhnz5w545dhkwbN26oqpmLFhmn4WEdnypVDJOwebMqeNq/P9ijJ0mcNY73173PoUuqql+erHbyISXizJkzvP3221SpUoUTJ07w448/cvToUZo3b254jaD0iM3GXwjhAkwGmgClgXZCiP/WLjgLdAYW2zpfRqdcuXIcO3aMevXqMXToUHx9fZk9ezZxDqgBH3QtiOITi/NzsO3L9UyZYPp0VTRs0CA7iEsrOXOqzDOjdqAvXICfflLB9AYVDgsPV9sfJUqoOzJ7cCniEhtObWDXOcfE9R45coSSJUvyyy+/MHToUEJDQ+nRoweZM2d2yHwZAXus/KsAoVLKMCllNLAUeCRjRUp5Rkp5BDC+tvEzgJ+fH7/88gt//PEHhQoVokuXLlSoUIGNGzfaNaTOO4c39YvVp2zesnYZr0oV5XafNs3AhFYhVNjnnj2wf7/z558+HaxWVQTJID75RG2+z55tv+tPwewFOdbzmF2brz948IC9e/cCKoJn6NChnDp1ilGjRj0TnbSMRthqLOLdOI2llF3jH3cAqkopeydx7FxgnZRy5RPG6gZ0A8ibN6//0qVL06wrIiICT0/PNJ/vKOytS0rJ9u3bmTFjBhcvXqRixYr06NEDX19fu+u6GX2TnK62bZY+eGChWzd/IiNdmD17P56eT08AcMTf0SUigpfbtOFqnTqEpPE2JE264uKo1rYtET4+HHVQz8vkdB0+7EW/fhVp1eo8vXuH2jzfvDPzEELQscjTS1Gn5fc1fvx4tmzZwtKlSx1m7J9FO1G3bt0DUsrkN0CklDZ9Aa2BmYkedwAmPeHYuUDrlIzr7+8vbSEwMNCm8x2Fo3Q9ePBATpw4UebKlUuOGzcu1ecnp+v73d/LHGNyyFM3TqVR4f+zZ4+UFouUnTvbrivN9OghZZYsUl67lqbT06wrNFTK48fTdm4KeJquiAgpixeXslgx9bOtWK1W2fHnjrLzL52l1WpNs67E7NixQ4aGhkoppQwJCZGbNm2yVeZTeRbtBLBfpsDG2iOn/AJQKNHjgvHPaZyIq6srffr0oWPHjrjFV7BcsmQJBw8eZNSoUQnPpZWWfi25GH6RojmK2qy1ShUVaTJ6NLRqBc2a2Txk6unTRzm9nV3t0xF1klNI//4QFqaSnLNmtW2smLgYMrtkZk6LOUgpbd5wDQ4O5tNPP+WXX36hS5cuzJw5kxIlSiSEO2vsjz18/vsAXyFEUSGEK9AWWGOHcTVpwMvLiyzxseNHjhxhx44dCcku0gYXX5EcRfi6wde4WFy4FXmLI1eO2KRz2DAoW1ZVNngY8u5USpVSIUjOuuU/fFgVbzt92jnz/YfVq9Vey4ABtnfnmrp/KtVmVeNW5C0swoKLJe3FgM6fP0/Xrl0pU6YMW7ZsYdSoUUyYMME2gZoUYbPxl1LGAr2BX4FgYLmU8rgQYqQQojmAEOJ/QojzQBtgmhDC+DY+GYCvvvqKP/74A4vFwvXr16lUqZJdegp3WdOFRgsbcT8m7WGmWbLA/Pkq0ejddw0Ku4+JgTlzVNy9o5k8WRVxy5HD8XP9h0uXVBJXxYqqXLOtFPYqTLHnipHVNe23D7du3WLQoEH4+vqyYMECPvjgA8LCwhg6dChZbb0t0aQIu8T5Syk3SClLSCmLSym/jH9umJRyTfzP+6SUBaWUWaWUuaSUjsn+0DzGw1X/w6SwgIAAKlasyIYNG9J8JzC+0XjmtZyHR2YPm7RVqABjx6rkrx9+sGmotPPZZ6phrSO5dUvlFbRvD88959i5/oPVCp06wf37sHixaraTVsJuhQHwqu+rLG+zPE0NWSIjI/nmm28oVqwYY8eO5c033yQkJITvvvtOl1l2MjrDN4NQunRpDhw4wOLFi4mIiKBp06bUrl2bnTt3pnos7xzeCXXZN57ayM6zqR/jIX36wGuvqYZbhw6leZi0kTmzKqf8229w8qTj5pkzByIjVYipk5kwQd1wfPcd+PmlfZxZB2dRenLphCSutHL9+nWGDRtG9erVOXz4MPPmzcPb29umMTVpQxv/DITFYqFdu3YEBwfz448/curUKWrUqEHz5s0JCwtL9XhWaeXTrZ/yyZZP0nwXIYSKN8+TB956y4Buhu+9py4CP/7omPGtVjV29er2SaVNBX/9pRLqWrSAbt1sG+v1Uq8zuMbgNOV8bNy4kXHjxiGlpFChQoSEhLBu3TrKlStnmyiNTWjjnwFxdXVNaGU3evRo/vjjD7p27cq4ceNSNY5FWPg14FdWvbkKIUSaLwC5cyuvyD//qPLCTvX/v/CCKj06d65qZ2VvoqOV38XJac3Xrqlq0QULqhuPtATjxFnjmHVwFlZpJad7TobXGU4mS8oDBB9+Hk6ePMmRI0e4GV9NtUiRIqkXo7E72vhnYLJmzconn3xCWFgYbdu2TagfdOXKFa5cuZKiMZ7P+jx5subBKq10XdOV6Qemp0lL7drw+eewcKEBlRd694ZixeD8efuP7eam9hVee83+Yz+BuDh4+211AVi1Ku3bDGtPrqXr2q6sP7k+VecdOnSIpk2bMmfOHAB69uzJ7NmzyZUrV9qEaByCNv4acubMSbdu3RKqIn766ae8+OKLqSoaFx0XzeV7l23qBDZ0qGr83q+fkwteVqsGBw5AyZL2HffMGVixQkUVOZFhw1Thth9/VBE+aaWlX0u2d97OayVTduEKDg6mTZs2VKpUid27dyfUm8qcOTMu9uwNqbEL2vhrHmPQoEF8//33eHioaJ6lS5cmeyFwy+TG6rarGVJzCACnb53mQeyDVM1rsajmIj4+ymXhtOYvQqiv27ftO+mkSWoJ7qDy20mxc2cuRo9WhdvS0oj97J2z1J9fnzO3zwBQq0itZM8JCwujU6dOlClThk2bNjFs2DDCwsJ4z+D2lJqno42/5jFKlChBQEAAAPv376ddu3YUL16ciRMnEhUV9cTzMlkyIYQgMiaSuvPq0umXTqmeO3t2+OUXiIqCN95QtYCcQlycyjobaJ8mNty/r3ayX39d1e53An//DV98URp//7SHzt6PuU/ozVAu3E0+Sf/8+fO8//77lCxZkuXLl/PRRx9x+vRpRowYQQ4D8hk0qUMbf81TqVy5Mtu3b6dkyZL07dsXHx8fpkyZQnR09BPPcc/szpj6Y/i4+sdpmtPPT/n+DxyAb74piY05aSnDxUWFG/30kyr5bCtLlqj4/t6P1Td0CJcuqW0FT89Y1qxJfdWKvy//DYBfbj9CeodQvXD1px5/8OBBfHx8mD17Nt27dycsLIyxY8fqWP10hDb+mmSpVasWgYGBbNmyhSJFitCzZ098fX2ZOXPmE5tkty3Tlkr5KgEwcc/EVG8aNm8OY8bA1q15GTLE5reQMnr0UHcA09O2aZ2AlMrlU6YM1KxpH21PITJShXPeuAGjRx8lf/7Unf/LiV+oMK0Cm0I3AZAlU9KtJW/dusUf8dnQ5cuXp3///pw8eZJJkyaR72GHNE26QRt/TYoQQlCvXj3+/PNPNm3axAsvvMB7772Hn58fy5cvf+J5MXExLDiygIVHF6Z6zo8/htdeu8iYMaoujcMpXhyaNFGTPeXOJlmuX1dun1690hZjmQri4lTi8P79KoPX1zf1ndde9X2V8Q3H80rRV556XPfu3WnVqhVRUVG4uLjw5Zdf6gStdIw2/ppUIYSgUaNG/PXXX6xduxYvLy9OxmfHWq3WxzqKZXbJzPbO25n52kwArt27xt0Hd1M4F/Tte4pXX1Xx/+vW2fe9JEmvXnD5sm31fvLkgeBg+3ZFTwIp4f33VV/k775Tq/+UcujSIVoubcn9mPu4urjSr1o/Mrs82hXrxo0bDB06NCEBcPjw4WzZssXmCrEac6CNvyZNCCFo1qwZBw4c4OOPlW9/1apVlC1blnPnzj1yrEdmD7K6ZkVKSdtVbak9tzZWmTJHvouLZNkyFbLYurUqR+xQGjeG48ehfv20nR8erlb9FovKHHYgn34KM2fCkCHQt2/qzr167yqHLh/i39v/Pvba9evX+fTTT/H29ubLL7/kt/i2a6VLl9ZZuc8Q2vhrbEIIkVA8Lnv27JQqVYr88U7nQ4cOPbIxLIRgeO3hfFrjUyxCffRSkhXs6QmbNqkQ0ObNYfduB7yRh1gsUDq+BXVaUo0nTIBChdRmrwMZO1btiXTvDqNGpeyc8AfhbDuzDYBGPo0I6R1CqTylEl6/du0agwcPxtvbmzFjxvDqq69y9OhR3n//fQe8A43RaOOvsRuNGjVi1apVuLi4cO/ePerXr0/x4sWZNGkSkZGRANQsUpM2L7YBYG3IWlosbcHtqNvJjp07typQli+fcssfOODQt6JcNn36pO6c2FiYOhX8/R1avfPbb9V+yFtvqUrRKd1W6P9bf5oubsr1+6qBglsm5b65dOkSH3/8Md7e3nzzzTc0b96cY8eOsWzZMsqUKeOot6ExGG38NQ7Bw8ODxYsX4+3tTZ8+fShatChjx44lPFHltqv3rnLt/rUEI5Qc+fLBli3g5QWvvOLgOwAhVFGc1KzgV6+GCxccGt75zTcqFeGtt1Q4bHKJs9Fx0Ql7LCPqjODXgF/J7fH/4ZiLFi2iaNGijBs3jtdff52goCAWL15M6Yd3P5pnFm38NQ7h4cbwjh072L59O+XKlUtYXY4cOZIbN27QpVIXdr67E7dMbkTFRjF482BuRt586riFC6u92Ny5oUED2LbNQW+gVy/lu587N+XnTJ4MRYpA06Z2lyOlans5aBC0basMf6ZkaqzFWeOoPbc2Xdaojed82fJRo3ANDh06RFBQEABVqlShU6dOhISEsHDhQvxsqfusSVdo469xOLVq1eK3335jz5491KhRg88//5xChQrRu3dv4mJVdNDOszsZt3sc+y7sS3a8IkVgxw71vUkTWJ+6FIKUUbEivPyyKpCTkiyzU6cgMFDlCti5jk1cnNrQHTJEhXUuWPB0wx9rjQXAxeJCh3IdaF+2fcJrUVFR1K9fn5EjRwLg6+vLtGnT8PHxsatmjfnRxl/jNKpUqcLq1as5duwY7dq148SJE2SOj4gpJooR9kEYjXwaAbD6xGpO33pyv9t8+VTxt9Kl1SbwlCkOENy7N4SGqmYvyeHjowrod+1qVwlRUWql/8MPqgH7/PlPN/xn7p2h1ORSCQ123qvwHg/+fsDbb7+N1WrFzc2NX375halTp9pVpyb9oY2/xum8+OKLzJo1KyGE8PLly/j5+bF46mIAHsQ+oMf6Hgz4fcBTx8mdW10AmjRReQD9+6tVst1o1UrVmU7JpqcQULUq2LFs8ZUryrW1ciWMG6c2ei1P+I99GDqb1y0v3jm8CQ8P5+uvv6ZYsWK0bduWvXv3cja+aF3NmjV17R2NNv4a47DEW7Ls2bMzbtw43njjDQAO7T9EF2sXhlcdDsCl8EusDFqZZFiop6faZ+3TB8aPV4lOdouydHWF4cNVR5SnMWOG6gj2IHVVTJ/Gvn0qaOjAAVi6FD766MnHfrf7O+rNq4dVWrl24Rq+u31pVaUVgwcPpkSJEqxZs4aTJ0/qbFzNI6S8LY9G4yA8PDzonShCZsOGDXwx6gu+HfYtderUoVCHQsz5Zw6n+pzCO4f3Y+e7uMDEiaog3IcfQqVKqox+fHsC29m0SWX9du78+GtSqmW5lxdkSbomTmqQUhUD7dVLubZ27Uq6+2NMXAwWYcHF4kJO95zIO5KGTRuyZeMWsmTJQvv27enbt69OytI8Eb3y15iOkSNHcvjwYTp27Mi2bduYETCDEn+WYPvq7URGRjLhrwnsvbD3sfN69lQbwVarapk7cWLK9mqTZdYs5VOKz1V4hC1bICTELuGdN2/Cm2+qbYMaNdTqPynDf+7OOfwm+zHvwDwAOpTrwPmJ5zlx5ASdO3fm7NmzzJo1Sxt+zVPRxl9jSsqXL8+0adNYsWIFEydMxHrWSufOnclXJB+fbPiESYH/3+sxsTuoalU4eFD5yvv2VfkAaehN/yi9einLvHTp469Nnqw2H9q0sWmKzZuhXDnVy2DMGPj1VzXsQyJjIjly5QgABbIXwOuOF/279ScqKgqLxcJvv/3GmTNn6NSpE88//7xNWjQZA238NabG09OTPn36EBQUxJYtW2jWsBlyoqTQ6UIA/PXvX1T4sQInrp9IOCdXLli7VtW9OXhQGdUJE2zopli7Nrz4oirTnHjf4exZWLNG+fvTWOzs8mUICFAXK09PFTA0aNCj0aJSSl6d9SrVJ1fnyPEjWISFCXUm0PPVnjyI32coXrw4mZIL/NdoEqGNvyZd8LCk9MKFC7ly9goD+6iOWxsDN3LkyBGuhqlWiUcuHyHsVhhCqAoNx46pkvoffgjlyyv3fRomV6v/gwdhz57/f97FRcX1p6H2zYMHyi1VsqTan/jsMzh0SG3yAhy/epzmc5szZOQQ/Pz82PbFNh4sfcCRQ2r1X7NmTb788ku8vLzS8IY0Gr3hq0mHZM+ePeHnd+u+S+bzmanurzpPtfyxJZdiLjG+8HhatmxJvgJ52LAhE2vXKrd9kybQsKEytjVqpGLSDh1UMsG1a///XIEC6m4gFcTEqFj9kSPVjUODBmqIEiWUwf/3wL/s3LiTxdsWc6bmGdYuWUudYnX4OOBjWrdurY29xm5o469J1xQpUoShQ4cmPO70XCdm/jSTnn/2pGfPnrh95EZF94qMrj+agwdfZvp0V77+Wt0N1KypCqQ1bpx8qQQ8PVWT3Pgqal5//62C7mvWTFFltZs3VamgyZPh9GmoUgWmz7BSrspV8uV4gZuRNykzuQz8CS7bXahZqyYfFPiANjvbUDC5UFONJg3Yxe0jhGgshAgRQoQKIQYn8XoWIcSy+Nf3CCG87TGvRvNfPu/3OWf/OMvRo0cZ9dUo8tzOw541e6hbty75iuRk5GUv2gzpy8SJcOaM6ntbpAh88gkEBSVTxVkI1eErKIji06ZBt25P1RITo5KD331XpQoMGCB5ruAxVq9Wvv2OO/Ph288HKSU53XPyrue7jG83nsuXLxO4NZB+ffppw69xGDav/IUQLsBkoAFwHtgnhFgjpQxKdFgX4JaU0kcI0Rb4GnjL1rk1mqQQQlCmTBnKlCnDUIZy9+5dAgMDWf77clbfW82ls+eYPA6qNj9EjYkvk/XwJ3zzzTDGjL/JC8WP0rx2VZo0cqNKFR7vh9uhAyxfTnZQTvtEq/64OFUNYscO+P33KDbs+Z0I1x1kPuNGgQLHuVxrIwezRlKp0jmEKEibIm24Kq4SExODq6srswbOcuavSZPBsYfbpwoQKqUMAxBCLAVaAImNfwtgePzPK4FJQgghU9LJQ6OxkezZs9OiRQtaxPc5tMYH/98Jv85zMdnp914eWiyAHpOnscb1U6ZPsTB9akEolg1Lldvk2lObPO75yVwkAul5itmZoUKsCx1Ds7EtoCZlb87h5nUfDtGX2PIzYWZW4Bo0AipD7FcCV1df/F0qUahIoYQ2iJPeTd1+gUZjT4St9lcI0RpoLKXsGv+4A1BVStk70THH4o85H//4n/hjrv9nrG5AN4C8efP6L00qrjqFRERE4OnpmebzHYXWlTqcqSv4cjCrD68m+7mc/Bt6l1OZQ7hd4SyW+TmICw+HqpHQBF78uieZI7tyuNosaDCZXDNX4J2vHveKjeCs10L8w6pTvHAuPAt5kid/HqoUrYJbFuf0vdV/x9TxLOqqW7fuASll8vntUmy6UlMAABgUSURBVEqbvoDWwMxEjzsAk/5zzDGgYKLH/wC5nzauv7+/tIXAwECbzncUWlfqMJOu+5H35aVrN+TVq9Fy48btMjIqWsbFxRkt6xHM9PtKjNaVOmzRBeyXKbDd9nD7XAAKJXpcMP65pI45L4TIBHgBN+wwt0bjNNzd3HF3cwfAzc2KW5bUN2iPiYnh/PnzREVF2VseAF5eXgQHBztkbFvQulJHSnS5ublRsGDBhLLoqcUexn8f4CuEKIoy8m2Bt/9zzBqgE7AbdaewNf4KpdFkKM6fP0+2bNnw9vZGpLT5bioIDw8nW7Zsdh/XVrSu1JGcLiklN27c4Pz58xQtWjRNc9gc6imljAV6A78CwcByKeVxIcRIIUTz+MNmAbmEEKHAR8Bj4aAaTUYgKiqKXLlyOcTwazIOQghy5cpl0x2kXZK8pJQbgA3/eW5Yop+jANsqX2k0zwja8Gvsga2fI13bR6PRaDIg2vhrNBkYb29vjh079shzVquVVq1aUbJkScqXL0+DBg34559/kjx/7ty5tG7d2hlSNXZGG3+NRvMYnTp1Ijg4mL///psWLVrQLZlSFpr0hzb+Go3mESwWC82bN0/osVytWjX+/fffVI0RFxfHgAEDEspsDBkyhLi4OACmT59OqVKlqFChAuXKlePEiRNYrVZ69uyJn58f5cuXp3r16nZ/X5pH0VU9NRqD+PBDOHzYvmPGxbnj4qLaP37/vX3GnDRpEs2bN0/+wERMnz6dw4cPc/DgQQAaNmzI9OnT6dGjBwMHDuTEiRPky5ePBw8eEBcXx99//01gYCBBQUFYLBZu3bplH/GaJ6JX/hqN5ol88803BAcH88UXX6TqvM2bN9O5c2dcXV1xdXWlffv2bN68GYB69erRqVMnfvjhBy5cuICHhwfFihUjJiaGLl26sGDBAke8Fc1/0Ct/jcYg7LUyT0x4eKTdkpZ++OEHFi9ezNatW/Hw8LDLmAA//fQT+/btY+vWrdStW5epU6fSpEkTjh8/zrZt29i8eTODBg3i4MGDvPDCC3abV/MoeuWv0WgeY9q0aUyfPp3ff/+dnDlzpvr8+vXrM2/ePGJiYoiJiWHJkiU0aNCA2NhYwsLCqFKlCoMHD6Zhw4YcOnSIa9eucf/+fRo1asSYMWPw8vIiLCzMAe9M8xC98tdoMjj169d/pPn77t276dGjB0WKFKFBgwYAZMmShT2J+xcnYsOGDY80nXnnnXcYPnw4oaGhVKxYEYC6devy3nvvERsbS+fOnbl9+zYWi4VChQoxZswY/v3334TXY2NjadKkCS+99JID37VGG3+NJgNz5syZJJ9/2PMgOTp37kznzp2TfG3cuHGMGzcOULVqXFxccHFxYceOHY8dmytXLg4cOJCiOTX2Qbt9NBqNJgOijb9Go9FkQLTx12g0mgyINv4ajUaTAdHGX6PRaDIg2vhrNBpNBkQbf41Go8mAaOOv0WRgkqrnD/Dtt99SsmRJLBYL69ate+L527Zto3Llyo6UqHEQ2vhrNJrHqF27Nhs2bKBWrVpGS9E4CJ3hq9EYSZ06jz/35pvQsyfcvw+vvvr46507q6/r1+E/XbTc4+Kgd2946y2bZP3vf/+z6XyAr7/+OqFCZ4UKFZg6dSqenp6sXr2aoUOH4uLiQmxsLJMmTaJOnTqMGDGCJUuW4ObmhhCCwMBAcuTIYbMOTdJo46/RaOzOxo0bWbBgAbt27SJbtmy8/fbbjBo1iq+//pphw4Yxffp0qlWrRlxcHPfu3ePmzZt89913XLp0CXd3d8LDw3F3dzf6bTzTaOOv0RjJtm1Pfs3D4+mv58792OuR4eF2K+lsC5s3b6Zt27Zkz54dUMXePvnkE0DV8+/Xrx+tWrWiSZMmlClThri4OHx8fOjYsSMNGzakWbNmpngfzzLa56/RaJzKd999x4wZM3B1daVNmzbMmDEDFxcX/vrrL3r37s358+fx9/fnyJEjRkt9ptHGX6PR2J369euzbNkywsPDkVIyb968hPLQISEhlC1blr59+xIQEMC+ffsIDw/n2rVr1K5dmxEjRlCmTJkko5A09kO7fTSaDM5/6/kfPXqUmTNnMmHCBK5du0bnzp1xc3MjKCgowY2TmCNHjjxSz79+/frMnTuXI0eOUK1aNQDKly/P0KFDARg8eDCnTp0iU6ZM5MiRg1mzZnHnzh1atWpFZGQkVquVSpUq8cYbbzj4nWdstPHXaDIwT6rnP3DgQAYOHJjs+XXq1CE6OjrJ1wYNGsSgQYMAVc/f09MTgJ9//jnJ45/ULEbjGLTbR6PRaDIgNhl/IUROIcTvQohT8d+fe8Jxm4QQt4UQT04V1Gg0Go3TsHXlPxjYIqX0BbbEP06KsUAHG+fSaDQajZ2w1fi3AObF/zwPaJnUQVLKLUC4jXNpNBqNxk7YavzzSikvxf98Gchr43gajUajcQJCSvn0A4TYDLyQxEtDgHlSyhyJjr0lpXyS378OMEBK2ewpc3UDugHkzZvXf+nSpcm+gScRERGREF1gJrSu1PGs6fLy8sLHx8cBihRxcXG4uLg4bPy0onWljpTqCg0N5c6dO488V7du3QNSyuRLrUop0/wFhAD54n/OB4Q85dg6wLqUju3v7y9tITAw0KbzHYXWlTqeNV1BQUH2FfIf7t6969Dx04rWlTpSqiupzxOwX6bAxtrq9lkDdIr/uROw2sbxNBqNE/H29sbPz4/y5ctTpkwZnnS3nVTd/mPHjuHt7Z3sHGfOnEnRcWlBCEFERIRDxk6K+/fvU7lyZe7duwfAlStXaNiwISVKlKB8+fJPzFWYMWMG5cqVo2zZspQr93/tnX9wFGWaxz8PISERRY1Ew5KAgFm4HJDJQm0J5NhBwUiwAhagEQPiAXus1BVV5w9+WWhpXQmW1p54Z4GFa7iAEcRVsRYDCASILiiUUQkYEhAhiAJhVQQSfr33R/f0TZKZZCbzI+g8n6qu9Pur+ztPd55+++23nx7IypUrnbKnn36am2++GZfLhcvlYtasWU5ZQUEBH3/8cUR+S6gveS0C1ojINOAb4D4AERkMzDTGTLfTO4B+wLUiUgtMM8ZsCHHfiqKEgbVr19K/f38+++wzhg4dysiRI+natWt7y2rG1TBE8/LLLzN+/Hg6d+4MwLx58xg+fDgbN26kvLycwsJCDhw4gIg0apeRkUFZWRnJycnU1tbicrnIyclxLopTpkzhhRdecOqfOWPNj5k7dy6zZ89m27ZtYf8tIfX8jTF1xpg7jTEZxpiRxpjTdv5uj+O30/9ijEkxxiQZY9LU8SuKhdvtbnXxdgput5uioiIATp061axuXl4ebl/fCAiA7OxsrrvuOr7++uug2x4+fLjRBaNpGuDRRx91er87duxw8tevX8+wYcMYNGgQQ4YMYefOnYB1tzFw4EAefvhhXC4XH3zwQcB6SktLyc7OZuDAgdx5553U1NQAVlyhIUOGOHc6S5YsAeC9995jwIABuFwu+vfvT5mfaKqvvvoqkyZNctJr1qxh5syZAOTk5NCpUyd2797drJ3b7SY5ORmAtLQ0unXrRm1tbau/w+VyceLECaqrqwP+7YGi4R0URQFg69at1NfXk5GR4bN83759uFwuJ11fXx/wtk+fPk1WVhYvvvgiZWVlPPDAAxw8eJDa2lqeffZZNmzYQJcuXaisrGT06NEcOXIEgMrKSpYtW+bECAqEEydOMHnyZLZt20ZmZiavvfYaDz74ILt27eKVV14hPz/fCS/t2Y+vbww05ejRo5w9e5aePXsCUFdXhzGm0UWuR48eHD16tMWP4ZSVlfHDDz8waNAgJ+/NN99k48aNpKamOoHtPAwZMoTNmzf7PS5tRZ2/orQj/nqYgdTv2rVrs/Zn2hDPf8KECSQmJtKlSxfefvttv1/PyszMbNSr3bt3L/fc43fyXiMSEhIoLCwErF5wUlISVVVVlJeXc/DgwUafi7x06RLff/89YA2XBOP4wYoRlJWVRWZmJmB9S+CRRx7hzJkzDB8+nCeeeIJz584xYsQI5zmGr28MNKW2tpZbbgltNvu+ffuYMmUKJSUlzsdqZs6cyYIFC4iPj2fTpk2MHTuWTz75xDmOqampAd0lBIvG9lGUGGft2rVUVFSwfft2J+zyvffe6zyA9Iw/t0THjh25cuWKkw70rsAYw913301FRYWzfPvtt46TDfc03/Hjx7Njxw769OnDokWLmDFjBuD7GwNNSUpKavS7brrpJsAafvNw5MgR0tPTfe67urqavLw8li1bRk5OjpOfmppKfHw8AKNGjSI9PZ39+/c75fX19RH5qpk6f0VRmvHOO+84zjiQO4nU1FQuXrzojK2/8cYbjcovXLjg5O3YsYPz58/Tr18/7rrrLkpLS6msrHTqfvrppyFpv/322/n888/56quvAFixYoXzPKOmpobU1FSmTp3KU089xZ49ewDf3xhoSt++fTl+/DgNDQ1O3sSJE1m6dCkA5eXlnD9/vtFwjodDhw6Rm5vLkiVLGD16dKOyY8eOOesVFRUcPny40RDP/v37ycrKCsEivtFhH0VRQqZjx4689NJLjBo1ipSUFMaMGdOoPDk5mYqKCp5//nmMMZSUlJCQkEBGRgYrV65k2rRpnD9/ngsXLjBs2LCgPiDft29fZ3ZN586dqaqqori4mEmTJnHp0iVSUlKcqZVr1qxh1apVJCQkICIsXrwY8P2NgaYkJSUxYsQIysrKyM3NBWDRokUUFhayYsUKkpKSKC4upkMHq089ffp08vPzyc/PZ86cOdTV1bFw4UIWLlwIWB+4z83NZf78+ezZs4e4uDgSEhIoLi527nzOnj1LZWUld9xxRzCHIzACeRmgPRZ9ySu6qK7g0Je8guPXouujjz4yY8eOjZCa/8eja+nSpebJJ5/0W689X/JSFEWJGYYOHco999zjczZQJIiLi2PuXH/BkkNDh30URVGCYPr06a1X+gXsS3v+iqIoMYg6f0VRlBhEnb+iKEoMos5fURQlBlHnrygxjIZ0Do6mIZ3dbje9e/d23oZ+/fXXfba7fPkys2bNok+fPtx2220sX77cKZszZ06zl+Kigc72UZQYR0M6B07TkM4AS5YsaTXG0apVq6ipqaG6upq6ujqys7MZOXIkt956K4899hg5OTkUFBQ4L4hFA+35K0o74i5yU1RRBMDFyxdxF7lZ+YX1Nuq5i+dwF7lZvXc1AD/W/4i7yM1f9/8VgFPnTuEucvN+1fsAfPfzd+StyaO0prRNWjSkc/AhnQNl9erVzJgxgw4dOpCSksK4ceN46623AEhJSaF3795s3rw56O2Ggvb8FUUBNKRzsCGdPTz++OPMmzePrKwsFi9eTPfu3Zu1PXLkSKN2ntDPHjxhmz2B9aKBOn9FaUfKppY56/Fx8Y3S18Rf0yh9feL1jdJdr+naKJ16bSrr71uvIZ2jGNK5uLiY9PR0Ll++zHPPPcf9999PeXl5UHrBCoy3ffv2oNuFgg77KEqMoyGd2xbSGXDCN8fFxTF79mx27tzZyA4eevTowTfffOOkm4Z+jlTY5pZQ568oSjM0pHPrIZ2971AASkpKGDBggM+Htp4LypUrVzh58iTvvvsuEyZMcMojFba5JXTYR1GUkInFkM4NDQ2MGTOGCxcuYIyhe/fujabK5uXl8cwzzzB48GAmT57Mrl27nOcpCxcupFevXoB197Nlyxbmz58fhMXDQCChP9tj0ZDO0UV1BYeGdA6OX4uuSIR0Li0tNYWFhY3yAtWlIZ0VRVGiQCRCOv/000/OHUg00WEfRVGUIAh3mOWJEyeGdXuBoj1/RYky1p25ooRGqOeROn9FiSKJiYnU1dXpBUAJCWMMdXV1JCYmtnkbOuyjKFEkLS2N2tpaTp48GZHt19fXh+QQIoXqCo5AdCUmJpKWltbmfajzV5QoEh8f70zxiwRlZWVkZ2dHbPttRXUFRzR06bCPoihKDBKS8xeRZBHZJCLV9t8bfdRxicjfRaRSRL4QkftD2aeiKIoSOqH2/OcCm40xGcBmO92Uc8AUY8w/A3cD/yUiviNHKYqiKFEhVOc/Flhhr68AxjWtYIw5YIyptte/BU4AKSHuV1EURQmBUB/43mKMOW6vfwfc0lJlEfk9kAAc9FP+R+CPdvJnEakKQVtX4FQI7SOF6goO1RUcqis4fo26erZeBaS1+cYi8iGQ6qNoAbDCGHODV91/GGOajfvbZd2AMuAhY8zOQMSFgojsNsYMbr1mdFFdwaG6gkN1BUcs62q152+MGemvTES+F5FuxpjjtnM/4adeF+BvwIJoOH5FURSlZUId818HPGSvPwS817SCiCQA7wD/a4xZG+L+FEVRlDAQqvNfBIwSkWpgpJ1GRAaLyHK7zn3AcGCqiFTYi8v35sLKq1HYR1tQXcGhuoJDdQVHzOpqdcxfURRF+fWhb/gqiqLEIOr8FUVRYpBftPMXkYl22IgrIuJ3WpSI3C0iVSJSIyJzvfJ7icguO3+1/XA6HLoCCXsxwusZSIWI1IvIOLusSES+DvczkkB02fUue+17nVd+e9rLb5iQcNrL37niVd7J/u01ti1u9SqbZ+dXiUhuWzW0Udd/iMg+2zabRaSnV5nP4xlFbVNF5KSXhuleZQ/Zx71aRB5q2jaCmv7speeAiPzgVRYxe4nIX0TkhIjs9VMuIrLE1v2FiPzOqyy8tgrkW49X6wL8E9AX6/2BwX7qxGG9VNYb6wWzz4FMu2wNUGCvLwX+FCZdzwNz7fW5wOJW6icDp4Fr7HQRMCEC9gpIF/Czn/x2sxfwWyDDXv8NcBy4IZz2aulc8arzCLDUXi8AVtvrmXb9TkAveztxYbJPILpGeJ0/f/Loaul4RlHbVOC/fbRNBg7Zf2+012+MhqYm9f8d+EuU7DUc+B2w1095HvABIMDtwK5I2eoX3fM3xuw3xrT2FvDvgRpjzCFjzAXgTWCsiAhwB+CZfuozPEUbaTXsRRMmAB8YY86Faf/+CFaXQ3vby0QnTIjPc6UFrWuBO23bjAXeNMY0GGO+Bmrs7UVFlzFmq9f5sxNoe6D3MGtrgVxgkzHmtDHmH8AmrPhf0db0AFAShv22ijFmO1ZHzx9jsabFG2O9E3WDWO9Qhd1Wv2jnHyDdgaNe6Vo77ybgB2PMpSb54SCosBdYPcimJ99/2rd9fxaRTlHWlSgiu0Vkp2coiqvIXuI7TEg47OXvXPFZx7bFj1i2CaRtWwl229Oweo8efB3PcBGotvH28VkrIulBto2UJuzhsV7AFq/sSNqrNfxpD7utrvqPuUgL4SWMMc1eKosWLenyThhjjIj4nU9rX9UHABu8sudhOcEErPm+c4BnoqirpzHmmIj0BraIyJdYTq7NhNlexVhhQq7Y2W22168NESkEBgN/8MpudjyNMT7ja0WI94ESY0yDiPwb1p3THVHcf0sUAGuNMZe98trbXlHhqnf+poXwEgFyDEj3SqfZeXVYt1Qd7R6cJz9kXRJg2Aub+4B3jDEXvbbt6QU3iMjrwGPR1GWMOWb/PSQiZUA28DbtbC/xEyYkFHs1wd+54qtOrYh0BK7HOpcCadtWAtq2iIzEupj+wRjT4Mn3czzD5cxa1WaMqfNKLsd6xuNp627StiwamrwoAGZ5Z0TYXq3hT3vYbRULwz6fAhlizVRJwDrY64z1FGUr1ng7+AlP0UZaDXvhRbPxRtsBesbZxwE+ZwZEQpeI3OgZNhGRrsAwYF9720taCBMSRnv5PFda0DoB2GLbZh1QINZsoF5ABvBJG3UErUtEsoFlQL4x5oRXvs/jGSZdgWrr5pXMB/bb6xuAu2yNNwJ30fgOOGKabF39sB6e/t0rL9L2ao11wBR71s/twI925yb8tgr30+xoLsC9WGNfDcD3wAY7/zfAeq96ecABrKv3Aq/83lj/oDXAW0CnMOm6CevjNtXAh0CynT8YWO5V71asK3qHJu23AF9iObGVwLXR0gUMtff9uf132tVgL6AQuAhUeC2ucNvL17mCNYSUb68n2r+9xrZFb6+2C+x2VcDoMJ/rren60P4f8NhmXWvHM4rangMqbQ1bgX5ebf/VtmUN8HC0NNnpp4FFTdpF1F5YHb3j9rlci/V8ZiYw0y4X4H9s3V/iNYsx3LbS8A6KoigxSCwM+yiKoihNUOevKIoSg6jzVxRFiUHU+SuKosQg6vwVRVFiEHX+iqIoMYg6f0VRlBjk/wD+XPQOR3+6CwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_array = sess.run(x_vals)\n",
    "plt.plot(x_array, l2_y_out, 'b-', label='L2 Loss')\n",
    "plt.plot(x_array, l1_y_out, 'r--', label='L1 Loss')\n",
    "plt.plot(x_array, phuber1_y_out, 'k-.', label='P-Huber Loss (0.25)')\n",
    "plt.plot(x_array, phuber2_y_out, 'g:', label='P-Huber Loss (5.0)')\n",
    "plt.ylim(-0.2, 0.4)\n",
    "plt.legend(loc='lower right', prop={'size': 11})\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Categorical Predictions\n",
    "\n",
    "-------------------------------\n",
    "\n",
    "We now consider categorical loss functions.  Here, the predictions will be around the target of 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Various predicted X values\n",
    "x_vals = tf.linspace(-3., 5., 500)\n",
    "\n",
    "# Target of 1.0\n",
    "target = tf.constant(1.)\n",
    "targets = tf.fill([500,], 1.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Hinge Loss\n",
    "\n",
    "The hinge loss is useful for categorical predictions.  Here is is the `max(0, 1-(pred*actual))`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Hinge loss\n",
    "# Use for predicting binary (-1, 1) classes\n",
    "# L = max(0, 1 - (pred * actual))\n",
    "hinge_y_vals = tf.maximum(0., 1. - tf.multiply(target, x_vals))\n",
    "hinge_y_out = sess.run(hinge_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Cross Entropy Loss\n",
    "\n",
    "The cross entropy loss is a very popular way to measure the loss between categorical targets and output model logits.  You can read about the details more here: https://en.wikipedia.org/wiki/Cross_entropy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Cross entropy loss\n",
    "# L = -actual * (log(pred)) - (1-actual)(log(1-pred))\n",
    "xentropy_y_vals = - tf.multiply(target, tf.log(x_vals)) - tf.multiply((1. - target), tf.log(1. - x_vals))\n",
    "xentropy_y_out = sess.run(xentropy_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Sigmoid Entropy Loss\n",
    "\n",
    "TensorFlow also has a sigmoid-entropy loss function.  This is very similar to the above cross-entropy function except that we take the sigmoid of the predictions in the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# L = -actual * (log(sigmoid(pred))) - (1-actual)(log(1-sigmoid(pred)))\n",
    "# or\n",
    "# L = max(actual, 0) - actual * pred + log(1 + exp(-abs(actual)))\n",
    "x_val_input = tf.expand_dims(x_vals, 1)\n",
    "target_input = tf.expand_dims(targets, 1)\n",
    "xentropy_sigmoid_y_vals = tf.nn.sigmoid_cross_entropy_with_logits(logits=x_val_input, labels=target_input)\n",
    "xentropy_sigmoid_y_out = sess.run(xentropy_sigmoid_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Weighted (Softmax) Cross Entropy Loss\n",
    "\n",
    "Tensorflow also has a similar function to the `sigmoid cross entropy` loss function above, but we take the softmax of the actuals and weight the predicted output instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Weighted (softmax) cross entropy loss\n",
    "# L = -actual * (log(pred)) * weights - (1-actual)(log(1-pred))\n",
    "# or\n",
    "# L = (1 - pred) * actual + (1 + (weights - 1) * pred) * log(1 + exp(-actual))\n",
    "weight = tf.constant(0.5)\n",
    "xentropy_weighted_y_vals = tf.nn.weighted_cross_entropy_with_logits(logits=x_vals,\n",
    "                                                                    targets=targets,\n",
    "                                                                    pos_weight=weight)\n",
    "xentropy_weighted_y_out = sess.run(xentropy_weighted_y_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Plot the Categorical Losses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd8zdf/wPHXyZZBCEJsESKyNIlVRawa39q7jS1qVGu1NjWKDmp2UMTeVVVaK6FGrYq9olaomokE2ef3xyfuz0jIuLk34zwfj/vI/dx77ue8XXze+Xw+55y3kFKiKIqi5D0mxg5AURRFMQ6VABRFUfIolQAURVHyKJUAFEVR8iiVABRFUfIolQAURVHyKL0kACGElRDisBDihBDijBDi8xTaWAoh1gghwoQQh4QQZfXRt6IoipIx+joDiAXqSym9AG+giRCixkttegEPpZQVgJnAdD31rSiKomSAXhKA1EQnb5onP16eYdYSCEp+vh5oIIQQ+uhfURRFST8zfe1ICGEKHAMqAPOklIdealICuAEgpUwQQkQCDsC9l/YTCAQC5MuXz6dUqVIZiicmJobw8HDMzc0pVaoUJibGvd2RkCC4ds0GExNJmTJPMDHJXjOwk5KSjP4dpSQpKYn8V66QYGtLjKOjscPRyc7fl4or7XJjXBcvXrwnpSySpsZSSr0+AHsgGHB/6fXTQMnnti8DhV+3Lx8fH5lRwcHBcuvWrdLExES2adNGJiYmZnhf+rJ3r5SmpomyZUsps0E4LwgODjZ2CCkKDg6Wslw5KT/4wNihvCBbf1/ZkIorfTITF3BUpvF4rffUJ6WMSE4ATV566yZQCkAIYQYUAO7ru//nNW3alG+++YaNGzcybty4rOwqTd55Bz788DK//ALTphk7mhxkyBBo3drYUShKrqOXS0BCiCJAvJQyQgiRD2jEqzd5NwPdgINAO2B3crbKUh9//DFnzpxhypQpuLm50aVLl6zu8rXatr3Jw4cujBkDvr7QuLFRw8kZBg40dgSKkivp6wygOBAshDgJHAF2SCm3CCEmCiFaJLf5CXAQQoQBQ4AReur7tYQQzJs3j7p169KzZ08OHz5siG5fEw/8+CO4u0PnznD1qlHDyRkePIBbt4wdhaLkOno5A5BSngSqpvD6uOeexwDt9dFfellYWLBhwwaGDx9OhQoVjBHCC2xs4OeftTOANm1g/37Il8/YUWVj3bpBeDgcP27sSBQlV8l+t7+ziIODA4sWLaJQoULExsby+PFjo8bj7AzLl2vHtH79QJVleA0bGzDy35ei5EZ5JgE8k5iYSOPGjQkICMAAtyBeq3lzmDABgoLg+++NGkr2ZmMDT54YOwpFyXX0Ng8gpzA1NaVjx44UKVKE7DAPbexYOHoUPv4YvLygVi1jR5QNqTMARckSeS4BAPTv31/3/MGDBxQqVMhosZiYwLJl2v2Adu3g77+hWDGjhZM9qQSgKFkiz10Cet5vv/1GmTJlOHjwoFHjsLeHjRshMhI6dID4eKOGk/289x7MmKFulCiKnuXpBFCjRg2KFStGq1atuHbtmlFj8fSEhQvhzz9h+HCjhpL91KqlzQXIBpfsFCU3ydMJwMHBgV9//ZXY2FhatGhBdHT0mz+UhTp3hk8+gVmzYOVKo4aSvURFwcmTEBNj7EgUJVfJ0wkAwNXVlbVr13LmzBnef/99kpKSjBrPl19CnTrQu7d2zFOAP/7Q7pBfvGjsSBQlV8nzCQCgcePGfPvtt2zevJkRIwwyQTlV5uawZg0ULKhNEnv40KjhZA/582s/Hz0ybhyKksuoBJBswIAB9O/fn6+++ooFCxYYNZZixWD9erh+HT74AIx8UmJ8KgEoSpZQCSCZEIJZs2bRpEkT+vfvz86dO40aT82a2r2ArVth4kSjhmJ8BQpoP1UCUBS9UgngOWZmZqxZswY/Pz/is8FYzA8/hO7d4fPPYcsWY0djROoMQFGyRJ6cCPY6+fPnZ//+/bpZwomJiZiamholFiFg/nztZvAHH2gzhrPBWnaG5+AAP/0Eb79t7EgUJVdRZwApeHbw/+6776hbty5Pnz41Wiz58sGGDWBqqt0UzpMTYq2soGdPqFTJ2JEoSq6iEsBrFC1alKJFixp9aGjZsrBqFZw+DX365NEJsceOwYULxo5CUXKVTCcAIUQpIUSwEOKsEOKMEOLjFNrUE0JECiFCkx/Gr8+YBm3btmXDhg3Y2NgY/Z5A48YwZYqWCGbPNmooxtGqFUx/ucicoiiZoY8zgARgqJTSDagBDBBCuKXQ7k8ppXfyI8eMaxFCcPfuXapVq0ZQUJBRYxkxQjsODh0Ke/YYNRTDs7dXkyIURc8ynQCklP9KKf9Ofh4FnANKZHa/2Ym9vT0ODg706dOHkJAQo8UhhFY7wNlZWzTu5k2jhWJ4Dg5aaUhFUfRGr/cAhBBl0UpDHkrh7ZpCiBNCiG1CiCr67DermZubs379eipUqECbNm24YMRr0fnza+UknzzRlo+OjTVaKIZVqBDcv2/sKBQlVxH6qoolhLAF9gBTpJQbX3ovP5AkpYwWQjQDZkkpXVLZTyAQCODo6OizevXqDMUTHR2Nra1thj6bmn///Zf+/ftjbW3N/PnzKfBsgpIR4tqzpwgTJlShRYubDB58KdP7y4rvSx+exVXx669xOHiQgxs2GDskIPt/X9mNiit9MhOXv7//MSmlb5oaSykz/QDMgT+AIWlsfxUo/KZ2Pj4+MqOCg4Mz/NnXOXDggLS0tJS1a9eWMTEx6f68PuP69FMpQcrFizO/r6z6vjJLF9fx41Ju327UWJ6X7b+vbEbFlT6ZiQs4KtN47NbHKCAB/ASck1LOSKVNseR2CCGqoV16ypHn8zVr1iQoKIh9+/bRu3dvo9YVnjIF6tfXZgz//bfRwjAMb29o1MjYUShKrqKPmcBvAwHAKSFEaPJro4DSAFLK74F2QD8hRALwFOgkjXnkzKSOHTsSFhbGmDFjcHZ2ZsKECUaJw8wMVq8GHx9tktixY9q90lzp9m346y8t4z1bGkJRlEzJdAKQUu4DXluqSUo5F5ib2b6yk1GjRhEWFkZYWBhJSUmYmBhnTl2RItpM4dq1tYIy27Zps4Zznb/+gtattSz31lvGjkZRcgU1EziDhBD8+OOPLFu2DBMTE6NeCvLz09YM2rEDxuWIKXYZ8OzURo0EUhS9UQkgE8zNzRFCcOXKFapVq8aJEyeMFkuvXhAYCF98oQ0TzXVUAlAUvVMJQA/Mzc2Jj4/nsZFXaps9G6pVg27d4Px5o4aif4UKaT/VZDBF0RuVAPSgZMmS/P3339SqVQuAhIQEo8RhaalVErOy0m4KR0UZJYys8SwBqDMARdEblQD05NlN4KlTp9KkSRPi4uKMEkepUlpN4QsXoEePXLRyqIUFBAdrfyhFUfRCJQA9K1myJLt27TLqHAF/f/jyS2100NdfGyWErFGvHpQsaewoFCXXUBXB9CwgIIBr164xduxYypQpw6RJk4wSx5AhcPiwtoLoW29BgwZGCUO/duyAuDho3tzYkShKrqASQBYYPXo0165dY/LkyZQuXZo+ffoYPAYhtCqKp09Dp07a8PnSpQ0ehn599ZVWF1glAEXRC3UJKAsIIfjuu+9o0qQJ/fr1Y9u2bUaJw9ZWGxIaFwdt20JMjFHC0B8HB7h3z9hRKEquoRJAFjEzM2Pt2rV4enrSvn17/jbSYj0VK8LSpVpB+YEDjRKC/jg6wn//GTsKRck1VALIQnZ2dvz2228ULlyY5s2bc/XqVaPE0bIljBmjXRJasMAoIehH8eIQHa09FEXJNJUAsljx4sXZunUrBQsWJCIiwmhxTJgA776rnQUcPmy0MDLHyUn7+e+/xo1DUXIJlQAMwM3NjVOnTuHt7Q1glALzpqawciWUKKHdD7hzx+AhZN7//qdNcS5b1tiRKEquoBKAgZgmL9G5YMECWrZsaZQkUKiQNjfg3j3o2BGMNGE54woWhEqVwNzc2JEoSq6gEoCBFS9enPLlyxtt+eiqVeHHHyEkBEaONEoIGZeQADNnwt69xo5EUXIFlQAM7H//+x9z587F1NSUyMhIo8QQEAADBmizhNeuNUoIGWNqCqNGwa+/GjsSRckV9FESspQQIlgIcVYIcUYI8XEKbYQQYrYQIkwIcVIIkecreoSHh+Pu7s6MGSlW0cxyM2ZArVrQsyecOWOUENJPCG0kkLoJrCh6oY8zgARgqJTSDagBDBBCuL3UpingkvwIBL7TQ785WvHixalZsyZDhw5lxYoVBu/fwgLWrQM7O63QVnR0DikjVrw43Lpl7CgUJVfIdAKQUv4rpfw7+XkUcA4o8VKzlsDS5KL1fwH2Qojime07JzM1NWXZsmX4+/vTvXt3/vjjD4PH4OSkJYErV2Dq1MokJRk8hPRzclJnAIqiJ0KfK1YKIcoCewF3KeWj517fAkxLrh+MEGIX8JmU8mgK+whEO0vA0dHRZ/Xq1RmKJTo6Gltb2wx9Niu9HFd0dDSDBw8mPDycGTNmULlyZYPHtHFjCebMcaFXr3/44IPrBu//dV7+virMnk2x7dvZt2WLEaPKOf++sgsVV/pkJi5/f/9jUkrfNDWWUurlAdgCx4A2Kby3Baj93PYuwPdN+/Tx8ZEZFRwcnOHPZqWU4vr3339luXLlZOHCheWFCxcMHlNSkpQNG96WQki5bZvBu3+tV76vhw+ljI42SizPy0n/vrIDFVf6ZCYu4KhM43FbL6OAhBDmwAZghZRyYwpNbgKlntsumfyaAhQrVozt27cjhKBx48bcMvA1biFg6NALeHpCly7aJaFsy94ebGyMHYWi5Ar6GAUkgJ+Ac1LK1Ia0bAa6Jo8GqgFESinVhdznVKhQgW3btnH//n0aNWrEo0eP3vwhPbKySmLDBq2CWJs28OSJQbtPu1u3YOhQCA01diSKkuPp4wzgbSAAqC+ECE1+NBNCfCiE+DC5zVbgHyAMWAD010O/uY6Pjw+//vorrVu3xs7OzuD9Oztry0WcOAEffphNy0kmJGhjWI8cMXYkipLjZbogjNRu7Io3tJHAgMz2lRfUq1ePevXqAfDPP/9QvHhx8uXLZ7D+mzbVFo4bPx6qV9cmjGUrTk5gZgZGWllVUXITNRM4m4qKiuLtt9+mX79+Bu97zBht3bVPPoH9+w3e/euZmWl1ga9dM3YkipLjqZKQ2ZSdnR3Tp0+nevXqBu/bxASWLQM/P2jfXisnWTw7zdooW1adASiKHqgzgGysa9euVKpUCSklmzZtejaE1iDs7bVykpGR0KEDGGHx0tSVLavVBlYUJVNUAsgBNm/eTOvWrRlp4OU73d21KmL79sGwYQbt+vUWLoSTJ40dhaLkeCoB5AAtWrSgX79+TJ8+nWnTphm0706dYPBgmD0bli83aNepM80h6xYpSjanEkAOIIRg7ty5dOnShZEjR/Ldd4ZdS2/6dKhbFwIDtSGiRnflinZdKsfWtlSU7EElgBzCxMSEJUuW0KJFCwYMGMCSJUsM1re5OaxZo1UUa90aHjwwWNepB7RuHfz9t5EDUZScTSWAHMTc3Jw1a9bQsGFDevbsycqVKw3Wt6MjrF8P4eHwwQcYd+VQJyewtoaLF40YhKLkfCoB5DBWVlZs2rSJevXq0bVrV9atW2ewvmvUgDlzYNs2+Pxzg3X7KhMTcHFRCUBRMkklgBzI2tqazZs3U6NGDT777DNiY2MN1ndgIPToARMnGrkyY8WKcOGCEQNQlJxPTQTLoWxtbdm6dSsPHjzA0tLSYP0KAfPmaaMwAwK0JXlcXAzW/f+rWlWbDZyUpJ0RKIqSbup/Tg6WP39+ypYtS1JSEoMHD2b79u0G6TdfPtiwQVuVoU0bePzYIN2+aORIOHRIHfwVJRPU/55c4PHjxwQHB7N3716D9VmmDKxaBWfPQu/e2XTlUEVRXkslgFzAzs6Offv2MWnSJAASEhIM0m+jRjBlCqxeDd9+a5Au/19MDNSqBQaeE6EouYlKALmEra0tQgjOnj1L5cqV2bdvn0H6/ewzbW7A8OEQEmKQLjVWVvDPP6ougKJkgr5KQi4SQtwRQpxO5f16QojI5wrGjNNHv8qr7O3tMTU1pUmTJuzZsyfL+xMClizRbgR37KjNEzAYd3c4dcqAHSpK7qKvM4AlQJM3tPlTSumd/Jiop35TFPYgjE9CP+HCvbw3TNDJyYng4GBKly5N06ZN2b17d5b3mT8/bNyolZFs1w4MNirVwwPOnDHyrDRFybn0kgCklHsBYy8QoHM98joP4h6Q3zK/sUMxiuLFixMcHIyzszPNmzdnx44dWd5n5coQFKQNzPnkkyzvTuPhAU+fapeCFEVJN6GvNeaFEGWBLVJK9xTeqwdsAMKBW8AwKeWZVPYTCAQCODo6+qxevTpD8TyKekR+Oy0BfH3xa9zzu9Ok2JtOUrJedHQ0tra2BukrIiKCoUOHcuPGDSZPnky1atWyPK4ffyzPqlWl+fTT8zRtejvT+3tdXDaXL1Nu0SL+CQzkSZkyme5LX3EZk4orfXJjXP7+/seklL5paiyl1MsDKAucTuW9/IBt8vNmwKW07NPHx0dmVHBwsJRSysdxj2WdxXXklL1TMrwvfXoWl6Hcu3dPent7SwsLC7lly5ZU2+krrvh4KRs2lNLSUsqjRzO/P0N/X2ml4kofFVf6ZCYu4KhM43HbIKOApJSPpJTRyc+3AuZCiMKG6Nva3JqQbiF8+vanAOy9tpc+m/sQGRNpiO6NzsHBgV27duHh4UHr1q05f/58lvZnZqbND3B01CaJ3buXpd1pDLgUhqLkJgZJAEKIYkIIkfy8WnK/9w3Rd3KfmJloq16E3g4l5FoI5qbmhure6AoVKsTOnTuZO3culSpVyvL+ChfWZgr/9x907gyJiVnYWd++2mggRVHSTV/DQFcBB4FKQohwIUQvIcSHQogPk5u0A04LIU4As4FOyacqBjeo+iBO9TuFtbk1iUmJ9PylJ4dv5v7CIvb29gQGBiKEIDQ0lKCgoCztz9cX5s+HnTthzJgs7KhcOQgLywZFChQl59HLYnBSys5veH8uMFcffemDlZkVAFcjrrL98nYalW9EtRKp3yDNbb755hv27t1Lu3btsLGxybJ+evbUinZNmwZ+ftolIb3z89N+Hj0KjRtnQQeKknvl6dVAnQs5c2HgBazNrQFYe2Ytdx7foZ9vP0xNcm/d2YULF3L79m1sbGxISkoi+epclpg1C0JDoVs3cHMDV1c9d+Djo/1UCUBR0i3PLwVhY2GjOwBuvrCZFadWZOkBMTuwtLSkTJkySCkZMGAAQ4cOJauuyFlaapXErK21JSOiovTcgb29Ng1ZLQmhKOmW5xPA85a1Xsa297dhIkyIjotm0LZB3Iq6ZeywsoyUEgsLC2bOnMnXX39NYhbdrS1ZUqspfOkSdO+eBSuHfvqptg6FoijpohLAc4QQ2FvZA3DgxgEW/L2A65HXjRxV1jExMeHbb79lzJgxbN26lc6dO2dZdbF69eDLL7UlI778Us87790bOnXS804VJffL0/cAXqexc2Ouf3KdIjZFAJh3eB4l85ekpWtLI0emX0IIJk2axP379/nuu++4f/8+P//8M/nz638ZjcGDtZvCo0Zpl+4bNtTTjpOStPrAtrba6YaiKGmizgBe49nBPzEpkSUnlrD6TMaWpcgJOnTowNKlS9m7dy916tTh33//1XsfQsDChdq6QZ06aRUd9SImBqpUgR9/1NMOFSVvUAkgDUxNTDnY6yDfNdeKj9yIvMHg3wfz8OlDI0emXwEBAWzZsoWwsDBq1qzJhSwoum5rCz//DPHx0LatduzONGtr8PICA9VAUJTcQiWANDIzMdPdH9h9ZTcL/l5AZGzuW07i3XffJSQkhCdPntCzZ88sGR3k4gLLl8OxY9C/v55uCtetCwcPQlycHnamKHmDSgAZ0M27G9c+uUZZ+7IAfB7yOSFXQ4wakz75+vpy8OBBli9fjhAiS5LAe+/B2LGweLGertzUqaOdTqjhoIqSZioBZJCDtQMAUbFRLDmxhO2Xtxs5Iv1ydnamXLlyJCUl0aNHDxYuXKj3PsaPh6ZN4aOP4K+/Mrmzd97Rfu7dm+m4FCWvUKOAMsnO0o6z/c8i0X5LPnrrKJvOb2Jk7ZHYWGTdMguGEhsby3///cfdu3f1vm9TU+1SkK+vVkns2DFtFdEMKVwYfv/9/2cGK4ryRuoMQA/ymefTLSfxR9gf/HT8J5Jk7ihTmC9fPn799VdGjBgBQGhoKI8fP9bb/gsV0m4KP3igzeVKSMjEzt59V0sEiqKkiUoAeja6zmjO9j+LnaUdUkp6b+6d4+8PmJmZIYQgKiqKhg0bUrduXW7d0t8MaS8v7T7Anj3w2WeZ2NH9+/D115DFNQ8UJbdQCSALFMxXEIBbUbfYfWU3YQ/CjByRftjZ2bF48WLOnz9P9erVOXHihN72/cEH2r2AGTMgg1VAtbGlw4fDL7/oLS5Fyc1UAshCJfKX4NyAc/Tw7gHAhrMbmHlxJtFx0UaOLOPee+899u3bh5SS2rVrs2XLFr3t++uv4e23oVcvOH06AzsoVgw8PeGPP/QWk6LkZvoqCLNICHFHCJHif1uhmS2ECBNCnBRCvKWPfnMCSzNL3dLSF+9f5FzUOfKZ5TNyVJnj7e3N4cOHqVixIi1atGDq1Kl6GSpqYQHr1kH+/NrKoRERGdhJkybahDC9LzuqKLmPvs4AlgBNXvN+U8Al+REIfKenfnOUke+MZH7V+ZiamBKTEIN/kD+/h/1u7LAyxMnJiT///JNOnToxatQoOnToQHR05s9sihfXlo++ehW6dtWW+UmXJk20S0G7d2c6FkXJ7fSSAKSUe4HX1eRrCSxNLlr/F2AvhCiuj75zmme1iW9H3yYiJkK3nRNHDVlbW7NixQq++uorNm7cSJ06dYiPj8/0ft9+G2bOhF9/heXLy6T/wwULamUiFUV5LaGvWZ5CiLLAFinlKxW6hRBbgGlSyn3J27uAz6SUR1NoG4h2loCjo6PP6gzeEYyOjsbW1jZDn81Kz8eVJJMwEVoOXnV9FSciT/C52+dYmloaNa6MOHr0KP/++y/vvfeeXuKREqZOdWXnTkemTj1F9eppr/kr4uKQFhZ6iSM1OeHfV3ai4kqfzMTl7+9/TErpm6bGUkq9PICywOlU3tsC1H5uexfg+6Z9+vj4yIwKDg7O8GezUmpxzT88X36w8QPd9uO4xwaKSKPP72vz5s3yyy+/lElJSZnaz+PHUjo7R0l7eynDwjKwg0z2/zo57d+Xsam40iczcQFHZRqP24YaBXQTKPXcdsnk15Rk/fz6saz1MgDuPr5LmW/LEBQaZOSoMmbTpk2sWbMm05eDrK1h4sTTCKGtHPrkSRo/GBurLQ73zTeZ6l9RcjtDJYDNQNfk0UA1gEgppf4XnM8lJJJWlVpRvWR1AB4+fcjjOP3Nvs1qCxcuZMeOHVhYWPDw4UOOHTuW4X05OcWwciWcPAmBgWlcOdTSEp4+1e4mK4qSKn0NA10FHAQqCSHChRC9hBAfCiE+TG6yFfgHCAMWAP310W9uVdSmKAtaLMC1sCsAo3ePpvK8yjyNf2rkyNJGCEHBgtpkuFGjRlGzZk3mzp2b4aGiTZrAxImwYgXMnZvGD7VpA4cOQXh4hvpUlLxAX6OAOkspi0spzaWUJaWUP0kpv5dSfp/8vpRSDpBSOkspPWQKN3+V1AV4BvDp25+Sz1ybP3DgxgESk7KmgLu+TZkyhXfffZePPvqIjh078ujRowztZ9QoaNEChgxJY92XNm20n5s2Zag/RckL1EzgHKBmqZoMrDYQgEv3L/HO4nf46sBXRo4qbQoVKsQvv/zC9OnT2bhxIz4+Phw+fDjd+zExgaVLoVw5aN8e3lixsmJFrUzkxo0ZC1xR8gCVAHIY50LOrG67ml5VewFw6r9T/Hrh1ywp2qIvJiYmfPrpp4SEhBAXF0etWrWYMmUKiYnpO4spUEA7nkdFaUngjcW/hg3T7h4ripIilQByGBNhQvsq7XUF62cfmk23Td2Iisv+Sx/Url2bEydO0KFDB8aMGUO9evW4evVquvbh7g6LFsH+/TB06Bsad+8OAwZkNFxFyfVUAsjh5jefT0j3EPJb5kdKyZA/hrD/+n5jh5Uqe3t7Vq5cyfLlyzl58iTffZf+VUE6dNAO/nPnwrJlb2gcEaEuAylKKlQCyOHMTc3xdPQE4L/H/7H69Gr+/vdvgOcn4mU777//PidPnuTzzz8H4OTJk+mqOjZtGtSrpw0NPX78NQ0XLtQuA126lLmAFSUXUgkgFylmW4zLgy7T17cvAL9c+IV3Fr/DrSj9FW/RpzJlymBlZUVSUhKdOnWiZcuWaU5YZmawZg04OGjH9weprRTRuTMIoY0hVRTlBSoB5DL5zPNhYaqtg5OYlIiFqQVFbYoCcPPRzWx5RmBiYsLatWv59ttvEULw5MkT7ty588bPFS0KGzbAzZvQpQukeE+5RAnw99eKD6d7aVFFyd1UAsjF2rq1ZXe33ZiZmJGYlEi9oHp03dTV2GGlyN3dnWrVqgEwfvx43NzcWL169RsTVvXqMGeOVgNmwoRUGvXoAZcvQ3CwfoNWlBxOJYA8ZGydsXT11BJATEIMq06tIj4x88s361uPHj1wdnamc+fOjBgxgsuXL7+2fZ8+WhWxyZNh8+YUGrRrp1WfDwnJkngVJadSCSCPMDUxpatXVxo5NwJg3Zl1dNnYhYPhB40c2avc3Nw4cOAAs2bN4vTp07i7uzNlyhRiY2NTbC+ENiLI1xcCAuDixZcaWFlpheInTcr64BUlB1EJII963/N9grsF807pdwBYH76e0btGZ5t7BKampgwaNIglS5bw3nvvMWbMGLy9vQlJ5bd4KyvtfoCFhbYKxCvFyYpo8yZISMjSuBUlJ1EJII8yESbUK1sPIQQA159c5/Td07rtB0/TXoAlKxUpUoS1a9fy22+/ERsbi7+/P++//z4JKRzIS5eG1avh3DntktAruWzqVK1ovLoZrCiASgBKsiEVh7ChwwYA7j25R+mZpZnfqn6KAAAgAElEQVR7OK1Lb2a9Zs2acfr0acaMGYOFhQVmZlopzZcTQYMG2nF+7VqYMeOlnZQtq2WHHTsME7SiZHMqASg6z+oTmwpTBtcYTINyDQC4/OAyi44vIiYhxpjhYW1tzaRJk1i0aBEAZ86coUKFChw6dOiFdsOHa3MDPvvspYE/bdpAsWJawWFFUVQCUF5VMF9BJtWfROUilQFYfXo1H275kIiYCMD4BeyfXaaKj4/HxcUFZ2dnAN1S00LA4sXg4gIdO8KNG8kftLSEQYO0MaMnTxojdEXJVvRVEKaJEOKCECJMCDEihfe7CyHuCiFCkx+99dGvYhij3hnFiQ9PUMy2GADvb3yfflv6GTkq8Pb2ZseOHRQuXBgpJfXr16dx48YcPXoUOzv4+WeIidFGgeoGEPXtCzY2KVwfUpS8J9MJQAhhCswDmgJuQGchhFsKTddIKb2THwsz269iOEII3dmAlJLS+UtTMn9J3furTq0iMibSWOEBkJiYSJcuXfj777/x8/OjXbt2wHmCguDwYe0Xf0CbD7B0KYwfb8xwFSVb0McZQDUgTEr5j5QyDlgNtNTDfpVsSAjB9EbTGV1nNACn75ymy8YuLD2xFDDeAnRmZmYMGTKEf/75h/Hjx/PHH39QpUoVNm4MoE+fs/z4I/z0U3LjNm20yjKKkseJzP5nFUK0A5pIKXsnbwcA1aWUA59r0x2YCtwFLgKDpZQ3UtgdQohAIBDA0dHRZ/Xq1RmKKzo6Gltb2wx9NivltriklFyMvkiJfCWwNbNl3719LLm6hCnuU3C0cjRaXBEREaxatYrNmzcTGxuLvf17REWNZ84ciatrFNZXrlD+p584P3w4CQUKGCyurKbiSp/cGJe/v/8xKaVvmho/+40tow+gHbDwue0AYO5LbRwAy+TnfYHdadm3j4+PzKjg4OAMfzYr5fa4tl7cKpsubyrjE+N12zsu7zBaXHfv3pWjR4+Wdnb5palpcVmyZJy8c0dKeeqUlEJIOXq0UeLKKiqu9MmNcQFHZRqP3/q4BHQTKPXcdsnk155PMvellM9uwy0EfPTQr5INNXVpytb3t+qGlH6x7wvG7B6je//ek3sGjadw4cJMnjyZ69ev8dNPG7l715yOHeNpNWYsIXXrwqxZcP++QWNSlOxCHwngCOAihCgnhLAAOgEvLMklhCj+3GYL4Jwe+lVygB0BO1jZdiUAsQmxVJxTkdG7Rhs8Dnt7e7p1q8H330Nw8DX27DlNVLt28PgxDydPJiIiwuAxKYqxZToBSCkTgIHAH2gH9rVSyjNCiIlCiBbJzQYJIc4IIU4Ag4Dume1XyRmszKwoX7A8AIkykbF1xtLMpRmg1SdotqIZobdDDRZP9+7w4YcViIi4QEzRftChA1/Nm0epkiXp168fx19bXkxRchczfexESrkV2PrSa+Oeez4SGKmPvpScy9rcmsE1B+u2r0Rc4dy9c1iZWQFw6r9TnPjvBO3c2uleywrffguhoSb07AlvrRpPRwcHbkVGsmTJEr7//nv8/PwIDAykU6dO2fIGoaLoi5oJrBhN7dK1+WfQP7gWdgVg5amV9Pm1j65GwaX7l3gU/0jv/Vpawvr1YG0NzYdVptzUeSxZvpxbt24xe/Zsnjx5Qp8+fXBycqJv377s378/26ySqij6pBKAYlTPlnUAmNJgCqF9Q7GztAPg498/ZuBx3WhivU42K1FCWzAuLAy6dYOkTZspOHkyH330EadOnWL//v20bt2a5cuXU7t2bT777DO99a0o2YVKAEq2YSJMqFS4km57ov9E+jv3B7Thym/9+Bb9f+uvez+zv5XXrQtffw2bNsG+eSe05SH270cIQa1atQgKCuL27dssXbqULl26AHDkyBH8/Py4cuVKpvpWlOxAJQAl2/J18qWGQw1Au4E80G+g7gZydFw0TjOcWHZiWab6+Phj6NQJmu8aQoyDk/bCc9Xl7ezsCAgIwNvbW+s3OhozMzOKJBeYWbNmDVOnTuXChQuZikNRjEElACVHMDMxY3DNwfyv4v8AiIqNommFproRRmfunKHK/CocCj/0ut28QghYuBDKVrFhYMw3cOwY/PBDqu39/f05ePCg7uZwcHAwo0aNwtXVlSpVqjBmzBgOHjxI4nNJRFGyK5UAlBypuF1xFrVcxNul3wa0Ivcl7EpQ3E6bcvLzuZ+pvrA64Y/C37gvGxtt5dD1ph05ZNcAOWoU3L2bpji+//57bty4wZw5c3B0dGTq1KnUqlWLIkWK0KlTJ5YsWcLt27cz/gdVlCykEoCSK/g4+bA9YDulC5QGtDMGWwtbHG209Yim7ZtG9YXViUuMA9D9fKZCBVi+QtA1ah4LPWYjHQqnue+SJUsycOBAdu/ezb1791izZg2tWrVi79699OjRg4ED//9G9rZt23j48GFm/7iKohcqASi50nuV3mNX112Ym5oDUMKuBFWKVMHC1AKAnr/0pMbCGrr2VyOu0rBJDJ3HVyJwX1d++FGkUFT4zQoWLEiHDh1YtGgRN2/eJDQ0lLFjxwJw/fp1mjVrxrJl2n2Le/fusWnTJu6rpSgUI9HLRDBFye4CvAII8ArQbTd2bkzVYlV12x3WdcDO0o4d43Zx5Aismz2Yt5bsoNruw9qEgQwQQuDl5aXbLlasGHv27KFChQoA7NixQze6qEKFClSvXl338PLywtLSMkP9KkpaqQSg5Eldvbq+sD2u7jjMTMwwMYFlyyTFv17IolPRVBk8GpsfZtL/t/40d2lO84rNgYwNQbWwsKBOnTq67bZt27Jv3z727dvHoUOHCA4OZsWKFbq23t7e+Pr6Mm7cOBwdHZFSvjBvQlEyS10CUhTgfxX/R5MKTQAoWBB+aXGaUgc+IN+Ps4jc8QdbLm7h/L3zgDYEtcWBFiw6rhWnj0uM47eLv6V7pVMLCwvefvttPvvsMzZu3MjNmze5ceMG69ev5+OPP8bKyorly5frzgQmTZqEm5sbCQkJAJw5c4ZLly7pthUlvdQZgKK8RAhBkxpliPzyO670PkCB9v25fvMcSdb5AG3EUcOiDXEp5ALAxfsX+d+q/7GyzUo6e3Tmn4f/MHT7UMbWGctbxd8iOi6a29G3KWtfVrdMdmpKlixJyZIladu2LcALv/VXqlSJ+vXrY2am7WPIkCFs374dc3NzKlSogKurK/ny5ePatWu4urri4uJCwYIF1VmDkip1BqAoqejYy5bf2i6mYORV9o36DROh/XcpbF2Yj10+5p0y7wDgXNCZ/T3307B8QwDuP7nPxfsXSUzS5gL8ee1PXOa48Ff4XwAcu3WMPpv76Iao3ntyj3N3z+nWQHre8wfvjh07MnfuXN32tGnTWLx4MUOGDKFSpUqcO3eONWvW0L17d2rUqIGDgwM1avz/je45c+awfv163faTJ0/08j0pOZc6A1CU1+i3qg4Btc7zy0IX/uoNHh6vtslnno9apWrptv1K+HGm/xndtqejJ4tbLsa9qDugjTj69eKvjKurLZi78dxG+m7py/VPrlOqQCk2nN3AwuMLWdV2FfZW9pz67xQX7l+gZaWWmJuaE58Yj5mJGVWrVqVq1aovxLJz505Kly7NhQsXuHTpEtbP3cCeP38+fn5+tGvXDikljo6OmJiYUKJECZycnFL8Wa5cOYoWLarPr1TJRlQCUJTXMDeHGb+6EPIWTGx6kIXbSlDAo3S69lEifwm6e3fXbbd1a0tbt7a67cbOjVnRZoVuEltsYiz3n9zH1kKbbbzu7Dqm/DmFuDHa3IUJIROYdWgWj0Y+wkSYsPTEUg7cOMD3//seMzMz7uW7R5xzHEPeGwJoZySmJqacPXuW2FitMF9iYiJjx44lPDycW7ducfPmTUJCQrh169YL9xQCAwP54YcfSExMpHLlygwbNozAwEAiIiKYOnUqRYoUoUiRIhQuXJjChQtjb2+ve6hRTNmfXhKAEKIJMAswRasPPO2l9y2BpWilIO8DHaWUV/XRt6JktWLFYGNQFC6N/8fNupWxux2i1/2XtS9LWfuyuu0uHl3o4tFFtz2k5hA6VumIqYkpAPXK1sPSzFJ3SerKwyscvXVU137B3wvY9c8uXZL55I9P2Hd9H1c+voKVlRWDfx/M9UfX2fDpBgC+/etbomKjGFt3LElJSSw9spT79+/jKl0pVqwYp++cJjEuET8/PxwdHYmOi+bGrRt8++23xMW9OKHueVZWVnz11VcMHDiQO3fu0KRJE0aNGkWdOnW4evUqK1aswNbWFhsbm1R/Ojo6YmWVdbUh8rpMJwAhhCkwD2gEhANHhBCbpZRnn2vWC3gopawghOgETAc6ZrZvRTGUGo3s2N5jLo0Xd+HPhuNhYiOD9W1vZY+9lb1uu5FzIxo5/3//4+uNZ3y98brtbxp/Q0TM/5e47OHdgybOTXTbxWyLvbD/0Nuh3H+qTUYzMTFh8bnFAAztPhSAtxe9TT6zfOxcsRMAvwV+FLEuQkxMDFFRUTRe2ZiiZkXpW7wvkZGRzLw+k/zx+fGN9sXLy4vh24cTcS+Chw8fkpCQwPR903l87TGTxkzSAngLuAdcTw7IFXgA3IENGzbg6OvItVPXGPDBALZv3459eXsOBR9i5tSZWFpZYmpnio25DTYWNlhaWWKezxxrS2usrawZNGgQZcqU4fTp0+zevZsePXpgZ2fH6dOnOXv2LBcuXCA6Ohpzc3PMzMwwNzd/4VG5cmUsLCyIjIwkOjoaJycnhBDExcUhpcTc3BwTk5x7K1UfZwDVgDAp5T8AQojVQEvg+QTQEpiQ/Hw9MFcIIaSqsqHkII1+6syeP3fh/uf3jJjdgZMnjR3Rqy5dKsHJk4WAQs+9Wh+A2SHaVj4+oxww+6K2/RZLtPdna9vN5a8kyDjddvWEb1543zV2EBaRNsyZI4D8FHraDCuTolx+os2RQG7DyqISJUqM4fhxWPdoOPbRlen5/iFOn4apEQG4m7fgm29iiYt7zITYsngmtqTOk0HExkbzncO7uEa+i8ftlpw86cOkU87USOiFp2dXtmwtwkTK4RsVQGysE4+ePiascTAFjpTA+mBBYsVjHgRewWK3DeKQINa0Od/ZVMD73w78/cNKbkTU5jsa43ypDieX/wwF0ArU/gGcBwqi/Wq6A7gMgz7fyyaToRQ76cbhdUGM+OY0a59+iM1+B05t+wWKAk1A7DTF9I4ZFBMk1YvHfI81Zg8s+PDzP9gROwWbg4UJP3Gc9p/MJyRmBmK3KTfPHCfeMYanVR5S4GgJzGPyEef4hMcu92hpN4969fT2zyJVIrPHYCFEO6CJlLJ38nYAUF1KOfC5NqeT24Qnb19ObvPKwGkhRCAQCODo6OizevXqDMUVHR2dLcv5qbjSJ7vFFf8onvnjC7MptOqbGyuvIYHkEU757kOSOcTm17aLnIGYghDlpG2X3wkRZeCBC4hE8FgF/3lqD5N48FkA4TXg37fA7CnUmAX/NIBbfmARBe9MhQtNIdwN8knwnwCnmsGNsmD7LzSaAUdbwI3KUCAcms6BA63gugsUdIVmQ2FPBwhPgMK1ofkA2NkCbj6GYjeg2W/we0O4VRRKhEOzXfCrP9x2gNIdtfYb2sOdB1C+OTQbAKubw73rUPEmNDsBS6vDAytwuwVNz2O75nd+XZax4bv+/v7HpJS+aWmb7RLA83x9feXRo0df1yRVISEh1DNECk0nFVf6ZNe4tmzZR61atY0dxiv27dtH7doqrrTKrnHt37+P997LWFxCiDQnAH1cAroJlHpuu2Tyaym1CRdCmKGdeKkVsLKZ+Ph4wsPDiYmJMXYoOgUKFODcuXPGDuMVJUrY8N9/2S+uUqVUXOmRXeMqWdLmjf/uraysKFmyJObm5hnuRx8J4AjgIoQoh3ag7wR0eanNZqAbcBBoB+xW1/+zn/DwcOzs7Chbtmy2mT0aFRWFnZ2dscN4hYorfVRc6fOmuKSU3L9/n/DwcMqVK5fhfjJ9+1pKmQAMRLuNcg5YK6U8I4SYKIRokdzsJ8BBCBEGDAFGZLZfRf9iYmJwcHDINgd/RVFSJoTAwcEh02frepkHIKXcCmx96bVxzz2PAdrroy8la6mDv6LkDPr4v5pzB7AqiqIomaISgJJtlS1blrNnz77wmq+vLyEhIQCMGzeONWvWGCweIQTR0dEG609RsppaC0jJsSZOnGjsEBQlR1NnAEqO1b17d93yyBMmTKBz5840a9YMV1dXmjdvrlvuODIykrZt2+Lq6kqDBg3o2rUrw4YNAyAuLo7hw4dTrVo1vLy8CAgISPdv+b///jtVq1bF09OTBg0aEBYWBsCFCxeoWbMmXl5euLu78/XXXwPwyy+/4OHhgbe3N+7u7rozGkUxNHUGoKTok08gNDRr9u3tDd9+m7a2AQEBLyxpfPHixVTbHj16lCNHjlCgQAHeffddVqxYQZ8+fZg4cSIFCxbk/PnzPHjwAB8fH13BlS+//JICBQpw+PBhAD777DOmTp3KlClT0hTfnTt3CAgIYM+ePbi5ufHTTz/x/vvvc+jQIebPn0+LFi0YOXIkAA8fPgS0S1c//vgjNWvWJDExkcePH6fty1AUPVMJQMnWli1bRvXq1XXbvr6pT3B89913sbfXFk2rXr06ly9fBiA4OJg5c+YAUKhQIVq1aqX7zObNm3n06JGuUEpsbOwLhdzf5NChQ3h5eeHm5gZAjx496N+/P1FRUdSpU4dPP/2UJ0+e4O/vj7+/PwD169dn8ODBtG3blqZNm+Lu7p7m/hRFn1QCUFKU1t/Qs5Pnlw02NTXl6dOnb/yMlJL58+dTv359vcfTtm1batasyfbt25k2bRqLFi1i+fLlzJw5k1OnTrF7927at2/PkCFD6NOnj977V5Q3UfcAlFyvXr16LF26FICIiAh++eUX3XstWrRgxowZumQRFRWVrqUnatSowYkTJzh/XisYHxQURNWqVbGzsyMsLIxixYrRvXt3xo8fr7vMdOHCBTw8PPj444/54IMPOHLkiL7+qIqSLuoMQMn1xo0bR48ePXB1daV48eL4+vpSoEABAEaMGMGECRPw8/PDxMQEIQTjx4+ncuXKKe6rUqVKCCGQUmJra8uFCxdYtmwZXbp0ISEhgSJFirB8+XIA1q5dy4oVK7CwsEAIwaxZs3R9Xrp0CTMzM+zt7fnpp58M80UoyksyvRpoVlKrgRpOSEgIjo6OqR74jEUfa7XEx8eTmJiIlZUVjx49onbt2syYMYOGDRsaNa6soOJKn5we17lz5175P2vo1UAVJVt7+PAhTZs2JTExkZiYGLp06ZKpg7+i5BYqASi5XtGiRTl27Jixw1CUbEfdBFYURcmjVAJQFEXJo1QCUBRFyaMylQCEEIWEEDuEEJeSfxZMpV2iECI0+bE5M30qiqIo+pHZM4ARwC4ppQuwi9QrfT2VUnonP1qk0kZRFEUxoMwmgJZAUPLzIKDVa9oqSrrEx8czefJkKlasiKenJ1WrVmXo0KHEx8cbNI6yZcvi6uqKt7e37nHt2rU3fm7ChAnExcUZIMKUXb16lcKFCxutfyX7y+wwUEcp5b/Jz28Djqm0sxJCHAUSgGlSyk2p7VAIEQgEAjg6OmZ4qdzo6Ohsucxudo6rQIECREVFGTsUnd69e/P06VNCQkKws7MjISGBZcuWce/ePWxtbV9om5iYiKmpaZbEIaUkKChIt+Dbs/7e9F19/vnn9O3b95VYARISEjAz0/8o7Ofjio6ORkqZLf5O0/J9GUNOjysmJiZzxxMp5WsfwE7gdAqPlkDES20fprKPEsk/ywNXAec39SulxMfHR2ZUcHBwhj+blbJzXGfPnn3xxbp1X33Mm6e99/hxyu8vXqy9f/duyu+vXp2meC5evCitra3ltWvXUnx/8eLFskGDBrJVq1aySpUq8vjx4/LSpUuyfv360sPDQ1atWlVu27YtOdTHsl27drJy5crS09NTtm/fXkop5fnz52WNGjWkp6enrFKlivzqq69S7KtMmTLy1KlTL7z26NEjKaWUgJwyZYr09fWV5cqVk+vXr5dSStm/f38JSA8PD+nl5SUfPnwou3XrJnv16iVr164tvby8pJRSbtu2TXp7e0sPDw9Zv359eenSJSml9vfh6ekpAwICpJubm/Tz85NnzpyRUkrZrFkzuXbtWl0sGzZskI0aNXohLimlvHLlinRwcEjxzxQUFCTd3d2lh4eHbNWqlfzvv/+klFLu379fVq1aVXp5eUk3Nze5cuVKKaWUP/zwg3R1dZVeXl7Sw8NDnjt3LsX9pub5uLKTnB7XK/9npZTAUZmG46uU8s1nAFLKVKdMCiH+E0IUl1L+K4QoDtxJZR83k3/+I4QIAaoCl9+YnZQ86/jx47i4uFCwYIrjCgD466+/OHHiBM7OzoC2BHRgYCC9evXi7Nmz1KlTh3PnzrFv3z4ePXqkKy/5bF3+1NbrT0m7du10q42amZkRHBysey9//vwcOXKE/fv306FDB9q2bcu8efOYP38+Bw4ceOEMIDQ0lD179mBjY/PaWgIAJ0+eZPbs2SxdupSgoCC6du3K0aNH+eijj5g+fTrt27cHYN68eQwaNCjN3+3p06cZMWIEx44do3jx4owdO5aPPvqINWvWMH36dIYPH07nzp2RUhIZGQnA8OHDOX/+PMWLFyc2NpbExMQ096dkY2nNFCk9gK+AEcnPRwBfptCmIGCZ/LwwcAlwS8v+1RmA4aR4BmBEa9askV5eXqn+JrR48WL57rvv6rYfPXokLSwsZGJiou61Bg0ayM2bN8vLly/LUqVKyf79+8u1a9fKx48fSymlXL9+vSxfvrwcM2aM3LVrl0xKSkqxrzedAdy9e1dKKWVCQoIE5NOnT3XvRUVF6T7TrVs3OXXqVN325s2bZYMGDXTbiYmJ0sLCQj569EgGBwfLChUqvPJeZGSkTEpKkpUqVZJnz56VZ8+elWXKlJEJCQkvxCVl6mcAs2fPlr169dJt37hxQxYqVEhKKeXMmTOlm5ubnDRpkvzrr790bVq1aiUbNWokZ8+eLS9fvpzi9/Q6Of03bUMz1BlAZm8CTwMaCSEuAQ2TtxFC+AohFia3qQwcFUKcAILR7gGcTXFvipKsatWqXLp06bW/lad0bT0l5cuX58yZMzRq1IidO3fi5eVFTEwMbdu25c8//8TZ2Zlp06YREBCQoVifnRk8uweRkJCQ6ZhfRwjBwIEDmT9/PvPnz6dv3756u//xySefsHnzZooUKcJHH33EmDFjANi4cSOTJ0/m8ePH+Pv7s23bNr30pxhXpu5CSSnvAw1SeP0o0Dv5+QHAIzP9KHmPi4sLLVq04JNPPiEoKAg7OzsSExNZvHgxnTp1eqW9nZ0d3t7eBAUF0aNHD86dO8eJEyeoUaMG4eHhukpgjRs3xsnJiQcPHvDkyRPKly9P9+7dcXFxoUePHnr9M9jZ2REZGZnqQb9GjRr07NmT8+fP4+rq+kItAYDLly/z559/8s4777By5Uo8PDzInz8/AN26dcPNzY3Y2FjOnDmTrrj8/f2ZOnUqt2/fplixYixYsIBGjRoBWsnNihUr4uzsjK2tLUFBQSQkJHDt2jWqVatGtWrVuHz5MsePH6dp06aZ+HaU7EAtBqdkW0FBQYwePRofHx8sLCxISkqiWbNmWFpapth+xYoV9O3bl5kzZ2JmZsayZcsoUqQI27ZtY8QIbYpKYmIiI0eOxMnJiS+++CLF9fpT8vw9AIBZs2ZRt27d18Y/dOhQ6tevT758+VIcqVGkSJFUawkAeHh4sHDhQvr164e1tbWuqA1oyaVJkyY8ffqUIkWKpBrDw4cPKVmypG7b1dWVnTt3Mm3aNBo1aoQQgvLly/PDDz8AMHv2bIKDg7GwsMDS0pI5c+aQmJhI9+7diYiIwMTEhFKlSjFt2rTX/tmVHCKt14qM8VD3AAwnu90DeCanX6PNqODgYPm6f//x8fGycuXK8vDhwwaNK6NUXOmTU+4BKIpiYJs3b8bZ2ZnGjRvj5+dn7HCUHExdAlKUbKhevXqkVg2vRYsWtGihVlRRMk+dASiKouRRKgEoiqLkUSoBKIqi5FEqASiKouRRKgEo2ZZaDjpzpJSMGzeOKlWq4OXlhZubGzNmzADg6NGjvP/++waLZfLkyaxZsybF9yZMmMCwYcNSfK9s2bKcPn06K0PL09QoICXb6tGjB1FRURw7dky3HPSiRYuIjY3F3Nz8hbZZuRw0wPr163F3d9dtp2Wp3s8//5xhw4ZhYWHxyntZtRz089avX8/u3bs5duwYVlZWxMbGcvmytgajr68vK1asyNL+nzdmzBjdDGcl+1BnAEq2dOnSJX7++WfmzJmjO3CYmZkRGBiIra0tS5YsoWHDhrRu3Rp3d3dOnTpFWFgYDRo0wNPTk7feeovff/8dgCdPntC+fXvc3Nzw8vKiQ4cOAFy4cIGaNWvi5eWFu7s7X3/9dbrjFELwxRdf4OfnR/ny5dmwYQMAAwYMAKBWrVp4e3sTERFB9+7d6d27N++88w6+vr4A/P7771StWhVPT08aNGhAWFgYACEhIXh5edG1a1eqVKlCtWrVdKuZNm/enHXr1uli2LhxI40bN34ltvDwcAoXLqybOW1paamraRASEqKLAWDu3Lm4uLjg5+fH+PHjdYVknhWVGTlyJFWrVsXV1ZVjx47Rp08fPD09qV69Ordv3wa0JDxs2DDc3d1xd3dn2LBhulVDP/zwQ+bOnQtAZGQk7dq1w9XVlXr16umSUnocOXKEmjVr4unpSc2aNTly5AgAd+7coWHDhnh4eODh4cHgwYMBOHDgAG+99Rbe3t5UqVKFVatWpbvPXCmtM8aM8VAzgQ0npZnAdevWfePj+TX069atKxcn1wO4e/duqp9Ji7SsBmpjYyPDwsJ0r1WrVk0uXLhQSinlmTNnpIODg7xz5yzFxGQAABafSURBVI7cuHGjbNy4sa7dgwcPpJRSDho0SH7xxRevvP6yMmXKyEqVKkkvLy/p5eUlfXx8XlgNdM6cOVJKKfft2yednJx0nyOF1UB9fHxkdHS0lFLK//77TxYuXFi3zv/ChQtltWrVpJTa3wcgQ0JCpJRSLlmyRDczeNu2bbJevXq6/davX19u2rRJSvniDNJbt27JihUrSmdnZ9m9e3e5bNkyGR8fr9v/s/2dOHFCOjk5yTt37ui+l2eriF65ckUCcsuWLVJKKb/88ktZoEABefz4cSmllP369ZOjR4+WUko5f/582aBBAxkbGytjY2Nl/fr15fz586WUUnbp0kX3PQ0ZMkT26NFDSqn9OylVqpQcOnRoqt/9yyuxxsbGylKlSsmdO3dKKaXcsWOHLFWqlIyNjZUzZsyQgYGBurbP/k5btGihq22QlJQkHz58+Mr3lZ2omcCK8ga1a9fW1QKIiooiNDRUt6Cbm5sb3t7e/PXXX3h5eXHu3DkGDBjAunXrdL8R16lTh4ULFzJ27Fh2796Nvb19qn2tX7+e0NBQQkNDX5mg9Wxxuho1anDr1i1iYmJS3U+7du2wsbEB4NChQ7pr86Bd8goNDdVdXqpQoYJuvaGAgABOnTrF/7V37lFRnecefl5RRGPQNuSAFj0NLVovXLwhtyB4Id6i0cRiqgaD0bQcNHalJuaiidGkpO1KTjSupVYTjKYxNlSbGlBBHQt6VFRQFKL1kniJRsUQBDUqfOePgZ0ZnQFGLjPK96w1C/beH/v77Rlmv/u7/d6SkhIeeeQRzp49S2FhIYWFhRw7dowRI0bcVlf79u05dOgQH374IZ07d+bNN9+0Wc5kMjFs2DDDUyghIcHqeJs2bRg+fDgAvXr1wtfXl+DgYAB69+5ttFoyMzOZNGkS7u7uuLu78/TTT5OZmXlbfVu3bmXy5MkAeHl5MWbMGLvvly0OHz6Mu7s7AweafSgHDRqEu7s7hw8fJjQ0lPT0dGbOnMn69esNI76YmBjmz5/P/Pnz2b17d7WfdVNCjwFo7OJoqjnL8l5eXnVKVWdpB22v79hRO+jNmzeTnp7Oyy+/TH5+Po8//jhhYWFs2rSJ5ORkPvjgAyszttriTDtooFo76ObNm/Pwww/z8MMPk5CQgI+PD5cuXXKoPkvzPTc3NytTPDc3t2qvt7EJCwsjNzeXjIwMVq5cSXJyMtnZ2cyYMYNHH32UzMxMpk2bRmxsLPPnz3e2XKejWwAal8TSDrrqibi8vJxly5ZRWlp6W3lLO2jgNjtoNzc3HnvsMd59910uXLjApUuXOHr0KD4+PkyaNInXXnuN3bt31+s1VNlB2yM0NJT9+/fz5ZdfAti1gwZs2kGvW7eOTz/9lGeeecbm+ffu3ctXX31lbO/bt4+f/OQntz399u/fn/T0dC5evGjouBMGDRrEihUruHHjBjdu3GDFihWGzbQlAwYM4MMPPwSgqKiItWvXOlRPly5duH79upGVbcuWLdy4cYMuXbpw4sQJPD09GTduHO+88w579+6loqKCI0eO8Itf/IJnn32W5557rt4/67uVOrUARGQs8DrmpC8hypwHwFa5IcB7gBuwTCmlvWQ1NaLtoOtmB33x4kUSExMpKSmhZcuWtG7dmnXr1tGsmfVzX1BQEC+88AJhYWF4enoycOBA2rZtW+212WLq1KkcPXqUnj17AvDII48wZcqU28rNnj2bhIQEfvWrX+Hj40NUVFS15x00aJDVjKn8/HxSU1OZPn06ZWVl3HfffXz22We4u7tjMpl45513cHNzo6KigsWLF9OsWTObNtca6pwSsivQBTABfeyUccOc/9cPcAf2o1NCuhzaDtox7jU7aMu/e+2119T48ePv6Dy1Ob8rcbfrcuogsFKqUCl1uIZiIcBRpdRxpdR1YDUwqi71ajRNmYawg541axbBwcF069aNPXv28Kc//alezqtxbRpjEPhnwCmL7dNAv0aoV6O5a2lsO+hFixbV6/k0dwc1BgARyQR8bBx6RSn1z/oWJCJTgakA3t7edzyTpLS0tE6zUBoKV9bVtm3bWq1wbUzKy8tdThNoXY6idTlGbXVdu3atTveTGgOAUmrQHZ/dzBmgo8W2b+U+e/UtBZYC9OnTR0VHR99RpSaTiTv924bElXV5eHi43HL9y5cvu5wm0LocRetyjNrq8vDwMAbd74TGmAaaA/iLyEMi4g6MAz5vhHo1Go1GUw11CgAiMlpETgNhwBcisrFyfwcRSQNQSt0EkoCNQCGwRil1qG6yNRqNRlNX6jQIrJRaC9y2ikMp9Q0wzGI7DUirS10ajUajqV/0SmCNS/Lqq6/yu9/9zthev349IsKhQz82HkeMGMHy5curPU9tfe+rXC9tUVxcXKdpkdV52h85coTRo0fj5+dHnz59iIiIYN26dXdc151gMplo3bq1Vb6Dp556qsa/y8vLY82aNY2g0D7V5RJoKBYsWFAr59hvv/2W2NhYOnfuTFBQELt27bJZLiUlhXbt2hnv/ejRo41jL774In/729/qTfut6ACgcUliYmKsZjds27aNfv36GfvKy8vJzs6ucUC9Pnzv6xoA7HH27FmioqIYM2YMx48fZ8+ePaSmplJSUnJb2Yb22+nWrZthdpeXl2e16tgeNQUAV/IIqi+uXLnCe++9Z9h9V8dLL71EVFQUR44cYdGiRUyYMKFqcextDBo0yHjvLa0x/vCHPzB37lwqKirq7Ros0QFAY5folGhS8lIAuFF+g+iUaFYdMFsVXLlxheiUaD49aM7y9P2174lOieYfhf8A4OKVi0SnRPOvw/8C4FzpOaJTotlwdEOt6g4PD+fEiROcP38eMAeA2bNnGwEgNzcXT09Pww00LS2NiIgIevfuTVhYGDt37gRq73tfxSuvvELPnj3p0qUL2dnZgNnbv7i4mODgYMLDw83Xc+4cTzzxBCEhIQQEBPDWW28Z58jKyjL86JOSkux+6RctWkRMTAwTJ0409vn4+BhP39HR0cyYMYPQ0FBj3v9HH31EQEAAgYGBjB492nh/qvzuIyIirPzuly5dSteuXQkODiYwMNDwHaotKSkpxMbGEhcXR/fu3YmIiODcuXMUFRUxZ84cMjMzCQ4OZvr06YDZqO7111+nb9++zJ0718gR0K9fv9tyBEyaNIkpU6YQHh5O586dmTJlCtevX+ebb76hffv2Vq6qI0eOdOhJuLS0lKefftrITWAZwOfOnWtkeIuMjKS4uNhuzohbSU1NJSoqilatWgHmTGdVbqZXrlwhICCAtDRzb/eaNWv47W9/C5ida1u2bGl3bYc9HnzwQfz8/Ni8ebNDf1dbdADQuCStWrUiJCSErKwsLl++TFlZGUOGDCEvLw+wnk577Ngx5s2bR3p6Onv37mXZsmU2v8AHDhzgj3/8Izt27CAnJ4fi4mKr40VFRYab5Jw5c3jxxRcB8426Xbt25OXlsWPHDsDswDl9+nR2797N3r17SU9PJyMjgx9++IFx48axcOFC8vPziYqK4uTJkzavcd++ffTrV/2ayOPHj5OdnU1aWhoHDx5k1qxZbNq0iQMHDtCjRw+mTZsGwNtvv83MmTPZvn07Bw8eZOjQoQDMnDmTLVu2kJeXR05ODp06dbJZT0FBgVUX0BtvvGEcy8nJ4S9/+QuHDh2iW7duLFy4kAceeIA33njDeHJdsGCB1WeXk5PDvHnzWLp0KXl5eWRlZbFv3z5yc3NZunSpUXbXrl1s2rSJgoICvv76a5YuXUqHDh3o37+/kULyq6++Ys+ePTzxxBPVvleWzJs3j4qKCvLz89mxYwcrVqwgPT2dS5cu8e6775Kbm0teXh7p6em0adOGjRs3UlJSQkFBAfv372fJkiU2z2symaw+s5dffpnS0lIWLlxIUlISQ4cOZdiwYRQVFaGUsnrA6NSpE6dOnbJ1WrZt20ZwcDBRUVF88cUXVsfCwsIaLABoO2iNXUyTTMbvLdxaWG23btHaarutR1urba/WXlbbPm18rLZrQ3R0NNnZ2Xh7exMZGYmbmxv+/v4cOnQIk8nE448/DsDGjRs5duyYlanYzZs3+fbbb62vx4bvvWX3UJs2bQy//NDQUJ5//nmbusrKysjKyjKeesE8b7uwsBBvb29at25tBKdf//rXTJ061aHrtuQ3v/mNYYS2detWhg0bRvv27QFzEAoKCgJ+9LsvKChgxIgRxk1qwIABxMfH8+ijjzJ8+HD8/Pxs1lNlAWGLiIgIOnY0L+UJDQ0lIyOjWs3x8fHG77ZyBKxdu9YY34mLizMssuPj40lNTSUpKYnp06fz+9//nvj4eBYvXkxCQoLN1Jr2yMzM5L333kNE8PT05MknnyQzM5PY2Fh++ctf8tRTTxEbG0t0dDTNmze3yhkRHR1t5D+4ldOnT1vlVGjWrBmrVq0iODiYTp06Ga1GRxgxYgRxcXG0atWK3Nxchg4dyvr1642Wq4+PD//+978dPm9t0C0AjcsSExNDVlYW27ZtM5w3o6Ki2Lx5s1X/v1LKaB1Uvb755hu8vb0dqu9W33t7fdgVFRWICDk5OUZ9x44dswoIloiIzf29evWq0Za4tvkDZsyYweeff46XlxfTpk3j1VdfBczpIufPn09ZWRkxMTGkp6fX6nyWOOr/Xx85D8LDwykvL2f79u2kpKTw7LPP1vmcYNa/c+dOkpKSOH36NFFRURw4cMDIGTF48GAyMzMJCgqymdinVatWt+0/ceIEzZo1o7i4mKtXrwLwwAMPABgW2wAnT540AqklXl5eRpdSz549iYiIsArG165dM47XNzoAaFyWsLAwTp48SWpqqnGzj4qK4v3336ddu3Y89NBDAMTGxrJhwwarGUJVOWItuVPfe09PT65cuWLc+O6//37Cw8NJTv7R1fzUqVOcO3eOLl26cPXqVcPH/7PPPrutq6mKxMRENm/ebNW3ff78ebsDsDExMaSlpRk5eP/6178afvtVfvcJCQmG3/3Nmzc5fvw4ISEhzJo1i9jYWHJzc2t1zbXB09Oz2nwHUHOOgL///e+UlZVx8+ZNVq5cyYABA4xj06ZNY9y4cYSHh9u8cdZU7/Lly1FKcfnyZVavXs3gwYO5fPkyFy5coH///sydO5euXbty8OBBuzkjbiUgIIDDh3/0v/zuu+8YP348q1evJi4uzsr+euzYsSxevBiA7Oxsrl69Su/evW8755kzPxojfP311+zcuZMePXoY+woLC42WXn2jA4DGZfHw8DCawR06dACgb9++nDlzxmr2j7+/P6tWrWLy5MkEBQXRtWtXm324lr73vXv3pnnz5rXyvf/pT3/K+PHjCQgIMAaBly1bRkFBgTHYGxcXR3FxMS1btuSTTz4hMTGRwMBATCaT3X73Dh06sG3bNj799FP8/PwICAhg1KhRdjX16NGD5ORkBg8eTGBgIPv37zdyGCxYsIDu3bsTGRnJwoULefPNNykvL2fSpEkEBAQQFBTE2bNn7T5J3zoGMGzYMJvlLBk4cCBlZWUEBQXZbf1MnTqVwMBAIiMj6dmzJ4GBgVY3yb59+xIbG0vXrl3p2LGjVXfZuHHj+O6770hMTKxWx5IlS/D19TVeS5YsYfbs2SilCAgIICwsjIkTJzJkyBC+//57HnvsMQIDA+nRowfe3t6MGTOG/Px8wsLCCAoKIiQkxMgZcStjxoxh48aNxnZCQgIJCQlERkYyZ84czp07Z9z0k5OTMZlM+Pv7k5iYyMqVK41cDM888wyff242RFi0aBHdu3cnKCiIkSNH8tZbbxk3fKUUW7ZsYdSoBjJQrq1vtDNeOh9A49FU8gHUl+/93e4j39jY0hUfH28kirdFVlaW6t69u6qoqGhUXTURGxur9u3b1wBqfqRK14YNG9SECRPslqtrPgA9CKxpUsyaNYvt27dz/fp1/Pz8rGakaFyHyZMnk5GRwUcffWR3DMVZvP/++/znP/+pkwlbbSkpKeHtt99usPPrAKBpUmjfe9chJSXF7rGaVng7E39/f/z9/RulrrFjxzbo+fUYgMYKZWfRkkajcS3q47uqA4DGwMPDw1jAotFoXBelFEVFRVZTdO8E3QWkMfD19eX06dNcuHDB2VIMrl27Vud/8oZA63IMrcsxaqPLw8MDX1/fOtWjA4DGoEWLFsbcelfBZDI1ymCbo2hdjqF1OUZj6dJdQBqNRtNEqWtGsLEickhEKkSkTzXlvhKRfBHJExHH7PA0Go1G0yDUtQvoIDAGsG2dZ02MUupizcU0Go1G0xjUNSVkIdg3u9JoNBqN69JYg8AK2CQiCliilLK7/FJEpgJVhiClInLYXtka8AJcscWhdTmG1uUYWpdj3Iu6/ru2BWsMACKSCfjYOPSKUuqftawnUil1RkT+C8gQkS+VUjYNriuDQ53X54vIHqWU3XEJZ6F1OYbW5Rhal2M0dV01BgCl1KC6VqKUOlP587yIrAVCgIbJcKDRaDSaWtHg00BF5D4Rub/qdyAW8+CxRqPRaJxIXaeBjhaR00AY8IWIbKzc30FE0iqLeQPZIrIf2A18oZSqXWbwuuGqNo9al2NoXY6hdTlGk9Yl2vdFo9FomiZ6JbBGo9E0UXQA0Gg0mibKPR0ARGSeiByotKDYJCK3J/l0AiLyZxH5slLbWhFp52xNUHtrj0bSMkREDovIURGZ5UwtlojIByJyXkRcaiKDiHQUka0iUlD5GT7nbE0AIuIhIrtFZH+lrrnO1lSFiLiJSK6IrHe2Fksa0zrnng4AwJ+VUoFKqWBgPTDH2YIqyQB6KKUCgSPAS07WU0WVtYdTp+iKiBuwCBgKdAOeFJFuztRkQQowxNkibHATeF4p1Q0IBf7HRd6zH4ABSqkgIBgYIiKhTtZUxXNAobNF2CFGKRXc0GsB7ukAoJQqsdi8D/OKZKejlNqklLpZubkTqJupdz2hlCpUSt3pyuv6JAQ4qpQ6rpS6DqwGRjlZEwCVCxgvOVvHrSilziql9lX+fhnzje1nzlUFlXnKSys3W1S+nP49FBFfYDiwzNlanMk9HQAARORNETkFjMd1WgCWJADpzhbhYvwMOGWxfRoXuJndLYjIz4GewC7nKjFT2dWSB5wHMpRSrqDrf4EXgApnC7FBlXXO3kprnAbjrg8AIpIpIgdtvEYBKKVeUUp1BD4GklxFV2WZVzA33T92JV2auxcRaQOkAjNuaQE7DaVUeWU3rC8QIiI9nKlHREYA55VSe52poxoilVK9MHeB/o+IRDVURXd9RjAHrCo+BtKA1xpQjkFNukRkEjACGKgacTFGfVh7NAJngI4W276V+zTVICItMN/8P1ZK/cPZem5FKVUsIlsxj6E4cxA9AhgpIsMAD8BTRFYppSY4UZNBY1rn3PUtgOoQEX+LzVHAl87SYomIDMHc/ByplLribD0uSA7gLyIPiYg7MA743MmaXBoxe7IvBwqVUu84W08VIvJg1Sw3EWkFDMbJ30Ol1EtKKV+l1M8x/29tcZWbf2Nb59zTAQBIruzeOID5jXSJqXHA+8D9mJ1R80RksbMFgX1rj8amcoA8CdiIeTBzjVLqkDO03IqIfAL8H9BFRE6LyGRna6okApgIDKj8n8qrfMJ1Nu2BrZXfwRzMYwAuNe3SxWhU6xxtBaHRaDRNlHu9BaDRaDQaO+gAoNFoNE0UHQA0Go2miaIDgEaj0TRRdADQaDSaJooOABqNRtNE0QFAo9Fomij/D5ZMqSmPBhxGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the output\n",
    "x_array = sess.run(x_vals)\n",
    "plt.plot(x_array, hinge_y_out, 'b-', label='Hinge Loss')\n",
    "plt.plot(x_array, xentropy_y_out, 'r--', label='Cross Entropy Loss')\n",
    "plt.plot(x_array, xentropy_sigmoid_y_out, 'k-.', label='Cross Entropy Sigmoid Loss')\n",
    "plt.plot(x_array, xentropy_weighted_y_out, 'g:', label='Weighted Cross Entropy Loss (x0.5)')\n",
    "plt.ylim(-1.5, 3)\n",
    "plt.legend(loc='lower right', prop={'size': 11})\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Softmax entropy and Sparse Entropy\n",
    "\n",
    "Since it is hard to graph mutliclass loss functions, we will show how to get the output instead"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.1601256]\n",
      "[0.00012564]\n"
     ]
    }
   ],
   "source": [
    "# Softmax entropy loss\n",
    "# L = -actual * (log(softmax(pred))) - (1-actual)(log(1-softmax(pred)))\n",
    "unscaled_logits = tf.constant([[1., -3., 10.]])\n",
    "target_dist = tf.constant([[0.1, 0.02, 0.88]])\n",
    "softmax_xentropy = tf.nn.softmax_cross_entropy_with_logits_v2(logits=unscaled_logits,\n",
    "                                                              labels=target_dist)\n",
    "print(sess.run(softmax_xentropy))\n",
    "\n",
    "# Sparse entropy loss\n",
    "# Use when classes and targets have to be mutually exclusive\n",
    "# L = sum( -actual * log(pred) )\n",
    "unscaled_logits = tf.constant([[1., -3., 10.]])\n",
    "sparse_target_dist = tf.constant([2])\n",
    "sparse_xentropy =  tf.nn.sparse_softmax_cross_entropy_with_logits(logits=unscaled_logits,\n",
    "                                                                  labels=sparse_target_dist)\n",
    "print(sess.run(sparse_xentropy))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}