{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Discriminative Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Preliminaries\n",
    "\n",
    "- Goal \n",
    "  - Introduction to discriminative classification models\n",
    "- Materials        \n",
    "  - Mandatory\n",
    "    - These lecture notes\n",
    "  - Optional\n",
    "    - Bishop pp. 203-206 \n",
    "    - [T. Minka (2005), Discriminative models, not discriminative training](./files/Minka-2005 -Discriminative-models-not-discriminative-training.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "###  Problem: difficult class-conditional data distribitions\n",
    "\n",
    "Our task will be the same as in the preceding class on (generative) classification. But this time, the class-conditional data distributions look very non-Gaussian, yet the linear discriminative boundary looks easy enough:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj4AAAG2CAYAAAB/OYyEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xt8FfWd//HPyaEJUCEucgtyi10U1FUErLhICXSL9YJUHmptRW1RgR+gUPbhIggkBgxaLMXLgkRd9CFqfbQGq48t29L1Al3qioCtVdC1XkCIJbZ6gkpDTeb3x+kkcyZzPzNnZs68no9HHuGcM2fmeybR7zvfa0pRFEUAAAASoCTsAgAAABQKwQcAACQGwQcAACQGwQcAACQGwQcAACQGwQcAACQGwQcAACQGwQcAACQGwQcAACQGwQcAACRGbILPF198IUuXLpXKykrp1q2bnHTSSVJbWyttbW1hFw0AAMREl7AL4NSdd94p999/vzzyyCNy2mmnySuvvCLf//73pby8XObPnx928QAAQAzEJvj89re/lalTp8pFF10kIiJDhw6VJ554Ql555ZWQSwYAAOIiNsHnvPPOk/vvv1/eeustOfnkk+V3v/ud/OY3v5G1a9eavqelpUVaWlraH7e1tclf/vIXOeGEEySVShWi2AAAIE+KosiRI0dkwIABUlKS5ygdJSba2tqUW265RUmlUkqXLl2UVCql1NXVWb6nurpaERG++OKLL7744qsIvg4cOJB3nkgpiqJIDPzkJz+Rm2++WVavXi2nnXaavPrqq7JgwQJZs2aNXHvttYbv0bf4ZDIZGTx4sBw4cEB69uxZqKIDAIA8NDc3y6BBg+STTz6R8vLyvM4Vm+AzaNAgueWWW2Tu3Lntz61cuVI2bdok+/btc3SO5uZmKS8vl0wmQ/ABACAm/Ky/YzOd/fPPP+/Ur5dOp5nODgAAHIvN4OYpU6bI7bffLoMHD5bTTjtN9uzZI2vWrJEZM2aEXTQAABATsenqOnLkiCxbtkw2b94shw8flgEDBsh3vvMdWb58uZSWljo6B11dAADEj5/1d2yCjx+c3DhFUeSLL76Q1tbWApeu+HzpS1+SdDoddjEAADHnZ/CJTVdXIRw7dkwaGxvl888/D7soRSGVSsnAgQPluOOOC7soAACICMGnXVtbm7z77ruSTqdlwIABUlpayiKHeVAURZqamuSDDz6QYcOG0fIDAIgEgs/fHTt2TNra2mTQoEHSvXv3sItTFPr06SPvvfee/O1vfyP4AAAiITbT2Qsl76Ww0Y4WMwBA1FDLAwCAxCD4AACAxCD45KmmRmTFCmfHrliRPR4AAISD4JOndFpk+XL78LNiRfY4xvgCABAegk+eli0Tqa21Dj9q6KmtzR4fV+vWrZPKykrp2rWrjB49WrZv3x52kQAAcIXp7D5Qw8zy5bmPRYon9Dz55JOyYMECWbdunYwbN042bNggF1xwgbzxxhsyePDgsIsHAIAjtPj4xKjlp1ChZ+DAgbJu3bqc53bs2CHdu3eX999/35drrFmzRq677jq5/vrrZcSIEbJ27VoZNGiQrF+/3pfzAwBQCAQfH2nDT1lZ4Vp6xo4dKzt37mx/rCiKLFiwQBYsWCBDhgzJObaurk6OO+44yy99F9axY8dk165dMnny5JznJ0+eLDt27AjugwEA4DO6uny2bJnIypUix46JlJYWpntr7Nix8vDDD7c/fvTRR2X//v2yePHiTsfOnj1brrjiCsvznXjiiTmPP/roI2ltbZV+/frlPN+vXz/58MMPvRccAIACI/j4bMWKjtBz7Fj2cSFafBYtWiSffvqplJSUyJIlS2TlypXSo0ePTsf26tVLevXq5ek6+pWYFUVhdWYAQKzQ1eUj7Zielhb72V5+GTNmjKTTadm9e7fccccdcsIJJ8iMGTMMj/XS1dW7d29Jp9OdWncOHz7cqRUIAIAoo8XHJ0YDma1me/mpa9eucuaZZ0pDQ4PU19fLs88+a7rnmJeurtLSUhk9erRs3bpVLr300vbnt27dKlOnTs3/AwAAUCAEHx9Yzd4qVPgZO3as3HPPPXLxxRfL17/+ddPjvHZ1LVy4UK6++moZM2aMnHvuuVJfXy/79++X2bNn51NsAAAKiuCTJydT1gsRfkaOHCldunSR1atX+39yEfn2t78tf/7zn6W2tlYaGxvl9NNPl1/84hedZo0BABBlBJ88tbY6m7Kuvt7aGkw5HnvsMZkzZ46ccsopwVxARObMmSNz5swJ7PwAAASN4JMnN5uO+t3S09bWJk1NTfLQQw/Jm2++KZs3b/b3AgAAFBmCT4xt27ZNJk2aJMOHD5eGhgYpLy8Pu0gAAEQawSfGqqqqpK2tLexiAAAQG6zjAwAAEoPgAwAAEoPgAwAAEoPgAwAAEoPgAwAAEoPgAwAAEoPgAwAAEoPgAwAAEoPgE4TGxuxeFo2NYZckVIcPi3zyibNjV6xwt/0HAABeEHyC0NgoctttiQ8+IiKZjMi6ddbHqDvcp9OFKRMAILkIPnBk27ZtMmXKFBkwYICkUil5+umnbd/Tt69IebnIPfdkw40RNfQ42eEeAIB8EXzgyGeffSZnnnmm3Hfffa7ed/zxIjfdlA03+vBD6AHgqyQOM0jiZ84TwacIDBw4UNbp+pN27Ngh3bt3l/fff9+Xa1xwwQWycuVKmTZtmuv3zpmTDTfa8EPocaemxrzVTI/xUgaoHJIh7GEGTn/P/Px9DPszxxDBJ1+NjSK7d3f+EjF+PoBfzrFjx8rOnTvbHyuKIgsWLJAFCxbIkCFDco6tq6uT4447zvJr+/btvpdx2bKO8FNWRuhxK502bjXTY7yUCSoHFILT3zN+H0PVJewCxN6GDdlfYCM33ND5uepq3/8cHzt2rDz88MPtjx999FHZv3+/LF68uNOxs2fPliuuuMLyfCeeeKKv5VMtWyaycqXIsWMipaWEHjfUe7V8ee5jLVrRCqyxMfvf/6xZIhUVYZcmnriHCAHBJ1+zZolccknuc7t3Z0PPAw+IjBqV+1oA/3GPHTtWFi1aJJ9++qmUlJTIkiVLZOXKldKjR49Ox/bq1Ut69erlexmcWLGiI/QcO5Z9TAXtnFX4iWzoKeaKTf2r/ZJLovHZ4nivo3YPkQgEn3xVVJj/BztqVOfgE4AxY8ZIOp2W3bt3y69//Ws54YQTZMaMGYbH1tXVSV1dneX5tmzZIuPHj/e1jPqKWX0sErGKOuKMwk9kQ49ItCu2OAYFK/p7HefP56TsjY2du4q0wwz0rP5fHRdJ/MwBIPgUga5du8qZZ54pDQ0NUl9fL88++6yUlBgP3wqjq2vdus4Vs5OuGxjT3ju167DgocesYgqysnV6bm3l0NQk0tAgMmxY9rG2cti7NxsUzj238JVDIUJJlEOnHSdlD3uYwauviqxfLzJtmkifPtnnjEJIU5PIRx+J9O5tfZzKKqyE/ZmLhZIgmUxGERElk8l0eu3o0aPKG2+8oRw9ejT/C+3apSgi2e8FMm/ePCWVSilTpkwJ5PxHjhxR9uzZo+zZs0cREWXNmjXKnj17lPfff9/0PUePHlV27HhDGTLkqFJba3xMbW32Vpm9DnOlpdl7V1oawsXNfsf1z/v534LTc1VXZ49z+jVzZrDl8freQ4eyn+XQIW/ntLqG23MHxenvkZFDh7Kva78eeCD7vgce6Pya35915kx3v2dOv6qrO39O9Wdl9ZmXLs1+37QpuM8cIqv62y1afIrEyJEjpUuXLrJ69epAzv/KK6/IxIkT2x8vXLhQRESuvfbanIHVWocPZ1duvukmkb8f3gktP97YjZeqqcnO7HJyP1esEGltLaI/DLXj7vbuFZk+XWTp0mzzmHbcnfqakyUavHQxiOTXqhNki00YrUFu7uHevfbnC3uYwbRpIvX1Ips2iYwYkX3OaHynUYvPc8+J3HyzyOrVIpMm5Z5X/5m0P6tRo8w/8/Dh2e8jRhRkiEWcEXyCUFGRbWIsYPPyY489JnPmzJFTTjklkPNXVVWJoiiu31denl3Hx4paObe2eihYAjkZL6VOf9c+Z3euvDU2dlRY6vcwxh8YnVetFIwqRLUysuKli+GSS4INLnYhQvuziMJYD7f3MOrU35sRI7L3dsOGbLepiH3wUn82FRWElBAQfIJQUVGQP5/b2tqkqalJHnroIXnzzTdl8+bNgV/Tjb59RT77zNmxtPQ4YzSQ2ajVLJTp79qKbfr03Nf8HH+wYUP2fW4q8o8/zn7XtiS4CWVeZm8GuUaLmxAxfbrIzJnZz6BqagqubGas7uHSpR3hVERk375sC11cBuyqrTKbNuU+F9fB5UWO4BNj27Ztk0mTJsnw4cOloaFBysvLwy4SAmQVVNyGH1ehx6p1YfPmjjDRu7fIddeJPPRQR0WmVmBLl4qcd15u64rTAcr6a9bXi3ztax3dC9pzmb3npz/NftcHMhFnocxLt4p2gLW+As+3JUwfIjZsyN4XM/X1ua+H0cpg9blWrjR+PioDdu1a2Lp1y/773Xez39VgmW+LnxqerMqj/ve3b19uubSM7n2Sg1n+Q47io2CDm6EoCvfUT04HgRsdp3/O9YByt4OFnQ7a9POa1dXu3zNlSn4DYe0G4Kqvux0EO3Om+4G6hw5lB7VqB7eq71+9Ovuc+rViRce1ghwE7GQAtXqP1LKp//Y6SDnfQdtm7/cyYN7q90P7WfXX135W9Tj9vXD7O2X0314Ik3DyweDmBDt0KPt9wAB/j0W0tbY6a50xGi+V9/R3L10UavePOoB406bOgzi9XlMkd0CpSMdfrGbvWb1a5P/+L3fqcVOTyLPP+jMQ1qpFYNiw3C4Q9fibb+64T9oWG33rjNMWKfV+6Ae3/t//Za9lJMgWFTcDqLU/S+2/3f5s8h1mYFZmu67Obt2yv+f33ivy2msi/+//ibS1ZY8zavFTf1fULY9UVi13Rj8rbRdmgRfOjTOCTww5CTSHDmW/CD3Fwc3/y40CTV7bhVh1UVx6ae7/ZHfvzl5IX2GpA0D9uKZ6PqMK0ew9kyaJXHVVbtO+UZeAV1ZjboxCx8yZ2e/qfaqpyR2DI+JfRTZtWu651TAqQiXplPb3UQ25avdWt24iR49m/921a/Zet7V1dEFt2tQ5+KrMfjfUn5eTn5X+51WghXPjjOATM2qQsQo/2tBD8IFITLcL0beiGA1M1rILS35P4dbO3nQ7+LmpKfcv+yCnZvfpY/7+sCtJ7T2My4ad+pCrHTdm1Crzpz/Zt/hpmf0uhP2zKiIEnxiyCj+EHuiFul1IPks7RH2VWn23ipvg4mdrk/b6RvdavyaOfvVq/TkK0eKjDbWXXJLb5aMdLKyd9h2Vlig15Gq7cZ98MtttamTr1uyXlr7FT6TjnmjvjV8/K7a6yEHwiSmj8EPogZ7T6e+ByWfMhb4Vxe8umhDW2wqUURCbMMF4JptK/5rX8Oi2Yn3sMZE1a4zPpQ21avmCCLVew4A+FIwYITJ3bjb42C1mqNK3+IlYB32R/H5WUf8josBiFXwOHjwoixYtki1btsjRo0fl5JNPloceekhGjx4ddtFCoQ0/jY3Z4fuEHqjcTn93xSw0+Bkm9BVMRUX2L+X6eufN/lZdKQVab8tQIUJXRYXIE08YV+52g8TdcluxLlwosmuXcbkKNe4o3zBg9DM0GnvmtMXP64B+q/I4OXcCx3nFJvh8/PHHMm7cOJk4caJs2bJF+vbtK3/84x/l+OOPD7tooRowoCP0pFKEHmQ5Wacnr/BjFhqCDBPqWBqr9WpUdl0pRucu5P/sndwnP8KR1eeaOTM76NuPz+2lYg1zuwmR/MOA9mfox/ikfH9WVr9TUbjfERKb4HPnnXfKoEGDZOPGje3PDR061PI9LS0t0tLS0v64ubk5qOKF5tChjtCjKMzkQlY+09+LQhSa9vMNLkG3SPm5cF0cK9Ywy+z2dyOJiwwGKDbB55lnnpHzzz9fLr/8cnnxxRflxBNPlDlz5sgNFnu8rFq1Sm6z6jONOf2YHvWxCOEn6fKd/h57UWjaD7MrDYXlNsjwuxGq2ASfd955R9avXy8LFy6UJUuWyMsvvyw33XSTlJWVyTXXXGP4nsWLF7fvIi6SbfEZNGhQoOVsbW2V7du3S2Njo1RUVMj48eMlnU77fh2jgcxOproDsea0goljCwTiyyjIFNvg+SJSEnYBnGpra5NRo0ZJXV2dnHXWWTJr1iy54YYbZP369abvKSsrk549e+Z8BamhoUGGDh0qEydOlO9+97syceJEGTp0qDQ0NPh6HavZW0atP36444475LTTTpPu3bvLySefLI8//rh/JwecUisYKhPvolohR7VcXvnxuxrkPSm2++1CbIJPRUWFnHrqqTnPjRgxQvbv3x9SiXI1NDTIZZddJh988EHO8wcPHpTLLrvMt/DjZMp6EOFn+/bt8uMf/1j+8Ic/yPTp0+Waa66Rd955x5+TAyicQoVHr90/YVbEUQsDQd6TKNzvkMQm+IwbN07efPPNnOfeeustGTJkSEgl6tDa2irz588XRVE6vaY+t2DBAmn1aQSpPvQMHDhQ1q1bl3PMe+/tkPHju8sHH7zvyzX/8z//UyZPniwnnXSSzJs3T1pbW+WQn01KAIpLHCvWOJYZrsUm+PzgBz+Ql156Serq6uTtt9+Wxx9/XOrr62Xu3LlhF022b9/eqaVHS1EUOXDggGzfvj3vaxm19IwdO1Z27tyZc70FCxbID36wQL761dxgWFdXJ8cdd5zll1U5FUWRf/3Xf5XTTz9dvvrVr+b9efJVU5Oduu3EihWMJ0y0qP01DyAUsRncfPbZZ8vmzZtl8eLFUltbK5WVlbJ27Vq56qqrwi6aNDpcw8HpcW6NHTtWHn744fbHjz76qOzfv18WL17c6djZs2fLFVdcYXm+E0880fS166+/Xnbs2CHPPfeclJaWei6zX9JpZ+vQaNe1QUIxkwaAxCj4iIhcfPHFcvHFF4ddjE4qHP4F6fQ4t8aOHSuLFi2STz/9VEpKSmTJkiWycuVK6dGjR6dje/XqJb169fJ0nd///vfyH//xH7Jv3z7LcFRIThbhc7KYHwAgGWIVfKJq/PjxMnDgQDl48KDhOJ9UKiUDBw6U8ePHB3L9MWPGSDqdlt27d8uvf/1rOeGEE2TGjBmGx9bV1UldXZ3l+bZs2WJY1nfffVdERE455ZT8C+0jq/BD6Mmqqcm2jjm5BytWZBc0pHEEQDEi+PggnU7L3XffLZdddpmkUqmc8JNKpUREZO3atYGs5yMi0rVrVznzzDOloaFB6uvr5dlnn5WSEuPhW/l0dU2YMCFnLFEh2VXc2vDzwgsi48d3dIMlPfSI0CUIAO2UBMlkMoqIKJlMptNrR48eVd544w3l6NGjns//1FNPKQMHDlREpP1r0KBBylNPPZVPsR2ZN2+ekkqllClTpgR2jYaGBuWUU05xfLwf91RVW6soItnvViZNyh6XTjs7Pkns7qHTewwAhWZVf7tFi4+Ppk2bJlOnTi3Iys16I0eOlC5dusjq1asDu0Ymk+m0pIBbXrtcnI7lee657PlbW7PfnUpC9w5dggAgtPio/GydCMPEiROV+fPnh12MHEb31Gmrgtlxds+rLT6lpdnv+VyrWOk/b9I+P4D48bPFh+Dzd3EMPq2trcqHH36o3H777cqAAQOUTz75JOwi5TC7p/l2uZhV3Gro0T8fRPdOdbXz99TWZo+PEvVzqwGR0AMgygg+HhVb8Hn++eeVVCqljBgxQnnppZfCLk4nVvfUbYuO2fvVilsfevTH5XMtq+vHuTVJvXelpWGXBACsEXw8KrbgE3V29zTfLhe14rYbyGwUfvwIJHEeLEyLD4A4Ifh4RPApLCf31GsFrL5PDT2TJjk73u/KPt+WqzAUaoxP3LsDAUQHwccjgk9hOb2nbrtctBV1dbV5N5fR+0pK/O/eidNg4UIGtWLoDgQQDQQfjwg+hRVEi08+FXeQ3Ttx6DoKo2suzt2BAKKD4OORk+Dz+eefh1Cy4vT555/nNcZH31ViVUnW1ipKVZX160G3ykR5sHCYrS9x7A4EEC0sYBiAL33pSyIi8vnnn0u3bt1CLk1xOHbsmIiI4QKORgvm6RfY026zoD5vtMCe9lyTJnVeoM/Jtbwu2qcuyJj9vCKlpdnvK1Z0nFN7jN0iifkupGi2QGRra+d7Z3Qt9fXWVm/XN2J0r1kwEUBYCD5/l06n5fjjj5fDhw+LiEj37t3b99lCZ3+/TdK3r/HrbW1t0tTUJN27d5c77uiSU8FaVXraSrK2Nvu1fLlIVZV96NG+plbcTq+lfeyGNpyp11CvqZ5Tf4wZP/bJMtuTSx+krK4VRBDR3uuVK7PhkNADIAwEH43+/fuLiLSHH5j75BORTEbk4EGR4483PqakpEQaGgbL8uWp9grWyV/6ZuFn0qTc48zOZdXSY3Ut7eN86M/phF8tIE639gijtWXZso7QU1pK6AEQDoKPRiqVkoqKCunbt6/87W9/C7s4kbduncg994jcdJPInDmdX7/rrlJZtqwkp4I16nIxoj2+pia7B5fTrhK1C0fE/bXc0LeaaMunDyBGxxidJ4jgFZU9uVasMO8OBICC8WHMUWz4OTgKWYUauOp0HR676/q1tozRdcwGUFsNqg5ygK/ZtezWPNK+36+1deI05R9A9DCryyOCTzAKVanpw49dOfTcrPljd5xZgDKa1q4PEIWc+q6/lps1j5K8yCOAaCH4eETwCY5RZe73yr3a4GPX2mL1frvKX33dacuInpNp7YWc+q6/ViHX1mEdHwB+IPh4RPDxnzbc2FWwZuHGTWhRW1C04cdN5WkXfvINPU5ac8Js8bFrlStk6AnimgCKE8HHI4KP//RBwqyCtds93Wno0T9n1Prjtszqe/0KPVbhIgpjfAo13oi9ugD4heDjEcEnGPrAoK9AzQKF19CjUkOWl8paH36cbnbqtpxGLVWFGOvi9FqFbH0CAK8IPh7FKfjE5a9ls9YT/fNmrzsdk6OlHaSsDT+TJnW+D1b3Rt9qlE57uAEOPou+a87LOYIoj1kXJQBEDcHHozgFnziMj7DrKjFrCXLaumA2C8vsvPpB1U7uTSrlvstMG6bcBLh8Wreccvt7Y9ZFCQBRQvDxKE7BR1Hc/+VeSHZdKWbTp/WtC05atrSVtLoRqfa8+mDhNGhUVmaPcRN+9J/bSfm1QczJzLV8Wu/ctBR6aYUDgDAQfDyKW/BRlMKOC3HDqoI1mt2lHq9vXXD6OdRK2qii1p/L6Fg9NfRUVnY+l9n77Moal+5J9fpR/L0CACMEH4/iGHwUJfjZN34y686y+gxuxshYBR9Fye0GMzuvPvQoSm63mlH40ZfX67R8N8cFJcotiQBghODjUVyDj6K4Hx8TBrNw42Qqu10LRFVV7nFGrUba7jWzMThGoceorEYtSW5CWpRChbYlykn5tK1zUfw9A5A8BB+P4hp81IrIyewbbWtEIbtezCpKu7VxrEKF2TmN7oPVvVHPo05ZV0OP/v4YhZ+Sks7ji7y26IQVJrTXtfqdMLr/rK0DIAoIPh7FNfhou3TcbNBZqK4Xu4rebosIo/Bj9jmdtPgYXUd9TTtl3ajc2jKrQUn97uYeRa17MootUQDgFMHHo2IIPmbdLV5bGYIKPYrirYtFUcxbb8zChF3AsApGVsd7DT1OrhuGqLVEAYBTBB+P4hh8zLqC/Bh34keFF9RGpE72ltIGFG1XmtW9MWp9sgpUZoOdnYra4oBRa4kCACcIPh7FLfjYtewYtXg4PVcUKzyzMtqFFaNxRFatQVbn0680rYYWL/crai0+qqiWCwDMEHw8ilPwsapo9RWX08oryhWe2ee1CjXaY+1Wc3YS/PT7dpmNJXJzr6MaNKPWEgUAVgg+HsUp+Nh1IWkrLiddSPr3Od2Xys25vbIbJ6RvoTG7N0b7gtXWms/GMmo9MxrT46Zb0erzRCX8RDkAA4ARgo9HcQo+VrxWXPrK3W4n8kJU1E6vYReOjLqzjN6nD3Lq4+pqRRk61HxMjz78VFV5W8QwiHvqZZsKs/ujV4jgCwB2CD4eFUPw8dqFYjbI18n6OkFyWmmrLT9OwobZxptOug/Ve2I1vkoNSE5adIyCg9/31un51PuiLuJod1+i0kIFAAQfj+IefLxWUG7Gzzg5Xxicfkb9OB3tfmFOQ49Vt5bVPmBmCyKaXdPPlhS7+2PX0hP1MUkAko3g41Gcg4/XcJNvhRgmt0FCbYlRWzTMdojXv0+kY0sM/XWM/u1kFecw7mO+AVcfHqPwOwAAikLw8Syuwcdrd5bbLhC1lSQqFZ5Vd5PRc9rQo1byVl16TlqBtIHHSVnszhs0qzDj5HhCD4AoIvh4FNfg43WRQDfv03cNhcVJK49Ry4u2pUI/xsesMre7P+p11PNUVZkfE6UuIn25nQZmZnkBiCqCj0dxDT5Bi1LF57SVR7+Luj7c6Nep8RpInG4MG5X7p3JSbm3wszve7zFJAOAGwccjgk9nUW6xMAs/2q4noy46fRDxOnXbTaCJ0oKATsutH9NjtzFsFAIdgGQi+HhE8MkVxTEqVmXQBx4n4cZo8LaTKfFuAmGUWnzcBlm3s70AIAwEH48IPh2ctH54rfD82rjUrBLWDzS2m43kds0iN4EwSkHBbZA1u2929xMACo3g4xHBJ0tbEVqFFKNK3ck4D6eVv5Pj9K0pRsFHUToq6aFDOwcT7bYX+j2/9NPS3QRCN0HD713srcrl5HW7x3ZhEQAKieDjEcEny2iLB7txL25bMry0KJmFA33o0S8iqG+hMAsm2spc23rktrXG6L12n8/PMOj1PXYtOfqQSYsPgKgg+HhE8DGmrzj9WoHYa9eL0bW1oUf/nNHig2bBRD8bzEurjHoNo+ntRsc5GVxdXd15Kwmr82rvudvWpAkTzI83mg3HbC4AYSP4eETwMWfWhWMXepx2VeXTlaSGDLMWHbNuL6twpa3c3fKyKajd51dDT2Wl9fm8tAg5pW/x0V8j6O46ADBD8PGI4GPNLHR4CT1G3WlW06WNppGbdUVZPWcV2vShp1DdOE7J65GzAAAgAElEQVTH09i1+DgJPV7DiZOxTtoWLKtQE2Q4A5BMBB9FUerq6hQRUebPn+/4PQQfe/qQ4qaryuoYs3Vu7Lq3jLqJzIJDKmV+Lu3nCWNfMrPwo65BZFcmv8cIOQm5RudyO0MOAPyQ+ODz8ssvK0OHDlXOOOMMgk8AtCFFX5HZbc6pZbaFhJNWBW2FbNQ6o3+vGiDSafPPY1Rhhxl+zMrstnvQ7jpWr9t1axq9bjc4mtADwG+JDj5HjhxRhg0bpmzdulWZMGECwceG264PNdhow4aTViD99dT36DcN1Vee+gHC+vJarYisdtWYtZ5or5dPt52f9GU12xjWrnvQ6XWswold0NE/ZxXGCD0AgpTo4HPNNdcoCxYsUBRFsQ0+f/3rX5VMJtP+deDAgcQFH7ddH2aVmVUrkNl59C0++q4pp91lVpW/WbeXfvaX9rGX8OPnwF59S4/ZtfPdBsMunJh9JrP36UNqvuEMAJxKbPB54oknlNNPP105evSooij2wae6uloRkU5fxRZ8nO4yrv6Vr6+UzcZ46F8zagXSV6JmLSxGrUZOu2OMHps9pyjGG5jq32M0RsXs3rhZx8juOLVsZvuLmd0vr6HC63mcvi9Ke5QBKF6JDD779+9X+vbtq7z66qvtz9Hik+WkUjYLN1ahx+h1u/Eh2hYe/Tn1Y23sumO06/PoP6dRUFBDi7YsqVTuObTr5RiN99EGH6dT0Y3KZ8Tpvlhur2vHazhxsmM7LT4ACiGRwWfz5s2KiCjpdLr9S0SUVCqlpNNp5YsvvrA9RzGP8XFaKRtVqE66f+z+rQ01amAxC0ZmgcXqmlafQzvgWvu62qpi1BVmNvbIqMvMyX12G3rM3uf2unaCavFx0iIHAH5JZPBpbm5WXnvttZyvMWPGKNOnT1dee+01R+co5uCjKM4qZX2Fpm9VsTqXUatKba31+B994HHS0mPXnWUWdGprO88607Y+6cccacOPk/BhVianoVO7OKHROkfack2Y0Pm+uw0XXsOJ3fu8hj8A8CqRwccIs7o6c1JJqS0hdl0YarCw25JC3zJg1tKjnbZtNC3ebKyS0fRpfcixar3SvqZv4dG3/DjdnNNpS4rT7iuzRQz9Ci92zzt9n93+XYQfAEEg+PwdwceYVaVs1uWkpR+obFfJqbN9zAKPttvJqCXIqpJ0GhycBh+rsGE0Hd6Kk7EzVhvCah8b7dVlFULyWTnZayhyGwoJPwD8QvDxKCnBR1GMK2WzYOK0q8vJ6/qgom3pcRNWrI63WgjRKlwYtVDpZ4AZLYBoVC6j9Y6cMCufXwOazd5nt/Gsk1a+6mrnO7bbhTMAcIPg41FSgo9RQDCqEM2Ch9fQY3Rep9O2zdaI0V/DKNCZVepmLV4lJbnn0A6CdtoClU9IMevm8mMrDadr86jPqeHEaSuf/n0AUAgEH4+SEHy8tqpYBSSr8xsx20XdSZeP1TWsAo2eekxJiXUo0ndzublXiuI9MOjLYbath5/8+NkCQBgIPh4Ve/Bx06pj9D4nLR52iyXqV042mgVm9diP6dP6c5kFMDX0VFZ2vgdeWseclEn7mr71qhCLAdqFSkIPgCgi+HhUzMHHbmyHXcWmbSHJtwxupoFru0zc7OBu9rxVi5dR95IafvThxew5u1DjpEza5/QtPmZ7dxld00l3k1FQNbtHkybRhQUgmgg+HhVr8HH617ofXUiFLoOb7hmzY/Vdb2b7eZmFJRHj9Y7MZmwZtXRpQ4W+nHbrCjm9h26ONQtetPgAiCKCj0fFGnzy2UDTTRdSocvgZgaRWYuM/lzql34hRauWInWml9uZUerML+3nsPusZhu7ml3TCbP3qKHHaSsTAISF4ONRsQYfLTcBxM/K1Su7Fgkna+qox+pnhimKcauMWeuWUUuNOgPMaTeWvrXISeixC0H5hlKra7lduwgAwkDw8SgJwcdp5eh2W4YguOnGsuK2tcnNytX6x/rX9Gvb6FuK9M8bLfZodG2nLUBuBHluAAgSwcejJAQfRbEPDFFYgTffMUFGqqvN9x7Tn08NJU7XDzILP2ZdafqWFP0WHVZhTW150pfVj5+D2SDqQrbyAYBbBB+PkhJ8FMWfLiT1+CBm+uQzJsjsPFZjfRSlcwuH/lg3LVBG/zYKPfm2qPg5xV0fpJzOngOAsBF8PEpC8HG6N5T6OO7Tl63G2GgrcLvp624HUmvfq10vKJXKDRVOW9esruV1tp36u6DtjjNbKFF/HOEHQJQQfDxKQvAxCwJup4l75VdLjht24ccs9Ojf76UVTBt6jDY79driYxda3ZzDbud3ozE/cQ/EAIoLwcejJAQfRelcSeq7S+I2dsfLdfXdT2ahJ5/y6K+hDQ/6MT5uzu9knJHbMmrXCNIyC0VREUaQBhA9BB+PkhJ8FKXzX/OFXKjOzViZIK+rH8tjd003Fad6raoqRRk6NPf8Zuvj1NZmj7e6RhD3zq7FxywURUFYQRpAtBB8PIpz8PHyl69+fEk+403clslsPFHQFZS+a0+7+ahfe2CZDWzWBi2joOlXqPHSemQ2hT0OU9rDCtIAooPg41Gcg4/bStFsMKufFZx2irbVnlDqNdWWkaDHkBiFHictPk5oW3rM1u/RP2fXzaYKolvHaLC7el+MWqSi2lXkZ/cfgPgh+HgUdvDJt2Jz+pevPtwENcbHqLVDf06jrSIK0eKjDT36YOL12kYtPfoxPPrPqP38RitLF5p2MLafrWCF4MeAbwDxRPDxKOzg40dXht1fvmbbHAQ1q8sq/OhbQfJdiM8uOOrvgT7o5Bt+zPbqMmr5mTSpo0XMr9amfNm1+MSB2e8zgOJG8PEo7OCjKP6MV7D6yzeMdXyMwo/dSsb5XMcqEGqvq9+IVF9WPypNo3FTVt1eYVXUdmN94hQg/FzUEUA8EHw8ikLwURR/xivY/eVb6DER2speP7YmiP2mrFpezIJIEOHHaNE/7fm1lXNYY2jsWgOjFn6cDJrXjlmL6rgkAP4h+HgUleCjKP6MVzD7y9ePViUv9C0dRt0pflzbaKxNbW1H15JRq5LR/babWu6lTNoxNEahtNDr0tiFmyiGH6eh3e9ZigCii+DjUZSCj6LkN14h37E7QYQf/ZgW/fYN+mvnM9hXew6jriSzWWZGQcIuYDgNK9rQ47S70e7z+REQ7UJNHMKP2eMolRlAcAg+HkUt+CiKt/EKTsf4ODnPhAnGx7sNDvrQY9bioy2vXxW7UXD0M2C4Ocboc7kNP36FUu1eXU7ug/b4KLDrovOzFRFAtBF8PIpa8PHS4uP32B2n57M7Tv3SrmKsH+Ojb5nxo5K1Co5+BgyrY7X3wGhAtdH7/f45Fit9dxb3DUgmgo9HUQo+Xv5qDaqlwK4ysRsfom3l0IcAo3/7VTk5CY5+VpRWgUY/1kS7uKP+WH33nDpA1+z8UWmBCYN28LjdHwhJv1dAMSP4eBSV4OOlMg56bIjZ+8wGkFqFHn2lrm8B8iP4uAmOfnaNGH1Gu/ujDzLaUKTf1NRLd10SMIUdSDaCj0dRCD5eW20KMRvILCCYjbFQu7X0oUd/PrVSHzrUn8o8n+Dox8J3RqHP7jir1jL1/uinvhN6svz82QGIJ4KPR2EHn6BbbfxgVskYPa8NY2bBTLtvlhrG8umS0N8bs9WUjV43Chj6czstl9259OXRz2DThyftfSX0dPCztQ5AfBF8PAo7+BR6DRev19eGFaPnnXY3+P2XulWrktlz6r/VjULNtmlwU6G6/Vx2LWlGXWdU7N5a9gAUJ4KPR2EHn7A5qTDMunH8quzzqazMgptZ+FGDoxp6Kiu9l029tpP3GoVW/f0zmulmtA5QUnntEgZQnAg+HiU9+CiKs3EwZgOUnQaFMP5SNzu3WnY1/NiNX7I7v93nsvqM2pY0/THa1rSkV+px6BIGUFgEH48IPllW3UP65+1mdbkNN4UMP3ZrwLjZ8sCuG8rp60abtRq1piW5Ug+7SxhA9BB8PCL4dDBqpTALQ3br+LjtzipE+DHbDd6qy8lJee0Cn9190m9qanUPqdQBIIvg4xHBJ5fZeB7ta3ahRl+pR+EvdbPB2frX1UHaVVXW+4ZpP1dtbefFCa3CSr5ByY0w7n9UfuYAihvBx6NiDz5eKqGSEuOZWn5u8llIRt1Gdq97aamyu47+eLPXvLzXSfn8OC6q1wSQPAQfj4o9+LithJxuBRAXdq1Udt1Kbscmed0nzOl1gwg/QQSQMK4JIFkIPh4Ve/BRFOeVkNuZWlFn16XkZJySk+48/XNmodGq9c1o0UWzFjMvrWluPodfwrgmgOQg+HiUhOCjKP6EgDixK7fd7C2r8GMVeqIcGsMoYxzuC4B4Ivh4lJTgoyjmlZDdLKa4VVZeW7iszmPVmhOnlg0n45CK4ZoAih/Bx6MkBR9F6VwJuV2sLw6Vlt2Abv2MLKtuI+3rRuN34jiWJYxdzdlJHYDf/Ky/uwiK1rJlIitXihw7JlJaKjJ+vEhVVfZ5u/eJiLS2Bl7EvNXUOH/d6edesaLjnh07ln0sIrJ8uUhtrfl51OeXL3d2vaAZfY6gyxTGNQHAjZKwC4Dg6CuhdNp5JbRsmX2oKEYrVnQEnJaW7Pfly0Wee8469KiWLcseF3ZoNPscaogrlmsCgGs+tEDFRpK6uqI+0DSKC9/FafyOltEsMavPYbVYo1dxvXcA4oExPh4lJfjEoRJyWpZClTmO43dUTqbka49jHR8AcUPw8SgJwSdOlVBUyhq1EOaFXahxGo68XjfO9w5A9DG4GYa0Yyz8GoBbU+N8bNCKFdmxLU7HBlmVxcln8Utrq/PxO+rxUbNsWXYc0gsvdH7N7F768TmK4d4BSBgfglhsFHuLTxDjZgrxF33UxyPFCfcSQDHys/5OKYqihB2+CqW5uVnKy8slk8lIz549wy5ObNi1vvjROqOeQ52BVoiWnmLFvQRQbPysvwk+cMQs3PjZJVVW1jH9vqUlv3MlHfcSQDHxs/72tI7P0aNH5eDBg52ef/311/MqDKJLXZ9Guy6Ln6HHbNFAuMe9BABzroPPz372Mzn55JPlwgsvlDPOOEP+93//t/21q6++2tfCIVq04aeszN/Qw8J3/uBeAoANt4OCzjzzTOXw4cOKoijKzp07lVNPPVV57LHHFEVRlJEjR+Y96MhKXV2dMmbMGOW4445T+vTpo0ydOlXZt2+f4/cX++DmQvFzL6Y4rDkUF9xLAMUq1Onsf/vb36RPnz4iIjJmzBjZtm2bTJs2Td5++21JpVI+x7JcL774osydO1fOPvts+eKLL+TWW2+VyZMnyxtvvCFf/vKXA702svzci8mqqyxq+15FHfcSABxym5SqqqqU3/3udznPtbS0KFdeeaWSTqfzTmJuHD58WBER5cUXX3R0PC0++fFzqjQL3/mHewmg2IXS4nPkyBHp0aOHPProo9KlS+7bSktL5YknnpB58+b5m8psZDIZERHp1auX4estLS3SopnS0tzcXJByFSOjFoV8WhJY+M4/3EsAcM7xdPaRI0fKf/3Xf0n//v2DLpMjiqLI1KlT5eOPP5bt27cbHlNTUyO33XZbp+eZzu5OIdbxAQDATCjT2ceMGSPnnHOO7Nu3L+f5PXv2yIUXXphXIbyYN2+e/P73v5cnnnjC9JjFixdLJpNp/zpw4EABS1gcnG6DwewhAEAcOA4+Dz74oMyYMUPOO+88+c1vfiNvvfWWXHHFFTJmzBgpKysLsoyd3HjjjfLMM8/I888/LwMHDjQ9rqysTHr27JnzBXfcdKPU1lp3o9TUOA9GK1Y43/MLAACnXM3qqq6ultLSUvnGN74hra2tcv7558vOnTtl1KhRQZUvh6IocuONN8rmzZvlhRdekMrKyoJcN8nchA+7cJROOxsPpG1lAgDAT46DT2Njo6xatUoefPBBOfXUU2Xfvn1y5ZVXFiz0iIjMnTtXHn/8cfn5z38uPXr0kA8//FBERMrLy6Vbt24FKwe8cTIYmvFCAIAgOQ4+J510kgwfPlx++tOfykUXXSS//OUv5YorrpAPPvhAFi1aFGQZ261fv15ERKqqqnKe37hxo3zve98rSBmQH6vwQ+gBEqixUWTDBpFZs0QqKsIuDRLAcfDZuHGjXHnlle2Pzz//fHn++efl4osvlvfff1/WrVsXSAG1HE5AQ8QZhR9CD5BQjY0it90mcsklBB8UhOPgow09qlGjRsmOHTtCmdWFeNOGn5Urs6tAE3oAAEHztDu71tChQ+V//ud//CgLEmbZso6tL0pLCT0AgODlHXxERP7hH/7Bj9MgYYz2/QIAIEiuNykF/KAf06M+FqHlByhajY3ZL63du3O/a1VUMO4HviP4oOD83vcLQExs2JAdyGzkhhs6P1ddzUqm8B3BBwVlNXuL8AMUuVmzsrO3tHbvzoaeBx4Q0a8LR2sPAkDwQSBqarIrNTtZp2fFiuxWFzU1hB+gqFl1XY0a1Tn4AAHwZXAz4i2IPbTU7Sm05zXa90sNQ+l0x3NO9v0CAMALWnwQyB5aRi03+sDkpNsLAAA/EXwQ2B5abE8B+IRtHQDfEHwgIsGFFLanAHxQzNs6VFRkZ28V6nMRIhOP4IN2QYUUtqcAYKqiorBT1os5RMIRgg9yBBVSli3rOB/bUwAAwsKsLnQSxB5abE8BAIgCWnzQiVFIySf8sD0F4ILbbR1KSkSeftqfMSuMf0ECEHyQw++QksTtKYwWbzSjXbwREBH32zrMnClSX+/PmJViG//C3mAwQPBBO79DSlK3pwhiXSQkiNttHZqassEHnbE3GAwQfCAi/ocUJ7PBijX8BLUuEhLC7bYORi0XyGJvMBgg+CCQkGK0PYXVeYttewoWb/RHa2urbN++XRobG6WiokLGjx8vae3+JoAV9gaDAYIPAgkpblqLi7XyZ/HG/DQ0NMj8+fPlgw8+aH9u4MCBcvfdd8u0adNCLFmImpo6t/C4GbOiHbysPvZ6LiCulATJZDKKiCiZTCbsoiBBamsVRURRSkuz32trwy5R9D311FNKKpVSRCTnK5VKKalUSnnqqafCLqK1Q4cUpbo6+90Pu3Zlf3lmzsx+d/qllkH9rp5n167sc27PVQy09wCx4Wf9nVIURQkzeBVSc3OzlJeXSyaTkZ49e4ZdHCRIWVnHEgEtLWGXJtpaW1tl6NChOS09WqlUSgYOHCjvvvtudLu9du8WGT1aZNcuf7pT1Jaab31LpK2t87Wsxqw0NnaURaTj3+prbs5VDC0+fv9sUBB+1t90dQEB83tdpGK3fft209AjIqIoihw4cEC2b98uVVVVhStYmJxs62A2ZkUfbrTnTOL4l0LvDYbIYeVmIEDaMT0tLdnvy5ezcrWVRrOK2uNxQA41RBJ8EosWHyAgSVy80Q8VDiskp8cllrp439692cd794ocPZr9t9ngZSABCD5AAJK6eKMfxo8fLwMHDpSDBw+K0RBEdYzP+PHjQyidgSitDqwty4YNuQsbTp/e8W+zxfv0a94ARYjgA/gsyYs3+iGdTsvdd98tl112maRSqZzwk0qlRERk7dq10RnYHPbqwNoxK1ZlMTJzZsfUdjWMMf4FRY5ZXYDP2KvLH0br+AwaNEjWrl0brXV8zFp8wpgdZVWW1atFfvtbkYYGkU2bREaMKJ6ZWih6ftbfBB8AkRXblZujNGVaLcumTR3dXVEoF+AC09kBJEI6nU7OlHUABcF0dgAAkBgEHwDQa2zMDrwqxrWCZs5kXA8Sja4uANBrbMzOjrrkEm8hwWx1YO0moUGGD+0g56ambNhRH599dudB0AxyRoIQfADAb2ZbTOQbqJwKe4o9EGEEHwAoNrNmdV6M0G6KPZAQBB8AKDaF3oC0UF14gA8IPgCSLUpbTsRVobrwAB8QfAAkW1DjYQhUQCQRfAAkW1DjYZI0wLipKfc7EGEEHwDJFtR4mKgNMDabYu+Hjz7K/Q5EGMEHAIJQ6AHGTsrjR4uSURfevn0d3/XdeHThIWIIPgAA56y68FauzH5pxbkLD0WJ4AMAcM6oC2/z5mzgWbpU5NJLc1+jtQcRw15dAKAX5HgYAKFKKYqihF2IQmlubpby8nLJZDLSs2fPsIsDIMryWZTP7L3FsNBfTY15V5cRurrgAz/rb4IPABjZvVtk9GiRXbvcD0R2+t44BiGjwc12XV1x+WyILD/rb8b4AEBY4rjisVGQ2bs3+3348MLPVgNcYowPACA/vXvnfgcijOADAMhPnz6534EIo6sLAPLZV4s9uYBYiV3wWbdunaxevVoaGxvltNNOk7Vr18r48ePDLhaAOMtnXy237505MzuYWaR4AhLT/xEjsZrV9eSTT8rVV18t69atk3HjxsmGDRvkwQcflDfeeEMGDx5s+35mdQEwZNZqY7Wvll2Lj/69GzaI1Nc7LxPTwIF2iZ3Ofs4558ioUaNk/fr17c+NGDFCvvWtb8mqVats30/wAeCY39PZnQSkvXtFpk8X2bRJZNKk4lsDCPAokdPZjx07Jrt27ZJbbrkl5/nJkyfLjh07DN/T0tIiLS0t7Y+bm5sDLSMAmHKzaemIEZ2PdTv1naAEGIrNrK6PPvpIWltbpV+/fjnP9+vXTz788EPD96xatUrKy8vbvwYNGlSIogJA+NSgpG9lAhIuNsFHlUqlch4ritLpOdXixYslk8m0fx04cKAQRQQAABEVm66u3r17Szqd7tS6c/jw4U6tQKqysjIpKysrRPEAFJuKCpGFC0Uee8z9DCsns5waGztWPFa/W83yampyfv1iRfcdfBCb4FNaWiqjR4+WrVu3yqWavWC2bt0qU6dODbFkAIpSRYXIVVdlBylfdZX74GM3I0s7DX769NzXjKbBT5mS/e526vuGDdmyVFTEPzjEcYsPRE5sgo+IyMKFC+Xqq6+WMWPGyLnnniv19fWyf/9+mT17dthFAwB3Zs3KVuBa6iyvKVNEnn029zX1sVEoWrgwG86amkQaGkSmTRNRu/br60W+9rXsgOm9e7PB4dxzCQ5IrFgFn29/+9vy5z//WWpra6WxsVFOP/10+cUvfiFDhgwJu2gA4I5VK83cuZ1bjKzWFXrssWzLlEq/XpC+RamhQeT88z0VG4i7WAUfEZE5c+bInDlzwi4GAASnTx/ztYP0U99FOrrltOsAHT3a0Tq0aVNHi8/06dkWISChYhd8ACAQcd5zS1+WESNyXx8xIjcsWW0mGpVxQHH+eSDSCD4AIJLffl1BlEUdkOyUfpbYxx93vLZ5c/a5ffs6Hjc15QYgNThEZQBxlH4eKCoEHwAQsR5sbLZfl98qKrKbmNbX27e46FtEtHuB6cf0rFxp/VgkesEhCj8PFCWCDwCIuNtSIsgyzJplvJmpfm0gqxYRI5deKnLaadnQs3SpyHnndW7xcSvIbrEo/DxQlAg+AKClrcyjRL82kF2LyKBB2QDV0NCx6WljYzb4XHqpP8EhKt1igAsEHwDQ0lbmUeakRaRPn2zwUTc91Q8WZgAxEojgAwB+8NLtE3bwKMYBxFGZlYbIIvgAiK+gKzkne25py+K22yfs4KF2l+lXfI7qAGKne6DR/QYLBB8A8RV0Jedkzy0rdsEs6JlL+qBg9LiiIntNdSaZOuBZP4BYbZ3StlAVulss358HIAQfAEkWdFeTXTDze+aSUbDRBoV8gkPYrVOATwg+AJKr2CrzIFtEWFcHRYLgAyC5vFbmTluK1JWUm5r8K7NX+Y6HiuK6OmEPDkcsEXwAxEMQlZzXytxtS5Efu6HnG1zUbrdzz7W+j4MGZVePbmrKPo5yWCi2FjsUBMEHQDxEqZIzail67jmRm2/Oroo8fHj2uX37sgsGDhvWOZyZBQqzmUt+DeRuaDBeGVok9z6qx0Q5LND9Bg8IPgDiIUqVnFFo2bAh+91oH6ybb+78nFmgCHrm0rRpnVeljmtYiGL3GyKP4AMgHqJeyU2blm0l2bQpu1KySDQDRZ8+5vfK7X10s84REBEEHwDQ8lqZq+vfbNuW3RdL+34vwSyfMT1ux0N5HXzNujqIIYIPAGjlW5mrCwHm0wrS2JgdM3TbbdnxQSNGuBvI7XY81MyZ3ssKxAzBBwD8UFGRDRBGA4c3bMiGKS9bWUyfnvuak4HcbsdDNTWZD3iOG7rfYIPgAyC+olTJVVRkA4dRgHDbCjRrVralZ/r0jjFDbsYLuR0P1dgYnfuYL7rfYIPgAyC+ol7JWbUC2b1PHSA9YkRuUAliIHfU7yPgI4IPAHjhdADx176WDT5WA4idnEtdBXrv3mgvKghEHMEHALzwc/VmN+eaPj3aiwoCEUfwAQAv/Fy92clg5L17O8b8TJrkvrxOxkPluy0GEAMpRVGUsAtRKM3NzVJeXi6ZTEZ69uwZdnEAFBuzwc1mrFpudu8WGT1aZNeubPApRCjRXxOICD/rb1p8AMAvQa7ezABkwBcEHwDwi7p6s34mlkg0ttUAICVhFwAAAKBQaPEBAL/4uaBi0Iszut3Piyn0KBIMbgaAIEV1wHBNjfkUeiNMoUeIGNwMAHERpW01tNzu5xW18gMeEXwAIEhRnY3ldj8voEgwuBkAACQGwQcAACQGwQcAACQGwQcAkBWFgdiNjdkxUfqp9oBPCD4AgCx1IHbYwee22wg+CAzBBwAAJAbBBwCiiC4fIBAEHwCIIrp8gECwgCEAIBzsF4YQEHwAAOHYsMF8v7Abbuj8HPuFwQcEHwBAONgvDCEg+ABA2JLa5cN+YQgBwQcAwkaXD1AwBB8ACBtdPkDBEHwAIGx0+QAFwzo+AIDoiMJ+YShqtPgAAKJD3S8MCAgtPgAAIDEIPgAQRXT5AIGIRfB577335LrrrpPKykrp1q2bfOUrX5Hq6mo5duxY2EUDgGCoXT4EH8BXsRjjs2/fPmlra5MNGzbIP/7jP8of/vAHueGGG+Szzz6Tu+66K+UQiIYAAA1nSURBVOziAQCAmEgpiqKEXQgvVq9eLevXr5d33nnH8Xuam5ulvLxcMpmM9OzZM8DSAQAAv/hZf8eixcdIJpORXr16WR7T0tIiLS0t7Y+bm5uDLhYAAIiwWIzx0fvjH/8o9957r8yePdvyuFWrVkl5eXn716BBgwpUQgAAEEWhBp+amhpJpVKWX6+88krOew4dOiTf/OY35fLLL5frr7/e8vyLFy+WTCbT/nXgwIEgPw4AAIi4UMf4fPTRR/LRRx9ZHjN06FDp2rWriGRDz8SJE+Wcc86Rhx9+WEpK3OU2xvgAABA/RTPGp3fv3tK7d29Hxx48eFAmTpwoo0ePlo0bN7oOPQAAALEY3Hzo0CGpqqqSwYMHy1133SVNTU3tr/Xv3z/EkgEAgDiJRfD51a9+JW+//ba8/fbbMnDgwJzXYjobHwAAhCAW/UXf+973RFEUwy8AAACnYhF8AAAA/EDwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiUHwAQAAiRG74NPS0iIjR46UVColr776atjFAQAAMRK74PNv//ZvMmDAgLCLAQAAYqhL2AVwY8uWLfKrX/1KnnrqKdmyZYvt8S0tLdLS0tL+OJPJiIhIc3NzYGUEAAD+UuttRVHyPldsgs+f/vQnueGGG+Tpp5+W7t27O3rPqlWr5Lbbbuv0/KBBg/wuHgAACNif//xnKS8vz+scKcWP+BQwRVHkwgsvlHHjxsnSpUvlvffek8rKStmzZ4+MHDnS9H36Fp9PPvlEhgwZIvv378/7xiVdc3OzDBo0SA4cOCA9e/YMuzixxX30D/fSP9xLf3Af/ZPJZGTw4MHy8ccfy/HHH5/XuUJt8ampqTFskdHauXOn7NixQ5qbm2Xx4sWuzl9WViZlZWWdni8vL+eX0Cc9e/bkXvqA++gf7qV/uJf+4D76p6Qk/6HJoQafefPmyZVXXml5zNChQ2XlypXy0ksvdQoxY8aMkauuukoeeeSRIIsJAACKRKjBp3fv3tK7d2/b4+655x5ZuXJl++NDhw7J+eefL08++aScc845QRYRAAAUkXRNTU1N2IWwU15eLn379m3/SqfTcvfdd8vixYvl5JNPdnWudDotVVVV0qVLbMZ1Rxb30h/cR/9wL/3DvfQH99E/ft3LWAxu1nM6uBkAAEArlsEHAADAi9it3AwAAOAVwQcAACQGwQcAACQGwQcAACRGIoPPe++9J9ddd51UVlZKt27d5Ctf+YpUV1fLsWPHwi5aLKxbt04qKyula9euMnr0aNm+fXvYRYqdVatWydlnny09evSQvn37yre+9S158803wy5W7K1atUpSqZQsWLAg7KLE0sGDB2X69OlywgknSPfu3WXkyJGya9eusIsVO1988YUsXbq0vY456aSTpLa2Vtra2sIuWuRt27ZNpkyZIgMGDJBUKiVPP/10zuuKokhNTY0MGDBAunXrJlVVVfL666+7ukYig8++ffukra1NNmzYIK+//rr8+Mc/lvvvv1+WLFkSdtEi78knn5QFCxbIrbfeKnv27JHx48fLBRdcIPv37w+7aLHy4osvyty5c+Wll16SrVu3yhdffCGTJ0+Wzz77LOyixdbOnTulvr5ezjjjjLCLEksff/yxjBs3Tr70pS/Jli1b5I033pAf/ehHee+LlER33nmn3H///XLffffJ3r175Yc//KGsXr1a7r333rCLFnmfffaZnHnmmXLfffcZvv7DH/5Q1qxZI/fdd5/s3LlT+vfvL9/4xjfkyJEjzi+iQFEURfnhD3+oVFZWhl2MyPvqV7+qzJ49O+e54cOHK7fccktIJSoOhw8fVkREefHFF8MuSiwdOXJEGTZsmLJ161ZlwoQJyvz588MuUuwsWrRIOe+888IuRlG46KKLlBkzZuQ8N23aNGX69OkhlSieRETZvHlz++O2tjalf//+yh133NH+3F//+lelvLxcuf/++x2fN5EtPkYymYz06tUr7GJE2rFjx2TXrl0yefLknOcnT54sO3bsCKlUxSGTyYiI8Dvo0dy5c+Wiiy6Sf/mXfwm7KLH1zDPPyJgxY+Tyyy+Xvn37yllnnSUPPPBA2MWKpfPOO0/++7//W9566y0REfnd734nv/nNb+TCCy8MuWTx9u6778qHH36YUweVlZXJhAkTXNVBrKEtIn/84x/l3nvvlR/96EdhFyXSPvroI2ltbZV+/frlPN+vXz/58MMPQypV/CmKIgsXLpTzzjtPTj/99LCLEzs/+clPZPfu3bJz586wixJr77zzjqxfv14WLlwoS5YskZdfflluuukmKSsrk2uuuSbs4sXKokWLJJPJyPDhwyWdTktra6vcfvvt8p3vfCfsosWaWs8Y1UHvv/++4/MUVYtPTU2NpFIpy69XXnkl5z2HDh2Sb37zm3L55ZfL9ddfH1LJ4yWVSuU8VhSl03Nwbt68efL73/9ennjiibCLEjsHDhyQ+fPny6ZNm6Rr165hFyfW2traZNSoUVJXVydnnXWWzJo1S2644QZZv3592EWLnSeffFI2bdokjz/+uOzevVseeeQRueuuu+SRRx4Ju2hFId86qKhafObNmydXXnml5TFDhw5t//ehQ4dk4sSJcu6550p9fX3ApYu/3r17Szqd7tS6c/jw4U4JHM7ceOON8swzz8i2bdtk4MCBYRcndnbt2iWHDx+W0aNHtz/X2toq27Ztk/vuu09aWloknU6HWML4qKiokFNPPTXnuREjRshTTz0VUoni6+abb5ZbbrmlvT76p3/6J3n//fdl1apVcu2114Zcuvjq37+/iGRbfioqKtqfd1sHFVXw6d27t/Tu3dvRsQcPHpSJEyfK6NGjZePGjVJSUlSNX4EoLS2V0aNHy9atW+XSSy9tf37r1q0yderUEEsWP4qiyI033iibN2+WF154QSorK8MuUix9/etfl9deey3nue9///syfPhwWbRoEaHHhXHjxnVaUuGtt96SIUOGhFSi+Pr888871SnpdJrp7HmqrKyU/v37y9atW+Wss84SkezY0xdffFHuvPNOx+cpquDj1KFDh6SqqkoGDx4sd911lzQ1NbW/piZKGFu4cKFcffXVMmbMmPaWsv3798vs2bPDLlqszJ07Vx5//HH5+c9/Lj169GhvRSsvL5du3bqFXLr46NGjR6dxUV/+8pflhBNOYLyUSz/4wQ/kn//5n6Wurk6uuOIKefnll6W+vp7WcA+mTJkit99+uwwePFhOO+002bNnj6xZs0ZmzJgRdtEi79NPP5W33367/fG7774rr776qvTq1UsGDx4sCxYskLq6Ohk2bJgMGzZM6urqpHv37vLd737X+UV8m3cWIxs3blRExPAL9v793/9dGTJkiFJaWqqMGjWKKdgemP3+bdy4MeyixR7T2b179tlnldNPP10pKytThg8frtTX14ddpFhqbm5W5s+frwwePFjp2rWrctJJJym33nqr0tLSEnbRIu/55583/H/jtddeqyhKdkp7dXW10r9/f6WsrEz52te+prz22muurpFSFEXJP6MBAABEHwNbAABAYhB8AABAYhB8AABAYhB8AABAYhB8AABAYhB8AABAYhB8AABAYhB8AABAYhB8AABAYhB8AMTOE088IV27dpWDBw+2P3f99dfLGWecIZlMJsSSAYg6tqwAEDuKosjIkSNl/Pjxct9998ltt90mDz74oLz00kty4oknhl08ABGWyN3ZAcRbKpWS22+/XS677DIZMGCA3H333bJ9+/b20HPppZfKCy+8IF//+tflZz/7WcilBRAltPgAiK1Ro0bJ66+/Lr/61a9kwoQJ7c8///zz8umnn8ojjzxC8AGQgzE+AGLpl7/8pezbt09aW1ulX79+Oa9NnDhRevToEVLJAEQZwQdA7OzevVsuv/xy2bBhg5x//vmybNmysIsEICYY4wMgVt577z256KKL5JZbbpGrr75aTj31VDn77LNl165dMnr06LCLByDiaPEBEBt/+ctf5IILLpBLLrlElixZIiIio0ePlilTpsitt94acukAxAEtPgBio1evXrJ3795Oz//85z8PoTQA4ohZXQCKzvnnny+7d++Wzz77THr16iWbN2+Ws88+O+xiAYgAgg8AAEgMxvgAAIDEIPgAAIDEIPgAAIDEIPgAAIDEIPgAAIDEIPgAAIDEIPgAAIDEIPgAAIDEIPgAAIDEIPgAAIDEIPgAAIDE+P8uAWVAgedq7wAAAABJRU5ErkJggg==",
      "text/plain": [
       "PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x1366e32d0>)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Generate dataset {(x1,y1),...,(xN,yN)}\n",
    "# x is a 2-d feature vector [x_1;x_2]\n",
    "# y ∈ {false,true} is a binary class label\n",
    "# p(x|y) is multi-modal (mixture of uniform and Gaussian distributions)\n",
    "using PyPlot\n",
    "include(\"scripts/lesson8_helpers.jl\")\n",
    "N = 200\n",
    "X, y = genDataset(N) # Generate data set, collect in matrix X and vector y\n",
    "X_c1 = X[:,find(.!y)]'; X_c2 = X[:,find(y)]' # Split X based on class label\n",
    "X_test = [3.75; 1.0] # Features of 'new' data point\n",
    "function plotDataSet()\n",
    "    plot(X_c1[:,1], X_c1[:,2], \"bx\", markersize=8)\n",
    "    plot(X_c2[:,1], X_c2[:,2], \"r+\", markersize=8, fillstyle=\"none\")\n",
    "    plot(X_test[1], X_test[2], \"ko\")   \n",
    "    xlabel(L\"x_1\"); ylabel(L\"x_2\"); legend([L\"y=0\", L\"y=1\",L\"y=?\"], loc=2)\n",
    "    xlim([-2;10]); ylim([-4, 8])\n",
    "end\n",
    "plotDataSet();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "###  Main Idea of Discriminative Classification \n",
    "\n",
    "- Again, a data set is given by  $D = \\{(x_1,y_1),\\dotsc,(x_N,y_N)\\}$ with $x_n \\in \\mathbb{R}^D$ and $y_n \\in \\mathcal{C}_k$, with $k=1,\\ldots,K$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "-  Sometimes, the precise assumptions of the (multinomial-Gaussian) generative model $$p(x_n,\\mathcal{C}_k|\\theta) =  \\pi_k \\cdot \\mathcal{N}(x_n|\\mu_k,\\Sigma)$$ clearly do not match the data distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Here's an **IDEA**! Let's model the posterior $$p(\\mathcal{C}_k|x_n)$$  _directly_, without any assumptions on the class densities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Of course, this implies also that we build direct models for the **discrimination boundaries** \n",
    "  $$\\log \\frac{p(\\mathcal{C}_k|x_n)}{p(\\mathcal{C}_j|x_n)} \\overset{!}{=} 0$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 1. Model Specification \n",
    "\n",
    "- <span style=\"color:blue\">[Q.]</span> What model should we use for $p(\\mathcal{C}_k|x_n)$?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "-   <span style=\"color:blue\">[A.]</span> Get inspiration from the generative approach: choose the familiar softmax structure **with linear discrimination bounderies** for the posterior class probability\n",
    "$$\n",
    "p(\\mathcal{C}_k|x_n,\\theta) = \\frac{e^{\\theta_k^T x_n}}{\\sum_j e^{\\theta_j^T x_n}}\n",
    "$$\n",
    "but **do not impose a Gaussian structure on the class features**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- $\\Rightarrow$ There are **two key differences** between the discriminative and generative approach: \n",
    "  1. In the discriminative approach, the parameters $\\theta_k$ are **not** structured into $\\{\\mu_k,\\Sigma,\\pi_k \\}$. This provides discriminative approach with more flexibility.\n",
    "  2. ML learning for the discriminative approach by optimization of _conditional_ likelihood $\\prod_n p(y_n|x_n,\\theta)$ rather than _joint_ likelihood $\\prod_n p(y_n,x_n|\\theta)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    " ###  2. ML Estimation for Discriminative Classification\n",
    " \n",
    "\n",
    "-  The conditional log-likelihood for discriminative classification is \n",
    "\n",
    "     $$\n",
    "    \\mathrm{L}(\\theta) = \\log \\prod_n \\prod_k {p(\\mathcal{C}_k|x_n,\\theta)}^{y_{nk}} \n",
    "     $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "     \n",
    "- Computing the gradient $\\nabla_{\\theta_k} \\mathrm{L}(\\theta)$ (NB: revised text) leads to (for proof, see next slide) \n",
    "\n",
    "$$\n",
    "\\nabla_{\\theta_k} \\mathrm{L}(\\theta) = \\sum_n \\Big( \\underbrace{y_{nk}}_{\\text{target}} - \\underbrace{\\frac{e^{\\theta_k^T x_n}}{ \\sum_j e^{\\theta_j^T x_n}}}_{\\text{prediction}} \\Big)\\cdot x_n \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "  \n",
    "- Compare this to the gradient for _linear_ regression:\n",
    "\n",
    "$$\n",
    "\\nabla_\\theta \\mathrm{L}(\\theta) =  \\sum_n \\left(y_n - \\theta^T x_n \\right)  x_n\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In both cases\n",
    "\n",
    "$$\n",
    "\\nabla_\\theta \\mathrm{L} =  \\sum_n \\left( \\text{target}_n - \\text{prediction}_n \\right) \\cdot \\text{input}_n \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The parameter vector $\\theta$ for logistic regression can be estimated through iterative gradient-based adaptation. E.g. (with iteration index $i$),\n",
    "\n",
    "$$\n",
    "\\hat{\\theta}^{(i+1)} =  \\hat{\\theta}^{(i)} + \\eta \\cdot \\left. \\nabla_\\theta   \\mathrm{L}(\\theta)  \\right|_{\\theta = \\hat{\\theta}^{(i)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    " ###  2. <span style=\"color:red\">(OPTIONAL)</span> Proof of Derivative of Log-likelihood for  Discriminative Classification\n",
    "\n",
    "\n",
    "- The Log-likelihood is $\n",
    "    \\mathrm{L}(\\theta) = \\log \\prod_n \\prod_k {\\underbrace{p(\\mathcal{C}_k|x_n,\\theta)}_{p_{nk}}}^{y_{nk}} = \\sum_{n,k} y_{nk} \\log p_{nk}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "     \n",
    "- Use the fact that the softmax $\\phi_k \\equiv e^{a_k} / {\\sum_j e^{a_j}}$ has analytical derivative:\n",
    "\n",
    "$$ \\begin{align*}\n",
    " \\frac{\\partial \\phi_k}{\\partial a_j} &= \\frac{(\\sum_j e^{a_j})e^{a_k}\\delta_{kj}-e^{a_j}e^{a_k}}{(\\sum_j e^{a_j})^2} = \\frac{e^{a_k}}{\\sum_j e^{a_j}}\\delta_{kj} - \\frac{e^{a_j}}{\\sum_j e^{a_j}} \\frac{e^{a_k}}{\\sum_j e^{a_j}}\\\\\n",
    "     &= \\phi_k \\cdot(\\delta_{kj}-\\phi_j)\n",
    " \\end{align*}$$\n",
    "\n",
    "<!---\n",
    "%    -  Again we try to minimize the cross-entropy ($\\sum_{nk} y_{nk} \\log \\frac{y_{nk}}{p_{nk}}$) between the data `targets' $t_{nk}$ and the model outputs $p_{nk}$.\n",
    "--->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    " -  Take the derivative of $\\mathrm{L}(\\theta)$ (or: how to spend a hour ...)\n",
    "$$\\begin{align*} \n",
    "\\nabla_{\\theta_j} \\mathrm{L}(\\theta) &= \\sum_{n,k} \\frac{\\partial \\mathrm{L}_{nk}}{\\partial p_{nk}} \\cdot\\frac{\\partial p_{nk}}{\\partial a_{nj}}\\cdot\\frac{\\partial a_{nj}}{\\partial \\theta_j} \\\\\n",
    "  &= \\sum_{n,k} \\frac{y_{nk}}{p_{nk}} \\cdot p_{nk} (\\delta_{kj}-p_{nj}) \\cdot x_n \\\\\n",
    "  &= \\sum_n \\Big( y_{nj} (1-p_{nj}) -\\sum_{k\\neq j} y_{nk} p_{nj} \\Big) \\cdot x_n \\\\\n",
    "  &= \\sum_n \\left( y_{nj} - p_{nj} \\right)\\cdot x_n \\\\\n",
    "  &= \\sum_n \\Big( \\underbrace{y_{nj}}_{\\text{target}} - \\underbrace{\\frac{e^{\\theta_j^T x_n}}{\\sum_{j^\\prime} e^{\\theta_{j^\\prime}^T x_n}}}_{\\text{prediction}} \\Big)\\cdot x_n \n",
    "\\end{align*}$$\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 3. Application - Classify a new input\n",
    "\n",
    "-  Discriminative model-based prediction for a new input $x_\\bullet$ is easy, namely substitute the ML estimate in the model to get\n",
    "\n",
    "$$\n",
    "p(\\mathcal{C}_k |\\, x_\\bullet,\\hat\\theta) = \\frac{ \\mathrm{exp}\\left( \\hat \\theta_k^T x_\\bullet \\right) }{ \\sum_{k^\\prime} \\mathrm{exp}\\left(\\hat \\theta_{k^\\prime}^T x_\\bullet \\right)} \n",
    "  \\propto \\mathrm{exp}\\left(\\hat \\theta_k^T x_\\bullet\\right) \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "-  The contours of equal probability (**discriminant boundaries**) are lines (hyperplanes) in feature space given by\n",
    "$$\n",
    "\\log \\frac{{p(\\mathcal{C}_k|x,\\hat \\theta )}}{{p(\\mathcal{C}_j|x,\\hat \\theta )}} = \\left( \\hat{\\theta}_{k} - \\hat{\\theta}_j\\right) ^T x = 0\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### CODE EXAMPLE\n",
    "\n",
    "Let us perform ML estimation of $\\theta$ on the data set from the introduction. To allow an offset in the discrimination boundary, we add a constant 1 to the feature vector $x$. We only have to specify the (negative) log-likelihood and the gradient w.r.t. $\\theta$. Then, we use an off-the-shelf optimisation library to minimize the negative log-likelihood.\n",
    "\n",
    "We plot the resulting maximum likelihood discrimination boundary. For comparison we also plot the ML discrimination boundary obtained from the generative Gaussian classifier from lesson 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj4AAAG2CAYAAAB/OYyEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzs3Xt4FOWhP/DvJpAQ5CY3CUIICCRcDhcJCkU0REWtqIWqpafgORUFxBveqvATguGiLS3FWrEEtVpFTp9KaOEc76UqiAIC3oAkRLlECJCIbABJINn398d2wmQyMzszO7Mzs/v9PM8+MbuzM+9OIu837zUghBAgIiIiSgBJbheAiIiIKFYYfIiIiChhMPgQERFRwmDwISIiooTB4ENEREQJg8GHiIiIEgaDDxERESUMBh8iIiJKGAw+RERElDAYfIiIiChh+Cb41NXV4fHHH0fPnj2RlpaGXr16oaCgAKFQyO2iERERkU80c7sARv3617/Gn/70J7z88ssYMGAAPv30U/zyl79E27Ztcf/997tdPCIiIvIB3wSfjz/+GDfddBOuv/56AEBmZiZWrVqFTz/91OWSERERkV/4Jvhcdtll+NOf/oTS0lL07dsXn3/+OTZu3IilS5dqvqe2tha1tbUN34dCIRw7dgwdOnRAIBCIRbGJiIgoSkIInDhxAl27dkVSUpSjdIRPhEIh8dhjj4lAICCaNWsmAoGAWLRoke578vPzBQA++OCDDz744CMOHuXl5VHniYAQQsAH/ud//gePPPIIFi9ejAEDBuCzzz7DzJkzsWTJEvzXf/2X6nuULT7BYBAZGRkoLy9HmzZtYlV0oriwadMmXHfddcjIyMCXX37pdnFs9fbbb+PZZ5/FjTfeiDvuuMPt4hCRQnV1Nbp3747jx4+jbdu2UZ3LN11djzzyCB577DFMnDgRAPAf//Ef2L9/P5588knN4JOamorU1NQmz7dp04bBh8ikYcOGAQDKy8vRrFkztGzZ0uUS2WfLli344IMPkJ2dzX8biDzMjmEqvgk+P/zwQ5N+veTkZE5nJ4qRjh07on379jh27Bj27NmDwYMHu10k20ydOhV9+vRB//793S4KETnMN8HnhhtuwMKFC5GRkYEBAwZgx44dWLJkCW6//Xa3i0aUEAKBALKzs7Fp0yYUFxfHVfDp06cP+vTp43YxiCgGfBN8nnnmGcyZMwczZszA0aNH0bVrV0ybNg1z5851u2hECSMrK6sh+BAR+ZFvgk/r1q2xdOlS3enrROSs7OxsAEBJSYnLJbHPJ598gsrKSowcORIdO3Z0uzhE5DDfbFlBRO6Tgk88tfhIs7meffZZt4tCRDHA4ENEhslbfOJlYkFmZib69++PkSNHul0UIooB36zjY4fq6mq0bdsWwWCQU1aJLDh79izOO+88nD17Fvv370dGRobbRSKiBGBn/c0WHyIyrHnz5ujduzeA+OruIqLEweBDRKZkZWUBiI/gI1/ZnYgSA4MPEZkSTzO7rr/+evTq1Qvvvfee20UhohjxzXR2IvKGeJnZFQqF8OmnnyIYDKJTp05uF4eIYoTBh4hMiZfgk5SUhAMHDmDLli0YMGCA28UhohhhVxcRmSKN8Tl06BCqq6tdLk102rRpg6uuugrNmvFvQKJEweBDRKa0a9cOXbp0ARAf43yIKLEw+BCRafEws+uhhx7C0qVLcezYMbeLQkQxxOBDRKb5fWbX8ePHsWTJEjzwwAOoq6tzuzhEFEPs2CYi0/w+wDkUCmHhwoX45ptv0LlzZ7eLQ0QxxOBDRKb5Pfi0b98es2fPdrsYROQCdnURkWlS8NmzZw+7iojIVxh8iMi0jIwMtGjRAmfOnMG+ffvcLo4pQgh88sknqKmpcbsoROQCBh8iMi0pKQl9+/YF4L/urrKyMowcORIXXHABW6uIEhCDDxFZ4teZXeXl5ejUqRMGDhzIhQuJEhD/ryciS/w6wDkvLw9HjhxBMBh0uyhE5AK2+BCRJX4NPgAQCATQrl07t4tBRC5g8CEiS/wcfIgocTH4EJEl0uDmqqoqVFVVuVwaYz766CNccsklKCgocLsoROQSBh8isuS8885D9+7dAfhngPNHH32ErVu34osvvnC7KETkEg5uJiLLsrOzUV5ejpKSEowaNcrt4kQ0adIkZGRkcJsKogTGFh8issxv43y6du2KiRMnIi8vz+2iEJFLGHyIyDK/BR8iIgYfIrLMT8Fn27ZtWLlyJfbv3+92UYjIRQw+RGSZFHy++eYb1NbWulwafStXrsSkSZPw29/+1u2iEJGLGHyIyLL09HS0atUK9fX1+Prrr90ujq6ePXtixIgRGD16tNtFISIXMfgQkWWBQMA3e3bde++9+Pjjj3Hrrbe6XRQichGDDzlm3jxg/nxjx86fHz6e/MdP43yIiBh8yDHJycDcuZHDz/z54eOSk2NTLrKXH4LPiRMnIIRwuxhE5AEMPuSYOXOAggL98COFnoKC8PHkP34IPlOnTkXHjh2xatUqt4tCRC5j8CFH6YUfhp74IA8+Xm1V2bFjB44dO8YVm+NdRUW4z7yiwu2SxE4ifuYoMfiQ49TCD0OPOV4eL9W7d28kJSWhuroahw8fjt2FTfji7bex5Y47MDIz0+2ikJMqKoAnnnAvBBgNIXaGFbc/sw8x+FBMyMNPaipDj1leHi+VmpqKnj17AvDuzK6U777D8OefR8tg0O2iUDwzGkIYVlzF4EMxM2cOkJICnDkT/srQY5zXx0v5YZyP7djFED3eQ3IBgw/FzPz550LPmTPGu24ozMvjpTSDjwcqtsceewwFhYX41u4Te+2vdg/ca9O8dg8pITRzuwCUGJQVs/Q9wJYfM6R7Jb93boceIELweeIJ4MYbgfT0mJerrq4OzzzzDH744QfcrHyxogJYvhyYNs2VstlOea/9/PmMlL2iomlg2r698Ve59HT/3QelRPzMDmDwIcepVcxqFTgZI793CxaEW89iHnoUFVND8Nm5M9zq4ERla7Qil1UOZ7/9Fr8eMgTbz55F9tatjSuH3bvDQWHkyNhXDrEIJS6HzqgYKfvy5eFj1Nx5Z9Pn8vPtHfX/2WfAc88BEyYAnTqFn1MLIZWVQFUV0LGj/nESvbDi9meOFyKBBINBAUAEg0G3i5IwCgqEAMJfrbxO2lJSwvcuJcWFi2/bFr74tm1CCCGOHj0qAAgA4pTseeVxdl5TU35++Dijj6lTnS2P1fceOhT+LIcOWTun3jXMntspWmU0en+2bWv8WLEi/L4VK5q+ZvdnnTrV3O+Z0Ud+ftPPKf2s9D7z44+Hv776qnOf2UV21t9s8SHHGOmCYcuPNWrjpeT3bt688MwuI/dz/nygvj66Pww7duyI9u3b49ixY9gDYLD1U0Vv2rRwSwEQbtWZNAl4/PFw89iKFcDFFzd+bcKEyOe00sUARNeq42SLjRutQWbu4e7dkc+n1zJy8cXnfs5OmTABKCwEXn0V6Ncv/Nz27eGWF/nvmVqLz/r1wCOPAIsXA3l5jc+r/Ezyn9XFF2t/5n+3uqJfP+c/u88x+JBj6uuNdcFIr9fXO1+meGBkvJQ0/V3+XKRzRSMQCCC7Vy9sOnYMxQAGSxWXG+MPZOf9YNs2DAJwvlQpqFWIUmWkx0oXw403OhtcIoUI6Wewe7c3xnqYvYdeJ/3e9OsXvrfLl4e7TYHIwUv62aSnM6S4gMGHHGOmBYEtPcYYHS9lpCXN7kHR2XV12ASgGAi3pMjZOf5g+fLw+yJU5N999x1yp05FAMCxgwfRDmjckmAmlMlbkeTvV/51L3+/kzOVzISISZOAqVPDn0FSWelc2bTo3cPHHz/XYgEAxcXhFjq/DNiVWmVefbXxc34dXB7nGHyIfEIvqJgNP6ZCj17rwpo1DWEiu0cP4LPPwsFHqsikCuzxx4HLLmvcumJwgHKTaxYWApdffq57QX4u2XsO7tmD3p07I3D0KNr93/+Fn1QGMsBYKLPSrSKVpbKyaQUebUuYMkQsXx6+L1oKCxu/7kYrg97nWrBA/XmvDNiN1MKWlhb+7717w1+lYBlti58UnvTKI4V5aUal0d+pRA5mNow58g0Obia/MjoIXO045XOmB5QbHCy89t+Dm4cYHbRpwzUbnVvjPae03nPDDdENhI00AFd63ewg2KlTzQ/UPXQoPKhVPrhVev/ixeHnpMf8+eeu5eQgYCMDqKV7JJVN+m+rg5SjHbSt9X4rA+b1fj/kn1V5fflnlY5T3guzv1Nq/+/ZOfEgBji4OYHFetAqeUM046Winv5usIsiq6ICeOQRlAAILV+OpJyccwOIX3216SBOq9cEGg8oBc79xarynpZ33hkeRLpnT+Opx5WVwLp19gyE1WsR6NOncReIdPwjj5zrJpO32ChbZ4y2SEn3Qzm4dc+e8LXUONmiYmYAtfxnKf9vsz+b9PToyq5V5khdnWlp4d/zZ54BvvwSuOsuIBQKH6fW4if9rlRUNH5Nr+VO7Wcl78KM1PVKDRh8fCbWg1bJG6IdLzVnzrnQY3q7EL0uivHjG/6R7Xn2LJrPmoXTdXX49sILkSH/x1caAGrHNaXzqVWIWu/JywN+8YvGTftqXQJW6Y25UQsdU6eGv0oVu7T2kZxdFdmECY3PLYVRgJWkUfLfRynkSt1baWnA6dPh/27RInyvQ6FzXVCvvto0+Eq0fjekn5eRn5Xy5xWLGW0+x+DjM24MWiX/izT93Q7NmzdH7+7dsXvvXhTv24eMaE+obEVRG5gsJ6sEiouL8ZOf/AR5//EfWCY/n52zrNLTw60j6enmBz9XVjb+y97JqdmdOmm/3+1KUn4P/bJthTLkyseNqbXKHDkSucVPTut3we2fVRxh8PEh2watUkKI5XYh2ZmZDcFnLNC4YjMrilVqN23ahJKSElxw3nnmr2uUslvFTHCxs7VJfn21e61cE0e5erXyHLFo8ZGH2htvbNzlIx8sLJ/27ZWWKCnkyrtx//rXcLepmnffDT/klC1+wLl7Ir83dv2suNVFIww+PuXVPZvIW2K9XUh2ZiYAoHjfvvAT0Yy5ULaimOiiGT9+PLp06YJme/dqh4xoQpkXqQWxK65Qn8kmUb5mdXyP2Yp15UpgyRL1c8lDrVQ+J2ZyWQ0DylDQrx9w993h4BNpMUOJssUP0A/6QHQ/K2510Yivgs/Bgwfx6KOP4s0338Tp06fRt29fvPDCCxg2bJjbRXOFJ/ZsIs8yO/3dFI3QkD1kCACg2I5uC2UFk54e/ku5sDBis//555+PH//4x+GKrbJSvSsl2oGw0YhF6EpPB1atUq/cIw0SN8tsxfrgg8C2berlitW4o2jDgNrPUG3smdEWP6sD+vXKY+TcCTjOyzfB5/vvv8eoUaMwZswYvPnmm+jcuTO+/vprtGvXzu2iuSqqQasUtxzfLkQjNGRdeikAoPibb0yczMQ1p03TX69GEqkrRe3csfzH3kjosiMc6X2uqVPDg77t+NxWKlY3t5sAog8D8p+hE0FfzsjPSu93ygv320N8E3x+/etfo3v37vjzn//c8Fzmv5vVtdTW1qK2trbh++rqaqeK55pYDFol/3Fru5CsrCwAQEVFBaqrq9GmTRt7TmzCzp078cGvfoXL33gDA9UOiFXTfrTBxekWKTsXrvNjxepmmc3+biTiIoMOSnK7AEatXbsWOTk5uOWWW9C5c2cMHToUK1as0H3Pk08+ibZt2zY8unfvHqPSxob8r/ra2vDXuXPDz1NimzfPeACeM8e++rVdu3bo0qULAKCkpMSek5r0j3/8A3e/8QYWjB0b7k6RHtK/FytWNH5+27amU8ntIAUXVljxz2yQ4e+Gq3zT4vPNN9/gueeew4MPPojZs2djy5YtuO+++5CamorbbrtN9T2zZs3Cgw8+2PB9dXV13ISfWA9aJTIqOzsbhw8fRnFxMYYPH27vyQ1UML169cLYsWOR99Ofqv/V7tUWCPIvtda5eBs8H0d8E3xCoRBycnKwaNEiAMDQoUOxc+dOPPfcc5rBJzU1FampqbEsZkw4OmiVKErZ2dl4//33USztHWQnA90/EydOxMSJE+2/drzwaoXs1XJZZUdXpZP3JN7utwm+CT7p6eno379/o+f69euH1atXu1Qidzg+aJUoStn/3sLCkeBD0YvVbDar3T9u8loYcPKeeOF+u8Q3wWfUqFFNxgyUlpaiR48eLpXIHW4NWiUyShrg7EbwOX78OM477zw0b9485tcmBT9WrH4sM5nmm8HNDzzwAD755BMsWrQIZWVleO2111BYWIi7777b7aLFlFuDVr1q3jzjg7nnz4//++EFUotPWVkZ6urqYnrtOXPmoE2bNnj66aebvui1v+aJyBW+CT7Dhw/HmjVrsGrVKgwcOBDz58/H0qVL8Ytf/MLtopGLpE1bI4UfqYswOTk25UpkGRkZaNGiBc6cOYN90grOMbJz507U1NQ0zCxrhDNpiAg+6uoCgHHjxmHcuHFuF4M8hJu2ek9SUhKysrLw+eefo7i4GL17947Ztd977z3s2bNHPfgQEcFHLT5EWubM0V7DiKEnLNZdgm4NcJZCV9u2bWN6XSLyD1+1+FDimjcv3E1lZCbb++8Do0ef6wZL9NADnLsXgP69kAfFaHBmFxF5FYMP+YKRinvOnHDoWb8e+OAD4zPgEkGsuwTdmNmVn5+PU6dO4Y477mgIXkRESgw+FFORWm7k5s8Phxf5TLZIFff69eHz19ebG8gsv1a80ruHdncJSsEjlttWvPDCCzh48CBuuOEGBh8i0sQxPhRT0czCMjKWJy8vHGBSUsJfOeOrMbV76MQ4qL59+wIAqqqqUFVVZc9JdYRCISxatAjTp09HTk6O49cjIh8TCSQYDAoAIhgMul2UhFZQIAQQ/mrH69L3eXnqz0dzLS35+cbfU1AQPt5LpM+dkmLt8xuRkZEhAIiNGzfaf3IiSih21t8MPuQKrcBhNIgoK25l6FEeF821zJTf6nFukO5dSooz5x87dqwAIJ5//nlnLkBECYPBxyIGH2/RarkxGhKkijs52VirTjTXMlJ+s6+7KRYtPvfdd58AIB5++GH7T67w3nvviQMHDohQKOT4tYgo9hh8LGLw8R6rFbD0Pin05OUZO97uyj7alis3RBs4jfrxj58VAMS4ceMMlclqd2Btba1o0aKFACBKS0utnYSIPI3BxyIGH28y2+Uir6jz87W7udTel5Rkf/dOrIKEHWIZ1P77v/8pAIgOHfpYKpNR5eXl4uKLLxadO3dmiw9RnGLwsYjBx3vMtvhEU3E72b0Ti66jaMW6a+7gwYMCgACSxdy5NY5f88yZM9GfhIg8icHHIgYfb4nUUqKcOaVXSRYUCJGbq/+6060yTg8WjoYbg7FDoZBo3br1v8PPTl91BxKRt9hZf3MBQ3KF2toxygX25Ks1S8+rrTUjP1deXtMF+oxcy+r6NdKCjABw5kx4/aAzZ8LXlM4pPybSIonRLqSotUCk2irWateSXq+vt3Z9uUAggOzsbGzduhUTJxZj7tz+DdfgHmpE5BobgphvsMXHPtGsY2Omy0X679xcYy05yus53b1jZMaYkfWE7CiLmXPEqrVl8uTJAoBYuHChI92BR44cEenp6WLChAmirq4u+hMSkSexq8siBh/7WK1grbzP6rieWIQArVCjVv5Yja/x0jT7BQsWCABi8uTJQgj7uwPXrl0rAIj+/fvbc0Ii8iQGH4sYfOxlpYK12lKkHL8TabxPfr7zqytHCmfKwBPLGVWxvJae119/XQAQl1xyiSMtPj/88IPYsGGDePPNN6M/GRF5FoOPRQw+9otVBSsPEXoVZ6Tr2hWG9IKOXhdXLAZZRypPpDWP5O+PdquNr776SgAQqaltBBCKyecmovjD4GMRg48zYlWZK8NPpHIomVnzJ9JxWgFKrVVDa4xTLKa+K69lZs0jO8pWU1MjAoEkAUA88sghR65BRPGPwcciBh/nqFXmdnc1yYNPpNYWvfdHqvyl1422jCgZGccSy6nvymvFcgxQ+Fy9BQCxfv16W6/19ddfiyeffJKboBIlAAYfixh87CcPN5EqWK1wYya0qA0WNlN5Rgo/0YYeI605brb4RGqVc2KQdd++4wQAsWzZMt3jzF6zsLBQABB5Vn9YROQbDD4WMfjYTxkktCrYSLunGw09yucizZYyUmbpvXaFHr1w4YUxPrEabySF4oceekgAEPfdd59uWc2OJ3r77bfFLbfcIn73u99FVU4i8j4GH4sYfJyhDAzKClQrUFgNPRIpZFmprJXhx+hmp2bLqdZS5WQri5nyyL93svVpxYoVAoC45ppr7D85ESUEBh+L/BR8nJ6KbRet1hPl81qvGx2TIycfpCwPP3l5Te+D3r1RtholJ1u4AQY+i7Jrzso5nCiPVhel3TZs2CAAiB49ejhzASKKeww+Fvkp+BitCN2cGROpq0SrJcho64LWLCyt8yoHVRu5N4GA+S4zeZgyE+Ciad0yyuzvjVYXpZ0qKysFAAFAnDp1ypZzHjt2TJw4ccKWcxGR9zH4WOSn4COE+b/cYylSV4rW9Gll64KRli15JS0tZCg/rzJYGA0aPXuGjzETfpSf20j55UHMyMy1aFrvzLQUWmmFs6pDhw4CgNixY4ct53viiSdEUlKSePTRR205HxF5G4OPRX4LPkLEdlyIGXoVrNrsLul4ZeuC0c8hVdJqFbXyXGrHKkmhp2fPpufSel+ksvqle1K6fix/r0aNGiUAiFWrVtlyPmkPsGeeecaW8xGRtzH4WOTH4COE87Nv7KTVnaX3GcyMkdELPkI07gbTOq8y9AjRuFtNLfwoy2t1Wr6Z45ziRkvi7bffLgCIfBvT3sGDB8V3331n2/mIyLsYfCzya/ARwvz4GDdohRsjU9kjtUBIu7PrtRrJu9e0xuCohR61sqq1JJkJaV7qnpS3RBkpn7x1zq5y/uY3vxEAxMSJE+05IRElFAYfi/wafKSKyMjsG3lrRCy7XrQqykhr4+iFCq1zqt0HvXsjnUeasi6FHuX9UQs/SUlNxxdZbdFxq6VHfl293wm1+29XA420i/qQIUPsOSERJRQGH4v8GnzkXTpmNuiMVddLpIo+0hYRauFH63MaafFRu470mnzKulq55WWWgpL01cw98lr3pNstUaWlpQKASEtLE/X19VGda9GiReKOO+4Qn3zyiU2lIyKvY/CxKB6Cj1Z3i9VWBqdCjxDWuliE0G690QoTkQKGXjDSO95q6DFyXTe42RJ19uxZ0bx5cwFA7Nu3L6pzDRkyRAAQr7/+uk2lIyKvY/CxyI/BR6sryI5xJ3ZUeE5tRGpkbyl5QJF3pendG7XWJ71ApRzvY1YsNyM1ws2WqP79+wsA4q233orqPGvXrhWPPfaYOHToUOSDiSguMPhY5LfgE6llR63Fw+i53O56UaNVxkhhRW0ckV5rkN75lCtNS6HFyv3yWouPxK1yjR8/XgAQS5cujc0FiShuMPhY5Kfgo1fRKisuo5WXVytiIbQ/r16okR8baTVnI8FPuW+X1lgiM/faq0HTjZaoWbNmCQDirrvuit1FiSguMPhY5KfgE6kLSV5xGelCUr7P6L5UZs5tVaRxQsoWGq17o7YvWEGB9mwstdYztTE9ZroV9T6PV8KPWwH45ZdfFgDEmDFjLJ/jnXfeEV999VXUA6SJyF8YfCzyU/DRY7XiUlbukXYij0VFbfQakcKRWneW2vuUQU76Pj9fiMxM7TE9yvCTm2ttEUMn7qmVbSq07o+SncF38+bNAoBIT0+39P5QKCS6dOkiAIgNGzbYUygi8gUGH4viIfhY7ULRGuRrZH0dJxmttKWWHyNhQ2vjTSPdh9I90RtfJQUkIy06asHB7ntr9HzSfZEWcYx0X+wu5/Hjx4W0Wenx48dNv7+6ulrk5uaK888/X/zwww/2FIqIfIHBxyK/Bx+rFZSZ8TNGzucGo59ROU5Hvl+Y0dCj162ltw+Y1oKIWte0swsx0v2J1NITqzFJ6enpAoDYvHmz5XOwm4so8TD4WOTn4GM13ERbIbrJbJCQWmKkFg2tHeKV7wPObYmhvI7afxtZxdmN+xhtwFWGRyfKnpubKwCIl19+2f6TE1HcYvCxyK/Bx2p3ltkuEKmVxAuhRwj97ia15+ShR6rk9br0jLQCyQOPkbJEOq/T9MKMkeOdDD1CCDF9+nQBQMyePdv0e0OhkAMlIiI/YPCxyK/Bx+oigWbep+wacouRVh61lhd5S4VyjI9WZR7p/kjXkc6Tm6t9jNNdRGYoy200MMdiltfSpUsFADFhwgRT76upqREdOnQQeXl5lsYHEZG/MfhY5Nfg47RYVnxGyxIp/Ch3UVeGG+U6NVYDidGNYb1y/yRGyi0PfpGOt2tM0ltvvSUAiP79+5t638cffywAiE6dOrHlhygBMfhYxODTlJdbLLTCj7zrSa2LThlErE7dNhNovLQ1hdFyK8f0RNoY1o7fiX379gkAonnz5uLs2bOG31dXVye++OIL8eabb0ZfCCLyHQYfixh8GvPiGBW9MigDj5FwozZ428iUeDOB0EstPmaDrNnZXtGqr68XaWlpAoAoLS2156REFPcYfCxi8DnHSOuH1QrPro1LtSph5UDjSLORzK5ZZCYQeqnFzGyQ1bpvke5ntAYPHiwAiLVr19p7YiKKWww+FjH4hMkrQr2QolapGxnnYbTyN3KcsjVFLfgIca6SzsxsGkzk214o9/xSTks3EwjNBA27d7HXK5eR1yN9HyksRuNnP/uZACAWL15s6PjDhw+LWbNmiXXr1tlfGCLyBQYfixh8wtS2eIg07sVsS4aVFiWtcKAMPcpFBJUtFFrBRF6Zy1uPzLbWqL030uezMwxafU+klhxlyHSqxSc/P18AEFOmTDF0fFFRkQAgBg0aZG9BiMg3GHwsYvBRp6w47VqB2GrXi9q15aFH+Zza4oNawUQ5G8xKq4x0DbXp7WrHGRlcnZ/fdCsJvfPK77nZ1qQrrtA+Xm02nN2b1K5atUoAEKNGjTJ0/ObNm8WUKVNEgRv9h0TkCQw+FjH4aNPqwokUeox2VUXTlSSFDK0WHa1uL73sEJr0AAAgAElEQVRwJa/czbKyKWikzy+Fnp499c9npUXIKGWLj/IadnXX7dixQwAQHTp0iKa4RJRAGHwsYvDRpxU6rIQete40venSatPItbqi9J7TC23K0ONUgFAyOp4mUouPkdBjNZwYGeskb8HSawWKVM5Tp04JabPSyspKY4UlooTG4COEWLRokQAg7r//fsPvYfCJTBlSzHRV6R2jtc5NpO4ttW4ireAQCGifS/553NiXTCv8SGsQRSqT2XE8Zo7TCrlq5zI7Q05LRkaGACA2bNige9yxY8dERUWF/smIKO4lfPDZsmWLyMzMFIMGDWLwcYA8pCgrskibc8ppbSFhpFVBXiGrtc4o3ysFiORk7c+jVmG7GX60ymy2ezDSdfRej9StqfZ6pMHRRso5duxYAUA8//zzusctW7ZMABCTJk2KfFIiilsJHXxOnDgh+vTpI959911xxRVXMPhEYLbrQwo28rBhpBVIeT3pPcpNQ5WVp3KAsLK8eisiS101Wq0n8utF021nJ2VZtTaGjdQ9aPQ6euEkUtBRPqcXxszew/vuu08AEA8//LDucY899pgIBAJi7ty5xk5MRHEpoYPPbbfdJmbOnCmEEBGDT01NjQgGgw2P8vLyhAs+Zrs+tCozvVYgrfMoW3yUXVNGu8v0Kn+tbi/l7C/591bCj53r8ChberSuHe02GJHCidZn0nqfMqRGE86klpxx48ZFPDYYDIqqqirjJyeiuJOwwWfVqlVi4MCB4vTp00KIyMFHWi9E+Yi34GN0l3Hpr3xlpaw1xkP5mlorkLIS1WphUWs1Mtodo/a91nNCqG9gqnyP2hgVrXtjZh2jSMdJZdPaX0zrflltjbJ6HqPvsxrO1q9fLwCI3r17m3sjESWkhAw+Bw4cEJ07dxafffZZw3Ns8QkzUilrhRu90KP2eqTxIfIWHuU5lWNtInXHyNfnUX5OtaAghRZ5WQKBxueQr5ejNt5HHnyMTkVXK58ao/timb1uJFbDiZEd262Gs4MHDwoAIikpSdTU1JgrGBElnIQMPmvWrBEARHJycsMDgAgEAiI5OVnU1dVFPEc8j/ExWimrVahGun8i/bc81EiBRSsYaQUWvWvqfQ75gGv561KrilpXmNbYI7UuMyP32Wzo0Xqf2etG4lSLj5EWOT2hUEi0bt1aABA7d+5UPebpp58Wt956q3jrrbeMnZSI4lZCBp/q6mrx5ZdfNnrk5OSISZMmiS+//NLQOeI5+AhhrFJWVmjKVhW9c6m1qhQU6I//UQYeIy09kbqztIJOQUHTWWfy1iflmCN5+DESPrTKZDR0yhcnVFvnSF6uK65oet/Nhgur4STS+6yGP6Xhw4cLAGL16tWqr1999dUCgHjuueeMnZCI4lZCBh81nNXVlJFKSmoJidSFIQWLSFtSKFsGtFp65NO21abFa41VUps+rQw5eq1X8teULTzKlh+jm3MabUkx2n2ltYihXeEl0vNG3xdp/y4z4Wfy5MkCgFi4cKHq6xs2bBCLFi0Se/bsiXwyIoprDD7/xuCjTq9S1upyklMOVI5UyUmzfbQCj7zbSa0lSK+SNBocjAYfvbChNh1ej5GxM3obwsq/V9urSy+ERLNystVQZDYURgo/CxcuFADE5MmT9Q8kooTH4GNRogQfIdQrZa1gYrSry8jryqAib+kxE1b0jtdbCFEvXKi1UClngKktgKhWLrX1jozQKp9dA5q13hdp41kjrXz5+cZ3bI8UzoQQYvXq1QKAGD58uP6BRJTwGHwsSpTgoxYQ1CpEreBhNfSondfotG2tNWKU11ALdFqVulaLV1JS43PIB0EbbYGKJqRodXPZsZWG0bV5pOekcGK0lU/5vmh89dVXAoBo3bq1CIVCjV575513xKZNmxqWriCixMbgY1EiBB+rrSp6AUnv/Gq0dlE30uWjdw29QKMkHZOUpB+KlN1cZu6VENYDg7IcWtt62MmOn62dampqRFJSkgAgDh061Oi1gQMHCgDi73//e2wKQ0SexuBjUbwHHzOtOmrvM9LiEWmxROXKyWqzwPS+t2P6tPJcWgFMCj09eza9B1Zax4yUSf6asvUq2pWajYgUKmMVeiS9e/cWAMT69esbnquvrxfjx48XnTt35galRCSEYPCxLJ6DT6SxHZEqNnkLSbRlMDMNXN5lYmYHd63n9Vq81LqXpPCjDC9az0UKNUbKJH9O2eKjtXeX2jWNdDepBVWte5SXZ08Xlhnjxo0TAMSyZcuavKbs/iKixMXgY1G8Bh+jf63b0YUU6zKY6Z7ROlbZ9aa1n5dWWALU1zvSmrGl1tIlDxXKckZaV8joPTRzrFbwinWLz8MPPywAiPvuuy+2FyYiX2HwsSheg080G2ia6UKKdRnMzCDSapFRnkt6KBdS1GspkmZ6mZ0ZJc38kn+OSJ9Va2NXrWsaofUeKfQYbWVywvPPPy8AiLFjxzY8d/bs2dgXhIg8jcHHongNPnJmAoidlatVkVokjKypIx2rnBkmhHqrjFbrllpLjTQDzGg3lrK1yEjoiRSCog2letcyu3aR3TZs2CAAiIyMDCFEuHura9euIicnR+zbt8+dQhGR5zD4WJQIwcdo5Wh2WwYnmOnG0mO2tcnMytXK75WvKde2UbYUKZ9XW+xR7dpGW4DMcPLcVlVWVgoAAoA4deqUKCsrEwBESkoKNy8logYMPhYlQvARInJgsHsFXifKaKUM+fnae48pzyeFEqPrB2mFH62uNGVLinKLDr2wJrU8Kctqx89BaxB1LFv5lDp06CAAiB07dohQKCT27dsn3n777dgXhIg8i8HHokQJPkLY04UkHe/ETJ9oxgRpnUdvrI8QTVs4lMeaaYFS+2+10BNti4qdU9yVQcro7DmnjRo1SgAQq1atiu2Ficg3GHwsSoTgY3RvKOn7WE9ftpveGBt5BR5p+rrZgdTy98rXCwoEGocKo61reteyOttO+l2Qd8dpLZSoPC6W4WfKlCkCgMj3+y8jETmGwceiRAg+WkHA7DRxq+xqyTEjUvjRCj3K91tpBZOHHrXNTq22+EQKrWbOEWnnd7UxP7HMIIsXLxYAxE9/+lNx9913i1deeUXU1dXFrgBE5HkMPhYlQvARomklqewu8dvYHSvXVXY/aYWeaMqjvIY8PCjH+Jg5v5FxRmbLKF8jSE4rFMXS2rVrBQBx0UUXCQAiPT29YfFCN4I0EXkPg49FiRJ8hGj613wsF6ozM1bGyesqx/JEuqaZilO6Vm6uEJmZjc+vtT5OQUH4eL1rOHHvIrX4aIWiWCktLRUARGpqqpg5c6aYNWtWw2tuBWki8hYGH4v8HHys/OWrHF8SzXgTs2XSGk/kdAWl7NqTbz5q1x5YWgOb5UFLLWjaFWqstB5pTWH3wpT2s2fPiubNmwsAqmv3uBWkicg7GHws8nPwMVspag1mtbOCk0/R1tsTSrqm1DLi9BgStdBjpMXHCHlLj9b6PcrnInWzSZzo1lEb7C7dF7UWKbe6ivr37y8AiLfeekv1dTu7/4jIfxh8LHI7+ERbsRn9y1cZbpwa46PW2qE8p9pWEbFo8ZGHHmUwsXpttZYe5Rge5WeUf361laVjTT4Y285WsGjdcMMNAoD4/e9/r3mMHQO+icifGHwscjv42NGVEekvX61tDpya1aUXfpStINEuxBcpOCrvgTLoRBt+tPbqUmv5ycs71yJmV2tTtCK1+Lhp/PjxAoDo2rWr7nFav89EFN8YfCxyO/gIYc94Bb2/fN1Yx0ct/ERayTia6+gFQvl1lRuRKstqR6WpNm5Kr9vLrYo60lgftwPEL37xC0PBRwh7F3UkIn9g8LHIC8FHCHvGK0T6yzfWYyLklb1ybI0T+03ptbxoBREnwo/aon/y88srZ7fG0ERqDfRC+Nm8ebMAIC644AJDg+blY9Y4hZ0o/jH4WOSV4COEPeMVtP7ytaNVyQplS4dad4od11Yba1NQcK5rSa1VSe1+R5pabqVM8jE0aqE01uvSRAo3Xgk/x48fF9JmpbNnHzcU2u2epUhE3sXgY5GXgo8Q0Y1XiHbsjhPhRzmmRbl9g/La0Qz2lZ9DrStJa5aZWpCIFDCMhhV56DHa3Rjp89kRECOFGq+En/T0dAFAbN682VA3rRfKTESxweBjkdeCjxDWxisYHeNj5DxXXKF+vNngoAw9Wi0+8vLaVbGrBUc7A4aZY9Q+l9nwY1cole/VZeQ+yI+PpRdffFFce+21DVPaX3755YYy6XXR2dmKSETexuBjkdeCj5UWH7vH7hg9X6TjpId8FWPlGB9ly4wdlaxecLQzYOgdK78HagOq1d5v98/Rz37+858LAGL48OECQKOVm5XdWbxvRImJwcciLwUfK3+1OtVSEKkyiTQ+RN7KoQwBav9tV+VkJDjaWVHqBRrlWBP54o7KY5Xdc9IAXa3zx/vg3S+++EL88Y9/FL/61a8EADF+/PiG1+SDxyP9gZAI94ooUTH4WOSV4GOlMnZ6bIjW+7QGkOqFHmWlrmwBsiP4mAmOdnaNqH3GSPdHGWTkoUi5qamV7rp48dZbbwkAon///k1e4xR2osTG4GORF4KP1VabWMwG0goIWmMspG4tZehRnk+q1DMz7anMowmOdix8pxb6Ih2n11om3R/l1PdECj1CCLFv3z4BQDRv3lycPXu24Xk7f3ZE5E8MPha5HXycbrWxg1Ylo/a8PIxpBTP5vllSGIumS0J5b7RWU1Z7XS1gKM9ttFyRzqUsj3IGmzI8ye9rIoWe9957T7z55pvi+++/F/X19SItLU0AEKWlpUIIe1vriMi/GHwscjv4xHoNF6vXl4cVteeNdjfY/Ze6XquS1nPSf0sbhWpt02CmQjX7uSK1pKl1nSVKxT5mzBgBQKxYsUIIIcSQIUMEALF27VpLLXtEFJ8YfCxyO/i4zUiFodWNY1dlH01lpRXctMKPFByl0NOzp/WySdc28l610Kq8f2oz3dTWAYp3d911l+jTp4/48ssvhRBCTJw4UQAQY8f+xlKXMBHFJwYfixI9+AhhbByM1gBlo0HBjb/Utc4tlV0KP5HGL0U6f6TPpfcZ5S1pymPkrWmJXKnn5+eL8ArOt3u6S5iIYovBxyIGnzC97iHl85FmdZkNN7EMP5HWgDGz5UGkbiijr6tt1qrWmpaolfqqVasEAJGRMcrQ8U50CROR9zD4WMTgc45aK4VWGIq0jo/Z7qxYhB+t3eD1upyMlDdS4It0n5Sbmurdw3iv1E+fPt3kuR07dggAokOHDi6UiIi8isHHIgafxrTG88hfixRqlJW60TDjZKWuNThb+bo0SDs3V3/fMPnnKihoujihXliJNiiZ4cb9j+aa/fr1E3379hWfffZZw3OnTp0S0mallZWV0ReQiOICg49F8R58rFRCSUnqM7Xs3OQzltS6jSK9bqWlKtJ1lMdrvWblvUbKZ8dxTl4zGAyKQCCgGnB69OghAIgNGzZEX0AiigsMPhbFe/AxWwkZ3QrALyK1UkXqVjI7NsnqPmFGr+tE+HGim9HqNauqqsQ///nPJsdfc801Qj7FnYiIwceieA8+QhivhMzO1PK6SF1KRsYpGenOUz6nFRr1Wt/UFl3UajGz0ppm5nPYxc5r3nfffQKAeOihh+wtJBH5FoOPRYkQfISwJwT4SaRyR5q9pRd+9EKPl0OjG2W065rLli0TAMS4cePsLyQR+RKDj0WJEnyE0K6EIs1i8mIlrsdqC5feefRac9xoTbHKyDgkN65ZX18v7rjjDvHHP/5RnDp1qsnr69evFwBE7969nS8wEfkCg49FiRR8hGhaCZldrM9LlbiWSAO6lTOy9LqN5K+rjd9xY/xMtNzY1TzSNXfv3i0AiBYtWogzZ840ef3QoUMCgEhKShI1NTUOl5aI/MDO+rsZKG7NmQMsWACcOQOkpACjRwO5ueHnI70PAOrrHS9i1ObNM/660c89f/65e3bmTPh7AJg7Fygo0D6P9Pzcucau5zS1z+F0mYxcs1WrVsjPz8eJEyfQvHnzJufo0qUL2rRpg+rqapSVlWHAgAHOFpqIEosNQcw3Er3Fx0stEV6l1UWYm2v8/lkZkGw3P4/xEUKISy65RAAQr7/+ur2FJCJfYleXRYkUfLw+ANcrix0qr+OX8TtyarPE9D6H3mKNVtl97yZPniwAiAULFthXSCLyLXZ1ka7585t2y3itGyY52VhZ5J/FSWr3TOK1e6ckv5dA5C6599+3t9vLzL17+OHT2LlzJwYPHqzazSXJzs4GAJSUlNhTSCIiiQ1BzDcSocXHTwNwvVJWo9fx0r1TksoW7aKJVq9r9N798pf/EgBEnz59dI9fvXq1ACCGDx8efSGJyPfY4kOq9P7ylphtvZg3L9yiYKR1YP788IDoSAOOjZTFyGexS329set4edD3nDnA+vXh1hwlrXtpx+cwe+8+//wIzj//fAwZMkT3eKnFp7i4GEIIBAKB6AtLRASwxSeeODFuJhatIV4fj+QnfriXoVAo4v+DNTU1Ijk5WQAQBw8ejFHJiMirOLjZongPPk6JRZcUZ6DZJ17uZZ8+fQQA1f28iCix2Fl/J7nZ2kT+MGdOuDtj7txza9pI7OqSmjPn3NovKSneG0DsJ/FyL7OysgCEu7uIiOxiKficPn0aBw8ebPL8zp07oy4QeZNa+LFzHI7WooFknlfv5erVqzFq1Cg888wzho7nzC4icoLp4PP666+jb9+++PGPf4xBgwZh8+bNDa9NnjzZ1sKRt8jDT2qqvaFHOldtrXbrEkXm5Xv54YcfYtOmTSgrKzN0vHyAMxGRbcz2jQ0ePFgcPXpUCCHE1q1bRf/+/cXKlSuFEEIMGTIk6r43PYsWLRI5OTmiVatWolOnTuKmm24SxcXFht/PMT72sHP/J78uGuhFXr+Xe/fuFX/5y1/Eli1bDB2/ceNGAUBkZGQ4XDIi8jpXp7OfPXsWnTp1AgDk5OTgww8/xIQJE1BWVub4lNMPPvgAd999N4YPH466ujr8v//3/zB27Fjs2rUL5513nqPXpjA793/y86KBXuOHe5mZmYnMzEzDx0stPgcOHMCpU6f4/zgR2cNsUsrNzRWff/55o+dqa2vFxIkTRXJyctRJzIyjR48KAOKDDz4wdDxbfKJj51TpeFg00Cvi+V527NhRABDbt293uyhE5CJXWnxOnDiB1q1b45VXXkGzZo3flpKSglWrVuGee+6xNZRFEgwGAQDt27dXfb22tha1tbUN31dXV8ekXPHI7m0w4mHRQK/ww7384IMPcOTIEYwePRrp6emG35eVlYWqqioUFxdj6NChDpaQiBKG0YQ0ePBgUVFREXXSsksoFBI33HCDuOyyyzSPyc/PFwCaPNjiY45XtpYg/7r11lsFALFo0SJT75syZYoAIPLd3u6eiFzlyjo+OTk5uPTSS5vMsNixYwd+/OMf2xbEjLrnnnvwxRdfYNWqVZrHzJo1C8FgsOFRXl4ewxLGB6PbYHhp9hB5T//+/TFs2DCMGjXK1Ps4s4uI7GY4+Dz//PO4/fbbcdlll2Hjxo0oLS3FrbfeipycHKSmpjpZxibuvfderF27Fv/617/QrVs3zeNSU1PRpk2bRg8yx0w3SkGBfjfKvHnGg9H8+cb3/CLvy8/Px6efforLL7/c1PsYfIjIbqZmdeXn5yMlJQVXX3016uvrcc0112Dr1q24+OKLnSpfI0II3HvvvVizZg3ef/999OzZMybXTWRmwkekcJScbGw8kLyViRKbfBHDUCiEpCQuNk9E0TEcfCoqKvDkk0/i+eefR//+/VFcXIyJEyfGLPQAwN13343XXnsN//jHP9C6dWscPnwYANC2bVukpaXFrBxkjZHB0LHclZ1i48SJE2jVqpWl5S4yMzORkpKCmpoaHDhwwNR0eCIiNYb/fOrVqxc2bNiAv/3tb9i2bRuKioowY8YM/PrXv3ayfI0899xzCAaDyM3NRXp6esPjr3/9a8zKQNGJxb5f5C3XX389unbtin/+85+m39usWTP07t0bALu74lZFRbhpuaLC7ZJQgjDc4vPnP/8ZEydObPj+mmuuwb/+9S+MGzcO+/fvx7JlyxwpoJwQwvFrkPPUWn4YeuJTfX09vvjiCwSDQVx44YWWzpGdnY1du3ahpKQE1157rc0lJNdVVABPPAHceCNgYqkDIqsMBx956JFcfPHF2LRpkyuzusjf5OFnwYLwKtAMPfEnOTkZFRUV2L59O/r27WvpHBzgTER2inqkYGZmJj766CM7ykIJZs6cc1tfpKQw9MSrtLQ0jBo1yvLAZAYfIrKTLVMkzj//fDtOQwlGbd8vIiUGHyKyk+lNSonsoBzTI30PsOUnnkybNg2ZmZmYNm2a5tYykWRlZQEADh8+jOPHj6Ndu3Z2FpFiqaKi6SDm7dsbf5VLT+e4H7Idgw/FnN37fpE3VVVVobCwEAAwffp0y+dp06YN0tPTUVFRgZKSElx66aV2FZFibfny8EBmNXfe2fS5/HyuZEq2Y/ChmNKbvcXwE1+Sk5OxZMkS7Nu3L+ru8OzsbAafeDBtWnj2ltz27eHQs2IFoFwXjq095AAGH3LEvHnhlZrl4UUr9MyfH97qYt48hp94cv755+OBBx6w5VzZ2dn417/+xXE+fqfXdXXxxU2DD5EDuP47ObKHlrQ9hfy8avt+SWEoOfncc0b2/aLEwgHORGQXtviQI3toqbXcKAOTkW4v8qdQKIT3338fw4cPR+vWraM+H4MPEdmFwYcc20NL77xcqTm+7d69G1deeSXatGmDY8eOIVnepGeBFHzKyspw9uxZNG/e3I5i+kdFRXhg8LRpHPdCFCUGHwLgXEjh9hSJ6ciRI+jRowd69uwZdegBgG7duiEtLQ2nT5/G3r17La8C7VvxvK1Denp49lasPhdDZMJj8KEGToUUbk+RePLy8rBv3z6cPn3alvMlJSUhKysLn332GUpKShIv+MSz9PTYTlmP5xBJhnBwMzUi3z09NdW+lhluT5GY0tLSbDsXx/kQkR0YfKgJJ0IKt6egaDH4EJEdGHyoCbtDiry7rLb2XIsSw098Wr9+PQYOHIjHH3/c1vMmTPCpqAgv6qd8AOrPf/ZZuKtIuRWE1WvbdS4ij+IYH2rE7j20EnF7CrXFG7XIF2+MF5s2bcLOnTsxaNAgW88rBZ/du3dDCIFAIGDr+T3D7LYOU6cChYX2jFmJt/Ev3BuMVDD4UAO7Q0qibk/hxLpIfjJt2jQMGjQIHTp0sPW8ffr0AQB8//33qKqqQqdOnWw9v2eY3dahsjIcfKgp7g1GKhh8CID9IcXIbLB4DT9OrYvkF506dcKNyorbBi1btkSPHj2wf/9+lJSUxG/wMbutg1rLBYVxbzBSweBDjoQUte0p9M4bb9tTcPFGe9TX12PDhg2oqKhAeno6srKysH//fhQXF+Oyyy5zu3jkddwbjFQw+JAjIcVMa3G8Vv6JuHjjxx9/jJ07dyI3Nxe9e/eO6lxFRUW4//778e233zY816pVKwAJMMBZS2Vl0xYeM2NW5Iv3Sd9bPReRX4kEEgwGBQARDAbdLgolkIICIQAhUlLCXwsK3C6Rc2bMmCEAiAcffDCq86xevVoEAgEBQPUxbNgwm0rskEOHhMjPD3+1w7Zt4V+eqVPDX40+pDJIX6XzbNsWfs7sueKB/B6Qb9hZfweEEMK11BVj1dXVaNu2LYLBINq0aeN2cSiBpKaeWyKgttbt0jjnueeew9/+9jfcf//9uOmmmyydo76+HpmZmY1aepSaNWuGmpoaW7bDcMT27cCwYcC2bfZ0p0gtNT/5CRAKNb2W3piViopzZQHO/bf0mplzxUOLj90/G4oJO+tvdnUROUxtXaR47OYCgLvuugt33XVXVOfYsGGDbugBgLq6Ovzzn//E2LFjo7qWbxjZ1kFrzIrWmjyJOv4l1nuDkedwAUMiB3HxRvMqDC6e9/nnnztcEopLUohk8ElYbPEhckiiLd743XffoV27dlF3P6UbrJB++OGHqK4T96TF+3bvDn+/ezcgbRqrNXiZKAEw+BA5IBEXb5w6dSreeecdFBYW4uc//7nl84wePRrdunXDwYMHoTcEMSnJIw3WXlodWF6W5csbL2w4adK5/9ZavM+B9ZeIvIbBh8hmibp4486dO3Hy5El069YtqvMkJyfj6aefxs0334xAINAo/Mi/Ly0tjeo6tnF7dWD5mBW9sqiZOvXc1HYpjHH8C8U5zuoislmi7tVVX1+P3bt3o0+fPkhNTY36fGrr+HTv3h0///nP8Zvf/AY5OTnYunVr1NeJmlaLjxuzo/TKsngx8PHHQFER8OqrQL9+8TNTi+KenfU3gw8ReZZy5ebRo0ejpKQEAwYMQKtWrVBdXe3NzUq9NGVaKsurr57r7vJCuYhM4HR2IkoIycnJyM3NbfTcRRddhOTkZJw8eRIVFRXo2rWrO4UjIl/yyOhAIvKze++9F7NmzUJ5ebnj10pNTUWvXr0AJPDWFURkGYMPEUWltrYWK1aswFNPPYXaGC1LnZ2dDcDB4FNRER54ZXBNIV+ZOpXjeiihMfgQUVRCoRCeffZZ3HPPPbjoootics2YBJ8nnrAefLRWB45VoKqoCI/t2b49vLHp1Knnrjl8eOPXt2+Pz4BHpIFjfIgoKmlpaZgyZUpMr+l48ImW1hYTUqC68UZnW13cnmJP5GEMPkTkO1lZWQA8HHzcNm1a08UII02xJ0oQDD5EFJU33ngDF198Mbp06RKza0otPuXl5Th16hTOO++8mF3bF2K9Aam0e/y0aQxR5HkMPkRk2eHDh3H99dcjEAggGAyidevWMbluhw4d0LFjR1RVVaG0tBRDhw61fjIvbTnhV7HqwiOyAYMPEVl25MgRDBo0CIFAIGahR5KdnY2NGzeiuLg4uuDj1HgYBioiT2LwIYnh98EAACAASURBVCLLBg8ejM8//xxnz56N+bXlwScqTo2HSaQBxpWVjb8SeRiDDxFFrXnz5jG/pm0zu5waD+O1AcZaU+ztUFXV+CuRhzH4EJElQghX98ny/MyuWA8wNlIeO1qU1LrwpJ9BcXHTbjx24ZHHcAFDIrLkiy++QI8ePXDHHXe4cn2pxae0tBShUMiVMiSk5cvDm57KHwsWhF9bsKDpa8uXu1teIgW2+BCRJR9//DEOHDiA/fv3u3L9zMxMpKSkoKamBgcOHEBmZqYr5Ug4al14a9aEQ8/jjwPjxzd+ja095DEMPkRkyaRJk5CVlYXk5GRXrt+sWTP06dMHO3fuRHFxsb3Bx8nxMETkKnZ1EZElrVq1wpgxY3D55Ze7VgbHtq5ITw+3bCxfbm0fK609ueIhULGri3yOLT5E5FuO7tkVzaJ8Wu9VDjD244rH7Ooin2PwISLTduzYgfXr12PMmDG4ONazk2Q8P7MrEj+ueKw2S2v37vDX7OzYz1YjMoldXURk2j/+8Q88/PDDWLp0qavlkFp8SkpKXC1HwuvYsfFXIg9j8CEi0wYMGICbbroJV199tavlkFp8Dh8+jOPHj7taloTWqVPjr0Qexq4uIjLtlltuwS233OJ2MdCmTRt07doVhw4dQklJCS699FJrJ4pmXy3uyUXkK74LPsuWLcPixYtRUVGBAQMGYOnSpRg9erTbxSIil2RnZ+PQoUMoLi62Hnyi2VfL7HunTg0PEAbiJyDFw2w1ShgBIYRwuxBG/fWvf8XkyZOxbNkyjBo1CsuXL8fzzz+PXbt2ISMjI+L7q6ur0bZtWwSDQbRp00bzuPr6elc2XaTE1Lx5c9fWwrGisrISrVq1QlpamttFAQDcfffdWLZsGR577DE8+eST1k6i1Wqjt69WpBYf5XuXLwcKC42Xyc+blhLZzGj9bYSvWnyWLFmCKVOmNCyRv3TpUrz99tt47rnnrP+DJyOE4FgBckW7du3QpUsXV/e+Murxxx/Hiy++iMWLF2PmzJluF8eemV3R7Ktl9L3z5p1r6ZEoA9Lu3cCkScCrrwJ5eY2P9ePUdyIP8k3wOXPmDLZt24bHHnus0fNjx47Fpk2bVN9TW1uL2trahu+rq6t1ryGFns6dO6Nly5a+qITI34QQ+OGHH3D06FEAQLoPKrSvv/4adXV16NGjh9tFAeCjmV1mwlW/fk2PNTv1nUGJSJVvgk9VVRXq6+txwQUXNHr+ggsuwOHDh1Xf8+STT+IJrb53hfr6+obQ06FDh6jLS2SU1GV09OhRdO7c2fPdXu+++y7Ky8vRvn17t4sC4FzwKSsrw9mzZ9G8eXOXS+QRflwjiCgGfDedXdkKI4TQbJmZNWsWgsFgw6O8vFzzvNKYnpYtW9pXWCKDpN87P4wtCwQCyMjIQKtWrdwuCgCgW7duaNmyJc6ePYu9e/e6XRwi8jjftPh07NgRycnJTVp3jh492qQVSJKamorU1FRT12H3FrmBv3fWJSUlISsrCzt27EBxcTH69u1rz4nT04EHHwRWrjQ/w8rILKeKinMrHktf9WZ5VVYav368Yvcd2cA3LT4pKSkYNmwY3n333UbPv/vuu/jRj37kUqmIEsvDDz+MGTNmYNeuXW4XpRFHtq5ITwd+8QtgyRLzG5VKe3LpVc7Ll4cHMgPhr8OGnZv+fuedTTf7fPbZ8Gvbtzd96JVPvtGq1uapfiF13/m1/OQJvmnxAYAHH3wQkydPRk5ODkaOHInCwkIcOHAA06dPd7toRHFPCIG//OUvqKysxOTJk90uTiO+GeAsp7bZpzTL64YbgHXrGr8mfa+2NtCDD4ZDWmUlUFQETJgASF37hYXA5ZeHB0zv3h0ODiNHssWEEpavgs/PfvYzfPfddygoKEBFRQUGDhyIN954w9XZJfPmAcnJwJw5kY+dPx+or+fSHORPoVAIzz77LD755BMMHTrU7eI04ugu7U7R6z67++6m/1DorSu0cmW4VUiiXC9IalmSFBUB11xjqdhEfuer4AMAM2bMwIwZM9wuRoPkZGDu3PB/64Wf+fPDxxUUxKZcRHZLTk72zFYVSlLw2b17t+6EB9/o1El77SC1dYWkbjn5OkCnT59rHXr11XMtPpMmhVuEiBKU74KP10hhRy/8yEOPkZYhr+J2IeRVffr0QSAQwPfff4+qqip0srJZpp/33FKWpV+/xq/369c4LOndH68MIPbzz4M8jcHHBnrhJ15Cz1//+lfMnDmz0XYh1113neHtQsj/1q1bh759+6Jv376ea1Fp2bIlevTogX379qG4uNha8Ilmvy67LV8eeXC0knKW2Pffn3ttzZrwc1JX4Jo14fFA8vskBQevrP/jpZ8HxReRQILBoAAggsFgk9dOnz4tdu3aJU6fPm35/AUFQgDhr2rfO+XCCy8Uzz77bKPnPvroI5GWlib27dtnyzUuueQSMX369EbPZWdni8cee8yW8yc6O37/nPTDDz+IZs2aCQDim2++cbs4qq655hoBQBQWFlo7waFDQmzb1vixYkX4f+IVK5q+duiQvR9AKsPUqeFrbtvW+LVt2xo/ryyv9D6rj/x89eu4xQs/D/IMvfrbLLb42Eje8rNgAXDmTGxaekaMGIGtW7c2fC+EwMyZMzFz5swmA78XLVqERYsW6Z7vzTffbNSFZWW7EIovlZWVGDFiBL799ltkZma6XRxV2dnZePvtt63P7Ipmvy67pKeHu5jUNjNVrg2k1yKiZvx4YMCA8D9Ojz8OXHZZ0xYfs5zsFvPCz4PiEoOPzebMORd6UlJi0701YsQIvPTSSw3fv/LKKzhw4ABmzZrV5Njp06fj1ltv1T3fhRde2Oh7K9uFUHzJyMjAhg0bEAqFPNfNJbFtZpe8MvcSaW0gid50+BUrgO7dwwGqqOjcpqcVFeF/oMaPtyc4eKVbjMgEBh+bzZ9/LvScORP+PhYtPo8++ihOnjyJpKQkzJ49GwsWLEDr1q2bHNu+fXvLeyyZ2S6E4lNSknfXPLU1+EiVuZcZaRHp1CkcfKRNT5WDhTmAmBIQg4+NlAOZpe8BZ8NPTk4OkpOTsX37drz33nvo0KEDbr/9dtVjrXR1WdkuhOKLH0KuFHz27t2LmpoatGjRIrYFsNLt43bwiMcBxF6ZlUaexeBjE7XZW0amutuhRYsWGDx4MIqKilBYWIh169Zp/mVupatLvl3I+PHjG55/9913cdNNN0X/AcjTvv32WwwaNAg/+tGPsHbtWm+1+sgquQu6dEHbtm0RDAZRVlaGgQMHRn9+I3tuyctittvH7eAhdZcpV3zWWijR7SBhdA80dr+RDgYfG+hNWY9V+BkxYgT+8Ic/YNy4cbjyyis1j7Pa1cXtQhLX5s2b8f333+PgwYPeCj1Ao0oukJ6OrKwsbNmyBcXFxfYFn2iCRqTWh0jjdKINHsqgoPZ9enr4moWF4fJIA56VA4il1il5C1Wsu8Wi/XkQgcEnakbW6YlF+BkyZAiaNWuGxYsX239yeHO7EIqNG2+8EZ9++ilOnDjhdlEiys7OxpYtW4zP7HK6qylS64PdM5fUgo08KEQTHNxunSKyCYNPlOrrjU1Zl16vr3emHCtXrsSMGTMadql2gte2C6HYaN68OYbJ94HyMNMDnOOtMneyRcTp1imiGGHwiZKZf2PsbukJhUKorKzECy+8gJKSEqxZs8beCxD5jOngY7UyN9pSJK2kXFlprDxOinbQrxfX1XF7cDj5EoOPj3344YfIy8tDdnY2ioqK0LZtW7eLRHGmpKQEr732Gq644grk5eW5WxgDlVx2KAQAKN61C+LQIQS6dtU/p9XK3GxLkR27oUcbXKRut5Ej9e9j9+7A1KnhsLZ9u7fDQry12FFMMPj4WG5uLkL//oeeyAnvvvsuCgoKcO2117offAxUchcBSAZw8ocfcOi3v8WFS5Y4Uxa1lqL164FHHgmvivzvlicUF4cXDOzTp2kLhFag0Jq5ZNdspaIi9ZWhgcZhQTrGy2GB3W9kAYMPEWnKzs7GbbfdhhEjRrhdFEOVXAqAXuPHY8+BAygeMQIXqp7IBmqhZfny8NcFC5oe/8gjTZ/TChROz1yaMKHpqtR+DQte7H4jz2PwISJNV111Fa666iq3ixFmsJLLHjwYew4cQElVFbQXdnDAhAnhVpJXXw2vlAx4M1B06qQdCMyGBTPrHBF5BIMPEcWV7OxsrFu3zvrWFVYrc2n9mw8/DO+LJX+/ldaHaMb0mB30a3XwNdfVIR9i8CEiVUePHkVycjI6dOjgdlFMiXrPrmgrc2khwGhaQSoqwmOGnngiPD6oXz9zs5XMDvqdOtV6WYl8xmPLsBKRV/zhD39Ax44d8atf/crtophi22alZqWnaweI5cubtsDoWb4cmDQp/N+TJgHDhp0LLHfeGf5e/pDGF0mmTQO2bWv8WLEi/NqKFU1fmzDB3Gf1Mna/UQRs8SEiVd9++y0AoHfv3i6XRIdKJSct4lleXo6TJ0+iVatWsSvLtGnqM6bMtgJNmxZu6Zk06dyYITPjhcwO+q2oiJ+wwO43ioDBh4hUvfTSS1iyZAmaNfPwPxMqlVyHDh3QsWNHVFVVobS0FBe7ObNHagXSmj6u9z5pgHS/fo2DihOzlRgWKIF4+F80InKblQ1tvSA7OxsbN25ESUmJc8HH6ADiyy8PBx+9AcRGziWtAr17t7cXFSTyOAYfJ0S7wioRRUUKPo6O87Fz9WYz55o0yduLChJ5HIOPE+xaYZXIJXPmzMHevXtx9913Y+TIkW4Xx7SYDHC2c/VmIysQ7959bsyPlVW0jQz65R9tlAAYfMiQDz/8EIsXL8a2bdtQUVGBNWvW4Cc/+YnbxSKHFBUVYdeuXfjZz37mdlEsiUnwsXP1ZiODkaXgolwjyEx5I7US8Y82SgAMPmTIqVOnMHjwYPzyl7/ET3/6U7eLQw5bunQpPv74Y1+29gDngk9paSnq6+uRnJwcmws7uXozByAT2YLBJw5069YNs2fPxowZMxqe27RpE6666irs3r0bPXr0iPoa1113Ha677rqoz0P+cPXVV+Pqq692uxiWZWZmIiUlBTU1NThw4AB69uwZmwtLqzcrZ2IB3DuKyCMYfKJldml4B2ZjjBgxAlu3bm34XgiBmTNnYubMmU1Cz6JFi7Bo0SLd87355psYPXq0rWUkiqXk5GT06dMHO3fuRElJSeyCDxF5HoNPtMzO7HBgNsaIESPw0ksvNXz/yiuv4MCBA5g1a1aTY6dPn45bb71V93wXXujYntbkA2vXrkV6ejoGDx6MlJQUt4tjWXZ2Nnbu3Ini4mJce+21sbmonasGO70CsQf+aCNyA4NPtIzMxpBz4B+OESNG4NFHH8XJkyeRlJSE2bNnY8GCBWjdunWTY9u3b+/btVnIeUII/PKXv8SxY8ewZcsWDB8+3O0iWebK1hV2jsNxekyPB/5oI3IDg0+0zC4N74CcnBwkJydj+/bteO+999ChQwfcfvvtqseyq4v0VFdX49JLL8Xnn3+OwYMHu12cqLi2Z5eSV/eO8sAfbURuYPCJAy1atMDgwYNRVFSEwsJCrFu3DklJ6vvPsquL9LRt2xZvvPEGhBAIBAJuFycqngo+Xmwp8cAfbURuYPCJEyNGjMAf/vAHjBs3DldeeaXmcVa7uk6ePImysrKG7/fu3YvPPvsM7du3R0ZGhqUyk3f5PfQAQN++fQEAR44cwfHjx9GuXTuXS0REXqDeLEC+M2TIEDRr1gyLFy925Pyffvophg4diqFDhwIAHnzwQQwdOhRz58515Hrkjrq6OreLYJs2bdqga9euAICSkhKXS0NEXsHg4wQX+vRXrlyJGTNmICsry5Hz5+bmQgjR5CGfTUb+durUKbRr1w4/+tGPcPLkSbeLYwvPdHcRkWcw+DhB6tN3OPiEQiEcOXIEixYtQklJCZ7QmqFBZMD27dtx6tQpHDhwAK1atXK7OLZg8DHJCwOxKyrC/34qp9oT2YRjfHzsww8/RF5eHrKzs1FUVIS2bdu6XSTyscsuuwx79uzBwYMH3S6KbRh8TPLCQGzuF0YOY/DxsdzcXIRCIbeLQXEiEAigd+/e6N27t9tFsQ2DDxEpsauLiOKWNOatrKwMZ8+edbk0JrHLh8gRDD5EhIMHD+KBBx7A6tWr3S6Krbp164aWLVuirq4Oe/fudbs45khdPgw+RLZiVxcRYePGjVi6dCk2btyIn/70p24XxzZJSUnIysrCjh07UFxc3LC2D3kE9wsjFzD4EBEuuugi3HPPPcjMzHS7KLbLzs5uCD43KrdoIHdxvzByAYMPESEnJwc5OTluF8MRHODsYdwvjFzA4ENEcc0XwSdRu3y4Xxi5gMGHKMFVVlbi5MmTyMzMjIs9upSkmV3FxcXe3XyVXT5EMcPgQ5TgVq5ciQceeAD/+Z//iZUrV7pdHNv16dMHgUAA33//PaqqqtCpUye3i9QUu3yIYobBhyjBVVZWonnz5ujfv7/bRXFEy5Yt0aNHD+zbtw/FxcXeDD7s8iGKGa7jY7P6+nq8//77WLVqFd5//33U19e7XSQiXQsXLkQwGMS9997rdlEc44txPhTmhf3CKK4x+NioqKgImZmZGDNmDP7zP/8TY8aMQWZmJoqKitwuWtSeeuopDBgwAC1btkTfvn3x2muvuV0kslFaWhratGnjdjEcw+DjIzHa5JkSF4OPTYqKinDzzTfj22+/bfT8wYMHcfPNN/s+/GzYsAG///3v8dVXX2HSpEm47bbb8M0337hdLCJDGHyISMLgY4P6+nrcf//9EEI0eU16bubMmY51e3Xr1g3Lli1r9NymTZvQsmVL7N+/35Zr/N///R/Gjh2LXr164Z577kF9fT0OHTpky7nJPU899RRuvPFGvPHGG24XxVHymV2+wS4fIkf4Ivjs27cPU6ZMQc+ePZGWloaLLroI+fn5OHPmjNtFAxBuDVG29MgJIVBeXo4NGzY4cv0RI0Zg69atja43c+ZMzJw5Ez169Gh07KJFi9CqVSvdh145hRB46KGHMHDgQFxyySWOfB6Knbfeegvr1q3D4cOH3S6Ko6QWn3379qGmpsbl0hjELh8iR/hiVldxcTFCoRCWL1+O3r1746uvvsKdd96JU6dO4be//a3bxUOFwU0EjR5n1ogRI/DSSy81fP/KK6/gwIEDmDVrVpNjp0+fjltvvVX3fBdeeKHma3fccQc2bdqE9evXIyUlxXKZyRsWL16Mjz76CFdeeaXbRXHUBRdcgLZt2yIYDKKsrAwDBw50u0hE5BJfBJ9rr70W1157bcP3vXr1QklJCZ577jlPBJ90g3+RGT3OrBEjRuDRRx/FyZMnkZSUhNmzZ2PBggVo3bp1k2Pbt2+P9u3bW7rOF198gRdffBHFxcW64Yj8Y/jw4Rg+fLjbxXBcIBBAdnY2Nm/ejOLiYgYfogTmi64uNcFgMGIFXltbi+rq6kYPJ4wePRrdunXTXBE2EAige/fuGD16tCPXz8nJQXJyMrZv346nnnoKHTp0wO233656bDRdXXv37gVwbrwEkZ9wgDMRAT5p8VH6+uuv8cwzz+B3v/ud7nFPPvkkntBaBt5GycnJePrpp3HzzTcjEAg0GuQshaGlS5ciOTnZkeu3aNECgwcPRlFREQoLC7Fu3TokJaln2mi6uq644opGY4nI39auXYvzzjsPI0aMwHnnned2cRzH4ENEAADhovz8fAFA97F169ZG7zl48KDo3bu3mDJlSsTz19TUiGAw2PAoLy8XAEQwGGxy7OnTp8WuXbvE6dOnLX+e1atXi27dujUqf/fu3cXq1astn9Ooe+65RwQCAXHDDTc4do2ioiKRlZXl2PkTmR2/f2b17dtXABBvvPFGzK7ppqKiIgFADBs2zO2iEJFJwWBQs/42y9UWn3vuuQcTJ07UPSYzM7Phvw8dOoQxY8Zg5MiRKCwsjHj+1NRUpKamRltMwyZMmICbbroJGzZsQEVFBdLT0zF69GjHWnrkhgwZgmbNmmHx4sWOXSMYDKKkpMSx81Ps1NXVYdiwYThz5kzCzM6TWnxKSkq8u1kpETkuIITK4jMedPDgQYwZMwbDhg3Dq6++ailMVFdXN8zsUK5SW1NTg71796Jnz55o0aKFXcWOmby8PAwaNAhLly51uyhkgd9///zgzJkzaNmyJerr6/Htt99ygD6Rj+jV32b5YozPoUOHkJubi4yMDPz2t79FZWVlw2tdunRxsWTuCoVCqKysxAsvvICSkhKsWbPG7SIReVZKSgouuugilJaWcmYiUQLzRfB55513UFZWhrKyMnTr1q3Raz5psHLEhx9+iLy8PGRnZ6OoqAht27Z1u0jkEzU1NQnZspSdnd0QfOJ97SIiUueL6ez//d//DSGE6iOR5ebmIhQKYdeuXbj00kvdLg75RCgUQteuXTFw4ECUl5e7XZyY4swuIvJFiw8R2aesrAzff/89amtrHVtU06t8uWcXEdmKwYcowfTt2xeHDx/Gnj170KxZYv0TIJ/ZRUSJKbH+1SMiAOG9qy644AK3ixFzUotPeXk5Tp48iVatWrlcIiKKNV+M8SEiskOHDh3QqVMnAEBpaanLpSEiNzD4ECWQEydOYMqUKVixYgVCoZDbxXEFBzgTJTYGH6IEsnXrVrz44otYuHCh5n5u8Y7BhyixcYwPUQLp1q0bZs+ejbS0NLeL4hrO7CJKbAw+RAmkb9++WLhwodvFcBVndhEltsRs605QgUAAf//7310tQ7du3fDHP/7R1TJoqaurQyAQwP/+7/+6XRRykBR8SktLUV9f73JpiP5/e/ceF0W9/w/8tSxyXVi5yLWDrIICKXKto6KINyTvkeYFxRDLQAMtzVCTSvGSl1TygiV51EzMvKSg4SXSPAQioAlIGmRyOVDi4q4CAp/fH36dXyuIoOjs7L6fj8c+Hu7MZ2ZezNlH8z6f+XxmyPNGhY/ATZs2DSKRCCKRCB06dIC1tTWGDBmC7du3Nxm8WlZWhqCgIJ6SEr5VVVUhNzcX9fX1fEfhlaOjI/T09FBTU4Pr16/zHYcQ8pxR4aMBhg0bhrKyMhQXFyMlJQUBAQGIiorCiBEjVC5yNjY20NfXb/fjNzQ0aO0MoYfV1dXxHeGRkpOT4eHhgSFDhvAdhVdisRjdunUDQON8CNFGVPi0gDEGpVL53D9tfQeZvr4+bGxsYG9vDy8vL8TExODQoUNISUnBV199xbX7562uuro6zJo1C7a2tjAwMICjoyOWL1/Otb116xbefPNNWFtbw8DAAD169OBuAX311Vfo2LEjjhw5Ajc3N+jr6+OPP/5odV65XI7XX38dxsbGsLe3x6ZNm1TWFxcXY9SoUTA2NoZUKsWECRNQWVnJrQ8JCcFrr72mss2sWbMwePBg7rufnx/mzJmDd999F2ZmZrC1tcUnn3yiss2VK1fQr18/GBgY4MUXX8SpU6eaZH3vvffg7OwMQ0NDdOnSBUuWLFEpJhctWgQfHx9s27YNMpkMxsbG2L59O6ysrHDv3j2VfY0ePRphYWGtPk/t7ebNmzAxMYGHhwdvGdQFzewiRHvR4OYW3Llzh5cnuyoUChgbGz/VPgYOHIhevXrhu+++Q3h4eJP1GzZswOHDh5GUlAQHBwf8+eef3AsrGxsbERQUhNu3b2PXrl3o2rUr8vLyIBaLue3v3LmD5cuX44svvoCFhQWsrKxanW3VqlVYuHAhPvnkEyQnJ2P27NlwdXVFQEAAGhsbMWrUKJibm+PMmTOoq6vD22+/jYkTJ+LEiRNtOgfbt2/HvHnzkJGRgbNnzyIsLAx+fn7cccaMGQN7e3v88ssvqKqqQnR0dJN9SKVS/Oc//4GtrS1yc3MxY8YMSKVSzJ07l2tTUFCAgwcP4rvvvoOOjg6cnJwQHR2NI0eOYOzYsQCAiooKJCcnN1tcPS+zZ89GREQE7t69y1sGdUEzuwjRXlT4aDAXFxdcvHix2XXXr1+Hs7Mz/Pz8IBKJ0LlzZ27diRMnkJGRgfz8fO6WQJcuXVS2v3fvHjZt2oRevXq1OVf//v0xf/58APdnGZ09exbr1q1DQEAAjh8/jvz8fBQXF8Pe3h4AsGPHDvTq1QvZ2dnw9PRs9XG8vLywaNEiAICzszM2btyIkydPIiAgAMeOHcO1a9dw6tQp7kWdS5cuxciRI1X2sXjxYu7fjo6OyMvLQ1JSkkrhU19fj507d8Lc3JxbNmHCBCQmJnKFz86dOyGTydCvX7+2nKp2JxaL6TUNoJldhGgzKnxaYGRkBIVCwctx2wNjDCKRqNl106ZNw5AhQ9C9e3cMGzYMI0aMwNChQwEAOTk5eOGFF7iipzl6enpwd3d/oly9e/du8n3Lli0AgPz8fDg6OnJFDwC4u7tDIpEgPz+/TYXPw/lsbW1RUVGhcpx/vp384VwAkJSUhPXr1+Pq1atQKpWor69XKXAAQCaTNVk2Y8YM9OnTB+Xl5bCxsUFiYiLeeOONVmcnzxbd6iJEe1Hh0wKRSPTUt5z4lJ+fD5lM1uw6Ly8vFBUVISUlBSdOnMD48eMxePBgfPvtt616uJ2hoeEji6on8WBfLRVrD5br6Og0GQf18HgaAOjQoUOT7R8Mwm7NOKqff/4ZkyZNwtKlSzF48GBIpVLs2rULn3/+uUq75n4jvr6+cHNzw86dO+Hv74+CggKEhoY+9pjPyqZNm5CUlITw8HCEhITwlkNdPLjV9b///Q9VVVUwMzPjOREh5Hmhwc0a6tSpU7h06RKCg4Mf2cbU1BSvv/46tm3bhr1792L//v24efMm3N3dcePGjWf2Esf09PQm3x/8P3A3NzcUFRWhtLSUW3/x4kUoFAq4uroCADp16oSysjKVfeTk5LQpg5ubG4qLi1FeXv7IXGfPnkXXrl2xYMEC+Pj4wNnZuU2DuMPDw5GYmIjEB6wX4QAAGKJJREFUxEQEBgbCzs6uTRnb06lTp5CWloaSkhLeMqgTExMTrleRbncRol2o8NEAtbW1KC8vR0lJCS5cuIC4uDiMHj0aI0aMwNSpU5vdZt26dfjmm29QUFCAwsJC7Nu3DzY2NujYsSP8/f3Rv39/BAcHIzU1lesZOnbsWIs5vv32W/To0eOxedPS0rBmzRoUFhZiw4YNOHDgAKKiogAAgYGBcHV1xeTJk5GdnY309HRMmzYNgwYN4mYjDRw4EOnp6di9ezd+++03LFq0qM23LAIDA9GlSxdMnToVFy9eRFpamsp4HgBwcnJCUVERkpKScO3aNXz22Wc4fPhwq48REhKCoqIibN++ndfZXACwbNkyfPnllxg9ejSvOdQJ3e4iRDtR4aMBjh07BltbWzg6OmLYsGE4ffo0NmzYgEOHDqnMxPoniUSClStXwsfHB76+viguLkZycjL34sr9+/fD19cXEydOhJubG+bPn//Yp9zeunWrVReR+fPn47///S88PT2xYsUKfPbZZxg0aBCA+7exDh8+DIlEAj8/PwQGBqJbt27Ys2cPt/3w4cMRExODuXPn4qWXXkJtbS0mT57c2tMF4P4g34MHD0KpVMLX1xdvvfUW4uLiVNq8+uqr3EwoT09PZGRkYOHCha0+hpmZGcaOHQtTU9Mmg6aft+7duyMsLIy72BOa2UWIthKxtj40RsCqq6shlUohl8thamqqsq6mpgZFRUWQyWQwMDDgKSHRNAEBAfD09MTatWtbbEe/v+dv48aNeOeddzBmzBgcOHCA7ziEkBa0dP1uKxrcTMgzcPPmTaSkpODMmTPYtm0br1mOHDmCuro69O/fH5aWlrxmUSd0q4sQ7US3ugh5Btzd3REZGYnVq1fDycmJ1ywrVqxAcHAwkpOTec2hbh4UPlevXm12ViAhRDNRjw8hz8CNGzf4jsDx8fGBQqHAv//9b76jqBV7e3sYGxtDqVTi999/58b8EEI0G/X4EKLhPvvsM+Tk5LT4QEptpKOjQwOcCdFCVPgQQrQWFT6EaB8qfAjRYNXV1XxHUGv0zi5CtA8VPoRoME9PTzg4ODzyZbXajmZ2EaJ9aHAzIRqqqqoKxcXFYIyhc+fOfMdRS/8sfFp6TxwhRHNQjw8hGsrMzAxyuRznzp2DVCrlO45acnZ2hkgkQlVVFSorK/mOQwh5DqjwIe1OJBLh4MGDfMdoVnFxMUQiUZtfaipUEomEprG3wNDQEI6OjgDodhch2oIKHw1QXl6OqKgoODk5wcDAANbW1vDz88OWLVtw584dvuMRotZoZhch2oXG+Ajc77//jr59+6Jjx46Ii4tDz549UV9fj8LCQmzfvh12dnYYNWoU3zE1Sl1dHfT09PiO0aKGhgZMmTIFXl5eiIyMhKGhId+R1JaLiwuOHTtGM7sI0RLU49MKSqUSSqUS/3yfa11dHZRKJWpra5tt29jYyC27d+8elEolampqHtu2rSIiIqCrq4vz589j/PjxcHV1Rc+ePREcHIyjR4+qvBVcLpfjzTffhJWVFUxNTTFw4EDk5uZy62NjY+Hh4YGdO3fC0dERUqkUEyZMwO3bt9ucq6ysDEFBQTA0NIRMJsO+fftU1l+6dAkDBw6EoaEhLCws8Oabb0KhUHDrBwwYgOjoaJVtxowZg2nTpnHfHR0dERcXh7CwMJiYmMDBwQEJCQkq22RkZMDT0xMGBgbw8fFBdna2yvqGhgZMnz4dMpkMhoaG6N69O9avX6/SZtq0aRgzZgyWL18OOzs7dOvWDR9//DF69uzZ5O/29vbGhx9+2KZz9SxcvnwZe/bswccff6z2RRrfaGYXIdqFCp9WkEgkkEgk+Ouvv7hln376KSQSCWbNmqXS1srKChKJBNevX+eWff7555BIJJg+fbpKW0dHR0gkEuTn5z9Rrr///hs//PADIiMjYWxs3GybB7NUGGMYPnw4ysvLkZycjKysLHh5eWHQoEG4efMm1/7atWs4ePAgjhw5giNHjiAtLQ0rVqxoc7bFixcjODgYubm5CAkJwcSJE7m/886dOxg2bBjMzMyQmZmJffv24cSJE03OZWusWbOGK2giIiLw9ttvcxcwpVKJESNGoHv37sjKykJsbCzee+89le0bGxvxwgsvICkpCXl5efjwww8RExODpKQklXYnT55Efn4+UlNTceTIEYSFhSEvLw+ZmZlcm4sXLyI7O1ulOOOLpaUlVq5ciTlz5kAsFvMdR61R4UOIlmFaRC6XMwBMLpc3WXf37l2Wl5fH7t6922QdAAaAVVRUcMuWLl3KALDw8HCVtkZGRgwAKyoq4patW7eOAWCTJk1SaWtpackAsF9//fWJ/p709HQGgH333Xcqyy0sLJixsTEzNjZm8+fPZ4wxdvLkSWZqaspqampU2nbt2pVt3bqVMcbYkiVLmJGREauurubWz5s3j7388sttygWAzZw5U2XZyy+/zN5++23GGGMJCQnMzMyMKRQKbv3Ro0eZjo4OKy8vZ4wx5u/vz6KiolT2MXr0aBYaGsp979y5MwsJCeG+NzY2MisrK7Z582bGGGNbt25l5ubmTKlUcm02b97MALDs7OxH5o+IiGDBwcHc99DQUGZtbc1qa2tV2gUFBXF/E2OMRUdHswEDBjxyvy1p6fdHnq3y8nIGgIlEIjr/hKiplq7fbUU9Pq2gUCigUChgaWnJLZs3bx4UCgXi4+NV2lZUVEChUMDBwYFbFhkZCYVCgS+//FKlbXFxMRQKBVxdXZ8q38PPHsnIyEBOTg5efPFF7lZcVlYWFAoFLCwsuB4siUSCoqIiXLt2jdvW0dERJiYm3HdbW1tUVFS0OVPv3r2bfH/Q45Ofn49evXqp9FL17dsXjY2NbR5n4e7uzv1bJBLBxsaGy/vgOEZGRo/MBQBbtmyBj48POnXqBIlEgm3btqn02AFAz549m9wymjFjBvbs2YOamhrcu3cPu3fvRlhYWJvyE/5ZWVmhY8eOYIzht99+4zsOIeQZo8HNrdDcbSQ9Pb1mx04017ZDhw7o0KFDq9q2hZOTE0QiUZMu+i5dugCAyoDWxsZG2Nra4scff2yyn44dO6pk/SeRSPRUY5Ae3heAFh8U92C5jo6Oypgq4P5YqYe1lPfh7ZuTlJSEOXPmYM2aNejduzdMTEzw6aef4pdfflFp19z/ViNHjoS+vj4OHDgAfX191NbWIjg4+LHHfNYUCgWys7Ph7e2tUvSR5olEInTv3h2//PILCgoKmh27RQjRHNTjI2AWFhYYMmQI4uPjoVQqW2zr5eWF8vJy6OrqwsnJSeXzz56s9pKent7k+4OxFG5ubsjJyVHJ/PPPP0NHR4d7g3inTp1QVlbGrW9oaMCvv/7apgxubm7Izc3F3bt3H5nrzJkz6NOnDyIiIuDp6QknJyeVHrCW6OrqIjQ0FImJiUhMTMSECRPUotA4d+4c+vfvD09PT76jCAa9s4sQ7UGFj8Bt2rQJ9fX18PHxwd69e5Gfn48rV65g165dKCgo4Aa2Dh48GL1798aYMWNw/PhxFBcX49y5c1i0aBHOnz/f6uPFx8dj0KBBj223b98+bN++HYWFhViyZAkyMjK4wcuTJ0+GgYEBQkND8euvv+L06dOYPXs2pkyZAmtrawDAwIEDcfToURw9ehQFBQWIiIjArVu32nRuJk2aBB0dHUyfPh15eXlITk7G6tWrVdo4OTnh/PnzOH78OAoLC7F48WKVAcuPEx4ejlOnTiElJUVtbnPdunULtra28PX15TuKYNAAZ0K0B93qEriuXbsiOzsbcXFx+OCDD3Djxg3o6+vDzc0N7733HiIiIgDc785PTk7GwoULERYWhsrKStjY2KB///5csdEaf/31V6t6RD766CN88803iIiIgI2NDXbv3g03NzcAgJGREY4fP46oqCj4+vrCyMgIwcHBWLt2Lbd9WFgYcnNzMXXqVOjq6mLOnDkICAho07mRSCT4/vvvMXPmTHh6esLNzQ0rV65UuR01c+ZM5OTk4PXXX4dIJMLEiRMRERGBlJSUVh3D2dkZffr0wd9//42XX365TfmelfHjx2PcuHFNHp9AHo0KH0K0h4i1ZiCEhqiuroZUKoVcLoepqanKupqaGhQVFUEmk8HAwICnhERoGGNwcXHBW2+9hblz5z7xfuj3x6+CggK4urrC2NgYt2/fppeVEqJmWrp+txXd6iLkCVVUVGDt2rUoKSnBG2+8wXcc8hS6du0KXV1dKJVKlJSU8B2HEPIMUeFDyBOytrbGihUrkJCQADMzM77jAAB2794NLy8vrFu3ju8ogtKhQwduNiTd7iJEs1HhQ8gTYoyhsrISkyZN4jsK5+zZs8jOzkZpaSnfUQSHZnYRoh2o8CFEgyxevBj79u1DSEgI31EEhwY4E6IdaFbXQ9rrYX2EtEV7/e7s7Ozw2muvtcu+tA0VPoRoByp8/o+enh50dHRQWlqKTp06QU9Pj2Z2kGeOMYa6ujpUVlZCR0eH3qTOIyp8CNEOVPj8Hx0dHchkMpSVldH4CPLcGRkZwcHBATo6T373+fjx4ygrK8PAgQNV3hVHWqd79+4AgBs3buD27dsq76wjhGgOKnz+QU9PDw4ODqivr0dDQwPfcYiWEIvF0NXVfeoexs2bN+PQoUNYvXo13n333XZKpz3Mzc3RqVMnVFZWorCwEN7e3nxHIoQ8A1T4PEQkEj3ypaKEqLOXXnoJf//9N/z8/PiOIlguLi6orKzElStXqPAhREPRrC5CNERMTAzOnDmjNq/OECIa50OI5hNc4VNbWwsPDw+IRCLk5OTwHYcQokGo8CFE8wmu8Jk/fz7s7Oz4jkGIWqmsrKRHMbQDKnwI0XyCGuOTkpKCH374Afv372/V27Nra2tRW1vLfZfL5QDuv+yMEE0SFBSEq1evYufOnfD39+c7jmDZ29sDuP/05qqqKojFYp4TEUKA/3/dbpf3qjOBKC8vZ/b29iwzM5MVFRUxACw7O7vFbZYsWcIA0Ic+9KEPfehDHw34XLt27anrCRFj7VE+PVuMMbzyyivo27cvFi1ahOLiYshkMmRnZ8PDw+OR2z3c43Pr1i107twZ169fh1QqfR7RNVZ1dTX+9a9/4c8//4SpqSnfcQSLzmP7oXPZfuhctg86j+1HLpfDwcEBVVVV6Nix41Pti9dbXbGxsfjoo49abJOZmYlz586huroaH3zwQZv2r6+vD319/SbLpVIp/QjbiampKZ3LdkDnsf3QuWw/dC7bB53H9vM0D3l9gNfCZ9asWZgwYUKLbRwdHbF06VKkp6c3KWJ8fHwwefJk7Nix41nGJIQQQoiG4LXwsbS0hKWl5WPbbdiwAUuXLuW+l5aWIjAwEHv37qVnlhBCCCGk1cSxsbGxfId4HKlUCisrK+4jFouxfv16fPDBB+jWrVub9iUWizFgwADo6gpqQptaonPZPug8th86l+2HzmX7oPPYftrrXApicPPDWju4mRBCCCHknwRZ+BBCCCGEPAnBPbmZEEIIIeRJUeFDCCGEEK1BhQ8hhBBCtAYVPoQQQgjRGlpZ+BQXF2P69OmQyWQwNDRE165dsWTJEtTV1fEdTRA2bdoEmUwGAwMDeHt748yZM3xHEpzly5fD19cXJiYmsLKywpgxY3DlyhW+Ywne8uXLIRKJEB0dzXcUQSopKUFISAgsLCxgZGQEDw8PZGVl8R1LcOrr67Fo0SLuGtOlSxd8/PHHaGxs5Dua2vvpp58wcuRI2NnZQSQS4eDBgyrrGWOIjY2FnZ0dDA0NMWDAAFy+fLlNx9DKwqegoACNjY3YunUrLl++jHXr1mHLli2IiYnhO5ra27t3L6Kjo7Fw4UJkZ2ejX79+CAoKwvXr1/mOJihpaWmIjIxEeno6UlNTUV9fj6FDh0KpVPIdTbAyMzORkJAAd3d3vqMIUlVVFfr27YsOHTogJSUFeXl5WLNmzVO/F0kbrVy5Elu2bEF8fDzy8/OxatUqfPrpp9i4cSPf0dSeUqlEr169EB8f3+z6VatWYe3atYiPj0dmZiZsbGwwZMgQ3L59u/UHeerXnGqIVatWMZlMxncMtffSSy+xmTNnqixzcXFhCxYs4CmRZqioqGAAWFpaGt9RBOn27dvM2dmZpaamMn9/fxYVFcV3JMF5//33mZ+fH98xNMLw4cNZWFiYyrJXX32VhYSE8JRImACwAwcOcN8bGxuZjY0NW7FiBbespqaGSaVStmXLllbvVyt7fJojl8thbm7Odwy1VldXh6ysLAwdOlRl+dChQ3Hu3DmeUmkGuVwOAPQbfEKRkZEYPnw4Bg8ezHcUwTp8+DB8fHwwbtw4WFlZwdPTE9u2beM7liD5+fnh5MmTKCwsBADk5ubi7NmzeOWVV3hOJmxFRUUoLy9XuQbp6+vD39+/TdcgeoY2gGvXrmHjxo1Ys2YN31HU2l9//YWGhgZYW1urLLe2tkZ5eTlPqYSPMYa5c+fCz88PPXr04DuO4HzzzTe4cOECMjMz+Y4iaL///js2b96MuXPnIiYmBhkZGXjnnXegr6+PqVOn8h1PUN5//33I5XK4uLhALBajoaEBy5Ytw8SJE/mOJmgPrjPNXYP++OOPVu9Ho3p8YmNjIRKJWvycP39eZZvS0lIMGzYM48aNQ3h4OE/JhUUkEql8Z4w1WUZab9asWbh48SL27NnDdxTB+fPPPxEVFYVdu3bBwMCA7ziC1tjYCC8vL8TFxcHT0xNvvfUWZsyYgc2bN/MdTXD27t2LXbt24euvv8aFCxewY8cOrF69Gjt27OA7mkZ42muQRvX4zJo1CxMmTGixjaOjI/fv0tJSBAQEoHfv3khISHjG6YTP0tISYrG4Se9ORUVFkwqctM7s2bNx+PBh/PTTT3jhhRf4jiM4WVlZqKiogLe3N7esoaEBP/30E+Lj41FbWwuxWMxjQuGwtbWFm5ubyjJXV1fs37+fp0TCNW/ePCxYsIC7HvXs2RN//PEHli9fjtDQUJ7TCZeNjQ2A+z0/tra23PK2XoM0qvCxtLSEpaVlq9qWlJQgICAA3t7eSExMhI6ORnV+PRN6enrw9vZGamoqxo4dyy1PTU3F6NGjeUwmPIwxzJ49GwcOHMCPP/4ImUzGdyRBGjRoEC5duqSy7I033oCLiwvef/99KnraoG/fvk0eqVBYWIjOnTvzlEi47ty50+SaIhaLaTr7U5LJZLCxsUFqaio8PT0B3B97mpaWhpUrV7Z6PxpV+LRWaWkpBgwYAAcHB6xevRqVlZXcugcVJWne3LlzMWXKFPj4+HA9ZdevX8fMmTP5jiYokZGR+Prrr3Ho0CGYmJhwvWhSqRSGhoY8pxMOExOTJuOijI2NYWFhQeOl2mjOnDno06cP4uLiMH78eGRkZCAhIYF6w5/AyJEjsWzZMjg4OODFF19EdnY21q5di7CwML6jqT2FQoGrV69y34uKipCTkwNzc3M4ODggOjoacXFxcHZ2hrOzM+Li4mBkZIRJkya1/iDtNu9MQBITExmAZj/k8T7//HPWuXNnpqenx7y8vGgK9hN41O8vMTGR72iCR9PZn9z333/PevTowfT19ZmLiwtLSEjgO5IgVVdXs6ioKObg4MAMDAxYly5d2MKFC1ltbS3f0dTe6dOnm/1vY2hoKGPs/pT2JUuWMBsbG6avr8/69+/PLl261KZjiBhj7OlrNEIIIYQQ9UcDWwghhBCiNajwIYQQQojWoMKHEEIIIVqDCh9CCCGEaA0qfAghhBCiNajwIYQQQojWoMKHEEIIIVqDCh9CCCGEaA0qfAghhBCiNajwIYQIzp49e2BgYICSkhJuWXh4ONzd3SGXy3lMRghRd/TKCkKI4DDG4OHhgX79+iE+Ph4fffQRvvjiC6Snp8Pe3p7veIQQNaaVb2cnhAibSCTCsmXL8Nprr8HOzg7r16/HmTNnuKJn7Nix+PHHHzFo0CB8++23PKclhKgT6vEhhAiWl5cXLl++jB9++AH+/v7c8tOnT0OhUGDHjh1U+BBCVNAYH0KIIB0/fhwFBQVoaGiAtbW1yrqAgACYmJjwlIwQos6o8CGECM6FCxcwbtw4bN26FYGBgVi8eDHfkQghAkFjfAghglJcXIzhw4djwYIFmDJlCtzc3ODr64usrCx4e3vzHY8Qouaox4cQIhg3b95EUFAQRo0ahZiYGACAt7c3Ro4ciYULF/KcjhAiBNTjQwgRDHNzc+Tn5zdZfujQIR7SEEKEiGZ1EUI0TmBgIC5cuAClUglzc3McOHAAvr6+fMcihKgBKnwIIYQQojVojA8hhBBCtAYVPoQQQgjRGlT4EEIIIURrUOFDCCGEEK1BhQ8hhBBCtAYVPoQQQgjRGlT4EEIIIURrUOFDCCGEEK1BhQ8hhBBCtAYVPoQQQgjRGlT4EEIIIURr/D+u01smBW5XWwAAAABJRU5ErkJggg==",
      "text/plain": [
       "PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x132271b90>)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "using Optim # Optimization library\n",
    "\n",
    "y_1 = zeros(length(y)) # class 1 indicator vector\n",
    "y_1[y.==false] = 1.\n",
    "X_ext = vcat(X, ones(1, length(y))) # Extend X with a row of ones to allow an offset in the discrimination boundary\n",
    "\n",
    "# Implement negative log-likelihood function\n",
    "function negative_log_likelihood(θ::Vector)\n",
    "    # Return negative log-likelihood: -L(θ)\n",
    "    p_1 = 1.0 ./ (1.0 + exp.(-X_ext' * θ))   # P(C1|X,θ)\n",
    "    return -sum(log.( (y_1.*p_1) + ((1.-y_1).*(1.-p_1))) ) # negative log-likelihood\n",
    "end\n",
    "\n",
    "# Use Optim.jl optimiser to minimize the negative log-likelihood function w.r.t. θ\n",
    "results = optimize(negative_log_likelihood, zeros(3), LBFGS())\n",
    "θ = results.minimizer\n",
    "\n",
    "# Plot the data set and ML discrimination boundary\n",
    "plotDataSet()\n",
    "p_1(x) = 1.0 ./ (1.0 + exp(-([x;1.]' * θ)))\n",
    "boundary(x1) = -1./θ[2] * (θ[1]*x1 + θ[3])\n",
    "plot([-2.;10.], boundary([-2.;10.]), \"k-\");\n",
    "\n",
    "# Also fit the generative Gaussian model from lesson 7 and plot the resulting discrimination boundary for comparison\n",
    "generative_boundary = buildGenerativeDiscriminationBoundary(X, y)\n",
    "plot([-2.;10.], generative_boundary([-2.;10.]), \"k:\");\n",
    "legend([L\"y=0\";L\"y=1\";L\"y=?\";\"Discr. boundary\";\"Gen. boundary\"], loc=3);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Given $\\hat{\\theta}$, we can classify a new input $x_\\bullet = [3.75, 1.0]^T$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P(C1|x•,θ) = 0.6476513551215346\n"
     ]
    }
   ],
   "source": [
    "x_test = [3.75;1.0]\n",
    "println(\"P(C1|x•,θ) = $(p_1(x_test))\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The generative model gives a bad result because the feature distribution of one class is clearly non-Gaussian: the model does not fit the data well. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The discriminative approach does not suffer from this problem because it makes no assumptions about the feature distribition $p(x|y)$, it just estimates the conditional class distribution $p(y|x)$ directly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Recap Classification\n",
    "\n",
    "<table>\n",
    "<tr> <td></td><td style=\"text-align:center\"><b>Generative</b></td> <td style=\"text-align:center\"><b>Discriminative</b></td> </tr> \n",
    "\n",
    "<tr> <td>1</td><td>Like <b>density estimation</b>, model joint prob.\n",
    "$$p(\\mathcal{C}_k) p(x|\\mathcal{C}_k) = \\pi_k \\mathcal{N}(\\mu_k,\\Sigma)$$</td> <td>Like (linear) <b>regression</b>, model conditional\n",
    "$$p(\\mathcal{C}_k|x,\\theta)$$</td> </tr>\n",
    "\n",
    "<tr> <td>2</td><td>Leads to <b>softmax</b> posterior class probability\n",
    "$$ p(\\mathcal{C}_k|x,\\theta ) = e^{\\theta_k^T x}/Z$$\n",
    "with <b>structured</b> $\\theta$</td> <td> <b>Choose</b> also softmax posterior class probability\n",
    "$$ p(\\mathcal{C}_k|x,\\theta ) = e^{\\theta_k^T x}/Z$$\n",
    "but now with 'free' $\\theta$</td> </tr>\n",
    "\n",
    "<tr> <td>3</td><td>For Gaussian $p(x|\\mathcal{C}_k)$ and multinomial priors,\n",
    "$$\\hat \\theta_k  = \\left[ {\\begin{array}{c}\n",
    "   { - \\frac{1}{2} \\mu_k^T \\sigma^{-1} \\mu_k  + \\log \\pi_k}  \\\\\n",
    "   {\\sigma^{-1} \\mu_k }  \\\\\n",
    "\\end{array}} \\right]$$\n",
    "<b>in one shot</b>.</td> <td>Find $\\hat\\theta_k$ through gradient-based adaptation\n",
    "$$\\nabla_{\\theta_k}\\mathrm{L}(\\theta) = \\sum_n \\Big( y_{nk} - \\frac{e^{\\theta_k^T x_n}}{\\sum_{k^\\prime} e^{\\theta_{k^\\prime}^T x_n}} \\Big)\\, x_n$$ </td> </tr>\n",
    "</table>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "---\n",
    "The cell below loads the style file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!--\n",
       "This HTML file contains custom styles and some javascript.\n",
       "Include it a Jupyter notebook for improved rendering.\n",
       "-->\n",
       "\n",
       "<!-- Fonts -->\n",
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<!-- Custom style -->\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "#notebook_panel { /* main background */\n",
       "    background: rgb(245,245,245);\n",
       "}\n",
       "\n",
       "div.container {\n",
       "    min-width: 960px;\n",
       "}\n",
       "\n",
       "div #notebook { /* centre the content */\n",
       "    background: #fff; /* white background for content */\n",
       "    margin: auto;\n",
       "    padding-left: 0em;\n",
       "}\n",
       "\n",
       "#notebook li { /* More space between bullet points */\n",
       "    margin-top:0.8em;\n",
       "}\n",
       "\n",
       "/* draw border around running cells */\n",
       "div.cell.border-box-sizing.code_cell.running {\n",
       "    border: 1px solid #111;\n",
       "}\n",
       "\n",
       "/* Put a solid color box around each cell and its output, visually linking them*/\n",
       "div.cell.code_cell {\n",
       "    background-color: rgb(256,256,256);\n",
       "    border-radius: 0px;\n",
       "    padding: 0.5em;\n",
       "    margin-left:1em;\n",
       "    margin-top: 1em;\n",
       "}\n",
       "\n",
       "div.text_cell_render{\n",
       "    font-family: 'Alegreya Sans' sans-serif;\n",
       "    line-height: 140%;\n",
       "    font-size: 125%;\n",
       "    font-weight: 400;\n",
       "    width:800px;\n",
       "    margin-left:auto;\n",
       "    margin-right:auto;\n",
       "}\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Nixie One', serif;\n",
       "    font-style:regular;\n",
       "    font-weight: 400;\n",
       "    font-size: 45pt;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0,51,102);\n",
       "    margin-bottom: 0.5em;\n",
       "    margin-top: 0.5em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Nixie One', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 30pt;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0,51,102);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.3em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Nixie One', serif;\n",
       "    margin-top:16px;\n",
       "    font-size: 22pt;\n",
       "    font-weight: 600;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: regular;\n",
       "    color: rgb(102,102,0);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {    /*Use this for captions*/\n",
       "    font-family: 'Nixie One', serif;\n",
       "    font-size: 14pt;\n",
       "    text-align: center;\n",
       "    margin-top: 0em;\n",
       "    margin-bottom: 2em;\n",
       "    font-style: regular;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {  /*Use this for small titles*/\n",
       "    font-family: 'Nixie One', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 16pt;\n",
       "    color: rgb(163,0,0);\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.8em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 { /*use this for copyright note*/\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 9pt;\n",
       "    line-height: 100%;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "    font-family: \"PT Mono\";\n",
       "    font-size: 90%;\n",
       "}\n",
       "\n",
       ".boxed { /* draw a border around a piece of text */\n",
       "  border: 1px solid blue ;\n",
       "}\n",
       "\n",
       "h4#CODE-EXAMPLE,\n",
       "h4#END-OF-CODE-EXAMPLE {\n",
       "    margin: 10px 0;\n",
       "    padding: 10px;\n",
       "    background-color: #d0f9ca !important;\n",
       "    border-top: #849f81 1px solid;\n",
       "    border-bottom: #849f81 1px solid;\n",
       "}\n",
       "\n",
       ".emphasis {\n",
       "    color: red;\n",
       "}\n",
       "\n",
       ".exercise {\n",
       "    color: green;\n",
       "}\n",
       "\n",
       ".proof {\n",
       "    color: blue;\n",
       "}\n",
       "\n",
       "code {\n",
       "  padding: 2px 4px !important;\n",
       "  font-size: 90% !important;\n",
       "  color: #222 !important;\n",
       "  background-color: #efefef !important;\n",
       "  border-radius: 2px !important;\n",
       "}\n",
       "\n",
       "/* This removes the actual style cells from the notebooks, but no in print mode\n",
       "   as they will be removed through some other method */\n",
       "@media not print {\n",
       "  .cell:nth-last-child(-n+2) {\n",
       "    display: none;\n",
       "  }\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n",
       "<!-- MathJax styling -->\n",
       "<script>\n",
       "    MathJax.Hub.Config({\n",
       "                        TeX: {\n",
       "                           extensions: [\"AMSmath.js\"],\n",
       "                           equationNumbers: { autoNumber: \"AMS\", useLabelIds: true}\n",
       "                           },\n",
       "                tex2jax: {\n",
       "                    inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
       "                    displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
       "                },\n",
       "                displayAlign: 'center', // Change this to 'center' to center equations.\n",
       "                \"HTML-CSS\": {\n",
       "                    styles: {'.MathJax_Display': {\"margin\": 4}}\n",
       "                }\n",
       "        });\n",
       "</script>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "open(\"../../styles/aipstyle.html\") do f\n",
    "    display(\"text/html\", readstring(f))\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Julia 0.6.1",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}