{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# The SciPy Library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- The SciPy Library includes modules for\n",
    "    - optimization,\n",
    "    - interpolation,\n",
    "    - signal processing,\n",
    "    - linear algebra,\n",
    "    - sparse matrix representation,\n",
    "    - KD-Trees, Delaunay triangulation, convex hull computation\n",
    "    - statistics, and others"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "The SciPy library includes modules for optimization, interpolation, signal processing, linear algebra, statistics, sparse matrix representation and operation, among others. It also includes the `spatial` module, containing an implementations for the KD-Tree data structure (used on nearest-neighbor queries), Delaunay triangulation and convex hull computation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Optimization\n",
    "\n",
    "- The `optimize` module presents algorithms for\n",
    "    - function minimization\n",
    "        - Broyden-Fletcher-Goldfarb-Shanno (BFGS)\n",
    "        - simulated annealing\n",
    "        - least-squares\n",
    "        - Newton conjugate gradient\n",
    "    - root finding\n",
    "    - fitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x4c3a150>]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH+ZJREFUeJzt3Xec01XWx/HP0EQEBVxUigoiKCAoWBZXVmOhKAqsBWw0\nuzyKHRhwddZKUcGyurqUB1RQEEVcESlLAFcfkV5nqOMyiCggSBEdmDx/nIwMAwzpN/nl+3698kom\nk+R3jOHMzfndey6IiIiIiIiIiIiIiIiIiIiIiIiIiMhhDAc2AUuK3DcIWAEsAj4Ejivyu0xgFZAN\ntEpQjCIiEqE/A005MMm3BEoFb/cPXgAaAguBskBtYHWRx4mISAKFmnxnAz8Vu28qUBC8/TVQK3i7\nPTAGyAdysSR/QVRRiohIRGI1wr4NmBS8XQPIK/K7PKBmjI4jIiJhiEWS7wf8Bowu4TGBGBxHRETC\nVCbK53cDrgIuL3LfBuDkIj/XCt53gLp16wbWrFkT5eFFRNLOGuD0UB8czUi+DfAYVoPfU+T+icCN\nQDmgDlAPmHNQlGvWEAgEdInR5cknn3Qeg5cuej/1fibrBagbTqIOdSQ/BrgE+AOwHngSmyZZDjsB\nC/AV0ANYDowNXu8N3qdyjYiIA6Em+ZsOcd/wEh7/XPAiIiIOaf66R/h8PtcheIrez9jS++lOhsNj\nB4L1JRERCVFGRgaEkbs1khcR8TAleRERD1OSFxHxMCV5EREPU5IXEfEwJXkREQ9TkhcR8TAleRER\nD1OSFxHxMCV5EREPU5IXEfEwJXkREQ9TkhcR8TAleRERD1OSFxHxMCV5EREPc5vkCwqcHl5ExOvc\nJvlZs5weXkTE69wm+bffdnp4ERGvc7vHa+XK8N13cPTRDsMQEUkdqbXH63nnwcSJTkMQEfGyUJP8\ncGATsKTIfVWBqcBKYApQucjvMoFVQDbQ6rCv2rmzSjYiInEU6pD/z8BOYBTQOHjfQGBz8Lo3UAXo\nAzQERgPnAzWBaUB9oPhUmkBgxw6oVQtWroQTTojmv0NEJC3Eq1wzG/ip2H3tgJHB2yOBDsHb7YEx\nQD6QC6wGLjjkq1asCFdfDe+9F2q8IiIShmhq8idiJRyC1ycGb9cA8oo8Lg8b0R+aSjYiInFTJkav\nEwheSvr9QbKysmxBVHY2vlGj8HXpEqNwRES8we/34/f7I35+OFMoawOfsL8mnw34gO+B6sAM4Eys\nLg/QP3g9GXgS+LrY6wUCgWDuf/RROOooePbZsIIXEUk3iZxCORHoGrzdFZhQ5P4bgXJAHaAeMKfE\nV+rcGd55R20ORERiLNQkPwb4EjgDWA90x0bqLbEplJexf+S+HBgbvP4M6EHJpRw4+2w47jiYPTvM\n8EVEpCRuV7wGiuT+QYMgOxuGDXMXkYhIkgu3XJM8SX7DBjjrLLU5EBEpQWq1NSiqZk21ORARibHk\nSfKgOfMiIjGWPOUagJ071eZARKQEqVuuAbU5EBGJseRK8qCSjYhIcXv3QlYWbN8e9lOTL8lffjnk\n5dl0ShGRdLd3L3TpAl99ZZ0BwpR8Sb5MGbj5Zo3mRUTy8626sXkzTJgA5cuH/RLJdeK10MKF0L49\nrFsHpZLv75CISNzt3g033AClS8P77/++fii1T7wWUpsDEUln27dDmzZQpQqMHx/VAtHkTPIZGToB\nKyLp6bvv4NJLoUkTGDUKypaN6uWSs1wDanMgIuln/nwrVd97L2Rm2oC3GG+Ua0BtDkQkvXz4IbRu\nDUOGQN++h0zwkUjeJA8q2YiI9wUC8Pzz8MADMHkyXHddTF8+ecs1oDYHIuJtv/4Kd90FS5da1aLm\n4bfDLuSdcg1Ym4N27WD0aNeRiIjE1o8/2uLPnTth1qyQEnwkkjvJA3TtCiNHuo5CRCR2li2DP/4R\nLrkExo2DY46J26GSP8lfeils2QKLF7uOREQkelOmWF7729/g2WfjvuAz+ZN8qVJ2AlajeRFJdW+9\nZX1oPvzQ8loCJPeJ10I5Ofa1Ji/PetuIiKSSggKb9/7RR/Dpp1CvXsQv5a0Tr4XOOAPq1IHPP3cd\niYhIeH75BTp1gi+/tE6SUST4SKRGkgedgBWR1PPjj3DZZVCuHEybBscfn/AQUqNcA/DTTzaaX7fO\nmvaIiCSzvDxo2RKuvRaeeSZmK1hdlGsygWXAEmA0cBRQFZgKrASmAJWjPkqVKtCqlbXcFBFJZmvW\nwMUXw2232QyaGCX4SESb5GsDdwLNgMZAaeBGoA+W5OsD04M/R08lGxFJdsuW2USRXr3gscdcRxN1\nkv8ZyAcqAGWC198B7YDCbDwS6BDlcUzr1lauycmJycuJiMRUdraVaPr3h3vucR0NEH2S3wq8CPwX\nS+7bsBH8icCm4GM2BX+OXpkycMstalomIsln3TpL8M89B7fe6jqa30U76bwu8CBWttkOjAOK/9cF\ngpeDZGVl/X7b5/Ph8/mOfMSuXeHqq+Gpp7Q1oIgkh7w860OTmQndusX0pf1+P36/P+LnR3s2oBPQ\nErgj+HNnoDlwGXAp8D1QHZgBnFnsueHNrinq3HOtNWerVpE9X0QkVrZsgYsugttvT0gNPtGza7Kx\npH508KBXAMuBT4Cuwcd0BSZEeZwD3XEHDB0a05cUEQnbr7/CX/5i3XKT4CTrocRiXk8vLJEXAPOx\nUX0lYCxwCpALdMTq9UVFPpLfvh1q17Y+89WqRfYaIiLRCASsfLxrl3WSTFD5ONyRfOoshiquWzdo\n3BgeeSRmAYmIhOzpp22jj5kzoUKFhB3Wm71rDqWwZBPNHwoRkUh88IHln4kTE5rgI5G6Sf6ii+z6\nP/9xG4eIpJecHLj3XusoWb2662iOKHWTfEaGTsCKSGLt3g3XX2+9aJo1cx1NSFK3Jg/W4a1ePcjN\nhcrRt8cRETmsQAC6d4e9e21BpqN+NOlTkwebWdOqFYwZ4zoSEfG6ESPgm2/gH/9w2nAsXKk9kgfb\nLzEzE+bNi/61REQOZfVqaN4cZs2Chg2dhpJeI3mAK66wFWfz57uORES8aN8+m7L9+OPOE3wkUj/J\nlyply4l1AlZE4uHFF6FsWejZ03UkEUn9cg1Yc6AmTWD9ejjmmNi8pojIkiW2fd/cuXDqqa6jAdKx\nXANQq5btwvLuu64jERGvyM+HLl1gwICkSfCR8EaSB7jvPnjtNa2AFZHYeOklOOkkmzaZwrxRrrFX\ns5Mib75po3oRkUitWwfnnw9z5sBpp7mO5gDpWa4Bm7daOJoXEYlUIGC55JFHki7BR8I7I3mAHTus\ndrZkCdSsGdvXFpH0MH48PPEELFgA5cq5juYg6TuSB6hUyfaAffNN15GISCr6+Wd48EFb1ZqECT4S\n3hrJg+2W7vPBt9/CUUfF/vVFxLsefRS2boXhw11Hcljps2lISVq2tKlPnTvH5/VFxHtWrYILL4Rl\ny+DEE11Hc1jpXa4p9NBDtkpN0ylFJFS9etlIPokTfCS8meSvvNIWMkyf7joSEUkFM2bAwoVWj/cY\nbyb5jAz7izxokOtIRCTZ7dtn3/4HDYLy5V1HE3PeTPIAN98MS5fCokWuIxGRZDZiBBx7LFx3netI\n4sKbJ14LDRhgif7tt+N7HBFJTbt32+5yH38M553nOpqQaHZNUdu22Yq1RYvg5JPjeywRST39+9te\nFGPHuo4kZC6SfGVgKNAICADdgVXA+8CpQC7QEdhW7HnxT/IADz9sPedfeCH+xxKR1PHTT1C/Pnzx\nBZxxhutoQuYiyY8EZgLDgTLAMUA/YDMwEOgNVAH6FHteYpL8+vVwzjm2SKpatfgfT0RSQ2YmbN4M\n//yn60jCkugkfxywACjexScbuATYBJwE+IEziz0mMUkeoEcPa3kwYEBijiciyW3jRmjUCBYvtv0o\nUkiik/w5wJvAcuBsYB7wIJCHjd4Lj7G1yM+FEpfk8/Lg7LNhxQo44YTEHFNEklePHlChQkqWccNN\n8mWiPF4ZoBlwH/ANMIRDlGWCl4NkZWX9ftvn8+Hz+aIM5zBq1YKbbrJ5sJo7L5Le1q6F99+HnBzX\nkYTE7/fj9/sjfn60I/mTgK+AOsGfWwCZWPnmUuB7oDowA5flGti/D+yKFZ5btiwiYbjjDqhRA556\nynUkEUl075rvgfVA/eDPVwDLgE+ArsH7ugITojxO9GrVsjbEGsmLpK+1a2HCBE+2LzicWMyuORub\nQlkOWINNoSwNjAVOwfUUyqI2bIDGjW2BVI0aiT22iLh3xx1QvTo8/bTrSCKmxVBH0ru3TZsaNizx\nxxYRd9autX1bV62CqlVdRxMxJfkj2b7dFj589hk0bZr444uIGx4YxYOSfGjeeAPGjbNWxBku3wIR\nSQiPjOJBm4aE5s47YdMm+OQT15GISCIMHAj33JPyCT4S6TmSB5g8GXr2tJOwHtmwV0QOYeNGaNjQ\n5sV7YDGkRvKhatPGOlS+8orrSEQknl5+2aZPeyDBRyJ9R/Kwf+PeOXMs4YuIt2zfbv+2586FOnWO\n/PgUoJF8OOrVs817775bm36LeNE//mF7PnskwUcivUfyAHv3wh//CPfdB927u45GRGJlzx5L7p9/\nbi1NPEIj+XCVKWMLo3r3thM0IuINI0dCs2aeSvCR0Ei+UN++trHI+PGaOy+S6vbts0WPI0bAn//s\nOpqY0kg+Uk88AWvWqN2BiBeMH2/dZlu0cB2JcxrJF7ViBVx8McycafNqRST1BAJw7rmQlQXt2rmO\nJuY0ko9Ggwa2ReB118GOHa6jEZFITJ0Kv/4KV1/tOpKkoJH8odx9t3Wq/OAD1edFUs3ll0PXrtCl\ni+tI4kIj+Vh45RWbafPEE64jEZFwzJkDq1fbdp8CKMkf2lFH2e4xo0fD8OGuoxGRUA0YAI88AmXL\nuo4kaahcU5KcHPD54O9/h2uvdR2NiJQkJ8emS65bB8cc4zqauAm3XFMmfqF4wBlnwKRJ1sysXDmd\nyBFJZoMGwf/8j6cTfCQ0kg/FnDlwzTVWq+/UyXU0IlJc4f7Nq1bB8ce7jiauNJKPhwsusGlZV14J\n331nO71r1o1I8hg82GbUeDzBR0Ij+XB8+62VbJo3h1dfhfLlXUckIj/9BHXrwqJFcPLJrqOJO02h\njKdTT4Uvv4Rt26wPfU6O64hE5PXXbWVrGiT4SCjJh6tSJRg7Fu66Cy66CF54wdoVi0ji7d5t58p6\n9XIdSdKKVZIvDSwACnfGrgpMBVYCU4DKMTpOcsjIgHvvtROyU6ZAo0bWECnVyk8iqW7ECPtWrV5T\nhxWrmvzDwLlAJaAdMBDYHLzuDVQB+hR7TurV5A8lELCTsr172zTLrCxo3RpK6UuSSFzt3Wu7u40e\nbYk+TbioydcCrgKGFjlwO2Bk8PZIoEMMjpOcMjKgVSuYNw8eeAD69YPTT4f+/WHTJtfRiXjX2LFw\nyilpleAjEYskPxh4DCgoct+JQGGG2xT82dtKlYKbb7Zk//77Nl+3fn249FIYMsRW4YlIbAQCNpDq\nU7xAIMVFO0/+auAHrB7vO8xjAsHLQbKysn6/7fP58PkO9xIpJCMDzj/fLq++CtOmwccfw3PPwbHH\nWr/6wkudOppvLxKJSZPs306bNq4jiTu/34/f74/4+dFmmOeAzsBeoDxwLPAhcD6W9L8HqgMzgDOL\nPdcbNflQFRTA8uUwezbMmmUbk5QqZb02WrSw67POUi1fJBQtWsB998GNN7qOJOHCrcnHchh5CfAo\ncA12wnULMAA74VoZr554jVQgYNsNfvGFJf7Zs+HHH+FPf4KWLaFtWzupJCIHmj0bune3PZnLpN+i\nfddJ/hHspGtVYCxwCpALdAS2FXt8eif5Q9m0yT7Akyfb19GKFaFDB1uu3aiR6+hEksNVV9m/i7vu\nch2JEy6TfLiU5EsSCMDChTaDYNQoqFHDRi/dukGFCq6jE3Fj4UL7lrt2re37kIaU5L1o3z47gfvm\nm9ZW4YEHrKXqsce6jkwksTp1soaBjzziOhJnlOS9btkyeP55K+n06WMJX7vgSDpYtcrOWa1da+1F\n0pSSfLrIyYH774fvv4c33rA+OiJeduedVrb8299cR+KUknw6CQRg3Dh4+GFrgTx4MBx9tOuoRGIv\njTYFORK1Gk4nGRnQsaPNv9++3frcr1zpOiqR2HvpJZt0kOYJPhIayXtFIGAnZv/6V3jtNW1TKN6x\nZYutGVm8GGrVch2NcyrXpLv58+G66+COO6BvX7VNkNT3+OPwww/w1luuI0kKSvICGzdaT4+LL4aX\nX1arBEldW7ZYo79586B2bdfRJAXV5AWqV7f+OEuWwE03wa+/uo5IJDKDB9s3UyX4iGkk72V79liS\nL5yFo/n0kkq2brVavEbxB9BIXvYrX9562//2m7VEKCg48nNEksXgwXDttUrwUdJIPh388gtceSU0\naGA72+tkrCS7wlH83Lm274L8TiN5OdjRR8PEifa1t29f19GIHNlLL1mnSSX4qGkkn062bLEFU337\nWvlGJBlt2gQNG9p04FNPdR1N0tEUSilZdrZNrRw/3najEkk2999vm4EMHuw6kqSkJC9HNmWKbUTy\n5Zf6OizJZe1a2x85OxuqVXMdTVJSTV6OrFUryMyEa66BnTtdRyOy3xNPQM+eSvAxpJF8ugoE4Pbb\nIT/fdp7SjBtxbdEiG4CsXp3W/eKPRCN5CU1GhjUyW7AAhg1zHY2ITQjo21cJPsY0kk93K1bYidjp\n06FJE9fRSLqaMgV69LCdz9J079ZQaSQv4WnQwGYx3HAD7NjhOhpJR/n58OCDNjdeCT7mNJIXc+ed\n9o/tf//XdSSSbl5+GT79FD7/XOeGQqAplBKZnTuhaVPo39+6/okkwo8/2sKnmTPtWo4o0eWak4EZ\nwDJgKdAzeH9VYCqwEpgCVI7yOBJvFSvCO+9YXXTjRtfRSLp4/HG45RYl+DiKdiR/UvCyEKgIzAM6\nAN2BzcBAoDdQBehT7LkaySejJ5+EOXNg0iR9dZb4mj/fGudlZ0OVKq6jSRmJHsl/jyV4gJ3ACqAm\n0A4YGbx/JJb4JRU8/rj1uHn9ddeRiJfl59s6jUGDlODjLJZDtdrATOAs4L/Y6L3wGFuL/FxII/lk\ntXIl/OlPNqI/7TTX0YgXPfssfPGFvjFGINyRfJkYHbciMB54ACg+Dy8QvBwkKyvr99s+nw+fzxej\ncCQq9etDnz62Gfj06fpHKLG1fLlN250/X5+tEPj9fvx+f8TPj8U7XBb4F/AZMCR4Xzbgw8o51bGT\ns2cWe55G8sls3z4bzd9+O9x1l+toxCv27bPup7feaif5JWyJrslnAMOA5exP8AATga7B212BCVEe\nRxKtdGkYPhz69YP1611HI17xyivWRviee1xHkjaiHcm3AGYBi9lfkskE5gBjgVOAXKAjsK3YczWS\nTwXPPGMtiT/9VF+tJTpz58JVV8FXX0Hduq6jSVlaDCWxlZ8PF1wADz0EXbq4jkZS1bZtcO65ttju\nhhtcR5PSlOQl9hYsgNatrRVs9equo5FUEwjA9dfbZ+e111xHk/LUoExir2lT623To4f9gxUJx2uv\nQW4uvPii60jSkkbyEppff4VzzoGnn7ZRmUgoZsyATp1Uh48hlWskfv7zH6unLl0KVau6jkaSXXY2\nXHIJjBkDl13mOhrPUJKX+Lr/futYOWKE60gkmf33v7YZTVYWdOvmOhpPUZKX+NqxAxo3hrfesv04\nJTZ27rSRb26utd/95Re7//jj4Q9/gBNOgDPOgGOPdRpmSDZutBF8jx62GYjElJK8xN/kyXDvvbBk\nibUolvDl54PfDxMn2vXatdZOok4dS+gVKthJ7i1bLOlv2gQ5OVCzJjRrZquRr7wS6tVz/V9yoNxc\naNkSbrsNMjNdR+NJSvKSGF26WF1+yJAjP1b2y8uzVZ+jRkHt2tChA1x+uZ3ULlu25Ofu3WuJft48\n22Tjs8/sj0HbtnDjjdC8udsFa/PnQ/v20KuXlfUkLpTkJTG2bIGzzoKPPrLkIiXLy7OFQGPGWI36\n7rtt5B6NQAAWL4aPP4bRo+G33+Dmm20TjgYNYhJ2yN57zxL7G29o9lWcKclL4rz/Pjz1lI3gtAHz\noe3ebScfhw61Zm+PPWblmFgLBGzR2ujR9ofkhBOgc2dL+iedFPvjFfr5Z6u7f/GFfR6aNo3fsQTQ\nYihJpI4d4fTT4fnnXUeSnKZNs5PUGzZYe91Bg+KT4MHKNM2awQsv2MyWF1+0cyYNGkCbNvDuu7Br\nV+yOV1Bgr9mggZWZ5s9Xgk9SGslLdDZssHryv/9tCU1spkzPnvaevPGGnSB1ZfduK+e8/bY1mrv8\ncqubt21rM3cieb1x42DgQDvpPmQIXHhh7OOWw1K5RhLvrbdg2DBLIqVLu47GreXLrSbdvDm8/DJU\nquQ6ov02b7ZuohMm2GYw9evbLJ0LL7TmYbVrQ7lyBz5n717bKeybb2DKFDvZ27y5/RFr3VqdSR1Q\nkpfEKyiwEWK7dtatMl2984799w8cCN27u46mZHv22CydL7+0lgOLFtnJ4UqV4LjjrMa/axds3WrT\nOps2hSuusFbBNWu6jj6tKcmLG6tX2wgvHfeF3bfPpg1+8gl88AE0aeI6osjs3WtJ/eefbYReoQJU\nq2abfEjSUJIXdwYNgs8/h6lT0+dr/K5dNmVx+3b48EOoUny/epHY0uwaceehh2xziHTpa7NxI/h8\nULmy/XFTgpckpJG8xNaiRbasfe5cOOUU19HEz9q1VqPu1g3++tf0+eYizqlcI+71728j2+nToZQH\nvywuW2YzS/r1sx4+Igmkco2499hjdjLypZdcRxJ733xjM4kGDFCCl5SgkbzER24unH++rfo8+2zX\n0cTGzJm2acrQoTZdVMQBjeQlOdSubUvsb7nF5mSnukmTLMGPGaMELylFI3mJn0AAbrrJFte8+abr\naCI3frxtgPHxx+q4Kc4l00i+DZANrAJ6x/E4kqwyMqzlgd9vvVNS0bvvwn332YlkJXhJQfEayZcG\ncoArgA3AN8BNwIoij9FIPl0sWWIbOfv90KiR62hCN3SotQmeMgUaNnQdjQiQPCP5C4DVQC6QD7wH\ntI/TsSTZNW5sq2Gvv946NKaCV1+Fp5+GGTOU4CWlxSvJ1wTWF/k5L3ifpKtu3eCii2zbwIIC19GU\nbMAA6yA5a1by7aEqEqZ4dR4KqQ6TlZX1+22fz4fP54tTOJIU/v53aNXKmnm98ILraA4WCFh5ZuxY\nmy6pbouSBPx+P36/P+Lnx6sm3xzIwk6+AmQCBcCAIo9RTT4dbd1qPcwfeCC5FhMVFFjvHb/fGqzF\nawcnkSiFW5OP10h+LlAPqA18B3TCTrxKuqta1TauaNECTj3V+pO7tmePlZF++MFG8JUru45IJGbi\nVZPfC9wHfA4sB97nwJk1ks7q1rW2vN262QbQLm3bZnugAkyerAQvnqPFUOLO1Km2Ivbjj93sE7py\nJXToYF0zBw/2ZjM18ZxkmUIpcmQtW8KoUbax9LRpiT32v/5lJaMHH7SZNErw4lEayYt7s2bZHPrB\ng21kH0/79sGzz9pK3HHj3HyDEImC+slLalq61Bp/dewIzzwTn31FV660DbbLloXRo6FGjdgfQyTO\nVK6R1HTWWfD11zBvnm2pl5sbu9cuKIAhQ2zqZqdO8O9/K8FL2lCSl+RRrZo1AmvfHs47DwYOhN9+\ni+41p0+31xo/Hv7v/6BnT9XfJa2oXCPJafVqOym6ZAlkZsKtt0LFiqE997ff4JNPbGeqH36A556z\nmr/2YRUPUE1evOWrr6yXzMyZ0Lat7a16/vk2175sWXtMfj6sWQMLFtgsnYkToUEDaxF83XVQurTb\n/waRGFKSF2/auNGS99SpsGiR1ezLl7fR+e7dtnq2SRNrady2LZx2muuIReJCSV7Sw759sGuXNRWr\nVEl1dkkbSvIiIh6mKZQiIvI7JXkREQ9TkhcR8TAleRERD1OSFxHxMCV5EREPU5IXEfEwJXkREQ9T\nkhcR8TAleRERD1OSFxHxMCV5EREPiybJDwJWAIuAD4HjivwuE1gFZAOtojiGiIhEIZokPwVoBJwN\nrMQSO0BDoFPwug3wepTHkRD4/X7XIXiK3s/Y0vvpTjTJdypQELz9NVAreLs9MAbIB3KB1cAFURxH\nQqB/RLGl9zO29H66E6sR9m3ApODtGkBekd/lATVjdBwREQlDmSP8fipw0iHu7wt8ErzdD/gNGF3C\n62h3EBERB6LdGaobcCdwObAneF+f4HX/4PVk4EmspFPUaqBulMcXEUk3a4DTE3GgNsAy4A/F7m8I\nLATKAXWCAbncZlBERCKwCvgWWBC8vF7kd32xkXo20DrxoYmIiIiISMzdgJV59gHNiv1Oi6iik4XN\nZir8dtXGaTSpqQ32+VsF9HYcixfkAouxz+Mct6GkpOHAJmBJkfuqYpNiVmLrlSo7iKtEZwL1gRkc\nmOQLa/llgdpYuUeLqMLzJPCw6yBSWGnsc1cb+xwuBBq4DMgD1mFJSSLzZ6ApByb5gUCv4O3e7J/k\nckgukmg29heoOC2iig2d5I7cBdjnLhf7HL6HfS4lOvpMRm428FOx+9oBI4O3RwIdSnqBZBopaxFV\nbNyP9RMaRhJ+jUtyNYH1RX7WZzB6AWAaMBebbi3ROxEr4RC8PrGkBx9pMVSkQllEFQotojrY4d7b\nfsAbwFPBn58GXgRuT1BcXqDPW+xdBGwEqmGf3WxsdCqxEeAIn9t4JfmWETxnA3BykZ9rBe+TA4X6\n3g4lvD+ocvBn8GQO/HYp4dsYvP4R+AgriSnJR2cTNtD7HqgO/FDSg12Xa4rW6iYCN7J/EVU9dDY+\nXNWL3P4LB56skSObi33uamOfw07Y51IiUwGoFLx9DDZjTp/J6E0EugZvdwUmOIzlkP6C1T1/wf4S\nfVbkd1pEFZ1R2HS1Rdj/+BJrdXJIVwI52Ocw8wiPlZLVwWYoLQSWovczEmOA77D+YOuB7thspWkk\n8RRKEREREREREREREREREREREREREREREREREZGw/T/QuyBtUuJS0QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x4a5ff10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def f(x):\n",
    "    return x**2 + 10 * sin(x)\n",
    "\n",
    "x = arange(-10, 10, 0.1)\n",
    "plot(x, f(x), 'r-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### BFGS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: -7.945823\n",
      "         Iterations: 5\n",
      "         Function evaluations: 24\n",
      "         Gradient evaluations: 8\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([-1.30644003])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.optimize import fmin_bfgs\n",
    "\n",
    "xmin = fmin_bfgs(f, 0)\n",
    "xmin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x4c18c10>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEeCAYAAABv8mXfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYVOX5//H30kRAacqiFEEUpFgABVHKqqhIIthJbCBf\njT0iahQxsCZRwRaViCagsBLFBiKoGBAYBFHUn7QgTaRXBYNIh53fH/dZd1h22Z2d8sw583ld114z\nc6ac+zDMPc/c5ykgIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIifrAS2AlsB7YCHwB1vftGAnu8+/L+\nro547u+A2cAvwCbgC+D2iPvrAmOAH4D/AQuAngk5ChERKdQK4Hzv+hHAK8B73u0RwF+KeN59wEbg\nCqCyt+0M4N9Aee/2NOBZ4EigjHd/lzjGLiIixYhM8gBdgSXe9ZEUnuSrYq33y4t57e3AaTHGJ5Jw\nZVwHIJJgGd5lJaAH8Ll3OxxxX6R2WKv//WJe9wtgqPea9WMPU0REorUSa3H/BOwF1gItvPtGAru8\n+34CNnvbrwc2FHidWd5jdgIdvG3VgCeA/wL7gTnAmfE/BBERKUpkuSYDK8FsATIpuiZ/CbCPwn/l\nrgE6FrK9pvd6a2OMVyTuVK6RdBHGTroeANp72wor13yO9bq5LIrX3gI8AxwPVI8hRpG4U5KXoMuI\nuOyOlVm+pfAED9Yd8lGs3n4lcBT5vWcqRzxuMNAcKOc95nZgGVbWERGRJFhBfj/5n4H5wO+9+w7X\nhRLgWqyf/A6sXv8FcDP5XShfAJZ6r70ZGA80iW/4IsnzKjYgZEHEtqeARcA8YCzW9SxPP6xVsxi4\nKEkxiohIKXUAWnJwkr+Q/HLPIO8PoBkwF2vxNAC+Q2UhEREnSpp8Z3BorXEykOtdn03+cPHuwGis\nh8JKLMm3iSlKEREplXi1sHsDH3nXj+fgrmRrgTpx2o+IiEQhHkm+PzbQ5I3DPCYch/2IiEiUysX4\n/F7YfCAXRGxbB9SLuF3X23aQRo0ahZcvXx7j7kVE0s5y4KSSPjiWlnwX4AGsBr87Yvt4bJrWCkBD\n4GTgy0OiXL6ccDgc2L+BAwc6j0HHp+NLx+ML8rGFw2GARtEk6pK25EcDnYBjsKHdA7FukhWwE7Bg\nIwXvwAaavO1d7ve2qVwjIuJASZP87wvZ9uphHv+49yciIg6p/3qCZGVluQ4hoXR8/hbk4wvysZVG\nUfN3JEPYqy+JiEgJZWRkQBS5Wy15EZEAU5IXEQkwJXkRkQBTkhcRCTAleRGRAFOSFxEJMCV5EZEA\nU5IXEQkwJXkRkQBTkhcRCTAleRGRAFOSFxEJMCV5EZEAU5IXEQkwJXkRkQBTkhcRCTAl+URYvRr+\n/W/XUYiIKMknxL598Kc/gVa+EhHHlOQT4cQToUwZ+O4715GISJpTkk+EjAzo1AmmT3cdiYikOSX5\nRFGSF5EUUNIk/yqwCVgQsa0GMBlYCkwCqkXc1w9YBiwGLoo9TB/KS/Kqy4uIQyVN8iOALgW2PYQl\n+cbAFO82QDOgh3fZBRgaxX6Co3Fj2LsXVq50HYmIpLGSJt8ZwE8FtnUDcrzrOcBl3vXuwGhgH7AS\n+A5oE1OUfqS6vIikgFha2JlYCQfvMtO7fjywNuJxa4E6MezHv7KylORFxKl4lVHC3t/h7k8/nTpB\nKOQ6ChFJY+VieO4moDawETgO2OxtXwfUi3hcXW/bIbKzs3+9npWVRVZWVgzhpKCmTWHHDhsBW7++\n62hExIdCoRChGBqLGVE8tgEwATjVu/0ksAUYjJ10reZdNgPewOrwdYBPgJM4tDUfDqdDz5OrroLu\n3eGGG1xHIiIBkJGRAVHk7pKWa0YDs4AmwBrgJmAQcCHWhfJ87zbAt8Db3uVE4A7StVwDOvkqIk5F\n05KPt/Royc+fD1deCcuWuY5ERAIgUS15Ka0WLWDrVli/3nUkIpKGlOQTrUwZ6NBBJRsRcUJJPhlU\nlxcRR5Tkk0FJXkQc0YnXZDhwAI45BhYvhszM4h8vIlIEnXhNRWXLQvv2as2LJNL+/fDxx66jSDlK\n8smiko1IYg0eDM8+q+m9C4hlWgOJRqdOMHKk6yhEgmnePHjuOfjmG5sBVn6llnyytGwJa9bAjz+6\njkQkWPbuhRtvhKefhnr1in98mlGST5Zy5eCcc+DTT11HIhIs/fpBgwaW6OUQSvLJpLq8SHy99Ra8\n9x6MGKEyTRGU5JNJSV4kfhYuhLvugjFjoEYN19GkLPWTT6Z9++w/4+rVUL2662hE/GvbNmjTBh5+\nGHr2dB1NUqmffCorXx7OPhtmzHAdiYh/5eZCr15wwQVpl+BLQ0k+2bTuq0hsBg+GjRuty6QUS0k+\n2bTuq0jpTZ4MQ4bAO+9AhQquo/EF1eSTbc8eqFkT1q2DqlVdRyPiH6tWQdu28Oab9os4Takmn+qO\nOMJOGM2c6ToSEf/YvdtWWHvggbRO8KWhJO+CulKKROfuu+HEE6FvX9eR+I6SvAtK8iIlN3IkfPYZ\nvPKKBjyVgmryLuzaBcceCxs2wFFHuY5GJHUtXGjlmVAImjd3HU1KUE3eD448Elq1glmzXEcikrp2\n7ICrr4annlKCj4GSvCsq2Ygc3p13WieFXr1cR+Jr8Ujy/YCFwALgDeAIoAYwGVgKTAKqxWE/waIk\nL1K0nBz48kt48UXXkfherDX5BsBUoCmwB3gL+AhoDvwIPAk8CFQHHirw3PStyYP9FM3MhE2boHJl\n19GIpI7Vq6F1a5gyBU47zXU0KSfZNfmfgX1AJWyVqUrAeqAbkOM9Jge4LMb9BE/lynD66fD5564j\nEUkd4TDcfDP06aMEHyexJvmtwDPAaiy5/w8r02QCm7zHbPJuS0Eq2YgcbNgw2LoVHnzQdSSBEWuS\nbwT0wco2xwNVgOsLPCbs/UlBSvIi+TZuhP79rV98OS0/HS+x/kueCcwCtni3xwLtgI1Abe/yOGBz\nYU/Ozs7+9XpWVhZZ6TZc+ZxzbOHhXbusW6VIOnvgAejdG1q0cB1JSgmFQoRimNQw1hOvpwOvA2cB\nu4GRwJfACVjiH4ydcK2GTrwWrm1bmzo13b7gRCJ9+ilcdx0sWgRVqriOJqUl+8TrPOA14Gtgvrft\nX8Ag4EKsC+X53m0pjEo2ku7277c+8c8+qwSfAJrWwLUPP4RnnoGpU11HIuLG8OEwapRNXaC5aYoV\nbUteSd61bdugTh3YssWmIRZJJzt3QuPGthh327auo/EFzV3jN1WrQpMm8NVXriMRSb4hQ2zdYyX4\nhFE/pVSQt+5r+/auIxFJnq1b4emntYBOgqklnwq07quko+efh27d7JesJIxq8qngp5+gfn2ry2tx\nYkkHP/9sKz198QWcdJLraHxFNXk/ql7dWjOax0bSxdChcPHFSvBJoCSfKi6+GP7zH9dRiCTezp3w\n3HPQr5/rSNKCknyqUJKXdPHqq9CunaYvSBLV5FPFvn227uuyZXYpEkS5uVaaHDkSzj3XdTS+pJq8\nX5Uvb10pJ092HYlI4nz0kY0NOecc15GkDSX5VHLRRSrZSLA9/zzcc4+mL0gilWtSyfLlNiBq/Xp9\nCCR4Fi6ECy+EFSs0hUcMVK7xs0aNbFnA+fOLf6yI3wwZArfdpgSfZJrWINXk9bI5/XTXkYjEz/bt\n8NZb8O23riNJO2rJpxp1pZQgevNNOO88OO4415GkHdXkU8327XD88bbeZeXKrqMRiY82beDRR+GS\nS1xH4nuqyfvdUUdB69aasEyCY+5c2LTJeo9J0inJp6KuXW3FKJEgGDbMFuguW9Z1JGlJ5ZpUtHgx\ndO4Ma9aoK6X4265dULeutebr1XMdTSCoXBMETZpApUowZ47rSERiM2ECtGqlBO+QknwqysiwxRQm\nTHAdiUhsRo2CG25wHUVaU5JPVZdeCuPHu45CpPQ2b4YZM+CKK1xHktaU5FPVuefCypWwdq3rSERK\n5803rbFSpYrrSNJaPJJ8NeBdYBHwLdAWqAFMBpYCk7zHSDTKlbNeNirZiF+NGgU33ug6irQXjyT/\nPPAR0BQ4DVgMPIQl+cbAFO+2RKt7dxg71nUUItFbvBjWrYPzz3cdSdqLtX9eVWAOcGKB7YuBTsAm\noDYQAk4p8Bh1oSzOjh02+nX5cjjmGNfRiJTcX/4CW7faMn8SV8nuQtkQ+AEYAXwDDAMqA5lYgse7\nzIxxP+mpcmXo0gXee891JCLReest6NHDdRRC7LNQlgNaAXcBXwHPcWhpJuz9HSI7O/vX61lZWWRl\nZcUYTgBdcw28/DLccovrSERKZuFCm4OpbVvXkQRCKBQiFMM0J7GWa2oDn2MteoD2QD+sfHMesBE4\nDpiGyjWls3OnlWy09qv4xYAB9v/26addRxJIyS7XbATWYCdYAToDC4EJQE9vW09gXIz7SV+VKqlk\nI/4RDsPbb9svUEkJ8ZgY5XRgOFABWA7cBJQF3gbqAyuBa4D/FXieWvIlNXYsvPgiTJniOhKRw5s/\n33qFff+95l1KkGhb8pqgzA80yZP4Rf/+sH8/DB7sOpLA0gRlQXTkkXD11Ta4RCRVqVSTkpTk/aJX\nLxg50j5IIqlo7lzIzbVZJyVlKMn7Rdu2UKYMfP6560hECpfXilctPqUoyftFRgbcdBOMGOE6EpHC\njRunGSdTkE68+sm6ddCihc1MqUW+JZXkrWa2erX94pSE0YnXIKtTBzp2hNdfdx2JyMHef9+6TirB\npxy9I37zxz/CCy/oBKyklnHj4LLLXEchhVCS95u8qVunTnUbh0ieDRtgyRLo1Ml1JFIIJXm/ycjI\nb82LpILx4+GSS6BCBdeRSCGU5P3ouuvgs89snnkR11SqSWnqXeNX/fvDDz/Av/7lOhJJZz//bFNu\nrFsHRx3lOpq0oN416aJvXxgzBlascB2JpLOJE6FDByX4FKYk71c1a8Idd8Bjj7mORNKZSjUpT+Ua\nP/vpJzj5ZJg9Gxo1ch2NpJs9e6B2bVi0yC4lKVSuSSfVq8Pdd8Mjj7iORNJRKATNminBpzgleb+7\n/36btEz95iXZVKrxBZVrgmDcOOjXD+bNU19lSY7cXOtVM326lQwlaVSuSUfdu1tN/tlnXUci6eKr\nr6xcqASf8pTkgyAjA4YMsSQ/Z47raCQdqFTjG0ryQdGwoSX6a66xASoiiaQk7xuqyQfN7bfD1q3w\n5ptaoUcSQ3PHO6WafLr7+99h1Sp44AFNRyyJobnjfUXvUtBUrAgffQSTJ8Nf/uI6GgkilWp8JV5J\nviwwB5jg3a4BTAaWApOAanHaj5REjRowaRKMHg333Qf797uOSIJCc8f7TryS/D3At0BefeAhLMk3\nBqZ4tyWZMjNh1iyYPx+6doUtW1xHJEEwfjx06aLxGD4SjyRfF+gKDCf/ZEA3IMe7ngPot50LNWrY\nLIFnnGELgL/2mur0Eptx4+Dyy11HIVGIR/eLd4DHgaOB+4FLgZ+A6hH72BpxO4961yTT11/DbbdB\n+fI2Ova3v9WJM4nOtm1Qr57mjncs2t415WLc32+BzVg9PquIx4TJL+McJDs7+9frWVlZZGUV9RIS\nszPPtNkq330XsrPhwQfhppvg2mtteLpIcSZOhI4dleCTLBQKEQqFSv38WFvyjwM3APuBilhrfixw\nFpb0NwLHAdOAUwo8Vy15V8JhmDEDRo2yhUdatYLrr4crroCjj3YdnaSq3/3O+sfffLPrSNJatC35\neI6W6UR+ueZJYAswGDvpWo1DT74qyaeC3bthwgR4/XWYNg0uvhh69bKTayrnSJ49e+xk/pIldinO\nuB4MlZe1BwEXYl0oz/duSyqqWBGuvtpOqK1YYS21P/8ZTjkFXnoJ9u1zHaGkgqlT4dRTleB9SNMa\nyKHCYfjsM/jrXy3xP/UUdOumaRLS2a23QuPGNu5CnHJZromWkrwffPwx9OkDrVtby141+/Rz4ADU\nqWNf/Fpm0jnX5RoJmi5d4JtvLLmfcQbMnes6Ikm22bOhVi0leJ9SkpfiVapkrfgnnoCLLrLVgCR9\naK4aX1OSl5Lr0cOmML76avjgA9fRSDKEw/Deexrl6mNK8hKd88+3BH/TTbaAuATbt9/C3r1WqhNf\nUpKX6LVpAzk51rpbutR1NJJIeaUa9azyLSV5KZ2uXeFvf7M5cLZvdx2NJMq776pU43PqQimxueUW\n2LXLpkhQay9Yli61eePXroWyZV1HIx51oZTkev55mDPHpjGWYHnnHbjqKiV4n1NLXmL33//CeefB\nl19Cw4auo5F4Oe00GDoU2rd3HYlEUEtekq9FCxvufscdWpQkKBYtstXEzjnHdSQSIyV5iY/77rPF\nJN56y3UkEg9vv23jITQTqe+pXCPxM3u2dbdbuNCWHhT/at4chg+Hdu1cRyIFqFwj7rRtC9272+yV\n4l8LF1q32LZtXUcicaCWvMTX5s3QrBl88QWcdJLraKQ0BgyAHTvgmWdcRyKFUEte3KpVC/r2tcXC\nxX/CYavH9+jhOhKJEyV5ib8+faw+/9lnriORaH3zja0GdtZZriOROFGSl/irVAkefRT693cdiUQr\nJwduvFGjlwNENXlJjP37bZ3Y4cMhK8t1NFISe/dC3bp2PuXEE11HI0VQTV5SQ7lytiB4drbrSKSk\nJk60L2Yl+EBRkpfEue46GyA1bZrrSKQk8ko1Eigq10hivfYajBihRJ/qtmyxNVxXrYKqVV1HI4eh\nco2klt//Hr7/Hr76ynUkcjijRtnaAErwgRNrkq8HTAMWAv8F/uhtrwFMBpYCk4BqMe5H/Kp8ebj3\nXnjqKdeRSFHCYXj5ZbjtNteRSALEmuT3AfcCzYGzgTuBpsBDWJJvDEzxbku6uvlmmDoVli93HYkU\nJhSyE+Xnnus6EkmAWJP8RmCud/0XYBFQB+gG5Hjbc4DLYtyP+FmVKnDrrfDss64jkcK89BLcfrv6\nxgdUPN/VBsB0oAWwGqgesY+tEbfz6MRrOtm4EZo2tSXljj3WdTSSZ8MGm2to1So4+mjX0UgJRHvi\ntVyc9lsFGAPcAxRc1Tns/R0iO6IPdVZWFlkaNBNctWvb/OQvvqi+86lk2DB7X5TgU1YoFCIUCpX6\n+fFoyZcHPgAmAs952xYDWVg55zjs5OwpBZ6nlny6WbIEOnSAlStt6gNxa9cuaNDAavJNm7qORkoo\n2V0oM4BXgG/JT/AA44Ge3vWewLgY9yNB0KSJzVH+xhuuIxGw8Qtnn60EH3CxtuTbA58C88kvyfQD\nvgTeBuoDK4FrgP8VeK5a8ulo8mRbKnDePJ3oc2n/fmjcGP79b63j6jPJrsnPpOhfA51jfG0Jos6d\nLcFMn66Jy1x6912oU0cJPg1oxKskV0YG3H03vPCC60jS14ED8Nhj8JCGr6QDJXlJvhtusJb8qlWu\nI0lPo0ZBtWrQtavrSCQJNEGZuNG3r015MHiw60jSy65dVot/5x076Sq+E21NXkle3Fi+3HrarF6t\n7pTJNGgQfP211eTFl5TkxT+6dYNLL4VbbnEdSXpYuxZatrS1dxs3dh2NlJKSvPhHunSn3L/fBoBt\n3mwlqvr1oVat5B5zOGw1+HbtYMCA5O1X4s7VtAYi0Qtyd8rcXHj/fRtwNH061KgBmZmwb5+VqPbs\nseP/zW+SM63AiBGwaRP065fY/UjKUUte3HrpJWvRjx3rOpL4mTTJuieWKWPdRX/7W6hZ8+DH/PAD\nfPwxjBtnq2Zdd52djG7YMP7xrFwJZ50FU6bAaafF//UlqVSuEX/55Rc44QT45hu79LPdu22BlEmT\n4Mkn4YorSlaSWbsWhg6Ff/4Trr0WHnnEWv3xsGOHzRPfqxf06ROf1xSntPyf+EuVKtCzp81O6Wer\nV9vo0R9/tC+sK68sec29bl14/HFYvNgW72jWzBL9tm2xxbR3r5WCWraEe+6J7bXEt5Tkxb277oJX\nX7VWpx8tXw4dO9p6tm+/Xfp1Uo89Fv7+d/uSWLcOTj4ZnnnGfiFEa+dO+yVRoQL861/BPrEth6Uk\nL+6deKJNQfzaa64jid6SJXbSuF8/eOCB+CTTE06wE6XTpsHMmZbsX3nFWuYlsXix/ao45hgb9FS+\nfOwxiW+pJi+pYfp0W0h64UI7YekHGzbkd0ns3Ttx+/niCyvfLFxo++nRA1q0OPTf6fvvYcgQm7bg\nscfgD39QCz6AdOJV/Ckchlat4IknoEsX19EU75dfrERzxRWWgJNh0SIrvXz4IWzdCq1bW6+dXbvs\nvi1b7PzG/ffbSlwSSEry4l8jR8Kbb1rXwlSWmwuXXWYDmoYNc9NaXrfOBpFt3QoVK8JJJ8Gpp0LZ\nssmPRZJKSV78a/duW45u6lTrYZKqHnsMJk60mrnq3ZJk6kIp/lWxotXlU3mu+cmTrbvnW28pwYsv\nqCUvqWXTJjjlFFi2zHqHpJKNG63P+ejRwZuGQXxDLXnxt8xMuOoq+Mc/XEdysHDYerbcfLMSvPiK\nWvKSepYuhfbtYcUKqFzZdTRm6FDruz5rlso04pROvEowXHWVDZBKheH4S5bYl87MmdCkietoJM0p\nyUswfPWVJfrvvnPbct63zwY8/d//we23u4tDxKOavATDWWfZcH7XUx08+qidJ7jtNrdxiJRSIlvy\nXYDngLLAcKDgis1qycvhzZwJN9xg5ZIKFZK//1mzbDbJOXM0glRSRqq05MsC/8ASfTPg90DTBO1L\ngqp9e1uLdMSI5O97+3b7gnn5ZSV48bVEteTbAQOxJA/wkHc5KOIxaslL8WbPtjnRly2DI45I3n57\n97YJwIYPT94+RUogVVrydYA1EbfXettEotO2rS1Z989/Jm+f774LM2bAc88lb58iCZKohbxL1ETP\nzs7+9XpWVhZZGmQihXniCbjgAlsHteBaqfG2di3ceSdMmGCrVok4FgqFCIVCpX5+oso1ZwPZ5Jdr\n+gG5HHzyVeUaKbk777TyyZAhidtHbi507mxfKP37J24/IjFIlX7y5YAlwAXAeuBL7OTroojHKMlL\nyf34o81MOW0aNG+emH089RSMHw+hkKbslZSVKkke4BLyu1C+AjxR4H4leYnOCy/Ae+/BlCnxXz1q\n5kzrLvnll7b8nkiKSqUkXxwleYnO/v021cG118Ldd8fvdTdsgDPPtJ40l1wSv9cVSQAleQm2pUtt\nkepZs6wPfaz27rUafOfOMHBg7K8nkmBK8hJ8Q4bA66/b4t+x9J3PzYXrr7c1UseM8c8C4pLWlOQl\n+HJzbYBU5cqQk1O6NVbDYbjvPpsIbdIkOPLI+McpkgCpMhhKJHHKlIFRo2DxYvjrX6N/fjgMjzxi\nyX38eCV4CbREDYYSSaxKlSxBt2tng5b69i3Z8/btg1tugUWLrDtm9eqJjVPEMbXkxb9q14ZPP4Vh\nw2wq4N278+/7/nvr775zZ/62VavsBOuPP8LUqXDssUkPWSTZlOTF3+rVs0nMtm2zQVKvvw7vvAMt\nWkC3bnD66dZqf/hhaNUKunaF999PnWUFRRJMJ14lOD75BAYPtlZ6bq5ty8iwhN6zJ/zpT1C/vtsY\nRWKk3jUiXbpYwj9wACpWtBOsHTq4jkokLtS7RtJe+NVX4dxz4fjjYcAAJXhJa+pdI4GRm5vL0KFD\n6dOnD3PnzqVFixauQxJxTkleAmHRokXceuutLFmyBIBj1XNGBFC5Rnxu7969DBgwgI4dO9KyZUsq\neAt+10z04iIiPqEkL762YMECVq1axaxZs/jggw8YNGgQ1apVo1w5/UgVASV58bnWrVuTk5PDSy+9\nRLt27TjjjDOoVauW67BEUoaaO+J7M2fOZPTo0SxYsIBFixaRmZnpOiSRlKF+8uJr69ev56yzzmLY\nsGF07dqVPXv2sHbtWho1auQ6NJGE0GAoSRt79uzhvPPOo2vXrjzyyCOuwxFJCiV5SRu33XYbmzZt\nYsyYMZTRgh+SJqJN8qrJiy8NHz6c6dOnM3v2bCV4kcNQS158Z/bs2Vx66aXMmDGDJk2auA5HJKk0\nd40E2saNG7nqqqsYPny4ErxICcSS5J8CFgHzgLFA1Yj7+gHLgMXARTHsQ+RX+/bt45prrqF37950\n69bNdTgivhBLkp8ENAdOB5ZiiR2gGdDDu+wCDI1xP74UCoVch5BQLo7vvvvuo2rVqgwcODDh+9L7\n519BPrbSiCX5Tga8lRmYDdT1rncHRgP7gJXAd0CbGPbjS0H/j5bs48vJyeHjjz9m1KhRSTnRqvfP\nv4J8bKURr941vbHEDnA88EXEfWuBOnHaj6ShTz75hPvvv59QKES1atVchyPiK8Ul+clA7UK2PwxM\n8K73B/YCbxzmddSNRkrlo48+olevXowdO5bmzZu7DkfEd2LtQtkLuAW4ANjtbXvIuxzkXX4MDMRK\nOpG+AzT2XEQkOsuBk5Kxoy7AQuCYAtubAXOBCkBDLyCX/fHFn3oAG4FWrgMRSVfLgFXAHO9vaMR9\nD2Mt9cXAxckPTXzuRmA9cKrrQEREJL5uwU7WN3UdiIiUztVYmecAB/8UbwDsovBfBn5S1PFB8AaJ\nZWMJOe896xLj692F/TpMSr2xCF2w92cZ8KDDOBJlJTAfe7++dBtKXLwKbAIWRGyrgXUaWYqN5/Fz\nl6zCji+b+H7u4u4UoDEwjUOT/ILCnuAzRR1f3rmK8tixfof/B4kNBPrG6bXux87fNIjT65VGWex9\naYC9T3MJ3i+KFVgSDIoOQEsOzh1PAn/yrj9IficQPyrs+KL63LlIMouxb9igKur4gjpILNaT6hnA\nn4E/AJ2wfxtX2mDvy0rsfXoTe9+CJkgdIWYAPxXY1g3I8a7nAJclNaL4Kuz4wMcTlDXEfn6EgPZu\nQ4m747GfWHmCMkjsbmz+oleI/mdxNeBd7EPYiYP/fVyoA6yJuB2U9yhSGPgE+Bo7/xFEmViJA+8y\niOtBlvhzl6gkPxn7eVHw79LDPGc9UA/7adIXG1x1VILii1Vpjq8wfhgkVtSxdgNewr6YzwA2AM9E\n8bptgG+AdcA53vNd88P7Eatzsc/YJcCdWDkgyMIE732N6nOXqEVDLizFc/Z6f2Af/uXAyd71VFOa\n41uHfYnZyqcXAAAEK0lEQVTlqettS3UlPdbh5I+CPpwM4F5s0NytwHuljCsRCr5H9XD/6yLe8r5M\nf8D+7dtgJYEg2YSN1N8IHAdsdhtO3EUeT7GfO9flmsi60jHYiS+AE7EE/33SI4qvyOMbD/yO/EFi\nJ+P/3g3HRVy/nOJPnNfE/h16YMkllRI8WAnjZOzEawUszvEuA4qzSuT/Oq6M9fAKQmeHgsYDPb3r\nPYFxDmNJhGg/d0l3OVb33IV90070tl8J/Beryf8/4DdOootdUccHwRsk9hrWHW8e9kE6XO2zPbAa\nW4egQuJDK7VLgCXY+9SvmMf6TUOsx9Bc7LMWhOMbjZV692Kfu5uw3kOfEIwulAWPrzfRfe5EEq4M\nlkw24t8vbhERKUQm8B+s5lu3mMeKiIhPZADXYT8zHyNxJ/hFRCTJmmPjHL4B2rkNRURE4uUo7KTq\nD1j/67KHf7iIiPhFQ+zM/0h0pl9EJHAqAae5DkJEREREREREREREpARaYIOGfgByi3lsTeAz4Edg\nGzYNRMH5su/FJqLahk1FWnAagd8Bi4BfsOH7hU3rPMCL5fxC7qvgPX9Nge0NsMVSdnj3X1DMsYiI\npIXG2Jwb3Sg+yR8BNCF/crnu2LwWVbzbF2NTCDTF5uyYBjwR8fwLsUUx8hYsOQ6b6z5SI2xujLUU\nnuT7A9OxuWgifQ487cV4BbbYwTHFHI+ISNo4ieKTfKQy2Bz268lvrb8B/C3iMedx8Fzts7AvlMOZ\niE3UtYJDk3xD4FtsXcnIlnxjYDc2u2Ge6dh0wiLiiOuphqX05mMzXY7EZr7Mm4u/GTY7XeTjMoHq\n2ACl1kAtbKHqNcAQoGLE46/GknXk7JmRhmATju0usL05NjX0joht87ztIuKIkrx/nYaNMs0GxpDf\ngq6C1eLz/OxdHoUl+/LYtM7tsZVlWgKPRDzmMeCeIvZ5OTYvzfuF3Fdwv3n7TtXVvUTSgpK8G9cB\n272/D2N4nb1Yy3o7+Sc5fwGOjnhMVe9yO9byx3vOJmAL8CzQ1dueDYzi4Fp73sInlYEnKfoLoOB+\nwc4J/FzIY0VE0lK0Nfk8y4DO3vXXObgmfwEH1+RXAzdE3L4CW6QFrKfOD97jNwD7sS+CB4DTsS+V\nvPu2ePdvAOpjNfld5J8ABpti+A+lOB4RkcCpiNXTc7HeKUcU8bi2WKmlAnAk8CBWW4/sXbMB611T\nHZsR8vGI5z+KLTt4rHf/DG8b2Io6tby/TOwL4UqsFV824r5aWOlmnXc97xfh59gkZRXJ711TM6p/\nBRGRAGqAJfdc4IB3Gbm+7UfYwtcAHbHl237GWt0fcujJzXuxbpR5/eTLR9xXDngRS8AbgOcoejm+\nwnrX5Mni0C6UJ2BdNndi/eSLeq6IiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIikhf8PeTWP\nvg9gNT4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x4b81e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(x, f(x), 'r-')\n",
    "scatter([xmin],[f(xmin)], 10, color='red')\n",
    "annotate('%f' % xmin, \n",
    "        xy=(xmin, f(xmin)), fontsize=12, xycoords='data',\n",
    "          xytext=(-90, -50), textcoords='offset points',\n",
    "          arrowprops=dict(arrowstyle=\"->\", connectionstyle=\"arc3,rad=.2\"))\n",
    "title('BFGS')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Problem\n",
    "\n",
    "Initialization and **local minimima**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.83746663])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xmin = fmin_bfgs(f, 3, disp=0)\n",
    "xmin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x4bb4510>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEdCAYAAADpZhdxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8TfX+x/HXMU+VzFOZhahUSHQ7KVMDKTQLv+t2K3Sp\nWzRRubrSIHXTIByNGugSCjenFKGbKfNMAyIuokzn98dnnc52nMM5Zw/fvdZ+Px+P/dh7r732Wp9l\nO5+99md9BxAREREREREREREREREREREREREREZEAOQrUiOL2uwGzI7StW4BPo7CuiEiObQT2A3uB\nX4CPgSohr48FfvdeT791Dnn9RmAesA/YBnwN3BnyehXgQ+BnYDewFLg9jHj9lORFoiKf6wDEV9KA\nq4FTgIpYon4h0+tDvdfTb+97r90LDPdeL+/d/go0Bwp667wBbALOBEoBt3n7EBGRGNgAtAx5fiWw\nKuT5GODxLN53Gnb23vEk298LnBNOgJmEnsmfBowDtmO/SB4CkkLW7QksB/YAy4BG3vL+wNqQ5deG\nvKcb2Z/JV/P23w3YDOzEvtQaA0uAXRz7BZl5W0eBO4DV3rovnmTdO4E1XpyPAzWBudgvonfJ+CIt\nif0C2479GpsMVM7mGEQkwWwALvceFwNSsBJNujHAE1m8ry1wiJP/cpwBfAncgJ3Nhys0yY8DJgLF\ngarYl1MP77XOwPfABd7zmiH77wRU8B53wb6synvPu3HyJP8SUAhohZWyJgJlgErYr5Q/ZbOto8Ak\n4FTgDCwptznBuhOBEkB9bz+feTGcin05dfXWLYV92Rbx1n/Pe6+ICBuxs+1dwEEsMTYIeX0scMB7\nfReWmABuBX7KtK053jr7gUu8ZSWBJ4HvgMPAQuDCMOJNT/L5scRXN+S1vwCzvMefAr1zuM2FQHvv\ncTdOnuQrhizbwbHXKD4A7slmW0eBi0OejwceOMG6zUKefwP8PeT508Bz2cR5HnZGLwGlmrzkRhrQ\nATgdKIwlxs+BciGvD/NePz1k+U7s7DX0/9vF3jo7ySib7AYGYF8c5YFFwEfZxDKNjIu7N50k7jJY\nuWJTyLLNZJQpqgDrsnlvVyyxp39xNQBKn2R/oUKvKRzI4nnxE7x3a8jj/SdZ92T7KeE9Lga8gn1h\n/w/7/E7j2NKVBIiSvORVGvYz/wjQImR5VsliLnYmfW0Wr2VnJ/AMVtY4PYvX25Fxcfedk2xrB1Yu\nqhay7EzslwjAFqBWFu+rCrwK3I2VOU7HfmX4OSHeC9QBmmDJ/VLsePx8THICSvKSW0kh9+ln9Ssy\nvZbZbuAxrD59PZaY82GlgtCz06HA2UABb530i4m7woz5CFZ7/gd2RlsV6Au86b0+CrgPON87hlrY\nl0Bx7Mtshxdvd44tT0VCTpNrbhNxUjaPS2Bn9v/DvrgG5mKb4kMFXAcgvjMZS5pp2E/+rmQk+TTv\nlpVhwA/A/dhF0F+B9d7zud46RbFfBxWxRPQ1GfXvvAiNpTfWmmU98Bt2hj7Ge+0DrATzNlbC2YA1\n31yM/ZqYi9W9x2EXhkO3n93xZt7/ydbJvK3M703LxbqZl4WuPxw7zh3Y5/Es4f0bS0CMxmp8S0OW\nDcP+uBcDE7CffukGYGdgK4HWMYpRRETy6BKs3XBokm9FRrnnn94NrAnXIuxCVzWsjbHKQiIiDuQ0\n+c7m+LroDOwnLFhX9fTu7R2wC2GHsJ/za7GLPCIiEmOROsPuAUz1Hlcio9UC3mP1qBMRcSASSf4h\nrGPM2ydYJycXoEREJMLCbV3TDRu/5PKQZT9g3bDTVfGWHaNmzZpp69Zl1/9ERESysY6s+3VkKZwz\n+bZY1+kOWJO0dJOwIWULAdWB2sD846Jct460tLTA3gYOHOg8Bh2fji8Rjy/Ix5aWlgY2tlKO5fRM\n/h2sZ1wZrHfgQKyZZCHsAixYW+K7sJH83vPuD3vLVK4REXEgp0k+q7FBRp9g/SHeTUREHFL79ShJ\nTk52HUJU6fj8LcjHF+RjywuXgxKlefUlERHJoaSkJMhF7taZvIhIgCnJi4gEmJK8iEiAKcmLiASY\nkryISIApyYuIBJiSvIhIgCnJi4gEmJK8iEiAKcmLiASYkryISIApyYuIBJiSvIhIgCnJi4gEmJK8\niEiAKcmLiASYknw0rFsHL73kOgoREc0MFRWbNkHjxrBtGyS5/CcWkaDRzFDxoGpVKF4cVqxwHYmI\nJDgl+Wi59FL4/HPXUYhIglOSjxYleRGJAzlN8qOBbcDSkGWlgBnAamA6UDLktQHAGmAl0Dr8MH0o\nPckH9bqDiPhCTpP8GKBtpmX9sSRfB/iP9xygPnCDd98WeCkX+wmO6tWhQAFYs8Z1JCKSwHKafGcD\nuzItaw+keI9TgGu9xx2Ad4BDwEZgLdAkrCj9KClJJRsRcS6cM+zyWAkH776897gS8H3Iet8DlcPY\nj38pyYuIY5Eqo6R5txO9nnhUlxcRxwqE8d5tQAVgK1AR2O4t/wE4I2S9Kt6y4wwaNOiPx8nJySQn\nJ4cRThyqXRsOH4YNG6BGDdfRiIgPpaamkpqamuf356Y7ZjVgMtDQe/4UsBMYil10Lend1wfexurw\nlYGZQC2OP5sPbo/XUDfdBK1bQ/furiMRkQCIVo/Xd4A5wFnAFqA78E+gFdaEsqX3HGA58J53Pw24\ni0Qt14CVbML4FhYRCYfGrom2FSugXTvYuNF1JCISABq7Jt7UrQsHDtigZSIiMaYkH21JSfCnP6kp\npYg4oSQfC2ovLyKOKMnHgpK8iDiiJB8LZ58Nu3fDD1l2FxARiRol+VjIlw8uuURn8yLR9Pvv8Oab\n6mGeiZJ8rKhkIxJdAwfChAmuo4g74QxrILmRnAwvv+w6CpFgmjsXxo6FJUs0r3ImOpOPlYYNYft2\n+Okn15GIBMv+/XD77fCvf0G5cq6jiTtK8rGSPz+0aAFffOE6EpFg6dMHmjSB6693HUlcUpKPJdXl\nRSLrtdesVKNSaLaU5GNJSV4kchYsgIcesoutJUq4jiZuKcnH0nnnwfffw88/u45ExN9+/hk6dYJX\nXoGzznIdTVxTko+lAgWgeXPV5UXCcfiwzdNw883QsaPraOKeknysqWQjEp5HHrFmkoMHu47EF5Tk\nY01JXiTvJk6Ed96xW/78rqPxBU0aEmuHDkHp0jaJSKlSrqMR8Y+VK23Y7ilToHFj19E4o0lD4l3B\ngtCsGcye7ToSEf/Ytw+uuw6GDEnoBJ8XSvIuaN5XkZxLS4MePazRwp//7Doa39HYNS5cein07u06\nChF/eOEFWL8evvzSdSS+pJq8CwcPWl1+yxYoWdJ1NCLxa8ECuOoq+PprqFHDdTRxQTV5PyhUyMba\n0JmJSPZ274YbbrAhC5Tg80xJ3hU1pRTJXnod/uqr7YKr5FkkkvwAYBmwFHgbKAyUAmYAq4HpgGoS\nmSnJi2TvX/+CzZth2DDXkfheuDX5asBnQD3gd2A8MBU4G9gBPAU8AJwO9M/03sStyQP89huUKWPj\ny59yiutoROLH6tVw8cU2umTt2q6jiTuxrsnvAQ4BxbCWOsWAH4H2QIq3TgpwbZj7CZ4iReCCC+Cr\nr1xHIhI/jhyB7t3h0UeV4CMk3CT/C/AMsBlL7ruxMk15YJu3zjbvuWSWnKySjUio4cNtIL9evVxH\nEhjhJvmawN+wsk0loARwa6Z10rybZKa6vEiGTZvgySdh9GjIpzYhkRJuZ6gLgTnATu/5BKAZsBWo\n4N1XBLZn9eZBgwb98Tg5OZnk5OQww/GZiy6yiYd//RWKF3cdjYhbffvCPfdAzZquI4krqamppIbR\nQz7cC6/nAm8BjYHfgLHAfKAqlviHYhdcS6ILr1lr0QIGDoRWrVxHIuLOtGnWC/y77+x6lWQr1hde\nFwPjgG+AJd6yV4F/Aq2wJpQtveeSFZVsJNH9/rtNxv3CC0rwUaBhDVybPh2eeEKjUkriGjECPvkE\npk51HYkv5PZMXknetX37oEIFm7OyaFHX0YjE1p491lRyxgw45xzX0fiCxq7xmxIloEEDG4BJJNE8\n8wy0basEH0UaajgepNflL7vMdSQisbNtG7z4Inz7retIAk1n8vFAnaIkET3zDNx8M1St6jqSQFNN\nPh7s2QOVKsGOHWpdIIlh506rxS9eDGec4ToaX1FN3o9OPdVqkhpfXhLFiBE2hLASfNQpyceL1q3h\n009dRyESfXv2wEsvQf/M/SMlGpTk40WbNkrykhheecV6eNeq5TqShKCafLw4fBjKlbNu3ZUquY5G\nJDoOH7axaSZMsKG2JddUk/erAgXg8sutB6xIUH30EZx5phJ8DCnJx5M2bZTkJdief95GmpSYUbkm\nnmzZAuefb51ENJ62BM2330LHjrBunf1ylTxRucbPzjgDypZVD0AJphEj4O67leBjTEk+3qiVjQTR\nrl1Wj+/Rw3UkCUdJPt6ovbwE0ZtvQrt2UKaM60gSjmry8Wb/fihfHn74wXrCivhdWhqce65N0t2y\npetofE81eb8rVszmfv3sM9eRiETG/Pl28pJoczjHCSX5eHTVVfDxx66jEImM116Dnj3VYswRlWvi\n0bp10Lw5/Pij/jDE3/bts1ZjK1bYDGgSNpVrgqBmTbtAtWCB60hEwjNxop2wKME7oyQfr9q3h0mT\nXEchEp433oDbbnMdRUJTko9X11yjJC/+9sMP8M03dsIizijJx6smTWD7dli/3nUkInnz9ts2MUjR\noq4jSWiRSPIlgQ+AFcByoClQCpgBrAame+tIbuTPD1dfDZMnu45EJPfS0mDcOJVq4kAkkvzzwFSg\nHnAOsBLojyX5OsB/vOeSWx062LjbIn6zdKnNAHXJJa4jSXjhNqE8DVgI1Mi0fCVwKbANqACkAnUz\nraMmlCfz229QsSIsX273In7x0EM2QcjQoa4jCZxYN6GsDvwMjAG+BV4DigPlsQSPd18+zP0kpiJF\n7ALshx+6jkQk59LS4L33oEsX15EIEO6YnwWA84FewAJgOMeXZtK823EGDRr0x+Pk5GSS1e35eJ07\nw9NPQ69eriMRyZmFCy3Rn3++60gCITU1ldTU1Dy/P9xyTQVgLnZGD9ACGICVby4DtgIVgVmoXJM3\nv/9upRrN/Sp+0b+/9dQeMsR1JIEU63LNVmALdoEV4ApgGTAZuN1bdjvwUZj7SVyFC6tkI/6hUk3c\niUTrmt7AW8BirHXNP4B/Aq2wJpQtveeSV507w/jxrqMQOblvvoGCBW1oYYkLGqDMDw4ehCpVYO5c\nG9dGJF79/e/WYOCJJ1xHElgaoCyIChWCm26yziUi8UqlmrikJO8X3bpBSgocPeo6EpGszZsHxYtD\ngwauI5EQSvJ+0agRlCwJYTSlEomq99+360dJLqvAkpmSvJ906wZjx7qOQuR4aWnw0Uc2IJnEFV14\n9ZPt26FOHdi0CU47zXU0IhmWLrUhhdev15l8lOnCa5CVKwdt2uhsXuLPRx/BtdcqwcchJXm/6dMH\nXnhBF2AlvqQneYk7SvJ+c/HFdgF22jTXkYiYzZvt1ry560gkC0ryfpOUZGfzI0a4jkTE/PvfNsFN\ngXDHO5RoUJL3oxtugMWLYcUK15GIqFQT59S6xq8GD4ZVq+CNN1xHIonsl1+genX46ScoVsx1NAlB\nrWsSRZ8+8OmnsHKl60gkkU2ZAi1bKsHHMSV5vzr1VOjbFx5/3HUkkshUqol7Ktf42d69UKsWzJoF\n9eu7jkYSzYEDUKGCdYAqXdp1NAlD5ZpEcsopcN99NhOPSKzNnGlT/CnBxzUleb/r08cuwE6e7DoS\nSTQq1fiCyjVBMHMm9OwJy5bpApjExpEjNvfw/PlQrZrraBKKyjWJ6IoroGlTTZwssTNnDlSurATv\nA0ryQfHsszBqFHz5petIJBGoVOMbSvJBUakSjB4NN98MO3a4jkaCLH3seCV5X1CSD5Irr7S5YG+7\nzWqmItHw3Xc2Cuo557iORHJAST5oBg+Gw4ftQqyGI5Zo0NjxvqIkHzQFC9of4apV8Le/2U9rkUhS\nqcZXIpXk8wMLgfTG2qWAGcBqYDpQMkL7kZwoXtzGFJkzB7p3h99+cx2RBIXGjvedSCX5e4DlQPpp\nY38sydcB/uM9l1gqWRI+/xz274dLL4UtW1xHJEHw73/DVVdp7HgfiUSSrwJcCYwio4F+eyDFe5wC\n6LedC8WLw/jxcN110KgRPPec1etF8mriROjY0XUUkguRSPLPAX8HQq/ylQe2eY+3ec/FhaQkeOAB\nK91MnQoNG8KYMXDwoOvIxG927oT//hdatXIdieRCuL+5rga2Y/X45GzWSSOjjHOMQYMG/fE4OTmZ\n5OTsNiFhq1MHpk+H//wHhg6Fhx+GW2+15pYNGriOTvxgyhS4/HINnRFjqamppKam5vn94baBGgLc\nBhwGigCnAhOAxljS3wpUBGYBdTO9V2PXuLRsmc0q9dZbUKYM3HKLtbGvXNl1ZBKvrrvOWtV07eo6\nkoSW27FrItnQ9VLgPuAa4ClgJzAUu+hakuMvvirJx4MjR+wC7VtvWb21USPo1g26dIHChV1HJ/Fi\n/34bkGzDBihVynU0Cc31AGXpWfufQCusCWVL77nEo/z5bfq211+HH3+Eu+6CN9+0gacGD4Z9+1xH\nKPFgxgy48EIleB/SUMOStWXLbFTL1FR44gk7u8+nvnMJq3t3myCkd2/XkSQ8l+Wa3FKS94P586FX\nL5v9Z+xYKK+GUgnn8GGb5u/bb+HMM11Hk/Bcl2skaJo0ga++ggsusHr97NmuI5JY+/JLK98pwfuS\nkrycXMGCVp8fOxauv15TDSYajVXjayrXSO4sWADt28PTT1uzSwm2tDQ7i586Fc4+23U0Qu7LNRqA\nQnKncWPrUHXZZVC2LLRu7ToiiaZFi6BQIahf33Ukkkcq10ju1a8PH35oPWYXLXIdjUSTxo73PSV5\nyZsWLeBf/7LSzc6drqORaPngA+vpKr6lJC9517mz3Xr00OQkQbRsGezdC02buo5EwqAkL+F58knr\nKfvii64jkUgbP96Gt1AnOF9T6xoJ37p1cNFF8MUXUK+e62gkEtLS7LMcN876SkjcUGcoib2aNWHg\nQLjjDk0eHhRLlsDvv1trKvE1JXmJjDvvtIlIRo92HYlEwnvvWalGrWp8T+UaiZwlS+CKK2DpUo1x\n42dpaTbJzLvv2nAWEldUrhF3zjnHZpp69FHXkUg4Fi60stv557uORCJAZ/ISWbt2wVlnwWefaVpB\nv+rf31rUDBniOhLJgoYaFveGD7f5ZKdOdR2J5FZaml1InzABzjvPdTSSBZVrxL277oLVq202IfGX\nuXNt2sdzz3UdiUSIkrxEXqFC8I9/wMMPqyes36SkwO23q1VNgKhcI9Fx9Cg0bGhDErdr5zoayYnf\nfoPKlWHxYqhSxXU0kg2VayQ+5MtnHaQGDtTZvF9MmmQtapTgA0VJXqKnUyc4cACmTXMdieRESgp0\n7eo6CokwlWskut5/H555xi7oqc4bv7ZutbFqvv8eihd3HY2cgMo1El+uu87Gm9cE4PFt7Fj7rJTg\nAyfcJH8GMAtYBnwH9PGWlwJmAKuB6UDJMPcjfpU/P9x3Hwwb5joSyc7Ro/DKKzb+kAROuEn+ENAX\nOBu4CLgbqAf0x5J8HeA/3nNJVF272gTgy5e7jkSy8umnULo0XHih60gkCsJN8luB9Ek+9wErgMpA\neyDFW54CXBvmfsTPihaFXr2sOaXEn5EjdRYfYJG8ElYN+BxoAGwGTg/Zxy8hz9Ppwmsi2bkTateG\n776DSpVcRyPpNm+GRo3sXvV4X8jthdcCEdpvCeBD4B5gb6bX0rzbcQYNGvTH4+TkZJKTkyMUjsSd\n0qXh1lvh+edh6FDX0Ui6l1+GW25Rgo9jqamppKam5vn9kTiTLwh8DEwDhnvLVgLJWDmnInZxtm6m\n9+lMPtFs3Gjjk2/YAKee6joa2bsXqleH+fOhRg3X0UgOxboJZRLwOrCcjAQPMAm43Xt8O/BRmPuR\nIKhWDS6/3OYNFfdefdUmeVGCD7Rwz+RbAF8AS8goyQwA5gPvAWcCG4EuwO5M79WZfCKaPRv+/GdY\nscKGPhA3Dh605D55stXkxTdiXZP/kux/DVwR5rYliFq0gGLFbLz5tm1dR5O43noL6tdXgk8AOpWS\n2EpKgj59YMQI15EkrkOHbNanAQNcRyIxoCQvsXfTTfDNN7BmjetIEtOrr1qp5rLLXEciMaABysSN\nBx+EX3+1JpUSO3v3Qp06NjKopvfzJc3xKv6wZYtNMbdxo5pTxtKjj8KmTTassPiSkrz4R5cucMkl\n0Lu360gSw7p10LQpfPstnHmm62gkj5TkxT++/BL+7/+C35xyzx47xu3boWBBOOMM6zMQy16mR49a\nDf7aa6Fv39jtVyLO1bAGIrnXvHlwm1MePGidvsaMsTlT69aF8uWtZcvmzTY5xwUXwFVX2XAP0R7P\n58UX4cgRa9kkCUVn8uLWmDE2e9TUqa4jiYy0NHj3XXj4YahZ05Jqq1ZQuPCx6x04ALNmwUcf2fG3\nawf33x+di6ErVlhZbO5cGyROfE3lGvGX336z+vCXX1qrDz/bswd69oSVK2H48Jw3Udy9G15/3YZi\nvuwyePxxqFUrMjHt3m11+P79oXv3yGxTnNL0f+IvRYpYYnzxRdeRhGflSiu/lCoF8+blrg16yZJw\n773Wb6B+fbjoIhvf/aefwotp/3645horCSnBJywleXHvzjvhzTfhf/9zHUneLFkCLVtaD9KRI+2L\nKy9KlLAyz6pVdlG2QQPrT7A787BPObBrl13nqFlTk7UkOCV5ca9KFWjTBkaPdh1J7n37LbRubeWZ\nHj0is83SpS0xL1pkLXJq14annrJyUKhff4XUVOtrEGrePGjc2G6jRwe75ZKclGryEh/mzYMbb4S1\na23ybz/YsMFaCL34Ilx3XfT2s2IFPPaYtULq0gVuuAHq1YNmzWzGrSNH7GJvxYo2JtCMGXbfuXP0\nYhJndOFV/KtZM3jgAWvLHe9++QUuvtg6ct19d2z2+dNPdoF2yhRYuNCaYx49aq8VKGBt73v0gLvu\ngtNOi01MEnNK8uJf48dbTTuMqc5i4vBhaxZ5wQXu6t1TpkCnTtY6KV8+aN2aYS1bUqJECZKSkjh8\n+DCHDh3iyJEj3HrrrVSoUMFNnBJxSvLiX4cO2eiIkybF9zjn999vF1unTHFbWnriCftSPOssGD+e\nsmefza5du2jatCkNGzakaNGiFCxYkF69enGmhjEIDCV58behQ60GPXas60iyNmEC9OtnQyWXKeM6\nmmOsX7+exo0bU69ePTZs2ED//v3p2bMnRfLa2kfiktrJi7/17Gln8j/+6DqS423cCH/9q/VQjbME\nD1CjRg3GjRvHxo0bGT16NDNmzKBWrVq88cYbHE2v3UvC0Zm8xJ8+faBoUTurjxdHjlgHp2uugb//\n3XU0JzRw4EA+//xzZs6cyYIFC+jTpw8FChRgxIgRNG7c2HV4EiadyYv/3XsvjBqVt05A0fL003aB\ns18/15Gc1KOPPkrVqlVZu3YtzZo1Y968edxxxx106NCB7t27s3XrVtchSgwpyUv8qVoVrrwSXn7Z\ndSRm4UJ45hmbaMMHbfjz589PSkoKdevWBSBfvnx069aNlStXUq5cORo0aMCwYcP4/fffHUcqsaBy\njcSnpUutJ+mGDXkfJiASDhywppIPPmhDAgfAmjVr6NevHytXruTZZ5/l6quvTi8BiA+odY0ER/v2\nluh79XIXwz33wLZt8M47ELBE+Mknn9C3b1+qVq3KiBEjqOP3UUATRDzV5NsCK4E1wANR3I8E1aBB\n8OSTdjbtwvTp1mRy5MjAJXiAtm3bsmTJElq3bk3z5s0ZOXIkOvEKnmj9z80PrAKuAH4AFgA3AStC\n1tGZvJxcx47wpz/Ffsq6nTttovGUFLj88tju24FVq1Zx6623UrZsWV5//XUqVqzoOiTJRrycyTcB\n1gIbgUPAu0CHKO1Lguyxx2wExl9/jd0+09LgjjtsgK8ESPAAZ511FnPmzOHCCy+kUaNGTJgwwXVI\nEiHRSvKVgS0hz7/3lonkzjnn2NR1I0bEbp9jx8Lq1VYqSiAFCxbk8ccfZ+LEidx///10796dPZmH\nNxbfidZE3jmqwwwaNOiPx8nJySQnJ0cpHPG1IUNstqRu3Ww43Whau9bGppk1y22rHoeaNWvGokWL\n6NevH+eeey7jxo3jkksucR1WwkpNTSU1jEH7olWTvwgYhF18BRgAHAVCuzCqJi85d//9sGNHdCcW\nOXQIWrSwppK9e0dvPz4yadIk7rjjDrp168Zjjz1GoUKFXIeU8OKlCWUB7MLr5cCPwHx04VXCsWeP\njbb48cfWbj0aHnnEBh6bOjWQrWnyavv27XTv3p19+/bx4YcfUiYOx+1JJPFy4fUw0Av4FFgOjOfY\nBC+SO6eeCoMH24QYhw9HfvuffGK/EsaMUYLPpFy5ckyePJmLL76Ypk2bsmzZMtchSS6oM5T4x9Gj\n1jmqZUvrgRopGzZYzf+DD+wir2TrjTfe4N5772XMmDFcddVVrsNJSPFSrskJJXnJvc2brVwzc6a1\nYw/X/v1Wh+/aFf72t/C3lwDmzJlDp06duO++++jbt6+GRIgxJXkJvrFj4dlnYe5cKF4879s5dMg6\nW5UpozJNLm3atIn27dtz4YUXMnLkSF2QjaF4qcmLRM/tt9vZ/M032zjveZHe4enoUXjtNSX4XKpa\ntSpfffUVO3fupFWrVuzYscN1SJINJXnxn6QkeOUV2LsX7rsv9+8/cgTuvBOWLbNZngoWjHyMCaBE\niRJMmDDhjwuyK1aobUU8UrlG/GvXLqunX3uttbzJydn4/v32C+DXX+HDD63VjoRt7NixDBgwgOnT\np9OwYUPX4QSayjWSOE4/HVJTYcYMuOGGk88k9d130Lw5nHIKTJmiBB9B3bp147nnnqN169YsWbLE\ndTgSQkle/K1sWfjiCyhXDurVg+efh19+yXg9LQ0WL4a//MXmaL37bhg3DnShMOJuvPFGhg8fTps2\nbVi8eLG/CetIAAAIRklEQVTrcMSjco0Ex3//C8OG2Vl6xYpQogRs2WL3XbvaBOGlS7uOMvDef/99\nevfuzSeffMJ5553nOpzAURNKkYMHYd06q7tXrAiVNQBqrH3wwQf06tWLadOm0ahRI9fhBEpuk3y0\nRqEUcadQISvdiDOdOnUiKSmJtm3bMm3aNM4//3zXISUsJXkRiYrrr7+efPny0a5dO6ZOncoF0RpY\nTk5ISV5EoqZjx47ky5ePK6+8klmzZlG/fn3XISUcJXkRiaoOHTqwe/durrnmGr7++mvKli3rOqSE\noguvIhITDz74ILNnz2bmzJkULlzYdTi+pdY1IhKXjh49SpcuXShWrBgpKSkavTKP1ONVROJSvnz5\nGDduHMuXL2fIkCGuw0kYqsmLSMwUK1aMSZMmcdFFF1GnTh06d+7sOqTAU7lGRGJu0aJFtGrViilT\nptCkSRPX4fiKyjUiEvfOO+88Ro0aRceOHdm8ebPrcAJN5RoRcaJDhw6sWbOG9u3b8/XXX1OkSBHX\nIQWSyjUi4kxaWhpdunShcuXKDB8+3HU4vqAmlCLiK7/88gvnnnsuo0aNok2bNq7DiXuxrMkPA1YA\ni4EJwGkhrw0A1gArgdZh7ENEAq5UqVKkpKTQo0cPfv75Z9fhBE44SX46cDZwLrAaS+wA9YEbvPu2\nwEth7seXUlNTXYcQVTo+f4u342vZsiU333wzPXv2JNxf+PF2bK6Fk3xnAEe9x/OAKt7jDsA7wCFg\nI7AWSLg2UkH/j6bj87d4PL7BgwezadMmRo0aFdZ24vHYXIrUGXYPYKr3uBLwfchr3wOatUFETqhw\n4cK89dZbDBgwgNWrV7sOJzBOluRnAEuzuF0Tss5DwEHg7RNsR1dYReSk6tevz2OPPcYtt9zCoUOH\nXIcTCOG2rukG9AQuB37zlvX37v/p3X8CDMRKOqHWAjXD3L+ISKJZB9SKxY7aAsuAMpmW1wcWAYWA\n6l5AGm5ORHKjPPATCXg9L56sATYBC73bSyGvPYidqa8E1PBVRPLidmA+Cdg6T0QkEeQD5mINO8RH\nOmNlniNA6BTu1YADZP3LwE+yOz4IXiexQVjrqfTPrK3TaCKjLfb5rAEecBxLNGwElmCf13y3oeTI\nBVjZpmQ2r48GtmENQtKVwhqNrMb682T3Xj/I6vgGEed/d3WBOsAsjk/yS7N6g89kd3zp1yoKYse6\nFv//DB0I9HMdRATlxz6XatjntAio5zKgKNiAJUE/eRXIbmCbS4BGHJs7ngLu9x4/QEYjED/K6vhy\n9XfnIsmsxL5hgyq74wtqJ7EgXVRvgn0uG7HP6V3scwsav31mDwE3Aw2yeG02sCvTsvZAivc4Bbg2\neqFFXVbHBz4eT7469vMjFWjhNpSIC2onsd7Y+EWv4++fxWCfx5aQ50H5jEKlATOBb7Dmz37wM/A4\nMIKcJbfyWIkD7758lOJyKcd/d9FK8jnpRJXZj8AZ2E+TfljnqlOiFF+48nJ8WfFDJ7HsjrU9MBL7\nYj4Pq5s+4yjGSPHD5xGu5tjfWDvgbqwc4AcvY821O+XyfWkE73PN1d9dtCYNaZWH9xz0bgDfYu3r\na3uP401eju8H7EssXRVvWbzL6bGOAiZHM5AYyPwZncGxv76C4Cfv/mdgIlaimu0unBw7jJ29jsOG\nUPn1BOtuAyoAW4GKwPaoRxdbocdz0r871+Wa0J9eZbALXwA1sAS/PuYRRVbo8U0CbiSjk1ht/NG6\n4UQqhjzuiP8vnH+DfS7VsM/pBuxzC4piZPw6Lo618PLTZ/Y58BUZveqzMwlrY493/1E0g3Ig7v/u\nOmJ1zwPYN+00b/n1wHdYTf6/wFVOogtfdscHweskNg5rjrcY+0MKQu2zHbAK+5wGnGRdv6mOtRha\nhP2t+fH4zgB2kvF/7R2s1HsQ+7vrjrUemkkwmlBmPr4eBPPvTkTkD88Dz7kOQkREoqMCdjYftJZP\nUZH/5KuIiMSVfUBZoCUZ81iIiEiAlEVn8yIigdYK/w3RICIiIiIiIiIiIiIiIiKSZ29i4+bswYYr\neegE6z6C9e7cjc3LUD+H27kF2Bty+xU4ig3IFqoQsIJjRxxNd4+33X3AcjImzH4w07b3YxMD6cKv\niAhwNlDEe3wWNuRHVjMatccGiquGja01BBvmJLfbARu3Zk0Wyx/CxsHZnGn5n7GhAup6z6sDp2ez\n7YHY0AkiIpLJWdjonpmnxwQbT2d8yPOzsXGgcrsdsF8Bj2RaVh07Q2/LsWfy+bznl50ocE8SdrZ/\nWw7WFRFJGC9hJZTDwF+zWacJdoZdG5uC8SlgQh62U9V7vWqm5R9js30lc2ySPxMr7fTx9r8em0s1\nq0lK/oSVbIpls28RkYSVhCXYHWQ//eUTWMI9hM0rUS0P23kE+CzTso7AFO9xMscm+Yu9fU4GTsW+\nHFZhJZzMXscm2BYRkWyMJOuRJHsBc7DpMvNhdfX1QNFcbmcNGWPKg42fv4aMC6nJHJvkG2FJPnSm\nrH4c/yuiGPA/4NJs4ok7ricNEZHEVJCsZ3dqS8YY6kexibhPB+rlYjvNsYk1PghZVhs7O5+Ntc75\n0FvnJ6xUs4qMmelCZZ46sCM2Zs7n2cQjIpJwymKzohXHRr5tg50NN85i3SFYIi6HnYTehtW/T83F\ndl4FxmZalt/bZvqtI9aKJ30/YF8ok4ES2PScK7BJSEJNx2r1IiLiKQOkAruwtu/zsaaSYGfRe7Gk\nClYOGYU1jfwfNiVj6xxsJ10R7/WTtZJJ5vgmlKdgvyL2eK89nOn1ytjZfo2TbFtERERERERERERE\nRERERERERERERERERERERERERP7w/6066jSdhNWCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x4c254d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(x, f(x), 'r-')\n",
    "scatter([xmin],[f(xmin)], 10, color='red')\n",
    "annotate('%f' % xmin, \n",
    "        xy=(xmin, f(xmin)), fontsize=12, xycoords='data',\n",
    "          xytext=(40, -75), textcoords='offset points',\n",
    "          arrowprops=dict(arrowstyle=\"->\", connectionstyle=\"arc3,rad=.2\"))\n",
    "title('BFGS - local minima')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Brute force\n",
    "\n",
    "- Compute $f(x)$ for all $x$ in a *grid*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.30641113])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.optimize import brute\n",
    "grid = (-10, 10, 0.1)\n",
    "xmin = brute(f, (grid,))\n",
    "xmin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x5162c50>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEeCAYAAABv8mXfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYU2X6//H30KVIEQWkSBEQRFlxBVGUUVGQVbCArroq\nsNgXCy6riF8ZdS3YVmXFAgrIqjQVQUVhgSBFQX+C6EgT6VWBBaTD5PfHfcYJw8CUlCfn5PO6rlxJ\nTk5y7hDmzpP7PAVEREREREREREREREREREREREREREREjuhcYCmwA+jkOBYREV9bAezCEuoW4GOg\nVpSvmQXUj+L5U4BeUcYgklDFXAcgcgRh4DKgAlAD2AgMPMr+Bf2/nBZFTHWAH4v43OJRHFdEJHCW\nAxdG3O8ILI64Pwx4FfgU+A24CAgBf43Ypxsww7v9BdaS/w37ddDV234ZMB/YCswCTjtCPMuAg9iv\ni+1ASeBEYDywGSvj9IzYPwMYC4wAtgE9gCrAUGAt9uvkw4j9CxqHiEggLMcSN0BZYDiW2LMNA/4H\ntPbulwamYck0WzdykjwcXq45A/uFcBbWwr/JO26po8QU+cXzBfBvb//mwCbgAu+xDGAfObX7MsAn\nwHtARaAEcF4R4xAR8b0VWIt7K5Ys1wDNIh4fyqFJHwqf5F8FHsv1GouA848QU2SSrw0cAMpFPP6k\nFxdYkg9FPFYD+yVQMY/XLWwcIgWmmrwkqzDQGaiMtdJ7AdOBEyL2WR3lMU4C7se+SLIvtbCEnJ8T\nsZLLzohtq4CaEffXRNyu7e2/LcZxiByVkrz4QRirXx8E2hxlv50c2rKuns/rrgKewL5Isi/lgVEF\niGkdVmMvH7GtDocm9nDE7dXe/nm15KOJQ+SolOQlmaVFXGe36hfmeizSfOAq4BjgZA49CQtW924Q\ncX8wcDvQ0nu9csCfODRxH8lqYDbwFPZL43SsVPSfI+y/HpgIDAIqYSdus8sx0cQhIuJLy8npJ78d\nWABcF/H4UA6vYx8HfO7tPwPoj50czXYb1gLfCnTxtrUH5nrb1mGt5yMl19wnXmsCE7DeNT8Bt0Y8\n1h94O9fzK2PnETZgpZuxEY8VJg6RmHsLawV9H7HtWaxV9R3wAYf+DO2LdSlbBFySoBhFRKSIzsO6\neUUm+YvJKfc87V0AmmI/m0sCdbEWjspCIiIOFDT5zsB+RkaajHVJA5hDzpDzzlhf4P1YN7ifsFqj\niIgkWKxa2D2wkYdgXcsiexis4dBuZSIikiCxSPL9sMEq7x5ln/BRHhMRkTgpEeXzu2FzilwUsW0t\nNvAjWy1v2yEaNGgQXrZsWZSHFxFJOcuwLsIFEk1LvgPQB6vB74nYPh74MzbvRj2gIdY17NAoly0j\nHA4H9tK/f3/nMej96f2l4vsL8nsLh8Nw6FiPfBW0Jf8e0Baoig0C6Y91kyyFnYAF+BK4E5uKdbR3\nfcDbpnKNiIgDBU3y1+Wx7a2j7P+kdxEREYfUfz1O0tPTXYcQV3p//hbk9xfk91YU0aySE62wV18S\nEZECSktLg0LkbrXkRUQCTEleRCTAlORFRAJMSV5EJMCU5EVEAkxJXkQkwJTkRUQCTEleRCTAlORF\nRAJMSV5EJMCU5EVEAkxJXkQkwJTkRUQCTEleRCTAlORFRAJMSV5EJMCU5ONh1Sr4z39cRyEioiQf\nF/v3wz/+AVr5SkQcU5KPh/r1oVgx+Okn15GISIpTko+HtDRo2xamT3cdiYikOCX5eFGSF5EkUNAk\n/xawEfg+YlsVYDKwBJgEVIp4rC+wFFgEXBJ9mD6UneRVlxcRhwqa5IcCHXJtexBL8o2AKd59gKbA\ntd51B2BQIY4THI0awb59sGKF60hEJIUVNPnOALbm2tYJGO7dHg5c4d3uDLwH7AdWAD8BLaOK0o9U\nlxeRJBBNC7saVsLBu67m3T4RWBOx3xqgZhTH8a/0dCV5EXEqVmWUsHc52uOpp21bCIVcRyEiKaxE\nFM/dCFQHNgA1gE3e9rVA7Yj9annbDpORkfH77fT0dNLT06MIJwk1aQI7d9oI2Dp1XEcjIj4UCoUI\nRdFYTCvEvnWBCcBp3v1ngM3AAOykayXvuinwLlaHrwn8FziZw1vz4XAq9Dzp0gU6d4Ybb3QdiYgE\nQFpaGhQidxe0XPMeMBtoDKwGugNPAxdjXSgv9O4D/AiM9q4nAneSquUa0MlXEXGqMC35WEuNlvyC\nBXD11bB0qetIRCQA4tWSl6Jq1gy2bIF161xHIiIpSEk+3ooVg/POU8lGRJxQkk8E1eVFxBEl+URQ\nkhcRR3TiNREOHoSqVWHRIqhWLf/9RUSOQCdek1Hx4tCmjVrzIvF04AB89pnrKJKOknyiqGQjEl8D\nBsALL2h671yimdZACqNtWxg2zHUUIsH03Xfw4ovw7bc2A6z8Ti35RDnjDFi9Gn791XUkIsGybx/c\ndBM89xzUrp3//ilGST5RSpSAc86BL75wHYlIsPTtC3XrWqKXwyjJJ5Lq8iKxNWoUfPghDB2qMs0R\nKMknkpK8SOxkZsLf/gbvvw9VqriOJmmpn3wi7d9v/xlXrYLKlV1HI+Jf27ZBy5bw0ENw882uo0ko\n9ZNPZiVLwtlnw4wZriMR8a+sLOjWDS66KOUSfFEoySea1n0Vic6AAbBhg3WZlHwpySea1n0VKbrJ\nk2HgQBgzBkqVch2NL6gmn2h798Jxx8HatVCxoutoRPxj5Upo1QpGjrRfxClKNflkV7q0nTCaOdN1\nJCL+sWePrbDWp09KJ/iiUJJ3QV0pRQqnVy+oXx9693Ydie8oybugJC9ScMOGwaxZ8OabGvBUBKrJ\nu7B7Nxx/PKxfDxUquI5GJHllZlp5JhSCU091HU1SUE3eD445Blq0gNmzXUcikrx27oSuXeHZZ5Xg\no6Ak74pKNiJHd9dd1kmhWzfXkfhaLJJ8XyAT+B54FygNVAEmA0uASUClGBwnWJTkRY5s+HCYOxde\necV1JL4XbU2+LjAVaALsBUYBnwKnAr8CzwAPAJWBB3M9N3Vr8mA/RatVg40boVw519GIJI9Vq+DM\nM2HKFDj9dNfRJJ1E1+S3A/uBstgqU2WBdUAnYLi3z3DgiiiPEzzlykHz5vDll64jEUke4TD07An3\n3qsEHyPRJvktwPPAKiy5/w8r01QDNnr7bPTuS24q2YgcavBg2LIFHnjAdSSBEW2SbwDci5VtTgTK\nA3/JtU/Yu0huSvIiOTZsgH79rF98CS0/HSvR/kv+EZgNbPbufwC0BjYA1b3rGsCmvJ6ckZHx++30\n9HTSU2248jnn2MLDu3dbt0qRVNanD/ToAc2auY4kqYRCIUJRTGoY7YnX5sA7wFnAHmAYMBc4CUv8\nA7ATrpXQide8tWplU6em2hecSKQvvoAbboCFC6F8edfRJLVEn3j9Dngb+AZY4G17A3gauBjrQnmh\nd1/yopKNpLoDB6xP/AsvKMHHgaY1cO2TT+D552HqVNeRiLgxZAiMGGFTF2humnwVtiWvJO/atm1Q\nsyZs3mzTEIukkl27oFEjW4y7VSvX0fiC5q7xm4oVoXFj+Ppr15GIJN7AgbbusRJ83KifUjLIXve1\nTRvXkYgkzpYt8NxzWkAnztSSTwZa91VS0UsvQadO9ktW4kY1+WSwdSvUqWN1eS1OLKlg+3Zb6emr\nr+Dkk11H4yuqyftR5crWmtE8NpIqBg2C9u2V4BNAST5ZtG8Pn3/uOgqR+Nu1C158Efr2dR1JSlCS\nTxZK8pIq3noLWrfW9AUJopp8sti/39Z9XbrUrkWCKCvLSpPDhsG557qOxpdUk/erkiWtK+Xkya4j\nEYmfTz+1sSHnnOM6kpShJJ9MLrlEJRsJtpdegnvu0fQFCaRyTTJZtswGRK1bpz8CCZ7MTLj4Yli+\nXFN4REHlGj9r0MCWBVywIP99Rfxm4EC4/XYl+ATTtAbJJruXTfPmriMRiZ0dO2DUKPjxR9eRpBy1\n5JONulJKEI0cCRdcADVquI4k5agmn2x27IATT7T1LsuVcx2NSGy0bAmPPgqXXuo6Et9TTd7vKlSA\nM8/UhGUSHPPnw8aN1ntMEk5JPhl17GgrRokEweDBtkB38eKuI0lJKtcko0WLoF07WL1aXSnF33bv\nhlq1rDVfu7braAJB5ZogaNwYypaFefNcRyISnQkToEULJXiHlOSTUVqaLaYwYYLrSESiM2IE3Hij\n6yhSmpJ8srr8chg/3nUUIkW3aRPMmAFXXeU6kpSmJJ+szj0XVqyANWtcRyJSNCNHWmOlfHnXkaS0\nWCT5SsBYYCHwI9AKqAJMBpYAk7x9pDBKlLBeNirZiF+NGAE33eQ6ipQXiyT/EvAp0AQ4HVgEPIgl\n+UbAFO++FFbnzvDBB66jECm8RYtg7Vq48ELXkaS8aPvnVQTmAfVzbV8EtAU2AtWBEHBKrn3UhTI/\nO3fa6Ndly6BqVdfRiBTcY4/Bli22zJ/EVKK7UNYDfgGGAt8Cg4FyQDUsweNdV4vyOKmpXDno0AE+\n/NB1JCKFM2oUXHut6yiE6GehLAG0AP4GfA28yOGlmbB3OUxGRsbvt9PT00lPT48ynAC65hp47TW4\n5RbXkYgUTGamzcHUqpXrSAIhFAoRimKak2jLNdWBL7EWPUAboC9WvrkA2ADUAKahck3R7NplJRut\n/Sp+8cgj9v/2uedcRxJIiS7XbABWYydYAdoBmcAE4GZv283AuCiPk7rKllXJRvwjHIbRo+0XqCSF\nWEyM0hwYApQClgHdgeLAaKAOsAK4BvhfruepJV9QH3wAr7wCU6a4jkTk6BYssF5hP/+seZfipLAt\neU1Q5gea5En8ol8/OHAABgxwHUlgaYKyIDrmGOja1QaXiCQrlWqSkpK8X3TrBsOG2R+SSDKaPx+y\nsmzWSUkaSvJ+0aoVFCsGX37pOhKRvGW34lWLTypK8n6Rlgbdu8PQoa4jEcnbuHGacTIJ6cSrn6xd\nC82a2cyUWuRbkkn2amarVtkvTokbnXgNspo14fzz4Z13XEcicqiPPrKuk0rwSUefiN/cfTe8/LJO\nwEpyGTcOrrjCdRSSByV5v8meunXqVLdxiGRbvx4WL4a2bV1HInlQkvebtLSc1rxIMhg/Hi69FEqV\nch2J5EFJ3o9uuAFmzbJ55kVcU6kmqal3jV/16we//AJvvOE6Ekll27fblBtr10KFCq6jSQnqXZMq\neveG99+H5ctdRyKpbOJEOO88JfgkpiTvV8cdB3feCU884ToSSWUq1SQ9lWv8bOtWaNgQ5syBBg1c\nRyOpZu9eqF4dFi60a0kIlWtSSeXK0KsXPPyw60gkFYVC0LSpEnySU5L3u7//3SYtU795STSVanxB\n5ZogGDcO+vaF775TX2VJjKws61UzfbqVDCVhVK5JRZ07W03+hRdcRyKp4uuvrVyoBJ/0lOSDIC0N\nBg60JD9vnutoJBWoVOMbSvJBUa+eJfprrrEBKiLxpCTvG6rJB80dd8CWLTBypFbokfjQ3PFOqSaf\n6v71L1i5Evr00XTEEh+aO95X9CkFTZky8OmnMHkyPPaY62gkiFSq8ZVYJfniwDxggne/CjAZWAJM\nAirF6DhSEFWqwKRJ8N57cP/9cOCA64gkKDR3vO/EKsnfA/wIZNcHHsSSfCNgindfEqlaNZg9GxYs\ngI4dYfNm1xFJEIwfDx06aDyGj8QiydcCOgJDyDkZ0AkY7t0eDui3nQtVqtgsgX/4gy0A/vbbqtNL\ndMaNgyuvdB2FFEIsul+MAZ4EjgX+DlwObAUqRxxjS8T9bOpdk0jffAO33w4lS9ro2Msu04kzKZxt\n26B2bc0d71hhe9eUiPJ4lwGbsHp8+hH2CZNTxjlERkbG77fT09NJTz/SS0jU/vhHm61y7FjIyIAH\nHoDu3eH66214ukh+Jk6E889Xgk+wUChEKBQq8vOjbck/CdwIHADKYK35D4CzsKS/AagBTANOyfVc\nteRdCYdhxgwYMcIWHmnRAv7yF7jqKjj2WNfRSbL685+tf3zPnq4jSWmFbcnHcrRMW3LKNc8Am4EB\n2EnXShx+8lVJPhns2QMTJsA778C0adC+PXTrZifXVM6RbHv32sn8xYvtWpxxPRgqO2s/DVyMdaG8\n0LsvyahMGeja1U6oLV9uLbX/+z845RR49VXYv991hJIMpk6F005TgvchTWsghwuHYdYsePxxS/zP\nPgudOmmahFR2223QqJGNuxCnXJZrCktJ3g8++wzuvRfOPNNa9qrZp56DB6FmTfvi1zKTzrku10jQ\ndOgA335ryf0Pf4D5811HJIk2Zw6ccIISvE8pyUv+ypa1VvxTT8Ell9hqQJI6NFeNrynJS8Fde61N\nYdy1K3z8setoJBHCYfjwQ41y9TEleSmcCy+0BN+9uy0gLsH244+wb5+V6sSXlOSl8Fq2hOHDrXW3\nZInraCSesks16lnlW0ryUjQdO8I//2lz4OzY4ToaiZexY1Wq8Tl1oZTo3HIL7N5tUySotRcsS5bY\nvPFr1kDx4q6jEY+6UEpivfQSzJtn0xhLsIwZA126KMH7nFryEr0ffoALLoC5c6FePdfRSKycfjoM\nGgRt2riORCKoJS+J16yZDXe/804tShIUCxfaamLnnOM6EomSkrzExv3322ISo0a5jkRiYfRoGw+h\nmUh9T+UaiZ05c6y7XWamLT0o/nXqqTBkCLRu7ToSyUXlGnGnVSvo3NlmrxT/ysy0brGtWrmORGJA\nLXmJrU2boGlT+OorOPlk19FIUTzyCOzcCc8/7zoSyYNa8uLWCSdA7962WLj4Tzhs9fhrr3UdicSI\nkrzE3r33Wn1+1izXkUhhffutrQZ21lmuI5EYUZKX2CtbFh59FPr1cx2JFNbw4XDTTRq9HCCqyUt8\nHDhg68QOGQLp6a6jkYLYtw9q1bLzKfXru45GjkA1eUkOJUrYguAZGa4jkYKaONG+mJXgA0VJXuLn\nhhtsgNS0aa4jkYLILtVIoKhcI/H19tswdKgSfbLbvNnWcF25EipWdB2NHIXKNZJcrrsOfv4Zvv7a\ndSRyNCNG2NoASvCBE22Srw1MAzKBH4C7ve1VgMnAEmASUCnK44hflSwJ990Hzz7rOhI5knAYXnsN\nbr/ddSQSB9Em+f3AfcCpwNnAXUAT4EEsyTcCpnj3JVX17AlTp8KyZa4jkbyEQnai/NxzXUcicRBt\nkt8AzPdu/wYsBGoCnYDh3vbhwBVRHkf8rHx5uO02eOEF15FIXl59Fe64Q33jAyqWn2pdYDrQDFgF\nVI44xpaI+9l04jWVbNgATZrYknLHH+86Gsm2fr3NNbRyJRx7rOtopAAKe+K1RIyOWx54H7gHyL2q\nc9i7HCYjog91eno66Ro0E1zVq9v85K+8or7zyWTwYPtclOCTVigUIhQKFfn5sWjJlwQ+BiYCL3rb\nFgHpWDmnBnZy9pRcz1NLPtUsXgznnQcrVtjUB+LW7t1Qt67V5Js0cR2NFFCiu1CmAW8CP5KT4AHG\nAzd7t28GxkV5HAmCxo1tjvJ333UdiYCNXzj7bCX4gIu2Jd8G+AJYQE5Jpi8wFxgN1AFWANcA/8v1\nXLXkU9HkybZU4Hff6USfSwcOQKNG8J//aB1Xn0l0TX4mR/410C7K15YgatfOEsz06Zq4zKWxY6Fm\nTSX4FKARr5JYaWnQqxe8/LLrSFLXwYPwxBPwoIavpAIleUm8G2+0lvzKla4jSU0jRkClStCxo+tI\nJAE0QZm40bu3TXkwYIDrSFLL7t1Wix8zxk66iu8UtiavJC9uLFtmPW1WrVJ3ykR6+mn45huryYsv\nKcmLf3TqBJdfDrfc4jqS1LBmDZxxhq2926iR62ikiJTkxT9SpTvlgQM2AGzTJitR1akDJ5yQ2Pcc\nDlsNvnVreOSRxB1XYs7VtAYihRfk7pRZWfDRRzbgaPp0qFIFqlWD/futRLV3r73/P/0pMdMKDB0K\nGzdC377xPY4kHbXkxa1XX7UW/QcfuI4kdiZNsu6JxYpZd9HLLoPjjjt0n19+gc8+g3HjbNWsG26w\nk9H16sU+nhUr4KyzYMoUOP302L++JJTKNeIvv/0GJ50E335r1362Z48tkDJpEjzzDFx1VcFKMmvW\nwKBB8PrrcP318PDD1uqPhZ07bZ74bt3g3ntj85rilJb/E38pXx5uvtlmp/SzVats9Oivv9oX1tVX\nF7zmXqsWPPkkLFpki3c0bWqJftu26GLat89KQWecAffcE91riW8pyYt7f/sbvPWWtTr9aNkyOP98\nW8929Oiir5N6/PHwr3/Zl8TatdCwITz/vP1CKKxdu+yXRKlS8MYbwT6xLUelJC/u1a9vUxC//bbr\nSApv8WI7ady3L/TpE5tketJJdqJ02jSYOdOS/ZtvWsu8IBYtsl8VVavaoKeSJaOPSXxLNXlJDtOn\n20LSmZl2wtIP1q/P6ZLYo0f8jvPVV1a+ycy041x7LTRrdvi/088/w8CBNm3BE0/ArbeqBR9AOvEq\n/hQOQ4sW8NRT0KGD62jy99tvVqK56ipLwImwcKGVXj75BLZsgTPPtF47u3fbY5s32/mNv//dVuKS\nQFKSF/8aNgxGjrSuhcksKwuuuMIGNA0e7Ka1vHatDSLbsgXKlIGTT4bTToPixRMfiySUkrz41549\nthzd1KnWwyRZPfEETJxoNXPVuyXB1IVS/KtMGavLJ/Nc85MnW3fPUaOU4MUX1JKX5LJxI5xyCixd\nar1DksmGDdbn/L33gjcNg/iGWvLib9WqQZcu8O9/u47kUOGw9Wzp2VMJXnxFLXlJPkuWQJs2sHw5\nlCvnOhozaJD1XZ89W2UacUonXiUYunSxAVLJMBx/8WL70pk5Exo3dh2NpDgleQmGr7+2RP/TT25b\nzvv324Cnv/4V7rjDXRwiHtXkJRjOOsuG87ue6uDRR+08we23u41DpIji2ZLvALwIFAeGALlXbFZL\nXo5u5ky48UYrl5Qqlfjjz55ts0nOm6cRpJI0kqUlXxz4N5bomwLXAU3idCwJqjZtbC3SoUMTf+wd\nO+wL5rXXlODF1+LVkm8N9MeSPMCD3vXTEfuoJS/5mzPH5kRfuhRKl07ccXv0sAnAhgxJ3DFFCiBZ\nWvI1gdUR99d420QKp1UrW7Lu9dcTd8yxY2HGDHjxxcQdUyRO4rWQd4Ga6BkZGb/fTk9PJ12DTCQv\nTz0FF11k66DmXis11tasgbvuggkTbNUqEcdCoRChUKjIz49XueZsIIOcck1fIItDT76qXCMFd9dd\nVj4ZODB+x8jKgnbt7AulX7/4HUckCsnST74EsBi4CFgHzMVOvi6M2EdJXgru119tZspp0+DUU+Nz\njGefhfHjIRTSlL2StJIlyQNcSk4XyjeBp3I9riQvhfPyy/DhhzBlSuxXj5o507pLzp1ry++JJKlk\nSvL5UZKXwjlwwKY6uP566NUrdq+7fj388Y/Wk+bSS2P3uiJxoCQvwbZkiS1SPXu29aGP1r59VoNv\n1w7694/+9UTiTElegm/gQHjnHVv8O5q+81lZ8Je/2Bqp77/vnwXEJaUpyUvwZWXZAKly5WD48KKt\nsRoOw/3320RokybBMcfEPk6ROEiWwVAi8VOsGIwYAYsWweOPF/754TA8/LAl9/HjleAl0OI1GEok\nvsqWtQTdurUNWurdu2DP278fbrkFFi607piVK8c3ThHH1JIX/6peHb74AgYPtqmA9+zJeeznn62/\n+65dOdtWrrQTrL/+ClOnwvHHJzxkkURTkhd/q13bJjHbts0GSb3zDowZA82aQadO0Ly5tdofegha\ntICOHeGjj5JnWUGRONOJVwmO//4XBgywVnpWlm1LS7PSTrdu8I9/QJ06TkMUiZZ614h06GAJ/+BB\nKFMGPv8czj/fdVQiMaHeNZLywm+9BeeeCyeeCI88ogQvKU29ayQwsrKyeOWVV7jvvvtYsGABTZs2\ndR2SiHNK8hIImZmZ3HbbbSxZsgSAqlWrOo5IJDmoXCO+tnfvXvr160d6ejotWrSgdOnSpKWlcVy8\nFxcR8QklefG177//nrVr1zJ79mw+/vhjBgwYQOXKlSmu+eBFAPWukYC455572Lp1K3369OG6667j\nhx9+cB2SSFwUtneNavLie9OnT2fMmDH88MMPLFiwgGrVqrkOSSRpqCUvvrZmzRpatmzJ0KFDad++\nPXv27GHdunXUr1/fdWgicaHBUJIy9uzZQ9u2bbnyyit58MEHXYcjkhBK8pISwuEwPXv2ZPv27Ywe\nPTr7P75I4KkmLynh9ddfZ86cOXz11VdK8CJHoZa8+M6sWbO48sormTVrFg0bNnQdjkhCae4aCbR1\n69ZxzTXXMGzYMCV4kQKIJsk/CywEvgM+ACpGPNYXWAosAi6J4hgiv9u7dy9dunThzjvvpGPHjq7D\nEfGFaJL8JOBUoDmwBEvsAE2Ba73rDsCgKI/jS6FQyHUIceXi/d19991Ur16dvn375r9zlPT5+VeQ\n31tRRJN8JwPeygzMAWp5tzsD7wH7gRXAT0DLKI7jS0H/j5bo9/fGG28wY8YMhg8fTrFi8W8z6PPz\nryC/t6KIVe+aHlhiBzgR+CrisTVAzRgdR1LQxIkTefjhh5kxYwYVKlRwHY6Ir+SX5CcD1fPY/hAw\nwbvdD9gHvHuU11E3GimScePGceutt/LRRx/RuHFj1+GI+E60XSi7AbcAFwF7vG3ZQw+f9q4/A/pj\nJZ1IPwENojy+iEiqWQacnIgDdQAygdyrMzQF5gOlgHpeQBqtIoV1E7AeON11ICKpaimwEpjnXQZF\nPPYQ1lJfBLRPfGjic7di53KauA5ERERiqxfWeEjIz1ERib2uWJnnINAiYntdYDd5/zLwkyO9Pwje\nILEMrMWd/Zl1iPL1/oGV9+pG+TrR6IB9PkuBBxzGES8rgAXY5zXXbSgx8RawEfg+YlsVrNPIEmw8\nTyUHccVKXu8vg9j+3cXcKUAjYBqHJ/nv83qCzxzp/WWfqyiJvdef8P8gsf5A7xi8ThrwCJZcXXa3\nLY59LnWxz2k+wSsZLceSYFCcB5zBobnjGazBAPZF/XTuJ/lIXu+vUH93LpLMIuwbNqiO9P6COkgs\n2pPqxYBDmoSKAAAExElEQVQngWuAdGBttAFFoSX2uazAPqeR2OcWNEHqCDED2JprWydguHd7OHBF\nQiOKrbzeH/h4grJ62M+PENDGbSgxdyL2EytbUAaJ9cLmL3qTwv8srgqMx5J7OrAhloEVQU1gdcT9\noHxGkcLAf4FvsO7PQVQNK3HgXQdxPcgC/93FK8lPxn5e5L5cfpTnrANqYz9NemODq5J1eGNR3l9e\n/DBI7EjvtRPwKvbF/Aesu+PzhXjd84BvgR+B84FfYxdykfnh84jWudjf2KXAXdjnEGRhgve5Furv\nLl6LhlxchOfs8y5gf/zLgIbe7WRTlPe3FvsSy1YLt6WJgiroex1CzijooymODZjrBXQHJhYxrnjI\n/RnV5tBfX0Gw3rv+BfgQK1HNcBdOXGzERupvAGoAm9yGE3OR7yffvzvX5ZrIulJVLAEA1McS/M8J\njyi2It/feODP5AwSa4j/ezfUiLh9JfmfOK+OjYC+BDiT5ErwYCWMhtiJ11LYbKrjXQYUY2XJ+XVc\nDvscgtDZIbfxwM3e7ZuBcQ5jiYfC/t0l3JVY3XM39k2b/Yd+NfADVpP/f8CfnEQXvSO9PwjeILG3\nse5432F/SEerfbbDWsqPktzLTl4KLMY+p/jPaZxY9bAeQ/Oxv7UgvL/3sFLvPuzvrjvWe+i/BKML\nZe7314PC/d2JxF0J4HEswV/kOBYREYmhWsAXWGtKrQ0RkYAoBvTEToD1xf25HxERiZEW2KIxs7Gu\neiIiEgCVgVewk809UOtdRCQwmmH9rwcRrHlRREQEqEjwJvISERERERERERERkWBqBnyOTRqVlc++\nxwGzsFkbt2HTQOSeL/s+7EToNmwq0lK5Hv8zsBD4DRu+n9e0zo94sVwYse0CbDGU/2GLT+T2ODaP\nxn5sYQMREcFWkuqOTd+bX5IvDTQmp4tiZ2xei/Le/fZYN8Ym2Jwd04CnIp5/MbYoRvaCJTWwue4j\nNcDmxljDoUn+LOAGbB7yvJL8TdhSZOOwLwkREYlwMvkn+UjFsDns15HTWn8X+GfEPheQM70s2CCl\n7vm87kRsoq7lHJrks7Uj7ySfbQRqyYskBQ1Y8a8F2EyXw7CZL7Pn4m+KzU4XuV81bLBScWyK3xOw\nhapXAwOBMhH7dwX2kHzTAItIESjJ+9fp2NzgGcD72PzgYGWbbRH7bfeuK2DJviQ2rXMbbGWZM4CH\nI/Z5ArgnjnGLSAIpybtxA7DDu3wSxevsw1riO8iZxvc34NiIfSp61zuwlj/eczYCm4EXgI7e9gys\n1LIq4vlBWvRZJOUoybvxDtZqrkBsFkcpAezybmdiLfRszbGEvtW75LWcXfYamBcCd2M1/PXY8nej\ngT5FiClo62qKiESlDFZPz8J60JQ+wn6tsFJLKeAY4AGsth7Zu2Y91rumMhACnox4/qPYsoPHe4/P\n8LaBzTtzgnephrXoryanFJTmxXkp1kOnNId2zyzhPf4u1p2yDGpIiIhQF0vuWcBB7zpyfdtPscWv\nAc7Hlm/bjvWr/wQ4Ndfr3Yd1o8zuJ18y4rES2GyRW7Evgxc5vB99tty9a9LziHNqxOPDIh7Pvtx0\nhNcWERERERERERERERERERERERERERERERERERERERGRwPv/Uq3KArMc+wwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x4c21450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(x, f(x), 'r-')\n",
    "scatter([xmin],[f(xmin)], 10, color='red')\n",
    "annotate('%f' % xmin, \n",
    "        xy=(xmin, f(xmin)), fontsize=12, xycoords='data',\n",
    "          xytext=(-90, -50), textcoords='offset points',\n",
    "          arrowprops=dict(arrowstyle=\"->\", connectionstyle=\"arc3,rad=.2\"))\n",
    "title('Brute force')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Simulated Annealing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning: Cooled to 8.328202 at 3.78222236155 but this is not the smallest point found.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(-1.2167727301519746, 5)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.optimize import anneal\n",
    "\n",
    "xmin, status = anneal(f, 3)\n",
    "xmin, status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x5193890>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEeCAYAAABv8mXfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcjvX+x/EXYxASUfYtpVQO6SDpHNNCWoiKaJE6vzZ1\ntJ0WJ4eptHBSUSmqIykVWUubnG6UOiSiZCuOpca+HEVk7t8fn2tyGzNjZu7le1/X/X4+HvO4t+u6\n788198zn/t6f7/f6fkFERERERERERERERERERERERERERESidBXwYZye+xXg4Tg9t1/0AmZH3P4f\nUN9JJOILJV0HIL50FjAH2A5sAT4F/ug99jpwfpxeN+z9FEYI+EsMXvMVYB9QPQbPFQ9HAqtdByHJ\nS0leiqoi8C4wFKgM1AIeBH5N0OuXKOR2hf0wKEh54DJgCXB1DJ5PRCTp/RHYVsDjvTi4nJAN3AKs\nAHYCDwENgc+xbwJvAun57Juz/3He9VEcKNdUxj5sNgJbgXewDxyAR4DfgN1YOWOYd/9JwHTs28dS\noGuBRwo9gUVYCWpxrscygXHAaO+4vgFOj3h8NXA38HXEcZaJePxiYCH2u/wMaBLx2P3ASu95vwU6\nRzzWi0N/vzm/n1eA57Dfy07gi4jHANoDy7x4ngNmEptvOyISIEcCm7GE0gFLtpF6cWgSmgRUAE7G\nWvz/xurIFbEk1jOffXP2zyvJHw10Acp6zz3Oe50cnwDXR9wuD6wFrsW+wTYDNgGNCzjWGcAD3jHv\nBppHPJbp3dcB+3bxKPbBlWMVlmSrY7+jJcBN3mOnARuAFt6+Pb3tcz7sLudAeagbsAuo5t3uRcFJ\nfjP2QZwGvAa84T1WFdiBfWCUBPoAezn4dyQBpHKNFNX/sJp8GHgRa0lPAY4tYJ/BWKJagrWI38da\nuju966cVI46tWFLf4z33o0DbXNtElnYuxhLpaCwxLgQmkn9rvi6QAYzHjvlDDnwY5ZgNfID9Ll4D\nmuZ6fBiQhbXW38E+WABuBEYA87x9X8U+/Fp7j7/t7Qf24bUCaJVPnJHC3jF9CezH+kdyXvNC7NvG\nZOz4c2KTgFOSl+JYClwH1AFOBWoCTxew/YaI67tz3d6DtcSLqhyWKFdjLdSZwFEcnNgj6/L1sES5\nLeLnSg60kHO7BkuKy73b473t0yK2iTyOX7BvFZH/U5FJdDcHjrMeVsqJjKU2UMN7vCewIOKxU4Eq\n+cSZW+7fdc5r1gTW5do2920JoFKuAxDfW4a1jm8s5v6RifhnLHnnyGtES872dwONgJbYt4lmwFdY\nks9rFM4a7IOgfSHj6ol9iP3k3S6FJdqLgKmFfI78Yl+D9Rs8msc29YCRwDlY+SeMJfzCdjjn50eg\nY8TtEtgHiwScWvJSVCcCd3Ggk7MO0IOD69GHUyKf618Dp2Blj7JY3Tv3fjnbV8Baqjuw+vyAXNtu\nwDp4c7yLfShcjdW+07Ga+El5xNcaq3O38GJpirWmx3JoyaYocmJ/EbgZ+4AqgfUXXOQdU3kssW/G\n/j+v8167KM+fl/ewzt1LsA+sW0neYaESQ0ryUlT/w8oe/8Fq4Z9jI1Du9h7P3YrOayhj7sdzbi/H\nRt98jH1DmF3Atk8DR2DJcA5W24/cdijWgbnV23YX1orvDqzHWuiPAaXziK8nVrv+FvuWsBH70BiK\nJePKeRxnfseaV+zzgRuAZ734VnDgw2MJMAT7vWZhCf7TfJ4n92sWFNNmrP9hsHe9MVa7T9TQV0ly\n/8L+yCOHkf0T+A5rfU3E6qE5+mJ/uEsp/NdjEUmcktiHXe7OaklRf8JGQEQm+XYc+CbwuPcDNkxu\nIfZ1uD423lffGETcaw9Uwsbr98OSfJkC9xDfK2zync2hJ8BMx4ZigX11z+nEuQQbm7sPG/mwEqs9\niohbrbH/x01Y2akzKtcEXqxa2NdjHTtw6FCtdRzopBMRdx7EToqqiCX8eW7DkUSIRZJ/ADtzbmwB\n28RiHhERESmiaMfJ98LOpDs34r712LC6HLW9+w7SsGHD8Pfffx/ly4uIpJzvgeMLu3E0LfkOwD1Y\nDX5PxP1TsWFqpYEGwAnA3EOi/P57wuFwYH8GDBjgPAYdn44vFY8vyMcWDofh4PM/DquwLfk3sKFW\nVbFJngZgwyRLYx2wYON6e2PjfMd5l79596lcIyLiQGGTfI887vtXAds/St6nbIuISAJp/HqcZGRk\nuA4hrnR8/hbk4wvysRVHtJMeRSPs1ZdERKSQSpQoAUXI3WrJi4gEmJK8iEiAKcmLiASYkryISIAp\nyYuIBJiSvIhIgCnJi4gEmJK8iEiAKcmLiASYkryISIApyYuIBJiSvIhIgCnJi4gEmJK8iEiAKcmL\niASYkryISIApycfDzz/Dt9+6jkJEREk+LpYuha5dXUchIqLl/+Ji/36oUgWWL4djj3UdjYgEiJb/\nSwZpadCmDcya5ToSEUlxSvLx0rYtzJzpOgoRSXGFTfL/AjYAiyPuOxqYDiwHPgIqRTzWF1gBLAXa\nRx+mDynJi0gSKGySHwV0yHXf/ViSbwTM8G4DnAxc4V12AIYX4XWCo3lzWL0atmxxHYmIpLDCJt/Z\nwLZc93UCRnvXRwOdveuXAG8A+4DVwEqgZVRR+lF6OrRuDbNnu45ERFJYNC3salgJB++ymne9JrAu\nYrt1QK0oXse/VLIREcdiVUYJez8FPZ56MjIgFHIdhYiksFJR7LsBqA5kATWAjd7964E6EdvV9u47\nRGZm5u/XMzIyyMjIiCKcJPTHP8LKlbBtG1Su7DoaEfGhUChEKIrGYlFOhqoPvAM08W4PBrYAg7BO\n10re5cnAWKwOXwv4GDieQ1vzwT0ZKtJ558Htt0PHjq4jEZEAiNfJUG8Ac4ATgbXAdcDjQDtsCOU5\n3m2AJcA47/J9oDepWq4B1eVFxClNaxBvs2bB3XfDvHmuIxGRAChqS15JPt727IGqVeHHH6FiRdfR\niIjPae6aZFO2rHXAfvaZ60hEJAUpySeC6vIi4oiSfCIoyYuII6rJJ8Ivv8Axx8CGDVChgutoRMTH\nVJNPRuXK2YRlc+a4jkQk2DZvdh1B0lGSTxSVbETia/JkaNcOUqVCUEhK8omiJC8SP5s3Q+/e8Mwz\nUMJlFTr5qCafKD//DNWqwcaNVr4RkdgIh6FbN6hfH/75T9fRxJ1q8smqfHlo0gS++MJ1JCLB8tpr\n8O238PDDriNJSkryiaSSjUhsLVoEd90Fb75pJx7KIZTkE0lJXiR2tm+HSy+Fp5+GP/zBdTRJSzX5\nRNq5E2rWtE4itTpEii87Gy65BBo0gGHDXEeTUKrJJ7OKFaFxY5g713UkIv72yCO2GM8TT7iOJOkp\nySeaSjYi0Xn/fXjhBRg/HkqXdh1N0lOSTzSt+ypSfKtWQa9e1tFao4braHxBNflE274d6tSBLVvU\nChEpit274cwzLcnffrvraJxRTT7ZVaoEJ5yglaJEiiIchltusT6tPn1cR+MrSvIuqC4vUjQjRsD8\n+fDii5q2oIiU5F1QkhcpvLlzoX9/mDjRzhyXIlFN3oUtW2x875YtkJ7uOhqR5LVtm03TPWSInfgk\nqsn7QpUqNpnSV1+5jkQkeYXDcP310KmTEnwUlORdUclGpGDDhsG6dTB4sOtIfC0WSb4v8C2wGBgL\nlAGOBqYDy4GPgEoxeJ1gUZIXyd/cuXZW61tvQZkyrqPxtWhr8vWBfwONgV+Bt4D3gFOAzcBg4D6g\nMnB/rn1TtyYPNq98o0Y2j02pUq6jEUkev/wCzZpZku/a1XU0SSfRNfmdwD6gHFDKu/wR6ASM9rYZ\nDXSO8nWC59hjoVYtWLjQdSQiyaVfPzj9dCX4GIk2yW8FhgBrsOS+HSvTVAM2eNts8G5LbirZiBxs\n9mybsuDZZ11HEhjRJvmGwB1Y2aYmUAG4Otc2Ye9HclOSFzng11/hL3+B4cNtBJrERLTF4D8Cc4At\n3u2JQGsgC6juXdYANua1c2Zm5u/XMzIyyMjIiDIcn2nbFm6+Gfbvh7Q019GIuPXEE3DyydBZ1d1I\noVCIUBSTGkbb8doUeB1oAewBXgHmAvWwxD8I63CthDpe83biiTaCoFkz15GIuLN6tdXh58+3c0gk\nX4nueP0aeBX4Eljk3TcSeBxohw2hPMe7LXlRyUYE7rgD7rxTCT4ONK2Ba6+/DhMm2LwcIqnok0+s\nFr9kiZbFLISituSV5F1bt85KNRs3QkmdgCwpJhyGVq2sFd+jh+tofEFz1/hN7dpw1FHWihFJNRMm\nwG+/wRVXuI4ksJTkk4Hq8pKK9u2DBx6AQYP0LTaO9JtNBlr3VVLRa6/ZWd/t2rmOJNBUk08G//0v\ntGwJWVla9UZSw/79tpTfyJHWyJFCU03ej+rVsxVvvvnGdSQiiTF+PBxzjJUqJa6U5JPF+efDhx+6\njkIk/rKzbYbJBx7QN9cEUJJPFkrykirefdeWvbzgAteRpATV5JPFzp3WCbVhA5Qr5zoakfhp2xZu\nuQW6d3cdiS+pJu9XFSvCaafBrFmuIxGJn4UL4Ycf4LLLXEeSMpTkk0n79irZSLANHQq9e1u5RhJC\n684lk/PPh2uvdR2FSHxs3AiTJ8PKla4jSSlqySeT5s3tH2HtWteRiMTeiBFw+eVaECTBlOSTSVqa\nnf2nko0Ezf79duLTbbe5jiTlKMknGw2llCD64AOoWROaNnUdScpRkk827dvDjBk2M59IULz4Itx4\no+soUpKSfLKpWdOmH/7yS9eRiMTGjz/a0GBNJ+yEknwy6tABpk1zHYVIbIwaBV27QoUKriNJSUry\nyahTJ3jnHddRiEQvOxtefhluuMF1JClLST4ZtW4N69fbFMQifvbppzbD6umnu44kZSnJJ6O0NLjo\nIrXmxf/GjIGePTXbpENK8slKJRvxu927bQ3XK690HUlKU5JPVu3bw+ef2+yUIn70zjtWpqlVy3Uk\nKS0WSb4S8DbwHbAEaAUcDUwHlgMfedtIUVSoAG3a6MQo8a+cUo04FYskPxR4D2gM/AFYCtyPJflG\nwAzvthRVp042oZOI32zaBLNnQ5curiNJedH2hhwFLACOy3X/UqAtsAGoDoSAk3Jto0VDDicrC046\nyS7LlnUdjUjhPf+8JfmxY11HEjiJXjSkAbAJGAV8BbwIlAeqYQke77JalK+TmqpXt4VEVLIRvxk3\nTme4Jolok3wpoDkw3Lv8mUNLM2HvR4qjWzf7hxHxi6wsWwHq/PNdRyJEv2jIOu9nnnf7baAvkIWV\nabKAGsDGvHbOzMz8/XpGRgYZGRlRhhNAl14KffvacLQjjnAdjcjhvf02XHyxSowxEgqFCIVCxd4/\nFmcozAL+DxtJkwnkrEK9BRiEtewrkUcLXzX5Qjr3XPjrX6FzZ9eRiBzen/8M995riV5irqg1+Vgk\n+abAS0Bp4HvgOiANGAfUBVYD3YDtufZTki+sF16wWfzUiSXJbv16aNIEfvoJypRxHU0guUjyxaUk\nX1ibNsEJJ9iygEce6ToakfwNHWr1+FGjXEcSWIkeXSOJcMwxkJFhtU6RZDZunA0WkKShJO8XvXqp\ndSTJbe1aWLbM+pAkaSjJ+8WFF8LSpbBypetIRPI2fjxccgmULu06EomgJO8XpUvbbH6vvuo6EpG8\nTZoEl1/uOgrJRR2vfrJwobWUVq2Ckvp8liSycSM0agQbNmhUTZyp4zXImjWDqlXhgw9cRyJysHfe\nsTNcleCTjpK83/TpY8PURJLJ5Mk6WS9JqVzjN3v2QL16EApB48auoxGBXbtsYZA1a+Coo1xHE3gq\n1wRd2bJw003w7LOuIxExH30EZ5yhBJ+klOT96OabbYqD7blnihBxQKWapKZyjV9dfTWccorNUCni\nyr59tu7BokVayzVBVK5JFQ88AE89pYW+xa3Zs+H445Xgk5iSvF81bgzt28OwYa4jkVSmUk3SU7nG\nz5YvhzZtbKoDdXpJooXDNtLrww810iuBVK5JJY0a2Zw2Q4a4jkRS0YIFtlrZSSe5jkQKoJa8361Z\nY4t9z50LDRu6jkZSSf/+8OuvMGiQ60hSilryqaZuXVtqrU8f+/oskiiqx/uCknwQ3HmnTVo2ZYrr\nSCRVfP+9rVjWqpXrSOQwlOSDoHRpGD7cFvvetMl1NJIKpkyBTp00G6oP6B0KiowMuOoq6NkTsrNd\nRyNBp1KNb6jjNUj27YOzz4aOHeG++1xHI0GlueOdKmrHa6n4hSIJl54Ob7xhddITToBLL3UdkQTR\nu+9q7ngfUZIPmjp1YNo06NABypWzS5FYmjwZevRwHYUUUqzKNWnAl8A6oCNwNPAWUA9YDXQDck+Z\nqHJNPM2ZYzXTl1+28o1ILGjueOdcjZO/HVgC5GTt+4HpQCNghndbEunMM21Jtt69YeBAjaGX2Pjw\nQysHKsH7RiySfG3gQuAlDny6dAJGe9dHA+qGd6FVKzsTdto0q6GuWOE6IvG7yZOhSxfXUUgRxCLJ\nPwXcA0SO26sGbPCub/Buiws1asCsWZbkW7e2+ec3bnQdlfjRvn3WYOjUyXUkUgTRJvmLgY3AAvKv\nEYU5UMYRF9LT4e67YeFC2LYNTjwRbrkF5s1TGUcKb+ZMG7WlueN9JdrRNWdipZkLgbJARWAM1nqv\nDmQBNbAPgkNkZmb+fj0jI4OMjIwow5EC1a4NL7wAmZkwYoSNkChVCq65Bq68Eho0cB2hJDOVapwI\nhUKEQqFi7x/Lk6HaAn/DRtcMBrYAg7BO10oc2vmq0TWuhcPwxRcwZgyMH28nuFx1FVxxBVSp4jo6\nSSbhsA3P/fhjTS3smOtZKHOy9uNAO2A5cI53W5JNiRJWpx8+HNavt3p9znJuN94IS5a4jlCSxZdf\nQoUKSvA+pGkN5FAbN1o557nnbFGSgQOhZk3XUYlLDzxgcyI99pjrSFKe65a8BMGxx8I//gHLltn1\nP/wBXnpJnbSpTBOS+ZZa8nJ4S5ZA9+42Kuell3QiTKpZvtxmOV23TlMLJwG15CX2Tj7ZTqqqUgXa\ntoWsLNcRSSLltOKV4H1J75oUTtmy8PzzcNll0KYN/PCD64gkUVSq8TWVa6TonnsOnngCPv8cqld3\nHY3E008/wSmn2Le30qVdRyNoPnlJhFtvhc2b4eKLIRSyoXUSTFOnwgUXKMH7mMo1Ujz9+0PTpnbW\nrJYbDK6JE1Wq8TmVa6T49u2zjtjLLrO5cSRYNm+Ghg3hxx+hfHnX0YhH5RpJnPR0GDsWWraEP/8Z\nWrRwHZHE0qRJtrKYEryvqVwj0alf3yY9697dVg2S4HjrLZvHSHxN5RqJjWuvtXH0Tz7pOhKJhY0b\nbcK6n36CI45wHY1E0MlQ4saQIVa6mT/fdSQSCxMn2rxFSvC+pyQvsVG1KgwebLNX/vab62gkWirV\nBIaSvMTONdfYvDYjR7qORKKRlWWriJ1/vutIJAaU5CV2SpSAp56Chx6CHTtcRyPF9fbb0LGjTWUh\nvqckL7HVtKnVch/XOjG+NW4cdOvmOgqJEY2ukdhbv97moF+wAOrWdR2NFMXatdCsmZ0AVaaM62gk\nDxpdI+7VqgW9e8OAAa4jkaIaMwa6dlWCDxC15CU+tm+3tWK/+MIuJfmFw9C4MYwaZWv/SlJSS16S\nQ6VKcNtt8MgjriORwpo71yabO+MM15FIDKklL/Gj1ry/3Hor1KgB/fq5jkQKUNSWvJK8xNeDD8Kq\nVfDKK64jkYL8+qv1pcyfD/XquY5GCqBZKCW59Olj09WuXQt16riORvIzdaqNiFKCD5xoa/J1gE+A\nb4FvgD7e/UcD04HlwEdApShfR/yqcmXo1QuGDnUdiRTk+efhpptcRyFxEG25prr3sxCoAMwHOgPX\nAZuBwcB9QGXg/lz7qlyTKtassbHXP/xgHbKSXJYuhYwMe5+0zF/SS/TomiwswQPsAr4DagGdgNHe\n/aOxxC+pqm5duOgiGDHCdSSSlxEj4PrrleADKpYdr/WBmcCpwBqs9Z7zGlsjbudQSz6VfP21LQi9\napVOtEkmv/xiH8JffmkLwEjSczVOvgIwAbgd+F+ux8Lej6Sypk2tY+/1111HIpHeegtatVKCD7BY\njK5JxxL8GGCyd98GrFafBdQANua1Y2Zm5u/XMzIyyMjIiEE4krTuucdOkOrVC0rqPDznsrNtsRet\n5pXUQqEQoVCo2PtHW64pgdXctwB3Rtw/2LtvENbhWgl1vEo4DM2bw2OP2QLR4ta770L//jY2voTL\nU2akKBJ9MtRZwCxgEQdKMn2BucA4oC6wGugGbM+1r5J8Kho1CsaPh/fecx2J/OlPdpZr9+6uI5Ei\n0Bmvktz27LGOvk8/tYWixY05c+Dqq2H5ciilcyL9RBOUSXIrWxZuuAGefdZ1JKnt0Ufhb39Tgk8B\naslL4q1bZyNtVq+GihVdR5N6Zs2Cnj1h2TINZ/UhteQl+dWuDeedB6NHH35bia1wGO6916aAVoJP\nCUry4kafPvDMMzaMTxJnwgTYuxd69HAdiSSIkry40aYNVKgAH33kOpLUsXs33H8/DB6s8xRSiN5p\ncaNECWvNDxvmOpLU0b+/nadw3nmuI5EEUseruBP04ZRZWfDmmzB9OixeDBs32miWunVtpax27Wzi\ntuOOi38sn38Ol14KixbBMcfE//UkbtTxKv4R1OGUmzfD3XfDKafAwoXwl7/YiJZt22D9epsv5uqr\n4auvbMHss8+GSZNg//74xLN7N1x3nfWBKMGnHLXkxa2gDaccN87OIu3WzdZKrVGj4O337oWJE+Hp\np2HnThg4ELp0id00A+EwXHONPd+YMbF5TnFKZ7yK/3TrBmedZTV6v/r1V7jrLvjgA0v0p59etP3D\nYdu3b18b2vjoo3DuudHH9dBDMGUKzJ4N5cpF/3zinJK8+M9nn9nMlMuW+XPUx+7dVu9OT7fW8lFH\nFf+5srPtQ+If/4AGDWwyt6J+YIB9aAwaBC+/bH0e1aoVPyZJKqrJi/+ceaYlRj9OWrZrl3WeHn20\nlV2iSfBgH3Ldu8OSJfbB0bGjfdNZvrzwz7F7N9x8s83dHwopwac4JXlxr0QJuOMOq0v7yb59Vj+v\nXx9efTW288Ckp1uiXrHChj22aQOXXQYff5z/CWTZ2TB1Kpx2Gmzfbi34WrViF5P4kso1khz27rVk\n+dFHcOqprqM5vHDY1kXdssVGxqSlxff1du60lvmIETZCp21baNECqlSxlvuSJVbTr1zZ6vAXXaQ5\n4gNKNXnxr4ED4b//hRdfdB3J4T38sLWaQyEoXz6xr71uHcycCQsW2LDMsmVt3P0559hIJSX3QFOS\nF//atMlOilq+PLnHc0+bBjfdZItfV6/uOhpJMep4Ff865hirO48c6TqS/K1aZWWat95SghdfUEte\nksvixbb+6w8/JN9UuHv32kiga66B2293HY2kKLXkxd+aNLGO19dfdx3Jofr3t9Eqfj5pS1KOWvKS\nfP79b+jd20aMJMvJUTNn2hzsX3+d3P0FEnhqyYv/nX22zWMzZYrrSMyOHXDttTbqRwlefEYteUlO\nEybY4hZffOF+SGDPnjZM8vnn3cYhglryEhSdO9tZm5984jaOcePgP/+BJ55wG4dIMcUzyXcAlgIr\ngPvi+DoSRGlpNknXgAF2dqkL69fDX/8Kr72W+BOeRGIkXkk+DXgWS/QnAz2AxnF6LQmqHj1sAY7p\n0xP/2tnZNjPmbbfZ9AEiPhWvJN8SWAmsBvYBbwKXxOm1JKjS0iAz04YuJro1/8wz8PPPNr+7iI/F\nK8nXAtZG3F7n3SdSNF272nS+iZyGeNEim0dnzJjYziwp4kC8/oIL1ezKzMz8/XpGRgYZGRlxCkd8\nq2RJWyXpnnugfXubgjeedu+Gq66yjtaGDeP7WiKFEAqFCIVCxd4/XmPTzgAysZo8QF8gGxgUsY2G\nUErhhMOW4Dt1so7QeLr9dsjKgjffdD90UyQPyTILZSlgGXAu8CMwF+t8/S5iGyV5KbxvvrGpdL/7\nzuZQj4f337fZJb/+2uZlF0lCyZLkAS4AnsZG2rwMPJbrcSV5KZpbb7VW/fDhsX/u9ettFM3YsaCy\noSSxZEryh6MkL0WzbZtNYPbqq9aqj5W9ey2xX3wx/P3vsXtekThQkpdge/99uOUWK6lEu2h2jttu\ng7VrbRm/ZJkQTSQfSvISfDfeaItojxoV/XM99xwMG2ZTF1SqFP3zicSZ5q6R4BsyxCYue+656J5n\n/Hh47DFbAFsJXgJKZ3qI/xx5pK2z2qYNNGgAF15Y9Of44APryJ0+3Z5DJKDUkhd/Ou44mDjR5nn/\n7LOi7fvKKzYvzZQp0LRpPKITSRpK8uJfrVvbMoGdO9uUwHn59lv49FP47Ter4/frBw8+CKGQ7S8S\ncOp4Ff/76ivo1g3OOMOmQKhb1+4fMsSmKy5Z0koypUpB9erWYVu9utuYRYpJo2skNe3aZZ2oL7wA\nzZtDy5bw5JOwZ489XqKEPX7vvZquQHxNSV5S265dMGOGjaMfOhS2brX7y5SBlSuhdm238YlESUle\nxBNesoQSV15pif6xx2x2SRGfK2qS1xBKCZx9+/bRr18/nnzySfbs2UNaWprrkEScUZKXQJkzZw43\n3XQT69ato2TJkkrwkvI0hFICYceOHdx8881cfvnlnHXWWdSqVYuaNWu6DkvEOSV5CYRZs2ZRunRp\nZsyYwYQJE7jvvvs49thjXYcl4pw6XiUwwuEwXbp0oUmTJrRs2ZIRI0bw7rvvug5LJKY0QZmkrDfe\neIMVK1bQr18/tm7dSrVq1VyHJOKcWvISCIsXL+acc85h+vTpNGvWjO3bt/Pzzz9Tq1Yt16GJxJTG\nyUvK2bZtGy1atOChhx7iyiuvdB2OSFwpyUtK2b9/Px07duTEE0/kqaeech2OSNypJi8pZcCAAeze\nvZvBgwe7DkUkKelkKPGtSZMmMWbMGObNm0d6errrcESSkso14kvfffcdbdu2Zdq0abRo0cJ1OCIJ\nk8hyzT+B74CvgYnAURGP9QVWAEuB9lG8hsghduzYQefOnRk8eLASvMhhRJPkPwJOAZoCy7HEDnAy\ncIV32QEAKLyjAAAGyUlEQVQYHuXr+FIoFHIdQly5Or7s7Gx69uzJeeedR69eveL2Onr//CvIx1Yc\n0STf6UC2d/0/QM5E3ZcAbwD7gNXASqBlFK/jS0H/Q3N1fAMHDmTLli1xH0mj98+/gnxsxRGrjtfr\nscQOUBP4IuKxdYDOSJGojRkzhpEjRzJv3jxKly7tOhwRXzhckp8O5LUY5t+Bd7zrDwB7gbEFPI96\nWCUqL730EpmZmUyfPp0aNWq4DkfEN6IdXdMLuAE4F/AW0+R+7/Jx7/IDYABW0om0EmgY5euLiKSa\n74HjE/FCHYBvgaq57j8ZWAiUBhp4AWnlZCmuu4AfsL8lEUmgFcB/gQXez/CIx/6OtdSXAucnPjQJ\niL9jf2d1XAciIiKxUwJ4CFgCqAAv4jNdsTLPfqB5xP31gd3k/c3AT/I7PgjeSWKZ2OipnPesQwye\nswQwCFgEuFjaqQP2/qwA7nPw+vG2GvvdLgDmug0lJv4FbAAWR9x3NDZoZDl2Pk8lB3HFSl7Hl0ns\n/+9i6iSgEfAJhyb5xXnt4DP5HV9OX0U6dqwr8f9JYgOwmnmslACGAvOBKjF83sJKw96X+tj7tBBo\n7CCOeFqFJcGg+BNwGgfnjsHAvd71+zgwCMSP8jq+Iv3fuUgyS7FP2KDK7/iCepJYrDrV04DngVbY\naK0tMXreomiJvS+rsffpTex9C5ogDYSYDWzLdV8nYLR3fTTQOaERxVZexwc+nmq4Afb1IwSc5TaU\nmKuJfcXKEZSTxP6KzV/0MsX/WlwTmIENqW0HbI9NaEVWC1gbcTso71GkMPAx8CU2/DmIqmElDrzL\nIK4DWej/u3gl+enY14vcPx0L2OdHbBTFadhXkbHAkXGKL1rFOb68+OEksfyOtRPW8m4ANAN+AoYU\n4/k7YOWZGd71/0UfcrH54f2IVhvsf+wC4FasHBBkYYL3vhbp/y5e88m3K8Y+e70fgK+w8fUneNeT\nTXGObz0HDwWs7d2X7Ap7rC9x4CzowkgHHgauBroDM4sYVzzkfo/qcPC3ryD4ybvcBEzCSlSz3YUT\nFxuwM/WzsNFZG92GE3ORx3PY/zvX5ZrIulJVrC4LcByW4H9IeESxFXl8U7FklnOS2An4f3RD5PDG\nLhS+47wuVpJrirUqkyHBg5UwTsA6Xktjs6lOdRlQjJXjwLfj8tgIryAMdshtKnCtd/1aYLLDWOKh\nuP93CdMFq3vuxj5p3/fuvwz4BqvJzwcuchJd9PI7PgjeSWKvYsPxvsb+kQpT++yEtbTuxX0jIy8X\nAMuw96nvYbb1mwbYiKGF2P9aEI7vDazUuxf7v7sOGz30McEYQpn7+K6neP93InFXGngSO0v6TMex\niIhIDDXASlNTCNYYbRGRlJaGjeDYBNxJsMZni4iktDOwkVEzgVMdxyIiIjFSFXgR6yy6GrXeRUQC\nozk2cmYocJTjWEREJMYqAye6DkJEREREREREREREpBBOBT7ExpVnF2L7kdgUCfs5MFdHju7eYzuA\nzcBEbErfHI2Bf2PT+q4g/zm3+3uxnJPr/ubALGzWyCygTx77tvX2fbgQxyIiEniNsLk3OlG4JN8b\nS77zgJ65HqvDgeXzygOvYYtggM0+uhy4AxvieDawC5ucK1JDbI6MdRyc5Ktio2h6YLNJlsdWxIqU\ns7LSHGy9VhER8RxP4ZJ8jtkcmuQjVcBWyHnKu30qh87b/iGHJuP3sQm7VnFwkn+UAyvv5Od+bNm1\nUaglL+JMMs4CKLFzFlaO2YlN71vQwtQlOfjM1K7AHg6eRTNHK2xJss+wFv1UDp6HvR72jeRhdDKU\niFNK8sH2KTbNam1szdJ/evcvwxYeuAcrq7QH/gwc4T1+JPAIcHs+z1sH6wPog314rMKmRM0xDOgH\n/EwwV+YR8Q0lebeuwsom/wOmxfF1fgT+wYGSzj6so/UibKWgO4FxHFgFKRMYA6yJeI7IFvkvWEfu\nfOBX4EFs+uAjsSUQKwDjI/ZTa15EJEKsa/JgpZuClhucw4GFnRdgI3x+8n5+A7ZgLX+wRQtejtj3\naC/eiljdf0fEvr9gH2KTCncoIiLBVhY4GUuaZbyf/KR7238G/J93PafVfCUH6uT1sNkgh0Xs28Tb\nvhzwN2xN3XTvsaOxkTnHYqvOrMFW7irnPX42sBVbvi8dS+w5S/hVyLXvm9hCw35enUdEJCbqY8k9\nGxv7ns3B69y+h41ayRHKtW02VlsHGIgtF7YLq5k/jiX1HIOxRJ1TKjqugLhyj64BuBkr72zFFgOp\nlc++o9AQShERERERERERERERERERERERERERERERERERERERkRT1/3FoREYnbIk5AAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x361c850>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(x, f(x), 'r-')\n",
    "scatter([xmin],[f(xmin)], 10, color='red')\n",
    "annotate('%f' % xmin, \n",
    "        xy=(xmin, f(xmin)), fontsize=12, xycoords='data',\n",
    "          xytext=(-90, -50), textcoords='offset points',\n",
    "          arrowprops=dict(arrowstyle=\"->\", connectionstyle=\"arc3,rad=.2\"))\n",
    "title('Simulate Annealing')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Linear algebra using `scipy.linalg`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Linear algebra is particularly important to **projective geometry**\n",
    "- Consider a 3D point $\\mathbf{X}_i$\n",
    "- Let $\\mathrm{\\tt P}$ be a **camera matrix**\n",
    "- The projection of $\\mathbf{X}_i$ on the camera plane is $\\mathbf{x}_i = \\mathrm{\\tt P} \\mathbf{X}_i$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "Linear algebra is an essential mathematical tool for computer vision and machine learning. It is particularly important in problems involving projective geometry as in [multiple \n",
    "view computer vision](http://www.robots.ox.ac.uk/~vgg/hzbook/). Consider a 3D point $\\mathbf{X}_i$, represented as a *homogeneous* 4-d array, and a projective matrix $\\mathrm{\\tt P}$, represented as a $3 \\times 4$ array and corresponding to a camera. The\n",
    "projection of $\\mathbf{X}_i$ on the camera’s image plane, $\\mathbf{x}_i$, can be elegantly coded as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 900.15223192,  644.29279864,    3.81333274])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "P = array([[  1.41598501e+03,  -3.53656018e+01,   6.29880366e+02,  -9.80199681e+02],\n",
    "           [ -1.66130187e+01,   1.52666051e+03,   2.41551888e+02,   2.34435820e+02],\n",
    "           [ -2.20274424e-01,   4.97475710e-03,   9.75425256e-01,   6.34871255e-01]])\n",
    "\n",
    "X = array([-0.11652141, -0.24440939,  3.23347243,  1.])\n",
    "\n",
    "x = dot(P, X)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "The `dot` function, for 2-D arrays, computes the matrix multiplication, and for 1-D arrays it calculates the inner product of vectors. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The pixel inhomogeneous coordinates can be recovered using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(236.05394389057571, 168.95792816629523)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_coord = x[0]/x[2]\n",
    "y_coord = x[1]/x[2]\n",
    "x_coord, y_coord"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Singular value decompostion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Several problems in multiple view geometry involve **over-determined systems of equations**\n",
    "     - Homography estimation\n",
    "     - 3D points triangulation\n",
    "- $A\\mathbf{x}$ is a homogeneous systems of linear equations\n",
    "- Can be solved using **Singular Value Decomposition (SVD)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "Estimation problems in projective geometry involve the solution of over-determined systems of equations. More precisely, these problems are formalized as *linear least-squares*\n",
    "resolutions of homogeneous systems of linear equations in the form $A\\mathbf{x}$, minimizing $A\\mathbf{x}$ subject to $\\| \\mathbf{x} \\| = 1$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "The minimization problem can be solved using **singular value decomposition (SVD)**, a matrix decomposition particularly useful in numerical computations. SVD decomposes $A$\n",
    "as $A = UDV^\\intercal$ and the $\\mathbf{x}$ solution corresponds to the last columns of $V$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def dlt_triangulation(ui, Pi, uj, Pj):\n",
    "    \"\"\"Hartley & Zisserman, 12.2\"\"\"\n",
    "    ui /= ui[2]\n",
    "    xi, yi = ui[0], ui[1]\n",
    "    \n",
    "    uj /= uj[2]\n",
    "    xj, yj = uj[0], uj[1]\n",
    "    \n",
    "    a0 = xi * Pi[2,:] - Pi[0,:]\n",
    "    a1 = yi * Pi[2,:] - Pi[1,:]\n",
    "    a2 = xj * Pj[2,:] - Pj[0,:]\n",
    "    a3 = yj * Pj[2,:] - Pj[1,:]\n",
    "    \n",
    "    A = vstack((a0, a1, a2, a3))   \n",
    "    U, s, VT = linalg.svd(A)\n",
    "    V = VT.T    \n",
    "    \n",
    "    X3d = V[:,-1]    \n",
    "       \n",
    "    return X3d/X3d[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Examples of `spatial.KDTree` and `linalg.svd` will be presented in the Structure from Motion example."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}