{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "機械学習とは、その名の通り「機械」を「学習」させることで、あるデータに対して予測を行えるようにすることです。\n",
    "\n",
    "* 機械とは、具体的には数理・統計的なモデルになります。\n",
    "* 学習とは、そのモデルのパラメータを、実際のデータに沿うよう調整することです。\n",
    "\n",
    "![](./pictures/machine_learning.PNG)\n",
    "\n",
    "学習の方法は大きく分けて2つあります。  \n",
    "\n",
    "* 教師有り学習(Supervised learning): データと、そこから予測されるべき値(正解)を与えることで学習させます。\n",
    " * 分類(Classification): データがいくつかのカテゴリに分類できるとき、そのカテゴリを予測させます(例：手書きの数字が0～9の何れかであるか判別するなど)\n",
    " * 回帰(Regression): データから予測される連続的な値を予測します(例：年齢と体重から慎重を予測するなど)。\n",
    "* 教師なし学習(Unsupervised learning): データを与えることで、その裏側にある構造を学習させます\n",
    " * クラスタリング: 似ているデータをまとめることで、データがどれくらいの集合(クラスタ)から構成されるのかを予測します。\n",
    " * 分布推定： データを生み出している確率分布の推定を行います。\n",
    "\n",
    "[scikit-learn](http://scikit-learn.org/stable/index.html)は、Python製の機械学習ライブラリです。  \n",
    "この中には様々な「機械」が実装されており、その「学習」のための仕組みも備わっています。  \n",
    "\n",
    "以下では、このscikit-learnを利用しデータを準備するところから実際にモデルを構築し学習・評価を行うまでの手順を解説します。\n",
    "\n",
    "1. [データの準備](#Loading-the-Data) \n",
    "2. [データの整備](#Arrange-the-Data) \n",
    "3. [モデルの選択](#Select-the-Model)\n",
    "4. [データの分割](#Split-the-Data) \n",
    "5. [モデルの学習](#Training-the-Model)\n",
    "6. [モデルの評価](#Evaluate-Training-Result)\n",
    "7. [モデルの保管](#Store-the-Model)\n",
    "\n",
    "環境のセットアップについては、以下にまとめてあるのでご参考ください。\n",
    "\n",
    "[Pythonで機械学習アプリケーションの開発環境を構築する](http://qiita.com/icoxfog417/items/950b8af9100b64c0d8f9)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# enable showing matplotlib image inline\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scikit-learnでは、よく例として利用されるデータセット(irisのデータや手書き文字のデータなど)を以下のように簡単に取得することができます。\n",
    "\n",
    "[Dataset loading utilities](http://scikit-learn.org/stable/datasets/index.html#datasets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "digits = datasets.load_digits()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`dataset`は、以下の内容で構成されています。\n",
    "\n",
    "* `data`: データ本体(常にサンプル×特徴量の二次元配列。画像などデータ自体が2次元で表示される場合は、`images`からアクセスできる)\n",
    "* `target`: データから予測されるべき正解(教師データ)\n",
    "* `feature_names`: 特徴量項目の名前\n",
    "* `target_names` : 予測値項目の名前\n",
    "* `DESCR`: データの説明"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['DESCR', 'target_names', 'data', 'target', 'feature_names'])\n"
     ]
    }
   ],
   "source": [
    "print(iris.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通常のデータ読み込みには、Pythonに標準で搭載されている`csv`などが使えます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['1' 'good']\n",
      " ['2' 'bad']\n",
      " ['3' 'vgood']]\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "import numpy as np\n",
    "\n",
    "encoding = \"utf-8\"\n",
    "ratings = []\n",
    "with open(\"./data/ratings.txt\", encoding=\"utf-8\") as f:\n",
    "    content = csv.reader(f, delimiter=\"\\t\")\n",
    "    lines = list(content)\n",
    "    ratings = np.array(lines)\n",
    "\n",
    "print(ratings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "なお、このようなデータの読み込み、また読み込んだデータに対する操作をサポートするライブラリとして[pandas](http://pandas.pydata.org/)があります。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Arrange the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "データの中の各特徴量は、別々の平均・分散を持っています(例: 体重と身長では平均も分散も異なる)。  \n",
    "この状態だと学習が効率的に進まないため、各特徴量の平均を0・分散を1にそろえる正規化(Normalization)を行うことが一般的です。  \n",
    "(これに加え、特徴量間の相関を消す白色化(Whitening)まで行うこともあります)。  \n",
    "\n",
    "scikit-learnでは[`preprocessing`](http://scikit-learn.org/stable/modules/preprocessing.html)を使用することでこの作業をとても簡単に行うことができます。以下では、`StandardScaler`を使って処理を行っています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before scaling:\n",
      " {'std': array([ 0.82530129,  0.43214658,  1.75852918,  0.76061262]), 'mean': array([ 5.84333333,  3.054     ,  3.75866667,  1.19866667])}\n",
      "After scaling (mean is almost 0, std = 1):\n",
      " {'std': array([ 1.,  1.,  1.,  1.]), 'mean': array([ -1.69031455e-15,  -1.63702385e-15,  -1.48251781e-15,\n",
      "        -1.62314606e-15])}\n",
      "Inverse the scaling:\n",
      " {'std': array([ 0.82530129,  0.43214658,  1.75852918,  0.76061262]), 'mean': array([ 5.84333333,  3.054     ,  3.75866667,  1.19866667])}\n"
     ]
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "import numpy as np\n",
    "from sklearn import preprocessing\n",
    "\n",
    "iris_data = iris[\"data\"]\n",
    "scaler = preprocessing.StandardScaler().fit(iris_data)\n",
    "describe = lambda t, x: (t + \":\\n {0}\").format({\"mean\": np.mean(x, axis=0), \"std\": np.std(x, axis=0)})\n",
    "\n",
    "# before scaling\n",
    "print(describe(\"Before scaling\", iris_data))\n",
    "\n",
    "# scaling\n",
    "iris_data_scaled = scaler.transform(iris_data)\n",
    "print(describe(\"After scaling (mean is almost 0, std = 1)\", iris_data_scaled))\n",
    "\n",
    "# inverse\n",
    "iris_data_inv = scaler.inverse_transform(iris_data_scaled)\n",
    "print(describe(\"Inverse the scaling\", iris_data_inv))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "※`preprocessing`には`Normalization`というモジュールがありますが、これは一般的に言う正規化を行うためのものではないので注意してください。  \n",
    "\n",
    "また、データの中にはテキストである項目が含まれていることもあります。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['1' 'good']\n",
      " ['2' 'bad']\n",
      " ['3' 'vgood']]\n"
     ]
    }
   ],
   "source": [
    "print(ratings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上記の`good`などのテキスト項目は、最終的には数値にしないとデータを学習させることができません。  これも`preprocessing`を利用することで簡単に数値へ変換することができます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['good' 'bad' 'vgood'] is encoded to [1 0 3]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import preprocessing\n",
    "\n",
    "le = preprocessing.LabelEncoder()\n",
    "le.fit([\"bad\", \"nbad\", \"good\", \"vgood\"])\n",
    "\n",
    "encoded_rating = le.transform(ratings[:, 1])\n",
    "print(\"{0} is encoded to {1}\".format(ratings[:, 1], encoded_rating))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Feature Extraction](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_extraction)では、テキスト/画像についてより強力に特徴量の数値化(ベクトル化)を行う機能がサポートされています。以下では、cityというテキストの項目がDubai/London/San Fransiscoを表す0/1の特徴量へと変換されています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  1.   0.   0.  33.]\n",
      " [  0.   1.   0.  12.]\n",
      " [  0.   0.   1.  18.]\n",
      " [  1.   0.   0.  32.]]\n",
      "['city=Dubai', 'city=London', 'city=San Fransisco', 'temperature']\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_extraction import DictVectorizer\n",
    "\n",
    "measurements = [\n",
    "    {\"city\": \"Dubai\", \"temperature\": 33.},\n",
    "    {\"city\": \"London\", \"temperature\": 12.},\n",
    "    {\"city\": \"San Fransisco\", \"temperature\": 18.},\n",
    "    {\"city\": \"Dubai\", \"temperature\": 32.},\n",
    "]\n",
    "\n",
    "vec = DictVectorizer()\n",
    "vectorized = vec.fit_transform(measurements).toarray()\n",
    "print(vectorized)\n",
    "\n",
    "feature_names = vec.get_feature_names()\n",
    "print(feature_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`preprocessing`には、他にも欠損値の修正を行う`Imputer`などデータの整備に役立つモジュールが含まれています。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dimensionality reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "データを図示することは、この後のモデルの選択を行う時を含め、様々なシーンで非常に重要です。\n",
    "しかし、単純に特徴量が4つになっただけでもデータを図示することができなくなってしまいますし(4次元の図になってしまうため)、場合によっては非常に多くなることもあります(テキスト解析など)。\n",
    "\n",
    "そのため、データをなるべく少ない、必要最小限の特徴量で表現することが重要になります。これを行うのがDimensionality reduction(次元削除/次元圧縮)と呼ばれる手法です。\n",
    "\n",
    "具体的には、データの中に身長と体重があった場合、これらは体が大きくなれば両方とも増える特徴量のため、データの特性を表す上ではどちらかひとつで十分です。このように片方が増えれば片方も増えるといった、互いに相関のある特徴量を消していけば必要最小限の特徴量でデータを表現することができる・・・というのが基本的な考え方です。\n",
    "\n",
    "scikit-learnでは[`decomposition`](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.decomposition)を利用しこの処理を行うことができます。以下では、[TruncatedSVD](http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html#sklearn.decomposition.TruncatedSVD)によって数字データの特徴量を、上記で述べたとおり互いに相関のない、2つの特徴量へと圧縮しています。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "実際にデータを図示するには、scikit-learnではなく[matplotlib](http://matplotlib.org/examples/index.html)を利用します。\n",
    "以下では、最初の2つの特徴量をピックアップし、irisのデータをプロットしています。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x8c930d0>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhUAAAGHCAYAAAAHoqCrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xd4VNXWwOHfnknvlEAIPUAIRaS3SBEQEekoCihFFCzX\ngort4hUbYAPFi5+iIoiKIkpT0RtEREFBghDpvYUSamhJSGbW98eESCDdk0wS1vs88wB7zpq9znAY\nVs7sYkQEpZRSSql/yubuBJRSSilVOmhRoZRSSilLaFGhlFJKKUtoUaGUUkopS2hRoZRSSilLaFGh\nlFJKKUtoUaGUUkopS2hRoZRSSilLaFGhlFJKKUtoUaGUwhgzzBjjNMZUy+Pxe4wx04tLPkqp4kGL\nCqUUgKQ/8sqZz+PzK7/5ZDDGDDTGPGxxPkqpPDC694dSyhhjAE8RuZDH4z0Bp4g4CimfocB0oKaI\n7Mtn7CKggYhEFEZuSqns6Z0KpRTikmtBYYzxST8+tbAKCqVUyaVFhVIqyzEM6eMmFhpjuhpj/jDG\nJAEjL3lu+iXHehhjnjPGbDPGJBljjhljfjHGdM5D3/WNMUuNMeeNMfuNMf8mi88mY0wvY8w3xph4\nY0yyMWaHMWasMcZ2yTE/ATcD1dPPx2mM2ZX+nKcx5gVjzBpjzCljzFljzHJjTMd/8NYppS7h4e4E\nlFLFQlZjGASIAj4D3gOmAVsvee5SzwNPpR/zBxAENAeaAj9m16kxpiKwDFcRMR44j6twSc7i8GHA\nGeAN4CzQCXgBCASeTD/mJSAYqAw8Apj0Y0nP6S5gdnqegcAI4HtjTEsRicsuT6VU3mhRoZTKSS3g\nRhFZkstx3YFvReS+fL7+U0A5oKWIxAIYY2YCO7I4dqCIpFzy52nGmJPA/caYselfyfxojIkHQkRk\n9mXxJ4AaIpJ2scEY8z6uQulB4J585q6Uuox+/aGUysnuPBQUAKeABsaY2vl8/ZuA3y8WFAAichz4\n9PIDLy0ojDEBxphywK+AH647KjlKHzeSlh5vjDFlAC9gDa47Kkqpf0iLCqVUTnbn8bj/ACHANmNM\nnDHmVWPMNXmIqw5sz6J96+UN6WMv5hljTgGngaPArPSng/OSpDFmqDFmPa6vV44DCbjGYOQpXimV\nMy0qlFI5ScrLQSLyC66vSoYDf+Eaq7DWGHOXFUkYY4KB5cA1wFigB9CFv8dS5PpZZoy5A/gIVxFz\nF3Bj+msszUu8Uip3OqZCKWUJETkFzARmGmP8gF+AcbjWm8jOXqBOFu2Xf53RESgD9BaRFRcbjTG1\nskolm776AztF5JZLG40xL+SQn1IqH7Q6V0r9Y8aYspf+WUTO4xps6Z1L6HdAa2NM80teKxQYdNlx\nDlwzOS6dPuoF3J/Fa54j668zrlhXwxjTCmiTS45KqTzSOxVKKStsMsYsA2JxzbJoAdwCTMkl7lXg\nTuAHY8xbuKaU3gPsARpdctxK4CTwsTHm4mveQdZ3JWKBAcaYN3BNbz0rIt8A3wD9jDHzgW+BCGAU\nsBEIyM/JKqWypkWFUio7Oe2/cflzbwG9gBtw3Z3YCzwDvJ5jByKH0xefehvX+IjjwP8Bh4EPLjnu\nhDHmZlxrVLyIq8CYhWs8xA+Xvew7wLW41rV4JD2Xb0RkRvq6GKOArsAmYDAwAGifU55KqbzRvT+U\nUkopZQkdU6GUUkopS2hRoZRSSilLaFGhlFJKKUtoUaGUUkopS1w1RYUxxs8Y0zR9UR6llFJK5VFe\n/w+9mqaUNgZWAIONMVvcnYxSSilVgkTh2ugvGte6MVm6moqKGum/XrH7oVJKKaXypAZaVACuFfr4\n5JNPqFevXqF1Mnr0aCZPnlxor18S6HtQeO/B3Xffw/Zdu7j5zlFUj6zPjg3r+O6TaUS3acOrr75i\neX8FpdeAvgdX+/lD6XoPNm/ezB133AHp/5dm52oqKpIB6tWrR9OmTQutk+Dg4EJ9/ZJA34PCew9+\n+OF7Bg4cyBf//buA6NOnLzNnziAoKMjy/gpKrwF9D67284dS+x4k5/Tk1VRUKFXihYaGsmTJErZs\n2cLu3bupW7cuERER7k5LKaUALSqUKpGioqKIirp8d3CllHKvq2ZKqVJKKaUKlxYVFhs4cKC7U3A7\nfQ/0Pbjazx/0Pbjazx+uzvfgqtml1BjTFIiNjY0tjQNnlFJKqUKzdu1amjVrBtBMRNZmd5zeqVBK\nKaWUJbSoUEoppZQltKhQSimllCW0qFBKKaWUJbSoUEoppZQltKhQSimllCW0qFBKKaWUJbSoUEop\npZQltKhQSimllCW0qFBKKaWUJbSoUEoppZQltKhQSimllCW0qFBKKaWUJTzcnYBSpYXT6WTp0qWs\nW7eOypUr06dPH3x9fd2dllJKFRktKpSywNGjR+ne/WbWrPkDX39/ks6dI7RCBb5ZtIiWLVu6Oz2l\nlCoS+vWHUhYYNepeduzezdhpc/hg+WYmzV9OSFgVevfuQ0pKirvTU0qpIqFFhVL/0NGjR1mwYD79\nRj1K/eZtMMYQVq0mI//zOocPH+K7775zd4pKKVUktKhQ6h86duwYTqeTyjVrZ2qvVD0Cm83G4cOH\n3ZSZUkoVLS0qlPqHatasSUiZMqxe+n2m9tif/4fT6aRZs2ZuykwppYqWDtRU6h/y8fFhzOOP8+9/\n/xtHWipN29/Avm2bWPjRVLp06UKLFi3cnaJSShUJLSqUssDTTz+Nh4cHr732Oku+nIW3tzeDBw/m\nzTffxBjj7vSUUqpIaFGhlAWMMTzxxBOMHj2aQ4cOUbZsWQICAtydllJKFSktKpSykKenJ9WqVXN3\nGkop5RY6UFMppZRSltCiQimllFKW0KJCKaWUUpbQMRWqRBERli1bRkxMDL6+vgwYMIC6deu6Oy2l\nlFLonQpVgly4cIHevfvQqVMn3vtgOq+89jpRUVFMmDDB3akppZRC71SoEmTy5Mks/n4xj7w2jRad\nupGWeoGv3pvMM888Q6dOnWjVqpW7U1RKqaua3qlQJcZHM2bS5sbetOx8E8YYPL28GfDAE1QIr8LM\nmTPdnZ5SSl31tKhQJcbJEycIrVQlU5vNZqNcWGVOnDjhpqyUUkpdpEWFKjHaRrdl9Y/fknohJaPt\nyP49bIuLJTo62o2ZKaWUAh1ToUqQsf/+N9HR0bxwV3869rmdc2dP87/PP6Ja1WoMHTrU3ekppdRV\nT+9UqBKjWbNm/Pjjj4SXC+bD8U/z9btvcGOXTvzyy3KCgoLcnZ5SSl319E6FKlGio6NZvvxnUlJS\nsNvteHjoJayUUsWFfiKrEsnb29vdKSillLqMfv2hlFJKKUtoUaGUUkopSxS7osIY85QxxmmMmZTL\ncR2NMbHGmGRjzDZjjA7/V0oppdyoWI2pMMa0AEYC63M5rgbwDfAOMAjoAnxgjDkoIjGFnKZSJc6u\nXbuYPn068fHxNG7cmKFDhxISEuLutJRSpUyxuVNhjAkAPgHuBk7lcvh9wC4ReUJEtorIVGAuMLqQ\n01SqxJk7dy5RUVG89d+prIhdz+NjxlCvXn22bNni7tSUUqVMsSkqgKnAIhFZmodjWwNLLmv7AWhj\neVZKlWCJiYkMGz6cZh1v5L+LV/P8zIVMXrgCu68/I0eOcnd6SqlSplgUFcaY24HGwNN5DAkDjlzW\ndgQIMsboXEOl0n3zzTecO3uWOx77D14+vgCUq1iJ3iMe4pdflhMfH+/mDJVSpYnbx1QYY6oAbwJd\nRCS1sPsbPXo0wcHBmdoGDhzIwIEDC7trpYrc2bNnsdlsBASXydQeXLZcxvNKKXWp2bNnM3v27Ext\niYmJeYp1e1EBNANCgbXGGJPeZgfaG2P+BXiLiFwWcxioeFlbReC0iKSQg8mTJ9O0aVML0laq+OvY\nsSNOp5Nl8z+n623DABARfvz6M6pUqUqtWrXcm6BSqtjJ6gfttWvX0qxZs1xji0NRsQS45rK2GcBm\nYGIWBQXAb8BNl7V1TW9XSqWrW7cuI0aM4KPXnmPHX39SvW59/ly+hE2xvzNr1ixd5lwpZSm3f6KI\nyDlg06VtxphzwHER2Zz+5/FAZRG5uBbFu8ADxphXgOlAZ+AWoHuRJa5UCfHee+/RoEED/u/d91j7\n8w80vrYxixYtokePHu5OTSlVyri9qMjG5XcnKgFVM54U2WOMuRmYDDwEHABGiMjlM0KUuurZ7XZG\njx7N6NE641opVbiKZVEhIp0u+/PwLI5Zjms8hlJKKaWKgWIxpVQppZRSJZ8WFUoppZSyhBYVSiml\nlLKEFhVKZWHjxo106dKFimFhREZG8t///tfdKSmlVLGnRYVSl/nhhx9o3LgJy3/5lfDa9Uk8n8KD\nDz5It27d3J2aUkoVa8Vy9odS7nTnkCGUqRDG8zPmE1K+AiLCF/99hYUfTWXZsmV07NjR3SkqpVSx\npHcqlLrE4cOHOZqQQI+h9xJSvgIAxhj63v0wnl7evP76627OUCmlii8tKpS6REqKa+sYb1+/TO0e\nnp7Y7R5cuHDBHWkppVSJoEWFUpeoXr06gUFB/O/zGaRe+HtvuuWLviQ56RwjRoxwY3ZKKVW86ZgK\npS4z/uWXeeihh3m83/W07NydQ3t3sXb5EmrXqcNtt93m7vSUUqrY0jsVSl3mX//6F7Nnf4avh2HJ\nnJls//N3Bg0ayOZNm3IPVkqpq5jeqVAqC7fddpvelVBKqXzSOxVKKaWUsoQWFUoppZSyhBYVSiml\nlLKEFhVKKaWUsoQWFcptNm/eTGRkJF7e3nh5e9OoUSPi4+PdnVax98svv3Db7bfTomVLhg4dSmxs\nrLtTUkoVsfj4eJ588klat2xB1xtuYNasWTidTsv72bhxI/fccw933jE4T8drUaHcIi4ujmsaXcuu\n3Xto0q4L17Ruz8aNG6lZM4LDhw+7O71i68MPP6R9+/b8tuZPAsIj+N9Py2nVqhXz5s1zd2pKqSKy\nY8cOmjS+lnemTMb36DaObPidIUOGMHzYMETEsn6WLVtG82ZNWfD5J/gnHshTjLEygeLMGNMUiI2N\njaVp06buTueqV7NmTQ7EH2T87MVUiYgEYHvcWsYN70OzZs34448/3Jxh8XPmzBnCK1em2fU3cc9/\nXsNms+FIS+PNMSM5uG0je/fuwdPT091pKqUK2W0DBvDz94t4rUsVgn1cK0P8uOsUU1Yd5ueff6Z9\n+/b/uA8RoWH9+pgT+xnXsTL7Ey/w6A97AJqJyNrs4vROhXKLA/EHaX1Dj4yCAqBOo6Y0bHkdcXFx\nbsys+Fq2bBlnz5yhz4gHsdlc/3TtHh70HPYAhw4d1K9BlLoKiAgLFy6ka0RgRkEBcH3NYCoE+rBg\nwQJL+tm9ezebtmyhd90QvOx5LxW0qFBKKaWUJbSoUG5RpXI4v8d8w4Fd2zLatq2PZcPqX2nUqJEb\nMyu+OnbsSEBgIPM/fDtjQJYjLY1FM6ZSqVI4zZs3d3OGSqnCZoyhV69e/G/XGRKT0zLaf9qdSMKZ\nZHr37m1JPzVr1qR+VBQLtp7igiPvA0B1TIVyiw0bNtC4SVNAaNr+Bhxpqaz7dSl2uwf79u0lLCzM\n3SkWSx9++CF33303VWtFUqthE7bE/s7RQweYO3cuffr0cXd6SqkisGPHDtq2aU3S2dM0D/PlRLKT\nuMNnGXLnncyYORNjjCX9LFu2jJu63UiAp6FmkCd/HDwLOqZCFUcNGzbkr7j11IqIYN2vP/LX78tp\n0KABu3fv0oIiByNGjGD58uW0ad6Eswd30bVTB1atWqUFhVJXkdq1a/PnuvXc/9BokkIjqdiwNbNm\nzeKjGTMsKyjAdXc0du2f9Ln9Ts4FV8lTjN6pUEoppVSO1q5dS7NmzUDvVCillFKqKGhRoZRSSilL\naFGhlFJKKUtoUaGUUkopS2hRodzmyJEjPPbYY9SJrEtUvfqMHTuWU6dOWd7P1q1bGT58ODUjatG4\nSVMmTZrEhQsXcoxZt24dtw8cSI2aEbRs2ZL33nsPh8NheW5KKVWaaFGh3CIhIYHWrdsw7cPp1Gjc\nivB6jZn05pu0a9eeM2fOWNZPXFwcLVq25Jvv/0e9tp3wq1CFJ558kv79b8l2R79ffvmF1q1bs3zl\nKq5pfyMElOW+++7j7rvvtiwvpZQqjTxyP0Qp602ePJmEY8eY+EUM5StVBuCmQSN4ZtBNfPjhhzzy\nyCOW9DN27LMElQ3lhVnf4OsfAMCaZT8w6dG7WbJkCV27dr0iZsyYJ6gWWZ+x73+Jp5c3AD/Nm837\nLz7BQw89RJMmTSzJTSmlShu9U6Hc4tvvFtOi000ZBQVAlVp1uaZVe7799jtL+hARvv9+MR37Dswo\nKACadehKxSrV+O67K/s5deoUq1b9Tudbh2QUFADte96Kf2AQixcvtiQ3pZQqjbSoUG7h7eVJStL5\nK9pTks/j5e1lWT+eXl6kJCVlahOnkwvJyXh7e19xvIeHB8aYK3JLS03FkZaGl5d1uSmlVGmjRYVy\niwEDBhD78//Ytv7v7brXrfiJTWt+Y8Ctt1rShzGGW2+5hR/nziIhfh/gunuxePaHnDyWwC233HJF\nTEBAAN263cT3n77PqWMJGTHzPniLlOQk+vXrZ0luSilVGuky3cotzp8/zw03dOW331YS1aQlaWmp\nbI9by80392DevK/x9PS0pJ+DBw8SHX0d8fHx1GvehlNHj7BvxxZGjx7NpEmTsozZtm0b7dq15/SZ\nM0Q1a03C/j0c3LuLCRMm8NRTT1mSl1JKlSR5XaZbiwrlNikpKXz66ad888032O12+vXrx6233oqH\nh7Xjh0+dOsWHH37Izz//THBwMHfccQddu3bNceOdhIQEpk2bxqpVqwgNDWX48OG0a9fO0ryUUqqk\n0KLiMlpUKKWUUgWjG4oppZRSqkhpUaGUUkopS2hRoZRSSilLaFGhlFJKKUvkq6gwxtQzxjxvjFlq\njNlpjDlkjIkzxsw0xgwyxly5mpDKksPhYNKkSURG1sU/IIDo6OtYtGiRu9MqsIMHDzJy5EjKh4YS\nUqYMd955J7t27XJ3WkqpUujs2bM8/fTTVK4URmBAADff3J3Vq1e7Oy1FHosKY0xTY8wS4E/gOmAV\n8CbwLPAJYICXgYPGmCe1uMjdqFGjGDNmDGGR19B35GhOpTjo1asXn3zyibtTy7cTJ04QHX0dX349\nj+iet9F5wHC+j/mR1m3asH//fnenp5QqRdLS0rip2428+cbrNAlMoW9tPzb9toz27drx+++/uzu9\nq16eppQaY3YDrwGfiUi2e1MbY9oADwNxIjLesiwtUJymlG7dupWoqCiGPfkiXW8bBrhWbXzryfs4\nuCWO3bt3Wb5WQ2EaP348L7z4Eq/OXUpoeBUATp88zhP9O3HXsCFMnjzZzRkqpUqL+fPn07dvX17q\nVJVrKvoDkOpw8sSPB4ho3IaYJUvcnGHpZPWU0kgReSenggJARH4TkdtxFSAqG8uWLcNms9Gxz+0Z\nbcYYOvcbxIED+9m5c6cbs8u/H5cu5Zo27TMKCoCgMuVodn03/QeulLLU0qVLqRzil1FQAHjabXSq\nHsBPy5Zxtay9VFzlqagQkdT8vGh+j7/aBAQE4HQ6OZt4MlN74onjGc+XJAEBAZxJz/1SZ04eIygw\nyA0ZKaVKq4CAAM6mpJHmzFw8nEp24O/nm+NKuarwFWj2hzGmhTHmCWPM68aYSZc+rE6wNOrRowf+\nAQHMeuMFLiS7dtA8fuQQCz6cQrt27alcuXIur1C8DB40iK3r17D8m7kZPyWsXb6EtcuXMHjwIDdn\np5QqTQYOHEhi0gVm/3UMR3phsetkMj/sOs3gO+50c3Yq38t0G2OeAV4CtgJHgEtfQESkk3XpWac4\njakAmDt3LoMGDcLbz59KVWuye+sGypcrz08/LSUqKsrd6eWL0+lk2PDhzPr4YypVq4ndw4MDu7Zb\nvjmYUkoBTJgwgWeeeYZyAd6E+Hiw89g5rmnYgJ+W/Uy5cuXcnV6pVGh7fxhjjgBPisiMf5RhEStu\nRQXArl27+Oijj4iPj+faa69l6NChhISEuDutAhERfvzxR77++mscDgc9e/ake/fu2Gy6FIpSynpr\n167lk08+ITExkQ4dOjBgwAB8fHzcnVapVZhFxSGgvYhs/2cpFq3iWFQopZRSJUFhbig2GXigoIkp\npZRSqnQqyGIIrwPfGmN2ApuATDM9RKSfFYkppZRSqmQpyJ2KKcD1wDbgOJB42SNfjDH3GmPWG2MS\n0x8rjTHdcji+gzHGednDYYypUIBzUUoppZRFCnKnYijQX0S+tSiH/cCTwHZcy30PAxYYYxqLyOZs\nYgSIBM5kNIgkWJSP+gfOnz+P0+nM11obZ8+exWaz4efnl+eYlJQU7HZ7sVt5VERITk7G29tbB6kq\npa46BfnUOwFYtuSjiHwrIt+LyE4R2SEiY4GzQOtcQo+KSMLFh1X5qIKJiYkhPLwy/v4BBAYGEhpa\ngTlz5uQYM2fOHEJDKxAYGIi/fwDh4eHExMTkGLNixQrat++Aj48Pfn5+DBw0iPj4eCtPpcA++ugj\nIiPr4ufnR/nQUJ5++mlSUlLcnZZSShWZghQV44DnjTF5/7Eyj4wxNmPM7YAf8FtOhwLrjDEHjTH/\nM8a0tToXlXdxcXF0734zqRjuePRZhj35Ip7+QQwcNCjbIiEmJoaBgwbh6R/EsCdf5I5HnyUVG927\n30xcXFyWMWvWrKFz584cPJ7IiGcm0P++x/nfjz/Rrl17Tp8+XZinmKupU6dy1113Ua5GJPe+MJk2\n3fszadJkBg++w615KaVUUSrIlNI/gVq4/mPfw5UDNfM9X9MY0xBXEeGD6yuNQSLyfTbHRgIdgDWA\nN3APcCfQUkTW5dCHTiktJO3atWP1mlje+mYlwWXLA5B07iyje11H5bAKbNq06YqY+vXrE384gckL\nf8XX3/VVSeKJYzzcoy2tWjRn+fLlV8T07dePNes38PJni/H0cm2Ee2T/Hh7vfz2TJ03iwQcfLMSz\nzN6FCxeoUrUqDdpcz8jnXs9oX/7NXN79z2jWr19Po0aN3JKbUkpZoTCnlM4H3sA1C2QusOCyR0Fs\nAa4FWgL/B3xsjMlyWUkR2SYi74vInyLyu4iMAFYCowvYt/qHNm7cROPo6zMKCgBf/wBadL6JPXv3\nZRmzZ+8+WnbpnlFQAASXLU/j6OvZsHFjljErV6ykZeebMwoKgIpVaxDZqBkrVqyw6Gzyb9euXRxN\nSCC6e+aJT21v7I3NZmPlypVuykwppYpWvke5icjzVichImnArvQ//mmMaYlrC/X78vgSq4HovBw4\nevRogoODM7UNHDiQgQMH5rErdTlfP18S4q8sHo7G78PbyyvLGG8vLxIOXBlz5MDebAdslilblqOH\nDmRqczqdHD8cT9lW7rv7dHEV1KMH92dqP5FwCKfTSdmyZd2RllJKFcjs2bOZPXt2prbExLxN7sx3\nUWGMaQHYRGTVZe2tAIeIrMnva2bBhuurjbxqDBzKy4GTJ0/Wrz8sNuKuu3jxxRf5dtZ7dBs4AmMM\nyxd9Sdxvyxk2bFiWMX369GbGjBksm/857XveitPp5IfPp7N360aeffbZLGOGDxvK2GefpVmHrrTo\n1I201At89d5kEg4eYOjQoYV4hjkLCwvjxhu7MW/aZGo1uJaqtaM4ffI408c/TZmyZenRo4fbclNK\nqfzK6gftS77+yJmI5OuB665A3yza+wGrCvB644F2QHWgITABSAM6pT8/AZh5yfEPA71wjetoALyJ\na1xHx1z6aQpIbGysKGs5HA5p1qyZAOIbECgBwSECSK1atSQ1NTXLmNTUVKlVq5YAEhAcIr4BgQJI\ns2bNxOFwZBmTkpIiPXv2EkDKhlYU/8AgAWTChAmFeXp5snfvXqldu44AUrFKNfH09BL/gACJiYlx\nd2pKKfWPxcbGCq7lHJpKDv/XFmSSf30gqwGRf6Y/l18VgJlAJVyLZ8UBXUVkafrzYUDVS473wjWm\nIxw4n358ZxG5cmSfKhI2m401a9YwZ84cpk6disPhYPjw4QwfPjzbtRo8PDzYtm0bH330ER999BF2\nu50HHniAAQMGZNuPl5cXCxbMZ9myZcTExODr68uAAQOoW7duYZ1anlWrVo0NG/7iq6++Yv369VSu\nXJlBgwZRvnz53IOVUqqUKMjsj+NADxH57bL2tsC3IlLGwvwso7M/lFJKqYIpzNkf/wMmGGMyRjsa\nY0JwfY2R88pFSimllCq1CvL1x+PAcmBv+poV4BooeQTXehFKKaWUugoVZEppvDGmETAY19oSScBH\nwGwRSc0xWCmllFKlVoF2YxKRc8A0i3O5Kp09e5bjx48THh6Op6dnofWze/duEhISaNasWaFuwrVx\n40ZSU1Np1KhRnjbUEhEOHz6M3W6nQoW8bTQrIhw8eBBvb+9CHQiZnJzMn3/+SdWqValSpUqh9VPa\nnDx5krNnz1K5cuVC3VQtISEBh8NBWFgYxphcjy/ItaaUyp88/Ys3xuS2udelx/oZYxoUPKWrw5kz\nZxgxYgTly5enRo0ahIdX5tVXXyW/A2dzs2rVKsIqVSIiIoLWrVvjHxBQKMtZf/rppwSHhNCwYUOa\nNGlCUHAwb7/9do4xy5cvp2nTZoSHh1OxYkWio69j7dpsx/8A8O2331K/fgOqVKlCaGgoXbp0YevW\nrVaeCgCDBw8mKCiYtm3bUq1aNapXr1Eo/ZQm+/fvp1fPHpQrV45q1apRp1bEFQvoWGHt2rVEt21D\nxYoVCQ8Pp1mTJlku636p5cuX07Rx47+vtbZtcr3WlFIFkNN8U/l7jYftwA/ArYB/NsfUxzVY8xAw\nJC+vW5QPitE6FU6nU7p06SJ+AYFy+0NPy5Nvfyw33DpEAHn55Zct6+fo0aPi7e0jZUIryl3PjJfH\nJn8oza+/UQB56qmnLOvn119/FZvdLlXrRMn9L02Rhya+I7WvaSrGZpO5c+dmGbN+/Xrx8fGRyGub\ny0MT35H7X5oiNeo2kKDgYNmzZ0+WMT///LPY7Xa5pnU7Gf36NBn53OsSXj1CKoaFybFjxyw7nyFD\nXH8X7XosdTxhAAAgAElEQVTcImOmzJQ7Hx8n/kEh4h8QICkpKZb1U5qcO3dOakfUlCrB/jKpU32Z\n3bOJ9KhdUQCZN2+eZf3s2bNHggIDJKKcn4xuU0nGRIdLvQr+4uPtJXFxcVnGrF+/Xny8vaReBX8Z\nEx0uo9tUkohyfhIUGJjttaaUyiyv61TkaUqpMcYT15LZDwARwDbgIJAMlAGigABgHjBeRP6ypOKx\nUHGaUrp69WpatWrF6Nen0aLTTRntM197jlXfz+PQwYP4+Pj8437uvvtuPpw+ndfmLqVyzdqAq4gc\nf+9A9m6O48wZa3b2bNGiBRu3bOXtb3/HLzAIgAspyTzapz1lAvzYsWPHFTHDhg1j8ZKlvPbVTxl7\neZw/c5rRva5j1D0jeO21166Iuemm7mzds58XZ32DzW4H4OTRwzzS8zpeevEFnnjiiX98Lmlpafj7\nB9Ds+ht5cMLUjPYtf67mhRH9ee655xg3btw/7qe0mTFjBnfddRer72xLZFnXfi4iQp/5azlbrip/\nxFpzV2DMmDFMmzqFd7tXx9/LdQ1ccDh58Pt93NhnADNmzrwiZtjQofwwfw7/7VYNT7vr5uy5Cw7u\n/W4vIx94KMtrTSmVmaVTSkUkVUSmiEhdoA3wPrABiAeWAaOAcBEZWBwLiuImNjYWm81Gsw5dM7W3\n7NSNUydPsnv3bkv6Wb16NZWq1cwoKACMMbTsfBNnz57B6XRa0s/Onbto1KZDRkEB4OXtQ7MOXTl4\n6HDWuf2xhmujO2XaHMwvMIj6LaNZsyY2y5jY2FiaduiaUVAAlAkNo06jpsTGZh2T/3PZyYULKbTs\n3D1Te1STlvgHBed6m/1qtWbNGqJCgzIKCnBdaz0iQon9c51lX+ut+WM115T3zigoALzsNppW9GH1\n6lVZxqxe9TtNK/pkFBQA/l52rinvzZo/VluSl1LKJd+jqERkjYi8KSKjReReERkrIl+JyInCSLA0\nCgsLw+l0cmhf5uLhwK7t2Gw2ywYfVqhQgeNHDpF8/twV/Xh4elo2iC4oKJD9O7Zc8R/HgR1b8fXN\n+o5LpbAwDu7enqlNRDi0eweVKoVlGROWRYwjLY3D+3YTFpZ1TH5VqlQJm81G/K7M/Zw6lsD5s2cI\nDw+3pJ/SJiwsjAOnkzh7IS1T+9YTZwmrEJqngZR5UalSOPHnHFdea2fSqFSpUtYx4eEcOJM5LxEh\n/pyDSpX071MpKxXe0GyVre7duxMWVolpzz/G4X27ERE2/rGSr9+bRJ8+fQkNDbWknxdffJHUlGTe\nG/cYiceP4nQ4WPn9fJZ+9Slt27SxpA+ABx98kIO7d/D52xNJOneWC8lJLJzxDpvX/s4dgwdnGTNy\n5D1s/GMlC2e8w4XkJJLOneXztyeyb8cW7rnnnixj7rnnbn6P+YYlc2eRlnqBs6dP8dHEsZw8eoS7\n7rrLknMJCgoiKiqKhTPeIfbnmPRdUA/yf/95BIPh5ZdftqSf0mbIkCEkO5z8a8lGEs6l4HAKc7cc\nYubGg9wz6l7L+rn7nnvYc+I8H68/yvlUBylpTr7adJy4w2cZmU0/I0fdS9zhs3y16TgpaU7Opzr4\neP1R9pw4zz0jR1qWm1KK/G8oVlIfFKOBmiIiq1atktAKFVybcPn7CyDNm7eQhIQES/t59NFHxdhs\nYowRTy9vASQ8vLIkJiZa2k+3bt0EEJvdLnYPTwGkRYsW2W4O5nQ6ZcyYMQKIp5e3eHh6is1mk4kT\nJ2bbR1pamgwfPlwA8fL2EbvdLp6enjJt2jRLzyU+Pl7Kli3nys3bRzBGbHa7vPLKK5b2U9p8+eWX\n4uvjIzZjxM/LdQ3069tHkpOTLe1n4sSJYrPZxMNuEy8PuwAyZswYcTqdWR5/6bXm5WEXD7st12tN\nKZWZpQM1S4PiNFDzoqSkJBYsWMCBAwdo3LgxnTp1KpR5/Tt37uTFF1/k5MmT9O/fnyFDhljeB7im\nr77xxhs4HA7uv/9+OnfunGvMjh07+O6777Db7fTq1YuqVavmGrNhwwZiYmLw8/OjT58+VKxY0Yr0\nM3E6nbz77rssXryYSpUqMW7cOP3qIw9OnDjBvHnzSExMpH379jRv3rxQ+tm/fz8LFy7E4XDQvXt3\nateunWtMQa41pZRLXgdqalGhlFJKqRwV5oZiSimllFJXKNB6zcaYzkBnoAKXFSYiYs2IOaWUUkqV\nKPkuKowxzwH/AdbgWj3z6vj+RCmllFI5KsidinuBYSIyy+pkVMkmImzatIm0tDQaNmyI/ZJFqrLj\ncDjYsGEDHh4e1K9f37L1DJS63IULF5g/fz7e3t707NmzUDc7Kwr79u1j2bJl1K9fv9AGxCqVXwX5\nV+UFrLQ6EVWy/fTTT9StG0XDhg1p3LgxERG1WLBgQY4xCxYsICKiFo0bN6Zhw4bUrRvFTz/9VEQZ\nq6vJM888Q6C/H7fddht9+vTB39eHSZMmuTutAklOTqZ58+bUrFGdoUOH0qJFCypVrMjmzZvdnZpS\nBSoqPgAGWZ2IKrm2bt1K9+7d8Qwqy1NTP2HstDmUq1ab/v37s2pV1ksnr1q1iv79+1O+eh3GTpvD\nU1M/wTOoLN1vvll3A1WW+vDDD5kwYQKNKvjwUqeqjOtYlYhgD8Y8/hiLFy92d3r51rFjR9bGxjKg\nQTneuLEGD7YM49yp47Rq2cKypfeVKqi8bih2aUlvA4YCcemP1EuPFZFHrUzQKjqltPA8+OCDfPr5\nHCYv/BUvH1/AtXz2UwO6EN2qOV98/vkVMbfdfjsrVscy8YsY7B6ub+EuJCcxutd1DL59QK7bpiuV\nV9WrVSP15GGm3hyB3eb6ei0lzcndC3dSPbI+cXFxbs4w7xISEqgUVpF+9cpx57V/r7z756FzjFu2\nnylTpvDggw+6MUNVWlk9pbTJJY9rgXWAE2h42XNN/kHOqoT6668N1G3aKqOgALB7eNCgVTv++mtD\ntjENW7XLKCgAvHx8qdu0VbYxShXEsYQjNA0PyCgoALw9bDSq6MeBffvcmFn+rV69GqdA00r+mdob\nh/lhgJUr9Ztp5V55GqgpItcXdiKq5KpRozoxy37B6XRmDH4TEXZvWk/t6tWzjKlerRo7N63P1OZ0\nOtm7ZQNdr29f6Dmrq4d/YCDbjmXeVM8pwvbjyZQNq+amrAqmQYMGGGDHiWQaVPDLaN91MgUB6tat\n67bclIICjKkwxkw3xgRm0e5vjJluTVqqJLn33ns5uHcXH7z0JCcSDpF44hifvfkS2//6k/vvvy/L\nmAceuJ/tcWv5dPKLJJ44xvEjh/jgpSc5vH8P995r3QZUSt173/1sP5HMR38mcCo5jWPnU5m6+jBH\nzqXy1FNPuTu9fKlZsyY1IyL47K+j/LrvNClpTrYcS2LSbwfx8vQoceejSqGcNgbJ6gE4gApZtJcH\n0vL7ekX1oJhtKFbavP/+++Lr53dxwxnx8vKSV199NceYV199Vby8vDJifP385P333y+ijNXVpHv3\n7mLSrzNAbAYZMmSIu9MqkP3790to+XIZ5wKIt6enLFq0yN2pqVLM8g3FjDFBgAFOAnWAo5c8bQd6\nAhNFpFjuuqQDNQtfYmIiP/zwAw6HgxtuuIHy5cvnGnP06FGWLFmC3W7nxhtvJDg4uAgyVVejzZs3\n88477+Dh4cHo0aOpVq1kffVxuTlz5rBkyRLq1KnDww8/jJeXl7tTUqWY5RuKGWOc5Lx6pgDPicjL\n+Um0qGhRoZRSShVMXouK/KyoeT2uOxVLgf7AiUueuwDsFZGDBchVKaWUUqVAnosKEfkZwBhTE9gn\neb3FoZRSSqmrQp6KCmNMo8uarslujwYRKTkrySillFLKMnm9U7EO15gJQ+67kua+i1QJsHPnTrZt\n20bNmjWJiooqlD5EhL/++ov4+HgaNmxI1apVc41xOp188skn7Nu3j169etGo0eX1nvukpKSwcuVK\nHA4Hbdu2xc/PL/cgVWyJCLGxsRw7dozGjRsTFhbm7pSK3IEDB5g1axaBgYHcfffd+Pj45Bpz/Phx\n1qxZQ1BQEK1atSpWG5cV5HNt8+bN7Nmzh8jISGrVqpXr8QX5XCsIp9PJqlWrOH36NC1atKBs2bKF\n0o/Kp5ymhsjf0zGrX/LoA+wARgGN0h+jgG1An7y8njse5HFKaWJiovTu3SfTdK3OnTtLQkJC7nNu\n8mHPnj3SqlXrv6e42Wxy55AhkpSUlG3MokWLMk3bNMZIo0aNcowpKl999ZWEVqiQkVtwSIhMnz7d\n3WmpAtq0aZNc06B+xt+nh4dd/vWvf0lqaqq7UysyvXr1Epv5+3PA08Oe4zRpp9MpY8eOFR/vv6dJ\n16pZU1avXl2EWWctMTFRevfqlflzrdP1OX6uHTlyRDp3uj5TTO9evSQxMTHbmD179kjrli0yfa4N\nufNOyz+jVq1aJbVq1szox8fbS8aOHStOp9PSftTf8jqltCD/Oa8GumfR3h2Ize/rFdUjr0XFLbfc\nKv6BQTJq3Bvy9ner5KGJ70hIuVDp0KFj3t/9XDgcDmnQoKFUqFxVHps8XaZ8+7sMe/JF8fL2kfvu\nuy/LmJMnT4qnl5eE16glT7/zqbz1zUq5/aGnxdhs0rGjdbkVxLp168TDw0NaXN9Nxn+2WF79com0\n63GLALJ06VK35qbyLykpSapWDpd6oUEyr28ziRveXsZF1xG7zSbjxo1zd3pFYvTo0QLIzZFlZOrN\nNeXVG6rLNRX8xBjkl19+yTJm6tSpAsiABuXk3R4RMr5zNakb6i8hwUFy/PjxIj6DzG65pb8EeHvK\nQ63C5INetWRMdLiU8feWDu3bZRvTvt11UsbPW8ZEh8sHvWrJQ63CJMDbU265pX+WxzscDmlYv55U\nDwmQz3s1kQ13tZfXr68nPp4e2X6uFcSxY8ckJDhI6ob6y/jO1eTdHhFyawPXuh1Tp061rB+VmeXr\nVFxkjElKf9HNl7XXA9aKiG/Wke6Vlyml+/fvp3r16oz490Q69ft7I9Y/li5m8uMjWbduHddee+0/\nziUmJoauXbvy3PSvqdu4RUb7vA/eYtH0/3LkyBGCgoIyxTzwwAO88847vP71MsJr/H0LcuZrz7Hk\ny485e+ZMnm7NFoaRI0cy/5vveGP+L3h4egKuYvXfg7pxbVQk8+fPc0teqmA+++wzBg8ezJoh11Gn\n7N97TDy5bDNz953h0JEEPDzyM3Gs5AkOCqSGn5MXO/29lkVSqpNh87fTtGUbVqxYcUVMnVoRhDuO\n81jbv5fqOZmUxt2LdvH6G5N4+OGHiyT3y138XLu/eUW61g7JaP9t/xkm/hqf5efa+vXrady4MU+3\nq0zrKn8voPzDjlP835oj7Nu3jypVqmSKufi59sOAlrQOL5PR/tqqnby+dj9HEhKu+FwriLfeeosx\njz3KBz0jCPH9+zp8Y+VBDtrLsX3nrn/ch7qS1RuKXWoz8LQxJmOllfTfP53+XIm1a9cuRISopq0y\ntddr1hqA7du3W9LPjh07sNntRF7bPHM/TVuTnJxMfHz8FTGbNm0iMKRspoLiYowjLY29e/dakltB\nbNu2ndqNmmUUFADGGOo2acU2i94zVXS2b99OxUC/TAUFQHSVshw7cZJTp065KbOic/78eRpWyDwm\nyNfTRu2yvuzfv/+K40WEnbv30KBC5p+pyvh6UDXE17LPjoK4+LlW/7LcLp5fVrldbGsQmvk9aFDB\nFxFh164r/+PesWMHdpuhVaWQTO3RVcqSnJKS5edaQWzfvp0qIb6ZCgqA+qG+7Ny9h/z+oKysVZCi\n4l7gRuCAMWaJMWYJcCC9rURv2hAREYExhi1rV2Vq3xz7OwB16tSxpJ/atWvjdDjYtn5N5n7W/o6P\njw+VK1e+IqZ+/fqcOXWCg3t2XhFj9/CgejYbdxWFyMg67IiLJS01NaNNRNj65yoiLXrPVNGpU6cO\nR86cZ/uJzJtwrThwgvJlyxASEpJNZOnh5+fHhoTzmdqSUp3sOJGU5cBDYwy1atZgY0JSpvaTSWns\nP5Vk2WdHQVz8XNt0WW4Xzy+r3C62bTya+T3YmJCEMYaIiIgrYmrXro3DKaw6lLnoXHHgBD7e3ll+\nrhVEnTp1OHAqiZNJaZnaNx1NolbNGmQ3M1EVkZy+G8nuAfgDI4FJ6Y97AP+CvFZRPSjgmIoHJ0wt\n5DEVH8qUb3+XoU+U/DEVza+/UcZ/tlhemROjYypKMB1T8c/GVNx6yZiKyPLFc0zF423zOKbC31se\nb5v/MRWze7rGVLzWMcryMRXHjx/XMRVuUGgDNUvqI69FRWJiovTp07fQZ3/s3bu3VM/+CClTRmd/\nlGA6+6P0zf7o07t3vmZ/JCQkXDH7o0/v3jnO/ti7d2+RzP5YvXq1zv4oYpYO1DTG9AIWi0hq+u9z\nuvOxMNcXdIP87v1RFOtUAMTFxZWadSouXLjAihUrdJ2KUkJE16n4J+tUBAcH07JlyxK/TsWWLVvY\nvXs3devWzfJrj6zk93OtIHSdiqJl6YZi6ZuJhYlIQvrvsyMiUiwXv9INxZRSSqmCsXRDMRGxZfV7\npZRSSqmL8l0gGGPcsxiCUkoppYq1gqxgc8oYsxr4GVgGrBSRpJxDlFJKKVXaFeSrjC7A90ArYAFw\n0hjzqzHmZWPMDZZmV8o5HA6WLFnCzJkzWb9+faH1k5yczMKFC5k1axa7d+/OU8zu3bvp06cPXbp0\n4bvvviu03JQqzuLj4/n000/5+uuvOXfuXO4BRejVV1+lQ4cO3HPPPSQlFc7PdQ6Hg0ceeYQOHTrw\n7LPP4nA4CqUfVYrkNDUktweuOx1tgBlAKuD4J69XmA/yOKW0qGzcuFFq166TabpWt27dcpyuVRAx\nMTFSPjQ00zTUkSNHSlpaWrYxvXr1EpvN/nduxkh4eHiOMUqVJk6nU5566imx220Z/w6CAgNl7ty5\n7k5NNm3aJD5enpk+O+w2I++8846l/cydO1c8bCZTP54edvn1118t7UeVDIW6TgUQiWvxq8+Ag8Bx\nYB7wcEFerygexamoSE1NlRo1akq12lHywsyFMuO37fLQK/8nfgGBcueQIZb1c+jQIfHz95dGbdrL\na1/9JB8s3yRDxjwvNptNXnnllSxjpk2bJhgjLTt3lzcXrZBpy/6SW+57TAC57rrrLMtNqeJs+vTp\nAsjga8rLp/3ryLSeEdK2WpB42O2yZcsWt+YWGBAgPh5GnogOly8HRMqrN1SXKkFeYrcZOXPmjCV9\npKWliYfdJuX9POSlTlVl7oBIebZ9FQn0souPl6clfaiSJa9FRUEGasYDvwPd0n+9CSgvIn1F5K38\nvt7V6IcffmDPnt3c+8Jkal/TBC9vH1rf0IO+Ix/h89mzOXnypCX9fPzxxzgcTv41YSqVa9bGLyCQ\nbgPvon3PW5k69Z0sY5599ln8A4N44OUpVKhcjYCgEPrd8whN2nXm91WrLclLqeJu6ttTaFUlkAEN\nyxPgZadigBejW4fh72Xngw8+cFte69at4+zZswy6JpToakF42W3ULe/Lo23CcTiFkSNHWtLPuHHj\nSHM4eaBlGNdU9MfTbqN55QBGNK1A8oVUvvjiC0v6UaVPQcZUHAX8gLD0R0WgWO5MWlzt27cPm81G\n9boNMrVH1GtEamoqhw8ftqyfsKo1CAjKvFdDRINr2b9/38U7OJmcPn2a6pH18fTyztReq0FjRHJa\nokSp0mPfvn3UKpP534CX3Ub1YC/27dvnpqzgt99+Q4DaZTNPwoso440Btm3bZkk/69atA6BO2cwf\n7XXKufr99ddfLelHlT75LipEpDGuYmIi4A2MB44ZY1YaY162OL9SqUGDBjidTjas+iVT+/qVy/AP\nCKBatWpZB+ZTw4YNObBrG0cPHsjUHrdyGfXrN8hy452KFSuy/a+1nE38+26JiLBuxVLsxWhlQKUK\nU4OGDfnzSFKmwvtMioPtJ5Jp0KBBDpGFq1u3btgMrD2UedDousPnESA6OtqSfrp37w7A2kNnM7Wv\nPXgOA/Tt29eSflTpk6cVNbMNNqYc0BHoDQwEbKIrauZKRGjTpi1bd+zg1geepHpkfdb+HMOC6W8z\nZswYJkyYYEk/p0+fJiqqHnZff26573HKhIaxbMEX/DTvM2bNmsUdd9xxRUxMTAzdut1Etch69B/1\nKP5BwcR8MZPf/reQO+64g1mzZlmSm1LF2eLFi+nevTvtqgfRvU4IZy84mLPpJAkX7GzespVKlSq5\nLbfw8HAOHzrEgIblaFk5kN0nk5m5LoFkBySlXMBut+Yj2NfHGxypDLm2AlHlfVl/+Byf/nUU/4Ag\nTiUmWtKHKjnyuqJmQQY89gOmAHFAGpAAfA08BFyb39crqgfFaKCmiGuznp49e4kxrtHVPr6+8thj\nj1m+YdPmzZszbVxWtlw5mTJlSo4x48aNEw/Pv0eX2+0e0qFDB0vzUqq4mzlzplSs8PfMqUYNG8of\nf/zh7rTkzJkzUrFiBTGXzMrw8fa2fFbGpk2bJMD/kg0MQUJCQuTQoUOW9qNKBks3FLuUMSYBWI5r\n4aufReSvfL2AmxSnOxWXio+P59ChQ9SpU4fg4OBC62fnzp0kJiZSv379PG2K5HA4mD59OseOHeP+\n++8v1NyUKq5SU1PZsGEDfn5+REZGZvmVobts2LCBr776ihYtWmR8XVEYli1bxs8//8xNN91Ey5Yt\nC60fVbxZuqFYaVBciwqllFKquMtrUaEj75RSSillCS0qlFJKKWUJtxcVxph7jTHrjTGJ6Y+Vxphu\nucR0NMbEGmOSjTHbjDFDiypfpZRSSmWtILuUWm0/8CSwHTDAMGCBMaaxiGy+/GBjTA3gG+AdYBCu\nDc4+MMYcFJGY3DpLTk7ONaG0tDS+//57tmzZQs2aNenZsydeXl55P6NiZufOnbzwwgucOnWK/v37\nM2TIkFxjDh8+zPz580lKSuKGG26gYcOGucbs37+fhQsXkpaWxs0330zt2rWtSN8tRIQVK1awatUq\nQkND6du3L4GBgZb3k5yczPjx41m7di316tXjueeeIyAgIMeYCxcusGjRInbv3k1UVBTdunXDw8P6\nf8oHDx5k3LhxHDp0iJtuuol7770XWy5rlZw4cYKvv/6a06dP0759e5o3b55rP3FxcUycOJHz588z\ndOjQEr8GwvPPP8+XX35JUFAQb7/99sXvobNVkGstLS2NSZMmsXz5cmrWrMnzzz9P2bJlc40prp9r\nZ86cYd68eRw9epRWrVoRHR2d66DYglxrxdmGDRuIiYnB19eXvn37UrFixVxj1qxZw/LlywkKCqJf\nv365XgNFIqepIe564NpLZHg2z70CxF3WNhv4LpfXbApISJkysmrVqmynzezdu1eiouoJIH7+AQJI\ntWrVZdOmTXmceFO8PProo2JsNjHGiKeXtwASHh6e48Zl06ZNE09PT7F7eIiXt48AMnz48Bw3FJs4\ncaLYbDbx8PTM6GfMmDHidDoL47QK1enTp6VTx44CiL+XpxhjJDgoUGJiYiztZ+XKleLr7eW61jxc\nG1d5e3jIokWLso3ZuHGjVK9aRQAJ9HHF1qsbKXv37rU0t1deeUVs6dOdveyuX8uGhEh8fHy2MV9+\n+aX4+niLzRjx8fQQQPr06S3JycnZxtx5551iQGyGjM2r6tSpIykpKZaeT1FISEgQXx/Xte9pM2LS\np2H26NEj25iCXGuXTvW8+HfjYbPJ9OnTs43Zu3evREVGuq41b9d08WpVqxSLz7WYmBgJCgwUYxBf\nL9d1c33HDnL69OlsYwpyrRVXaWlpMnz4cNffp4dd7DabeHp4yLRp07KNSU5Olj69e7umE3t6iM0Y\n8fXxli+//LLQ8rR0QzFc61Dk6ZGX18uhHxtwO5AERGVzzM/ApMvahgEnc3ntpoBUrR0lYWGVsr34\nrruunVSoXFVe+HihfLZ2v7wyJ0aq1qorUVH1xOFw/OO/mKK0cuVKMcZIqy43y/8t+VM++WOP/Gv8\n22K3e0j79u2zjFm3bp0YY6RTv8Eybdlf8vGqnTLimQlijJG33347y5gff/xRAOk57H6Z/usWmbFy\nm9z+0NMCyOeff16Yp1goRo0aJYHeXvJFryZy8uGusnFEe+lcI1QCA/zl5MmTlvVTJihIKgf4yNLb\nW0niIzfKb3e0lcgy/uLr7ZXleiUOh0Pq1Y2U+qHB8tsdbSXxkRtl6e2tpHpIgLS/LtqyvPbs2SM2\ngzQO85MPe9eSebfXlbHtq4i33Uj9evWyjNm7d694enjIddWCZGbf2vL1bXXlsTbh4mm3yX/+858s\nY7744gsB5MZaIfJp/zoyd0Bdua9FRQHklltusex8ikrVqlXFbpDH2oTL17fVlZl9a0t01UAxIF9/\n/XWWMQW51iqHV5JAL7u81KmqzL+9rrzbI0LqlPURD7st2x8WrotuK2FBPvJa1+qyYGCUTLmpplQv\n4ytRkZFu/Vw7efKkBPj7S9PwgEzXmp+Xh4waNSrLmIJca8XZlClTxGaM3Neionx1W135tH8dubFW\niBhjZN26dVnGPPvss+Jpt2W61q6rFiRenp6W/4BxkdUbiiXm45FvxpiGxpgzQAqurzX6isiWbA4P\nA45c1nYECDLGeGdxfCb9Rj7C4cOH+O677654buvWrfz66y8MemQstRs2AaBq7SiGPfUSW7ZsZuXK\nlfk4K/d79tln8fT2YdS4NwguWx6b3U7bbn3o1H8wK3/7LcuY6dOnUya0IsOfeomAoBA8PL3ofMsd\ntOpyM9OmvZ9lzAcffEC12lHc/uBT+Pj54+XjS69h99OgRdtsY4qrlJQUZn08kwebVKVbRAVsxlAl\n0Jf/u6EB588nMWfOHEv6Wbx4MSdPn2ZCh7o0C3PtzVK/fCBvdq5PUsoFpk2bdkXMypUr2bx1G693\niKR+edft8WZhIbwQXYvlv66wbN+Hf//734jAI63DKe/nic0YWlQOoF/9cmzZspnTp09fEfPxxx/j\naSJz4qsAACAASURBVDf8q2UYIT4e2G2G9jWCuCEiiGnvvptlP+PHjyfY286o5hUJ8LLjaTd0q12G\n6KqBfPfNIkvOpSgdPLCfbrVDaF8jCLvNEOLjwYOtKuFpNzz++ONXHF+Qa23z5s3EHzzE4Ebluaai\nP8YYKgV68XDrSqQ5nLz88pU7JWzdupVfV6xkaKOyRJZz7eXx/+3dd3hUVfrA8e+ZSZn0RkgjCaET\nQAhN6UUFQZqoKAhiV3Z1wbaWddddXV32hy67a1msoCgIKgi6KrqoWAGpKkWkdwgEQiA98/7+mElM\nmYRkmGSS8H6e5z4yd+57z7l3jpN37j3nnuRwf27vGs3Wbdu8+r22cOFCcnKy+V3P2DJtbXTbcOa+\n/hp5eXkVYtxpa/XZSy/Mok9SCJe1isDHYgj2c/w/ERnkx6uvvlpJzAsMaRFapq3d2TMWqxFef/31\nOj6Csqp1I1ZEbqzlemwFOgNhwFXA68aY/lUkFm77eL7jQ3r00UeZPXs2AOPHj2f8+PEcOeLIVeJT\nyvYFSGjRBsBjE33VlaNHjxIVE4ctMKjM+mYtWlNYUIDdbq9wj/zw4cPEJqVgLXePPqFFG77c4HqW\n0kOHDhOX0qrCPdD4lNbs/2mNB46k7mRlZZGdk0vbqLL9GmKC/IkI9PdYG9i+fTsAbSPLltPOWe6O\nHTsqxBSX3a5c3do593H48GHatGlzznU7ePAgQX4WIgLKtoHEUD/sAocOHSI0NLRC3aKD/AjwLdue\nEsP8+Wj7UceT9sq1j4yMDJqF+mG1lF2fFObPqoNl57ao74qKiigSaBZW9ndNgK+F6EAfTp48WSHG\nnbZW3C4Sy5UTH+KHAZeTnRV/rzULLRuTGOboT+HN77XDhw8TYvNz2dayc3LJysrC39+/QkxN21p9\ndvjwYTrGle3bYrUY4oN9XX42IsKR9HSaJTUtsz7A10LTYM98R82fP5/58+eXWZdZzUeze330B4CI\nFIrIThFZLyJ/ADYCUyvZ/DCOmVFLiwFOiUjFtLacbgMuBRy/rpcuXcrSpUsZP3484Jjoy9/fnzWf\nfVwm5vvlH2KMaXAPzerZsyeH9u7iwK7tJetEhNXLPyI4OMRlp7tu3brxyw/rOJH+68Uge1ER61Z8\nUmmHs+7du7F59TdkZ/36CzY/L5eN33xGzx4Nq/NUZGQkzZMSeX/70TLrvztwgmOnc87a6a66iieG\nen9H2YtuS35xvB45cmSFmOL2t3R72Zj3tx/B38/PYxNd9e/fn9P5drakZ5dZ/+2+LPx8rLRs2bJC\nTPfu3dl74gz7T/36v6CIsOrAGbp26eLyS75jx45sPZZDRk5hyboiu/Dd/iwiIupBh7MasFqt+Fgt\nfLcvq/h2KwD7MvM4kFXg8rNxp6317dsXq8Xw3b6sMutXHziNAEOGDKkQ06FDB/z9fFm5v2zMt/uy\nvP691q1bN05m51Voa9/tP01yYqLLjofutLX6rFv3Hqw+lE2R/dd2czy7gJ+PuW4Dxhi6dunCqgNn\nKrS1vSfOeKTD6vjx40v+PhYvM2fOrF5wVfdGKltwXE1YCKwE1pVe3Nmfi/0vB16t5L3pwMZy6+ZR\nzY6atoBAueSSSyrtQDh16lSxWq1y+aTb5YFnXpext04TP3+bTLjuuhrcfaof0tPTxd/fJhHRMXLT\nw0/KvTNfke6DhgogDz74oMuYY8eOSUxsrMQnt5Db/jRD7n7qRel0UT+xWq2yYsUKlzG7d++W0LAw\nad62g/zm8X/J76Y/L206dxebzSY//PBDbR5irXjppZcEkGvbx8vC0V3lb/3bSpMgm3RL61JlZ9Wa\n6tKli1gMcle35vLumK7ywIUtxc9iJKV5cqUx100YLzZfH3ngwpby7piucle35mK1WGTq1Kkeq1de\nXp4EBwZIsJ9Fbk5rKn/s30wGNg8VQK6//nqXMWfOnJGWKSkSHWyTO7rHyMP9EuSiZiECyOLFi13G\nbNu2TXysFokN9pW7esbKg30TpHOMowPi2eanqY/Gjh0rgFzYLFge7pcgd3SPkcgAH7FajOzfv99l\njDtt7dJLLxVAhrUKlz8NaCaTOkeLzcdIVGREpf0jpk6dKlaLkTHtIuXRAc3kmo5R4udjlQnjx3vs\n+N1RWFgoXdO6SHigX6m2FiaAvPTSSy5j3Glr9dmKFSvEarVIl7hgebBvgtzVM1YSwgIkpmm0HDt2\nzGXM4sWLBZCLmoWUtLXoYJu0TEmR7OzsWqmnRztqStk/zr8DsoBncPSBmAV8CpwEnnBjf08C/YBk\noCPwNxwTlQ12vv834LVS2zd3lv93oC3wGyAfuOQs5XQFZNSoUVX2Ki4oKJBHHnlEwsLDnSNAgmTq\n1KmSk5Nz7p+KF6xcuVJiY2NLJgXy97fJnXfeWWXM1q1b5eKLLy6Jad8+VT744IMqY9auXSu9e/cp\niUlL61ppEtIQzJo1SxLiHOfNx8cq115zjaSnp3u0jJycHOnbt2/JqAerMdKlS5cqO4Pm5OTI1KlT\nJSgwQAAJDw2VRx55xOMT0W3dulWSkhJLJq3ytVplwoQJVcbs3btXRo4cUTJJXovmyTJv3rwqYz78\n8EOJiogoaTcB/n7y6KOPevBI6taQIUPE6vw8cU70tWTJkipjatrWioqKZNiwYSXtxoC0ad26ypE5\nJd9roY4/voEBAfXmey09PV2uueYa8bFaBZCEuDiZNWtWlTHutLX67IMPPpD2bduWtJtLLr5Ytm7d\nWmXMvHnzJCU52dEGjJGRI0fI3r17a62OtTmh2FbgLyIy39m5srOI7DTGPAZEisidNdzfy8BgIA5H\nR88fgOki8pnz/dlAsogMLhXTH5gJpAL7gcdEpMo5uWs690deXp7j3l10NIGBgTU5pHpp165dHD16\nlG7dulX7mQbHjx8nNzeX+Pj4al9SPHr0KEVFRcTGxja4y5DlFRYWcvDgQcLDwyv0IfCkU6dO8eOP\nP9K2bVuaNGlSrZjs7GzS09OJjY2tcM/Zk/bv38++fftIS0ur1kR0ACdOnOD06dMkJCSc9bkWxX75\n5RcyMzPp2rVrtWPqq/z8fD7++GOSkpLo0qVLtWLcaWvZ2dls3LiRlJQUYmNjqxVTn7/XTp06xcmT\nJ4mPj6/2d5Q7ba2+EhEOHjyIzWYjKiqqWjF2u50DBw4QHBxMRERErdav1iYUM8ZkA+1FZI9zxtJL\nRWSjMaY1sFJEqnc26phOKKaUUkq5pzYnFDsMFPee2Qtc5Px3Co4nYiqllFLqPOROUvEZMMr579nA\nTGPMp8ACYLGnKqaUUkqphsWdCQNuw5mMiMhzxpjjQG9gKfCCB+umlFJKqQakxkmFiNgBe6nXbwFv\nebJS3nbmzBkWLlxYMvHO+PHjCQsL83a1VCPkTlvLzMxk/vz5JROKjRs3jqCgoCpj6squXbt46623\nSiZ5Gjp06Fk70G3dupW33367ZPK6gQMHnrWT74YNG1i8eDFFRUWMGDGCCy+8sMoYEWHVqlV88MEH\nWK1Wxo4dS+fOnassQ0T44osvSiZ5GjduHG3btq0yxh0FBQW8//77rF69miZNmjBhwgTi4+M9Xo5y\nr62pGqpqaEhlCxAB3Ae84lzuxTHy45yfUVFbC84hpWvXrq1y2MzmzZslPj5BjDESm5gsVqtVIiIj\nq5yETCl3bN68WRLiYsUYIymRIWK1WCQyPLzKtrZq1SqJDA8Xq8UiKZEhYoyRhLjYejEx1AsvvCAW\ni0WC/X0lISxIAOnft69kZWVVGvPkk08KIGE2f4kNcTyjYtTIEZVOKGa32+Wee+4RQCICbRId7Bha\nO/n66yt9RkNRUZFMvv56Z4y/hAY4JmK75557Kn1eTV5enowaOUIAiQ0JlDDnRGFPPvlkzU9MFdLT\n0yWt8wWOCb7CgyXA10f8fH1rdWKo81VxWwv295WoYMfnOXJE5W1NlVWbz6noj+OZFHv5dSKxPTiG\ng/av6f7qaqlOUmG32yUtras0a9lGZi75Suat2yfPfrxaWl/QVZKTm3v0wUfq/Ga326Vrl87Srkmo\nrL+hn2ROGypbbhkgPeIjpHlSosu2VlhYKM2TEqVHfIRsuWWAZE4bKutv6CftmoRKt7QuXp0Rdtu2\nbWKxWOSmToly8LcXy8mpQ+S9sd0lyM9X7r33Xpcx3333nQByb48WcvTOS+Xk1CEyd0QX8bVaZfr0\n6S5j3n//fQHkr/3ayvHfXSonpg6R5y7tKIC88sorLmNeeeUVAeSuC2Nl8bVt5d1r2sqNXaIFqHRG\n2OnTp4uv1SpzR3SRk1OHyNE7L5V7eqQIICtXrnTvJLkwaeJEiQqyyfJrHJPK7Z0yWMa2jRObv78c\nPXrUY+Wc74rb2lWpUfLOuDby3rVt5cG+CeJjtVTa1lRZnp5QrLTncDxNM0VExorIWKAFjlsgz7mx\nv3pj06ZNrF+/jvG/e5iYxOYARDaNY9K9f2bPnt18+eWX3q2gajQ2bdrEug0beaxPK1qEO54XEB9s\nY3q/Nuzeu89lW/vyyy/ZvXcff+/fhvhgxzMjWoQH8pfeLVm7fgObN2+u02Mo7Y033iDM5svfBrQl\nyNcHYwyDkqK4sWMCr812PSnS66+/TnJ4MI/0boW/jwVjDKNaxXBVmxheq2QipTmzZ5MWF8Fd3Zrj\nY7FgMYaJHRK4pHl0peXMmf0q3RJCuKRFOBZj8LEYxrSPolWTIF6bM8dlzGuvvspVbWIY1SoGYwz+\nPhb+2Ls1SeFBvPbaa26do/JycnJYsGABv0tLonucY1K5MH9fnh7YjqLCAo9NXqccbS021MZ1FzTB\n1+poa70SQ+iXFMKcV1/xdvUaFXeSilbA0yJSVLzC+e9/ON9rsDIyMgBoEpdQZn10fLMy7yt1rorb\nUmJI2QdKJYUGlHnfVUzxNtWJqSsZGRk0DbJh87GWWZ8caiPjZGbx1cIKMc2C/bGUu6edGGoj44Tr\nY8nIOE5ikF+F9UkhNjKOH3cZc/z4MZoGWCusjw6wcLySmIwTGRXOs8UYmgXbPHaes7OzyS8oqFBO\nhM2XEJufft94UEZGBk0CfCq0tehAHz3PHuZOUrEOaO9ifXscE4E1WJ07dyYgMJCvPyw7MvbrDxdh\ntVq58MILvVQz1dh07tyZwAAbC7YeKrN+wdZDWK0Wl23twgsvxGKxuIwJCgzgggsuqNU6V6VPnz78\nnJ7J+iO/zmRYZBfe3naUPr0uctkZrnfv3qw6dIKdJ3+dTCqv0M5724/Rp28/1+X07cfn+06Qnv3r\nZFJZ+YV8uPs4ffoPcBnTr/9AVh/KIbug5HcQJ3ML2Xg0h779XJfTu09f3tuRTl5hSZ90dp7MZvWh\nE/Tp06eSs1AzkZGRtG3dioVbD5VJupbvOUbGmVyPlaMcbW1LejaHsvJL1hUU2fnuQDZ9KmkDyk1V\n3RtxtQDX4OhDcR/Q17ncB+xyvndB8VLTfdfmQjU7aj766KOODmYjr5Ipj/1TLr5qolitVpkyZUoN\n7j4pdXbFbW1Cary8MLST3NQpUawWS5Vt7Y477hCrs+/CC0M7yYTUeAG8Pl9GXl6edO7UUSIDbfLw\nRS3lmUs6SJ9mUWKxWOTTTz91GZOZmSktmidLXEig/LlPa5k5OFW6xIaLv5+frFmzxmXMoUOHJCa6\niTSPCJYn+7eVGQPbSbsmoRIaEizbtm1zGbNt2zYJCQ6WpIgAua1bjNyU1lRiQ23SNLqJHDp0yGXM\nmjVrxN/PT9Jiw2Xm4FT5c5/WEhcaKC2aJ0tmZqZ7J8mFt956yzHXQ/Noef7SjnJvjxYS7O8rA/v3\n92ofmcYmMzNTUpKTJSrIX67vHC1TesRIq6jAKtuaKqs2O2raz7IUFf+3pvuuzaW6SYXdbpd//etf\nkpiYJIDExMTKX//6V+2kqTyuuK0lNUtwjDJoGn3WtlZYWCh//etfJbapo6NhUrME+de//lUv/gAd\nO3ZMbrrpJgmw2QSQnt27ybJly6qM2bdvn0wYP178fH1LRot8/fXXVcZs27ZNxoweLVarVYwxctmQ\nIbJhw4YqYzZs2CBDhwwRY4xYrVYZPXpUpUlIsa+//lr69+0rgPj5+sqE8eMrnW30XLzzzjvSqUOq\nABISHCR33XVXlSNmlHvKt7V+ffucta2pX9XmhGLJNbgKsqdGO69FNZ37Q0TIzc3FZrPpOGZVq9xp\na/W5fRYVFVFQUFDtCcjAMaFWUVFRjSZHKygoQETw86vYx6Iy+fn5GGPw9fWtdkxeXh5Wq7Xak1y5\nKzc3Fz8/vwY/MVZ9505bU9Wf+8Odh1/Vm0ShNhljCAgIOPuGSp0jd9pafW6fVqsVq7Vix8iq+Pj4\n1PiPdk0Sg2I1SUCK1dUfn5okYcp97rQ1VX1upcTGmEnGmG+MMQeLr1wYY6YZY0Z7tnpKKaWUaihq\nnFQYY6bgGD76IRAOFP8kOQlM81zVlFJKKdWQuHOl4i7gVhF5AkenzGJrgE4eqZVSSimlGhx3biyl\nAOtdrM8D6sesRko1EEePHmXOnDls2bKFFi1acOONN9KsWbMqY77//nsefPBB9u7dS+vWrZkxYwYd\nOnSoMmbnzp28+uqrHDhwgC5dujB58mTCw8M9eShu+89//sOsWbPIy8tj2LBhPPHEEwQGBla6vYiw\nfPlyFi1aVDKh2OWXX15lB8eioiI+/PDDMhOKXXzxxVV2cs3OzuYPf/gDH330Ef7+/txxxx1MmTKl\nymPJzc1l4cKFrFixgrCwMCZOnFitjuF1wZ22dr5zp63VlXrb1qoaGuJqATYDo53/zgJaOP99F7Cu\npvurq4VqDilVqq6sXr1awkJDxc/HKu2aBkuAn48EBtgqfa6DiMjf//53sRgkwMciXWNCxc9ixGox\nMmfOnEpj3n77bfH18ZEQm5+0bRosPlaLxMY0lS1bttTGYVVbUVGRpKWlCSDxwf7SLtIxCVl4aIgc\nOXKk0phJkyY6YsICJCnCMQnZ5cOHVzoxVF5enlw+fLhjCG5EoMSHOSYhmzRpYqWTkB05ckTCQkME\nkMRQP4kK8BFA0tLSKo05duyYXNCxgwDSOTZc4kIDa2USMnesXr1awkNDxebrIz0SIiXY3/esbe18\n505bqyvHjh2TTs621rJJUMkEabXZ1mrzORW3APtxPOjqNHAt8Ifif9d0f3W1aFKh6hO73S7t2rSR\nNk2CZO7YVrJkfDuZf1Vr6RIXLDFNoyU/P79CTFZWlvhaLTIwMVL2TblYMqcNlZ23D5IuTUMl0N/f\n5R+7kydPSlBgoPRNCpWFV7eRJePbySujW0pieID069unLg61Uk8//bTji7B/WzkxdYhkThsqH13d\nU/ysRvr37+8yZsGCBQLItIvi5L1r28qS8e3kkf7NxGKMPPvssy5jnnnmGbFajDzSv5ksGd9O3ru2\nrUy7KE4AWbBggcuY/v37i4/FyJMXJ8mS8e1k8bVt5aa0pgLI008/7TJmypQpEhHgL19N6CWZ04bK\n8d9dKvf2aCGA/Pjjj+6dJA+w2+3Svm0b6RYXITtvHySZ04bKvikXy6DkJhJbSVtT7rW1ujJlyhQJ\nsfnKzMuay5Lx7WTRNW3lqtSoWm1rtTahmIi8DDwA/BUIBOYBU4CpIvJWTfen1Plo48aNbN22jes6\nRhLq77gLGehrZXLnJhw5ms4XX3xRIeaZZ56hoMjOX/u3LYmJCvDj0T6tyc7L4+23364Q88EHH3Am\nO5ub0qLx93H8794k0Jer2ofz1dffcODAgdo7yLN48cUXaR4WwG/SkkvmZOidEMG17eP5fuV3LmPe\nfPNN2jcNYlBKWMmtix4JwfRICObNuXNdx8ydW7INOIbjDkoJo33TIN58802XMau++5bBKaF0aOq4\nDWMxhlFtI4gJ8uXFF190GTPvjTe4qWMCFzQNBcDHYuHBi1oSFWRj/vz51Twrnrdx40a2/LyNRy5q\nQVSAY0htqL8Pj/VpzeFK2ppyr63VWd3emMvQFqG0iHAMQ7ZaDNd2bEJYgJ9X2xq416cCEXkTeNMY\nEwgEi8hRz1ZLqcbt9OnTAITZyj7PIczf8TorK6tCzIkTJwBoElD2WQvRgY7XribHOn36NBZjCPEv\nW064Mykproc35OTkEBPgV6FfQ3SAP0VFRS5jTmdlEepXsR9EmL+Fg1mnXMecziLer+Lvp1A/w2kX\n5xnAbreXJG7FjDGE2azk5ORU2F5EOJ2dXfJZFPOzWgi3+bn8POtK8Wdcvm7Fr71Zt/rMnbZWF0SE\nM9k5hNvK9jvytRqC/X28/nm6M6Q0wJlMICLZQIDzGRVDPF47pRqptLQ0QkOCWbbjZJn1n+w4ia+P\nD3379q0QM3nyZAww58f9ZdbP/nE/VovhqquuqhAzcOBA7CJ8uuPXib5EhGU7TtIsPp6WLVt65oDc\nMGDAANYczuSn9F+/BLPyC3lry0HimyW6jLn4kkvYcCSHo2cKStZl5hay8mAOlwwZ6jrm0iGsPJhD\nZm5hybqjZwpYfziHiy+5xGVMQrNEvtidWWYSst0nctl2PJcBAypOXGaMYfDAgby59VCZSci+2pfB\njuOnuPjiiys5C7UvLS2NsJCQCu1mzo/7K21ryr22VheMMQwaOJDP9pymoOjXtvbjkTMcOJnt1bYG\nuNWn4hPgDue/w4EjwD4gB5hS0/3V1YL2qVD1zMyZMwWQbgkhclNaU+mdFCqAPPzww5XG9O7dWwAZ\n2bKpPNG/rVyc7LiPOmbMmEpjbr75ZrEYIwObh8mNXaKlY0ywADJ37tzaOKxqS09PlyCbTYJ8rXJn\n12R5pFcrSQ4NEKsxsmTJEpcxx48fl+ZJSRIR6CdXpUbJNR2jJDrYX6KbRMnevXtdxuzdu1eimzi2\nu6ZjlFyVGiXhgX7SPClJjh8/7jJmyZIlYjFGYoJ85bpOTWR02wix+RgJtPlLenq6y5jvvvtO/P38\nJDU6TB7t01pu65wkgX6+0q9PHykoKHDvJHlIcVu7NCVanujfVka1jj1rWzvfudPW6kpxW2seESCT\nOkfL5a3DxebrI3379K61tlabHTWPAR3k106bG3Fc8bga2FLT/dXVokmFqo/mzZsn3dLSJDAgQDqm\ntpcXX3yxysnBHD3SJ0mQzV+sxkhIUKDceeedVZZRWFgo//jHP6RNq5YSGBAgvXtdJO+//76nD8Ut\n27dvly5duoif1SJWi5FmCQmyaNGiKmMOHDggt912m0RFRkh4WKhMmjhRduzYUWXMjh07ZNLEiRIe\nFipRkRFy2223yYEDB6qMWbRokSQkJIjFID5Wi3Tp0kW2b99eZcyqVatkxOXDJSQoSJrFx8mDDz4o\np0+frjKmrsybN0+6d02ToMAA6ZSaeta2ptxra3Vl1apVcvnw4RIcFCQJcbG13tZqc0KxbKCdiOw1\nxiwENonIX4wxicDPIlL5AHMvqumEYkoppZRyqO6EYu48wWM7MMaZRAzFcTsEoCngvd4rSimllPIq\nd5KKx4CngN3AKhEpHvs1BNdP2lRKKaXUecCdqc/fMcZ8DcTh6E9RbDmw2FMVU0oppVTD4u5zKg4D\nh8utW+2RGimllFKqQfL+rChK1cDp06d59tlnGT16NNdeey2LFy/GbrefPbCe2r59O/fddx8jLr+c\nu+66i02bNtVKObfddhv+/v5YLRZCQ0OZM2eOx8sQEZYtW8bEiRMZNWoUTz31FCdPnjx7YB3Iy8vj\nlVde4YorruDqq69m3rx5FBYWVhnT2NqaUnWiqqEhjWlBh5Q2eBkZGdKpQ6pYLRYZkBQlXWLDBZDr\nJ01qkEPjli9fLgE2m0QG2mRYi2iJDQkUHx+rvPPOOx4tJyUlRQCJCvCRHvFBYvMxYjHIHXfc4dFy\n7r77bgEkNTpMLk5uIn4+VklJTpL9+/d7tJyaOnPmjPTudZEYg3SMCZJ20Y6Jy4YPG1bpvBcZGRnS\nMTVVrBYjnWODpFWTwJJJyBpiW1PqXNXakNKGSoeUNnz3338/Lz77b5Zd1Z3UJiEAvLXlILcv+5H/\n/ve/DB8+3Ms1rL6ioiJat2xBMznDwpFpBPpayS+yc9NHP/DNsTz2HzxIQEDAOZfz8ccfM3zYMC5t\nGcYd3WOxWgyn84t4ZPle9mbmUWj3zP//33//PT179uSJ/m35bVoyxhh2Z2Yz5J01DLvyGmbPnu2R\nctwxY8YM/vDQg/x1cCLtmjjO6dqDp3lsxX7mzJnD5MmTK8Tcf//9/OeZf/G3wc1IDvcH4PNdmfxz\n5aEG19aU8oTaHFKqlFe8s2AB17aNLUkoAK5pF0f76DCXk2nVZ+vXr2fXnr081LMFgb6OeTn8rBb+\n2LsVGSdPemySp6lTpyLAxAuisVoc8xgE+1m5ukMURQKzZs3ySDlvv/02sSGBTOmSXDKXR/OwQG7q\nEM/bCxfgzR8vC996i54JwSUJBUC3+GA6xQazcOFClzFvL3iLgUlBJQkFwMDmoSRHBDa4tqZUXdKk\nQjUY+fn5BPiUbbLGGAJ9LOTn53upVu4prm9xQlEs0MfxOi8vzyPlFBQ45i3wL3febM7XnurzkJ+f\nj83HgqXc/EuBvlbyCwpcB9WR/Pw8bNaKE0P5WyC/kvOcn59f4ZwZY/D3MQ2urSlVlzSpUA3GZZdf\nzsJfjnIs+9cv9ZUHT7D20IkGdzm6W7duNImM4D/r95b5Ff/8+j0E2GwMHDjQI+X8/ve/B+CDn0+U\nrCuyC+//fAKrgWnTpnmknGHDhrH7xGk+3Jlesi4rv5DXNx9i+LBhFWYirUvDR4xk5cFs0ktNDLU3\nM48NR7K5fMQIlzHDLh/Bir1nykxCtiU9m23pZxpcW1OqTlXV4aIxLWhHzQZv586d0rRJlDQNDpA7\nuiTJte3jxd/HKn1795K8vDxvV6/GZs+eLYB0jYuQad1TpE+iY3KwGTNmeLScoCBHx8QusYEytn2k\nNAv1E0B69+7tsTKKiopk5IjLxcdqkbFt4+S3acnSLCxIQkOCZePGjR4rxx1HjhyR5klJEmLz/LXQ\nEwAAIABJREFUk2GtwuWSFmES4OcjHVNT5dSpUy5jdu7cKdFNoiQiyF9GtImQgc3DxM/HKn0aaFtT\n6lxVt6OmXqlQDUZKSgqr16zlykk38L9MC5tMOH/6y2N8/Mmn+Pn5ebt6NXbDDTfwySefEHNBT947\nUoB/y04sXryY++67z6PlnDx5ktTUVH48msPSnzM4dLqQCRMm8M0333isDIvFwjvvLuL/ZjzFLls0\nH2UIl1xxNau/X8MFF1zgsXLc0bRpU1auXs1tv72L7RLJfp+m3HP/A3z1zTeEhIS4jElJSeH7NWu5\n9vqb2FIQyrHAeB79y2Msa6BtTam6oqM/lFJKKVUlHf2hlFJKqTqlSYVSSimlPEKTCqWUUkp5hCYV\nSimllPIITSqU8pBNmzZx6623cmH37oy94gqWLVtWK+V89dVXXHvNNVzYvRuTr7+etWvXnjVm2bJl\njL3iCi7s3p1bb72VzZs310rdaqqgoIAXX3yRwYMG0uvCnjz66KMcO3bM29VSqt47cOAADzzwABf1\n7MGQSy9l7ty59WPCu6rGmzamBX1OhapFn3/+udj8/SUhLEgmpiaUTHY2ffp0j5bz8ssvCyDtmoTK\nxA4JkhIZIlarVRYtWlRpzPTp053PqQiXiakJkhAWJDZ/f/n88889WreaKiwslOHDhokxSNf4YOmX\nFCIBfj7SPClJDh486NW6KVWf/fLLLxLdJEqCbb4ysHmoXBAbXOuTK+qEYuXokFJVW0SETh1SCcs6\nyuIxXbH5WBER/vT1Np7fuI+9e/cRFxd3zuVkZWWREB/H6ORwnrmkAxZjKLTbmfTfjWw4Y9i9dx++\nvr5lYg4dOkRSUiK/6ZzIY33bYIwht7CIK95bx6nQGH74aZPXnnb5zjvvcPXVV/PHAc3oHh8MwNEz\nBdz76T6uv/k2nnnmGa/US6n67ppx41jx8fvMuKQZYTYfAJbvPMm/Vx1mxYoV9O/f3+Nl6pBSperI\nrl272LRlK3elJWNzzt1hjOHeHi0oLCzio48+8kg5X3zxBVmnz3BvjxZYnImAj8XC3d1SOHj4iMvb\nIB999BGFhUXc26NFSfJg87FyV1oyP23ewq5duzxSN3csXbqUllGBJQkFQNMgXwYmBfHeone9Vi+l\n6jMRYenSpQxpEVKSUAAMSgmjaYiNJUuWeLF2mlQopZRSykM0qVDqHKWkpNChfTueWb+H3MIiwPFr\n4unvd+LjY2XYsGEeKWfgwIGEBAfx9Pc7sTtvWxba7cxcu4v42Bi6d+9eIWbYsGH4+Fh5+vudxX2L\nyC0s4pn1e+iY2p6UlBSP1M0do0aNYsfxbNYcPF2y7uiZAr7Ye4YxY6/0Wr2Uqs+MMYwaNYpPdmaV\nmfDu812ZHM3KZfTo0V6sHdpRUylPKO6oGR8aWKcdNZtHBIvVapXFixdXGlO+o2Z8aKB21FSqAavP\nHTX1SoVSHjBw4EDWrlvH8HET2BYQS8pFA/n444954IEHPFrOzTffzJdffknnwZexzRZL/xFXsGrV\nKsaMGVNpzAMPPMDHH39MykUD2RYQy/BxE1i7bp3Hpld3l9Vq5b0lS3jhhRcJa9ONgth23P/gw3y/\ndq1HOrYq1Vi1atWK9Rs28pvf3U1OdBtiOl7E3LlzmT1njtc6XhfT0R9KKaWUqpKO/lBKKaVUndKk\nQimllFIe4fWkwhjzkDFmtTHmlDHmiDFmsTGmzVliBhhj7OWWImNM07qqt1JKKaXK8npSAfQDngEu\nBC4BfIFPjDEBZ4kToDUQ61ziRORobVZUKaWUUpXzelIhIsNFZK6IbBGRH4EbgCSgWzXC00XkaPFS\nqxU9j5w5c4YnnniCTh1SadWiOb/5zW/Ys2ePx8s5cuQI9957L21btSS1bRseeeQRTp486fFy6sq3\n337LFWNGk5KUSN/evXjjjTc4W0foTz/9lOGXXUZKUiIXDxrEe++9d9Zy3nvvPQYPGkhyUjOGXXYZ\nn376qacOodH66aef6N69OwH+fgTa/Onbt2+ttOm68u233zJmzGiSkxLpU822plSdqGq8qTcWoBVQ\nBKRWsc0AwA7sBA4CnwC9z7JffU5FNeTl5UnvXheJv49VBqWEyYg2ERIR5C/RTaJk586dHivnyJEj\nkpKcJBEB/nLzBYlyfccECfb3lU4dUuXUqVMeK6euLFmyRKxWq3RoGibTuqfIJSnRAsjvf//7SmNm\nz54tgHSNi5C7u6dI38QoAWTGjBmVxsyYMUMA6RgbLFe2j5Q20UECyOzZs2vhqBqHn376Sfx8fSTA\nxyLDWoXLpS3DxM9qJNDmLwcOHPB29WrM0dYs0jwyUMa2j5Su8SFnbWtKnasGOaGYcQywfR8IEZEB\nVWzXBkdisQbwB24FJgE9RWRDJTE6pLQa3njjDSZNmsT0S5JoHx0IQGZuIXd/so8x107k5Zdf9kg5\nDz30EM//8x98O+EiEkMdd7q2HD9Nv3kr+b+nnmLatGkeKacu2O122rZuRXP7aRaOSsNqcYwTf2r1\nTp5YuYOdO3eSnJxcJiYvL49m8XFc3DSAF4Z2LBlb/tCKrczecoQDBw8SERFRJubEiRMkxMdxSXIg\nt3SNARw/CmauPMSmLB/2HziIv79/HRxxw9KrVy82rFnFs8NbEB3kmHBtb2YeUz/axajRY1i8eLGX\na1h9drudNq1aEpZ3jEf6JZS0tbc3HWPeTxku25pSntBQh5Q+D6QC11a1kYhsE5GXRGS9iKwUkZuB\nb4G766KSjdlHH31Em+igkoQCIMzmw4CkID784H3PlfPBB4xqEV2SUAC0jwpmcFIkH/73A4+VUxd2\n7NjB9p27uKNzYsmXPMAdXZIQET755JMKMWvXruVYxgmmpCWVeVjNb9KSycnNZcWKFRViVqxYQU5u\nHqPbRpasM8Ywsk0Ex45nuJxQTMGPGzbQJzG0JKEASArzJy0uiK++rHie67MdO3awY9duRrQOL9PW\nRrSJRMTusq0pVZd8zr5J3TDGPAsMB/qJyCE3drEa6HO2je6++27CwsLKrBs/fjzjx493o8jGx8/P\nj7xC52WsUn/scgvt+Pn5ea4cfz/OZBVVWJ9daCfUr2H92i4+L9mFZY8nt9COiLg8byUxBWVjivfh\n6opDcUxuob3M+rwiKfO+KstitVQ4ZwC5BXas1nrzFVgtxZ9xXlHZ48kvsiOibUB5xvz585k/f36Z\ndZmZmdWKrRf/RzkTitHAABHZ6+ZuugBnTUZmzpyptz+qcNVVVzFnzhy+2H2KQSmO5GvPyTy+2Hua\n3/7uVo+Vc/U11/LIww+x+uBJesaHA/DprnS+2necOY+P81g5dSE5OZnuXdN4es0uBiRGEW7zpcgu\n/OXbbdj8/RkxYkSFmLS0NFKSk/jb6p0sjAkj0NdKfpGdx7/dTmR4OIMGDaoQM2jQICLCw3jzx2Pc\n2yseX6shr9DOgk0ZpCQna7uuRL8BA1n20YdsPZZDuyaOK2NrD55mU3oOt9xynZdrVzPJycl065rG\nO1t+5oKYIIL9rBTZhbk/pGPz93PZ1pSqKVc/tEvd/qhaVR0u6mLBccvjBI6hpTGlFlupbZ4EXiv1\neiowCmgJdAD+CRQAA6soRztqVoPdbpeJE68TQFpFBUrn2CCxWox0TE2VjIwMj5Vz5swZ6dOrlxhj\npE+zSOkRHyGAjBg+XPLz8z1WTl1Zs2aNhIWGSKjNTy5LiZak8CAxxshLL71Uaczy5cslwGaTyECb\nDGsRLbEhgeLjY5V333230ph33nlHfKxWiQrylx4JwRIa4CcBNn9Zvnx5bRxWo5Ceni5hIY7OjKnR\nAdI60iaAxDSNljNnzni7ejW2Zs0aCQ0JkSB/X+kRHyQxIbaztjWlzlWD6ahpjLE7K1rejSLyunOb\n2UCyiAx2vr4fuA2IB7KBH4C/iMiXVZSjHTWryW63s3TpUhYsWEB2djZDhgxh8uTJBAcHe7ScvLw8\n3nzzTT744AOsVitjx47l6quvxsenXlxAq7H9+/cza9YsNqxfT7PERG655RaX05GXtmPHDmbNmsWW\nzZtp0bIlt99+Ox06dKgyZtOmTbzwwgvs3LGD9qmp3HHHHbRs2dKTh9LonDp1iqlTp7Js2TIsFgtX\nXHEFM2bMwGazebtqbnGnrSl1LqrbUdPrSUVd0aRCKaWUck9DHf2hlFJKqQZKkwqllFJKeYQmFUop\npZTyCE0qlFJKKeURmlSoCjIyMpg6dSqxTaMJCwnhqiuv5KeffvJ2teq9P/7xj0SEheJjsRDo78fo\n0aPJz8/3drWUUqrONMyxe6rWZGdnM2hAf/bu2M7E9nGE+4cz74tP6dP7E75buYrU1FRvV7FemjJl\nCrNmzaJ3fDjDO7fhx/RTLFy6lM4XXMCWrVu9XT2llKoTmlSoMt58801+3LSZryf0omN0CAB3pCXT\nZ95KnvjrX3lz3jwv17D+yc3N5dWXX2JM6xjmDO9c8njzHnHh3Pf5FpYtW8bQoUO9XEullKp9evtD\nlfHZZ5/RMz6iJKEACPHz4arWMXz2v/95sWb11+eff05+YRGTOzYrM1/KpA4JGOD111/3XuWUUqoO\naVKhyggODuZYTgHlH4qWnpPn8SdqNhZNmjQB4HhO2f4TGbkFCFSYwE4ppRorTSpUGRMmTGBHRhbP\nr9+D3ZlYfHvgBG//fIQJkyZ5uXb1U48ePYgIC+VvK3dwICsXcMw++uAXW7Eaw8MPP+zlGiqlVN3Q\nPhWqjIEDBzJt2jQe/uc/mfXDAUL8rGw6mknf3r34/e9/7+3q1VtvzJvPmFEj6fTql3SKDmH7iTOc\nKSji7nvuoVmzZt6unlJK1Qm9UqHKMMYwc+ZMvv76a0ZPupFeo65mwYIFfPbFCoKCgrxdvXpr+PDh\n/LJjJ6PGjOFMWAxdLuzFZ59/ztNPP+3tqimlVJ3RKxXKpT59+tCnTx9vV6NBSU5OZtGiRd6uhlJK\neY1eqVBKKaWUR2hSoZRSSimP0KRCKaWUUh6hSYUHiQg5OTkVnvFwPikoKDiv57sobgN2u93bVVEu\n5OfnU1BQ4O1qKNVoaVLhAUVFRTzxxBPExcUTGBhIcnJz/v3vf59XycUvv/zCFWNGExAQgM1m47Ih\nQ9i4caO3q1WnZs+eTdvWrQgMDCQ6KoqHHnqIvLw8b1dLARs3buSyIUOw2WwEBARwxZgx/PLLL96u\nllKNjo7+8IA777yTl156iUFjJzC2Uzc2ff81U6dOJSMjgz//+c/erl6tO3z4MP369CagMJfH+7TC\nz2rh5fWr6N+vL2vWrqN169bermKte+6557jzzjsZ3TqWe4d05KdjWcx8agbbf9nG2++86+3qndd+\n+eUX+vfrS7y/hRkD25FfZOfFL5fTr09vNvzwI7Gxsd6uolKNhiYV52j//v28+OKLjJ/2By6feBsA\n/UZcSVhUU2Y89RR33313o39M83PPPUf2qVN8M7k30YH+AFzbPp7uc7/j6aefZtasWV6uYe3Kz8/n\nsT8/ysQOCTx3aceS9R2bhHDHu4v44YcfuOCCC7xYw/Pb008/TSB2/jfuQkL8HF9549rFkfbatzz/\n/PM89thjXq6hUo2H3v44R6tWrcJut9N3+Ngy6/sOv4LsM2f44YcfvFSzuvPN118xKDGiJKEAxyRk\nw5tH8c2XK7xYs7qxc+dOjh47zjXt4sqsv6ptHBZj+Pbbb71UMwXwzZcruDwlqiShAIgO9GdQYgTf\nfPWVF2umVOOjScU5ioyMBCD94L4y69MP7i/zfmMWGRnFvjMVO2fuzcolMirKCzWqW+Hh4QDsPZVb\nZv2B07nYRc6LNlCfRUZFsTcrt8L6fafzz4v2qVRd0qTiHPXv35/k5ObMferPZBw9BMCRfbt565m/\n0bVrN1JTU71cw9p3w403sv7QCZ5Zu5tCu50iuzB3037+tzudyTfe5O3q1brY2FguGzKE6at3sflY\nFgDHsvO5+/OtRIaHM2LECC/X8Pw2+cab+HRXOnM37afILhQU2Xlm7W7WHz7B5Btu8Hb1lGpcROS8\nWICugKxdu1Y8bdWqVRIZFSVWq1ViE5PFGCPx8QmyefNmj5dVH9ntdrnnnnsEkIhAm0QHBwggk6+/\nXoqKirxdvTqxZ88ead2yhQDSPCJE/HysEhwUKJ9++qm3q3beKyoqksnXXy+ARAcHSESgTQC59957\nxW63e7t6SjUIa9euFUCArlLF31oj58mwR2NMV2Dt2rVr6dq1q8f3n5mZyfz589m1axft2rVj3Lhx\n590EXBs3bmTRokUUFRUxcuRIevbsiTHG29WqM3l5ebz77rts3LiRhIQEJkyYQJMmTbxdLYXjx9Pq\n1at5//33sVqtjB07ls6dO3u7Wko1GOvWraNbt24A3URkXWXbaVKhlFJKqSpVN6nQPhVKKaWU8ghN\nKpRSSinlEZpUKKWUUsojNKlQSnnUrl27WLVqFYWFhbVaztGjRzl06NB5NceOUvWdJhVKKY9YtWoV\ncbExtGjRgosuuoigwADuuusuj5ezbt06+vbuTUxMDPHx8XTvmsaXX37p8XKUUjWnc38opc7ZsWPH\nGNCvL0E+cEf3GCIDfFi+K5Nnn32W4OBg/va3v3mknD179jB44ECSAi28dFknfCwWZm3cy9AhQ1j9\n/fd06tTJI+UopdyjVyqUUufswQcfJK+gkMcHJzKsdQQXNgvhob4JXBATyLP//pfHynn22WexFOXz\n37HdGNcunrFtYll6RTdiAnx5+qmnPFaOUso9mlQopc7Z6tWrSQjxo1nor5PKGWPolRjC6ewc7Ha7\nR8pZ+/33DEgIJ8zft2SdzcfKkOQI1qxe5ZEylFLu06RCKXXOmjZtyrHsAnIKyiYP+zLz8LVasFg8\n81UTGxfHtpO5FTpnbj2RQ2xcvEfKUEq5T5MKpdQ5e/zxx8kvEp5ZdYiTOYUU2YUvd59i2faT9OrT\n12Pl3HLrrWxOz+TP3/xCVn4hOYVFzPx+J1/tPcatt9/usXKUUu7RjppKqXPWq1cv7r7nHmbO/Aff\n7svCx2IosAsJ8XG8//77Hitn8ODBTJ8+nYcffpjn1u/BYgx5hUXcf//9jBs3zmPlKKXco3N/KKU8\nZseOHTz++OOcOHGCK6+8kuuvv75Wytm3bx9Lly6lqKiI4cOH06pVq1opRynlUN25P/RKhVLKY1q2\nbMmcOXNqvZzExER++9vf1no5Sqma0T4VSimllPIITSqUUkop5RGaVCillFLKIzSpUEoppZRHaFKh\nlFJKKY/QpEIppZRSHqFJhVJKKaU8QpMKpZRSSnmEJhVKKaWU8ghNKpRSSinlEZpUKKWUUsojNKlQ\nSimllEdoUqGUUkopj9CkQimllFIeoUmFUkoppTzC60mFMeYhY8xqY8wpY8wRY8xiY0ybasQNNMas\nNcbkGmO2GWMm10V9z2b+/PneroLX6TnQc3C+Hz/oOTjfjx/Oz3Pg9aQC6Ac8A1wIXAL4Ap8YYwIq\nCzDGNAc+AJYDnYF/AS8bYy6t7cqezfnYiIrl5eXx+eef8+9//5vs7GxvV8erzud2AHr8oOfgfD9+\nOD/PgdeTChEZLiJzRWSLiPwI3AAkAd2qCJsC7BSR34vIzyLyHPAOcHft11i5smjRIpKaJTB48GBW\nrlxJQlwcs2fP9na1lFJK1SGvJxUuhAMCZFSxzUXA/8qtWwb0qq1Kqcpt3LiRa64ZR89wH76c0Iu+\nzSIYlhDMTTfdxOeff+7t6imllKoj9SqpMMYY4J/A1yKyuYpNY4Ej5dYdAUKNMf61VT/l2nPPPUdc\nkI05wy6gc9NQQvx8+M+QjnSKCeff//qnt6unlFKqjvh4uwLlPA+kAn1qYd82gFtuuYWQkJAybwwd\nOpTLLrvMI4VkZmaybt06j+yroVi3Zg2tw2xsOn4agMy8QjamZ9E6zMaGjRvPu/MB52c7KO18P37Q\nc3C+Hz803HPw8ccfs2zZsjLrsrKyiv9pqyrWiEgtVatmjDHPAiOBfiKy9yzbrgDWisg9pdbdAMwU\nkYhKYnoD33iuxkoppdR5p4+IfFvZm/XiSoUzoRgNDDhbQuH0HTCs3LohzvWV2UDVnT+VUkopVbWt\nVb3p9SsVxpjngfHAKGBbqbcyRSTXuc2TQIKITHa+bg78iON2yavAxTj6YgwXkfIdOJVSSilVB+pD\nUmHHMdqjvBtF5HXnNrOBZBEZXCquPzATRx+M/cBjIjK3DqqslFJKKRe8nlQopZRSqnGoV0NKlVJK\nKdVwaVKhlFJKKY/QpMJNxpgHjTF2Y8w/qthmgHOb0kuRMaZpXdbVU4wxj7o4nqoeUlZvJ35zV03P\nQWNrAwDGmHhjzFxjzDFjTLYxZqMxputZYhpbO6jROWhM7cAYs8vFsdiNMc9UEdPYPv8anYPG9Pmf\nTb0YUtrQGGN6ALcBG6uxuQBtgJInh4jI0VqqWl34CcdoG+N8XVjZhqUmfnsemIBjwriXjTEHReTT\n2q1mrar2OXBqNG3AGBOO43kvy4GhwDGgNXCiipjmNKJ24M45cGos7aA7YC31uhPwCbDQ1caN7fN3\nqtE5cGosn3+VNKmoIWNMMPAGcAvwx2qGpYvIqdqrVZ0qFJH0am5bMvGb8/XPxpi+OCZ+a6hfJlCz\nc1CssbSBB4G9InJLqXV7zhLT2NqBO+egWINvByJyvPRrY8xIYIeIfFVJSGP7/N05B8Ua/Od/Nnr7\no+aeA94Xkc+qub0BNhhjDhpjPnE+2bMha22MOWCM2WGMecMYk1jFto114reanANoXG1gJLDGGLPQ\nGHPEGLPOGHPLWWIaWztw5xxA42oHABhjfIHrgFeq2Kyxff5lVPMcQCP8/F3RpKIGjDHXAl2Ah6oZ\ncgi4HbgSGAvsA74wxnSpnRrWupU4pqYfCtwBpABfGmOCKtm+MU78VtNz0NjaQAscvzx/xvEU2/8A\n/zbGTKoiprG1A3fOQWNrB8WuAMKA16rYprF9/uVV5xw01s+/Ar39UU3GmGY4ntp5iYgUVCdGRLZR\n9imhK40xLXFc9mtwHZVEpPQMMz8ZY1bjuOw7DpjtnVrVrZqeg8bWBnD8EFktIsW3/jYaYzriSLDO\nl4fP1fgcNMJ2UOwm4CMROeztinjRWc9BI/78K9ArFdXXDYgG1hljCowxBcAAYKoxJt8YY6oOL7Ea\naFVblaxLIpKJ43+Uyo7nMBBTbl0McEpE8mqzbnWlGufAlYbcBg4BW8qt2wIkVRHT2NqBO+fAlYbc\nDjDGJOHodPnSWTZtbJ9/iRqcA1ca9OdfGU0qqu9/OHr4dgE6O5c1ODptdpbqP5q0C44vpQbP2Wm1\nFZUfz3c4RkmUdraJ3xqUapwDVxpyG/gGaFtuXVuq7qjY2NqBO+fAlYbcDsDxC/0I8OFZtmtsn39p\n1T0HrjT0z981EdHFzQX4HPhHqddPAq+Vej0Vx0RpLYEOOG6fFAADvV13N493BtAfSAZ64+i5fQSI\nquT4m+MYPvV3HF+6vwHycdxC8vrx1NE5aGxtoDuQh6NfUUscQwSzgGtLbdOo24Gb56CxtQMD7Aae\ncPFeo/783TwHjerzr2rRPhXnpvzViTig9EgAP+BpIB7IBn4ALhaRL+umeh7XDJgHRAHpwNfARfLr\n8Koyxy8iu40xl+OY+O13OCZ+u1ka9kyyNToHNLI2ICJrjDFXANNxDKneBUwVkbdKbdao24E754BG\n1g5wXPJPxHVfqkb9+ZdS7XNA4/v8K6UTiimllFLKI7RPhVJKKaU8QpMKpZRSSnmEJhVKKaWU8ghN\nKpRSSinlEZpUKKWUUsojNKlQSimllEdoUqGUUkopj9CkQimllFIeoUmFUqpSxpjZxphFVbw/2Rhz\noi7rVBVjzC5jzO/ciIsyxhxxThBVK0qVEV9bZSjlbZpUKKXOVZ0/lrcWkpk/AO+JyF4P7rMM56Pc\nXwMeq60ylPI2TSqUUg2RwUPJjDEmAMdsky97Yn9nMQe4zhgTXgdlKVXnNKlQqp4yxlxljPnBGJNt\njDlmjPnE+Qew+P1bjDGbjTE5zv9OKfVesjHGboy5xhjzjXObH40x/UttYzHGvGyM2eksY6s7tw5c\n1Hu0MWats8ztxpg/GWOspd63G2NuNsYsMsacMcZsM8aMLLePUc712c7jnuSMCzXGDABeBcKc64qM\nMX8qFR5kjHnFGHPKGLPHGHPrWap8OZArIt+Xq0OqMeZ9Y0ymc18rjDEpzvdmG2MWG2MeMsYcNsac\nMMY8YoyxGmP+zxhz3BizzxhzQ+l9ishm4CBwRU3Pq1INgSYVStVDxphYHLOhvgy0AwYAi3D8QscY\ncx3wZxzTb7cDHgYeM8ZMKrer/8MxXXsX4DtgqTEmwvmeBdgHXAm0B/4CPGGMueoc6t0PxyX+mc56\n3Q5MdtavtD8BbwGdgA+BN4t/vTv/cL/tPN7OznPwJL9emfgWmAacAmJwzAj5VKl93wN87zzm54H/\nGGNaV1HtvsDacscRD3wJ5AADgTTgJSgzs/NgZ9n9gLtx3Nb4AMgAegKzgBdc9KFY7YxRqvHx9tzr\nuuiiS8UFxx+xIiCxkvd/Aa4pt+4PwDfOfycDduC+Uu9bgb2l17nY7zPAwlKvZwOLqth+MpBR6vWn\nwAPltrkOOFDqtR34c6nXgc51Q5yvpwMby+3jcef5CHVVbqntdgFzyq07DNxWxTEsBl4qt+5JYDtg\nrSRmNrCz3LotwBelXluALGBcue2eBpZ7u43pokttLKWzbqVU/bERWA78ZIxZBnwCvCMiJ40xgUBL\n4BVjTOl+AFbgZLn9rCz+h4gUGWPW4LgqAYAx5rfAjUASEAD4AevPod6dgd7GmEfK1cvPGGMTkVzn\nuh9L1SvbGHMKaOpc1QbHlYbSVtegDj+We3241L5dCQByy63rDHwlIkVVxG0q9/oIZY/Lbow57qLs\nHByJlFKNjiYVStVDImIHhhhjegFDgLtw3JroieOPEsAtVPxjW9UfwTKMMdfiuDVyN47aUK+fAAAC\ntklEQVTkIwv4PY5L9+4KxnFro8Iw1FIJBUBB+bfx3O3Ymu77GBBRbl2Oqw2rUU51yo4E0quxf6Ua\nHO1ToVQ9JiLfichfcNwOyQeuEJGjODr7tRSRneWWPeV2cVHxP5ydJbsBm52reuO4XfKCiGwUkZ04\nroCci3VAWxf12lmDffwMdC+3rnyik4/jCognrAdSy637AehXuoOpB3Xk3K4GKVVvaVKhVD1kjOnp\nHFnQzRiTiKMzZRN+TQgeBR4yxtxljGltjOlojLnBGDOt3K5+a4wZY4xpi6PTYjiO/gDg6JfR3Rgz\nxLmPx4Ae51j1x4DrnSM+Uo0x7ZwjUB6vwT5eANoZY6Y76zUORx8K+LWz5m4g2Bgz2PlQqQBXO6qm\nZUAHY0xYqXXPAqHAAudn0MoYM/EsHT7PylnPbs4ylWp0NKlQqn46BfQH/ovjl/tjwD0i8gmAiLyC\n4/bHjTh+VX+B4w/vrnL7edC5bMBxZWKkiGQ433sBx22Kt3Dc/ogEnjuXSjvrNwK4FMetme9wjNTY\nXXozV6Gl9rEbuArHsMuNOEaQPOF8O8+5zXc4RlcsAI4C91dn35XU+SccV1jGlVqXgWN0RxCOc7sG\nx/kuf3vjbOWUXzcG2CMi31ZVJ6UaKiNS5w/DU0rVMmNMMrATSBORH7xdn3NljPkDjhEcybW0/+HA\n/4lIx9rYf6lyvgP+KSILarMcpbxFO2oq1XgZb1fAXc4HeX0PHMfxHIn7gH/XVnki8qHzFkeCiByo\njTKMMVHAu5pQqMZMr1Qo1Qg19CsVxph/ANfgGJWxF3gdmO4cFaOUqqc0qVBKKaWUR2hHTaWUUkp5\nhCYVSimllPIITSqUUkop5RGaVCillFLKIzSpUEoppZRHaFKhlFJKKY/QpEIppZRSHqFJhVJKKaU8\nQpMKpZRSSnnE/wNH7WX00hCDCwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8c43e30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "features = iris.data[:, :2]  # select first 2 feature\n",
    "label = iris.target\n",
    "\n",
    "plt.scatter(features[:, 0], features[:, 1], c=label, cmap=plt.cm.Paired)\n",
    "\n",
    "for i in range(features.shape[1]):\n",
    "    f_data = features[:, i]\n",
    "    if i == 0:\n",
    "        plt.xlabel(iris.feature_names[i])\n",
    "        plt.xlim(f_data.min(), f_data.max())\n",
    "    else:\n",
    "        plt.ylabel(iris.feature_names[i])\n",
    "        plt.ylim(f_data.min(), f_data.max())\n",
    "\n",
    "plt.title(\"iris data\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dimension is reduced from 64 to 2.\n"
     ]
    }
   ],
   "source": [
    "from sklearn import decomposition\n",
    "\n",
    "digits_data = digits[\"data\"]\n",
    "show_dimension = lambda dset: len(dset[0])\n",
    "\n",
    "dimension = 2\n",
    "digits_recuced = decomposition.TruncatedSVD(n_components=dimension).fit_transform(digits_data)\n",
    "\n",
    "print(\"Dimension is reduced from {0} to {1}.\".format(show_dimension(digits_data), show_dimension(digits_recuced)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Select the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "機械学習に使えるモデルには色々なものがあり、scikit-learnでも様々なモデルが使えるようになっています。  \n",
    "ただ、その分一体どれを選べば良いのかは非常に悩ましい問題です。  \n",
    "\n",
    "一つの基準として、以下のようなフローチャートがあります。これは、scikit-learnの中のアルゴリズムをどのような基準で選択したらよいのかを図示したものです。\n",
    "\n",
    "[Choosing the right estimator](http://scikit-learn.org/stable/tutorial/machine_learning_map/)\n",
    "\n",
    "scikit-learnにはNeural Networkがないため図中にもありませんが、基本的にはSVC/SVRの代替であり、データが多いほど精度が向上します。\n",
    "\n",
    "ポイントとしては以下になります。\n",
    "\n",
    "* 最低でも50件以上はデータを集める\n",
    "* 単純なモデルから始める(ClassificationならLinerSVC、RegressionならRasso/ElasticNetなど)\n",
    "* Just lookingから始める(データを見て、必要に応じ次元削除を行う)\n",
    "\n",
    "機械学習で正しい結果を出すにはデータの整備(図中ではJust looking、[前章](#Arrange-the-Data)に当たる部分)が欠かせません。データ整備した上で単純なモデルで検証をしてみて、必要に応じ他のモデルを試していくというのが基本的な進め方になります。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select Model Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "特徴量が多い場合は、どの特徴量をモデルに使うのかも重要な問題です。scikit-learnには、どの特徴量が予測値に寄与しているか調べるための機能があります。  以下では、[Feature selection](http://scikit-learn.org/stable/modules/feature_selection.html)を利用し特徴量をもっとも有用な2つに絞っています(k=2)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(150, 4)\n",
      "(150, 2)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_selection import SelectKBest\n",
    "from sklearn.feature_selection import chi2\n",
    "\n",
    "X, y = iris.data, iris.target\n",
    "print(X.shape)\n",
    "\n",
    "X_new = SelectKBest(chi2, k=2).fit_transform(X, y)\n",
    "print(X_new.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "学習に当たっては、データを学習用(training set)と評価用(test set)に分けておきます。学習に使ったデータに対して予測がうまくできるのは当たり前なので、正確に精度を測定するため、評価用のデータは学習用とは別にしておきます。\n",
    "\n",
    "単純に学習用と評価用に2分割するのでなく、データ全体を何個かに分割し、評価用として使うデータを切り替えていくという方法もあります。これにより、少ないデータでも効率的に学習を行うことができます。\n",
    "\n",
    "[K-FOLD CROSS-VALIDATION, WITH MATLAB CODE](https://chrisjmccormick.wordpress.com/2013/07/31/k-fold-cross-validation-with-matlab-code/)\n",
    "\n",
    "![cross_validation](./pictures/cross_validation.PNG)\n",
    "\n",
    "これはCross Validationと呼ばれる手法ですが、scikit-learnでは単純な分割からこのCross Validationまで、[`cross-validation`](http://scikit-learn.org/stable/modules/cross_validation.html#cross-validation)で行えるようになっています。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test data is 30.0% of data\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "test_size = 0.3  # use 30% of data to test the model\n",
    "X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=test_size, random_state=0)\n",
    "test_data_rate = X_test.shape[0] * 100 / (X_train.shape[0] + X_test.shape[0])\n",
    "\n",
    "print(\"test data is {0}% of data\".format(test_data_rate))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cross Validationを利用する際は、データの分割と学習を合わせて行ってくれる`cross_val_score`を利用するのが簡単ですが(後述します)、データの分割のみ行う場合は`KFold`を利用します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: training 100, test 50\n",
      "1: training 100, test 50\n",
      "2: training 100, test 50\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "\n",
    "kf = KFold(n_splits=3)  # divide into 3 set\n",
    "\n",
    "i = 0\n",
    "for train_index, test_index in kf.split(iris.data):\n",
    "    x_train = iris.data[train_index]\n",
    "    y_train = iris.target[train_index]\n",
    "    \n",
    "    x_test = iris.data[test_index]\n",
    "    y_test = iris.target[test_index]\n",
    "\n",
    "    print(\"{0}: training {1}, test {2}\".format(i, len(y_train), len(y_test)))\n",
    "    i += 1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "これでデータの準備は整ったので、いよいよ学習を行っていきます。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下では、分類を行う際によく利用される[Support Vector Machine](http://scikit-learn.org/stable/modules/svm.html)をベースにその学習方法などを解説していきます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import svm\n",
    "clf = svm.SVC(gamma=0.001, C=100.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "モデルの構築はたったこれだけでおしまいです。そして、学習もたった一行で済ませることができます(以下の例では、最後の1データ以外を学習データとして渡しています)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=100.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape=None, degree=3, gamma=0.001, kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(digits.data[:-1], digits.target[:-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "そして、取っておいた最後の一つのデータについて、モデルを使って予測させてみます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([8])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.predict([digits.data[-1]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "実際判定した画像は以下です。「8」という予測はそこそこ的を得ているのではないかと思います。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQoAAAEQCAYAAABIhjo0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAD+dJREFUeJzt3X+sV/V9x/HnG0egIoLBTu20AtM6GgitaLVrlVmtGhPp\nmMaK2skMa9AuUbNkVbuGuD9Mk0Zh1WKW/fCiVBajMa1ZFadWO7UGkZbKxJporzqHqKhXg+JAPvvj\n+6W93AAfzuGc8z338nwk3z++h/v5vt/38r2ve77ne873HSklJGlPRvW6AUntZ1BIyjIoJGUZFJKy\nDApJWQaFpCyDQlKWQSEp6w/qfPCImAScBfQDW+qsJamUscBkYGVKadPuvqjWoKATEj+quYakfXcx\ncOfu/rHuoOgHWL58OdOmTSu8+Oqrr2bx4sVV91RLzUWLFpWu+fTTT3PiiSeWWrt69epS69555x0O\nOeSQUmvnzJlTat0DDzzA2WefXWrtRRddVGrdddddxw033FBq7fjx40utG07P2/Xr13PJJZdA93d1\nd+oOii0A06ZN4/jjjy+8eMKECaXW7YuyNSdNmlS65ujRo0uvHzNmTKl1o0aNKr32iCOOKLVu7Nix\npdfOnDmz1LqDDz649NqJEyeWWjecnreD7PHQgAczJWUZFJKyDApJWa0Oinnz5u0XNadMmdJ4zXHj\nxjVec/r06Y3XPO+88xqvORKftwZFC2pOnTq18Zq9CIoZM2Y0XvP8889vvOZIfN62OigktUOpoIiI\nb0XEbyPiw4h4KiLKnQQgaVgoHBQR8XXgRmAR8HlgLbAyIg6tuDdJLVFmj+Jq4J9SSrenlJ4HFgIf\nAJdV2pmk1igUFBExGpgFPLxjW+p8jPdDwBerbU1SWxTdozgUOADYOGT7RuDwSjqS1Dp1X+sBdC5Y\nmTBhwk7b5s2b15O3kaT91YoVK1ixYsVO2wYGBvZqbdGgeAv4GDhsyPbDgNd3t2jx4sWNXyQjaWe7\n+uO8Zs0aZs2alV1b6KVHSmkr8Axw+o5tERHd+08WeSxJw0eZlx43AX0R8Qywis67IAcCfRX2JalF\nCgdFSumu7jkT/0DnJcevgLNSSm9W3Zykdih1MDOltBRYWnEvklrKaz0kZRkUkrIMCklZBoWkLINC\nUpZBISnLoJCUZVBIymrk6tGm9ff3N15z2bJljdcEOProoxuvOXny5MZrqrfco5CUZVBIyjIoJGUZ\nFJKyDApJWQaFpCyDQlKWQSEpy6CQlFVm9ugpEfGTiHgtIrZHxJw6GpPUHmX2KMbR+UDdK4BUbTuS\n2qjMp3A/ADwAv5vpIWmE8xiFpCyDQlKWQSEpy2nm0n6iyWnmpTjNXOq9fZlmXjgoImIccAyw4x2P\nqRExE3g7pfRq0ceT1H5l9ihOAH5G5xyKBNzY3b4MuKyiviS1SJnzKB7Dg6DSfsVfeElZBoWkLINC\nUpZBISnLoJCUZVBIyjIoJGUZFJKyRuSQ4l4M0R160VtT3n333cZr9mIIdC/+T3vxs20r9ygkZRkU\nkrIMCklZBoWkLINCUpZBISnLoJCUZVBIyjIoJGUVCoqIuDYiVkXEexGxMSLujYjP1NWcpHYoukdx\nCnAzcBJwBjAaeDAiPlF1Y5Lao9C1Himlcwbfj4j5wBvALODx6tqS1Cb7eoxiIp2P7H+7gl4ktVTp\noIiIAJYAj6eUnquuJUltsy+XmS8FPgt8qaJeJLVUqaCIiFuAc4BTUkobcl/vkGKp9xodUtwNia8B\ns1NKr+zNGocUS73X2JDiiFgKzAPmAJsj4rDuPw2klLYUeSxJw0fRg5kLgYOBR4H/HXS7oNq2JLVJ\n0fMoPOVb2g/5iy8py6CQlGVQSMoyKCRlGRSSsgwKSVkGhaQsg0JS1ogcUtwLfX19Pak7d+7cxmte\nf/31jde89NJLG6+p33OPQlKWQSEpy6CQlGVQSMoyKCRlGRSSsgwKSVkGhaQsg0JSVtEhxQsjYm1E\nDHRvT0bE2XU1J6kdiu5RvAp8GziezrzRR4AfR8S0qhuT1B5FP1z3P4Zs+vuIuBw4GVhfWVeSWqX0\nRWERMYrOx/QfCPyiso4ktU6ZSWHT6QTDWOB9YG5K6fmqG5PUHmXe9XgemAl8AbgVuD0i/qTSriS1\nSuE9ipTSNuCl7t1fRsQXgCuBy3e3xiHFUu81OqR4F0YBY/b0BQ4plnqvySHFNwD3A68A44GLgdnA\nmUUeR9LwUnSP4g+BZcARwADwa+DMlNIjVTcmqT2KnkexoK5GJLWX13pIyjIoJGUZFJKyDApJWQaF\npCyDQlKWQSEpy6CQlGVQSMpymnlFlixZ0pO6Q6/KHan6+/t73cJ+zT0KSVkGhaQsg0JSlkEhKcug\nkJRlUEjKMigkZRkUkrL2KSgi4pqI2B4RN1XVkKT2KR0UEXEi8E1gbXXtSGqjUkEREQcBy4EFwLuV\ndiSpdcruUfwQuM+P6Zf2D2WGFF8IfA44ofp2JLVR0UlhRwJLgDNSSlvraUlS2xTdo5gFfBJYExHR\n3XYAcGpE/A0wJqWUhi5ySLHUe00OKX4ImDFkWx+wHvjerkICHFIstUFjQ4pTSpuB5wZvi4jNwKaU\n0voijyVp+KjizMxd7kVIGjn2+aPwUkpfqaIRSe3ltR6SsgwKSVkGhaQsg0JSlkEhKcugkJRlUEjK\nMigkZRkUkrJG5JDiRx99tPGajz32WOM1AW677bbGa06ePLnxmqeddlrjNfv6+hqvCTB//vye1N0T\n9ygkZRkUkrIMCklZBoWkLINCUpZBISnLoJCUZVBIyioUFBGxqDuUePDtufxKScNZmTMz1wGnAzvm\nemyrrh1JbVQmKLallN6svBNJrVXmGMWxEfFaRLwYEcsj4qjKu5LUKkWD4ilgPnAWsBCYAvw8IsZV\n3JekFik6KWzloLvrImIV8DJwAdD8ZYySGrFPl5mnlAYi4gXgmD19nUOKpd5rckjxTiLiIDohcfue\nvs4hxVLv7cuQ4qLnUXw/Ik6NiKMj4k+Be4GtwIrMUknDWNE9iiOBO4FJwJvA48DJKaVNVTcmqT2K\nHsz0oIK0H/JaD0lZBoWkLINCUpZBISnLoJCUZVBIyjIoJGUZFJKyDApJWQ4pHuZ68b32YkhxL/T3\n9/e6hdZwj0JSlkEhKcugkJRlUEjKMigkZRkUkrIMCklZBoWkrMJBERGfiog7IuKtiPggItZGhB+x\nLY1ghc7MjIiJwBPAw3Smhb0FHAu8U31rktqi6Cnc1wCvpJQWDNr2coX9SGqhoi89zgVWR8RdEbEx\nItZExILsKknDWtGgmApcDvwGOBO4FfhBRHyj6sYktUfRlx6jgFUppe9276+NiOl0JpvfUWlnklqj\naFBsANYP2bYe+Is9LXJIsdR7TQ4pfgI4bsi248gc0HRIsdR7jQ0pBhYDJ0fEtRHxxxFxEbAAuKXg\n40gaRgoFRUppNTAXmAc8C3wHuDKl9O819CapJQp/FF5K6afAT2voRVJLea2HpCyDQlKWQSEpy6CQ\nlGVQSMoyKCRlGRSSsgwKSVkGhaSsETmk+Kqrrup1C43pxZDiXtScPXt24zX3p+dRjnsUkrIMCklZ\nBoWkLINCUpZBISnLoJCUZVBIyjIoJGUVCoqI+G1EbN/F7ea6GpTUe0XPzDwBOGDQ/RnAg8BdlXUk\nqXUKBUVKadPg+xFxLvBiSum/Ku1KUquUPkYREaOBi4F/ra4dSW20Lwcz5wITgGUV9SKppfYlKC4D\n7k8pvV5VM5LaqdRl5hHxaeAM4M/35usdUiz1XpNDine4DNjIXk4Mc0ix1HtNDikmIgKYD/SllLYX\nXS9p+ClzjOIM4Cjgtop7kdRSZYYU/yc7n3QlaYTzWg9JWQaFpCyDQlJWq4Ni6Hu+Tbj77rsbr/ns\ns882XvONN95ovObmzZsbr9mL77MXz6G6f1cMiiHuueeexmuuW7eu8ZoGRX168Rzar4NCUjsYFJKy\nDApJWXXPHh0LsH79+lKLBwYGWLNmTeF177//fql6AO+99x5r164tvG7Dhg2la27ZsqX0+rLf67Zt\n20qv/eijj0qt2759e+m1vfg+yzwPoPxzaIfx48cXXlP2d2XQ7+bYPX1dpJQKP/jeioiLgB/VVkBS\nVS5OKd25u3+sOygmAWcB/cCW2gpJKmssMBlYOfSjLgerNSgkjQwezJSUZVBIyjIoJGUZFJKyWhkU\nEfGt7vjCDyPiqYg4seZ6p0TETyLite6IxDl11uvWvDYiVkXEexGxMSLujYjP1FxzYUSsjYiB7u3J\niDi7zppD6l/T/fneVHOdRbsYe/lcnTW7dT8VEXdExFsR8UH3Z13bh8U2OeKzdUEREV8HbgQWAZ8H\n1gIrI+LQGsuOA34FXAE09TbQKcDNwEl0Pl5wNPBgRHyixpqvAt8GjgdmAY8AP46IaTXWBKAb9t+k\n8//ZhHXAYcDh3duX6ywWEROBJ4CP6JwSMA34W+CdGsuewO+/v8OBr9J5/lY/4jOl1Kob8BTwj4Pu\nB/A/wN81VH87MKcH3/eh3dpfbrjuJuCvaq5xEPAb4CvAz4Cbaq63CFjT8M/xe8BjTT9vhvSwBHih\njsdu1R5Fd0zhLODhHdtS5yfwEPDFXvXVkIl0/hq83USxiBgVERcCBwK/qLncD4H7UkqP1FxnsGO7\nLyVfjIjlEXFUzfXOBVZHxF3dl5JrImJBzTV/p+4Rn60KCjp/VQ+gMzNksI10dq1GpO4IhCXA4yml\nWl9LR8T0iHifzi7yUmBuSun5GutdCHwOuLauGrvwFJ2REmcBC4EpwM8jYlyNNacCl9PZczoTuBX4\nQUR8o8aag9U64rPui8K0d5YCnwW+1ECt54GZdJ5U5wO3R8SpdYRFRBxJJwDPSCltrfrxdyeltHLQ\n3XURsQp4GbiA+sZMjAJWpZS+272/NiKm0wmqO2qqOVitIz7btkfxFvAxnYNQgx0GjMgZpxFxC3AO\n8GcppfKXoO6llNK2lNJLKaVfppS+Q+fg4pU1lZsFfBJYExFbI2IrMBu4MiL+r7snVbuU0gDwAnBM\njWU2AEMvk14PfLrGmsBOIz7/ua4arQqK7l+dZ4DTd2zrPplOB57sVV916YbE14DTUkqv9KiNUcCY\nmh77IWAGnZceM7u31cByYGb3+FPtIuIgOiFRZxA/ARw3ZNtxdPZk6lZoxGcZbXzpcRPQFxHPAKuA\nq+kccOurq2D3tesxdN5hAZgaETOBt1NKr9ZUcykwD5gDbI6IHXtRAymlWq60jYgbgPuBV4DxdA5+\nzabzmrpyKaXNwE7HXCJiM7AppVTuQ0r2QkR8H7iPzi/pHwHXA1uBOj9YcjHwRERcS+ftyZOABcBf\n11izuRGfvXw7Zw9v81xB59L0D+kckT+h5nqz6bw1+fGQ27/VWHNX9T4G/rLGmv8CvNT9ub4OPAh8\npeH/20eo/+3RFXTeUv+QTijeCUxp4Hs7B/g18AHw38BlDdT8avd5c0yddbzMXFJWq45RSGong0JS\nlkEhKcugkJRlUEjKMigkZRkUkrIMCklZBoWkLINCUpZBISnLoJCU9f/AHV+G/Uc7DAAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xdc4790>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.figure(1, figsize=(3, 3))\n",
    "plt.imshow(digits.images[-1], cmap=plt.cm.gray_r, interpolation='nearest')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "今度はCross Validationを使ってみます。`cv`ではデータの分割数(foldの数)を指定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.97 (+/- 0.03)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "scores = cross_val_score(clf, digits.data, digits.target, cv=5)\n",
    "print(\"Accuracy: %0.2f (+/- %0.2f)\" % (scores.mean(), scores.std() * 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Search Model Parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上記ではモデルのパラメータを固定で指定しましたが(`gamma=0.001`など)、実際どんなパラメータを設定すべきかは非常に悩ましい問題です。  \n",
    "最適なパラメータを探すため、各パラメータが取りうる範囲を決め、その組み合わせを試していくという手法があります。これをGrid Searchと呼びますが、scikit-learnではこれを行うための[Grid Search](http://scikit-learn.org/stable/modules/grid_search.html#grid-search)モジュールが提供されています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVC(C=10, cache_size=200, class_weight=None, coef0=0.0,\n",
      "  decision_function_shape=None, degree=3, gamma=0.001, kernel='rbf',\n",
      "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
      "  tol=0.001, verbose=False)\n",
      "0.972 (+/-0.008) for {'gamma': 0.001, 'kernel': 'rbf', 'C': 1}\n",
      "0.948 (+/-0.011) for {'gamma': 0.0001, 'kernel': 'rbf', 'C': 1}\n",
      "0.973 (+/-0.007) for {'gamma': 0.001, 'kernel': 'rbf', 'C': 10}\n",
      "0.959 (+/-0.011) for {'gamma': 0.0001, 'kernel': 'rbf', 'C': 10}\n",
      "0.973 (+/-0.007) for {'gamma': 0.001, 'kernel': 'rbf', 'C': 100}\n",
      "0.963 (+/-0.009) for {'gamma': 0.0001, 'kernel': 'rbf', 'C': 100}\n",
      "0.949 (+/-0.010) for {'kernel': 'linear', 'C': 1}\n",
      "0.949 (+/-0.010) for {'kernel': 'linear', 'C': 10}\n",
      "0.949 (+/-0.010) for {'kernel': 'linear', 'C': 100}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "\n",
    "candidates = [{'kernel': ['rbf'], 'gamma': [1e-3, 1e-4], 'C': [1, 10, 100]},\n",
    "              {'kernel': ['linear'], 'C': [1, 10, 100]}]\n",
    "\n",
    "clf = GridSearchCV(SVC(C=1), candidates, cv=5)\n",
    "clf.fit(digits.data, digits.target)\n",
    "\n",
    "print(clf.best_estimator_)\n",
    "\n",
    "for params, mean_, std_ in zip(clf.cv_results_[\"params\"], clf.cv_results_[\"mean_test_score\"], clf.cv_results_[\"std_test_score\"]):\n",
    "    print(\"%0.3f (+/-%0.03f) for %r\" % (mean_, std_ / 2, params))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ensemble Lerning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "学習の結果、優秀なモデルができることもあればそうでないこともあります。\n",
    "特徴量が増える、つまり多次元になるほど「よさそうに見えるパラメーターの組み合わせ」は増えていくので、何度も学習させてみないとなかなか最適と思える結果にはたどり着けなくなります。\n",
    "\n",
    "こうした問題に対応するため、複数のモデルを別々に学習させ、最終的にはそれらの組み合わせで決めることで精度を上げるという手法があります。  \n",
    "これがアンサンブル学習と呼ばれる手法です。「複数のモデル」は、同じモデルのこともあれば(Bagging)それぞれ異なるモデルを使うこともあります(Boosting)。\n",
    "\n",
    "scikit-learnでは、[ensemble](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.ensemble)によってこのアンサンブル学習を行うことができます。以下では、Baggingによって10個のモデルを並列で学習させ、その組み合わせで決めるモデルを作成しています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.97 (+/- 0.07)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import BaggingClassifier\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "base_clf = svm.SVC()\n",
    "bagging_clf = BaggingClassifier(base_estimator=clf, n_estimators=10, max_samples=0.9, max_features=2, n_jobs=4)\n",
    "\n",
    "scores = cross_validation.cross_val_score(clf, iris.data, iris.target, cv=5)\n",
    "print(\"Accuracy: %0.2f (+/- %0.2f)\" % (scores.mean(), scores.std() * 2))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`n_jobs`により並列で学習させる際のプロセス数を簡単に調整することができ、高速な学習が可能です"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate Training Result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "学習した結果、つまりモデルの精度はどのように測ればよいでしょうか。\n",
    "単純には予測と実際の2つを比較することで精度を算出することができます。ただ、この手法だと例えば「90%がAで10%がB」というデータがあった場合、常にAという単純なモデルが生まれていたとしてもその精度は90%となってしまいます。\n",
    "\n",
    "こうした問題を防ぐため、分類結果を以下のようにまとめ評価を行う混同行列(confusion matrix)があります。\n",
    "\n",
    "![confusion matrix](./pictures/confusion_matrix.jpeg)\n",
    "\n",
    "ここでは、以下3点の観点が重要です。\n",
    "\n",
    "* 正確度(Accuracy、単に精度といった場合この値): 全データのうち、予測=実際だったものの割合\n",
    "* 適合率(Precision): 真と予測したものについて、実際真だったものの割合\n",
    "* 再現率(Recall): 実際真だったものについて、予測て真だったものの割合\n",
    "\n",
    "先ほどの例だと、偽とは予測しないため再現率は常に1になりますが、その分予測したもののうち間違いだったものが増えるため、データが増えるにつれ適合率が悪化します。適合率を上げる場合は逆にほぼ間違いないもの以外を予測しなければ良いですが、その分本当は真だったものが増えるため再現率が下がることになります。\n",
    "\n",
    "このように適合率と再現率はトレードオフの関係にあり、良いモデルとは基本的にはこの2つの値のバランスが取れているものになります。このバランスを評価する値としてF値があり、これらの値を図ることがモデルの精度を測る上で重要になります。\n",
    "\n",
    "scikit-learnでは、[metrics](http://scikit-learn.org/stable/modules/classes.html#sklearn-metrics-metrics)を利用することでこれらの値を簡単に参照することができます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2 2]\n",
      " [1 3]]\n",
      "0.6\n",
      "0.75\n",
      "0.625\n",
      "0.666666666667\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import precision_score\n",
    "from sklearn.metrics import recall_score\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import f1_score\n",
    "\n",
    "actual = [1, 0, 1, 1, 0, 0, 0, 1]\n",
    "predict = [0, 0, 1, 1, 0, 1, 1, 1]\n",
    "\n",
    "c_mx = confusion_matrix(actual, predict)\n",
    "print(c_mx)\n",
    "\n",
    "# calculate each score\n",
    "print(precision_score(actual, predict))\n",
    "print(recall_score(actual, predict))\n",
    "print(accuracy_score(actual, predict))\n",
    "print(f1_score(actual, predict))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`classification_report`を使用することで、簡単に一覧表を取得できます(単純に値だけ取得したい場合は`precision_recall_fscore_support`)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "    class 0       0.50      1.00      0.67         1\n",
      "    class 1       0.00      0.00      0.00         1\n",
      "    class 2       1.00      0.67      0.80         3\n",
      "\n",
      "avg / total       0.70      0.60      0.61         5\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "actual = [0, 1, 2, 2, 2]\n",
    "predict = [0, 0, 2, 2, 1]\n",
    "target_names = [\"class 0\", \"class 1\", \"class 2\"]\n",
    "print(classification_report(actual, predict, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Store the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "学習したモデルはファイルとして出力し保存しておくことができます。  \n",
    "以下は、標準の`pickle`を使う手法です。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import svm\n",
    "from sklearn import datasets\n",
    "\n",
    "clf = svm.SVC()\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data, iris.target\n",
    "clf.fit(X, y)\n",
    "\n",
    "import pickle\n",
    "s = pickle.dumps(clf)  #serialize model data\n",
    "clf2 = pickle.loads(s) #load serialized model data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "このほか、`sklearn.externals`の`joblib`を利用しファイルに保管することもできます。大規模なモデルなどにはこちらの方がよいでしょう。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.externals import joblib\n",
    "\n",
    "joblib.dump(clf, \"data/model.pkl\") \n",
    "clf = joblib.load(\"data/model.pkl\") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上で解説は終了です。scikit-learnは[公式ドキュメント](http://scikit-learn.org/stable/index.html)が充実しているので、そちらもご参照ください。"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}