{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Машинное обучение, ФКН ВШЭ // Семинар 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:18.309671Z",
     "start_time": "2019-09-28T19:10:14.654448Z"
    },
    "cell_style": "center"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "%pylab inline\n",
    "\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "from tqdm import tqdm\n",
    "from sklearn.datasets import fetch_20newsgroups\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Работа с текстовыми данными\n",
    "\n",
    "Как правило, модели машинного обучения действуют в предположении, что матрица \"объект-признак\" является вещественнозначной, поэтому при работе с текстами сперва для каждого из них необходимо составить его признаковое описание. Для этого широко используются техники векторизации, tf-idf и пр. Рассмотрим их на примере [датасета](https://www.dropbox.com/s/f9xsff8xluriy95/banki_responses.json.bz2?dl=0) отзывов о банках.\n",
    "\n",
    "Сперва загрузим данные:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:18.959288Z",
     "start_time": "2019-09-28T19:10:18.346983Z"
    }
   },
   "outputs": [],
   "source": [
    "data = fetch_20newsgroups(subset='all', categories=['comp.graphics', 'sci.med'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Данные содержат тексты новостей, которые надо классифицировать на разделы."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:18.987312Z",
     "start_time": "2019-09-28T19:10:18.964475Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['comp.graphics', 'sci.med']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['target_names']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:19.012942Z",
     "start_time": "2019-09-28T19:10:18.997929Z"
    }
   },
   "outputs": [],
   "source": [
    "texts = data['data']\n",
    "target = data['target']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Например:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:19.060074Z",
     "start_time": "2019-09-28T19:10:19.019838Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'From: dyer@spdcc.com (Steve Dyer)\\nSubject: Re: Analgesics with Diuretics\\nOrganization: S.P. Dyer Computer Consulting, Cambridge MA\\n\\nIn article <ofk=lve00WB2AvUktO@andrew.cmu.edu> Lawrence Curcio <lc2b+@andrew.cmu.edu> writes:\\n>I sometimes see OTC preparations for muscle aches/back aches that\\n>combine aspirin with a diuretic.\\n\\nYou certainly do not see OTC preparations advertised as such.\\nThe only such ridiculous concoctions are nostrums for premenstrual\\nsyndrome, ostensibly to treat headache and \"bloating\" simultaneously.\\nThey\\'re worthless.\\n\\n>The idea seems to be to reduce\\n>inflammation by getting rid of fluid. Does this actually work? \\n\\nThat\\'s not the idea, and no, they don\\'t work.\\n\\n-- \\nSteve Dyer\\ndyer@ursa-major.spdcc.com aka {ima,harvard,rayssd,linus,m2c}!spdcc!dyer\\n'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "texts[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:19.164963Z",
     "start_time": "2019-09-28T19:10:19.131795Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'sci.med'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['target_names'][target[0]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bag-of-words\n",
    "\n",
    "Самый очевидный способ формирования признакового описания текстов — векторизация. Пусть у нас имеется коллекция текстов $D = \\{d_i\\}_{i=1}^l$ и словарь всех слов, встречающихся в выборке $V = \\{v_j\\}_{j=1}^d.$ В этом случае некоторый текст $d_i$ описывается вектором $(x_{ij})_{j=1}^d,$ где\n",
    "$$x_{ij} = \\sum_{v \\in d_i} [v = v_j].$$\n",
    "\n",
    "Таким образом, текст $d_i$ описывается вектором количества вхождений каждого слова из словаря в данный текст."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:19.833837Z",
     "start_time": "2019-09-28T19:10:19.175011Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "vectorizer = CountVectorizer(encoding='utf8', min_df=1)\n",
    "_ = vectorizer.fit(texts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Результатом является разреженная матрица."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:19.851338Z",
     "start_time": "2019-09-28T19:10:19.837509Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<1x32548 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 86 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectorizer.transform(texts[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:19.872060Z",
     "start_time": "2019-09-28T19:10:19.854688Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0 86]\n",
      "[ 3905  3983  4143  4345  4665  4701  4712  5074  5176  5198  5242  5619\n",
      "  5870  6348  6984  7232  7630  8267  8451  8460  8682  8733  8916  9557\n",
      " 10811 10812 10901 10933 10971 11312 11488 13133 13226 13463 13866 14726\n",
      " 14806 15682 15805 15952 16147 18002 18031 18373 18740 18781 18790 18936\n",
      " 20420 21036 21164 21166 21494 21518 21622 21769 21839 21856 23589 23602\n",
      " 24556 24592 24803 25502 25513 26464 26474 27021 27398 27518 27940 28199\n",
      " 28286 28687 29187 29189 29264 29300 29500 29837 30702 31915 32005 32052\n",
      " 32095 32392]\n",
      "[2 1 1 1 1 2 2 1 1 1 1 1 1 1 1 1 1 2 2 1 1 1 1 1 1 1 1 1 1 6 2 1 2 1 1 1 1\n",
      " 2 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 2 1 2 1 2 1 1 1 2 1 1 1 3 2 1 2 1\n",
      " 2 3 2 1 3 1 1 2 2 1 1 1]\n"
     ]
    }
   ],
   "source": [
    "print(vectorizer.transform(texts[:1]).indptr)\n",
    "print(vectorizer.transform(texts[:1]).indices)\n",
    "print(vectorizer.transform(texts[:1]).data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TF-IDF\n",
    "\n",
    "Ещё один способ работы с текстовыми данными — [TF-IDF](https://en.wikipedia.org/wiki/Tf–idf) (**T**erm **F**requency–**I**nverse **D**ocument **F**requency). Рассмотрим коллекцию текстов $D$.  Для каждого уникального слова $t$ из документа $d \\in D$ вычислим следующие величины:\n",
    "\n",
    "1. Term Frequency – количество вхождений слова в отношении к общему числу слов в тексте:\n",
    "$$\\text{tf}(t, d) = \\frac{n_{td}}{\\sum_{t \\in d} n_{td}},$$\n",
    "где $n_{td}$ — количество вхождений слова $t$ в текст $d$.\n",
    "1. Inverse Document Frequency\n",
    "$$\\text{idf}(t, D) = \\log \\frac{\\left| D \\right|}{\\left| \\{d\\in D: t \\in d\\} \\right|},$$\n",
    "где $\\left| \\{d\\in D: t \\in d\\} \\right|$ – количество текстов в коллекции, содержащих слово $t$.\n",
    "\n",
    "Тогда для каждой пары (слово, текст) $(t, d)$ вычислим величину:\n",
    "\n",
    "$$\\text{tf-idf}(t,d, D) = \\text{tf}(t, d)\\cdot \\text{idf}(t, D).$$\n",
    "\n",
    "Отметим, что значение $\\text{tf}(t, d)$ корректируется для часто встречающихся общеупотребимых слов при помощи значения $\\text{idf}(t, D)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:20.548074Z",
     "start_time": "2019-09-28T19:10:19.877266Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "vectorizer = TfidfVectorizer(encoding='utf8', min_df=1)\n",
    "_ = vectorizer.fit(texts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "На выходе получаем разреженную матрицу."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:20.569113Z",
     "start_time": "2019-09-28T19:10:20.550328Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<1x32548 sparse matrix of type '<class 'numpy.float64'>'\n",
       "\twith 86 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectorizer.transform(texts[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:20.601343Z",
     "start_time": "2019-09-28T19:10:20.574776Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0 86]\n",
      "[32392 32095 32052 32005 31915 30702 29837 29500 29300 29264 29189 29187\n",
      " 28687 28286 28199 27940 27518 27398 27021 26474 26464 25513 25502 24803\n",
      " 24592 24556 23602 23589 21856 21839 21769 21622 21518 21494 21166 21164\n",
      " 21036 20420 18936 18790 18781 18740 18373 18031 18002 16147 15952 15805\n",
      " 15682 14806 14726 13866 13463 13226 13133 11488 11312 10971 10933 10901\n",
      " 10812 10811  9557  8916  8733  8682  8460  8451  8267  7630  7232  6984\n",
      "  6348  5870  5619  5242  5198  5176  5074  4712  4701  4665  4345  4143\n",
      "  3983  3905]\n",
      "[0.02775776 0.030364   0.10357777 0.10097852 0.05551552 0.08913878\n",
      " 0.0751644  0.05521512 0.02543534 0.07527191 0.05440145 0.04646966\n",
      " 0.07125018 0.0955189  0.01649287 0.12280349 0.25018259 0.0710195\n",
      " 0.09802838 0.05646637 0.09712269 0.10057076 0.09482619 0.08113136\n",
      " 0.04893556 0.09057384 0.23738007 0.11869004 0.18429156 0.12343474\n",
      " 0.01703927 0.04332515 0.12343474 0.01848065 0.05632337 0.12343474\n",
      " 0.03769659 0.0854585  0.06358581 0.07172143 0.09057384 0.12343474\n",
      " 0.0833942  0.10531547 0.08659639 0.08913878 0.01987357 0.08913878\n",
      " 0.1325496  0.08719619 0.07172143 0.06089811 0.01649287 0.04338328\n",
      " 0.09689049 0.04826154 0.48428276 0.03809625 0.03886054 0.03418115\n",
      " 0.11500976 0.11869004 0.10357777 0.08984071 0.12343474 0.04302873\n",
      " 0.09925063 0.06487218 0.16490298 0.06679843 0.0833942  0.03432907\n",
      " 0.11869004 0.02634071 0.05392398 0.10946036 0.03071828 0.03099918\n",
      " 0.02850602 0.15610804 0.03925604 0.10531547 0.0819996  0.10946036\n",
      " 0.05913563 0.23738007]\n"
     ]
    }
   ],
   "source": [
    "print(vectorizer.transform(texts[:1]).indptr)\n",
    "print(vectorizer.transform(texts[:1]).indices)\n",
    "print(vectorizer.transform(texts[:1]).data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Заметим, что оба метода возвращают вектор длины 32548 (размер нашего словаря)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Заметим, что одно и то же слово может встречаться в различных формах (например, \"сотрудник\" и \"сотрудника\"), но описанные выше методы интерпретируют их как различные слова, что делает признаковое описание избыточным. Устранить эту проблему можно при помощи **лемматизации** и **стемминга**.\n",
    "\n",
    "### Стемминг\n",
    "\n",
    "[**Stemming**](https://en.wikipedia.org/wiki/Stemming) –  это процесс нахождения основы слова. В результате применения данной процедуры однокоренные слова, как правило, преобразуются к одинаковому виду.\n",
    "\n",
    "**Примеры стемминга:**\n",
    "\n",
    "| Word        | Stem           |\n",
    "| ----------- |:-------------:|\n",
    "| вагон | вагон |\n",
    "| вагона | вагон |\n",
    "| вагоне | вагон |\n",
    "| вагонов | вагон |\n",
    "| вагоном | вагон |\n",
    "| вагоны | вагон |\n",
    "| важная | важн |\n",
    "| важнее | важн |\n",
    "| важнейшие | важн |\n",
    "| важнейшими | важн |\n",
    "| важничал | важнича |\n",
    "| важно | важн |\n",
    "\n",
    "[Snowball](http://snowball.tartarus.org/) – фрэймворк для написания алгоритмов стемминга. Алгоритмы стемминга отличаются для разных языков и используют знания о конкретном языке – списки окончаний для разных чистей речи, разных склонений и т.д. Пример алгоритма для русского языка – [Russian stemming](http://snowballstem.org/algorithms/russian/stemmer.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:21.109437Z",
     "start_time": "2019-09-28T19:10:20.604439Z"
    }
   },
   "outputs": [],
   "source": [
    "import nltk\n",
    "stemmer = nltk.stem.snowball.RussianStemmer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:21.130340Z",
     "start_time": "2019-09-28T19:10:21.112772Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "машин обучен\n"
     ]
    }
   ],
   "source": [
    "print(stemmer.stem(u'машинное'), stemmer.stem(u'обучение'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:25.274844Z",
     "start_time": "2019-09-28T19:10:21.135760Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [00:04<00:00, 242.80it/s]\n"
     ]
    }
   ],
   "source": [
    "stemmer = nltk.stem.snowball.EnglishStemmer()\n",
    "\n",
    "def stem_text(text, stemmer):\n",
    "    tokens = text.split()\n",
    "    return ' '.join(map(lambda w: stemmer.stem(w), tokens))\n",
    "\n",
    "stemmed_texts = []\n",
    "for t in tqdm(texts[:1000]):\n",
    "    stemmed_texts.append(stem_text(t, stemmer))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:25.290119Z",
     "start_time": "2019-09-28T19:10:25.279611Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From: dyer@spdcc.com (Steve Dyer)\n",
      "Subject: Re: Analgesics with Diuretics\n",
      "Organization: S.P. Dyer Computer Consulting, Cambridge MA\n",
      "\n",
      "In article <ofk=lve00WB2AvUktO@andrew.cmu.edu> Lawrence Curcio <lc2b+@andrew.cmu.edu> writes:\n",
      ">I sometimes see OTC preparations for muscle aches/back aches that\n",
      ">combine aspirin with a diuretic.\n",
      "\n",
      "You certainly do not see OTC preparations advertised as such.\n",
      "The only such ridiculous concoctions are nostrums for premenstrual\n",
      "syndrome, ostensibly to treat headache and \"bloating\" simultaneously.\n",
      "They're worthless.\n",
      "\n",
      ">The idea seems to be to reduce\n",
      ">inflammation by getting rid of fluid. Does this actually work? \n",
      "\n",
      "That's not the idea, and no, they don't work.\n",
      "\n",
      "-- \n",
      "Steve Dyer\n",
      "dyer@ursa-major.spdcc.com aka {ima,harvard,rayssd,linus,m2c}!spdcc!dyer\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(texts[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:25.306834Z",
     "start_time": "2019-09-28T19:10:25.294638Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "from: dyer@spdcc.com (steve dyer) subject: re: analges with diuret organization: s.p. dyer comput consulting, cambridg ma in articl <ofk=lve00wb2avukto@andrew.cmu.edu> lawrenc curcio <lc2b+@andrew.cmu.edu> writes: >i sometim see otc prepar for muscl aches/back ach that >combin aspirin with a diuretic. you certain do not see otc prepar advertis as such. the onli such ridicul concoct are nostrum for premenstru syndrome, ostens to treat headach and \"bloating\" simultaneously. they'r worthless. >the idea seem to be to reduc >inflamm by get rid of fluid. doe this actual work? that not the idea, and no, they don't work. -- steve dyer dyer@ursa-major.spdcc.com aka {ima,harvard,rayssd,linus,m2c}!spdcc!dy\n"
     ]
    }
   ],
   "source": [
    "print(stemmed_texts[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как видим, стеммер работает не очень быстро и запускать его для всей выборки достаточно накладно."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Лемматизация\n",
    "\n",
    "[Лемматизация](https://en.wikipedia.org/wiki/Lemmatisation) — процесс приведения слова к его нормальной форме (**лемме**):\n",
    "- для существительных — именительный падеж, единственное число;\n",
    "- для прилагательных — именительный падеж, единственное число, мужской род;\n",
    "- для глаголов, причастий, деепричастий — глагол в инфинитиве."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Например, для русского языка есть библиотека pymorphy2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:25.442948Z",
     "start_time": "2019-09-28T19:10:25.311429Z"
    }
   },
   "outputs": [],
   "source": [
    "import pymorphy2\n",
    "morph = pymorphy2.MorphAnalyzer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:25.468072Z",
     "start_time": "2019-09-28T19:10:25.447023Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Parse(word='играющих', tag=OpencorporaTag('PRTF,impf,tran,pres,actv plur,gent'), normal_form='играть', score=0.16666666666666666, methods_stack=((<DictionaryAnalyzer>, 'играющих', 303, 34),))"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "morph.parse('играющих')[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Сравним работу стеммера и лемматизатора на примере:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:25.489016Z",
     "start_time": "2019-09-28T19:10:25.474203Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "игра\n"
     ]
    }
   ],
   "source": [
    "stemmer = nltk.stem.snowball.RussianStemmer()\n",
    "print(stemmer.stem('играющих'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:25.502206Z",
     "start_time": "2019-09-28T19:10:25.493051Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "играть\n"
     ]
    }
   ],
   "source": [
    "print(morph.parse('играющих')[0].normal_form)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Трансформация признаков и целевой переменной"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Разберёмся, как может влиять трансформация признаков или целевой переменной на качество модели. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Логарифмирование \n",
    "\n",
    "Воспользуется датасетом с ценами на дома, с которым мы уже сталкивались ранее ([House Prices: Advanced Regression Techniques](https://www.kaggle.com/c/house-prices-advanced-regression-techniques/overview))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:27.600964Z",
     "start_time": "2019-09-28T19:10:25.507394Z"
    }
   },
   "outputs": [],
   "source": [
    "!wget https://www.dropbox.com/s/h5x86yvaf384vnt/train.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:27.669451Z",
     "start_time": "2019-09-28T19:10:27.604504Z"
    }
   },
   "outputs": [],
   "source": [
    "data = pd.read_csv('train.csv')\n",
    "\n",
    "data = data.drop(columns=[\"Id\"])\n",
    "y = data[\"SalePrice\"]\n",
    "X = data.drop(columns=[\"SalePrice\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Посмотрим на распределение целевой переменной "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.386207Z",
     "start_time": "2019-09-28T19:10:27.671842Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuIAAAFNCAYAAABFQQqZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZhc13nf+e9b1V29b+hurI2VABdwAWVCJG1LsiPaJmiPBTmRYlKOLE/kMJ6Ik3miOBY5kyg2H9Exndgcz0SyzViyGckMSMuLIIcSbQ1lSZREkCDFDaBANrGDAHtfqpeqrq53/qhbYLHZS3Wjqm5V9e/zPHhUfeve95xbFG6/OPWec8zdERERERGR0oqE3QERERERkdVIibiIiIiISAiUiIuIiIiIhECJuIiIiIhICJSIi4iIiIiEQIm4iIiIiEgIlIiLiIiIiIRAibhIDjM7aWY/FVLbf2ZmnwmjbRERESk9JeIiBWJm0bD7ICIi+TOzmrD7IKubEnGRgJl9EdgCfNXM4mb2G2b2F2Z2wcxGzezbZnZ1zvl/ZmZ/aGaPmdkE8I/MrNPMvmpmY2b2jJl9xsyezLnmSjP7ezMbMrNjZvZPg+N3Ar8E/EbQ9ldLfPsiIlXDzO42s9fNbNzMjprZLwTHf8XMvmtmD5jZIPCbwfF/bmavmNmwmT1uZltzYv2BmZ0JnuvPmtl7w7krqUZKxEUC7v5R4DTw8+7e7O6/C3wN2AWsBZ4D/nzOZR8B7gNagCeBzwITwHrgY8EfAMysCfh74OEg3u3A58xst7s/GMT+3aDtny/ajYqIVL/XgfcCbcBvAV8ysw3BezcBx4F1wH1mth/4P4F/DHQD3wH+R06sZ4DrgTVknt9/YWb1pbgJqX7m7mH3QaRsmNlJ4Ffd/RvzvNcODAPt7j5qZn8GRNz9l4P3o8A0cI27HwuOfQb4SXd/j5n9InCXu783J+YfA2+4+28F8c66+78v6k2KiKwyZvY88B+BDuBed9+S897XgC+7++eDnyNAHLjK3U/NE2uYzHP9hZJ0XqqaRsRFFmBmUTP7neDrzTHgZPBWV85pZ3JedwM1c47lvt4K3GRmI9k/ZMpR1he+9yIiq5eZ/bKZPZ/zrL2Gt57dZ+acvhX4g5xzhwADNgWxfj0oWxkN3m/j7b8HRFZMkxRE3i73K6KPAPuBnyKThLeRGRG3Bc7vB1JAD/BqcGxzzvtngG+5+0/n0baIiKxAUN/934BbgO+7+2wwIp59ds991p4B7nP3uaWHBPXgvxHEOuLu6WBE3OaeK7ISGhEXebs3gR3B6xYgAQwCjcBvL3ahu88CfwX8ppk1mtmVwC/nnPK3wOVm9lEzqw3+vNvMrpqnbRERWZkmMsl2P4CZ/a9kRsQX8kfAPdnJ+GbWZmYfDt5rITPA0g/UmNmngdZidVxWHyXiIm/3n4B/H3z9uAY4BZwDjgJP5XH9XWRGzi8AXyQz4ScB4O7jwM+QmaT5RnDO/UBdcO3ngd3B16N/U6gbEhFZTdz9KPB7wPfJDHBcC3x3kfP/msyz+EBQhvgycFvw9uPA18l8y3mKzDyguaUtIiumyZoiRWRm9wPr3f1jS54sIiIiq4pGxEUKKFgn/DrLuBH4OPDXYfdLREREyo8ma4oUVguZcpSNZL4S/T3gK6H2SERERMqSSlNEREREREKg0hQRERERkRAoERcRERERCcGqrBHv6urybdu2hd0NEZEVefbZZwfcvTvsfpSSntsiUqkWe2avykR827ZtHD58OOxuiIisiJmdCrsPpabntohUqsWe2SpNEREREREJgRJxEREREZEQKBEXEREREQmBEnERERERkRAoERcRERERCYEScRERERGRECgRFxEREREJgRJxEREREZEQKBEXEREREQmBEnERERERkRAoERcRERERCUFN2B2Q4nv40Ol3HPvITVtC6ImIiIRJvw9EyotGxEVEREREQqBEXEREREQkBErERURERERCkFcibmb7zOyYmfWa2d3zvF9nZo8E7x8ys205790THD9mZrcuFdPMtgcxeoOYscXaMLNaM3vIzF4ys1fM7J6VfhgiIiIiIqWyZCJuZlHgs8BtwG7gDjPbPee0jwPD7r4TeAC4P7h2N3A7cDWwD/icmUWXiHk/8EAQaziIvWAbwIeBOne/FrgB+Je5/xAQERERESlH+YyI3wj0uvtxd08CB4D9c87ZDzwUvP4ycIuZWXD8gLsn3P0E0BvEmzdmcM37gxgEMT+4RBsONJlZDdAAJIGxvD8BEREREZEQ5JOIbwLO5Px8Njg27znungJGgc5Frl3oeCcwEsSY29ZCbXwZmADOA6eB/+LuQ3ncl4iIiIhIaKphsuaNwCywEdgO/Fsz2zH3JDO708wOm9nh/v7+UvdRRERERORt8tnQ5xywOefnnuDYfOecDUpE2oDBJa6d7/gg0G5mNcGod+75C7XxEeDr7j4D9JnZd4G9wPHcDrr7g8CDAHv37vU87ruqaVMHERERkXDlMyL+DLArWM0kRmby5cE55xwEPha8/hDwhLt7cPz2YMWT7cAu4OmFYgbXfDOIQRDzK0u0cZpMXTlm1gTcDPww3w9ARERERCQMS46Iu3vKzO4CHgeiwBfc/YiZ3QscdveDwOeBL5pZLzBEJrEmOO9R4CiQAj7h7rMA88UMmvwUcMDMPgP8IIjNQm2QWX3lT83sCGDAn7r7iyv/SEREREREii+f0hTc/THgsTnHPp3zeprMMoLzXXsfcF8+MYPjx8nUfc89Pm8b7h5fqG0RERERkXJVDZM1RUREREQqjhJxEREREZEQKBEXEREREQmBEnERERERkRAoERcRERERCYEScRERERGRECgRFxEREREJgRJxERHBzPaZ2TEz6zWzu+d5v87MHgneP2Rm23Leuyc4fszMbl0qppndYmbPmdnzZvakme0s9v2JiJQjJeIiIqucmUXJ7FJ8G7AbuMPMds857ePAsLvvBB4A7g+u3U1mp+OrgX3A58wsukTMPwR+yd2vBx4G/n0x709EpFwpERcRkRuBXnc/7u5J4ACwf845+4GHgtdfBm4xMwuOH3D3hLufAHqDeIvFdKA1eN0GvFGk+xIRKWt5bXEvIiJVbRNwJufns8BNC53j7ikzGwU6g+NPzbl2U/B6oZi/CjxmZlPAGHDzfJ0yszuBOwG2bNmyvDuStznyxijPnxnhO68O0N1Sx1UbWpe+SESKTom4iIiU2r8BftbdD5nZvwN+n0xy/jbu/iDwIMDevXu9tF2sDu7OF586xW999Siz6bc+wpt3dPJz124gGrEQeyciSsRFROQcsDnn557g2HznnDWzGjIlJYNLXPuO42bWDexx90PB8UeArxfiJuTt3J3/8JWX+dJTp/mpq9bymx+4mr878iZP/LCPJ3sHGBhP8LEf2xZ2N0VWNdWIrzLHLozx+39/jD/4/17lT548zmQiFXaXRCR8zwC7zGy7mcXITL48OOecg8DHgtcfAp5wdw+O3x6sqrId2AU8vUjMYaDNzC4PYv008EoR723Veuh7J/nSU6e58307ePCje+npaKS+NsrPXruBD+zZSG9/nMOnhsLupsiqphHxVea50yPEEym2dTbxwwvjvPzGGDduXxN2t0QkREHN913A40AU+IK7HzGze4HD7n4Q+DzwRTPrBYbIJNYE5z0KHAVSwCfcfRZgvpjB8X8B/KWZpckk5v+8hLe7Krx8bpTffuyH/NRVa7nntivJzKt9y03b1/Di2RG+8Uof49MztNTXhtRTkdVNI+KriLtzYmCCK9e38tGbt7KmKcbR86Nhd0tEyoC7P+bul7v7Ze5+X3Ds00ESjrtPu/uH3X2nu9/o7sdzrr0vuO4Kd//aYjGD43/t7te6+x53/8ncWHLpJhIp7nr4OTqbY/znD+15RxIOYGb87LUbmEik+ONv6eMXCYsS8VWkP54gnkixvasJM2P3hlZe759gemY27K6JiEiB/N7fvcqpoUn+4PZ30dEUW/C8no5G9vS08d++c5y+sekS9lBEspSIryInBiYA2NHVBMDuDa3Mpp1X3xwPs1siIlIgL5wZ4c++d4KP3rw1r7LD91+5jkQqzd++eL4EvRORuZSIryLH+ydora9hTTBCsqWzkaZYlKPnx0LumYiIXKqZ2TR3/9VLrG2p59/dekVe13S31HHl+ha+9rIScZEwaLLmKpGtD9+5tvlivWDEjKs2tPLSuVFS6XTIPRQRkaU8fOj0O4595KbMZkdfePIEr5wf448/esOyJl/+7LUbeOAbr9I3Ns3a1vqC9VVElqYR8VViIJ68WB+ea/eGVhKpNCf6J0LqmYiIXKrTg5M88I1XufXqddx69fplXXvbNetxh8ePXChS70RkIRoRXyWOD8QB3pGI7+huxoCTg0rERUQqkbvzf/3NS9REIvzWB64B5h85X8gzJ4fpbqnjT797kmgkMz6XHWUXkeLKa0TczPaZ2TEz6zWzu+d5v87MHgneP2Rm23Leuyc4fszMbl0qZrD5w6Hg+CPBRhALtmFmv2Rmz+f8SZvZ9Sv9QKrVqcFJWupr6Jwzgz5WE2Fdaz1nh6dC6pmIiFyKrzz/Bt95bYBP7buC9W0rKy25ZmMbJwYmiGuTN5GSWjIRN7Mo8FngNmA3cIeZ7Z5z2seBYXffCTwA3B9cu5vMpg9XA/uAz5lZdImY9wMPBLGGg9gLtuHuf+7u17v79cBHgRPu/vzyP4rqNhhPsLalbt71ZHs6Gjg7PEVmkzwREakUE4kU9/7tUX5kSzu/dNPWFce5ZlMrTmb3ZREpnXxGxG8Eet39uLsngQPA/jnn7AceCl5/GbjFMhnffuCAuyfc/QTQG8SbN2ZwzfuDGAQxP7hEG7nuCGLJHMOTM3Q0zr+e7KaOBqZmZjk9NFniXomIyKX42svnGZua4T/94+uIRN450JKv9a311NdG9HtApMTyScQ3AWdyfj4bHJv3HHdPAaNA5yLXLnS8ExgJYsxta6E2cv0i8D/yuKdVJZlKE0+kLi5bOFdPRyMAL5zVLpsiIpWity/Oc6dH+LWfuIwr1rdcUiwzY3NHo8oURUqsaiZrmtlNwKS7v7zA+3cCdwJs2bK6JqEMTyYBaF9gRHx9az01EePFMyN8YM/GUnZNRERWIJVOc/CFc3Q2xbjr/TsLErOno5FvvdpHMpVedJlEESmcfEbEzwGbc37uCY7Ne46Z1QBtwOAi1y50fBBoD2LMbWuhNrJuZ5HRcHd/0N33uvve7u7uRW63+owEifiaxvnXlY1GjA1t9byoEXERkYrw/dcHGYgn+V+u20B9bbQgMTd3NJB2ODeiUXGRUsknEX8G2BWsZhIjk/AenHPOQeBjwesPAU94ZubfQeD2YMWT7cAu4OmFYgbXfDOIQRDzK0u0gZlFgH+K6sPnNTQ5A0DHAqUpkBkJeencKKlZbewjIlLOxqdneOKHfVyxroUr1rcWLG7PmkyZ4tlh1YmLlMqSiXhQj30X8DjwCvCoux8xs3vN7APBaZ8HOs2sF/gkcHdw7RHgUeAo8HXgE+4+u1DMINangE8GsTqD2Au2EXgfcMbdj6/kQ6h2wxNJaiJGc93ClUg9wYTN3v54CXsmIiLL9fdH32RmNs3PXruhoHGb62roaKzljOrERUomrxpxd38MeGzOsU/nvJ4GPrzAtfcB9+UTMzh+nMyqKnOPL9bGPwA3L3YPq9nwZJKOxti8SxdmbepoAODFM6NcWcARFhERKZzBeIJnTw3zY5d10t1SV/D4PR2NnNGIuEjJaIv7VWB4MklH0/z14VldzXW01NXwwtmREvVKRESW61uv9hONGO+9vDhznXo6GhiZnGF8eqYo8UXk7ZSIrwLDEwuvIZ4VMePanjZN2BQRKVPnR6f4wekRbtjaQWv94oMrK7W5I1snrvIUkVKomuULZX5j0zNMzcwumYgDXNfTzp985zjTM7MFm4UvIiKF8eC3j+M475szGj7fUoMrtbG9gYhlJmxetUFliiLFpkS8yp0dyoxqLLZiStaenjZSaeeV82O8a0sHMP8DXmvJioiU1ujkDP/j6dNcv7kjr4GVlYrVROhsquPNsUTR2hCRtygRr3LZSTdr8hkR39wOwItnRy8m4iIiEo7cgZCnjg8yPZPmRy+bu6F04XW31NEfVyIuUgqqEa9yZ4YyifhSkzUBNrbV09Uc04RNEZEy89zpYda31rOxrb7obXU11zEUTzKb9qK3JbLaKRGvcmeHp6iridCQR823mbGnp50XzigRFxEpF2+OTXN2eIobtnYsugxtoXS31DHrznCwK7OIFI8S8Sp3dnhyyTXEc13X087xgQktXSUiUiaeOzVMxGBPUD5YbN3NmVLGgXGVp4gUmxLxKnd2eIqOxvyXubpucxvu8NI5LWMoIhK22bTzgzMjXLm+ddHdkQupK9goSHXiIsWnRLzKvTk2TWtD/on4np63JmyKiEi4jvfHiSdS/MiW0oyGAzTGamiKRenXiLhI0SkRr2LJVJrhyRla6vMfRVnTFGPzmgbViYuIlIFjb45TEzF2rWspabtdLXUMaERcpOiUiFex7EO0pW55O7Bdv7mD504P464Z8yIiYTp2YZwd3U3URkv767q7uU4j4iIloES8imUfossZEQfYu7WDN8cS2uJYRCREA/EEgxNJrijxaDhkVk6ZSM4ymUyVvG2R1USJeBXrCxLx5mUm4jdszWzm89zp4YL3SURE8vPqm+MAXLG+9FvNdzdnJmxq5RSR4lIiXsXeGhFfXmnKletbaIpFOXxSibiISFiOXRinqznGmqbibWm/kLdWTtFa4iLFpES8ivWNT2PGspe8qolGuH5LO4dPKREXEQnDVHKWEwMToZSlAHQ0xoiaqU5cpMiUiFex/vEEaxpjRCPL34nthq1rOHZhjOmZ2SL0TEREFvP94wOk0s7l68NJxKMRY01zTCuniBSZEvEq1jeeoDv4enG59m7tIO1wZniywL0SEZGlHDoxRNSMbZ1NofWhqynG0IRKU0SKSYl4Feu/hET8XVvaiRicGlQiLiJSas+fHmFDe33Jly3M1dEUY2gyqaVsRYpIiXgVu5REvKW+livWt3JaibiISEnNpp2Xzo3S09EYaj86GmMkU2kmkypRFCkWJeJVyt3pH0+wtqV+xTFu2NrO6eFJ0hoNEREpmdf6xplMzrK5oyHUfnQ0ZlZrGZ5UeYpIsSgRr1KjUzMkZ9MrHhEH2Lt1DclUmguj0wXsmYiILOaFMyMAbA57RLwps/Tt8ORMqP0QqWZKxKtUdsmpS0nEsxv7nBpSeYqISKk8f2aE1voa1jSXfv3wXBdHxDVhU6Ro8krEzWyfmR0zs14zu3ue9+vM7JHg/UNmti3nvXuC48fM7NalYprZ9iBGbxAzlkcb15nZ983siJm9ZGYrr8eoEtldNddeQiLe09FAa30NpwcnCtUtERFZwvNnRtmzuZ2ILX/p2UKqr43SUBtVaYpIES2ZiJtZFPgscBuwG7jDzHbPOe3jwLC77wQeAO4Prt0N3A5cDewDPmdm0SVi3g88EMQaDmIv1kYN8CXg19z9auAngVX/PVohRsTNjC2dTRoRFxEpsocPnebhQ6f5s++e5Ifnx6iJlMcX1h1NtUrERYoon7/pNwK97n7c3ZPAAWD/nHP2Aw8Fr78M3GJmFhw/4O4Jdz8B9Abx5o0ZXPP+IAZBzA8u0cbPAC+6+wsA7j7o7qt+ind/AUbEAbauaWRkcobRqVX/bxsRkaI7NzKFQ+gTNbM6GmMMT+j5L1Is+ex9vgk4k/PzWeCmhc5x95SZjQKdwfGn5ly7KXg9X8xOYMTdU/Ocv1AblwNuZo8D3WQS/9/N476qWt/4NPW1kWVtb//wodPvOLa1MzNZ6NTgBNf1tBesfyIi8k5ng03UetaEO1Ezq6MxxrEL41pLXKRIyuO7r0tTA7wH+KXgf3/BzG6Ze5KZ3Wlmh83scH9/f6n7WHLZpQvtEmsMN7Q1UBs1Tqs8RUSk6N4YmaKtoXZZgyjF1NFYSyrtxBOppU8WkWXLJxE/B2zO+bknODbvOUHNdhswuMi1Cx0fBNqDGHPbWqiNs8C33X3A3SeBx4AfmXsT7v6gu+91973d3d153HZlu5Tt7XNFI0ZPR6N22BQRKYH+eOKSSwoLqaMpu5a4ylNEiiGfRPwZYFewmkmMzOTLg3POOQh8LHj9IeAJz3yPdRC4PVjxZDuwC3h6oZjBNd8MYhDE/MoSbTwOXGtmjUGC/hPA0fw/guqUGREvzMN8a2cj50enSKbSBYknIiLv5O4MxJN0lVMiriUMRYpqyUQ8qNe+i0zC+wrwqLsfMbN7zewDwWmfBzrNrBf4JHB3cO0R4FEyifHXgU+4++xCMYNYnwI+GcTqDGIv1sYw8Ptkkvvngefc/X+u9AOpFoUaEYfMphJph/OjUwWJJyIi7zQ2nSKZStPdXIaJuFZOESmKvIrQ3P0xMiUfucc+nfN6GvjwAtfeB9yXT8zg+HEyq6rMPb5YG18is4ShAInULKNTMwV7mG9sz8zePzcyxdbOpoLEFBGRtxuIZ1a76iqjRDxWE6GprkaJuEiRlMdsECmYhw+dvrjU4PH+iXlXQlmu1voamutqeGNEW92LiBRLIfZ/KIaOxlotYShSJNWwaorMEZ/OzG5vKtCsezNjY3s9b4yoNEVEpFj64wli0Qit9eU1RtbRGNOIuEiRKBGvQtllpprrogWLuam9gb7xaWZmNWFTpBqZ2T4zO2ZmvWZ29zzv15nZI8H7h8xsW8579wTHj5nZrUvFtIz7zOxVM3vFzP51se+vEgyMJ+hqiV3ysrOF1tEYY2RyhnRaa4mLFJoS8So0kU3E62sLFnNTewNphwujKk8RqTZmFgU+C9wG7AbuMLPdc077ODDs7juBB4D7g2t3k1n56mpgH/A5M4suEfNXyCxHe6W7X0Vmd+VVrz+eKKv68Kz2xlpm3ekLSmdEpHCUiFeh7Ih4UwFHxHMnbIpI1bkR6HX34+6eJJMY759zzn7goeD1l4FbLDN0u5/MjsYJdz8B9AbxFov5vwH3unsawN37inhvFWFmNs3o5EzZ1YfDWyunnBvRfhIihaZEvArFEylqo0YsWrj/vG0NtTTGokrERarTJuBMzs9ng2PznhMsQTtKZonZha5dLOZlwC8Gux1/zcx2Feg+KtZAPIFDWS1dmNXemPl29eywnv8ihaZEvApNJFI019UUtM7QzNjU3qAJmyJSCHXAtLvvBf4b8IX5TjKzO4Nk/XB/f39JO1hq2RVTyrE0JTsirkRcpPCUiFeheCJVsBVTcm1sb+DNsWmmZ2YLHltEQnWOTM12Vk9wbN5zgl2M24DBRa5dLOZZ4K+C138NXDdfp9z9QXff6+57u7u7l3lLlaUc1xDPitVE9I2oSJEoEa9C8WBEvNCyEzaPXRgveGwRCdUzwC4z225mMTKTLw/OOecg8LHg9YeAJ9zdg+O3B6uqbAd2AU8vEfNvgH8UvP4J4NUi3VfFGIgnaW+oJVZTnr+WOxpjGhEXKYLyWqxUCmIikbo4ubKQ1rfVA/Dqm+Ps2dxe8PgiEg53T5nZXcDjQBT4grsfMbN7gcPufhD4PPBFM+sFhsgk1gTnPQocBVLAJ9x9FmC+mEGTvwP8uZn9GyAO/Gqp7rVcDZTpiilZ7Y21nBvWZE2RQlMiXmXcvWgj4h2NMaIRo7c/XvDYIhIud38MeGzOsU/nvJ4GPrzAtfcB9+UTMzg+AvzcJXa5qoxMznDl+vqwu7GgjsYYh08N4e5lt865SCUrz+/AZMWmZ9KknaIk4tGI0dUco/dNJeIiIoWSSM0ST6Roayjc3g+F1t5Yy/RMmsEJ7bApUkhKxKvMeGIGKNz29nN1t9RrRFxEpIDeHM1M1CznRPziWuKqExcpKCXiVWYikVnRpBgj4gBrW+o4MzSplVNERArkjdFMctvWWL6JuNYSFykOJeJVJrurZjET8bTDiYGJosQXEVltzmcT8TIeEW9v0O6aIsWgyZpVphjb2+fKbr/c2xfnqg2tRWlDRKRaPXzo9DuO/cOxPuCtZLccNcSitNTXaERcpMA0Il5lJhIpDGiMFeffWF3NdUQMXutTnbiISCGMTs3QUBst2zXEs3o6GlUjLlJg5f23XpYtnkjRGIsSjRRneanaaITNaxp5XYm4iEhBjE7NlHVZStam9gbtrilSYErEq0x8ujjb2+fa2d1MrxJxEZGCqJREvKejgbPDU2Q2VBWRQlAiXmUmirSZT66da5s5MTBBajZd1HZERFaDkcmZsl4xJauno4F4IsXYVCrsrohUDSXiVSaeKMGI+NpmkrNpTg9p9ryIyKVIptJMzczSXgEj4pvaGwA4o63uRQpGiXiVmUimaK4vfiIOqDxFROQSjU5lNmGrjNKURgDViYsUUF6JuJntM7NjZtZrZnfP836dmT0SvH/IzLblvHdPcPyYmd26VEwz2x7E6A1ixhZrw8y2mdmUmT0f/PmjlX4YlW56ZpbpmXTRS1MuCxLx1/u1lriIyKW4mIhXQGnKpo7MiLhWThEpnCUTcTOLAp8FbgN2A3eY2e45p30cGHb3ncADwP3BtbuB24GrgX3A58wsukTM+4EHgljDQewF2wi87u7XB39+bVmfQBUZmkgC0FykpQuzWutr6WqOcWpQibiIyKUYnco8t8t5DfGsjsZaGmqjWktcpIDyGRG/Eeh19+PungQOAPvnnLMfeCh4/WXgFjOz4PgBd0+4+wmgN4g3b8zgmvcHMQhifnCJNiQwGA8S8SKXpgBs7WzS7poiIpdoJBgRby3Bc/tSmRk9HQ3aXVOkgPJJxDcBZ3J+Phscm/ccd08Bo0DnItcudLwTGAlizG1roTYAtpvZD8zsW2b23jzuqSoNxBMARZ+sCbCts4lTg3oYi4hcitHJGZrraqiJVsaUrU3BEoYiUhiV8Td/ceeBLe7+LuCTwMNm9o69183sTjM7bGaH+/v7S97JUsgm4sWuEQfY1tnIhbFpppKzRW9LRKRaVcoa4lna1EeksPJJxM8Bm3N+7gmOzXuOmdUAbcDgItcudHwQaA9izG1r3jaCspdBACKpmA0AACAASURBVHd/FngduHzuTbj7g+6+1933dnd353HblWcwqBFvqosWva2tXU0AnBpSeYqIyEqNVFgi3tPRyMjkDPGE1hIXKYR8EvFngF3BaiYxMpMvD8455yDwseD1h4AnPLP11kHg9mDFk+3ALuDphWIG13wziEEQ8yuLtWFm3cHkT8xsR9DG8fw/guoxMJ6gNmrU1RQ/Ed/emUnETw6oPEVEZKXGpipjM58srZwiUlhL1jC4e8rM7gIeB6LAF9z9iJndCxx294PA54EvmlkvMEQmsSY471HgKJACPuHuswDzxQya/BRwwMw+A/wgiM1CbQDvA+41sxkgDfyauw+t/COpXIMTyZKUpQBs7cqsJ6uVU0REViaZSpNIpWkt0XO7EHqyifjIJFesbwm5NyKVL6+//e7+GPDYnGOfznk9DXx4gWvvA+7LJ2Zw/DiZVVXmHp+3DXf/S+Avl7yJVWAgnijJRE3ILGHY2RTjpBJxEZEVyZZ3lGKlq0LpCXbX1IRNkcKohsmaEhiMl25EHGBrZ6NKU0REVig+nVm6sJTP7UvV1VxHLBpRaYpIgSgRryID8URJHugPHzrNw4dO4w5Hz4/x8KHTRW9TRKTavDUiXjk14pGIaQlDkQJSIl4l0mlnaCJZstIUgM7mGKNTM8zMpkvWpohItRjPJuIVNCIOmSUMz2oJQ5GCUCJeJcamZ0ilvaQP9M6mOgCGgmUTRUQkf/HpykzEezoaVJoiUiBKxKtEKTfzyepsjgEwGLQtIiL5iydSNMaiRCMWdleWZVN7AwPxBNMz2tBN5FIpEa8SA/HsZj6lHxEf1Ii4iMiyxROpihsNh5y1xFWeInLJlIhXicEgES/lQ70hFqUxFr3YtoiI5C8+XZmJeE9HZh+JM0NaNUvkUikRrxLZ0pRSbG+fq7MpxsCESlNERJZrPJGqqDXEs7Z2KhEXKRQl4lViMJ7ArLSlKQCdzXUMaURcRGTZ4okULRU4Ir62pY762ggnB5WIi1wqJeJVYmAiyZrGGBEr7aSfzqbMEoaatCMikr9kKk0yla7I0hQzY1tnE6e0s7LIJVMiXiUG44mLq5iUUmdzHY6+ohQRWY5K3N4+19bORo2IixSAEvEqMRBP0tVcV/J2O5syyb8eyCIi+Xtre/vK2VUz17bOJk4PTjKb9rC7IlLRlIhXicyIeOkT8Wzyr68oRUTyV+kj4tu6mkjOpjk/qiUMRS6FEvEqMRhPXhydLqWGWJSG2ignBpSIi4jkq1K3t8/KrpxySt+GilwSJeJVYHpmlvFEiq4QasQhs8OmHsYiIvmLV3givq2zCYCT+jZU5JIoEa8C2Z0tw6gRz7arh7GISP7i05W5vX3W+tZ6YjURDcKIXCIl4lVgMNjMJ4wacYA1TTHeGJkikdIShiIi+ajU7e2zIhFj65pGTqosUeSSKBGvAtkt5sNYvhAyK6ekHc4MadKOiEg+KnV7+1xbO5s0Ii5yiZSIV4H+YES8qym80hRAIyMiInmKV+j29rm2dTZycnCCtJYwFFkxJeJVoH88k4h3t4STiL+1lrgScRGRfIxX6Pb2ubZ2NZFIpXlzfDrsrohULCXiVaB/PEFLXQ0NsWgo7TfW1dDWUKuvKEVE8lDJ29vn2p5dOWVAz36RlVIiXgX6xqfpbg1nNDwr+xWliIgsrtI388l6ay1xPftFViqvRNzM9pnZMTPrNbO753m/zsweCd4/ZGbbct67Jzh+zMxuXSqmmW0PYvQGMWNLtRG8v8XM4mb268v9ECpd31iCtSGVpWRt72rieL8exiIiS5kIEvGmCh8R39jeQKwmwnHNDxJZsSUTcTOLAp8FbgN2A3eY2e45p30cGHb3ncADwP3BtbuB24GrgX3A58wsukTM+4EHgljDQewF28jx+8DX8r3xatIfT7C2pT7UPuxa18K5kamLv2BERGR+FxPxWGUn4tGIsaOrid6+eNhdEalY+YyI3wj0uvtxd08CB4D9c87ZDzwUvP4ycIuZWXD8gLsn3P0E0BvEmzdmcM37gxgEMT+4RBuY2QeBE8CR/G+9Org7fWOJ0CZqZl3W3QzA6/16IIuILGYiWR0j4gCXrW1WIi5yCfJJxDcBZ3J+Phscm/ccd08Bo0DnItcudLwTGAlizG1r3jbMrBn4FPBbedxL1YknUkzNzIZemrJrXSYRf+1NPZBFRBYzkchsftZUF84E+0La2d3MmeFJpme0oZvISlTDZM3fJFPKsmgGaGZ3mtlhMzvc399fmp6VQHbpwrUhT9bcuqaR2qjxmkZGREQWNZFMURMxYtHK/xW8c20z7miOkMgK5fO92Dlgc87PPcGx+c45a2Y1QBswuMS18x0fBNrNrCYY9c49f6E2bgI+ZGa/C7QDaTObdvf/mttBd38QeBBg7969VbP7QF92DfHmcGvEa6IRtqtWUERkSROJWZrqagiqKyvaq2+OA/DQ906yZ3P7xeMfuWlLWF0SqSj5/HP8GWBXsJpJjMzky4NzzjkIfCx4/SHgCXf34PjtwYon24FdwNMLxQyu+WYQgyDmVxZrw93f6+7b3H0b8H8Dvz03Ca9mfWUyIg6wa20LvX3jYXdDRKSsTSRSVVGWApmdlY23fheJyPIsmYgHI9N3AY8DrwCPuvsRM7vXzD4QnPZ5MvXavcAngbuDa48AjwJHga8Dn3D32YViBrE+BXwyiNUZxF6wjdWubyyzo1nYNeKQ+Yry9JBqBUVEFjORTFX8iilZtdEIa5pi9Gt3TZEVyetJ4O6PAY/NOfbpnNfTwIcXuPY+4L58YgbHj5NZVWXu8QXbyDnnNxd7vxr1xxPEohHaGmrD7go71zaTDmoFd29sDbs7IrIMZrYP+AMgCvyJu//OnPfrgP8O3ECmLPAX3f1k8N49ZJaYnQX+tbs/nmfM/wf45+7eXMRbKzsTiRRdzeEPnhRKd0udRsRFVqjyZ4qscv3B0oXlUGuYXTmlV0sYilSUEPaLwMz2Ah1FvbEyNZGcpSlWHaUpkPlGdjCeZDZdNdOvREpGiXiF6xsPfw3xrO1dTUQMet9UnbhIhSnZfhFwMfH/z8BvFPm+ys7MbJpkKl0Va4hndbfUM+vO8EQy7K6IVBwl4hWufzz87e2z6mqibO1s0hKGIpWnlPtFQGaO0EF3P1+g/leMatnePlf2d5DKU0SWT4l4hesbny6bEXHI1IkrEReRhZjZRjLzff7fPM6tuv0fJpLBZj5VMlkTuPg7qE8TNkWWTYl4BUum0gxPzrC2Jdw1xHPtWtvMyYEJrZwiUlmWs18Eee4XsdDxdwE7gV4zOwk0BqthvYO7P+jue919b3d398rurMy8NSJePTXi9bVRWutrNCIusgLV80/yVWggXj5riGddu6mNVNp55fwY79qyKudhiVSii3s7kEmWbwc+Muec7F4O3ydnLwczOwg8bGa/D2zkrf0ibL6YwVK167NBzSweTABdFS4m4hU4Iv7wodMLvreutZ43xzQiLrJcGhGvYG/tqlk+ifh1wc5qL54dDbknIpKvEPaLWLUulqZUUY04wPq2evrGE1o5RWSZqutJsMpc3MynjEbEN7bV09VcxwtnR8LuiogsQyn3i5hzzqpbQzxiUF9bXeNg61vrmU07A/EE61rLp1xSpNxV15NglenPlqaUUY24mbGnp00j4iIi85hIZHbVLIe9HwppfVvm99AFlaeILIsS8QrWN5bADDqbY2F35W2u62nn9f448aAWUkREMiaSs1VXlgKZEsmIwZujSsRFlkOJeAXrG5+msylGbbS8/jNet7kNd3hJo+IiIm8zkUjRWEUrpmTVRCN0NddpRFxkmcorg5NlOT86zYa2hrC78Q57ejITNlUnLiLydtnSlGq0vq1eibjIMikRr2AXRqcv1uWVkzVNMXo6GnhRibiIyNtMJFNVWZoCsKG1npHJGe0jIbIMSsQrWGZEvPwScciMir9wRqUpIiJZM7NppmfSVbWZT651we8jrScukj8l4hVqMplidGqmLEfEAfZsbuPcyNTFTYdERFa74YkkUJmb+eRjfbBs4XlN2BTJmxLxCnUheNCV64j4DVvXAPDU8cGQeyIiUh4Gs4l4lZamtDXUUl8bUZ24yDIoEa9Q2UR8fWv5TdYE2NPTRmt9Dd9+tT/sroiIlIWhi4l4dZammBnrW+sv/n4SkaUpEa9Q58t8RLwmGuHHd3bxndcGcNeWxyIig1VemgKwvq2BC2PTpLXVvUheqvdpUOWyX/2VS434w4dOv+PY+y7v5msvX6C3L86udS0h9EpEpHwMBXNmqrU0BWBTez1PHU9zcnCCHd3NYXdHpOxpRLxCnR+doqOxlvra8v2K8727ugD49msDIfdERCR8QxNJDGiMle9z+1Jl97Y48sZYyD0RqQxKxCtUZg3x8qwPz+rpaGRHd5PqxEVEyJSmNMSiRMzC7krRrG2tI2qmRFwkT9X7/ViVK+c1xHO9b1c3B545zfTMLPW10XlLWD5y05YQeiYiUlpDE8mqLksBqIlEWNdax5E3tI+ESD40Il6hzpfprppzve/yLqZn0jxzcijsroiIhGpwIlnVEzWzNrQ3cOSNMU3UF8lDXom4me0zs2Nm1mtmd8/zfp2ZPRK8f8jMtuW8d09w/JiZ3bpUTDPbHsToDWLGFmvDzG40s+eDPy+Y2S+s9MMoZw8fOn3xz0PfO8nQRJINreWfiP/oji4aY1G+9vKFsLsiIhKqzIh49daHZ21sb2BoIqn1xEXysGQibmZR4LPAbcBu4A4z2z3ntI8Dw+6+E3gAuD+4djdwO3A1sA/4nJlFl4h5P/BAEGs4iL1gG8DLwF53vz5o44/NrKqHHMamZgA4OTj5tgR9vrKPsDXEotxy1Tq+9tJ5ZmbTYXdHRCQ0q6E0BWBj8G3tkXOqExdZSj4j4jcCve5+3N2TwAFg/5xz9gMPBa+/DNxiZhYcP+DuCXc/AfQG8eaNGVzz/iAGQcwPLtaGu0+6eyo4Xg9U/Xdho9OZRLytoTbknuTn56/bwPDkDN97XbtsisjqNJt2hidXR2nK+rZ6zLRyikg+8knENwFncn4+Gxyb95wgKR4FOhe5dqHjncBITmKd29ZCbWBmN5nZEeAl4Ndyrq9K2RHx1obyfqBnR+kvjE5TVxPhD77xWthdEhEJxchkEvfq3VUzV11NlO1dTbysCZsiS6qKyZrufsjdrwbeDdxjZu8onjazO83ssJkd7u+v7OX0Rqcy/85oq6+MEfGaaITdG1o5en6UlMpTRGQVemt7+/IeQCmUqze2cVQj4iJLyicRPwdszvm5Jzg27zlBfXYbMLjItQsdHwTac2q8c9taqI2L3P0VIA5cM/cm3P1Bd9/r7nu7u7uXvOlyNjo1Q31thLoy3sxnrut62pmeSfNaXzzsroiIlNxq2N4+1zUbWzk3MsVgsJuoiMwvn0T8GWBXsJpJjMzky4NzzjkIfCx4/SHgCc+sW3QQuD1Y8WQ7sAt4eqGYwTXfDGIQxPzKYm0EMWoAzGwrcCVwMu9PoAKNTc3QWiGj4Vk71zbTGIvywtmRsLsiIlJyb42IV84AyqW4rqcdgBfPqTxFZDFLJuJBvfVdwOPAK8Cj7n7EzO41sw8Ep30e6DSzXuCTwN3BtUeAR4GjwNeBT7j77EIxg1ifAj4ZxOoMYi/YBvAe4AUzex74a+BfuXtV76k+MpWkvbGyEvFoxLhmUxuvnB8jkZoNuzsiIiU1uMpKU67tacMMXjyjRFxkMXk9Edz9MeCxOcc+nfN6GvjwAtfeB9yXT8zg+HEyq6rMPT5vG+7+ReCLS95EFRmemKGnozHsbizb9T3tPH1iiKNvjPGuLR1hd0dEpGSG4qurNKW5road3c36FlRkCVUxWXM1SczMMjUzS0djLOyuLNuWzkbaG2r1YBaRVWdoIkFrfQ3RiIXdlZK5rqedF8+OaIdNkUUoEa8ww8HShZVWmgIQMWPP5nZ6++LEE1W9wqSIyNsMTiTpbK4Luxsl8/Ch08zMphmIJ/nDf3i9LDecEykHSsQrzEhQZ1iJI+IAeza3k3Z4SaPiIrKKDE0kWdNUmc/tlerpaADgzPBUyD0RKV9KxCtMdkS8owJHxAHWt9azvrWeF85qAo+IrB6rMRFf31pP1Ixzw5Nhd0WkbCkRrzAjE0lqIlbRM+/3bG7n9NDkxeW8RESq3eBEks5VlojXRCOsb6vnrEbERRakRLzCDE/N0N5YS8Qqd8LPnp42AJ4/o/IUEal+7s7wKhwRh0x5yrmRKdKasCkyLyXiFWZkMlmx9eFZ7Y0xtnU28cIZzaYXkeo3NpUilfZVmog3kkil6R/XDpsi81EiXmGGJ2cqcsWUua7f3E5/PMH50emwuyIiUlSDE5kktLN59SXiW9Zk9rw4PaQ6cZH5KBGvIMlUmolEquJHxAGu2dRK1EzlKSJS9bLzYdY0rZ7lC7O6mmM0xqKcGpwIuysiZUmJeAUZmcw8zNurIBFvjNVw+bpmXjw7Qjqt8hQRqV7Z7e1X22RNADNja2cTpwY1Ii4yHyXiFWSkwpcunOu6ze2MTad45uRQ2F0RESmat0bEV18iDrCts5HBiaTqxEXmoUS8ggxX0Yg4wFXrW6mNGl998Y2wuyIiUjSrPRHfGtSJP3tKgy4icykRryAjkzNEzWipr9w1xHPFaiJcub6Vx166QGo2HXZ3RESKon88QUtdDfW10bC7EoqN7Q3URIxnTg6H3RWRsqNEvIIMTyZpq/A1xOfa09PO0ESS774+GHZXRESKon88QXfL6puomVUTjdDT0cDhU0rEReZSIl5BRiZnqqY+POvydc201Nfw1RdUniIi1Wm1J+IAWzubOHJulKnkbNhdESkrSsQryPBksmrqw7NqohFuvXo9j798gURKD2gRqT79cSXiWzsbSaWdH5zRqLhILiXiFSKZSjM+narKyT4/v2cj44kU3zrWH3ZXREQKTiPisHVNExGDp1SGKPI2SsQrxNBk9c66/7HLOlnTFOOrL54PuysiIgU1mUwRT6RWfSLeEItyXU873+kdCLsrImVFiXiFGIpX74YQtdEIt12znm8cfZPJZCrs7oiIFEx27ey1LfUh9yR8793VxQtnRhgN9sQQESXiFWNwIvMw76zSLZJ/fs9GpmZm+cYrfWF3RUSkYLKJ+GofEQd4z84u0g7fV3mKyEVKxCvE0ESShtooDbHqXIf23dvWsK61TquniEhVuZiINysRf9eWDhpjUZ7s1XwgkSwl4hViaCJZlfXhWdGI8XPXbuRbx/r1taWIVI3+uEbEs2I1EW7e0cmTr6lOXCQrr0TczPaZ2TEz6zWzu+d5v87MHgneP2Rm23Leuyc4fszMbl0qppltD2L0BjFji7VhZj9tZs+a2UvB/75/pR9GORus8kQcYP/1G0nOpvlbbXkvIlWifzxBNGJV//zO13t2dnFycJIzQ5Nhd0WkLCyZiJtZFPgscBuwG7jDzHbPOe3jwLC77wQeAO4Prt0N3A5cDewDPmdm0SVi3g88EMQaDmIv2AYwAPy8u18LfAz44vI+gvKXmk0zMpmsyomaua7raePK9S0cePpM2F0RESmIvrEEnU0xopHq2RH5Urx3VxcA39GouAiQ34j4jUCvux939yRwANg/55z9wEPB6y8Dt5iZBccPuHvC3U8AvUG8eWMG17w/iEEQ84OLteHuP3D37BDqEaDBzKrqO8A3RqZJe3UuXZjLzLjjxi28dG6Ul8+Nht0dEZFLps183m7n2mY2tTfwxA81MV8E8kvENwG5Q5Rng2PznuPuKWAU6Fzk2oWOdwIjQYy5bS3URq5/Ajzn7ok87qtinBqaAGBNc3Un4gAfvH4TdTURDjxzOuyuiIhcMm3m83Zmxs9cvY5vv9ZPPKHlakWqZrKmmV1NplzlXy7w/p1mdtjMDvf3V9aM7VODmVq6al26MFdbYy0/d+0G/uYHb2hNcZESKvFcoD8Pjr9sZl8ws9pi319Y+scTrFUi/jb7rl5PMpXmH45pVFwkn0T8HLA55+ee4Ni855hZDdAGDC5y7ULHB4H2IMbcthZqAzPrAf4a+GV3f32+m3D3B919r7vv7e7uzuO2y8epwQlqIkZLfc3SJ1eB22/cQjyR0lKGIiUSwlygPweuBK4FGoBfLeLthSaddgZUmvIOe7etobMpxtdfvhB2V0RCl08i/gywK1jNJEbmgXtwzjkHyUyUBPgQ8IS7e3D89mAkZTuwC3h6oZjBNd8MYhDE/MpibZhZO/A/gbvd/bvLuflKcWpwko6mGBFbHZN93r2tgyvXt/Cn3z1J5v8SIlJkJZsLBODuj3mAzO+EniLfXyiGJ5Ok0q41xOeIRjLlKd/8YR/TM7Nhd0ckVEsm4kE99l3A48ArwKPufsTM7jWzDwSnfR7oNLNe4JPA3cG1R4BHgaPA14FPuPvsQjGDWJ8CPhnE6gxiL9hGEGcn8Gkzez74s3aFn0dZOj00WfUrpuQyMz7+nu388MI43+3VDmwiJVDKuUAXBSUpHyXz+6HqvLWGuLa3n2vfNRuYSM7y3V6tniKrW161Du7+GPDYnGOfznk9DXx4gWvvA+7LJ2Zw/DiZkZS5x+dtw90/A3xmyZuoUO7O6aFJrt/cHnZXSmoqOUtzXQ33/u0RfuXHtgPwkZu2hNwrESmwzwHfdvfvzPemmd0J3AmwZUvl/f3P7qq5tlUj4nP96I5OWupreOylC9xy1bqwuyMSmqqZrFmt+scTTCZnq37pwrlqohFu3rGGV9+M0zc2HXZ3RKpdKecCEcT4j0A3mW8451XJc3tA29svJlYTYd/V6/n6y+c1MV9WNSXiZe71/szShatxss+N2zupiRjffV3lKSJFVrK5QABm9qvArcAd7p4u8r2F5mIivgqf3/n4p+/ezERylv/wN0d4+NDpi39EVpPVsQxHBTs+EAeqe0RloQdvc10N79rSzg9OD/Mzu/XVpUixuHvKzLLzdqLAF7JzgYDD7n6QzDydLwbzdIbIJNYE52XnAqUI5gIBzBczaPKPgFPA9zPzPfkrd7+3RLdbMn3jCRpjUZrq9Kt2Pnu3dtDVHOPZU0PcsLUj7O6IhEJPhzL3et8EDbVRWhuqdpndRf3YZV08c3KYQyeG+Bfv2xF2d0SqVonnAq2K3z3azGdxZsYNW9fw+JELDIwn6NJnJauQSlPK3PGBONu7mlbN0oVzrWut5/J1zTx1fJBESstciUjlOD86xYY2rZiymHdtaSdi8Ozp4bC7IhIKJeJl7vX+OJetbQ67G6H68Z1dwQY/58PuiohI3s4NT7GxvSHsbpS11vpaLl/XwnOnhkmlq3a6gMiCVsXXg5VqemaWs8NT/ON3VeVeF3nb2d3MutY6/uQ7x/knP7IJW6XfDohI5ZiZTXNhbJoeJeIXLTQf6Ed3dPKn3zvJi2dG+RHVissqoxHxMnZycAJ3Vv2IuJnx45d18cML43xfK6iISAW4MDpN2mFThxLxpexc28z61nqe7B3Qbsqy6igRL2Ov92WWLtzR1RRyT8K3Z3M7Xc0x/uTJE2F3RURkSedGpgDY1N4Yck/Kn5nx4zu7uDA2TW9fPOzuiJSUEvEydrw/80Da0a1EvDYa4Z/dvJUnftjH6/16UItIeTs3HCTiGhHPy56eNlrqa3hSW97LKqNEvIy93h9nU3sDjTGV8gM0xmqoiRj3/NVL2vhBRMpadkRcq6bkpyYa4cd2dPJaX5wXzoyE3R2RklEiXsaOD0xoNDxHc10N12/ObPAzmdCWyCJSvs4NT9HdUkd9bTTsrlSMm3d00hiL8p8fPxZ2V0RKRol4mXJ3Xu+Lc1n36p6oOdeP7+xiZtZ5+uRQ2F0REVnQuZEpNmnFlGWpq43yk1es5cneAb6nEhVZJZSIl6k3xxJMJGc1Ij7HutZ6dq1t5vvHB7XmrIiUrXMjU6oPX4Gbtq9hY1s9v/v4Ma2gIquCEvEylZ2QuKNLI+Jz/fjOLsanU7x0djTsroiIvEM67ZwbmdIa4itQG43wf/zULp4/M8L/fEmbuEn10yzAMvXK+TEArtzQEnJPys+utc2sbam7uOasNvgRkXIyMJEgmUrzxui0JpWvwIdu2MyXnjrNvV89yk9c3k1LfW3YXRIpGo2Il6mj58fobqmjq7ku7K6Uneyas+dHp3nquGrFRaS8ZJcu7GhQArkS0Yhx3y9cQ388we/93athd0ekqJSIl6lXzo9z1YbWsLtRtq7f3E5jLMrnnzwedldERN4mu3RhW6MS8ZW6rqedj968lYe+f1LLGUpVU2lKGUqm0vT2jfO+y7vC7krZqo1GuHlHJ994pY/j/XF2LLG6zHxfD3/kpi3F6p6IrGIXR8QbYyH3pLL9+q1X8HdH3uTfPPI8X/3f30NTnVIWqT4aES9Dr/fHmZl1dmtEfFE3bV9DLBrhT797MuyuiIhcdG5kipb6Gq0hfola62t54Bev58TgBL958EjY3REpCv3zsgxlJ2oqEV9cS30t+6/fyF88e4Z/+zOX067RJxEpA+eGtYb4pZj7DeZPXt7NXzx7lvfs6mL/9ZtC6pVIcWhEvAwdfWOMWE2E7V1aQ3wpv/reHSRSaf7rE71hd0VEBMiMiPdoDfGCef+V63j3tg7u/suXOPKGlq2V6qJEvAy9cmGMK9a1UBPVf56lXLG+hV/cu5k/+95JevvGw+6OiKxys2nnxMCEBlIKKBoxPvtLP0JbQy13/vdnGYwnwu6SSMHklemZ2T4zO2ZmvWZ29zzv15nZI8H7h8xsW8579wTHj5nZrUvFNLPtQYzeIGZssTbMrNPMvmlmcTP7ryv9IMqFu/PK+XGVpSzDr996BQ2xKL/11aPaiU1EQnV6aJJEKs2uddoDopDWttTz4C/fwEA8wb/68+eYmdXOylIdlkzEzSwKfBa4DdgN3GFmu+ec9nFg2N13Ag8A9wfX7gZuB64G0UqviAAAH8dJREFU9gGfM7PoEjHvBx4IYg0HsRdsA5gG/gPw68u897LUN55gaCLJVdrIJ29dzXV88qcv5zuvDXDwhTfC7o6IrGLHLmS+mbtCiXjBXdfTzu/8k2s5dGKIz/ztUSBTTz73j0glyWdE/Eag192Pu3sSOADsn3POfuCh4PWXgVsss93hfuCAuyfc/QTQG8SbN2ZwzfuDGAQxP7hYG+4+4e5PkknIK97RYKKm1hBfnn9281Zu2NrBr//FC3zzh31hd0dEVqnX3swk4jvXLr6kqqzML7yrh3/x3u089P1THHhaSbdUvnwS8U3AmZyfzwbH5j3H3VPAKNC5yLULHe8ERoIYc9taqI28mNmdZnbYzA739/fne1nJvXw2MxHlSiXiy1IbjfCFX3k3V65v5V9+6Vkee+m8ylRE5P9v787j66rKhY//njPkJDlJ04xtSULT0pbSFigdoFAGAa0Fh6qAFOUKyvvyXsGrvtxXEVSu1+tw0fc6IKBwFUdKESm0LyJUKKhQOkPngc6kQ9Ikbeaccb1/7JVyGjI1zc4+aZ7v53M+OWeftfd6cs7OWitrr73WgNtR3URZfpbOee2iu+dO5LLxRdy3ePPxxZOUGqyGzN2AxphHjTEzjDEziouLvQ6nS6v3HeXsEbnk6dLIJy0vK8jvb7uQ8SU53PH4Oj7+8HKWbasikdQGuVJqYOw43KjDUlwW8Pv46fwLKAhnsHDVfiKxhNchKdVnvWmIHwDKU16X2W2dphGRAJAH1Hazb1fba4Hh9hgd8+oqj9NGImlYt+8oM8fkex3KoDU8O4Nn75zN9z9xLtUNbXzuN2u47P5lLNtWpTf3KKVcFUsk2V3TpDdqDoCCcAY/nT+VuuYoi9cf1CugatDqzbWz1cB4ERmD0xieD3yqQ5olwC3AG8D1wDJjjBGRJcACEfkRcAYwHlgFSGfHtPu8Yo+x0B5zcXd59O3XTk9bDzXQFIkzs6LA61AGtaDfx00XnsknppXy0pZqnlzzDi9trebN/cf4+LRSxhbp2E2lVP/bV9tMLGGYMELLmP7W1U2YV51Twstbq5k4MpfzyoYPcFRKnboee8TteOwvAC8CW4E/GmM2i8i3ReSjNtmvgEIR2QncBXzN7rsZ+COwBXgBuNMYk+jqmPZYdwN32WMV2mN3mQeAiOwFfgTcKiKVnczqMiis2lMHoA3xfhIK+PnQeaP43ecu5LZLx2CAX/5jD+v2H/U6NKXUaWj74SYAJmiP+IB534QSSodn8f/WH6QpEu95B6XSTK/uJjHGPA8832HbfSnP24Abutj3u8B3e3NMu303zqwqHbd3l0dFt7/AILF6bx2lw7M4Q5dG7rXOekk+ddGZ79l2VnEOX7xqPL9fsZdF6yrJDPgHIjyl1BCyo6oRn+iMKQPJ7xOum17GQ8t28tyGg8yf+d7yX6l0NmRu1kx3xhhW763jwjHaG+6WjICPm2eNpnR4Fk+s3s+avXVeh6SUOo28Xd3ImQXZZAb1H/2BNHJYJldOLGZDZT1b7RTASg0WOr9SmthT00xNU1SHpfSD7hZ0CAX83HJJBQ+/uosvLXyL5790mc5Qo5TqF9sPN+qwFI9cMaGEjQfqWbL+IF+7ZqJOH6kGDe0RTxOrbe/shTpjiuuyMwLcOKOcww1tfPPZTXq3vVLqlLXFEuytbdGGuEf8PuFjU0upb43x47/u8DocpXpNG+JpYsXuOgrCGZxVrGMLB0J5QTZfvno8S9Yf5Nm3Os7GqZRSJ2fdvqMkkoZpo3XmDq+MLgwzs6KAXy/fy6YD9V6Ho1SvaEM8DUTjSV7aWsVVE0sQEa/DGTLuuHIcM0bn82+LN3O4vs3rcJRSg9gbu2vx+0SHF3ps7uSR5GcH+fozG3UxNzUoaEM8Dby+q4bGtjjXnjvS61CGFL9P+OEN5xNNJLln0QYdoqKU6rM3dtUypTSP3Ey958RLWRl+vvGhSayvrOfxlfu8DkepHmlDPA38ZeMhckMBZo8r8jqUIWdMUZi7507kle1HeGptpdfhKKUGoZZonPWVx7h4bKHXoShg3tQzuHRcET94YTtVDXq1U6U3va3YY7FEkqVbqrj6nBJCdm7r7mb9UP2n/XMO+n2MKQrzjWc2Me3MfJ0DWCl1UtbsPUosYbj4LG2IpwMR4Tsfm8Kcn/ydf1u8mZ/fPE2Hfaq0pT3iHlu5u45jLTGuOXeU16EMWT4RPjmjnKBfuOPxtbRGE16HpJQaRN7YXYtPYPeRJhas3H/8obyxYOV+lu+q5coJxbyw+TD3LNrodUhKdUkb4h57ftMhsjP8XDGh2OtQhrS8rCCfnFnO29VN3PvMRpJ6k49Sqpfe2FVLWX728auaKj1cOr6YsvwsFr91kOpGHaKi0pMOTfFQQ1uMp9dWMnFkLovW6RR6Xhtfksu/fmAC/3fpDnwi3H/duQT8nf+v2llv16cu0qWVlRpqmiJxNh6o57Lxeo9PuvH7hOunlfHgKzu5d9Em/vsz03WIiko72iPuoQUr9xOJJ7l0vPaGp4s7rxzHXR+YwNPrKrlzwTqq9UYfpVQ3XtpSRSJp9N6SNFUyLJM5k0fy0tYqfrN8r9fhKPUe2iPukbZYgl+9tofxJTmUDs/yOhxliQhfvHo84VCA7z2/lVe3v8L8meWcXz6cktxMookER5tjvPb2EVqiCXw+oTCcwai8LIwx2tui1BCzYNV+zizIpqIw7HUoqguzzyokGk/wvee3csGZ+Uwt10WXVPrQhrhHFq07wJHGCB89/wyvQ1GduO3SMVw9sYSHXtnJH1bu57dvvHc+Wp+AMdA+mvylrVXcdGE5N8woJzOoY0WVOt3trG5k1Z467p47EZ/+E562RIT/umEq1z7wD+58fB3P3HkJJbmZXoelFKANcU9E4gke+fsuzi/LY2yR9qKkq4qiMD+84Xwmn5FHQ2uMhkiMoM9Hdoaf7IwAmUEfiaShriXK7iPN7K5p4puLN/PoP3Zz7zXnMHfKSO0hV+o0tmDlOwT9wg0zyli6ucrrcFQ38rKD/Pzmadz4yAo+++vVLLx9li6+pNKCjhH3wAMvv82+2hb+dc7Z2lAbBDICPopyQ4wtyqG8IJvCnBBZGX5EhIDfR0luJrPGFvLcv1zGH267iOxggM8/vo4bH13BpgP1XoevlHJBWyzB0+sqmTN5JEU5Ia/DUT1YsHI/mw40cOPMcrYeamDeg6/rVLUqLWhDfIBtqDzGL/62mxuml3G5Tll4Wlmwcj/761q4edZo5k09g00H6vnIg6/x1T+t16mzlDrNPLW2kvrWGJ++UGdLGkwmjMjlumll7Klp5pOPvMHhei2blbd0aMoAaosl+MpTGyjKyeAbH57kdTiqE/2xCIffJ1w0ppDzSodT1djGr1/fw583HOKzs8fw6VlnMipPb85VajA7VN/KD/6yjVljC3Q1zUHogjPzyQz6WbSuko88+Br/MW8KH5w8Qq9QK09oj/gAiSeSfOLh5eyoamTu5FH8ecMhXXntNJeV4aeiMMwXrxpPRVGYh17Zyez/XMbnfrOaJ1bt154YpQYhYwz3LtpIPGm4/7rztPE2SJ0zahiL7pjNsMwA//yHtcx76HWeWLWfndVNGKMLuqmBoz3iA8AYwzcXb2LLoQY+fN4ozh6Z63VIagAV5oT49EWjqWuOsmpPHdsPN7JsWzUA5QVZzBxdwPSKfGZWFDCuOAefTyt2pdLV4yv388r2I9z34UmM1ikLB7WzR+by4pcv55k3D/DAsre5Z9FGwJkRKxTwkxn0kRn0O4+Aj6wMPyW5mZTmZ1GWn8VnZ4/x+DdQpwNtiLssGk/yzWc38eSad7hiQjGXnKWrrw1VBeEM5k4ZiTGGqsYIO6sa2VfXwtItVSx601lZNTPoY3RBmNGF2dx26RjOLx+uUyEqlSYee20P335uC5eNL+KWSyq8Dkf1g4Dfxw0zyrl+ehm7a5p5cNlO6pqjtMUSROJJ2mIJ2mJJjrXGOHCslXX7jwFOY/3PGw5x8VmFXHxWIdPscBdddVmdLG2Iu6i2KcIdj69j5Z46/uWqcYwcpvOWKmdO25HDMhk5LJNLca6Y1DVH2Vfbwr66ZvbWtrC9qpGlW6oI+oUppXnMGJ3P9NEF7KttJicUOOFyuBbySrmrKRLnR0t38Njre5g7eSQ/mT8Vv165Oq2ICGcV5zCzoqDbdM2ROAeOtbKnppn61hgPvbKTny3bSUbAx4zRTmO8KCeDwnCIgnAG2RnakaK6J70ZCyUic4GfAn7gl8aY/+zwfgj4HTAdqAVuNMbste/dA9wGJIAvGmNe7O6YIjIGWAgUAmuBfzLGRPuSR1dmzJhh1qxZ0+Pv3VfGGJ5aW8n3n99KczTBD68/j3lTS3VMuOq1lkiciqIwa/YdZe2+OtZX1hONJwHIsgV9cW6IopwQH7+glLHFOYwuzNbe8yFCRNYaY2b08zE9L+e7i8/tcrszLdE49y7axMtbq2iMxJk1toAPnXuGNsIV4HSCNLTFWL2njuW7alm+q5ZthxpIbVVlBn2MLcqhLD+LsvxsyvKzGFscZkppnk57OYR0V2b32CMuIn7gIeADQCWwWkSWGGO2pCS7DThqjBknIvOB+4EbRWQSMB+YDJwBvCQiE+w+XR3zfuDHxpiFIvILe+yfn2wexpgBnyA0Ek/w5w2H+PXre9l4oJ4Zo/P57sfP1THh6qRlhwK8f9II3j9pBOCcW5sO1PPYa3s50hShpjHCzuom1u0/xtItzkIiIlA6PIuxxTmMLQpzVnGYscU5VBSFKckNEfT37t7sWCLJ0eYotc1Rapui1DZHWLq5CmMMwYCPoN95zJk8grys4AmP3uYxGJ3Ol5zTqJz3XH1LjOW7ali2rZrnNx6iOZqgPD+Lm2eNprwg2+vwVJoZlhnk6nNGcPU5Tln92+V7j5efdc1O+ZmdEWBvbTOv7ayhJWXu8pHDMplSOowppXmcV5bHeWXDPWmcR+NJWmM2rvb/IgRCAR8Zfp/et+Sy3gxNuRDYaYzZDSAiC4F5QGoBPQ/4ln3+J+BBca6dzwMWGmMiwB4R2WmPR2fHFJGtwFXAp2ya39rj/rwPebzRy8+g14wxROJJWqIJWqJxmiJx3qlrZU9NE6v21LFydx2NkTjjSnL44fXncd20Mj2BVZ911vCbPe7Eewwi8QQ1TVFqGiNOA70pws7qRlbsqiWaSJ6QtiCcQXFOiJJhIbIz/AT9PoxxptVsbItT0xyhrjnKsZZYr+J7YtV748sI+MgK+gn4BBHBJ5CfnYEIxJOGRNIQTyZpaI2TTBoSxpA0hqSBkN9HOBQgHPITDgXICQWYPa6I4tyQ88gJEQ4FyM5wbp5q/x36yth8Y4kkzRHn77mxzXkcaYpQ3dBGdWOEqoY2qhra2FndRHMkQSyRJJ40BHzC/S9sY3h2kMJwBgXhEEU5GRSEnUdRjnNputBeps4PBwn60rZSS5dyvt8Z45x3sYQhGk8STTiPo81RqhraOFTvfL/7alvYdLCePTXNGAM5oQDXnjuK/OwMRhdm6+wo6j06K6ODfh8lwzIp6WQoqjGGlmiCqsY2Dh5r4+CxVjZU1vPy1urj7d/S4VmcW5rH+BE5VBSGKRkWOqGzIzsjQFenYlssQUs0QXMkTks0QUNbjKPNMY62RDnaHKXO/txyqIHmiNOOaYk6Y+G7E/CJ0ygP+AgF/EQTSQI+ISPw7s2s55YOY1hmkGFZQYZlBuzPIMOyAinbg2QGff3yt9T+dx2JJ2mOxmmNJmiOJGiOxmlojdHYFuflbdVEYglaYwkisSQITBo1jKDfWYwvK+gnJxQgNzNAbmbQ/gzYbc7roN+HT3D17783DfFS4J2U15XARV2lMcbERaQe55JjKbCiw76l9nlnxywEjhlj4p2k70se/ea+xZv4w4p9JLsYyVNRmM1Hpp7BNVNGcum4Ii201YAIBfyUDs+idPiJc5MbY2hoi1PTFKG2KUpjW4yReZlUN0Y40hihuiFCNJHEJ5AZdBq+E0fmUhgOcfBYq20QOwVSOOQnJyOAzydEE0niCUM0keR9ZxdT3xqjoTVGfWuM+pYYK3bX0hpLkkgmSRonjtL8LIxxKii/Twj4hH11LfhsQ93nE3xANJGkKeJUIgeOttIUibN8V22Pn4EICE5BKcdfOxtTXxsMySQkjfMPQG9mKAsFfIzMy2REbiaj8rIIhwJk+AW/T4gnDWOKwhxtiVHXHKHyaAvrK49xtDlKvKuCwvL7nGPcOKOc//jYlJ4DcV+6lPP9Zsn6g3zlqfVEE8kev2ufwKi8LCadMYx555cye1wh55cPJ+j36ZBC1W9EhHAowNhQDmOLco5vj8QTHDzWRkluiPWVx9h0oJ6lWw532d7oq5xQgPxwEGMgHPJTnBsinOEnO+Q0OFNbLQZIJA3njMolGk8SiSftzwQ7qpqIJZzXbbEE9S0xqhraaGiN9diodz6H3pXZIhzvqGnvOEn2suxOFfAJITtsc9uhBuJJQyyRJJbo/YH8PsEvwriSHJ7/0mUnF0BP8fXr0dKYiNwO3G5fNonI9n44bBFQA7AP+Bvw/X446KnE4TGN40Qax4kGZRw7XIzjO1Dznb7tO7p/Q0lP/VRu9+r73gMs78PBXZQufy8nS+MeWBr3AHobiuTLfYq7yzK7Nw3xA0B5yusyu62zNJUiEgDycG7m6W7fzrbXAsNFJGB7S1LT9yWP44wxjwKP9uL37TURWdPfN0xpHBqHxqFxeCBdyvkT9Ee5nWafc69p3ANL4x5YGve7ejPAcjUwXkTGiEgGzk05SzqkWQLcYp9fDywzznQsS4D5IhKyd8mPB1Z1dUy7zyv2GNhjLu5jHkoppXonXcp5pZQaUnrsEbdjAb8AvIgzBdVjxpjNIvJtYI0xZgnwK+D39iadOpwCF5vujzg3/MSBO9tnM+nsmDbLu4GFIvId4E17bPqSh1JKqZ6lUTmvlFJDSq/mEVedE5Hb7aVTjUPj0Dg0jkEZx+lusH7OGvfA0rgHlsadckxtiCullFJKKTXwTt/VN5RSSimllEpj2hDvAxGZKyLbRWSniHztFI7zmIhUi8imlG0FIvJXEXnb/sy320VEHrB5bhCRaSn73GLTvy0it6Rsny4iG+0+D9jFNzrLY4qIvCIiW0Rks4h8yaNYXhaRtSKy3sbx7zbNGBFZafd90t74hb057Em7faWIVKTkd4/dvl1EPtjTd9dZHiLiF5E3ReQ5D+PYaz+3t0RkjYfnyGgR+ZOIbBORrSJysQdxvJ7yWbwlIg0i8mWPPo97xTlHN4nIEyKS6dH5kYHqUlefo4fxuFrmuxh3ubhcR7gUd6aIrBIX6xQXY3et/nE5blfrLJdiHi4u1m89Msbo4yQeODcd7QLGAhnAemBSH491OTAN2JSy7QfA1+zzrwH32+fXAn/Bme9+FrDSbi8Adtuf+fZ5vn1vlU0rdt9rusjjQWCafZ2LM3XyJI9i+ZF9HgRW2n3+CMy3238BfN4+vwP4hX0+H3jSPp9kv5cQMMZ+X/7uvrvO8gDuAhYAz3WVZgDiqAWKOpw3XnwvG4H/YV9nAMM9iqM9Dz9wGGdu1oGO43vAMSAr5Tu7tZPvbkDOU6/LxHR9dPc5ehiTq2W+i3GPwuU6wqW4Bcixz/u9TnH5M3el/hmAc3wvLtZZLsX8W1ys33rM3+0v5XR7ABcDL6a8vge45xSOV8GJhfJ2YJR9PgrYbp8/AtzUMR1wE/BIyvZH7LZRwLaU7cfTdZVHStrFwAe8jAXIBtbhrMRXAwQ6fv44szFcbJ8HbDrp+J20p+vqu7P7dMzjVeBlnKW4n+sizUDE0cp7C7WB/l4mAFHsPSXpcK4Cc4DXPYrjAiCGU+AG7PnxQY/Oj+Pp9OFuWd2PcVXgQpk/wL9Dv9YRAxRzv9cpLsZahkv1zwB8zntxqc5yKd48nPW8XKnfehODDk05eZ0tBd2fyzOPMMYcss8PAyN6yLe77ZVdxNlVHtjLWhfg9Bx4EouIvAVUA3/F+S++q+WwT1hyG0hdcvtk4utsye0LgK8C7ev1drcst5tx+IGl4gzZaV9hcKC/l2ycYWy/tpdKfykiYQ/iSM1jPvCER5/HWzgN8f3AIZzvey3enB/9vjT8acTtsrq/9Nf5OyBcqiNcY4d4uFWnuOUnuFf/uM3gXp3lhjHAEdyr33qkDfE0Zpx/q8xA5SEiOcDTwJeNMQ1exWKMmYrTI3AhMNHNPLtwNRA3xqz1IO+OdhtjpgHXAHeKyOWpbw7E94LTy+IHfm6MuQBoxrlUN6BxtOdhx0Z+FHiqqzRuxoFz2TKIU4CfAYSBuS7nqYaAATp/+8zrOqIvjDGJNKhTek1EPgxUp0n90xeXpkGddTICOMPFPKvftCF+8nqzFPSpqBKRUQD2Z3UP+Xa3vayLON+Th4gEcQrYx40xi7yMBcAYcwxn9b2Lscthd7Lv8fykd0tud7X9+JLbdvuVQFhE9gILcS4P/tSDOMqAffbzqAaewalIBvR7wRmWEjfGrLSv/4RTcHl1flwDrDPGVHWTxs3P43qg2RhzxBgTAxYBs/Hm/OjPsud043ZZ3V/66/x1lct1hOtcqlPcMBv4qIv1j6uMMQfsTzfqLDdUApUu1m890ob4yevNUtCnInUZ6Vt4d+nnJcBn7B27s4B6e9nkRWCOiOTbu3rn4IwdOwQ0iMgsERHgMx2O1TGPXwFbjTE/8jCWO4AXAEQkC2cM4la6Xg7brSW3W4G7jDEVNs0yY8ynPYjjtpTPI2w/z00efC/XAodF5Gz7+mqcVRS9Oldv4t1hKV2lcTOOqUBcRLJtuvbPY6DPj9Q81Hu5XVb3l345f90M0J7nrtURLsZdLCLD7XM36pR+Z4y5xxhT5mL94xoRCYtIbvtz+rnOciNmY8xh4B236rfeBqGPkx/cfy3OXeO7gK+fwnGewBljGsP5r+w2nLFdLwNvAy8BBTatAA/ZPDcCM1KO8zlgp318NmX7DJw/gl04M6O0L+DUMY9rcC67bMAZ//qW/R0HOpYVNoYNNu19Ns1YnAJkJ85whJDdnmlf77Tvj03J7+s2r+3YmS+6++66yeN9vHvX+kDH8bz9LNYDm9vTeXSOXA6ssfE8i3NXuBdxlOH09uSl7OtFHPcD22za3+PMTODZeaoPd8vqfozH1TLfxbgvxeU6wqW4zwPexMU6xeX434cL9Y+L8Y7Fqa9cq7NcinsqLtZvPT10ZU2llFJKKaU8oENTlFJKKaWU8oA2xJVSSimllPKANsSVUkoppZTygDbElVJKKaWU8oA2xJVSSimllPKANsTVkCYiXxeRzSKyQUTeEpGLukn7GxG5vqv3U9LsscdaJyIXd5Hun0XkM6cav1JKDRUiMkJEFojIbnGWUH9DRD7eSboKEdnUyfZvi8j7e5HPVBExIqIr5irXBXpOotTpyTaSPwxMM8ZERKQIyOiHQ3/FGPMnEZkDPIIzl21qvgFjzC/6IR+llBoS7KJCzwK/NcZ8ym4bDXy0Q7ou2zXGmPt6md1NwGv25wtdxCLGmGQvj6dUl7RHXA1lo4AaY0wEwBhTY4w5KCL3ichqEdkkIo/aQvcEIjJdRP5me2VelHeXP0/1d2CcTf+qiPxERNYAXxKRb4nI/7HvjRORl0Rkve1FP8tu/4qNY4OI/LtbH4JSSg0CVwHR1E4MY8w+Y8zPRORWEVkiIstwFmHpVPtVTRGZKyJPpWx/n4g8Z58LcANwK/ABEcm02ytEZLuI/A5ncaByEZlje+XXichTIpJj0/ZYhyjVThviaihbilOY7hCRh0XkCrv9QWPMTGPMFCALp9f8OBEJAj8DrjfGTAceA77byfE/grPyVrsMY8wMY8x/dUj3OPCQMeZ84BLgkO1NHw9ciLPq13QRufyUflullBq8JgPrunl/Gk6ZfEU3adq9BFxkl2EHuBFYaJ9fAuwxxuwCXgU+lLLfeOBhY8xkoBn4BvB+Y8w0nJUZ77Lpuq1DlEqlDXE1ZBljmoDpwO3AEeBJEbkVuFJEVorIRpxemMkddj0bmAL8VUTewimMy1Le/6HdfjvOEtbtnuwYg4jkAqXGmGdsTG3GmBZgjn28iVP5TMSpBJRSasgTkYfsVcTVdtNfjTF1vdnXGBPHGXLyETuU5UPAYvv2TbzbKF9oX7fbZ4xZYZ/PAiYBr9vy/hZgtH2vpzpEqeN0jLga0owxCZxej1dtofm/cMZ0zzDGvCMi3wIyO+wmwGZjTKc3YmLHiHeyvfkkQhPg+8aYR05iH6WUOl1tBq5rf2GMudPe17PGbjqZ8hWcRvYXgDpgjTGmUUT8No95IvJ1nHK40HaYdMxDcBr/qQ117FCWh+m+DlHqOO0RV0OWiJwtIqm9zFOB7fZ5jR3v19ksKduB4vYZUUQkKCJ96vEwxjQClSLyMXuskIhkAy8Cn0sZc1gqIiV9yUMppU4Dy4BMEfl8yrbsUzje33CGs/xP3u0BvxrYYIwpN8ZUGGNGA08D75mZBVgBzBaR9vuAwiIygXcb3d3VIUodpz3iaijLAX4mIsOBOLATZzjJMZybcQ4DqzvuZIyJijON4QMikofzd/QTnB6bvvgn4BER+TYQA24wxiwVkXOAN+x9Pk3AzUB1H/NQSqlByxhjbIfFj0XkqzjDCZuBu3HGYXd0tohUprz+3x2Ol7A3aN6KM6wEnGEoz3Q4ztPA53Fuvk/d/4gdyviEiITs5m8YY3aIyH/TTR2iVCoxxngdg1JKKaWUUkOODk1RSimllFLKA9oQV0oppZRSygPaEFdKKaWUUsoD2hBXSimllFLKA9oQV0oppZRSygPaEFdKKaWUUsoD2hBXSimllFLKA9oQV0oppZRSygP/H5K7dbkpOupPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12, 5))\n",
    "\n",
    "plt.subplot(1, 2, 1)\n",
    "sns.distplot(y, label='target')\n",
    "plt.title('target')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "sns.distplot(data.GrLivArea, label='area')\n",
    "plt.title('area')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Видим, что распределения несимметричные с тяжёлыми правыми хвостами.\n",
    "\n",
    "Оставим только числовые признаки, пропуски заменим средним значением."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.464075Z",
     "start_time": "2019-09-28T19:10:28.401693Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.3, random_state=10)\n",
    "\n",
    "numeric_data = X_train.select_dtypes([np.number])\n",
    "numeric_data_mean = numeric_data.mean()\n",
    "numeric_features = numeric_data.columns\n",
    "\n",
    "X_train = X_train.fillna(numeric_data_mean)[numeric_features]\n",
    "X_test = X_test.fillna(numeric_data_mean)[numeric_features]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-26T13:06:11.582867Z",
     "start_time": "2019-09-26T13:06:11.570901Z"
    }
   },
   "source": [
    "Если разбирать линейную регрессия с [вероятностной](https://github.com/esokolov/ml-course-hse/blob/master/2018-fall/seminars/sem04-linregr.pdf) точки зрения, то можно получить, что шум должен быть распределён нормально. Поэтому лучше, когда целевая переменная распределена также нормально.\n",
    "\n",
    "Если прологарифмировать целевую переменную, то её распределение станет больше похоже на нормальное:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.771830Z",
     "start_time": "2019-09-28T19:10:28.469382Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEGCAYAAAB1iW6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXicZ33u8e9vRvtq7ZYly/ua2Iltxc6+QEKWhqSUAElaGg6hKS20PV1oKfSEHmjPKaWHQktaSCkH6CkhEKB1WwdnTyBxEstxEq/ybnnRZlnSSJZG0sw8548ZBUXRZms078zo/lyXLmlm3pn39kTzy6PnfRZzziEiIqnP53UAERGJDxV0EZE0oYIuIpImVNBFRNKECrqISJrI8OrE5eXlbuHChV6dXkQkJe3YseOMc65irMc8K+gLFy6koaHBq9OLiKQkMzs+3mPqchERSRMq6CIiaWLSgm5m3zKzNjPbPc7jv2pmb5rZLjN7ycwuiX9MERGZzFRa6N8Gbpng8aPAdc65NcAXgIfjkEtERM7TpBdFnXMvmNnCCR5/acTNl4Ha6ccSEZHzFe8+9PuBx8d70MweMLMGM2tob2+P86lFRGa3uBV0M7uBaEH/k/GOcc497Jyrd87VV1SMOYxSREQuUFzGoZvZWuCbwK3OuY54vKaIiJyfabfQzawO+DHwYefcgelHEhGRCzFpC93MHgGuB8rN7CTwOSATwDn3deBBoAz4BzMDCDnn6mcqsEiife+Vpgkfv3dTXYKSiExsKqNc7pnk8Y8BH4tbIhERuSCaKSoikiZU0EVE0oQKuohImvBs+VyReNPFS5nt1EIXEUkTKugiImlCBV1EJE2ooIuIpAkVdBGRNKFRLjJrTDQKRiNgJB2ohS4ikiZU0EVE0oQKuohImlAfuqS9voEQ2452UJCdwbr5JWRlqB0j6UkFXdJWxDm2He7g6f2tBIciADyxp5Url5Zxw4pKfNH1+0XShgq6pK2XDnewZVczSysLuG1NNQNDYX528AxP72sj0D/EnZfWqKhLWlFBl7QUHArz7P42llUW8JErFxLbTYu60jye3NfKc43tACrqklZU0CUtvXCwnf6hMDdfNPetYg5gZty0qgqA5xrbyfL7uG1NtVcxReJKBV3STiA4xIuHzrC2tph5c3Lf8fhwUR8MRXjxcAdFuZn86uULPEgqEl+63C9p57nGdsIR91ZLfCxmxm1rqllTU8zju1v40Y6TCUwoMjPUQpe0Eo443jjRxcU1xZQVZE94rM+MD2yo5dxgiD967A0GwxHu2aglACR1qYUuaeX42XP0D4W5aF7xlI7P8Pv49csXct3yCv70x7v4xvOHcc7NcEqRmaGCLmll3+kAfp+xvLJgys/JyvDx8IfruX1tNf/78f184nuv0XlucAZTiswMFXRJG8459jYHWFpRQHam/7yem5Xh46t3r+NPblnJk3tbufkrL/D8gfYZSioyM1TQJW20BIJ09g2xurrogp7v9xm/df0SfvLbV1Gcm8l933qVz/37bgZDkTgnFZkZKuiSNvY1BzBgZXXhtF7n4ppi/uN3ruajVy3iO9uO84/PH6InOBSfkCIzaNKCbmbfMrM2M9s9zuNmZn9nZofM7E0zWx//mCKT29fcw/zSPApzMqf9WjmZfh5872q+89GNdJ4b4ps/O0pARV2S3FRa6N8Gbpng8VuBZbGvB4B/nH4skfPTGghyqqufVRfY3TKe65ZXcN+VC+nujxb13oFQXF9fJJ4mLejOuReAsxMccifwXRf1MjDHzDSXWhJq+7Hor+iSivy4v/ai8nzuu3IhnX2DbNnVHPfXF4mXePSh1wAnRtw+GbvvHczsATNrMLOG9naNIJD42XG8k0y/UV38zqn+8bCoPJ/rllfw+okuDrf3zsg5RKYroRdFnXMPO+fqnXP1FRUViTy1pLkdxzupLcnD75u5lROvW15BaX4Wm18/TSiikS+SfOJR0E8B80fcro3dJ5IQfYMh9pwOsKA0b0bPk+n38d6182jvHeDnB8/M6LlELkQ8Cvpm4Ndjo10uB7qdc+polIR540Q34YhjQdnMFnSAFXMLWVFVyM8PnSEUVitdkstUhi0+AmwDVpjZSTO738w+bmYfjx2yBTgCHAL+CfjtGUsrMoYdx6MXROfPcAt92JVLyugbDLP7dCAh5xOZqklXW3TO3TPJ4w74RNwSiZynHcc7WVZZQF5WYhYPXVJZQGl+Fq8c7eDS+XMSck6RqdBMUUlpkYhjx/FONiwoSdg5fWZsWlTK8Y4+WgLBhJ1XZDIq6JLSDrf3EgiGElrQAdbXlZDhM1492pHQ84pMRBtcSEr53itNb7u9/Wi0/7y5K0h54cQbWsRTfnYGF9cUs7Opi77BUMK6e0Qmoha6pLQTnX3kZvopK8hK+LnrF5QwEIrwfKMmyUlyUEGXlNbcHWTenBzMZm5C0XgWlOWTl+Xnib2tCT+3yFhU0CVlhSOOlkCQeXNmZrr/ZPw+Y+XcIp7e18qQxqRLElBBl5TV1hMkHHHMm6H1W6bionlFBIIhXjky0fp1Iomhgi4p63RXP4BnLXSApZUF5Gb6eWJvi2cZRIapoEvKOt0VJMvv8+SC6LBMv49rl5fzxJ5WIhHnWQ4RUEGXFHa6q5/q4hx8HlwQHenmi+bSEgiy61S3pzlEVNAlJUWco7k7SLWH3S3D3rWyEr/P1O0inlNBl5TU0TvIYDhCzZwcr6MwJy+L9XVzeOGAltQVb2l6m6Sk093RC6Lx2qFo9AzU83Xtsgq+/NQBOnoHKCtI3IxVkZHUQpeU1NzVj9+MyqLkKJ7XLK/AOfj5IbXSxTsq6JKSTncFqSrOJsOXHL/Ca2qKmZOXqW4X8VRyfBpEzoNzjtPd/Z5OKBrN7zOuXlrOzw62E90iQCTxVNAl5fQEQ/QNhplb7P0F0ZGuXVZBW88Aja09XkeRWUoFXVLO8KYSc4uSq6Bfs7wcgJ+p20U8ooIuKac1SQt6dXEuy6sKeOGgltMVb6igS8pp6Q5SmJNBXnbyjbq9ZlkFrxw9S/9g2OsoMgsl3ydCZBItgWBStc5HjmEPRxyDoQhf/Ol+llcVcu+mOg+TyWyjFrqklHDE0d4zkFQFfaSFZflk+IyDujAqHlBBl5TS0TtAKOKoSrIRLsOyMnwsLM/nYFuv11FkFlJBl5SSrCNcRlpWWUBbzwDd/UNeR5FZRgVdUkpLIIjPoLIwOab8j2VZZSGAul0k4VTQJaW0dgcpK8gmw5+8v7pVRdkU5WSo20USbkqfCjO7xcwazeyQmX16jMfrzOxZM9tpZm+a2W3xjyqSfCNcxmJmLK0s5FBbL2HtYiQJNGlBNzM/8BBwK7AauMfMVo867M+AHzjn1gF3A/8Q76AivQMhOvuGkm7K/1iWVRbQPxTWLkaSUFNpoW8EDjnnjjjnBoHvA3eOOsYBRbGfi4HT8YsoEtXYEu2TTvYWOkQ3jzbgZwc0a1QSZyoFvQY4MeL2ydh9I/058GtmdhLYAvzOWC9kZg+YWYOZNbS36xddzs/+lgCQGgU9PzuDeXNytQyAJFS8rizdA3zbOVcL3Ab8i5m947Wdcw875+qdc/UVFRVxOrXMFo0tPWRn+JiTl+l1lClZVlnAa01d9AQ1fFESYyoF/RQwf8Tt2th9I90P/ADAObcNyAHK4xFQZNj+lh6qinIwM6+jTMmyqkLCEcdLhzu8jiKzxFQK+nZgmZktMrMsohc9N486pgl4N4CZrSJa0PW3psSNc479zYGU6G4ZVleaR36WnxfUjy4JMmlBd86FgE8CW4F9REez7DGzz5vZHbHD/hD4DTN7A3gE+IjTti0SRy2BIIFgKGmn/I/F7zOuWFLOzw5qfXRJjCmttuic20L0YufI+x4c8fNe4Kr4RhP5hf0pNMJlpOuWl/PUvlaOnTnHwvJ8r+NImkve6XYiI6TSkMWRrl0evfiv0S6SCCrokhL2NweoLs4hN8vvdZTzsqAsn7rSPJ5rVEGXmaeCLilhf0sPK+YWeh3jgty4qoqfHzpD70DI6yiS5lTQJekNhSMcbu9l5dyiyQ9OQreumctgKMIz+9u8jiJpTgVdkt6R9nMMhR0rU7SFvqGuhIrCbH66u9nrKJLmVNAl6Q1P+U/VLhefz7j5oiqe3d+uzaNlRqmgS9JrbOkhw2csqSjwOsoFu/XiavqHwjyvSUYyg1TQJentb+lhSUUBWRmp++u6aVEpJXmZ6naRGZW6nxCZNfY1B1hZnZrdLcMy/D5uWl3F0/vaGAip20Vmhgq6JLWuvkGau4Osqk7NES4j/dLaefQMhHhWo11khkxp6r+IV/Y1R2eIpkNBv3ppOVVF2Ty24yS3XFzN915pmvD4ezfVJSiZpAu10CWp7WuOjnBZleJdLhBdrOt962p5trGd9p4Br+NIGlJBl6S2rzlAeUEWlYWptYbLeO7aUEM44vj310dvKSAyfSroktT2tQRSdoboWJZWFnLJ/Dn8sOEkWmFa4k0FXZJWKBzhQGtvWnS3jHTXhloaW3s43R30OoqkGV0UlaQy8kJhayDIYChCV9/QpBcQU8kda+fxhf/cy/ZjZ6m5dPR+6yIXTi10SVotsRbs3BTapWgqivMyueOSebze1KWlACSuVNAlaTV3B/GbUVGY7XWUuPvIlQsZDEfY0dTpdRRJIyrokrRaAv1UFGaT4Uu/X9OLa4pZUJrHy0c6iOjiqMRJ+n1SJG00dwepTrPulpGuWFLG2XODHIhtrycyXSrokpR6B0L0BENp138+0kXziinKyeClIx1eR5E0oVEukpSGL4hWF+d6nGR6Jhqd4/cZmxaX8eTeVtoCQSpTbANsST5qoUtSaunuB9JvhMtoly0sJcNnbFMrXeJABV2SUnN3kMLsDAqy0/uPyILsDNbWFrOzqYvgkIYwyvSooEtSagkE0751PuyKxeXRIYzHNYRRpkcFXZJOKBKhLTCQ1iNcRqopyaWuNI9tGsIo0zSlgm5mt5hZo5kdMrNPj3PMB81sr5ntMbPvxTemzCbtPQOEnWNuil8QPR9XDg9hbNUQRrlwkxZ0M/MDDwG3AquBe8xs9ahjlgF/ClzlnLsI+O8zkFVmiV+McJkdLXSIDmEszMngZV0clWmYSgt9I3DIOXfEOTcIfB+4c9QxvwE85JzrBHDOaY8tuWAt3UH8PqO8IP2m/I/H7zM2LirlQGsvZ3q1+YVcmKkU9BrgxIjbJ2P3jbQcWG5mL5rZy2Z2y1gvZGYPmFmDmTW0t7dfWGJJe82BIFWF2fh95nWUhNq4sBS/mVrpcsHidVE0A1gGXA/cA/yTmc0ZfZBz7mHnXL1zrr6ioiJOp5Z009wdnFX958MKczK5uKaIHcc7GQhpCKOcv6kU9FPA/BG3a2P3jXQS2OycG3LOHQUOEC3wIuelJzjEuYHQrOo/H+mKxWUMhCK8fqLL6yiSgqZS0LcDy8xskZllAXcDm0cd829EW+eYWTnRLpgjccwps0Rzmq6BPlXzS/OYNyeHbYc7tEWdnLdJC7pzLgR8EtgK7AN+4JzbY2afN7M7YodtBTrMbC/wLPAp55w6AuW8vTXCZZaua2JmXLG4nLaeAV4+ctbrOJJipjSv2jm3Bdgy6r4HR/zsgD+IfYlcsNPd/RTnZpKX5lP+J7K2tpjHdzfznZeOccWSMq/jSArRTFFJKqc6+6mZM/suiI6U6fdRv6CUJ/a2cKqr3+s4kkJU0CVpBIJDdJwbZN4sL+gAmxaXAvC9V457nERSiQq6JI09pwIAs76FDlCSl8WNq6p45NUTGsIoU6aCLklj96luILpYlcCHr1jA2XODPLGn1esokiJU0CVp7DrVTXFuZtqvgT5VVy0pp7Ykl+9vH3/XI5GRVNAlaew+3a3+8xF8PuND9fN58VAHTR19XseRFKCCLkmhdyDE0TPnqJkzO8efj+eu+lp8Bj9oODH5wTLrqaBLUthzqhvndEF0tOriXK5fUckPd5wgFI54HUeSnAq6JIVdsQui6nJ5pw9dNp/WwADPNWqFUpmYCrokhd2nuqkqyqYwJ9PrKEnnXSsrqSjM5vvb1e0iE1NBl6Sw61Q3a2qKvY6RlDL9Pu7aUMuzjW20BoJex5EkpoIunuvuH+Jw+znW1r5jCX2J+WD9fMIRx2M7TnodRZKYCrp47s2T0bW/19WpoI9nUXk+ly8u5dHtJ4hEtKyujE0FXTz3elMXZnDJfBX0idx9WR1NZ/u0RZ2MSwVdPLfzRBdLKwoo0gXRCd1y8VyKczN5RBdHZRwq6OIp5xw7mzq5VK3zSeVk+nnfuhq27mmhq2/Q6ziShFTQxVNNZ/vo7BtiXV2J11FSwgfr5zMYivBvO0dv6yuigi4e29mkC6LnY/W8ItbUFPNow0ntOSrvoGXtxFM7mzrJy/KzvKrQ6yhJ53uvjL3K4qLyfDa/cZrdpwKsqdXYffkFtdDFU6+f6GJtbTF+n3kdJWVcUjuH7AwfjzZoWV15OxV08UxwKMye0wH1n5+n3Cw/t148l39//TTBIe1mJL+ggi6e2XO6m1DEaYTLBfjgZfPpCYZ4fHez11EkiagPXTzz6tFOADYsUAv9fB1pP0dpfhZ/9/Qh+gffuazuvZvqPEglXlMLXTzz8pEOllYWUF6Q7XWUlOMzY8OCEo6eOUdH74DXcSRJqKCLJ0LhCA3HznL54lKvo6Ss9XUlGLDjeKfXUSRJqKCLJ/acDnBuMMymRWVeR0lZxbmZLK8q5LWmTsJasEuYYkE3s1vMrNHMDpnZpyc47v1m5sysPn4RJR0NLzC1SS30admwoIRAMMTBth6vo0gSmLSgm5kfeAi4FVgN3GNmq8c4rhD4PeCVeIeU9PPK0bMsLs+nslCbQk/HyupC8rMz2H5M3S4ytRb6RuCQc+6Ic24Q+D5w5xjHfQH4IqAtVWRC4Yhj+9GzbFqs7pbpyvD52FBXQmNLgO7+Ia/jiMemUtBrgJHrdZ6M3fcWM1sPzHfO/ddEL2RmD5hZg5k1tLdrw9vZal9zgJ6BkC6IxsnGRaU4Bw3HznodRTw27YuiZuYDvgz84WTHOuceds7VO+fqKyoqpntqSVFv9Z/rgmhclOZnsayqgO3Hzuri6Cw3lYJ+Cpg/4nZt7L5hhcDFwHNmdgy4HNisC6Mynm2HO1hQlsfcYvWfx8vGhWUEgiEaWwJeRxEPTaWgbweWmdkiM8sC7gY2Dz/onOt2zpU75xY65xYCLwN3OOcaZiSxpLTgUJgXD5/h+uX6Cy2eVswtpDg3k1eOqttlNpu0oDvnQsAnga3APuAHzrk9ZvZ5M7tjpgNKetl2uIPgUIR3raryOkpa8fuM+oUlHGzr5Yxmjs5aU+pDd85tcc4td84tcc79Zey+B51zm8c49nq1zmU8T+9vJS/Lz6ZFuiAabxsXluI3Y9thbSI9W2mmqCSMc45n97dz9dJycjL9XsdJO4U5maytLWbH8U4NYZylVNAlYRpbezjV1c+7V1V6HSVtXbm0nMFwhB82nJj8YEk7KuiSME/vawPghhUq6DOlZk4uC8vy+b8vHiMUfueyupLetB66JMwz+9tYW1vMU7HCLjPjqqVl/OsrTTy5t5Vb11R7HUcSSC10SYj2ngFea+pU6zwBVlUXsaAsj398/jDOaaLRbKKCLgnxbztP4Ry895J5XkdJez4zPn7dEt482c3PD53xOo4kkAq6zDjnHI/tOMm6ujksrSzwOs6s8Cvra5hblMPXnjnkdRRJIBV0mXG7TnXT2NrDXRtqvY4ya2Rn+PmNaxfzytGzWrRrFlFBlxn3w4aTZGf4uH2tulsS6Z6N8ynNz+Jrz6qVPluooMuMCg6F2fzGaW6+aC7FuZlex5lV8rIy+Ng1i3iusV37js4SKugyo57a10p3/xAfqFd3ixc+cuVCyguy+JutjV5HkQTQOHSZMc45vv78YRaU5XHlknKv48xKeVkZfOKGpfzP/9jLi4fOcNXScr73StO4x9+7qS6B6STe1EKXGfP0vjZ2nwrwiRuW4veZ13FmrXs31TGvOIe/3tqocelpTgVdZoRzjq88fYC60jzet65m8ifIjMnO8PN7Ny7jjRNdbN3T4nUcmUEq6DIjntkfbZ1/8oalZPr1a+a196+vZXlVAf9ry36t8ZLG1IcuF2SifthwxPH97U3ML83lfevVOk8GGX4f/+P21Xz4n1/lpcMdXKsdo9KSmk4Sdy8dPsOe0wH++OaVap0nkWuWVfCulZU829hG70DI6zgyA/Rpk7jq6B3gyb2t3LiqitvXaqW/ZPOZ21YxFI7w1N5Wr6PIDFCXi8RNxDl+vPMUfp+xYUEJj7yqTRaSzdLKAi5fXMa2wx1cvriMucU5XkeSOFILXeLmpUNnOHrmHLetqdas0CT2rpWV5GT6+a9dpzWMMc2ooEtcNHWc46d7WlhdXUT9ghKv48gE8rIyePeqSg63n2N/S4/XcSSOVNBl2voGQjyy/QTFuZm8f30tZppElOw2LSqjoiCbLbuaCUU0jDFdqKDLtESc44c7TtI7EOKejXXkZvm9jiRT4PcZv7S2mo5zg/z8oDbBSBcq6DItPzt4hsbWHm5bU01tSZ7XceQ8LK8q5KJ5RTyzv42z5wa9jiNxoIIuF+zomXM8ubeFi2uKuXxRqddx5ALcvnYePp/xH2/oAmk6UEGXC9I3GOLR7U2U5GXxK+tq1G+eoopzM7lxZSWNrT3sOR3wOo5M05TGoZvZLcBXAT/wTefcX416/A+AjwEhoB34qHPueJyzShLZsquZ3oEQv3X9UnIy1W+ebCZammG0K5aU81pTF/+1q5nP/tIq8rM1PSVVTdpCNzM/8BBwK7AauMfMVo86bCdQ75xbCzwG/HW8g0ryeLaxjdeaurhueQU1c3K9jiPT5PcZv3zpPLr7h/jKUwe8jiPTMJUul43AIefcEefcIPB94M6RBzjnnnXO9cVuvgxoe5o01RMc4jM/3kVlYTY3rKj0Oo7ESV1ZPpctLOFbLx5jX7O6XlLVVAp6DTByDvfJ2H3juR94fDqhJHl9+ckDtAaCvH99LRlaeCutDO/7+pmf7CIc0QXSVBTXT6SZ/RpQD3xpnMcfMLMGM2tob2+P56klAQ639/Iv247zocvqmF+qIYrpJi8rgwdvX83Opi4efuGI13HkAkzl6scpYP6I27Wx+97GzG4EPgtc55wbGOuFnHMPAw8D1NfXqwmQ5EZfWPvutmP4fcbCMhXzdHXnpfN4Ym8LX36yketXVLCqusjrSHIeptJC3w4sM7NFZpYF3A1sHnmAma0DvgHc4Zxri39M8dqhtl72t/Rww4pKCnO08Fa6MjP+4pfXUJybxe8/+joDobDXkeQ8TFrQnXMh4JPAVmAf8APn3B4z+7yZ3RE77EtAAfBDM3vdzDaP83KSgiLOsWVXMyV5mVy5pMzrODLDSvOz+Ou71rC/pYcv/Oder+PIeZjSgFPn3BZgy6j7Hhzx841xziVJZNepbloCQT5YP18XQmeJd62s4jevXcw3XjjC+roSfmW9Bq6lAn06ZULhiOPpfW1UFWWztrbY6ziSQJ+6eQWbFpXymZ/s0lDGFKGCLhN640QXZ3oHuHFVFT5N759VMvw+/v7edRTlZPKx7zTQFgh6HUkmoTm+Mq5QJMLT+1uZNyeH1RrtMCtVFubwz/ddxoce3sZHv7Od96+vJTtj7KUe7t1Ul+B0Mppa6DKuHcc76ewb4qZVVVp8axZbU1vMQ/euZ+/pAI+82qQNMZKYCrqMKTgU5tn9bdSV5rG8qtDrOOKxG1ZW8pfvW8OB1l4e3X5CM0mTlAq6jOmRV5sIBEPctFqtc4m6Z2Mdv7Smmj2nA/xwxwkiWj896agPXd6hbzDEQ88eZnF5PksqCryOI0nkqqXlhCOOn+5pwW/G+zfU6mJ5ElFBl3f47rbjnOkd4P3rJ1qDTWara5dXEHGOJ/a24vMZ71tXo6KeJFTQ5W06zw3y0LOHuH5FBQvK8r2OI0nq+hWVhCKOZ/a34TPjzkvneR1JUEGXUb769EHODYT4zG2raDjW6XUcSbDz2eno3SsriTjHc43tOOe4d2MdPp9a6l7SRVF5y9Ez5/h/L0eXx9XIFpmMmXHTqiretbKShuOdfOqxNzX6xWNqoctb/urxfWRn+Pj9m5Z5HUVShJlx46oqzOBHr50kHInwNx+4RGv+eEQFXQB4Zn8rW/e08oc3LaeyMMfrOJJi3r2yivV1JXxpayNhB3/7QRV1L6igC919Q/zpj3exoqqQB65b7HUcSVGfuGEpfp/xV4/vJxyJ8NW715Gpop5QKujC5/9zL2d6B/nmr1827jodIlPx8euWkOEz/uK/9hGOvMbf37OerAwV9UTROz3L/XR3Cz967SS/dd0S1mh5XImDj12zmM+9dzVb97TyW/9vB8Eh7XqUKGqhz2JvnOji9x99nbW1xfzOu5d6HUdS3Mghj9kZfu64ZB6b3zjNLV95gQ9fvpD7r1nkYbrZQS30Waqpo4/7v7Od8sIs/vk+dbVI/F2+uIwPXTafE2f7+aefHaGlW+upzzQV9Floz+lu7vmnlwlFHN/+bxupKMz2OpKkqUtq53DflQs52zfIHV/7OTubNFltJqmgzzI/fu0kv/IPLxGOOL770Y1afEtm3NLKAj5+3RKyM3186Bsv84PtJ3BaqXFGqKDPEi8f6eCeh1/mD37wBuvq5vCfv3s1a2vneB1LZom5RTls/sTV1C8s4Y9/9CaffGQn3X1DXsdKO7oomqb6BkPsa+7h+QPtPLm3lX3NASoLs/nce1fz4csXaNKHJFxJfhb/cv8mvv78Yf72yQO8dryTz733Im6+SGvux4t59adPfX29a2ho8OTc6SQUjvC1Zw7REgjSGgjSEhigNRCk89wgDvAZ1C8o5fZLqvlg/XxyMn9x8fN8FmISiaeTnX08tuMkbT0DLKnI56t3r+PiGg2bnQoz2+Gcqx/zMRX01NLc3c9zje3sON7J/pYAB1p7GQxF93j0GZQVZFNVlMPcouj3hWX55GfrDzFJPuGI49WjHTy5r5XgUISrlpbxsasXc/Wycs0wnYAKegobCkf44k/3c6ClhwOtvbQEokO/8qlwkdIAAAqESURBVLMzmFecw9yiHKpi3ysKs/VBkJTTPxhmMBzh/754lLaeAebkZXLz6rlctayc+gUlzJuT63XEpKKCnmJaA0Gea2zjucZ2fn7wDD0DIXwGC8vyWTG3kOVVhVQWZqvfUdLGvZvqGAiFeb6xnS27mnlqXxu9AyEAyvKzqCvLY0FpHnVl+dSV5jG/JJeaklzmFuXMuutBExV0/S0+ysh+5cFQhIFQmFDYEXEOn8/4QH0tJXlZcW0JD4Uj7Gzq4rnGNp5tbGdfcwCA6uIcbr+kGp8ZSyoK3tb/LZJusjP8vOeiubznormEwhH2NffQcPwsW3Y103FukOcOtNPdd5qRTVADinIzmZOXSUleFnNyM5mTl0VJfvT2b163eFZNmptSQTezW4CvAn7gm865vxr1eDbwXWAD0AF8yDl3LL5R48s5R1ffEE1n+zh+to8TZ/s43nGO7cc66eobpHcgxFD4nX+9fGlrIwB5WX4KsjMozMmgMCeT4txMblpdRc2caMuhsjCbwpxM/CN2cHHO0d0/REsgyMHWXg609rDjeCc7m7roHwrjM1hQls8tF81leVUhVUVqhcvslOH3saa2mDW1xW8ryKFIhK5zQ3T2DdLVN0RXf/R7Z98QxzvO8Wb/ECP32Pjbpw5QVZhDbUku80vzmFucE/3MZmfw5slucjL9ZGX4yPAZmf7o9wy/j0y/cffGOnJjj6eKSbtczMwPHABuAk4C24F7nHN7Rxzz28Ba59zHzexu4H3OuQ9N9Lrx6HJxzhGOOMKx76GIIxR2nBsIEQgO0RMM0RMMRYtodz+nu4M0d/XT3B3kVFc/PcHQ216vojCb3Ew/JXmZFOZkkp/lJzvTT6bf8Jm9dY5zAyF6B6Kv3Rs7V2DUL1L0fYH8rAx8Bo5oX2FoxEE+g5Vzi9i4qJTBUISllWqFi0xHOOLoCUYLfOe5QWpLcznZ2c+Js32c7OynNRB822dwKnIyfRTlZFKUm0lhTsZbPxfFGnNFuRnveLw4dl9hTiY5mb64Nsym2+WyETjknDsSe7HvA3cCe0cccyfw57GfHwO+ZmbmZqCD/vFdzfzeo69HC/l5/ocpzc+iujiH2pI8Ni4qpa40jwXDfXKlueRlZVzwUL6Ic9y4qopTXf2c7uqnrWeA7v4heoJDDL8LeVl+ygqyqSjMZmlFAYsr8t8q4BpCKDJ9fp8xJy+LOXlZLCrP595NdW973DnHQChCTzDEI682ERyKXpANhR2hcIShWKNwKBxhbW0x/YNhegZCBPqH3mokdvUN0nS2j57gEN39Q2P+JT+aGfjM8Fl0l6cHrlnMH928Iu7//qkU9BrgxIjbJ4FN4x3jnAuZWTdQBpwZeZCZPQA8ELvZa2aNFxL6Qh0Hdp7/08oZ9e9IUsoZX8oZX57k/NXzf0pCcn4q9nWBFoz3QEIvijrnHgYeTuQ5p8vMGsb78yaZKGd8KWd8KWdiTKW3/xQwf8Tt2th9Yx5jZhlAMdGLoyIikiBTKejbgWVmtsjMsoC7gc2jjtkM3Bf7+S7gmZnoPxcRkfFN2uUS6xP/JLCV6LDFbznn9pjZ54EG59xm4J+BfzGzQ8BZokU/XaRKF5FyxpdyxpdyJoBnM0VFRCS+UmfEvIiITEgFXUQkTczagm5m3zKzNjPbPeK+UjN70swOxr6XjPPcsJm9HvsafYE4ETk/YGZ7zCxiZuMOsTKzW8ys0cwOmdmnkzjnMTPbFXs/Z3TFtnFyfsnM9pvZm2b2EzMbcyunJHg/p5rT6/fzC7GMr5vZE2Y2b5zn3hf7rB00s/vGOiZJcibs8z5tzrlZ+QVcC6wHdo+476+BT8d+/jTwxXGe2+txzlXACuA5oH6c5/mBw8BiIAt4A1idbDljxx0Dyj18P98DZMR+/uJY/92T5P2cNGeSvJ9FI37+XeDrYzyvFDgS+14S+7kk2XLGHkvY5326X7O2he6ce4HoiJyR7gS+E/v5O8AvJzTUGMbK6Zzb55ybbJbtW0s2OOcGgeElG2bENHIm1Dg5n3DODS/s8zLRuRajJcP7OZWcCTVOzsCIm/nAWCMvbgaedM6ddc51Ak8CtyRhzpQyawv6OKqcc82xn1uAqnGOyzGzBjN72cw8L/rjGGvJhhqPskzGAU+Y2Y7Y8hBe+ijw+Bj3J9v7OV5OSIL308z+0sxOEJ19/+AYhyTF+zmFnJAan3dABX1cLvq31nj/x17gotOD7wW+YmZLEpcsLV3tnFsP3Ap8wsyu9SKEmX0WCAH/6sX5p2oKOT1/P51zn3XOzSea8ZOJPv9UTTFnynzeVdDfrtXMqgFi39vGOsg5dyr2/QjR/uF1iQp4HqayZENSGPF+tgE/Idq9kVBm9hHgduBXY/8zHy0p3s8p5EyK93OEfwXeP8b9SfF+jjBezlT5vAMq6KONXMLgPuDfRx9gZiUW3dADMysHruLtSwkni6ks2eA5M8s3s8Lhn4le+Ns98bPinuEW4I+BO5xzfeMc5vn7OZWcSfJ+Lhtx805g/xiHbQXeE/s8lRDNuTUR+YZNJWcKfd6jvL4q69UX8AjQDAwR7b+7n+iSv08DB4GngNLYsfVEd2oCuBLYRXSUwy7gfg9yvi/28wDQCmyNHTsP2DLiubcR3ZzkMPDZZMxJdNTIG7GvPR7lPES0P/f12NfXk/T9nDRnkryfPyL6P5E3gf8AakZ/jmK3Pxr7Nx0C/lsy5kz05326X5r6LyKSJtTlIiKSJlTQRUTShAq6iEiaUEEXEUkTKugiImlCBV1Smpl9Nrai4/CqeZsmOPbbZnbXJK/3bTM7Gnut18zsinGO+7iZ/fp084vE06Rb0Ikkq1ixvR1Y75wbiE38yIrDS3/KOfeYmb0H+AawdtR5M5xzX4/DeUTiSgVdUlk1cMY5NwDgnDsDYGYPAu8FcoGXgN90oyZcmNkG4MtAAXAG+Ij7xcJsw14AlsaOf47oZJ6rgUdiszF7nXN/Y2ZLga8DFUAY+IBz7rCZfQr4IJAN/MQ597k4//tF3kZdLpLKngDmm9kBM/sHM7sudv/XnHOXOecuJlrUbx/5JDPLBP4euMs5twH4FvCXY7z+e4nODhyW5Zyrd879n1HH/SvwkHPuEqIzC5tjrftlRNdRuRTY4NWiYzJ7qIUuKcs51xtraV8D3AA8GttJqMfM/hjII7qBwh6iU7uHrQAuBp40M4huXjGydf4lM/szoJ3oFPFhj47OEGup1zjnfhLLFIzd/x6i65PsjB1aQLTAvzCdf7PIRFTQJaU558JEV8B7zsx2Ab9JtM+73jl3wsz+HMgZ9TQD9jjnxrzgSawPfYz7z51HNAP+t3PuG+fxHJFpUZeLpCwzWzFqxbxLgeEdks6YWQEw1qiWRqBieASLmWWa2UUXksE51wOcHN74wMyyzSyP6MqBH41lwMxqzKzyQs4hMlVqoUsqKwD+PrZZcojoqn0PAF1EV9FrIbrs7ds45wZjwxf/zsyKiX4OvkK0a+ZCfBj4hpl9nuhqfh9wzj1hZquAbbFunV7g1xhnjX2ReNBqiyIiaUJdLiIiaUIFXUQkTaigi4ikCRV0EZE0oYIuIpImVNBFRNKECrqISJr4/6wlNkVRpeYoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(np.log(y+1), label='target')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Сравним качество линейной регрессии в двух случаях:\n",
    "1. Целевая переменная без изменений.\n",
    "2. Целевая переменная прологарифмирована.\n",
    "\n",
    "Не забудем вернуть во втором случае взять экспоненту от предсказаний!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.833250Z",
     "start_time": "2019-09-28T19:10:28.801394Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test RMSE = 32085.7681\n"
     ]
    }
   ],
   "source": [
    "model = Ridge()\n",
    "model.fit(X_train, y_train)\n",
    "y_pred = model.predict(X_test)\n",
    "\n",
    "print(\"Test RMSE = %.4f\" % mean_squared_error(y_test, y_pred) ** 0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.860025Z",
     "start_time": "2019-09-28T19:10:28.840089Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test RMSE = 26649.2742\n"
     ]
    }
   ],
   "source": [
    "model = Ridge()\n",
    "model.fit(X_train, np.log(y_train+1))\n",
    "y_pred = np.exp(model.predict(X_test))-1\n",
    "\n",
    "print(\"Test RMSE = %.4f\" % mean_squared_error(y_test, y_pred) ** 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Попробуем аналогично логарифмировать один из признаков, имеющих также смещённое распределение (этот признак был вторым по важности!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.883024Z",
     "start_time": "2019-09-28T19:10:28.866508Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train.GrLivArea = np.log(X_train.GrLivArea + 1)\n",
    "X_test.GrLivArea = np.log(X_test.GrLivArea + 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.915423Z",
     "start_time": "2019-09-28T19:10:28.887909Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test RMSE = 31893.8891\n"
     ]
    }
   ],
   "source": [
    "model = Ridge()\n",
    "model.fit(X_train[numeric_features], y_train)\n",
    "y_pred = model.predict(X_test[numeric_features])\n",
    "\n",
    "print(\"Test RMSE = %.4f\" % mean_squared_error(y_test, y_pred) ** 0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.946812Z",
     "start_time": "2019-09-28T19:10:28.919832Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test RMSE = 25935.0780\n"
     ]
    }
   ],
   "source": [
    "model = Ridge()\n",
    "model.fit(X_train[numeric_features], np.log(y_train+1))\n",
    "y_pred = np.exp(model.predict(X_test[numeric_features]))-1\n",
    "\n",
    "print(\"Test RMSE = %.4f\" % mean_squared_error(y_test, y_pred) ** 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как видим, преобразование признаков влияет слабее. Признаков много, а вклад размывается по всем. К тому же, проверять распределение множества признаков технически сложнее, чем одной целевой переменной."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Бинаризация"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Мы уже смотрели, как полиномиальные признаки могут помочь при восстановлении нелинейной зависимости линейной моделью. Альтернативный подход заключается в бинаризации признаков. Мы разбиваем ось значений одного из признаков на куски (бины) и добавляем для каждого куска-бина новый признак-индикатор попадения в этот бин."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:28.966252Z",
     "start_time": "2019-09-28T19:10:28.952975Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "np.random.seed(36)\n",
    "X = np.random.uniform(0, 1, size=100)\n",
    "y = np.cos(1.5 * np.pi * X) + np.random.normal(scale=0.1, size=X.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:29.287834Z",
     "start_time": "2019-09-28T19:10:28.972865Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x11b50b7f0>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAboElEQVR4nO3df5Dcd13H8ec7l0u9oHKFRG23TRPHGqxEOHoD1cyoLUgKanO2Aq0yUgfs+KMyrUzG68hIQWZ6mlHEkRErMBZBSKWd85jWuVGvTmeiYXrxUkOAYCi0zYISaC8zmCu93L39Y/d6e5vv97vf3f3u9+frMdPp7e53dz/fJPf+fr7vz/vz+Zi7IyIi5bcp6waIiEg6FPBFRCpCAV9EpCIU8EVEKkIBX0SkIjZn3YAw27Zt8507d2bdDBGRQjl69Oi33H170Gu5Dfg7d+5kfn4+62aIiBSKmT0Z9ppSOiIiFaGALyJSEQr4IiIVoYAvIlIRCvgiIhWhgC8iUhG5LcvMq+mFOgdnT/L1xSUuHR3hwL7dTIzVsm6WiEhHiQR8M/sY8AvAN9395QGvG/BB4I3AOeBWd//PJL47zCAC8/RCnbsePM7S8goA9cUl7nrwOICCvojkXlIpnb8Fro94/Q3Alc3/bgP+KqHvDbQWmOuLSzjrgXl6od7X5x6cPflCsF+ztLzCwdmTfX2uiEgaEgn47v4o8EzEIfuBj3vDEWDUzC5J4ruDdArM0wt19k7NsWvyIfZOzcW+EHx9camr50VE8iStQdsa8HTL49PN5zYws9vMbN7M5s+cOdPzl0UF5n56/5eOjnT1vIhInuSqSsfd73X3cXcf3749cO2fWKICcz9pmQP7djMyPLThuZHhIQ7s291zW+Pq9a5ERGRNWgG/Dlze8viy5nMDERWY6yG9/3qz9x9lYqzGPTfuoTY6ggEXbx3mos2buPPQsYEG4UGNSYhItaQV8GeAX7OGa4Cz7v6NQX1Ze2CujY5wz417ALCI98UJohNjNQ5PXscH3vJKnlteZXFpeeBBWIPFIpKEpMoyPwX8LLDNzE4D7wGGAdz9w8DDNEoyT9Eoy/z1JL43ysRY7YJSyb1Tc3jEe9aCaKcSy+mFOu+6/3FWfOOndXp/r6WiGiwWkSQkEvDd/ZYOrzvwO0l8Vy/WAm1YOqdVpyC6ll5pD/ad3t9PDf+loyOBbddgsYh0I1eDtoPQmv+Oo1MQDUqvxHl/UQeLRaQ8Sr+0QqcA3SpOEI26A4h6fz9pmbU7AC3pICL9KH3AjwqoF28dxh3OLi3HDqJh6ZUhM+65cU/o++OkZaJy/EFjEiIi3Sh9wA8LtLXREQ5PXtf15x3Yt3tDLh4aPfuoYB/1vrU7Aq3TIyKDVvocftL577CSz05BudP7VHopIoNW+h7+IPLfvaZXot6n0ksRGbTSB3woRv5bpZciMmilT+kUhUovRWTQKtHDL4Ko1JN22RKRJCjg50hQ6knVOyKSFAX8nAjrxUdV7yjgi0g3FPBzIKoXH6d6RykfEYlDg7Y5ENWL77TLltbKF5G4FPBzIKoXH1a9c+3LtrN3ao47Dh3ThC0RiUUpnRyIqsEPqt659mXbeeBoPXJROE3YEpF2Cvg50Gmdnfbqnb1Tcx1XANWELRFpp4CfExdt3vRCEL946zDv+cUfDyzRjLORiyZsiUgQBfyMtVfoADy3vBrruCA1VemISAgF/IzFrbPvtJFL+xLNKtUUkXYK+BmLu0pm1CBse69es3NFJIjKMjPWqc6+03FrG7l0uhtQqaaIKOBnLO4qmd2spqm19UUkiFI6GYu7QUvc46YX6mwyY8X9gu9SqaZItZkHBIY8GB8f9/n5+aybUShRlTxx9t0VkeIzs6PuPh70mlI6JRJWyTNkpmAvIkrp5FUvZZVhOfpVdwV7EVHAz6NeyypHtw7z7LnlC55X7l5EQAE/l3rZ9OTd08cDg/3wkHVcZkGTtESqQQE/h7otq5xeqPPJI08FvvaiLZsjg7cmaYlUhwZtcyjuZKw1B2dPElZrdXbpwl5/+3s1SUukGhTwc6ibSVYQPaGqU/5ek7REqkMBP4cmxmrcc+MeaqMjGI3lE6LKKsOCukHH/H23dxMiUlzK4edU+6Yn7VoHWl88MszwkLG8sp7YMeBXr9nRMQ/fafMVESkPBfwCah9oXVxaZniTcfHWYRbPLXdVaRN3yQYRKb5EAr6ZXQ98EBgCPuLuU22v3wocBOrNp/7S3T+SxHdXUdBA6/Kqs3XLZhb+8PVdf16nuwkRKYe+A76ZDQEfAn4OOA08ZmYz7v6FtkMPufvt/X6faKBVRHqTxKDtq4FT7v6Euz8PfBrYn8DnSggNtIpIL5JI6dSAp1senwZeE3DcTWb208CXgTvd/en2A8zsNuA2gB07diTQtHLqdaC1fUbttS/bziNfOqPcvUhFpFWW+Vlgp7v/BPDPwH1BB7n7ve4+7u7j27dvT6lpxdNt2SasD/TWF5dwGjNqP3HkqQ2P73rwONML9dDPEJFiS6KHXwcub3l8GeuDswC4+7dbHn4E+JMEvrfSuh1o7bQJOnRer0dEii2JHv5jwJVmtsvMtgA3AzOtB5jZJS0PbwC+mMD3ShfiDuhq4FekvPoO+O5+HrgdmKURyO939xNm9j4zu6F52DvN7ISZPQ68E7i13++VddMLdfZOzbFr8iH2Ts0FpmXiDug6hH6GiBSbtjgsuKBtDYO2M4za/jCItkQUKSZtcVhiYatdvuv+xy/o8V+0ef2v++Ktw7z1mh3UQnr+WjFTpHy0tELBheXcV5p3bvXFJQ585nHwxmzcNc8trzJ+xUsYv+Il3HHoWOBn1JXPFykVBfyCu3R0pGNgbl1Ubc3S8gp3z5zgu+dXQ99nNFJBSuuIlINSOgUXtHZ+XItLy5E5fQeldURKRD38gmtf7XKT2QvpnCSoTFOkPBTwS6B1ElZQNc4mg9UerwFan0ekPBTwS6a9x//ikWH+7/nzrAbk8aGRpw+7FmgjFJFyUcAvodYe/96pORYjNjJ3GmvxfH1xidGtw7g3Nj7XYmoi5aOAX3KdcvC10REOT16XUmtEJEuq0im5qBy8UjYi1aKAX3JhZZtmcNPV2tpQpEqU0im5tYD+3s+e4Nlz67l8d3jgaJ3xK16SStBv33xF4wMi6VMPvwImxmps3XLhtT2t9XKCNl/RZisi6VPAr4gsNz4PW+BNs3hF0qWAXxFZbnye5cVGRNYph18RvW583ov2fP3o1uEN4wdrNItXJF0K+BXRPgN3UAOn7Us71BeXGN5kDA/ZhlU7VRIqkj4F/ArpduPzXgTl65dXndGRYV500WZV6YhkSAFfEhWWlz+7tMyx97w+5daISCsN2kqishwcFpFoCviSqKCZvcrXi+SDUjoSqpfZsWkNDotI9xTwJVBQtc1dDx4HiBX0FeBF8kcpHQmk2bEi5aOAL4E0O1akfBTwJZCqbUTKRwFfAqnaRqR8NGgrgZKqttE6+CL5oYAvofqttumn0kdEkqeALwMTVekzMVZT718kZQr4MjBRlT7q/YukT4O2MjBRlT6q8xdJnwK+DExUpU9Y77+uOn+RgVHAl4GZGKtxz417qI2OYEBtdIR7btzDxFgttPdvoM3NRQbE3L3zUZ0+xOx64IPAEPARd59qe/0i4OPA1cC3gbe4+9eiPnN8fNzn5+f7bpvk0/RCnTsPHSPoX9+QGavuGsgV6YGZHXX38aDX+u7hm9kQ8CHgDcBVwC1mdlXbYW8HnnX3HwE+APxxv98rxTYxVgsM9gAr7jjrA7nq8YskI4mUzquBU+7+hLs/D3wa2N92zH7gvubPnwFea2aWwHdLgdViLNOggVwps+mFOnun5tg1+RB7p+YG3rlJIuDXgKdbHp9uPhd4jLufB84CL23/IDO7zczmzWz+zJkzCTRN8ixoUDeIFmyTMlorTa4vLqV2R5urQVt3v9fdx919fPv27Vk3RwasfVB3KOSmTwu2SRmFlSa/97MnBvadSQT8OnB5y+PLms8FHmNmm4EX0xi8lYqbGKtxePI6vjr18/zpm1+hBdukMsLuXJ89tzywXn4SAf8x4Eoz22VmW4CbgZm2Y2aAtzV//mVgzpMoD5JSiSrjFCmbqDvXQY1b9b20grufN7PbgVkaZZkfc/cTZvY+YN7dZ4CPAn9nZqeAZ2hcFEQuoO0RpUyi1os6sG83dxw6Fvi+QY1bJbKWjrs/DDzc9twftvz8HPCmJL5LqkELq0nRdVovamKsxt0zJ1hcWr7gvYMat8rVoK0IZFO9IJK0OOtF3X3Dj6c6bqXVMiV3Oi2rLJIHne5C4+wLndRGQ3Ep4EvuaAN1ybs4y3tfOjoSuBhge7omzXErpXQkd7SBuuRdnHRNHveFVg9fcufAvt0bek+Q/S+KSKuo5b33Ts29kJ656eoaj3zpTG6KDxTwJXfSzmuKdCssXWOs7+lQX1zigaP1XM0lUcCX3OimFFNlm5KloLtQgwtWgM1bsYECvuRCN3vcaj9cyVrQXWjYbm15KjbQoK2kKmw52G72uNV+uJIHretAHdi3m7D13vNUbKAevqQmqmfeTSmmyjYlbw7Ongzd0Ofc8+eZXqjn4u5TPXxJTVTPvJtSTJVtSt5EdTaePbecm5niCviSmqieeTc1y3msb5ZqG906HPl6XlKOSulIaqJmHnZTihl1rKp3JG3TC3W+89z5jsflIeWogC+p6TShqpsp5kHHqnpHsnBw9iTLq52398hDylEpHUnNoDc4UfWOZCFOzz0vKUf18CVVvSwUFTdNo+odyUJYqnLIjFX3XKUWFfAl17pJ08RdnVAkSWGpyjwtqbBGKR3JtW7SNKrekSxMjNW46eoaQ9aYejVkxk1X53OrTgV8ybVu0jTaBF2yML1Q54GjdVa8MXC74s4DR+u5qLtvp5SO5Fq3aRptgi5pK9IOberhS64pTSN5V6RiAQV8yTWlaSTvirTUh1I6knthaRrNqpU8KNIObQr4UkiaVSt5UaQd2hTwpZA6lWsW4ZdP0jXIO8KiFAso4EshRW0irZ6/tNMdYYMGbaWQwgbEhsy0no5cQOssNSjgSyGFlWuuTX5pl8cSOUlPkUonB0kBXwoprFyzVqASOUlPkUonB0k5fCmssIGyopTISXqKVDo5SAr4UipFKpGT9OjfRYN5SM4za+Pj4z4/P591M0SkpMo6cc/Mjrr7eNBr6uGLSOVUtUxTg7YiUjlVLdPsK+Cb2UvM7J/N7L+b/7845LgVMzvW/G+mn+8UEelXVcs0+03pTAL/6u5TZjbZfPz7Acctufsr+/wuEZFERO2z8O7p43zqc0+z4s6QGbe85nLeP7Eng1Ymr9+Uzn7gvubP9wETfX6eSOKmF+rsnZpj1+RD7J2ay+VORJKusIl7O186wieOPLVh96pPHHmKd08fz6KZies34P+gu3+j+fP/AD8Yctz3mNm8mR0xs9CLgpnd1jxu/syZM302TWR9cK6+uISzPjinoF9tYRP3jjzxbODxn/rc0+k2cEA6pnTM7F+AHwp46Q9aH7i7m1lYjecV7l43sx8G5szsuLt/pf0gd78XuBcaZZkdWy/SQZG2n5N0BU3cu+PQscBjw5bsKJqOAd/dXxf2mpn9r5ld4u7fMLNLgG+GfEa9+f8nzOzfgDHggoAvkrROg3NlrcWW3gyZBQb3IbMMWpO8fgdtZ4C3AVPN//9j+wHNyp1z7v5dM9sG7AX+pM/vFYklanCuqrXYVRZ0gYf1Gbhbtwzxf8+vXPC+W15zedpNHYi+Ztqa2UuB+4EdwJPAm939GTMbB37T3d9hZj8F/DWwSmPM4M/d/aOdPlszbSUJ7UEdYHjIeNGWzSwuLQe+pzY6wuHJ69JqoqQk8N/CJgOD5ZX1ODi0yVhddRwKWaUzsJm27v5t4LUBz88D72j+/O9Acf60pFTa11AZ3TrMd547Hxrsofy12FUVNJ6zvHphh3dl1Ut70dfSClJ6rYNze6fmePZceLCH6i2ZWxXdXMjLetHX0gpSKZ1+kau4ZG5VdHMhL+tFXwFfKiXqF3mtFlsDtuUUNNlqeJMxPLSxAqfMF32ldKRSwjbCUKBPT1alsGFr4gc9V9Z/Cwr4Uilrv8jv/eyJF3L5F23WjW5asiyFjbrQlDXAt1PAl0p6bnn1hZ8Xl5YvCDqakDUYWc181pyLBnVtpHI6rYWu9XcGJ6tliau6/n07BXypnKCZt7AedBQcBids0HzQVTFVXf++nQK+VMr0Qp2wVVHWgs6ggoOWaQ5flnjQVTFZXWjyRgFfKuXg7EmCFhMxeCHohAUBh54DdVnTRN1exMKWJR50Hj2rC03eaNBWKiWsl+6sD94d2LebA595fMP6Kmt6Hewr4zLNvQ6EBi1L3E8b4gyuh5VkFvXPvlcK+FIpYatn1lp69RNjNe6eORG63k4vgbqMOeSsL2LdXnCSvNAUlVI6Uilxb+3PRiyuBt0H6jLmkJO8iPUyvqHB9e4p4EulxM0hdwrE3QbqMuaQk7qI9Tq+Uca7pkFTSkcqJ86tfdASDGt6CdRlzCGHLVPR7Z9Nr6mhqM1tJJgCvkiA1gBdX1x6Yeu7Wh+Bumw55KQuYr321JO64FSJAr5IiG4DdJWWY2g/1w+85ZU9n2uvPfUy3jUNmgK+SAKqtFZL0ufaT0+9bHdNg6ZBW5EEVKliJOlzjRpI1+zkZKmHL5KAbvPQeUv/dNOeQVTHBPXUq3TXlBb18EUS0E2JYt6WWei2PWnNKajSXVNaFPBFEtBNnX0vgWwQqY21z7zj0LGu2pPWnALV2SdPKR2RBHRTMdJL+ifp1Eb7Z3bTnrSqY1RnnzwFfJEudNomb22g8eDsSe48dIyDsycvCIbdBrK4E5O6ycMHfWbc9rSe6yCpzj55SumIxBQn1x3nmG5TInHuCLrNw3dKi+QhsGa1lHKZqYcvElOcnnacY7pNicS5I+h2eYKwzwT6mk2cNNXZJ0sBXySmOD3tuPn5bgJZnNRGWPAOez7sM9WDLjcFfJGY4vS0BzHQGOeOYG2tn3ZDFryhY5bLEuRtDkKVKOCLxBSnpz2ogcZOdwRBwT7q+TifOQhBFUd3HjrG/JPP8P6JPam2pYo0aCsSU5xBxKwGGmshdxBhz2claKzBgU8eeUrLJqTAPKIHkKXx8XGfn5/PuhkihTC9UOfAPzzO8ur67/PwJuPgm16Rq3TJrsmHAjeRh8bF6fDkdYDSPv0ws6PuPh70mnr4ImXRnq4PTt9nKmosY21gO29LT5SJAr5ICRycPcnyysa+8/KK527dmQP7dodeh9YuBlpDZ3A0aCuSY3FTG0VZd2ZirMb8k8/wySNPbUjttA5sF+VciqivHr6ZvcnMTpjZqpkF5oyax11vZifN7JSZTfbznSJV0U1qI60VLJPw/ok9fOAtrwwd2C7SuRRNvymdzwM3Ao+GHWBmQ8CHgDcAVwG3mNlVfX6vSOl1k9pIawXLpEyM1Tg8eR1fnfp5Dk9et+GupWjnUiR9pXTc/YsAFjK5o+nVwCl3f6J57KeB/cAX+vlukbLrJrVRpv1dy3QueZNGDr8GPN3y+DTwmqADzew24DaAHTt2DL5lIjnW7azdMq07U6ZzyZOOKR0z+xcz+3zAf/uTboy73+vu4+4+vn379qQ/XqRQlNqQpHXs4bv76/r8jjpwecvjy5rPiUgEpTYkaWmkdB4DrjSzXTQC/c3Ar6TwvSKFl7fUhmbAFlu/ZZm/ZGangZ8EHjKz2ebzl5rZwwDufh64HZgFvgjc7+4n+mu2iKRNM2CLT2vpiFRQLz31vVNzgYPIrWvgSPai1tLRTFuRinn39PENM13jboquGbDFp7V0RCpkeqF+wbIGEG+tGs2ALT4FfJEKOTh7MnR54k499aAyUQOufZlKqItCAV+kQqKCeqee+sRYjZuurm1Y7dKBB47WNXBbEAr4IhUSFtQNYk3oeuRLZ3pKB0k+KOCLVEhYWuZXr9kRq55eA7fFpiodkQrpd/Zut+v7SL4o4ItUTD+zdw/s281dDx7fsGyz1vcpDgV8EYlN6/sUmwK+iHQlb+v7SHwatBURqQgFfBGRilDAFxGpCAV8EZGK0KCtSAlpoxIJooAvUjJrG5Ws1crHXf5Yyk8pHZGSOTh7csPEKNB6N9KggC9SMlrvRsIo4IuUjDYqkTAK+CIlE7Qipta7EdCgrUjpaL0bCaOAL1JCWu9GgiilIyJSEQr4IiIVoYAvIlIRCvgiIhWhgC8iUhEK+CIiFWHunnUbApnZGeDJBD5qG/CtBD6nKHS+5abzLbckzvcKd98e9EJuA35SzGze3cezbkdadL7lpvMtt0Gfr1I6IiIVoYAvIlIRVQj492bdgJTpfMtN51tuAz3f0ufwRUSkoQo9fBERQQFfRKQyShHwzex6MztpZqfMbDLg9YvM7FDz9c+Z2c70W5mcGOf7e2b2BTP7LzP7VzO7Iot2JqXT+bYcd5OZuZkVuowvzvma2Zubf8cnzOzv025j0mL8m95hZo+Y2ULz3/Ubs2hnEszsY2b2TTP7fMjrZmZ/0fyz+C8ze1ViX+7uhf4PGAK+AvwwsAV4HLiq7ZjfBj7c/Plm4FDW7R7w+V4LbG3+/FtlP9/mcd8HPAocAcazbveA/36vBBaAi5uPfyDrdqdwzvcCv9X8+Srga1m3u4/z/WngVcDnQ15/I/BPgAHXAJ9L6rvL0MN/NXDK3Z9w9+eBTwP7247ZD9zX/PkzwGvNzFJsY5I6nq+7P+Lu55oPjwCXpdzGJMX5+wX4I+CPgefSbNwAxDnf3wA+5O7PArj7N1NuY9LinLMD39/8+cXA11NsX6Lc/VHgmYhD9gMf94YjwKiZXZLEd5ch4NeAp1sen24+F3iMu58HzgIvTaV1yYtzvq3eTqO3UFQdz7d5y3u5uz+UZsMGJM7f748CP2pmh83siJldn1rrBiPOOd8NvNXMTgMPA7+bTtMy0e3veGza4rDEzOytwDjwM1m3ZVDMbBPwZ8CtGTclTZtppHV+lsbd26NmtsfdFzNt1WDdAvytu/+pmf0k8Hdm9nJ3X826YUVShh5+Hbi85fFlzecCjzGzzTRuCb+dSuuSF+d8MbPXAX8A3ODu302pbYPQ6Xy/D3g58G9m9jUaOc+ZAg/cxvn7PQ3MuPuyu38V+DKNC0BRxTnntwP3A7j7fwDfQ2OhsTKK9TveizIE/MeAK81sl5ltoTEoO9N2zAzwtubPvwzMeXN0pIA6nq+ZjQF/TSPYFz2/G3m+7n7W3be5+05330ljzOIGd5/Pprl9i/PveZpG7x4z20YjxfNEmo1MWJxzfgp4LYCZ/RiNgH8m1VamZwb4tWa1zjXAWXf/RhIfXPiUjrufN7PbgVkao/0fc/cTZvY+YN7dZ4CP0rgFPEVjsOTm7Frcn5jnexD4XuAfmmPTT7n7DZk1ug8xz7c0Yp7vLPB6M/sCsAIccPei3rHGPed3AX9jZnfSGMC9taidNjP7FI0L9rbmmMR7gGEAd/8wjTGKNwKngHPAryf23QX9MxMRkS6VIaUjIiIxKOCLiFSEAr6ISEUo4IuIVIQCvohIRSjgi4hUhAK+iEhF/D9JrQPwI7Kr4wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:29.304489Z",
     "start_time": "2019-09-28T19:10:29.292850Z"
    }
   },
   "outputs": [],
   "source": [
    "X = X.reshape((-1, 1))\n",
    "thresholds = np.arange(0.2, 1.1, 0.2).reshape((1, -1))\n",
    "\n",
    "X_expand = np.hstack((\n",
    "    X,\n",
    "    ((X > thresholds[:, :-1]) & (X <= thresholds[:, 1:])).astype(int)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:29.318935Z",
     "start_time": "2019-09-28T19:10:29.308660Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:29.345543Z",
     "start_time": "2019-09-28T19:10:29.322509Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.20553980048560808"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "-np.mean(cross_val_score(\n",
    "    LinearRegression(), X, y, cv=KFold(n_splits=3, random_state=123),\n",
    "    scoring='neg_mean_squared_error'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:29.378306Z",
     "start_time": "2019-09-28T19:10:29.349811Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.05580385745900118"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "-np.mean(cross_val_score(\n",
    "    LinearRegression(), X_expand, y, cv=KFold(n_splits=3, random_state=123),\n",
    "    scoring='neg_mean_squared_error'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Так линейная модель может лучше восстанавливать нелинейные зависимости."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Транзакционные данные"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Напоследок посмотрим, как можно извлекать признаки из транзакционных данных. \n",
    "\n",
    "Транзакционные данные характеризуются тем, что есть много строк, характеризующихся моментов времени и некоторым числом (суммой денег, например). При этом если это банк, то каждому человеку принадлежит не одна транзакция, а чаще всего надо предсказывать некоторые сущности для клиентов. Таким образом, надо получить признаки для пользователей из множества их транзакций. Этим мы и займёмся.\n",
    "\n",
    "Для примера возьмём данные [отсюда](https://www.kaggle.com/regivm/retailtransactiondata/). Задача детектирования фродовых клиентов."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:34.228625Z",
     "start_time": "2019-09-28T19:10:29.385812Z"
    }
   },
   "outputs": [],
   "source": [
    "!wget https://www.dropbox.com/s/zhgyiugyrzjs5xb/Retail_Data_Response.csv\n",
    "!wget https://www.dropbox.com/s/5xepi2t6d81s9o3/Retail_Data_Transactions.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:34.314276Z",
     "start_time": "2019-09-28T19:10:34.231504Z"
    }
   },
   "outputs": [],
   "source": [
    "customers = pd.read_csv('Retail_Data_Response.csv')\n",
    "transactions = pd.read_csv('Retail_Data_Transactions.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:34.339478Z",
     "start_time": "2019-09-28T19:10:34.316390Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>CS1112</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>CS1113</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>CS1114</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>CS1115</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>CS1116</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  customer_id  response\n",
       "0      CS1112         0\n",
       "1      CS1113         0\n",
       "2      CS1114         1\n",
       "3      CS1115         1\n",
       "4      CS1116         1"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "customers.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:34.393258Z",
     "start_time": "2019-09-28T19:10:34.345560Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>trans_date</th>\n",
       "      <th>tran_amount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>CS5295</td>\n",
       "      <td>11-Feb-13</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>CS4768</td>\n",
       "      <td>15-Mar-15</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>CS2122</td>\n",
       "      <td>26-Feb-13</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>CS1217</td>\n",
       "      <td>16-Nov-11</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>CS1850</td>\n",
       "      <td>20-Nov-13</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  customer_id trans_date  tran_amount\n",
       "0      CS5295  11-Feb-13           35\n",
       "1      CS4768  15-Mar-15           39\n",
       "2      CS2122  26-Feb-13           52\n",
       "3      CS1217  16-Nov-11           99\n",
       "4      CS1850  20-Nov-13           78"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:35.873166Z",
     "start_time": "2019-09-28T19:10:34.443594Z"
    }
   },
   "outputs": [],
   "source": [
    "transactions.trans_date = transactions.trans_date.apply(\n",
    "    lambda x: datetime.datetime.strptime(x, '%d-%b-%y'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Посмотрим на распределение целевой переменной:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:35.891923Z",
     "start_time": "2019-09-28T19:10:35.875688Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.09398605461940732"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "customers.response.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Получаем примерно 1 к 9 положительных примеров. Если такие данные разбивать на части для кросс валидации, то может получиться так, что в одну из частей попадёт слишком мало положительных примеров, а в другую — наоборот. На случай такого неравномерного баланса классов есть StratifiedKFold, который бьёт данные так, чтобы баланс классов во всех частях был одинаковым."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:35.909700Z",
     "start_time": "2019-09-28T19:10:35.898951Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import StratifiedKFold"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Когда строк на каждый объект много, можно считать различные статистики. Например, средние, минимальные и максимальные суммы, потраченные клиентом, количество транзакий, ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:36.003307Z",
     "start_time": "2019-09-28T19:10:35.914780Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>response</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>count</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>CS1112</td>\n",
       "      <td>0</td>\n",
       "      <td>67.466667</td>\n",
       "      <td>19.766012</td>\n",
       "      <td>15</td>\n",
       "      <td>36</td>\n",
       "      <td>105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>CS1113</td>\n",
       "      <td>0</td>\n",
       "      <td>74.500000</td>\n",
       "      <td>21.254102</td>\n",
       "      <td>20</td>\n",
       "      <td>36</td>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>CS1114</td>\n",
       "      <td>1</td>\n",
       "      <td>75.368421</td>\n",
       "      <td>21.341692</td>\n",
       "      <td>19</td>\n",
       "      <td>37</td>\n",
       "      <td>105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>CS1115</td>\n",
       "      <td>1</td>\n",
       "      <td>75.409091</td>\n",
       "      <td>18.151896</td>\n",
       "      <td>22</td>\n",
       "      <td>41</td>\n",
       "      <td>104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>CS1116</td>\n",
       "      <td>1</td>\n",
       "      <td>65.923077</td>\n",
       "      <td>22.940000</td>\n",
       "      <td>13</td>\n",
       "      <td>40</td>\n",
       "      <td>105</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  customer_id  response       mean        std  count  min  max\n",
       "0      CS1112         0  67.466667  19.766012     15   36  105\n",
       "1      CS1113         0  74.500000  21.254102     20   36   98\n",
       "2      CS1114         1  75.368421  21.341692     19   37  105\n",
       "3      CS1115         1  75.409091  18.151896     22   41  104\n",
       "4      CS1116         1  65.923077  22.940000     13   40  105"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agg_transactions = transactions.groupby('customer_id').tran_amount.agg(\n",
    "    ['mean', 'std', 'count', 'min', 'max']).reset_index()\n",
    "\n",
    "data = pd.merge(customers, agg_transactions, how='left', on='customer_id')\n",
    "\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:36.210653Z",
     "start_time": "2019-09-28T19:10:36.008305Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ekayumov/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "/Users/ekayumov/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "/Users/ekayumov/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.594866904556827"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "np.mean(cross_val_score(\n",
    "    LogisticRegression(),\n",
    "    X=data.drop(['customer_id', 'response'], axis=1),\n",
    "    y=data.response,\n",
    "    cv=StratifiedKFold(n_splits=3, random_state=123),\n",
    "    scoring='roc_auc'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Но каждая транзакция снабжена датой! Можно посчитать статистики только по свежим транзакциям. Добавим их."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:36.235061Z",
     "start_time": "2019-09-28T19:10:36.218040Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(Timestamp('2011-05-16 00:00:00'), Timestamp('2015-03-16 00:00:00'))"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions.trans_date.min(), transactions.trans_date.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:36.858305Z",
     "start_time": "2019-09-28T19:10:36.240648Z"
    }
   },
   "outputs": [],
   "source": [
    "agg_transactions = transactions.loc[transactions.trans_date.apply(\n",
    "    lambda x: x.year == 2014)].groupby('customer_id').tran_amount.agg(\n",
    "    ['mean', 'std', 'count', 'min', 'max']).reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:36.886707Z",
     "start_time": "2019-09-28T19:10:36.860734Z"
    }
   },
   "outputs": [],
   "source": [
    "data = pd.merge(data, agg_transactions, how='left', on='customer_id', suffixes=('', '_2014'))\n",
    "data = data.fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-09-28T19:10:37.020392Z",
     "start_time": "2019-09-28T19:10:36.889377Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ekayumov/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "/Users/ekayumov/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "/Users/ekayumov/anaconda/envs/py36/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.6483871707242365"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(cross_val_score(\n",
    "    LogisticRegression(),\n",
    "    X=data.drop(['customer_id', 'response'], axis=1),\n",
    "    y=data.response,\n",
    "    cv=StratifiedKFold(n_splits=3, random_state=123),\n",
    "    scoring='roc_auc'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Можно также считать дату первой и последней транзакциями пользователей, среднее время между транзакциями и прочее."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py36",
   "language": "python",
   "name": "py36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "position": {
    "height": "144px",
    "left": "792px",
    "right": "20px",
    "top": "166px",
    "width": "350px"
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}