{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Regression\n", "> A Summary of lecture \"Supervised Learning with scikit-learn\", via datacamp\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- author: Chanseok Kang\n", "- categories: [Python, Datacamp, Machine_Learning]\n", "- image: images/ridge_cv.png" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction to Regression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Importing data for supervised learning\n", "In this chapter, you will work with Gapminder data that we have consolidated into one CSV file available in the workspace as ```'gapminder.csv'```. Specifically, your goal will be to use this data to predict the life expectancy in a given country based on features such as the country's GDP, fertility rate, and population. As in Chapter 1, the dataset has been preprocessed.\n", "\n", "Since the target variable here is quantitative, this is a regression problem. To begin, you will fit a linear regression with just one feature: ```'fertility'```, which is the average number of children a woman in a given country gives birth to. In later exercises, you will use all the features to build regression models.\n", "\n", "Before that, however, you need to import the data and get it into the form needed by scikit-learn. This involves creating feature and target variable arrays. Furthermore, since you are going to use only one feature to begin with, you need to do some reshaping using NumPy's ```.reshape()``` method. Don't worry too much about this reshaping right now, but it is something you will have to do occasionally when working with scikit-learn so it is useful to practice." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Dimensions of y before reshaping: (139,)\n", "Dimensions of X before reshaping: (139,)\n", "Dimensions of y after reshaping: (139, 1)\n", "Dimensions of X after reshaping: (139, 1)\n" ] } ], "source": [ "# Read the CSV file into a DataFrame: df\n", "df = pd.read_csv('./dataset/gm_2008_region.csv')\n", "df.drop(labels=['Region'], axis='columns', inplace=True)\n", "\n", "# Create arrays for features and target variable\n", "y = df['life'].values\n", "X = df['fertility'].values\n", "\n", "# Print the dimensions of X and y before reshaping\n", "print(\"Dimensions of y before reshaping: {}\".format(y.shape))\n", "print(\"Dimensions of X before reshaping: {}\".format(X.shape))\n", "\n", "# Reshape X and y\n", "y = y.reshape(-1, 1)\n", "X = X.reshape(-1, 1)\n", "\n", "# Print the dimensions of X and y after reshaping\n", "print(\"Dimensions of y after reshaping: {}\".format(y.shape))\n", "print(\"Dimensions of X after reshaping: {}\".format(X.shape))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exploring the Gapminder data\n", "As always, it is important to explore your data before building models. On the right, we have constructed a heatmap showing the correlation between the different features of the Gapminder dataset. Cells that are in green show positive correlation, while cells that are in red show negative correlation. Take a moment to explore this: Which features are positively correlated with life, and which ones are negatively correlated? Does this match your intuition?" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAE9CAYAAAAF/alEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3debgcVZ3/8ffnhlU22Q1rAIOAGIKERUBlFx0EVHaQwIAZGBgdHB3BBQMMP3HFBRwMCIRFdpSoyGIg4ohgEgiBEDAQUGIYkGUEBEOW7++POg2Vpu+93berb3fX/byep57bVXXq1Omb3G+fPnUWRQRmZlZePe0ugJmZtZYDvZlZyTnQm5mVnAO9mVnJOdCbmZWcA72ZWck50JuZFUzSxZKelfRQL+cl6fuSHpM0U9J7c+fGSpqTtrFFlMeB3syseJcC+/Zx/sPAyLSNA/4bQNIawFeBHYEdgK9KWr3ZwjjQm5kVLCLuAl7oI8kBwGWRuQd4u6ThwIeA2yPihYh4Ebidvj8w6uJAb2Y2+NYHnsrtz0vHejvelGWazcD6phN3aukcE0vOGtfK7CGWtDR7rdj0t9J+xTOPtfYGqw9vbf4vPt3S7LXuO1uaP8MGIcyseICazaKhv9UL7v0XsiaXigkRMaGR29U4Fn0cb4oDvZkZoJ76PyuWZEG9kcBebR6wYW5/A2B+Or5b1fEpTdwHcNONmRmQBfp6twJMAo5OvW92Av4WEU8DtwL7SFo9PYTdJx1rimv0ZmY0VqPvNy/pKrKa+VqS5pH1pFkWICIuAG4GPgI8BrwKHJvOvSDpLGBqyurMiOjroW5dHOjNzCg20EfE4f2cD+CkXs5dDFxcWGFwoDczA0AqLtB3Ggd6MzOKrdF3Ggd6MzOgZ5ny9k0p5TuTNKK3OSaq0hyR2x8j6futL52ZdaJB7nUzqIZyjX4EcATwE4CImAZMa2eBzKx9ujGA16stNfpUm35E0sQ0c9v1kt4maU9J90t6MM3+tnxK/6Skr0v6Q9remY5fKumgXL6v9HKv30q6L207p1PnAO+XNEPSKZJ2k/SLdM0akn6WynaPpFHp+PhUrimS5kr6dKt/V2Y2OMpco29n0827yIYNjwJeAj5LNuPboRHxHrJvGyfm0r8UETsA5wHfbeA+zwJ7R8R7gUOBSvPMqcBvI2J0RJxbdc0ZwP2pbF8ELsud24Js4qHKzHLLNlAWM+tQDvSt8VRE/C69vgLYE3giIv6Yjk0EPpBLf1Xu5/sauM+ywIWSHgSuA7aq45pdgcsBIuIOYE1Jq6Vzv4yIBRHxHNmHyLrVF0saJ2mapGk8/GwDRTWzdukZ1lP31m3aWeJGJ+qJGq8Xkd6Dsk6wy9W47hTgGWAbYEwvaar1NbHQgtyxxdR4zhEREyJiTESMYat16ridmbWba/StsZGkSs38cODXwIhK+zvwSeA3ufSH5n7+Pr1+EtguvT6ANMS4ymrA0xGxJOU5LB1/GVill7LdBRwJIGk34LmIeKmud2VmXanMgb6dvW5mA2Ml/QiYA3wGuAe4TtIyZHM9XJBLv7yke8k+nCrDiy8EbpL0B2Ay8Pca9/khcIOkg4E7c2lmAoskPUD2bOD+3DXjgUskzSSbh6KQ5bzMrHN1YwCvVzsD/ZKIOKHq2GRg217Snx8RZ+QPRMQzwE65Q6el408CW6fXc4BRNdIsJHsukDclnXuB7BvCUiJifNX+1r2U1cy6jAO9mVnJOdAXLF/jrjP9iJYVxsyMck+B4Bq9mRmevdLMrPTcdGNmVnIO9GZmJedAb2ZWcj3lfRbrQN9qS84a19L8e74yoaX5Tz9zv5bmv9V3bmpp/gCzr5rT0vxH7rl+S/M/75iRLc1/zPsKXZ70LT5+0kYtzR/gpa++ZdhLw5brwjls6lXed2Zm1oBhUt1bfyTtK+lRSY9JOrXG+XPTFOkzJP1R0v/lzi3OnZtUxHtzjd7MDCiqQi9pGHA+sDcwD5gqaVJEPFxJExGn5NL/G0vPCPBaRIwupjQZ1+jNzCi0Rr8D8FhEzI2I14GrqTGlSs7hvDkNe0s40JuZUWigXx94Krc/Lx17C0kbA5sAd+QOr5DWs7hH0oHNvKcKN92YmQHLDau/e6WkcUC+p8WEiKj0jOhrPYtqhwHXR8Ti3LGNImK+pE2BOyQ9GBGP1124GhzozcyAYQ30o09Bvbcub/OADXP7GwDze0l7GHBSVd7z08+5kqaQtd83FejddGNmRqFNN1OBkZI2kbQcWTB/S+8ZSe8CVufNhZSQtLqk5dPrtYBdgIerr21UVwd6SZ+WNFvSlXWmHyHpiNz+GEnfT6+PkXReen2CpKNzx9drRfnNrHMM61HdW18iYhFwMnAr2QJL10bELElnSto/l/Rw4OqIyDfrbAlMSwsi3Qmck++tM1Dd3nTzr8CHI+KJ/hKmVatGAEcAPwGIiGnAtOq0EZFf2eoY4CF6/+plZiXQQBN9vyLiZuDmqmOnV+2Pr3Hd3cB7iitJpmsDvaQLgE2BSZKuBjYj+wUtA4yPiJskHQP8E7ACsBLwNmBLSTOAiWTLB34uIvaryns88ArZmrRjgCslvQZ8CTg+Ij6W0u0NnBgRH2/tuzWzVmukjb7bdG3TTVqGcD6wO1kQvyMitk/735S0Ukr6PmBsROwBnAr8NiJGR8S5ddzjerIa/5FpAMPNZB8Ua6ckxwKXFPm+zKw9luvpqXvrNt1X4tr2AU5NNfUpZDX4ygQbt6c1YJuW2tIuB46S9HayD5FfVaeTNC71g5024bLfFHFrM2uxYT31b92ma5tuqgj4REQ8utRBaUfg7wXf6xLg58A/gOvSg5el5LtexXMX99Z/1sw6SD1z2HSrLvxsqulW4N+U1gKTtG0v6V4GVmkw76WuSX1c5wNfBi5tuKRm1pGK6nXTicoS6M8ClgVmSnoo7dcyE1gk6QFJp/SSptqlwAVpJrkV07ErgaeK6PZkZp2hyNkrO01XN91ExIjc7r/UOH8puVp3RCwE9qxKNqU6bb7bU0TcANxQdc2uwIUDKrSZdaRubHuvV1cH+naQNJ2s3f8/2l0WMytOmRcecaBvUERs1+4ymFnxurFJpl4O9GZmuOnGzKz0XKM3Myu5Iue66TQO9GZmwLJd2D++Xg70ZmZAieO8A33LxZKWZj/9zP36T9SE7U7/RUvzX3LyR1qaP8A2o9Zpaf4avnb/iZqw1ZqvtzT/LcY0Oli8MVccXfisuy3hphszs5LrKXGV3oHezAzX6M3MSq/EFXoHejMzcK8bM7PSK3PTTYkH/ZqZ1a9H9W/9kbSvpEclPSbp1Brnj5H01zT9+QxJx+fOjZU0J21ji3hvrtGbmVHcFAiShgHnA3sD84CpkibVWL/imog4ueraNYCvAmOAAKana19spkyu0ZuZUWiNfgfgsYiYGxGvA1cDB9RZjA+R1rlOwf12YN+BvqcKB3pA0itV+8dIOi+9Hi/pc+nYVVXp1kpfv5YfzPKaWfGGqf6tH+sDT+X256Vj1T4haaak6yVt2OC1DXGgr9+NwN6S3pY7dhAwKSIWtKlMZlaQZYep7k3SOEnTctu4XFa1Pgqiav/nwIiIGAX8GpjYwLUNc6CvU0S8BNwFfDR3+DDgqtpXmFk3aaTpJiImRMSY3DYhl9U8YMPc/gbA/Py9IuL5XAXxQmC7eq8d0HtrNoOSWDH39HsGcGYv6a4iC+5IWg/YHLhzkMpoZi1UYNPNVGCkpE0kLUcWMyblE0gantvdH5idXt8K7CNpdUmrA/ukY01xoM+8FhGjKxtwei/pfgHsKmlV4BDg+ohYXJ0o/7VuwmV3tbDYZlaUoh7GRsQi4GSyAD0buDYiZkk6U9L+KdmnJc2S9ADwaeCYdO0LwFlkHxZTgTPTsaa4e2UDIuI1SbcAHyP7lD6ll3QTgAkA8deLmm5fM7PWK3KFqYi4Gbi56tjpudenAaf1cu3FwMWFFQYH+oG4CvgasCpwT5vLYmYFWbbE7RslfmstcxuwHtlgB9fWzUqiR6p76zau0QMRsXLV/qXApen1+Kpzi4DWrjRhZoOuzHPdONCbmUFX1tTr5UBvZoYDvZlZ6TnQm5mV3DI9w9pdhJZxoDczwzV6M7PSc6C3AdOKq7c0/62+c1NL819y8kdamn/PeTf3n6hJnz9qVEvzv3z6Iy3N/y+bDu8/URMu/Mb2Lc3/6Ita+/sB4Ozms+gp8bAiB3ozM1yjNzMrPQd6M7OSc68bM7OSc43ezKzkHOjNzEquR+51Y2ZWaj011+UuBwd6MzPK/TC2vN9V+iHpHZKulvS4pIcl3Sxpc0nvlnSHpD9KmiPpK1LWeCfpSEkz03a3pG3a/T7MrBhlXnhkSAb6FLh/CkyJiM0iYivgi8C6ZKu1nxMRmwPbADsD/5oufQL4YESMIlvAd8KgF97MWsKBvnx2BxZGxAWVAxExA9gc+F1E3JaOvUq2mvupaf/uiHgxXXIPsMGgltrMWqZHPXVv/ZG0r6RHJT0m6dQa5z+bWhJmSposaePcucWSZqRtUhHvbai20W8NTK9x/N3VxyPicUkrS1o1Il7KnToO+FULy2hmg6iomrqkYcD5wN7APGCqpEkR8XAu2f3AmIh4VdKJwDeAQ9O51yJidCGFSYZqjb43Anpb8PuN45J2Jwv0X6iZiTRO0jRJ0yZcfHvxpTSzwvWgurd+7AA8FhFzI+J14GrggHyCiLgztRjAILQODNUa/SzgoF6OfyB/QNKmwCsR8XLaHwVcBHw4Ip6vlXlETKDSfv/KDb19cJhZBymw1836wFO5/XnAjn2kr24dWEHSNGAR2fPCnzVboKFao78DWF7SpyoHJG0PzAF2lbRXOrYi8H2yr1VI2gi4EfhkRPxx0EttZi3TyMPY/Lf2tI3LZVWryl+zwifpKGAM8M3c4Y0iYgxwBPBdSZs1+96GZI0+IkLSx8h+iacC/wCeBP6d7CvWDySdDwwDLgfOS5eeDqwJ/DD1uFyU/kHMrMupgZGxS31rf6t5wIa5/Q2A+W+9n/YCvkTWk29BLu/56edcSVOAbYHH6y5cDUMy0MMbv8xDejm9Wy/XHA8c36oymVn7FLjwyFRgpKRNgL8Ah5HVzt8gaVvgR8C+EfFs7vjqwKsRsUDSWsAupBaFZgzZQG9mltdIjb4vEbFI0snArWStAhdHxCxJZwLTImISWVPNysB1qXXgzxGxP7Al8CNJS8ia1s+p6q0zIA70ZmbAMBUXDiPiZuDmqmOn517v1ct1dwPvKawgiQO9mRmevdLMrPRU4k6IDvRmZrhGb2ZWekU9jO1EDvQtFs881tL8Z181p6X5bzNqnZbm//mjRrU0f4BvXjGzpfmPO3jrlua/4K6n+k/UhIlbrNLS/Le+Ym5L8wfY5ezm8yiwe2XHcaA3MwOG9ZQ3HJb3nZmZNUCUd4UpB3ozM/ww1sys9Ny90sys5FyjNzMrOXevNDMruSLnuuk05X1nZmYNcNONmVnJlflhbMe8M0mLJc2Q9ICk+yTtnI6PkBSSzsqlXUvSQknnpf3xkj7XwrI9mRYBMLOS6lFP3Vu36aQSvxYRoyNiG+A04Gu5c3OB/XL7B5Mt5G1mVgjRU/fWbTq1xKsCL+b2XwNmS6qsz3oocG29mUmaIulcSXdJmi1pe0k3Spoj6b9y6X4mabqkWVWL/ebzOkrSH9K3jx9JKu9wOrMhZFjPMnVv3aaTAv2KKXg+AlwEnFV1/mrgMEkbAIupsdhuP16PiA8AFwA3AScBWwPHSFozpfnniNiObFX2T+eOAyBpS7IPmV0iYnQqx5HVN8qvED/h6nsbLKaZtYOi/q3bdNJH02speCLpfcBlkvLTAt5CFvyfAa4ZQP6T0s8HgVkR8XS611yyFdufJwvuH0vpNgRGpuMVewLbAVPTOo8rAs9SJb9CfDz+9S78b2E2BMWS+tOqdcVohU4K9G+IiN+nh59r5469Lmk68B/Au4GPNpjtgvRzSe51ZX8ZSbsBewHvi4hXJU0BVqjKQ8DEiDitwXubWadrJNB3mU5qunmDpC3IVk9/vurUt4EvRET18SKsBryYgvwWwE410kwGDpK0TirnGpI2bkFZzGywxZL6t35I2lfSo5Iek3RqjfPLS7omnb9X0ojcudPS8UclfaiIt9ZJNfoVJc1IrwWMjYjFqYkEgIiYRet629wCnCBpJvAocE91goh4WNKXgduUjZdeSNbW/6cWlcnMBktBNfrUQeN8YG9gHllT76SIeDiX7DiyiuU7JR0GfB04VNJWwGFkrRbrAb+WtHlELG6mTB0T6COiZu+ViHiS7KFp9fFLgUvT6/H95L1b7vUUYEqtc8CHe7l+RO71NQzsGYGZdbLFi+pPu2yfZ3cAHouIuQCSrgYOAPKB/gBgfHp9PXCeslrtAcDVEbEAeELSYym/39dfuLfqyKYbM7NBV1zTzfpAfv3HeelYzTQRsQj4G7Bmndc2rGNq9EWQdD6wS9Xh70XEJe0oj5l1kQaabtI4m/xYmwmptx3U7pNT3fuutzT1XNuwUgX6iDip3WUwsy61pP5An+9CXcM8su7ZFRvw1nE/lTTzJC1D1hnkhTqvbZibbszMoMimm6nASEmbSFqO7OHqpKo0k4Cx6fVBwB0REen4YalXziZkY3n+0OxbK1WN3sxswArqdRMRiySdDNxK1k384oiYJelMYFpETAJ+DFyeHra+QPZhQEp3LdmD20XASc32uAEH+tZbfXhLsx+5Z9PPafqk4Wv3n6gJl09/pKX5A4w7+C2dtgo14bqHWpr/DzZv7b/xVmu/raX5b7l7a/8PFSWWLKw7bX8DYyPiZuDmqmOn517/g2xyxlrXng2cXXdh6uBAb2YGDbXRdxsHejMzKPUUCA70ZmbgQG9mVnoO9GZmJdfIFAhdxoHezAxcozczKz0HejOzkitxoB+SUyBIWlfSTyTNTYuB/17SxyTtJulvku5Pk/7fJWm/3HXjJf0lrW37kKT92/k+zKxAS5bUv3WZIVejT3M+/4xsScAj0rGNgf2BF4HfRsR+6fho4GeSXouIySmLcyPiW2mh8N9KWieixFUBs6GixH/GQ7FGvwfwekRcUDkQEX+KiB9UJ4yIGcCZwMk1zs0mm4tirRaW1cwGy+JF9W9dZigG+ncD9zWQ/j5gi+qDknYkW1j8rwWVy8zaaUnUv3WZoRjolyLpfEkPSJraW5Kq/VPS2rbfAg5NU4tW5zlO0jRJ0yZMvLPoIptZK7iNvlRmAZ+o7ETESZLWAqb1kn5bYHZu/9yI+FZfN8gvShAvXNZ9H/9mQ1EXBvB6DcUa/R3ACpJOzB2rOU+rpFHAV8hWdDezMitx082Qq9FHREg6EDhX0n+StbH/HfhCSvJ+SfeTBf9ngU/netyYWVmVuEY/5AI9QEQ8TVrRpYbV+rhufEsKZGbtt6jphZw61pAM9GZmb+EavZlZyXVh23u9HOjNzKDUNfqh2OvGzOytBqkfvaQ1JN0uaU76uXqNNKPTHFyzJM2UdGju3KWSnkhzbs1IU7X0yYHezAyIxYvr3pp0KjA5IkYCk9N+tVeBoyPi3cC+wHclvT13/vMRMTptM/q7oQO9mRkM5sjYA4CJ6fVE4MDqBBHxx4iYk17PJ+vqvfZAb+hAb2YGgzlgat3UxbvS1XudvhJL2gFYDng8d/js1KRzrqTl+7uhH8a22otPtzT7844Z2dL8t1rz9Zbm/5dNh7c0f4AFdz3V0vx/sPn6Lc1/+T/+paX5z7n20Zbmf863dm5p/gDfKCKTBmrqksYB43KHJqSpTyrnfw28o8alX2qkSJKGA5cDY3PToZ8G/C9Z8J9ANtjzzL7ycaA3M4OGAn1+Pqtezu/V2zlJz0gaHhFPp0D+bC/pVgV+CXw5Iu7J5V2pPS6QdAnwuf7K66YbMzMYzKabScDY9HoscFN1AknLAT8FLouI66rODU8/Rda+/1B/N3SN3swMYNGgLShyDnCtpOOAPwMHA0gaA5wQEccDhwAfANaUdEy67pjUw+ZKSWuTTaE+Azihvxs60JuZwaCNjI2I54E9axyfBhyfXl8BXNHL9Xs0ek8HejMzKPXIWAd6MzNwoDczKz1PamZmVm6xsLzz0Q9K90pJi9PkOw9Iuk/Szun4CEkh6axc2rUkLZR0XtofL6nXfqKStkh53y9psxaVf4SkfrswmVkXK/FSgoPVj/61NPnONmSjur6WOzcX2C+3fzDZAt71OhC4KSK2jYjH+01tZlbL4qh/6zLtGDC1KvBibv81YHbqQwpwKHBtPRlJ+gjw78Dxku5Mx46S9IdUy/+RpGHp+CuSvi5puqRfS9pB0hRJcyXtn9KMkPTb9K3jjW8eVfccJumbkqamuSb+ZcC/CTPrGLEk6t66zWAF+hVT4H0EuAg4q+r81cBhkjYAFgPz68k0Im4GLgDOjYjdJW1J9kGxS0SMTnkdmZKvBEyJiO2Al4H/AvYGPsab80Q8C+wdEe9N+Xy/xm2PA/4WEdsD2wOfkrRJPoGkcZKmSZo24ep763krZtZuJa7RD9bD2NdS4EXS+4DLJG2dO38LWfB/BrimifvsCWwHTM1GB7Mib84j8Xq6D8CDwIKIWCjpQWBEOr4scF6ayH8xsHmNe+wDjJJ0UNpfDRgJPFFJkJ8HIx7/evf9rzAbiha7e2VhIuL3ktYiN7dyRLwuaTrwH8C7gY8OMHsBEyPitBrnFkZEJeguARakey+RVPk9nEL2YbMN2bedf/Ryj3+LiFsHWEYz60CxsLyBftDb6CVtAQwDnq869W3gC2l48EBNBg6StE661xqSNm7g+tWAp9N0oJ9M5ax2K3CipGXTPTaXtFITZTazTlDiXjeDVaNfUVJluSuRza28ODWvABARs2ist81bRMTDkr4M3CapB1gInAT8qc4sfgjcIOlg4E7g7zXSXETW1HNfmj3ur9RYIcbMukt0Ydt7vQYl0EdErZoxEfEksHWN45cCl6bX4/vJe3zV/jXUaOePiJX7uGbl9HMOMCp36rTqcqba/hfTZmZl4SkQzMxKzjX69pN0PrBL1eHvRcQl7SiPmZVLN/aPr1fXBPqIOKndZTCzEnvdTTdmZqXmGr2ZWdl5wJSZWbm5Rm8DpnXf2dL8x7zv4pbmv8WYVVqa/4Xf2L6l+QNM3KK172Grtd/W0vznXPtoS/MfOXrl/hM14cpdf9PS/AEoIkaXuNdNO2avNDPrOLFwSd1bM9KI/dslzUk/V+8lXWUdjxmSJuWObyLp3nT9NZKW6++eDvRmZjCYUyCcCkyOiJFk07ac2ku6yjoeoyNi/9zxr5PN2DuSbMr34/q7oQO9mRlkD2Pr3ZpzADAxvZ5IA1OopGlX9gCub+R6B3ozMwZ14ZF1I+JpgPRznV7SrZDWtbhHUiWYrwn8X0QsSvvzgPX7u6EfxpqZQUMPYyWNA8blDk1I61BUzv8aeEeNS7/UQIk2ioj5kjYF7khrZ7xUI12/BXegNzOjse6V+cWFejm/V2/nJD0jaXhEPC1pOG8ujlSdx/z0c66kKcC2wA3A2yUtk2r1G1DHinxuujEzA5YsXFL31qRJwNj0eixwU3UCSatLWj69Xotsnq+H0+JJdwIH9XV9NQf6HEmvpJ/rSbo+d/yqtBD4Ke0rnZm10pLFUffWpHOAvSXNIVu3+hwASWMkXZTSbAlMk/QAWWA/JyIeTue+AHxW0mNkbfY/7u+GbrqpIX1lOghA0juAnSOikZWqzKzLDNbI2LSK3p41jk8Djk+v7wbe08v1c4EdGrmna/Q1SBoh6aG0exuwThq08H5Jm0m6RdJ0Sb9NSyOaWZeLJUvq3rqNa/T92x/4RUSMBpA0GTghIuZI2pFs+cE92llAM2temZcSdI2+AZJWBnYGrktr4P4IGF4j3bjU/3XahItvH+ximtkADGI/+kHnGn1jesgGK4zuK9FSXa9euaH7/leYDUEF9KbpWK7RNyAiXgKekHQwZMORJW3T5mKZWQGWLIm6t27jQN+4I4HjUrenWWTzVphZl4vFUffWbdx0kxMRK6efTwJbV79O+08A+7aheGbWQt3Y9l4vB3ozMxzozcxKb8nCxe0uQss40JuZUe5+9A70Zma46cbMrPS6sdtkvRzozcxw042ZWem56cYGblhrf8UfP2mjluZ/xdE1Z0otzNEXPdLS/AG2vmJuS/Pfcve1W5r/Od/auaX5X7nrb1qa/5EnrN7S/AGOKCCPMk+B4EBvZoZr9GZmpec2ejOzknOvGzOzkuvChaPq5kBvZka5A72nKTYzAxYtrn9rhqQ1JN0uaU76+ZZuSZJ2T+tUV7Z/SDownbtU0hO5c30uhAQO9GZmQFajr3dr0qnA5IgYCUxO+0uJiDsjYnRazW4P4FXgtlySz1fOR8SM/m7oQG9mxqAG+gOAien1RODAftIfBPwqIl4d6A0HFOjTV4eDahxfT9L16fVukn7Ry/VPSlprIPdulKTRkj5SR7o3yitpf0mnptcHStqq1eU0s/YaxEC/bkQ8DZB+rtNP+sOAq6qOnS1ppqRzJS3f3w0LrdFHxPyIeMsHQLtIWgYYDfQb6PMiYlJEnJN2DwQc6M1KrpFAL2mcpGm5bVw+L0m/lvRQja2hpUclDQfeA9yaO3wasAWwPbAG8IX+8qmr142ko4HPAQHMBBYDH5D0WeAdwH9GxPWSRgC/iIitq65fk+wTaW3gD4D6uNcI4Bbgf4CdgAeAS4AzyD75joyIP0haA7gY2JSs/WpcRMyUNB5YDxgBPAfsCqwoaVfga8ATwHeBFYHXgGMj4tGqMhwDjAF+AuwPfFDSl4FPANdFxHtTupHA1RGxXT2/RzPrXIsW1Z82IiYAE/o4v1dv5yQ9I2l4RDydAvmzfdzqEOCnEbEwl/fT6eUCSZeQxeY+9Vujl/Ru4EvAHhGxDfCZdGo4WRDdDzinl8srvgr8T0RsC0wC+pug5Z3A94BRZJ9cR6R7fQ74YkpzBnB/RIxKxy7LXb8dcEBEHAGcDlyTHlpcAzwCfCCV5XTg//VWiIi4O5W38uDjceBvuafcxwKXVl+X/7Sf8ONbq0+bWQcaxKabScDY9HoscFMfaQ+nqtkmfTggSWQtDg/1d8N6avR7AOWxC9QAABM6SURBVNdHxHMAEfFClj8/i4glwMOS1u0njw8AH0/X/1LSi/2kfyIiHgSQNIvsCXVIepCspg5Z4P9EyvMOSWtKWi2dmxQRr/WS92rAxFQbD2DZfspS7SLg2PRt5lBgh+oES33av3ZTeYfbmZXIIPajPwe4VtJxwJ+BgwEkjQFOiIjj0/4IYEOgeta5KyWtTdYyMgM4ob8b1hPoRRYQqy2oStOfRgJePu8luf0lvFnmWves3OPvfeR9FnBnRHws/SKnNFAugBvIvqHcAUyPiOcbvN7MOlDE4NTJUszYs8bxacDxuf0ngfVrpNuj0XvW8zB2MnBIamcntY036i7gyHT9h4Ei5i3N57kb8FxEvFQj3cvAKrn91YC/pNfH1HGfpa6PiH+QPRj5b7JnB2ZWAoPYdDPo+g30ETELOBv4jaQHgO8M4D5nkD28vQ/Yh+zrSrPGA2MkzST7KjS2l3R3AlulEWSHAt8Avibpd8CwOu5zNfB5SfdL2iwdu5Ls28NtvV9mZt2kzIG+rl43ETGRNzv41zq/cvr5JLB1ej2F1CySvqrsk7vklD7yeiOPtH9MrXMR8QLZwIPq68dX7b9A1g0pb/Pc66/UKO+lpIesEfE73tq9clfg4ohocjC0mXWKRnrddBtPatYgST8FNiN7SG1mJdGNNfV6tS3Qpzb/yTVO7dnJDzgj4mPtLoOZFc+BvgVSMO931jUzs8HgQG9mVnIlXmDKgd7MDPww1sys9Nx0Y2ZWcmUO9BqsYb9WP0nj0nw5zt/5d1z+g3GPwXgPQ4lXmOpM4/pP4vydf9vyH4x7DMZ7GDIc6M3MSs6B3sys5BzoO1Or2yadv/Pv9Hu4fb5AfhhrZlZyrtGbmZWcA72ZWck50FtTJDW65u6QJGmtdpfBhi4H+g4haZik9SRtVNkKzPsGSf8kqRX/3n+RdKGkPdKq9F1LmaMknZ72N5L0lsXfG8zzo5L+CjwoaZ6knQspbO/321jSXun1ipJW6e+aBvLeXNJkSQ+l/VGSvlxg/tMknSSpiKVGLceBvgNI+jfgGeB24Jdp+0WBt/hv4AhgjqRzJG1RYN5bAtPIVup6StJ3Je1YVOaS3iPpHklPSZqQDwKS/lDUfZIfAu8DDk/7LwPnN5nn2cD7I2I48Anga03m1ytJnwKuB36UDm0A/KzAW1wInAYsBIiImcBhBeZ/GLAeMFXS1ZI+1O2Vh44REd7avAGPAWsOwn1WA04AngLuBo4Fli0w//WAzwC/Bx4Hzi4gz/8B9gXeDnwOmAVsls7dX/Dv577qfIEHisizt/2Cyz8DWK6q/A8WmP/UGr+fGS14Hz3A/sBf0v/VM4A1WvV7GwqbJzXrDE8Bf2vlDdKKXkcBnwTuJ1vgfFeyRdV3K+IeETFf0o+BF4HPAscDX2oy25Uj4pb0+luSpgO3SPok2QLtRVooaVglX0lrA81OdbWOpM/2th8R32ky/7wFEfF6pRIsaRmK/R09J2kz3vz9HAQ8XWD+SBpFVgH5CHADb/4/vQMvVDRgDvSdYS4wRdIvgQWVg0UFAUk3AlsAlwMfjYjKH+c1kqYVkP8KwEfJmjx2AW4h+4p/W7N5Z9lrtYj4G0BE3CnpE2RBYI0C8s/7PvBTsmB8NnAQ0Gwb9IXAKn3sF+k3kr4IrChpb+BfgZ8XmP9JZAOZtpD0F+AJ4MiiMk8f4v8H/Bg4NSIqfwv3StqlqPsMRR4w1QEkfbXW8Yg4o6D8PxIRN1cdWz73h9RM3j8B9gLuAq4GfhER/2g231z+RwBzI+KequMbAV+JiE8Vda+U7xbAnoCAyRExu8j8Wyk9bD8O2Ies/LcCF0WTf+SSPhMR35O0S0T8TtJKQE9EvNx8qZe6z6YRMbfq2CYR8USR9xmKHOg7SOohERHxSsH53hcR7+3v2ADzHgvcWPQffS/3Wpns9/P3gvPt85tBRLzQZP67AyeTfasCmA2cFxFTmsl3sEiaERGji/o/08d9av0/nR4R27XqnkOFm246gKStyZpV1kj7zwFHR8SsJvN9B7A+2Vf5bclqeQCrAm9rJu+ctYBP9dY5oojmJ0knkjUFrZTt6mXg6xHxw2bzTqaTtTvn30RlP4BNB5qxpH8CzgPOTJuA9wIXSzq5+pvWAO/xIH20xUfEqCZvMVvSk8Dakmbmb51l31z+6VvUu4HVJH08d2pVYIVm8raMA31nmAB8NiLuBJC0G1lbbrN9rj8EHEPWzS4fcF8Gvthk3hUrF5RPTamf9s7AbpWv9ZI2Bb4naY2I+K9m7xERmzSbRx8+DxwYEQ/kjs1Iz0Z+ADQd6IH9CsijVxFxeKo03ErWG6Zo7yJ7D28ne9ZT8TJQaNPcUOWmmw4g6YGI2Ka/Y03k/4mIuKGIvAabpEeBbarb/SWtSNb1cfOC77c6MJJcTTIi7moiv0cioua4hb7ODUWS3hcRv293OcrINfrOMFfSV8iabyDrBtn0AyhJR0XEFcCIqi5+QGHNKt/v63xEfLrZe9R6uBsRr0kqdJVPSceTjQPYgKxP+k5kYwL2aCLbvp4nFP2sYSeybwlbkvWnHwb8PSJWbTLfayPikBpNREU13fxnRHwDOELS4dXni/g/NNQ50HeGfyYbFHIj2R/PXWR9iZu1UvrZyuaV6bnXZwA1exA1YZ6kPSNicv6gpD0puA83WZDfHrgnInZPbcfN9nzaTNKkGsdFE23/vTiPbHTpdcAY4GjgnQXk+5n0s1VNRJWeTU139bXa3HRjhZF0f0RsW3Ce7wZuIhshW3louj1Zf/0Dmn1gXXWvqRGxvaQZwI4RsaDS46SJPD9Y43Dlj04R8ZuB5l3jXtMiYoykmZVatqS7I6Kl8+tY53ONvo0kfTci/l3Sz6nRayIimnrwNRjNKtVZFpwfZAPIjgE2J+uZUfnG82OgsP76yTxJbyebH+Z2SS8C85vM8+3ABhFxPrwxP8/aZL+rLzSZd7VXJS1H9rD3G2TfeFbq55p+pV5Otf5tK003zTYN1fz/X9Hs34G5Rt9WkraLiOm91PpotraX+rj3KiImNpN/jfsV3s9a0i+AL0Y2gVb++BjgqxHx0dpXNn3fD5LNDXRLRLzeRD6/Aw6LiKfS/gyyAVkrAZdExJ5FlDflvTHwLLAscApZ+X8YEY8VdY9W6O3/f0WR33qGKgf6DlAZedjfsU5UVdt7G/Bq5RTF1PYeioitezn3YES8p5n8a+S5OrAhuW+7EXFfE/lNjYjtc/vnRcTJ6fU9EbFTM+U1q4cDfQfoZURg0+3drW4aGgySHouImg8U+zo3wHudRdZMNJc3JzOLiBhwr5t+yv94RGw20Lxr5LcfcBawMdkHVSEftoNF0kiyaZy3YunurUU/tB5y3EbfRqkr2RHAJlU9M1YBni/gFpXumt8qIK92mSrpUxFxYf6gpONYusdPEQ4hmwJ5wE01NdzbS/n/BSh6Pv3vAh8nm5q4G2twl5D12joX2J2s55nnoy+Aa/RtlNpUNyGrxZyaO/UyMDMiFhV0n25uGlqXbEbJ13kzsI8h6yf+sYj43wLvdQNwYkQ8W2Ce65A93F0AVJqAtgOWJxsx+0yB97oT2DMiCh1fMFgq89rkm+Qk/TYi3t/usnU7B/ohoFVNQ4MpTQxWaaufFRF3tOAeY8i6cj7E0tNFN93EJWkPsl5D0Lryb0/WdPMbWjDddaulB9fvJ1sl6w6yhUfOiYh3tbVgJeBA3wFaOKKx0jT0frIuiRWrAIsjYq9m8i8bSbPIluF7kNyCI93S60PSbcArvLX8hUx33Wrpg2o2WZfUs8gmNftGRNzb1oKVgNvoO0OrRjTeTdaXei3g27njLwMza14xtD0XEX2OPehwa0TEPu0uRBNGRMRUsg+rYwEkHQw40DfJNfoO0MoRjcqWxrvVtff+SfoOWZPHJJZu+hhw98rBJOkc4I6IKGJlr0HXSxNjS+fAHypco+8MLRnRCBARiyW9qtxyfNaryjOLfN/2oLlJzQbTScB/Snqd7OF1V3SvlPRhsjVi168azb0qUEiHhKHOgb4zfJKsXf5kshGNGwKfKDD/fwAPSrqd3IyJnhVwaRGxe1/nJY0tejRxkSKiVWvRttp8sgnN9mfpLrMvk/09WJPcdDME9DYVQicHrU7U6c0IkkS2WPcmEXGWpA2B4RFRdH/9wqUmxssiorDFxu1NrtG3UY35vZfS7DzfuXwmpoU6NoqIR4vIc4jq9ME7PyTrbbMHWa+VV4DzyWb77GipiXFNScsVPGDNcKBvt5YuAVch6aNko2OXIxuFOxo4sxumQOgwnf71d8eIeK+k+wEi4sX07Kdb/An4XRolnm9i7IpxAJ3Mgb6NIuJPg3Sr8cAOwJR03xmSWrlOall1eo1+YWoCCQBJa5PrT98F5qeth2yshxXEgb4DVM0AuRzZNLNND5jKWRQRf8uacN/Q6bXTTvS7dhegH98nmy5iHUlnAwcBX25vkepXGdglaZVsN15pc5FKw4G+A1T3lpB0IFkNvCgPSToCGJZmCPw02WAqA1RjPd28StNBZXrhTiNpk4h4IiKulDSdbL57kc2lM7ufyzuGpK3JJuJbI+0/BxwdBa4iNlQ50HegiPiZpFP7T9k3SZdHxCeBx8nmWVkAXAXcSvawzjLd3kxwPbCdpMlpIZNH2l2gAZoAfDYi7gSQtBtwIeClEJvkQN8BJH08t9tDNg1CEU0r26UZMg8lm/Y1Pw3C2yh+Kb6u1C1zwfShR9JXgc1rfTvpooeZK1WCPEBETJFUyMDBoc6BvjPkl8NbBDwJHFBAvhcAtwCbkg1IqRDZB4kXdKAta+sW7TDgQLK/527+djJX0ld4cx2Fo4An2lie0vCAqSFA0n9HxIntLkenSlMGPARcS9brY+mn1l0ysEzShyPiV32c7+iRvWkZxzOAXXlzEfjxEfFiWwtWAg70HUDSpsD3yOZYCeD3wCkRMbetBRsiJK0JHEzWxLUIuAa4oWwBptNH9lrr9LS7AAbAT8hqk8OB9cimK76qrSUaQiLi+Yi4IM11cwzZfOizJH2yvSUrXEePA5A0RtKNku6TNLOytbtcZeA2+s6giLg8t3+FpI7syldmkt4LHA7sDfyK4tekbbdO//p+JfB5qhZOseY50HeGO1N3yqvJ/hgPBX4paQ2AiHihnYUrO0lnkE1HMZvs3+C0otbr7TAdXaMH/hoRk9pdiDJyG30HkNRXz4KICPeOaSFJS4C5wGvpUOWPojKfeyGTy7WbpPM6ddAXgKQ9yb5RTWbphV9ubFuhSsKB3oa8NNagV4M4J9GA1Duyt9NJugLYApjFm003ERH/3L5SlYObbjqApGWBE4EPpENTgB9FxMK2FWoIqRXIJa0FPB/dURPq5r7zedtExHvaXYgyco2+A0i6iGwis0of508CiyPi+PaVauiQtBNwDvAC2dQQl5MtqN5DNtfKLW0s3pAh6ULg3Ih4uN1lKRsH+g4g6YGI2Ka/Y9YakqYBXwRWI5tv5cMRcY+kLYCrImLbPjNosxKM7AVA0mxgM7LRsAso2TOSdnLTTWdYLGmziHgc3hhAtbjNZRpKlomI2wAknRkR9wBExCNVUzt3qhPoY2RvF9m3r5OSVi/bILbB4kDfGT5P1sWyMhJ2BHBs+4oz5OT7bL9Wda4bvvIOpwQje+t46D0Z8MjeAXDTTQeQtALwH2TziAPcTtZW6dklB4GkxWRL1wlYEXi1cgpYISKWbVfZGiVpfbIuip8FvlA1EK+rSbq/05vROpVr9J3hMuAl3pwj/nCyB4IHt61EQ0hEDGt3GYrgkb3WGwf6zvCuqgevd0p6oG2lsa4yhEb22gA50HeG+yXtVHkIKGlHOn99UuscXyEb2btN2v5feohctl4r3fqQue3cRt8BUreydwF/Toc2IqudLaFcf6jWAiUY2btGX+crcz1JWsPzPg2MA30H6PY/VOs83TSyN831FGQ19o2AF9PrtwN/johN2li8UnDTTQdwILdm9DWyV1LHj+ytBHJJFwCTIuLmtP9hYK92lq0sXKM363LdPrK3QtL0iNiu6ti0iBjTrjKVhVeYMut+y0TEbRFxHfC/+ZG9bS5Xo56T9GVJIyRtLOlLwPPtLlQZONCbdb9uH9lbcTiwNvBT4GfAOumYNclNN2Zdrkwje601HOjNrK0k/Zw+vnlExP6DWJxScq8bM2u3b7W7AGXnGr2ZWcm5Rm9mbSXp2og4RNKD1GjC8cjw5rlGb2ZtJWl4RDzd2whxDyhsngO9mVnJuR+9mXUESR+XNEfS3yS9JOllSS+1u1xl4Bq9mXUESY8BH42I2e0uS9m4Rm9mneIZB/nWcI3ezNpK0sfTyw8C7yCb/mBB5XxE3NiOcpWJA72ZtZWkS9LLypz0eRER/zzIRSod96M3s7aKiGMBJE0EPhMR/5f2Vwe+3c6ylYXb6M2sU4yqBHmAiHgR6Iq59DudA72ZdYqeVIsH3lhL1q0OBfAv0cw6xbeBuyVdT9ZefwhwdnuLVA5+GGtmHUPSVsAeZA9lJ0fEw20uUik40JuZlZzb6M3MSs6B3sys5BzozcxKzoHezKzkHOjNzEru/wPKHSkTePpkaQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.heatmap(df.corr(), square=True, cmap='RdYlGn')" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
populationfertilityHIVCO2BMI_maleGDPBMI_femalelifechild_mortality
count1.390000e+02139.000000139.000000139.000000139.000000139.000000139.000000139.000000139.000000
mean3.549977e+073.0051081.9156124.45987424.62305416638.784173126.70191469.60287845.097122
std1.095121e+081.6153544.4089746.2683492.20936819207.2990834.4719979.12218945.724667
min2.773150e+051.2800000.0600000.00861820.397420588.000000117.37550045.2000002.700000
25%3.752776e+061.8100000.1000000.49619022.4481352899.000000123.23220062.2000008.100000
50%9.705130e+062.4100000.4000002.22379625.1569909938.000000126.51960072.00000024.000000
75%2.791973e+074.0950001.3000006.58915626.49757523278.500000130.27590076.85000074.200000
max1.197070e+097.59000025.90000048.70206228.456980126076.000000135.49200082.600000192.000000
\n", "
" ], "text/plain": [ " population fertility HIV CO2 BMI_male \\\n", "count 1.390000e+02 139.000000 139.000000 139.000000 139.000000 \n", "mean 3.549977e+07 3.005108 1.915612 4.459874 24.623054 \n", "std 1.095121e+08 1.615354 4.408974 6.268349 2.209368 \n", "min 2.773150e+05 1.280000 0.060000 0.008618 20.397420 \n", "25% 3.752776e+06 1.810000 0.100000 0.496190 22.448135 \n", "50% 9.705130e+06 2.410000 0.400000 2.223796 25.156990 \n", "75% 2.791973e+07 4.095000 1.300000 6.589156 26.497575 \n", "max 1.197070e+09 7.590000 25.900000 48.702062 28.456980 \n", "\n", " GDP BMI_female life child_mortality \n", "count 139.000000 139.000000 139.000000 139.000000 \n", "mean 16638.784173 126.701914 69.602878 45.097122 \n", "std 19207.299083 4.471997 9.122189 45.724667 \n", "min 588.000000 117.375500 45.200000 2.700000 \n", "25% 2899.000000 123.232200 62.200000 8.100000 \n", "50% 9938.000000 126.519600 72.000000 24.000000 \n", "75% 23278.500000 130.275900 76.850000 74.200000 \n", "max 126076.000000 135.492000 82.600000 192.000000 " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.describe()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "RangeIndex: 139 entries, 0 to 138\n", "Data columns (total 9 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 population 139 non-null float64\n", " 1 fertility 139 non-null float64\n", " 2 HIV 139 non-null float64\n", " 3 CO2 139 non-null float64\n", " 4 BMI_male 139 non-null float64\n", " 5 GDP 139 non-null float64\n", " 6 BMI_female 139 non-null float64\n", " 7 life 139 non-null float64\n", " 8 child_mortality 139 non-null float64\n", "dtypes: float64(9)\n", "memory usage: 9.9 KB\n" ] } ], "source": [ "df.info()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The basics of linear regression\n", "- Regression mechanics\n", " - $y = ax + b$\n", " - $y$ = target\n", " - $x$ = single feature\n", " - $a, b$ = parameters of model\n", " - Define an error functions for any given line\n", " - Choose the line that minimizes the error function\n", "- The loss function\n", " - Ordinary least squares (OLD) : Minimize sum of squares of residuals\n", "- Linear regression in higher dimensions\n", "$$ y = a_1 x_1 + a_2 x_2 + b $$\n", " - To fit a linear regression model here:\n", " - Need to specify 3 variables\n", " - In higher dimensions:\n", " - Must specify coefficient for each feature and the variable b\n", " $ y = a_1x_1 + a_2x_2 + a_3x_3 + \\dots + a_nx_n + b $\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fit & predict for regression\n", "Now, you will fit a linear regression and predict life expectancy using just one feature. You saw Andy do this earlier using the 'RM' feature of the Boston housing dataset. In this exercise, you will use the 'fertility' feature of the Gapminder dataset. Since the goal is to predict life expectancy, the target variable here is 'life'." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "X_fertility = df['fertility'].values.reshape(-1, 1)\n", "y = df['life'].values.reshape(-1, 1)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5jU9ZXn8fepvtHdgN1ig0ZgMI4hwzgogjrKPhmNqyYZZhziNY54S4KMuZmJiZmZ+MRdZvYJatZ1NksATbxEzeh4WbNO1uhkZHcWLwkIIQ4JcUyMgAitdkMDTd/q7B9V1VZ3V3XX7VdVv/p9Xs/D03R1Xb7VDef37fM93/M1d0dERKIjVukBiIhIeSnwi4hEjAK/iEjEKPCLiESMAr+ISMTUV3oAuTjqqKN8zpw5lR6GiEiobNq06W137xh9eygC/5w5c9i4cWOlhyEiEipm9ttMtyvVIyISMQr8IiIRo8AvIhIxCvwiIhGjwC8iEjGhqOophXjceedgP/2DQzTW1zGttZFYzCo9LBGRsotE4I/Hne17evj0/RvZ2dXLzPZm7rpyEXNnTFHwF5HIiUSq552D/cNBH2BnVy+fvn8j7xzsr/DIRETKLxIz/v7BoeGgn7Kzq5f+waEKjShB6ScRqYRIBP7G+jpmtjePCP4z25tprK+rWPBV+klEKiUSqZ5prY3cdeUiZrY3AwwH2fbmBrbv6WHp6g0sXvUcS1dvYPueHuLx4E8lU/pJRColEjP+WMyYO2MKT1y/eMTMPlvwfeL6xXRMaSr6dcf7baJa008iUvsiEfghEfyntTby9sE+egcG2b1viLh7YMF3olTOeOknEZEgRSLVA+8F4o+vfp4vfH8Lv9pzALDh9E9KKYJvPO68tf8wB/sGuXnJPBbMahuTysmWfprW2ljUa4uITKSmZ/zpqRYz445nt9MxuYkbz5/LTY9tpWNyE7ddNJ8vP7p1xKy8mOCbaab/Py4/hZjBm/sOE4/Hgezpp2pZ2FXFkUjtqtnAnykAr7pwPu7OTY8lAv3Orl5ufXo7Ky84keOnT6a5YWSASw9+DfUx6mNGb//4gTDTusFnHnqZe64+lcc27WBmezMdUyYRixmxmJVkLaHUVHEkUttqNtWTKQDf9NhWjj5i0oi8+uYd3Vxz70+pM+iY0jQi6KdX/Hx89fNsf6uHzz60edzqn2yLtvt6B7jqzOO4859/VfWVO6o4EqltNRv4swXgulhuef1Mwe/Lj25lxVnHs7Orlzue3c5b+w+zq+sQnT19wxeB1KLt6Od/52A/Nz22lQsXzsq6eByPO509fWOes9xUcSRS22o28GcLwK2N9ay5YuGIRdU1VyykvblhxH2zBb+25gYWzGrjqjOP45K1L4yp/8+0aLvqwvmsWf8aO7t6mdbaOHyRSQ/07x7sq9iegtGyfe9UcSRSGwIN/Gb2RTP7NzN7xcy+b2aTzOw4M3vJzF41s4fNLJAylmxVM4lxwT1Xn8oT15/JzUvm8fc//hVdvQMjHp8t+HX3DrDirOOH1wlgZCoktWj7yHVn8OiKM7h5yTxu/9F2Nu/oZmZ7M9OnNDGttXFMKulnO/ZVTXpFFUcitS2wxV0zOxb4PDDP3XvN7BHgMuBjwB3u/g9mtgb4JPDtUr/+6KqZhvoYBw4P8vFvPz9isXfN+tfYvKObr//JyDRGKvilL3DedtF8bn16O3/zx783biokFjOOnjqJfb0D3PDwluHHr122kPcd0UwsZnT29I0I9C2NdVWTXqn2iiMRKU7QVT31QLOZDQAtwG7gw8Dlya/fB9xCAIEfGFE109nTx5Xf/cmYxd6bl8xj5VPbxqQxMl046mPGty5fgJmNu/kqVQ10ZEsDj1x3Bu4+4c7d7t6BqtrQVa0VRyJSvMBSPe6+C7gdeINEwN8HbAK63X0webedwLGZHm9my81so5lt7OzsLHo8mXL2HZOb+MCMyTzwydNxfEw+PRX8jm1vYfqUSRzZmvj70VMnjUmFrF22EMPZ2XWIHV2HeK3zACseeJlL1r5A32Acx9m9r3d40XZ0KmnN+te47aL5Sq+ISODMPZjFQzNrBx4DLgW6gX9Mfv51d//d5H1mAT909z8Y77kWLVrkGzduLGo87x7s42c79tHSWEd37wA/3raHpaccO2bzVq616uk1/nGHQ/0DvHtwYMTzffPik3hs086Mr3NCx2Re7TwwIpV0/7WnMXlSPQODcaVXRKRoZrbJ3ReNuT3AwH8x8BF3/2Ty8yuBM4CLgaPdfdDMzgBucffzx3uuYgN/pg1Jq//8FL71L6/yzLa9w/eb2d6cd4O2zp4+Xtm1D4Cbn3xlTKrmnqtP5Zp7fzrm9ieuXzzcKE55dBEJQrbAH2RVzxvAH5pZi5kZcA6wDXgOuCh5n6uAJwMcA5C5Jv/6B1/mwoWzRtyvkMXU/sEhWhrrsi7O1sUs66JteiopffOYiEiQgszxvwQ8CrwM/Dz5WuuAm4C/NLN/B6YB3wlqDCnZavJH588LWUxtrK/jUP8Qh/qHMpZ/pn8s5nVEREol0Dp+d/+6u3/Q3U9092Xu3ufuv3b309z9d939YnfvC3IMkLkm/7x50zn6iEk8uuIM1i5byHnzphe0mDqttZHfmdbCka0NYxZn77jkJGIxWLdsoRZtRaRqBJbjL6VS5/jPmzedz5/zAVY8sGlEjf3c6VOor8//WhiPO929/QwMxjk0MMRQ3Hlr32G++cyv6DzQx2MrziAWiymXLyJllS3HX7PdOdONrsk3My5Z+8KInP9139tU8MlbsZhxZGsTb7x7kLNv/z9jvt435Mw+4r3nTbVqKORCoHbJIlKsSAR+GLkhaVfXoUB2yU5qyHyq1qSG936LKKblsdoli0gp1GyTtvFk68MDFNUV86jWpow9bo5qfW+2X0zLY7VLFpFSiMyMP122PjyffWgznQf6Ct5IlUuPm2JaHqtdsoiUQiQDf3qA7h0Y4rW9B7j16UQHzQWz2tiz/zBXfrewHb0T9bgp5pB1HdAuIqUQyVQPvBeg6wyuufenbN7RDcCKs44fbq8ApU+nFNPyON/HVsvBLiJSXSI54083ehbd1txQcDoll4qbYloe5/NYLQSLSDaRnfGnjJ5FZ9uBO1E6ZfTBKuOdoFVMq4ZcHhuPO2/tP8zBvkFuXjKPBbPatBAsIsMiH/jTZ9Ebbjqbk2YdUVAqploqblIXoEvWvsBFa15g5VPbuPH8ucPBXwvBIhLJVE887rx9sI/DA0PUmdHcODJl0tbcmHcqJpeKm3Jsvsp0ARrvwBkRiZ7IBf5Mue/bLprPjKmTmDOtlVjMCjp9aqKKm3Ll3MdrSKceQSICEUz1ZJoRf/nRrfz2nUNFpWUmqrjJ9Lp3PLudt/YfLmnVTbbNae9ra9bCrogAEZzxZ5sRtzTWFZX/nqjiZvTrLpjVxlVnHjfcM6hUvwFk2px215WLOHrqJAV9EQEiGPizpWQO9Q8Vnf8eL0U0+nVXnHU8Nz02dr9AtkZxua4PFFMuKiLRELlUT6aUzG0Xzed3prUUnf8eb8PU6Ned1tqYdTF49PMMDsbHlIr+Yvd+3j2YOT2kk71EZDyRmPGPni2f0DGZx68/k8MDceoMmhvraGsufFac6se/u/sw16X1+E9P3WRqDZ3pN4+G+tiYReCHPnX6mPWB6x7YxMoLTuToIybllR4aHIyz90AfA0NxGupiTJ/cVNAZBCISXoH9jzezuWa2Je3PfjO7wcxuMbNdabd/LKgxQOaNVa92HuCo1iZmH9nCse0tHNla+Kw49fw/27FvOOhD5jr+9Jn40VMnjfnNY+2yhTTW2Zggv7enL+u6xKfv38hb+w/ntDA8OBjnl8ka/z+6bT2XrH2BX+7pYXAwXtB7F5FwCvLM3e3ufrK7nwwsBA4BTyS/fEfqa+7+w6DGAMFvrEo9f7bD1rMtGMdixgkdk3noU6fz6IozuHnJPO7851+xq+swHZNH5vjfOdifsVKnu3eAnV29vNndm3WXcLq9B/qGTx1LjW/FA5vYeyDw0y9FpIqUK9VzDvCau//WrLz55ng8zs1L5tHW3EB37wBr1r/G5h3dJdvBmqrW6e4dyLtzZlfvAJff/dKIx2zb3cPKC07kmnt/OnzbY5t2sHbZQq773ntppFUXzuf2H21nZnsz7xzs54aHt0x4gtjAUDzjxWlwSDN+kSgpV+C/DPh+2uefNbMrgY3Al9y9a/QDzGw5sBxg9uzZBb1oYoduPyuf2jYiYN73/G9KtoM1Va2zZv1rrLpw/nClTrZWD+nrDUPuGQPxcUe1Dl9EZrY388Vz5w6vSxzqG+I3bx/k9h9tp/NA3/AFIJd2DA11sYwXp/q6kSeE6WhHkdoW+GHrZtYIvAn8vrvvMbMZwNuAAyuBY9z92vGeo9DD1jt7+li6esOYQPfQp05nZntLSQJa+o7cjslNfP6cEzjuqFZamuo4atTawejdu/dcfSo3P/nKmPE9fv2ZGDYcfNubG+jqHaB/cIiG+hg4vPFuYsNZ6jeYme3NE874Uzn+9EPm11yxkA/OSBwyr46eIrWlkoetfxR42d33AKQ+Jgd1F/BUUC+cbbNWXbLKBoqf4eZTNz96veHvf/wqt100f7j/f/pRjenjGxOMly3iyNZGbnh4y7i/XYxWXx/jgzOm8Mh1ZzA4FKd+VFVPtvWQQg+hF5HqVI7A/wnS0jxmdoy7705+uhR4JagXLrZ/Tj6bplKBcbzHjL4Qbd7Rza1Pb+fh5X84PN7Rr/H2wb6xwfh7G3n8L84suKd/Q10Md6ehLlayYyFFJDwCDfxm1gKcC1yXdvOtZnYyiVTP66O+VlKZ2hesXbaQ9uYGYPwZ7rTWxrzTHhNdSDJdiDoP9NFYX5d1Rn14IHMwPjwYZ/aRLXl9PwoZn452FKk9ge7ccfdD7j7N3fel3bbM3f/A3ee7+5+mzf5LLlvJ5KudB4jHfdwZbiFloBM9ppBjF+uSG73SzWxvpq6AlHspxqfjHEXCr+Z37qZKJjsmN7HirOP55H94P2/tO8yMqU3jznALSXtM9JhC+ug0N9aNWQe47aL5tDbV0dnTV9IzAyYanxZ/RWpDzQf+/sEhOiY3ceP5c0eUWq69YiHHtk9i7RULx7RZmNbaOLxpKp+0Ry6pknx7/bc1NzJj6iRWXnAiLY11w0dD7t7Xl3cALnZ8WvwVqQ0136Slsb6Oz59zwphOmNc9sInNb+zja//zFVZecCLrbzyLx68/czh4FpKWKeQxE4nFjDnTWjnx2COY2d7MicceQVNDXUG7kYsdnxZ/RWpDzc/4p7U2ctxRrVl73Wze0c019/50uA4+NWMuJC0TVEvk0bPwXV2HCgrAxY5Pi78itaHmZ/yxmNHSlPlUqu7egeHPMwXOQtobl6MlcrZTtnIJwMWML4jfaESk/Go+8AMc1dqUsQf/mvWvDd8nTDPXSgXg9N8YNtx0Nk9cv1gLuyIhVPOpHngvYKV63by1/zCTGmJ0JrtSnjdvOl/743n0Dw7R2dNXVHqmHL1uKnnKViEH0YtIdYlE4IdEwDKMK76TKO380nkf4N5rTqWhLsbAUHy4S+bM9mbuv/Y0Jk+qZ2AwPm5QHR3k25sbeLXzQFnKHRWARaRQkQn8kL2087aL5tMxuYmdXb10TG5iz/7DXPndreMG70w17WuXLeTOf/6Vyh1FpKpFIsefkq2088uPbmXFWccDiUPQU5ulUl/PVCqZqab9uu9t4sKFs0bcT+WOIlJtIhX4xyvtbEv272lrbsipVDJbTfvoBdYwLRqLSDREKtWTXto5uhb9UH8isKd2xk5Uq54qqUy1gmhrbuBQ/xDHtk0acYiKyh1FpNoEfhBLKRR6EEsm2frNzJjaRG//EM2NdezZP3E7hHjcef2dg+zZf3hkP/1li5hxROK5dIKViFRStoNYIhf4YeKSy1xLMvf2HObjq58f89uBFnNz/x7qqEeR4FTyBK6qM1EpZK6lkgODmQ8vD/tibrHBONcunpXu9qmLjkRVpBZ3Sy3X1glh6mGfCsZLV29g8arnWLp6A9v39OQ15lzPMijkzINSKcX7FAkrBf4i5HpwSZgCTCmCca5dPCvZ7bOSFx2RSotkqqdUv+Ln0johbD3s04PxglltwxVL/YNDxOOe0/cp1y6elez2qRbTEmWBzfjNbK6ZbUn7s9/MbjCzI83sWTN7NfmxPagxZFLqGfhE3S7DFmBSwXjBrDZuPH8uK5/axqXrXuTSdS/m/H3KtYlcJbt9FtPhVCTsylLVY2Z1wC7gdOAzwLvu/g0z+yrQ7u43jff4Ulb1dPb0sXT1hrJV4pT79YqVujC+te8wNz/5SsHjrvaqnkovLIuUQ6Wres4BXnP335rZBcBZydvvA9YD4wb+Uso0A++Y3ET/4BC7ug6VPPikZrWjA0y1bupKpa9am+qK+k0l18qoSjWbq2SHU5FKK1fgvwz4fvLvM9x9N4C77zaz6ZkeYGbLgeUAs2fPLtlARueVF8xq4ysfmcul614MZOYXdIAJYsYcixnNDfVF5d+DGFepn1MdTiWqAk/1mFkj8Cbw++6+x8y63b0t7etd7j5unj/Inbv3XH1qUSmNSgoyXVHMcwcxLqVmRPJXyVTPR4GX3X1P8vM9ZnZMcrZ/DLC3DGMYNnoGPuQeqsVXeG/m2zswyFv7Dg+3lC5lxVAxv6kEUckUtuookWpWjsD/Cd5L8wD8ALgK+Eby45NlGMMI6b/id/b0heoA8Uwz31UXzuf2H21n847ukl60Ck2FBFHJFLbqKJFqFugGLjNrAc4FHk+7+RvAuWb2avJr3whyDBMJ2wHi6TPfBbPauHnJPJrqY9x60XwWzGoryUWr2J3GQZRKqvxSpHQi2aRttDD1bNnVdYjFq54brrMffZLYjKmTmDOttaK5dOX4RaqDunMSrgCfSTzu7Ow6xOV3v8TNS+ax8qltY1JUj19/JtOnTCr4NUq17yAMVT0ita7SdfwVVwszxncO9vO3/7SNVRfOp6k+ljHnPTAYL+o1SpVLD6JUUuWXIqURmSZttdCUq39wiGe27eX2H23niOaGQHLeyqWL1L7IBP5aqApJBeXNO7r5yqNbWXXh/JIvSodtsVtE8heZVM94nSDDkjtOb/+weUc39z3/Gx761OnUxYyG+hj1MWP3vt7AO46KSLhFZnE3W47/hI7JvNp5IDS5/0wXKSD06xdRFJYJh4SXqnrI/B/tnYP9oeqemUnYOoBKbRQbSPXLFvgjk+OHzL3zayH3XwvvIWpqodhAwitSgT+TWqhiqYX3EDW6WEslRT7w10IVSy28h6jRxVoqKVI5/mxqYZGtFt5DlCjHL+UQ+Z2749GO0NzpAlMaKpuVSlLgrwHlmj1qllpamnBIpUQ+x18LylUhokoUkdqgGX8NyLdCpNB0jSpRRGqDAn8NGK8dxWjFpGvyeR0RqV5K9dSAfMo5i0nXqGxUpDYEOuM3szbgbuBEwIFrgfOBTwOdybv9tbv/MMhx1Lp8KkSKSdeoEkWkNgSd6rkTeNrdLzKzRqCFROC/w91vD/i1IyXXCpFi0zWqRBEJv8BSPWY2FfgQ8B0Ad+939+6gXk9yo3SNiAQ5438/iXTOPWZ2ErAJ+ELya581syuBjcCX3L1r9IPNbDmwHGD27NkBDrO2ZargUbpGJNoCa9lgZouAF4HF7v6Smd0J7Ae+BbxNIue/EjjG3a8d77mCbtlQq7ThqrZpF7VMpKi2zGb2ATP7sZm9kvx8vpl9bYKH7QR2uvtLyc8fBU5x9z3uPuTuceAu4LTc34bkI6wbruJxp7Onj11dh+js6SMer/5+UuWWuqgvXb2BxaueY+nqDWzf06PvleQk1xz/XcBfAQMA7r4VuGy8B7j7W8AOM5ubvOkcYJuZHZN2t6XAK3mNWHIWxg1XCmi5CetFXapDroG/xd1/Muq2wRwe9zngQTPbCpwM/BfgVjP7efK2s4Ev5jxayUsYW/8qoOUmjBd1qR65Lu6+bWbHk8jLY2YXAbsnepC7bwFG55eW5TVCKVj64ezpOf5qruBRQHvPeDl87aKWYuQa+D8DrAM+aGa7gN8Afx7YqKQkwrjhKkoBbbzAPtHCfBgv6lI9xq3qMbMvuPudZrbY3TeYWSsQc/ee8g1RVT0pUajiiEol0kTvs7Onj6WrN4y5AD5x/eLhDXQT/XuIwr8XGV+hB7FcQ2L37X8nUZFzMIjBycRKGRCrOSCE8beUQmRby0gF9lxSXuPtoo7KBVQKM9Hi7i/M7HVgrpltTfuTWpyVMinVomcYqmZSAe3Y9hY6pjSVLVCVs4x0osBe7MK8FsllPOMGfnf/BPCHwL8Df5L2Z0nyo5RJqRY9FRAyK/cFcaLAnq21RntzQ04XJy2Sy3gmXNxN1uOfVIaxyDhKteipgJDZRKmXUptocTZTyqu9uYFXOw/klL6J0iK55G/cwG9mj7j7JWb2c5KlnKkvAe7u8wMdnQwrVRWHAkJm5b4g5rKWMTqH39nTxx3PbufmJfNoa26gu3eAO57dzt8tnT/m4lSuqp9qXi+S7Caa8aeaqi0JeiAyvlIteqoMMLNKXBDzbXEdj8e56szjuOmxrcM/u1UXzicej2d87qAXybWAHF6BNWkrJZVzlpZmaWOFIYi92d3LJWtfGHNxeuS6M3hfW/M4jwxGLiWnUlkFlXOaWQ8jUzzDXyKR6plaovFJGVXbYSrVcCEKQxmpu2dMR1Vq8qb1ovAaN/C7+5RyDUSiqZpm2uW8IBZysau29ZlqG4/kToety7BKtEOOYnlpoaWj1XZ6WrWNR3IX9Jm7EhKVmnkXmi6ohvRQoQotHZ0oHVXu70kY0mOSmQJ/mVVrwCp3HXtKIemCakoPFaKY3Hi2dFSlvifVtl4kuVGqp4yquV1CpRbqCkkXhD09FMQ5CWH/nkh5KfCXUTX/56zUoS3p6YINN53NE9cvnnCWGvZqkiBy42H/nkh5KdVTRtX8n7OSG7vyTReEvZqk1LnxeNwxs1B/T6S8FPjLqJoDVpgW6mph93GpcuOp9OEdz25n1YXzR+zqDdv3RMon0J27ZtYG3A2cSGIj2LXAduBhYA7wOnCJu3eN9zy1snM37IuS1aRaF8mLle/7St89u2BWGyvOOp5prY28r62Zo6dOqonviRQu287doAP/fcC/uvvdZtYItAB/Dbzr7t8ws68C7e5+03jPUyuBH2o3YEnxCpkY7Oo6xOJVz425fcNNZ3Nse0ver69/m9WhVD+LQk/gKpiZTQU+BFwN4O79QL+ZXQCclbzbfcB6YNzAX0tU/ibZFFJSW6r0oX4brR7l+FkEWdXzfqATuMfMNpvZ3ckze2e4+26A5MfpmR5sZsvNbKOZbezs7AxwmCLVoZDF/1JVCJWr4qwSu8PDphw/iyAXd+uBU4DPuftLZnYn8NVcH+zu64B1kEj1BDNEkepRyOy9VIvy5ag4028VuSnHzyLIGf9OYKe7v5T8/FESF4I9ZnYMQPLj3gDHIBIahc7eS3FGcTn2cVTzPpZqUo6fRWCBP3lk4w4zm5u86RxgG/AD4KrkbVcBTwY1BpEwKWQzW6mUo+FaNe9jqSbl+FkEXcf/OeDBZEXPr4FrSFxsHjGzTwJvABcHPAaR0Mh18b/UFTjl2MdRzftYqkk5fhaBBn533wKMKSUiMfsXkQIElSsPuuKsFjbelUvQPwsdvSiRE/Z69UKOPKyW91yqcVTL+6l2Za/jF6lGtVBZkm+uvFrecymDfjW8nzBTd06JlFqoLMm36qMa3nMpW5JXw/sJOwV+iZRaqCzJt+qjGt5zKYN1NbyfsFOqRyIln8qSas0j51v1UQ3VNKUM1qPfz4JZbXz+nBMY8sSu4Gr5OVUzzfglUnKdLVfzaWmQ36atajgUvdhNSemtHhzn/mtPY2Z7MwtmtfGVj8zl5idf4UO3rq+6n1O1UlWPRE4uM/lCKmeqWaV/eylmQTbbY2dMbaK3f4hL171YMz+nUlNVj0hSLjXStZZHrnRX2GI2JY3XtTT1ebow/5zKRYFfJINqyIuXQqVn+ukKvfiMdxGuxM+pmr6nhVKOXySDasiL5yNTu+NqX6fI1XjrA+X+OdXK91Q5fpEswjKzy5YDnza5kY+vfj70+e+J1gfK+XMK29qPcvwieap0XjxX2XLgD3369JrIf0+0PlDOn1OtrP0o8ItUmXxmsPG40z84xDcvPonu3gHWrH+NzTu62dnVS51ZTaxTQPVchGtl7Uc5fpEqkk8OOXXfS9e9yKXrXmTlU9u48fy5LJjVxsz2Zpob60K1TlEp+RwHGba1n2yU4xepIvnkkLPdd+UFJ3L0EZOYO2MKQCjWKSqlkP0FYVn7gew5fs34RapIPjnkbPc9fvrk4cBVimMZa1khPYRq4XuqwC9SRfJpbZDtvs0NdaEMRpVQK4u1+Qo08JvZ62b2czPbYmYbk7fdYma7krdtMbOPBTmGKMknVynVKZ8ccqnzzVH891OOg82rUaA5fjN7HVjk7m+n3XYLcMDdb8/1eZTjn5gOp6gd+Vb16HCTwtX6+1Ydf40br59JNZTBSe7yKV0sVZljlP/9NNXHWHnBibQ01nGof4im+trPgAcd+B14xswcWOvu65K3f9bMrgQ2Al9y967RDzSz5cBygNmzZwc8zPCLaq5SSiOq/37eOdjPld/9SWh24pZK0Je2xe5+CvBR4DNm9iHg28DxwMnAbuCbmR7o7uvcfZG7L+ro6Ah4mOEX1VyllEZU//1E9YIXaOB39zeTH/cCTwCnufsedx9y9zhwF3BakGOIilrZWCKVEdV/P1G94AW2uGtmrUDM3XuSf38W+M/Az9x9d/I+XwROd/fLxnsuLe7mJkwbS6T6RPHfjxZ3S28G8ISZpV7nIXd/2sy+Z2Ynk8j/vw5cF+AYIqVa+plIOEXx308xB8SEWWCB391/DZyU4fZlQb2miEi+InnBq/QARESkvBT4RUQiRoFfRCRiFPhFRCJGgV9EJGIU+EVEIkaBX0QkYtSdU0RKJoq7f8NIgV9ESqLW2x/UEqV6RKQkCjm/VipDM34RKcjotE5UWxyHkQK/SA0KOteeKa3z0KdOZ2Z7825KpcoAAAihSURBVJhDTWq9xXEYKdUjUmNSQXnp6g0sXvUcS1dvYPuenpIenp4prfO3/7SNtcsWRq6nfxhpxi9SY8pxfm6mtM4z2/ay8oITI9fiOIwU+EVqTDly7amTq0andWKxWORaHIeRUj0iNaYcxwlG9ajGWhHY0YulpKMXRXJXrnp6bdaqfpU4ehEzex3oAYaAQXdfZGZHAg8Dc0gcvXiJu3cFOQ6RKCnXcYJRPLmqVpQj1XO2u5+cdtX5KvBjdz8B+HHycxEpoVRQPra9hY4pTZqJywiVyPFfANyX/Pt9wJ9VYAwiIpEVdOB34Bkz22Rmy5O3zXD33QDJj9MzPdDMlpvZRjPb2NnZGfAwRUSiI+hyzsXu/qaZTQeeNbNf5vpAd18HrIPE4m5QAxQRiZpAZ/zu/mby417gCeA0YI+ZHQOQ/Lg3yDGIiMhIgQV+M2s1sympvwPnAa8APwCuSt7tKuDJoMYg0RGPO509fezqOkRnT19J2xOI1JogUz0zgCfMLPU6D7n702b2U+ARM/sk8AZwcYBjkAhQH3iR/AQW+N3918BJGW5/BzgnqNeV6ClHbxqRWqKWDRJ66gMvkh8Ffgm9cvSmEaklCvwSemoYJpIftWWW0CtXbxqRWqHALzVBDcNEcqdUj4hIxCjwi4hEjAK/iEjEKPCLiESMAr+ISMQo8IuIRIwCv4hIxKiOX0IjHnfeOdivTVoiRVLgl1BQ62WR0lGqR0IhW+vldw72V3hkIuGjwC+hoNbLIqWjwC+hoNbLIqWjwC9lUeyZuGq9LFI6gS/umlkdsBHY5e5LzOxe4I+Afcm7XO3uW4Ieh1ROKRZm1XpZpHTKMeP/AvCLUbd92d1PTv5R0K9xpVqYTbVePra9hY4pTQr6IgUKNPCb2Uzgj4G7g3wdqW5amBWpLkHP+P8b8BUgPur2vzOzrWZ2h5llPD3DzJab2UYz29jZ2RnwMCVIWpgVqS6BBX4zWwLsdfdNo770V8AHgVOBI4GbMj3e3de5+yJ3X9TR0RHUMKUMtDArUl2CXNxdDPypmX0MmARMNbMH3P2K5Nf7zOwe4MYAxyBVQAuzItUlsBm/u/+Vu8909znAZcC/uPsVZnYMgJkZ8GfAK0GNQaqHFmZFqkclevU8aGYdgAFbgBUVGIOISGSVJfC7+3pgffLvHy7Ha4qISGbauSsiEjEK/CIiEaPALyISMeaeX7OsSjCzTuC3wFHA2xUeTjE0/soK+/gh/O9B4y+v33H3MRuhQhH4U8xso7svqvQ4CqXxV1bYxw/hfw8af3VQqkdEJGIU+EVEIiZsgX9dpQdQJI2/ssI+fgj/e9D4q0CocvwiIlK8sM34RUSkSAr8IiIRE4rAb2bfNbO9ZhbKTp5mNsvMnjOzX5jZv5nZFyo9pnyY2SQz+4mZ/Sw5/v9U6TEVwszqzGyzmT1V6bHky8xeN7Ofm9kWM9tY6fHky8zazOxRM/tl8v/BGZUeU67MbG7y+576s9/Mbqj0uIoRihy/mX0IOADc7+4nVno8+Uq2oj7G3V82synAJuDP3H1bhYeWk2QL7VZ3P2BmDcD/A77g7i9WeGh5MbO/BBYBU919SaXHkw8zex1Y5O5h2jw0zMzuA/7V3e82s0agxd27Kz2ufJlZHbALON3df1vp8RQqFDN+d/+/wLuVHkeh3H23u7+c/HsPicPnj63sqHLnCQeSnzYk/1T/jCGNzn+uHDObCnwI+A6Au/eHMegnnQO8FuagDyEJ/LXEzOYAC4CXKjuS/CTTJFuAvcCz7h6q8ZP9/OewcOAZM9tkZssrPZg8vR/oBO5JptruNrPWSg+qQJcB36/0IIqlwF9GZjYZeAy4wd33V3o8+XD3IXc/GZgJnGZmoUm5jXP+c5gsdvdTgI8Cn0mmP8OiHjgF+La7LwAOAl+t7JDyl0xR/Snwj5UeS7EU+MskmRt/DHjQ3R+v9HgKlfwVfT3wkQoPJR+p859fB/4B+LCZPVDZIeXH3d9MftwLPAGcVtkR5WUnsDPtt8RHSVwIwuajwMvuvqfSAymWAn8ZJBdHvwP8wt3/a6XHky8z6zCztuTfm4H/CPyysqPKXbbznys8rJyZWWuyKIBkiuQ8QnRWtbu/Bewws7nJm84BQlHYMMonqIE0D1TmzN28mdn3gbOAo8xsJ/B1d/9OZUeVl8XAMuDnyTw5wF+7+w8rOKZ8HAPcl6xoiAGPuHvoSiJDbAbwRGL+QD3wkLs/Xdkh5e1zJM7bbgR+DVxT4fHkxcxagHOB6yo9llIIRTmniIiUjlI9IiIRo8AvIhIxCvwiIhGjwC8iEjEK/CIiEaPAL5FmZp9Pdot8MMf7zzGzy9M+X2Rmf5/8+9Vm9q3k31eY2ZVpt78viPGLFCIUdfwiAboe+Ki7/2aiO5pZPTAHuBx4CMDdNwJj2iS7+5q0T68mseHqzeKHK1I8BX6JLDNbQ6KB2A/M7B+A44E/IPH/4hZ3f9LMribR1XMS0Aq0AL+X3Ih3H7AZuHF0m2czu4VEK/HXSbSCftDMeoG/AT7l7kuT9zsX+At3/3iw71bkPUr1SGS5+woSs/CzSQT1f3H3U5Of35bWQfIM4Cp3/zCJ5mL/6u4nu/sdObzGoyR+I/jzZJO7H5K4cHQk73INcE8p35fIRBT4RRLOA76anMmvJzHDn5382rPuXpLzIDyxVf57wBXJ/kdnAP+7FM8tkiulekQSDLjQ3bePuNHsdBJthEvpHuB/AYeBf3T3wRI/v8i4NOMXSfgR8LlkJ1XMbEGW+/UAU/J87hGPSbZYfhP4GnBv3iMVKZICv0jCShJHSm41s1eSn2eyFRhMHjz/xRyf+15gTfKg7ubkbQ8CO8Jy7rLUFnXnFKmAZL3/5pC1F5caocAvUmZmtonEusG57t5X6fFI9Cjwi4hEjHL8IiIRo8AvIhIxCvwiIhGjwC8iEjEK/CIiEfP/AYBIziIy92T5AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.scatterplot(x='fertility', y='life', data=df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, there is a strongly negative correlation, so a linear regression should be able to capture this trend. Your job is to fit a linear regression and then predict the life expectancy, overlaying these predicted values on the plot to generate a regression line. You will also compute and print the $R^2$ score using sckit-learn's ```.score()``` method." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.6192442167740037\n" ] }, { "data": { "text/plain": [ "[]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3iUVfbA8e+ZFFIAA0hRQAKIKCBFEERcG4risoqAYKOIiogoKgiiyU+XgFIUEBcELIgiShFEbMgquHY3FDELRloQlBIw1ITU+/sjkyGTTJKZZPqcz/PkCZk7886dTDjvO+eee68YY1BKKRU6LL7ugFJKKe/SwK+UUiFGA79SSoUYDfxKKRViNPArpVSICfd1B5xx9tlnm/j4eF93QymlAsqGDRsOG2Pqlrw9IAJ/fHw8ycnJvu6GUkoFFBHZ4+h2TfUopVSI0cCvlFIhRgO/UkqFGA38SikVYjTwK6VUiAmIqh53KCgwHDmVQ05ePpHhYdSJjcRiEV93SymlvC4kAn9BgSH14AnufyuZfRlZNKoVzauDOtGyfg0N/kqpkBMSqZ4jp3JsQR9gX0YW97+VzJFTOT7umVJKeV9IXPHn5OXbgn6RfRlZ5OTl+6hHhTT9pJTyhZAI/JHhYTSqFW0X/BvViiYyPMxnwVfTT0opXwmJVE+d2EheHdSJRrWiAWxBtlZ0BKkHT3DrnG/pNmUdt875ltSDJygo8PyuZJp+Ukr5Skhc8VssQsv6NVg5opvdlX1ZwXfliG7UrVGtys9b3qcJf00/KaWCX0gEfigM/nViIzl8Kpus3Dz2H8unwBiPBd+KUjnlpZ+UUsqTQiLVA2cCcZ853zHq3c38dvAkILb0TxF3BN+CAsOB46c5lZ1HYq9WdGgcVyqVU1b6qU5sZJWeWymlKhLUV/zFUy0iwoy1qdStXo0xN7Rk3PtbqFu9GtP6teWJ5VvsrsqrEnwdXenPvvMSLAJ/HjtNQUEBUHb6yV8GdrXiSKngFbSB31EAntK3LcYYxr1fGOj3ZWQx9bNUkm5pQ/N61YmOsA9wxYNfRLiFcIuQlVN+IHQ0bvDQ4o0sGHIp72/YS6Na0dStEYXFIlgs4paxBHfTiiOlglvQpnocBeBx72+hwVlRdnn1TXuPcs+b/yVMoG6NanZBv3jFT58535F64AQjF28qt/qnrEHbY1m5DL68KS/9+ze/r9zRiiOlglvQBv6yAnCYxbm8vqPg98TyLQy/ujn7MrKYsTaVA8dP80dGJuknsm0ngaJB25LHP3Iqh3Hvb6Fvx8ZlDh4XFBjST2SXOqa3acWRUsEtaAN/WQE4NjKcuXd3tBtUnXt3R2pFR9jdt6zgFxcdQYfGcQy+vCn9531fqv7f0aDtlL5tmbt+J/sysqgTG2k7yRQP9H+dyvbZnIKSyvrdacWRUsHBo4FfRB4Tkf+JSIqIvCsiUSLSVER+FJHtIrJERDxSxlJW1Uxhv2DBkEtZOeJyEnu1YtYXv5GRlWv3+LKC39GsXIZf3dw2TgD2qZCiQdulD3Rl+fCuJPZqxQtrUtm09yiNakVTr0Y16sRGlkol/bz3mN+kV7TiSKng5rHBXRFpCDwCtDLGZInIUuB24CZghjHmPRGZC9wLvOLu5y9ZNRMRbuHk6Tz6vPKd3WDv3PU72bT3KM/8wz6NURT8ig9wTuvXlqmfpfL03y8qNxVisQgNakZxLCuXR5dstj1+3sCOnHtWNBaLkH4i2y7Qx0SG+U16xd8rjpRSVePpqp5wIFpEcoEYYD9wLXCntX0h8CweCPyAXdVM+olsBr3xU6nB3sRerUj6aGupNIajE0e4RfjXnR0QkXInXxVVA9WOiWDpA10xxlQ4c/doVq5fTejy14ojpVTVeSzVY4z5A3gB+J3CgH8M2AAcNcbkWe+2D2jo6PEiMkxEkkUkOT09vcr9cZSzr1u9GhfUr86ie7tgMKXy6UXBr2GtGOrViKJ2bOG/G9SMKpUKmTewI4JhX0YmezMy2Zl+kuGLNtJ/3vdk5xVgMOw/lmUbtC2ZSpq7fifT+rXV9IpSyuPEGM8MHopILeB9YABwFFhm/fkZY8z51vs0Bj4xxlxc3rE6depkkpOTq9Sfv05l8/PeY8REhnE0K5cvth7k1ksalpq85WytevEa/wIDmTm5/HUq1+54L97Wjvc37HP4PC3qVmd7+km7VNJbQztTPSqc3LwCTa8opapMRDYYYzqVut2Dgf824EZjzL3WnwcBXYHbgAbGmDwR6Qo8a4y5obxjVTXwO5qQNOeuS/jXl9v5fOsh2/0a1Yp2eYG29BPZpPxxDIDEVSmlUjULhlzKPW/+t9TtK0d0sy0Up3l0pZQnlBX4PVnV8ztwmYjEiIgA3YGtwDqgn/U+g4FVHuwD4Lgmf8Q7G+nbsbHd/SozmJqTl09MZFiZg7NhFilz0LZ4Kqn45DGllPIkT+b4fwSWAxuBX6zPNR8YBzwuIjuAOsDrnupDkbJq8kvmzyszmBoZHkZmTj6ZOfkOyz+Lf6/K8yillLt4tI7fGPOMMeZCY0wbY8xAY0y2MWaXMaazMeZ8Y8xtxphsT/YBHNfk92hVjwZnRbF8eFfmDexIj1b1KjWYWic2kiZ1YqgdG1FqcHZG/3ZYLDB/YEcdtFVK+Q2P5fjdyd05/h6t6vFI9wsYvmiDXY19y3o1CA93/VxYUGA4mpVDbl4Bmbn55BcYDhw7zYuf/0b6yWzeH94Vi8WiuXyllFeVleMP2tU5iytZky8i9J/3vV3O/4G3N1R65y2LRagdW43f/zrFNS98Vao9O99w3llnjlu0VENlTgS6XLJSqqpCIvCD/YSkPzIyPTJLNirC8a5aURFnPkVUZcljXS5ZKeUOQbtIW3nKWocHqNKqmGfHVnO4xs3ZsWeu9quy5LEul6yUcoeQueIvrqx1eEYu3kT6yexKT6RyZo2bqix5rMslK6XcISQDf/EAnZWbz85DJ5n6WeEKmh0ax3Hw+GkGvVG5Gb0VrXFTlU3WdYN2pZQ7BHWq58CBA3zxxRc4qlwqCtBhAve8+V827T0KwPCrm9uWVwD3p1OqsuSxq4/1l41dlFL+Jaiv+KdNm8b06dPp2rUrCQkJ9OzZk8JJxGeUvIqOi46odDrFmYqbqix57MpjdSBYKVWWoL3iT09PZ+7cuQB8//33/P3vf+fSSy/lgw8+oKCgwHa/klfRZc3ArSidUnJjlfJ20KrKUg3OPLagwHDg+GlOZeeR2KsVHRrH6UCwUsomaAN/QUEBgwYNIjLyTBpkw4YN3HrrrXTo0IGlS5eSn59vdxX97bhraNf4rEqlYvyl4qboBNR/3vf0m/s9SR9tZcwNLW3BXweClVJBG/jr16/PK6+8ws6dO3nkkUeIioqytW3ZsoUBAwbQ8qJWzJr3OodPZFInNpKGtWKoHVvN7kSwckQ3p9IjzlTceCPn7ugENO79wk3idSBYKQVBHPiLNGrUiJdeeondu3fzxBNPEBsba2vbuf03Rg2/j47tLmbay6+QnV14dV6ZVExFG5S7kgqqivIWpNM1gpRSEAKBv0iDBg2YOnUqySmpNLrmLiQyxta2b89unnz0IVpc0IK5c+eSne36unEVVdw4uhKfsTaVA8dPu/UTQFknoHPjonVgVykFhMgibcX9kZFJtynryD99khPJH3IieRUF2afs7nPuuecyduxY7r//fmJiYso4UmnlVfUUPW+RDo3jGHNDS8a9X7n5AuX1Qat5lFLggx243MmdgT/9RDa3zvnWduVdkJ2JZdsajv70Accyjtjdt169eowZM4YHH3yQ6tWru/V55w3sSNJHWx3uzOVoApgri7PpQm5KKfDNDlx+qWRK5rwGdXh39mS+2fg/pk2bRv369W33PXToEGPHjiU+Pp5JkyZx7Nixco9d3uBtyeetExtZ5mBwyePk5RWUGh/Ytv84f51ynB7Snb2UUuUJiSv+klfAtaIj+Csrh9O5BYQJREeGERddeFWclZXFa6+9xpQpU/jjjz/sjhMXF8cjjzzCqFGjqF27tt3xj2blsP/oaR4otsZ/yRRL8X6UXBoaCq/4V4y4nCMnc+xSNYvv68Kdr/1Y6r5Jt7ShwVlRLqVx8vIKOHQym9z8AiLCLNSrXq1SexAopfyf16/4RaSliGwu9nVcRB4VkWdF5I9it9/kqT6A42qa7eknOTu2GufVjrGVcBYFzujoaB5++GF27tzJvHnziI+Ptx3r6NGjTJgwgfj4eMaPH096errt+D/vPWYL+uC4jr/4lXiDmlGlBoPnDexIZJiUGgQ+dCLb4aeDmMgw7n8rmQPHTzs1MJyXV8Cv1hr/q6atp/+87/n14Any8goqfKxSKnh4cs/dVGNMe2NMe6AjkAmstDbPKGozxnziqT5A5SdWVatWjWHDhvHbb7/x5ptv0qJFC1vbiRMnmDx5MvHx8Tw06jEG/2tNmZutlzVhymIRWtStzuL7urB8eFcSe7XipX//xh8Zp6lb3T7Hf+RUjsNKnaNZuezLyOLPo1lOlYYeOplt23WsqH/DF23g0EmP736plPIj3vqM3x3YaYzZ46XnsykoKCCxVyuWDLuMeQM7ujyDNSIigsGDB7Nt2zYWL15Mq1atbG2ZmZnM/ddL/HfKXTyXOJa6lpN2j61owlRGVi53vvYj/eZ+zwNvb+DzrYd4YNEGHunewu5+72/Yy7wS+/ZO6duWuet30qhWtO3kVtHJLDe/wOHJKS9fr/iVCiXeWqTtduDdYj+PFJFBQDIw2hiTUfIBIjIMGAZw3nnnVepJCwoMh0/l2KpnigLmwu92uzyDNSwsjDvuuIMBAwawYsUKJk6cyM8//1zYmJ/Lh4vfIDzibep06EFYh1tp2rSpwwlTxfP8+cY4DMRNz461LRzXqFY0j13fkhZ1q7NixOVkZuez+/ApXliTSvrJbKb0bcsLa1KdOplFhFkcLuscHma/Q5hWBCkV3Dw+uCsikcCfQGtjzEERqQ8cBgyQBJxjjBla3jEqO7hbsoQSsA2WNqoVU6WAZoxh9erVJCUlUbJvYWFh3Hb7nTybmEDLlhfYbi9ZY79gyKUkrkpxOMAriN1gdEZWLjl5+USEW8DA739lcuRUDnPX72TT3qPlloIWKcrxF99kfu7dHbmwfuEm8zoHQKng4rM6fhG5BXjIGNPDQVs88JExpk15x6hs4C85aarIt+OuoWGtwolZVb3CNcbw6aef8eyECfz3xx/s2iwWC7fffjtPPfUUrVu3LnUi6tA4jrE3trSt/19WJVCpYDywE9UiLAx64yeXA3RRVU9efgHhJap6yjpRVnYTeqWUb5UV+L2R6rmDYmkeETnHGLPf+uOtQIqnnriiHasqusJ15qQgItx0U0969ryR9evXM2HCBNavX289fgGLFy/m3XffpW/fvgx7ZLRdXzbtPcrUz1JZMuwyW39LPsfhU9mlB6ffTmbFg5dXek3/iDALxhgiwixu2xZSKRU4PDq4KyIxwPXAimI3TxWRX0RkC3AN8Jinnt/R+jnzBnakVnQEUH7Fj6uLqokIV111NXMWr+Li4TOJiu9gazPGsHz5cnpc2ZUTHz5H9v7ttrb0k9lEhoeVOdnqdK7jYHw6r8DlSVoVvaaKFppTSgUHjwZ+Y0ymMaaOMeZYsdsGGmMuNsa0NcbcXOzq3+3KKpncnn6SggJT7hVuZcpAix5z/KzzqT8giQYDX6TWRV3t7vPXtu848NZjHFz6DDWP7axwxcwwEYfBOKwSKfeKXpMzWzvqdo5KBb6gn7JZVDI56eNtANx7RTMOHDvN0ayccq9wK5P2KPmYaue2pObNT7Pmq+/o27ev3X1P797AL3NH8dBdvfnPf75yuC8wFM4qntavrV0wntavLbHVwlwOwBW9ppKb0pTci8BbS0srpTwr6AN/Tl4+datXY8wNLUn6aCsD5v9A4qoU9h89TUQYzLu7o8Mr3MqkPcp6TIcOl7B8+XJSUlK44447sFjO/NrXrVvHNddcw5VXXsmaNWtKnQDioiOpXzOKpFvasGTYZSTd0oZGtaLZfyzb5QDszGsqb50ff9llTClVNUEf+CPDw3ikewvb8sdQGLAeWLSBTb8fI+GDFJJuacP6MVezYsTltitcZ9IeJVX0mNatW7N48WK2bdvG4MGDCQs7E3C/+eYbbrzxRi677DJWr15tOwFYLEJ8nVjaNDyLRrWiadPwLKpFhFUqAFfmNRWng79KBYegX6StoMDw+1+ZXP3C+lJtS4ZdxoD5hSWYjsoWK1Pq6cpjdu3axeTJk3nzzTfJzc21a2vfvj0JCQnceuutdp8QwLkyVXf0ryQt91QqsITssswWixBTzXGK42jWmWDr6Mq1Mssbu/KYZs2aMX/+fHbu3MnIkSOpVu1M8Ny8eTP9+vXj4osvZvHixeTnn+lbVapvqrJkc1U/MSil/EPQX/GD43r9af3aMvWzVDbtPQr4x5Xr/v37mTZtGnPnziUryz6l0qJFC5566inuuusuwsLCfTbDVpd0UCpwhPwOXIXr9mSTmZ3PgeOniYqwMHLxJvZlZNGjVT0S/t6KMItUOZi5IzAeOnSI6dOnM3v2bE6etF/4rWnTpjz55JMMHDiIk3miAVgpVaaQD/xwJkddt3o1Rve4gAZnRRERZiE3v4AhC/5ru3p+a2hnqkeFk5tXUG5QdbTBy/b0k267Ej9y5AgvvfQSs2bNKrX7V6NGjRg3bhz33XcfUVFRlf6dKKWCV8jm+IsrXtr55IpfuG76f7jrtR9JP5FtWwO/bvVqHDx+mj5zviu3VNJhTfuhE8xYm+q2csc6deowYcIE9uzZQ1JSkt2uX/v27ePhhx+madOmTJ8+nVOnTpVzJKWUOiOkAn9ZpZ1PLN/C8KubAzD86ua2RdOK2h0Fb0c17Q+8vYG+HRvb3c8d5Y5nnXUWCQkJ7Nmzh6lTp1KvXj1b24EDBxg9ejRNmzZl8uTJnDhxokrPpZQKfiEV+OvERtL07FiHtehx1vV74qIjnKpVL6umvWSFizvXuqlevTpPPPEEu3fvZubMmZx77rm2tvT0dMaPH0+TJk2YMGECGRmltjhQSikgxAJ/eaWdmTmFgT0zJ9+pUsmiksoOjeOYN7AjS4ZdxoIhl9IwLsrj5Y4xMTGMGjWKXbt2MWfOHLuNajIyMnjmmWeIj48nISGBw4cPu/W5lVKBL6QGd6HspZjr16xGVk4+0ZFhHDyeXeEAbUGBIe3IKQ4eP22/nv7ATtQ/q/BY3qq2ycnJYdGiRTz33HPs3LnTri02NpYRI0YwevRo6tev79F+KKX8i1b1FFNRyaWzJZmHThQOAvvLTNa8vDzee+89Jk2axK+//mrXFhUVxbBhwxg7diwNGzb0eF+c/R3qvAClPEereoqpaPaqs7Nbc/Mcb17uq7VrwsPDufvuu0lJSWHJkiVcfPHFtrbTp08za9YsmjVrxoMPPsiePWXve1/VpZedXcXT16t96hLTKlSFZOB3F2eXTvB2gAkLC6N///5s3ryZlStXcskll9jacnJymDt3Lueffz733nsvO3bsKNXXqgZjZ1fx9OVqn74+6SjlSxr4q8DZjUt8FWAsFgu9e/cmOTmZjz/+mMsuu8zWlpeXxxtvvEHLli0ZOHCgLTXkjmDs7CqevlztU5eYVqEsJAO/u67AK9q4BPwjwBTuC3wT3333HWvXruXKK6+0tRUUFLBo0SJatWrFgAED+HnLz3abwc8b2JEXb2tHTl6+078nZz8J+XKrR11iWoUyjwV+EWkpIpuLfR0XkUdFpLaIrBWR7dbvtTzVB0fcfQVe0XiAPwUYEeG6667jq6++Yv369XTv3t3WZoxh6dKlXH9FF06sfp6mHLTbvGbA/B+c/j05u4qnL1f71P2FVSjzSlWPiIQBfwBdgIeAv4wxk0XkSaCWMWZceY93Z1WPt9eU9/c17L///nsmTpzIJ598UqqtVssuVLv0Nqo1vBBwrd/+XtVTVlmvN1Y4VcpbyqrqCffS83cHdhpj9ojILcDV1tsXAuuBcgO/Ozm6Aq9bvRo5efn8kZHp9uBTdFVbMsD4yxr2Xbt25eOPP2bDhg1MnDiRDz74wNaWkfojpP5IVJP2nNXtdvbRxulPKkWfhNx1P3crnqbTUlIVarx1xf8GsNEY8y8ROWqMiSvWlmGMKZXuEZFhwDCA8847r2N55YeuKHkF3qFxHGNvbGk/CcvNV36evKp197G3bNnCpEmTWLZsWan9f2s2bcvrM5+n7z96IuK+ncicpTX/SrnGZxO4RCQS+BNobYw56GzgL86TM3cXDLmUxFUpfpuKKY8n0xX/+99Wxj8zgdUrloEpsGu77LLLSEhI4KabbnJ4AvBEvzQ1o5TrfDmBqyeFV/sHrT8fFJFzrJ06BzjkhT7YlKzEaV7P8aJt/lzdUVSVtO9oJgeOnbYtKe3OiqHWrVvxwdJ3+WHjL9x+92DCw89kBX/44Qd69epFp06dWLFiBQUF9icGT1Qy+UN1lFLBwhuB/w7g3WI/fwgMtv57MLDKC32wU7wSJzoiPKCqO4pXJV05dT2Jq1IYc0NLOjQu/BDlzpOWxSJ0ad+Kd99+kx07dvDggw8SGXlmbGLjxo307duXdu3asWTJEtu+wJ6oZPKn6iilAp1HA7+IxADXAyuK3TwZuF5EtlvbJnuyDxUJtA3Ei1/5dmgcR2KvVlQLtzC1X1s6NI5zy0nL0TyHJk2aMGfOHHbt2sWoUaPsdv1KSUnh9ttvp3Xr1rz11ltYMG4/mWr5pVLuE5KLtJUUSIOGf2Rk0m3KOjo0jmPMDS1tm8oUbSBfv2YU8XViPZ5LP3jwIC+++CJz5swptftXs2bNuGfEY3xyugV/nsjTHL9SPqKrcxJYAd6RggLDvoxM7nztRxJ7tSLpo62lBqVXjLicejUqvwevq/MODh8+zMyZM3n55Zc5fvy4XVujxo0ZMWo0Awffw7m1qx6gA/39U8rbQn51zmBYlOvIqRwmfryVKX3bUic20mHOOzevoIxHO8fVXPrZZ5/NxIkT2bNnD//85z+pVetMgda+vXt5asyjdGl3ES+/PIvMzMwq9c3ZVVOVUuULmcAfDFUhOXn5fL71EC+sSeWs6AiP5Lwrm0uPi4vj//7v/0hLS+P555+nbt26trY///yTRx99lKZNmzJt2jROnjxZpT4qpaomZAJ/MFSFFAXlTXuPMnb5Fqb0bev2QemqDnbXrFmTJ598kt27dzN9+nQaNGhgazt06BBjx44lPj6eSZMmcezYsSr1VSlVOSGT4y8vd10nNjIgcsclBzh7tKpHwt9bEWYRIsIthFvELVs+ujOXfvr0aV5//XWmTJnC3r177dri4uJ45JFHGDVqFLVr167U8ZVSZQv5wd2yqkJa1K3O9vSTAVMt4igoA35f8ZKTk8PChQt5/vnn2b17t11b9erVGTlyJI8//rhdiijY6WC18rSQD/zg+D/akVM5fr16pjP8fQXQ4nJzc1m8eDGTJk1i+/btdm0xMTEMHz6cMWPGcM455/ioh96h5anKG0K+qgccV4UEQ+4/kF5DREQEgwcPZtu2bSxevJhWrVrZ2jIzM5k+fTpNmzZl5MiRpVJDwSQYig1U4AqpwO9IMMwIDcTXEBYWxh133MEvv/zC8uXLadeuna0tOzub2bNn07x5c4YNG1YqNRQMAulkrYJPyAf+QFuywZFAfg0Wi4W+ffuyadMmPvzwQy699FJbW25uLq+++iotWrRgyJAh/Pbbbz7sqXsF4slaBY+QyvGXJRgG2YLhNUDhFpCff/45SUlJfPvtt3ZtFouFAQMG8PTTT9O6dWsf9dA9NMevvEEHd4OctwK/t57HGMNXX31FUlISX375pV2biNCnTx8SEhJo376925/bW4LlZK38lwb+IOatq0dfXaV+++23TJw4kc8++6xU2z/+8Q8SExPtUkRKqUJa1RPEvFUh4qtKlG7duvHpp5/y008/cfPNN9u1rV69ms6dO3PjjTeWSg0ppRzTwB8EXK0QcbTevieex90uvfRSVq1axebNm+nXr5/dto9r1qzhiiuu4Nprr2XdunWl9gtWSp2hgT8IuFIhUpVVSv2lEqVdu3YsW7aMlJQU7rzzTiyWM3/G69at49prr+Vvf/sbn332mZ4AlHJAA38QcKWcsyrpGn8rG23VqhXvvPMO27ZtY8iQIYSFnTkBffvtt/Ts2ZMuXbrw4Ycf6glAqWI8OrgrInHAa0AbwABDgRuA+4F0692eMsZ8Ut5xdHC3Ys5WiBTt4FXSt+OuoWGtGLc9jy/s3r2byZMns2DBAnJzc+3a2rVrR0JCAn369LH7hKBUMPPV4O5LwGfGmAuBdsA26+0zjDHtrV/lBn3lHGc3KalqusafN0Np2rQp8+bNY+fOnYwcOZJq1c6sU/Tzzz9z2223cfHFF7N48WLbxvBKhSKPBX4RqQlcCbwOYIzJMcYc9dTzKef4W7rGExo3bszLL7/M7t27GT16NDExZz7JbN26lbvuuouLLrqIN998s9QnA6VCgcdSPSLSHpgPbKXwan8DMAp4AhgCHAeSgdHGmAwHjx8GDAM477zzOu7Zs8cj/Qx2ZS3j7K/pGk9IT09nxowZ/Otf/+LEiRN2bfHx8YwfP57BgwfbfUJQKhh4fQKXiHQCfgC6GWN+FJGXKAz2/wIOU5jzTwLOMcYMLe9YmuOvHF0WwN5ff/3FrFmzmDlzZqndvxo1asS4ceO47777iIqq/Gb13uTP4y3KP1Qpxy8iF4jIFyKSYv25rYgkVPCwfcA+Y8yP1p+XA5cYYw4aY/KNMQXAq0Bn51+GckWgLv1b2XkGFalduzbPPvsse/bsYdKkSdSpU8fWtm/fPh5++GGaNm3K9OnTOXXqlFue01OqUparlLM5/leB8UAugDFmC3B7eQ8wxhwA9opIS+tN3YGtIlJ8h41bgRSXeqyc5usJV5XhjYB21lln8dRTT5GWlsa0adOoX7++re3AgQOMHj2a+Ph4Jk+eXCo15C8C9aSu/IOzgT/GGPNTidvynHjcw8A7IrIFaA88B0wVkV+st10DPOZ0b5VL/GXClSu8GdCqV6/OmDFj2L17Ny+99BLnnnuurfkdXvsAAB0LSURBVO3w4cOMHz+eJk2aMGHCBI4e9a+6hEA8qSv/4WzgPywizSnMyyMi/YD9FT3IGLPZGNPJGNPWGNPbGJNhjBlojLnYetvNxpgKj6MqJxAreHwR0KKjo3nkkUfYuXMnc+bM4bzzzrO1ZWRk8Mwzz9CkSROefvppDh8+7LF+lFReyisQT+rKfzg1uCsizSis0LkcyAB2A3cZY7xSaqODu5UXaAOA/rB/cE5ODosWLeK5555j586ddm2xsbE8+OCDjB49mgYNGlTpecp7byoamNeBe+WMSlX1iMgoY8xLItLNGPOtiMQCFmOMVxOfGvgLBVoQrwx/Cmh5eXm89957TJo0iV9//dWuLSoqimHDhjF27FgaNmzo8rErep3OnAAr+nsIhb8XVb7KBv7Nxpj2IrLRGHOJR3tYDg387g2I/h4Q/K1/+fn5rFixgokTJ7Jlyxa7tsjISIYOHcqTTz5JkyZNnD5mRYHdHUtr+MsJVPlOZcs5t4lIGtBSRLYU+yoanFVe4q5Bz0AoA/TVshBl5dTDwsK47bbb2LRpEx988AEdO3a0PSYnJ4e5c+dy/vnnc++997Jjxw6nnquisYyq5vC16keVp9zAb4y5A7gM2AH8o9hXL+t35SXuGvTUgOCYMydEi8XCLbfcwn//+18++eQTunbtamvLy8vjjTfeoGXLlgwcOJBt27Y5ehqbigJ7WQPztaIjnJrjoFU/qjwVVvUYYw4YY9oZY/aU/PJGB1Uhd1VxaEBwzJUToojQs2dPvv32W/79739z1VVX2doKCgpYtGgRrVu3pn///qVSQ0UqqriyWISW9WuwckQ3vh13DStHdKNF3epsTz/p1Kc1rfpR5Sk38IvIUuv3XzTV41vuKs3UgOBYZU6IIkL37t1Zv349//nPf7j++uttbcYYli1bRrt27ejduzcbNmywe6yjwF4y/14y5ZWRlcuMtakk9mrFkmGXkdirFTPWpjo8OXmrlNdTs6yVZ1U0uHuOMWa/iDgctdJyTu9yx6CnDvo55q4y0h9//JGkpCQ+/vjjUm09e/YkMTHRLkXkioPHstiRfopx72+xvXdT+rbl/Lqx1D8rutT9PT1Irn9L/s/ri7S5kwZ+9/K3qhl/4O4gtnHjRiZOnMjKlStLtXXv3p3ExES7FJEz/jyaRf9535c6OS19oCvnxpUO/J7mD3MuVPkqVdUjIidE5LiDrxMictxz3VWe5G+bqfhDusCZ1IsrLrnkElasWMGWLVsYMGCA3cbwX3zxBVdffTVXXnkla9eudXpbSGOMw3SUry7edLwocFVU1VPDGFPTwVcNY0xNb3VSBS9/Ki/1xAnx4osv5r333mPr1q0MHDjQbl/gr7/+mh49enBp58v48MPVFQZwfxuf8bf+KOfp5qPKxhdX3qFSXnrhhRfy1ltvkZqayr333kt4eLitbUPyT9xyy8107NiRFStWUFBQ4PAY/rb2kr/1RzlPc/wK8N1AXWVnqAbyOEX6iWxuem4Fv655h5Nb1kC+/UK3bdq0ISEhgX79+tl9QoCK1/fx9u8kkN+HUOCrzdZVCf6Qz3bEV1felUkX+FN6qDJy8vJJNzWp0+NBGj7wOjU63YKEnxkMTUlJ4fbbb6d169a8/fbb5OWdOTGUlY7y1e/E38aLlHM08HuRPwcsXw3UVSZdEOjpoeInu/Aadajd/X46jVvEyEdHExsba7tfamoqgwYN4sILL+T1118nJ6fs1xfovxPlXRr4vcif/3P6aqCuMtU0gV5N4uhkt/ChHrz04jT27NlDQkICNWueqZ3YuXMn9913Hy1atOCVV17h9OnTpY4Z6L8T5V0a+L3In/9z+nKgztV0QaBXk5R3sqtTpw5JSUns2bOHCRMmUKtWLdvjfv/9d0aMGEHz5s2ZOXMmmZmZQOEnSREJ6N+J8i4d3PUif5/wEigDdaE0Y/TEiRPMmTOHF198kfT0dLu2evXq8fjjj9O9z93M//5PBl/e1G5Wb7D+TpTzfDJzV0TigNeANhRu2zgUSAWWAPFAGtDfGJNR3nGCJfCHUsDytEA5SbmqrNd16tQp5s+fz7Rp09i/33630vCYGsR2vIUrbr6bh3u2o05sJOfGRdOgZlRQ/E5U5fkq8C8EvjbGvCYikUAM8BTwlzFmsog8CdQyxowr7zjBEvgheAOWqjpnLgxOnz7N66+/zpQpU9i7d6/d46VaLDU7/oManW7hh2dvdmrDlpLPr3+b/sFd74XXA7+I1AR+BpqZYk8iIqnA1dbF384B1htjWpZ3rGAK/EqVxZVUYE5ODm+99RYTJz3HnrTddm2WyGgeeughEp58gnr16jn13Ppp1H+4873wRR1/MyAdWCAim0TkNeuevfWNMfsBrN8d/mWKyDARSRaR5JK5TaWCkSuD/5GRkdx33338lprK8y/NJersRra2gpwsXp7xAvHx8Tz22GP8+eefFT63tyrO/HUeiz/xxnvhycAfDlwCvGKM6QCcAp509sHGmPnGmE7GmE5169b1VB+V8huVqVaKjIxg7Mhh7N6eyuxX3+TCi1rZ2rKyspg5cybNmjXjoYce4vfffy/zON6oOPPneSz+xBvvhScD/z5gnzHmR+vPyyk8ERy0pniwfj/kwT4oFTAqW1JrsQgN4mIYcd9g/pfyC++//z7t27e3tWdnZzNnzhzOP/98hg0bxq5du0odwxslsv48j8WfeOO98FjgN8YcAPaKSFH+vjuwFfgQGGy9bTCwylN9UCqQuGNpaIvFQp8+fdi4cSOrV6+mc+fOtrbc3FxeffVVLrjgAoYMGcJvv/1ma/PGPA5/nsfiT7zxXni6qqc9heWckcAu4B4KTzZLgfOA34HbjDF/lXccHdxVyp6zVR/GGNauXUtSUhLffPONXZvFYqF///4kJCTQunVrj1f1+Ps8Fn8SsFU97qSBX6kzKlP1YYzhq6++IikpiS+//LJUe58+fUhISKBDhw5+1W9VNRr4lbIK9Hr1ylw5F3/Nmzf8xMsvTGHNms9K3a9Xr14kJibapYjcyV2/+0B/D71Fl2VWiuCoLHE1V17yNT+/oYAZC5bwww8/cvPNN9vd96OPPqJLly7ccMMNpVJDVeXOoB/o76GvaeBXISUYKktcrfoo6zU3a9WOVatWsWnTJvr162e3L/Dnn3/O3/72N6655hq+/PLLKu/r685gHQzvoa9p4FchJRgqS1yt+qjoNbdv355ly5aRkpLCnXfeicVyJiysX7+e7t27c8UVV/DZZ59V+gTgzmAdDO+hr2ngVyHFlatlf51l6mrZp7OvuVWrVrzzzjts27aNIUOG2G37+N1339GzZ086d+7MqlWrXD4BuDNYl3w9HRrHsWDIpeQb41fvkz/TwK9CirNXy/6eR3ZlDwNXPyFccMEFLFiwgO3bt/PAAw8QERFha0tOTqZ37962TwllbQxfUlUnJRU/CRsMbw3tTKNa0XRoHMfYG1uSuCqFK6eu97v3yV9pVY8KOc4MMgZbzXlVBlb37dvH1KlTefXVV0vt/nXRRRfx9NNPM2DAAMLDw8t9/sqWcpb12Po1q5GVk8+A+T8EzfvkblrVo5SVM1fLwZZHrsqm6I0aNWLWrFns3r2b0aNHExNzZrnnbdu2cffdd3PRRRexYMECcnNzy3z+ys5KLmt8IN/6YSOY3idv0cCvlAOBvr1jEXeOUzRo0IAXXniBtLQ0xo8fT40aNWxtO3bsYOjQoVxwwQXMmzeP7OzsUo+v7MmnvJOwL94nfx37cYUGfqUc8OUexJXhKBh5apyibt26PPfcc6SlpfHMM88QFxdna0tLS2P48OE0b96cWbNmkZWVVc6RnFNecPf2++TvYz/O0hy/UmUIlNmhZeXA61SPpM+c7zye/z5+/DizZ89m+vTpHD582K6tfv36jBkzhuHDh1O9evVKHb+i8QFvvk+BNvajSzYoFaTKCkaL7+/ClVPXl7r/t+OucXlbRmecOnWKuXPnMm3aNA4ePGjXVqdOHR5//HFGjhxJzZo1XT62v5yE/8jIpNuUdaVu99TvtKp0cFepAOFKDrmgwJCTl8+Lt7Vj3sCOdGhcmHbZl5FFmIhX89+xsbGMHj2a3bt3M2vWLBo2bGhrO3LkCE8//TRNmjTh2WefJSMjw6VjV2Vw2p2CZexHA79SfsSVHHLRfQfM/4EB838g6aOtjLmhJR0ax9GoVjTRkWE+GaeIjo7m4YcfZufOncydO5cmTZrY2o4ePco///lPmjRpwlNPPVUqNeQLrpxoA23spyya6lHKj7iSQy7rvkm3tKHBWVG0rF9YdePrFElubi6LFi3iueeeY8eOHXZtMTExPPjgg4wZM4YGDRp4tV9QufkF/pJ2coamepQKAK7MHyjrvs3rVbcFLn9IkURERHDPPfewbds2Fi1axEUXXWRry8zM5MUXX6Rp06Y88sgj7Nu3z6t9q8waQv7wO60qDfxK+RFXcshl3Tc6Iswvg1F4eDh33XUXKSkpLF26lLZt29raTp8+zcsvv0zz5s0ZPnw4aWlpXulTsE3Uc5ZHA7+IpInILyKyWUSSrbc9KyJ/WG/bLCI3ebIPoSQYJpaEOldyyO7ON3vr78disXDbbbexadMmVq1aRadOZzIROTk5zJs3jxYtWjB06NBSqSF3C5bBWld5es/dNKCTMeZwsdueBU4aY15w9jia46+YbmsXPFzJIbt7cxNf/P0YY1izZg1JSUl89913dm0Wi4U77riDp59+2i5F5C7B/v/GJ3X8Gvi9J9Amlij/4g9/P8YY1q1bR1JSEuvXr7drExH69etHQkKCXYqoqgoKDGlHTrHnSCYxkWFk5uTTpE4M8XVigzrwezrHb4DPRWSDiAwrdvtIEdkiIm+ISC1HDxSRYSKSLCLJ6enpHu5m4AvVXKVyD3/4+xERrr32WtatW8fXX39Njx49bG3GGJYtW0a7du3o3bs3GzZscMtzHjmVw6A3fuKeN//LgPk/cM+b/2XQGz8F/W5eng783YwxlwA9gYdE5ErgFaA50B7YD7zo6IHGmPnGmE7GmE5169b1cDcDX6jmKpV7+NvfzxVXXMGaNWv44Ycf6NWrl11b0bjATTfdxPfff1+l5/GHE54veDTwG2P+tH4/BKwEOhtjDhpj8o0xBcCrQGdP9iFUBMvEEuUb/vr306VLF1avXs3GjRvp06ePXdunn37K5ZdfTvfu3Vm/fn2ltoX0txOet3gsxy8isYDFGHPC+u+1wATgZ2PMfut9HgO6GGNuL+9YmuN3TiBNLFH+JxD+flJSUpg0aRJLliwpFeivuOIKEhMTuf766+02ji+PDu66/wmbUXiVDxAOLDbGTBKRtylM8xggDXig6ERQFg38SqniUlNTee6553jnnXfIz7dPy3Tp0oWEhAT+/ve/O3UCCIQTXmXp6pxKqaCza9cunn/+eRYuXFhq968OHTqQkJBA7969sVhCc66qLtmglAo6zZo149VXX2XHjh2MGDGCyMgzYxKbNm2ib9++tGvXjvfee6/UJ4NQpoFfKRXwzjvvPGbPns3u3bt59NFHiY4+M2CbkpLCHXfcQevWrXnrrbfIy8vzYU/9gwZ+pVTQOPfcc5kxYwZpaWmMGzfObtev1NRUBg8eTMuWLXnttdfIyQnuWv3yaOBXSgWdevXqMXnyZNLS0khMTLTb9WvXrl3cf//9nH/++cyePZvTp0/7sKe+oYFfKRW06tSpw4QJE9izZw9JSUnUrl3b1rZ3715GjhxJs2bNmDFjBpmZmT7sqXdp4FdKBb24uDgSEhJIS0tjypQpFF8NYP/+/Tz++OPEx8czZcoUTpw44cOeeocGfqVUyKhRowZjx44lLS2NGTNmcM4559ja0tPTefLJJ4mPjycpKYmjR4/6sKeepYFfKeU2gbInRExMDI8++ii7du1i9uzZNG7c2Nb2119/8X//9380adKExMREjhw54sOeeoYGfqWUW7iyUby/iIqKYsSIEezYsYNXX32VZs2a2dqOHz/OxIkTiY+PZ9y4cRw6dMiHPXUvDfxKKbeozP61/iIyMpL77ruP1NRUFi5cyAUXXGBrO3nyJFOnTiU+Pp7HHnuMP//804c9dQ8N/EqpSimZ1gmGJY7Dw8MZNGgQW7du5d1336V169a2tqysLGbOnEmzZs146KGH+P33333Y06rRwK9UEPJ0rt1RWie/wATNEsdhYWHcfvvtbNmyhffff5/27dvb2rKzs5kzZw7nn38+999/P7t27fJhTytHA79SQcYbuXZHaZ2JH29l3sCOfremf1VYLBb69OnDxo0bWb16NZ07n9k+JDc3l9dee40LLriAwYMHk5qa6sOeukZX51QqyHhj/9w/MjLpNmVdqdt/HH8tFoslKJc4hsItINeuXUtSUhLffPONXZuI0L9/fxISEmjTpo2PemhPV+dUKkR4I9de1s5VFouFujWq0bBWDHVrVAuqoA+Fwb1Hjx58/fXXrF+/nu7du9vajDEsWbKEiy++mD59+rBp0yYf9rR8GviVCjLe2E7QX7dq9KarrrqKf//733z33Xf07NnTrm3lypVccskl9OrVix9//NFHPSybpnqUCjLe2k4wmHeuqozk5GQmTpzIqlWrSrVdf/31JCYm8re//c2rffLJDlwikgacAPKBPGNMJxGpDSwB4incerG/MSajvONo4FfKNRqUfWfLli1MnDiR5cuXl9oX+KqrriIxMZFrr73W6X2Bq8KXOf5rjDHtiz35k8AXxpgWwBfWn5VSbmSxSFDn2v1Z27ZtWbp0KSkpKdx111122z5+9dVXXHfddXTr1o1PP/201InBW3yR478FWGj990Kgtw/6oJRSHtWqVSsWLVrEr7/+ytChQwkPD7e1ff/999x000107tyZVatWef0E4OnAb4DPRWSDiAyz3lbfGLMfwPq9nqMHisgwEUkWkeT09HQPd1MppTyjRYsWvP7662zfvp3hw4fb7QucnJxM7969ad++PcuWLaOgoMArffJ04O9mjLkE6Ak8JCJXOvtAY8x8Y0wnY0yn4mtnK6VUIIqPj+eVV15h586dPPzww0RFRdnatmzZQv/+/WnTpg3vvPOOx/cF9mjgN8b8af1+CFgJdAYOisg5ANbvwbPknVJKVaBRo0bMmjWL3bt3M2bMGGJiYmxt27Zt4+677+aiiy5iwYIF5ObmeqQPHgv8IhIrIjWK/g30AFKAD4HB1rsNBkrXPinlokBZB16pIg0aNGDatGns2bOHp556iho1atjaduzYwdChQ2nRogVz5851ewrIY+WcItKMwqt8gHBgsTFmkojUAZYC5wG/A7cZY/4q71hazqnK4626daU8KSMjg1mzZjFz5ky73b+uvPJKvvrqq0od0yd1/O6igV+Vxxtr0yjlLcePH2f27NlMnz6dw4cP8/nnn3P99ddX6li6Vo8KWsGwDrxSRWrWrMn48eNJS0tj4cKFXHfddW5/Dg38KuB5Y20apbwtNjaWQYMGeWSGrwZ+FfB0wTClXBNe8V2U8m8Wi9Cyfg1Wjuima9Mo5QQN/CooFK1No5SqmKZ6lFIqxGjgV0qpEKOBXymlQowGfqWUCjEa+JVSKsRo4FdKqRCjgV8ppUKM1vGrgKEbiCvlHhr4VUDQpZeVch9N9aiAcORUji3oQ+Hqm/e/lcyRUzk+7plSgUcDvwoIuvSyUu6jgV8FBF16WSn30cCvvKKqe+Lq0stKuY/HB3dFJAxIBv4wxvQSkTeBq4Bj1rsMMcZs9nQ/lO+4Y2BWl15Wyn28ccU/CthW4rYnjDHtrV8a9IOcuwZmi5Zeblgrhro1qmnQV6qSPBr4RaQR8HfgNU8+j/JvOjCrlH/x9BX/TGAsUFDi9kkiskVEZoiIw90zRGSYiCSLSHJ6erqHu6k8SQdmlfIvHgv8ItILOGSM2VCiaTxwIXApUBsY5+jxxpj5xphOxphOdevW9VQ3lRfowKxS/sWTg7vdgJtF5CYgCqgpIouMMXdb27NFZAEwxoN9UH5AB2aV8i8eu+I3xow3xjQyxsQDtwNfGmPuFpFzAEREgN5Aiqf6oPyHDswq5T98sVbPOyJSFxBgMzDcB31QSqmQ5ZXAb4xZD6y3/vtabzynUkopx3TmrlJKhRgN/EopFWI08CulVIgRY1xbLMsXRCQd2AOcDRz2cXeqQvvvW4Hefwj816D9964mxphSE6ECIvAXEZFkY0wnX/ejsrT/vhXo/YfAfw3af/+gqR6llAoxGviVUirEBFrgn+/rDlSR9t+3Ar3/EPivQfvvBwIqx6+UUqrqAu2KXymlVBVp4FdKqRATEIFfRN4QkUMiEpAreYpIYxFZJyLbROR/IjLK131yhYhEichPIvKztf//9HWfKkNEwkRkk4h85Ou+uEpE0kTkFxHZLCLJvu6Pq0QkTkSWi8iv1v8HXX3dJ2eJSEvr773o67iIPOrrflVFQOT4ReRK4CTwljGmja/74yrrUtTnGGM2ikgNYAPQ2xiz1cddc4p1Ce1YY8xJEYkAvgFGGWN+8HHXXCIijwOdgJrGmF6+7o8rRCQN6GSMCaTJQzYishD42hjzmohEAjHGmKO+7perRCQM+APoYozZ4+v+VFZAXPEbY/4D/OXrflSWMWa/MWaj9d8nKNx8vqFve+U8U+ik9ccI65f/XzEUo/s/+46I1ASuBF4HMMbkBGLQt+oO7AzkoA8BEviDiYjEAx2AH33bE9dY0ySbgUPAWmNMQPWfsvd/DhQG+FxENojIMF93xkXNgHRggTXV9pqIxPq6U5V0O/CurztRVRr4vUhEqgPvA48aY477uj+uMMbkG2PaA42AziISMCm3cvZ/DiTdjDGXAD2Bh6zpz0ARDlwCvGKM6QCcAp70bZdcZ01R3Qws83VfqkoDv5dYc+PvA+8YY1b4uj+VZf2Ivh640cddcUXR/s9pwHvAtSKyyLddco0x5k/r90PASqCzb3vkkn3AvmKfEpdTeCIIND2BjcaYg77uSFVp4PcC6+Do68A2Y8x0X/fHVSJSV0TirP+OBq4DfvVtr5xX1v7PPu6W00Qk1loUgDVF0oMA2qvaGHMA2CsiLa03dQcCorChhDsIgjQP+GbPXZeJyLvA1cDZIrIPeMYY87pve+WSbsBA4BdrnhzgKWPMJz7skyvOARZaKxoswFJjTMCVRAaw+sDKwusHwoHFxpjPfNsllz1M4X7bkcAu4B4f98clIhIDXA884Ou+uENAlHMqpZRyH031KKVUiNHAr5RSIUYDv1JKhRgN/EopFWI08CulVIjRwK9Cmog8Yl0t8h0n7x8vIncW+7mTiMyy/nuIiPzL+u/hIjKo2O3neqL/SlVGQNTxK+VBI4CexpjdFd1RRMKBeOBOYDGAMSYZKLVMsjFmbrEfh1A44erPqndXqarTwK9ClojMpXABsQ9F5D2gOXAxhf8vnjXGrBKRIRSu6hkFxAIxwEXWiXgLgU3AmJLLPIvIsxQuJZ5G4VLQ74hIFvA0cJ8x5lbr/a4HHjTG9PHsq1XqDE31qJBljBlO4VX4NRQG9S+NMZdaf55WbAXJrsBgY8y1FC4u9rUxpr0xZoYTz7Gcwk8Ed1kXufuEwhNHXetd7gEWuPN1KVURDfxKFeoBPGm9kl9P4RX+eda2tcYYt+wHYQqnyr8N3G1d/6gr8Kk7jq2UszTVo1QhAfoaY1LtbhTpQuEywu60AFgNnAaWGWPy3Hx8pcqlV/xKFVoDPGxdSRUR6VDG/U4ANVw8tt1jrEss/wkkAG+63FOlqkgDv1KFkijcUnKLiKRYf3ZkC5Bn3Xj+MSeP/SYw17pRd7T1tneAvYGy77IKLro6p1I+YK333xRgy4urIKGBXykvE5ENFI4bXG+MyfZ1f1To0cCvlFIhRnP8SikVYjTwK6VUiNHAr5RSIUYDv1JKhRgN/EopFWL+H7BFJMjGKm+YAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.linear_model import LinearRegression\n", "\n", "# Create the regressor: reg\n", "reg = LinearRegression()\n", "\n", "# Create th prediction space\n", "prediction_space = np.linspace(min(X_fertility), max(X_fertility)).reshape(-1, 1)\n", "\n", "# Fit the model to the data\n", "reg.fit(X_fertility, y)\n", "\n", "# compute predictions over the prediction space: y_pred\n", "y_pred = reg.predict(prediction_space)\n", "\n", "# Print $R^2$\n", "print(reg.score(X_fertility, y))\n", "\n", "# Plot regression line on scatter plot\n", "sns.scatterplot(x='fertility', y='life', data=df)\n", "plt.plot(prediction_space, y_pred, color='black', linewidth=3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train/test split for regression\n", "Train and test sets are vital to ensure that your supervised learning model is able to generalize well to new data. This was true for classification models, and is equally true for linear regression models.\n", "\n", "In this exercise, you will split the Gapminder dataset into training and testing sets, and then fit and predict a linear regression over all features. In addition to computing the $R^2$ score, you will also compute the Root Mean Squared Error (RMSE), which is another commonly used metric to evaluate regression models." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "R^2: 0.7298987360907494\n", "Root Mean Squared Error: 4.194027914110243\n" ] } ], "source": [ "from sklearn.linear_model import LinearRegression\n", "from sklearn.metrics import mean_squared_error\n", "from sklearn.model_selection import train_test_split\n", "\n", "# Create training and test sets\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)\n", "\n", "# Create the regressor: reg_all\n", "reg_all = LinearRegression()\n", "\n", "# Fit the regressor to the training data\n", "reg_all.fit(X_train, y_train)\n", "\n", "# Predict on the test data: y_pred\n", "y_pred = reg_all.predict(X_test)\n", "\n", "# compute and print R^2 and RMSE\n", "print(\"R^2: {}\".format(reg_all.score(X_test, y_test)))\n", "rmse = np.sqrt(mean_squared_error(y_test, y_pred))\n", "print(\"Root Mean Squared Error: {}\".format(rmse))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cross-validation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Cross-validation motivation\n", " - Model performance is dependent on way the data is split\n", " - Not representative of the model's ability to generalize\n", " - Solution : **Cross-validation**!\n", "- k-fold Cross-validation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 5-fold cross-validation\n", "Cross-validation is a vital step in evaluating a model. It maximizes the amount of data that is used to train the model, as during the course of training, the model is not only trained, but also tested on all of the available data.\n", "\n", "In this exercise, you will practice 5-fold cross validation on the Gapminder data. By default, scikit-learn's ```cross_val_score()``` function uses $R^2$ as the metric of choice for regression. Since you are performing 5-fold cross-validation, the function will return 5 scores. Your job is to compute these 5 scores and then take their average." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0.71001079 0.75007717 0.55271526 0.547501 0.52410561]\n", "Average 5-Fold CV Score: 0.6168819644425119\n" ] } ], "source": [ "from sklearn.model_selection import cross_val_score\n", "\n", "# Create a linear regression object: reg\n", "reg = LinearRegression()\n", "\n", "# Compute 5-fold cross-validation scores: cv_scores\n", "cv_scores = cross_val_score(reg, X, y, cv=5)\n", "\n", "# Print the 5-fold cross-validation scores\n", "print(cv_scores)\n", "\n", "print(\"Average 5-Fold CV Score: {}\".format(np.mean(cv_scores)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### K-Fold CV comparison\n", "Cross validation is essential but do not forget that the more folds you use, the more computationally expensive cross-validation becomes. In this exercise, you will explore this for yourself. Your job is to perform 3-fold cross-validation and then 10-fold cross-validation on the Gapminder dataset.\n", "\n", "In the IPython Shell, you can use ```%timeit``` to see how long each 3-fold CV takes compared to 10-fold CV by executing the following ```cv=3``` and ```cv=10```:\n", "```python\n", "%timeit cross_val_score(reg, X, y, cv = ____)\n", "```" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.98 ms ± 36.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n", "0.6294715754653507\n", "6.27 ms ± 230 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n", "0.5883937741571185\n" ] } ], "source": [ "# Create a linear regression object: reg\n", "reg = LinearRegression()\n", "\n", "# Perform 3-fold CV\n", "%timeit cross_val_score(reg, X, y, cv=3)\n", "cvscores_3 = cross_val_score(reg, X, y, cv=3)\n", "print(np.mean(cvscores_3))\n", "\n", "# Perform 10-fold CV\n", "%timeit cross_val_score(reg, X, y, cv=10)\n", "cvscores_10 = cross_val_score(reg, X, y, cv=10)\n", "print(np.mean(cvscores_10))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Regularized regression\n", "- Why regularize?\n", " - Recall: Linear regression minimizes a loss function\n", " - It chooses a coefficient for each feature variable\n", " - Large coefficient can lead to overfitting\n", " - Penalizing large coefficients : **Regularization**\n", "- Ridge regression\n", " - Loss function = $ \\text{OLS loss function} + \\alpha \\sum^{n}_{i=1}a_i^2 $\n", " - Alpha : Parameter we need to choose (Hyperparameter or $\\lambda$)\n", " - Picking alpha is similar to picking k in k-NN\n", " - Alpha controls model complexity\n", " - Alpha = 0: get back OLS (Can lead to overfitting)\n", " - Very high alpha: Can lead to underfitting\n", "- Lasso regression\n", " - Loss function = $ \\text{OLS loss function} + \\alpha \\sum^{n}_{i=1}|a_i| $\n", " - Can be used to select import features of a dataset\n", " - Shrinks the coefficients of less important features to exactly 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Regularization I: Lasso\n", "In the video, you saw how Lasso selected out the ```'RM'``` feature as being the most important for predicting Boston house prices, while shrinking the coefficients of certain other features to 0. Its ability to perform feature selection in this way becomes even more useful when you are dealing with data involving thousands of features.\n", "\n", "In this exercise, you will fit a lasso regression to the Gapminder data you have been working with and plot the coefficients. Just as with the Boston data, you will find that the coefficients of some features are shrunk to 0, with only the most important ones remaining." ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [], "source": [ "X = df.drop('life', axis='columns').values\n", "y = df['life'].values" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[-0. -0. -0. 0. 0. 0.\n", " -0. -0.07087587]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEmCAYAAABiaec2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZhcdZ3v8fcn6SyEJN0daEJIdwWQsMlWbQSXURFIQEeFGdyVJyqIOFcdBzdcEAfUwXXGueOGihOdEUUvDhlHxZAr13EZJCQEI4thT0ggIQkJCdnzvX/8TpFKrE53uqrrVFV/Xs9TT/U5dbrq29VV53t+uyICMzOzEXkHYGZmjcEJwczMACcEMzPLOCGYmRnghGBmZhknBDMzA6CtFk8i6RzgS8BI4JsRcfVej48BvgM8B1gDvC4iHsoe+zBwIbATeE9E3NTf6x188MFx+OGH1yJ0M7Nh4/bbb38iIrr6erzqhCBpJPBlYCawHLhN0tyIuKvssAuBdRFxlKTXA58BXifpeOD1wLOBw4CbJR0dETv39ZqHH344CxYsqDZ0M7NhRdLD+3q8FlVGpwL3RcQDEbEN+D5w7l7HnAvMyX7+EXCmJGX7vx8RWyPiQeC+7PnMzKzOalFlNBVYVra9HDitr2MiYoek9cBB2f7/2et3pw4miH/5v0u5+e5Vg/lVM7M/c/oxXbz3rKPzDqOuapEQVGHf3vNh9HXMQH43PYF0MXAxQKFQ+LPHx44aycQDRu0zUDOzgXjoiU1841cP8O4zpjNyRKXTVGuqRUJYDvSUbXcDK/o4ZrmkNqAdWDvA3wUgIq4BrgGYMWPGnyWNi150JBe96MhB/glmZrvdsHA5l16/mKWrnuLYQyfmHU7d1KIN4TZguqQjJI0mNRLP3euYucDs7OdXA/830qx6c4HXSxoj6QhgOvD7GsRkZjZovYVOABY98mTOkdRX1QkhInYA7wJuAu4Gro+IP0q6UtKrssO+BRwk6T7gUuCy7Hf/CFwP3AX8HPhf/fUwMjMbatMOGkfnuFEsfHhd3qHUVU3GIUTET4Gf7rXv42U/bwFe08fvfgr4VC3iMDOrBUkUC50sWuYSgpnZsNdb6OC+VRtZv3l73qHUjROCmVkFxawd4Y5hVEpwQjAzq+Ck7nYkWPTI8GlHcEIwM6tgwthRHDN5wrDqaeSEYGbWh2Khg0WPrGPXruGx9rwTgplZH4o9nWzYsoMHntiUdyh14YRgZtaHYqEDGD7tCE4IZmZ9eFbXeCaMbWPhMGlHcEIwM+vDiBHilJ4OlxDMzCyNR/jT40+xceuOvEMZck4IZmb70FvoYFfAnctbv9rICcHMbB9O6Sk1LDshmJkNax3jRnNk14HDoh3BCcHMrB+9hU4WPfIkaRmX1uWEYGbWj2KhgzWbtvHI2qfzDmVIOSGYmfWj2DM8VlBzQjAz68cxh05g3OiRLd+O4IRgZtaPkSPEyd0dLT9i2QnBzGwAioUO7l65gc3bWnfZdycEM7MBKBY62bErWLJifd6hDBknBDOzASjNfLrw4dZtR6gqIUiaJGmepKXZfWcfx83OjlkqaXbZ/k9JWiZpYzVxmJkNtYPHj6EwaVxL9zSqtoRwGTA/IqYD87PtPUiaBFwBnAacClxRljj+M9tnZtbwioUOFj6yrmUHqFWbEM4F5mQ/zwHOq3DM2cC8iFgbEeuAecA5ABHxPxGxssoYzMzqorfQyaqntrJi/Za8QxkS1SaEyaUTenZ/SIVjpgLLyraXZ/v2i6SLJS2QtGD16tWDCtbMrBqtvoJavwlB0s2SllS4nTvA11CFfftd3oqIayJiRkTM6Orq2t9fNzOr2rGHTmRM24iWbUdo6++AiDirr8ckPS5pSkSslDQFWFXhsOXA6WXb3cAt+xmnmVnuRreN4KTu9uFbQujHXKDUa2g2cGOFY24CZknqzBqTZ2X7zMyaTrHQyZJHN7B1R+sNUKs2IVwNzJS0FJiZbSNphqRvAkTEWuAq4LbsdmW2D0mflbQcGCdpuaRPVBmPmdmQKvZ0sG3nLu5asSHvUGqu3yqjfYmINcCZFfYvAC4q274WuLbCcR8EPlhNDGZm9dQ7bffMp8VCxaFXTcsjlc3M9sPkiWM5rH0sC1uwHcEJwcxsPxWzFdRajROCmdl+KhY6ePTJzaza0FoD1JwQzMz2U6ntoNXWR3BCMDPbT88+bCKjRopFy1qrHcEJwcxsP40dNZLjD2tvuXYEJwQzs0HoLXRw5/In2b5zV96h1IwTgpnZIBQLnWzZvot7H3sq71BqxgnBzGwQij2tN/OpE4KZ2SB0dx5A14QxLdXTyAnBzGwQJFHs6XAJwczMUjvCQ2ueZu2mbXmHUhNOCGZmg9SbraB2R4uMR3BCMDMbpBO72xk5Qix8uDXaEZwQzMwGadzoNo49dELLjFh2QjAzq0JvoZPFy9azc9d+LxXfcJwQzMyqUCx0sHHrDpauav4Bak4IZmZVKM182grzGjkhmJlV4fCDxtE5blRLjEdwQjAzq4IkioXOlhixXFVCkDRJ0jxJS7P7iitOS5qdHbNU0uxs3zhJ/yXpHkl/lHR1NbGYmeWl2NPBfas2sn7z9rxDqUq1JYTLgPkRMR2Yn23vQdIk4ArgNOBU4IqyxPH5iDgWKAIvlPSyKuMxM6u7UjvC4mXNXUqoNiGcC8zJfp4DnFfhmLOBeRGxNiLWAfOAcyLi6Yj4JUBEbAMWAt1VxmNmVncn97QjwcImb0eoNiFMjoiVANn9IRWOmQosK9tenu17hqQO4JWkUoaZWVOZMHYURx8yoel7GrX1d4Ckm4FDKzz00QG+hirse2YEh6Q24DrgnyPigX3EcTFwMUChUBjgS5uZ1Uex0MHPljzGrl3BiBGVTnuNr98SQkScFREnVLjdCDwuaQpAdr+qwlMsB3rKtruBFWXb1wBLI+Kf+onjmoiYEREzurq6+gvbzKyuegudrN+8nQee2JR3KINWbZXRXGB29vNs4MYKx9wEzJLUmTUmz8r2IemTQDvw3irjMDPLVbHQ/CuoVZsQrgZmSloKzMy2kTRD0jcBImItcBVwW3a7MiLWSuomVTsdDyyUdIeki6qMx8wsF8/qGs+EsW0sauKeRv22IexLRKwBzqywfwFwUdn2tcC1ex2znMrtC2ZmTWfECHFKT0dTNyx7pLKZWY0UC53c+9gGNm7dkXcog+KEYGZWI8VCB7sC7lzenKUEJwQzsxop9pQalp0QzMyGtY5xozmy68Cm7WnkhGBmVkPFnk4WPfIkEc23gpoTgplZDRULHazZtI1lazfnHcp+c0IwM6uh3mzm02ac6M4Jwcysho6ePJ5xo0c2ZTuCE4KZWQ21jRzBSd3tTTli2QnBzKzGegud3LViA1u278w7lP3ihGBmVmPFQic7dgV/eHR93qHsFycEM7Maa9aZT50QzMxq7ODxYyhMGsfCh5urHcEJwcxsCBQLHSx8ZF1TDVBzQjAzGwLFng5WPbWVleu35B3KgDkhmJkNgWI2QK2ZJrpzQjAzGwLHTZnImLYRTTVi2QnBzGwIjG4bwYlT25uqp5ETgpnZECkWOliyYgNbdzTHADUnBDOzIdJb6GTbjl3ctWJD3qEMiBOCmdkQabaG5aoSgqRJkuZJWprdd/Zx3OzsmKWSZpft/7mkxZL+KOlrkkZWE4+ZWSM5tH0sU9rHNs1Ed9WWEC4D5kfEdGB+tr0HSZOAK4DTgFOBK8oSx2sj4mTgBKALeE2V8ZiZNZTeQicLH26OhuVqE8K5wJzs5znAeRWOORuYFxFrI2IdMA84ByAiShVrbcBooHmG9JmZDUCx0MGjT25m1YbGH6BWbUKYHBErAbL7QyocMxVYVra9PNsHgKSbgFXAU8CP+nohSRdLWiBpwerVq6sM28ysPp6Z6K4Jqo36TQiSbpa0pMLt3AG+hirse6YkEBFnA1OAMcAZfT1JRFwTETMiYkZXV9cAX9rMLF/PPqydUSPVFAPU2vo7ICLO6usxSY9LmhIRKyVNIV3p7205cHrZdjdwy16vsUXSXFIV1LwBxG1m1hTGjhrJ8Ye1N0VPo2qrjOYCpV5Ds4EbKxxzEzBLUmfWmDwLuEnS+CyJIKkNeDlwT5XxmJk1nGJPB3cuf5IdO3flHco+VZsQrgZmSloKzMy2kTRD0jcBImItcBVwW3a7Mtt3IDBX0p3AYlLp4mtVxmNm1nB6p3WyZfsu7nnsqbxD2ad+q4z2JSLWAGdW2L8AuKhs+1rg2r2OeRx4bjWvb2bWDIo9u1dQO2Fqe87R9M0jlc3Mhlh35wEcPH5Mw7cjOCGYmQ0xSRQLHQ3f9dQJwcysDnoLnTz4xCbWbtqWdyh9ckIwM6uD0gC1O5Y17ngEJwQzszo4qbudkSPU0O0ITghmZnUwbnQbxx46oaFHLDshmJnVSbHQweJl69m5qzHn8XRCMDOrk2JPJxu37uC+VRvzDqUiJwQzszrpnZaWgmnUaiMnBDOzOjn8oHF0jBvFIicEM7PhTRLFno6G7WnkhGBmVke9hU6WrtrI+s3b8w7lzzghmJnVUbGQ2hEWN+A0Fk4IZmZ1dFJPOxINWW3khGBmVkcTx45i+iHjG7KnkROCmVmd9RY6uWPZk+xqsAFqTghmZnVWLHSwfvN2HlyzKe9Q9uCEYGZWZ6WG5UZrR3BCMDOrs6O6xjNhTFvDtSM4IZiZ1dmIEeKUQuMNUKsqIUiaJGmepKXZfWcfx83OjlkqaXaFx+dKWlJNLGZmzaTY08G9j21g09YdeYfyjGpLCJcB8yNiOjA/296DpEnAFcBpwKnAFeWJQ9JfA4059Z+Z2RApTutkV8Di5Y1TSqg2IZwLzMl+ngOcV+GYs4F5EbE2ItYB84BzACSNBy4FPlllHGZmTeWU7rSkZiNVG1WbECZHxEqA7P6QCsdMBZaVbS/P9gFcBXwBeLrKOMzMmkrngaM58uADGyohtPV3gKSbgUMrPPTRAb6GKuwLSacAR0XE30k6fABxXAxcDFAoFAb40mZmjatY6OSWe1cREUiVTpX11W8JISLOiogTKtxuBB6XNAUgu19V4SmWAz1l293ACuD5wHMkPQT8Gjha0i37iOOaiJgRETO6uroG+veZmTWsYqGDNZu2sWzt5rxDAaqvMpoLlHoNzQZurHDMTcAsSZ1ZY/Is4KaI+GpEHBYRhwN/AfwpIk6vMh4zs6ZRLGTtCMsaYzxCtQnhamCmpKXAzGwbSTMkfRMgItaS2gpuy25XZvvMzIa1YyZPYNzokSx8uDESQr9tCPsSEWuAMyvsXwBcVLZ9LXDtPp7nIeCEamIxM2s2bSNHcFJ3O4saZG0Ej1Q2M8tRsdDJXSs2sGX7zrxDcUIwM8tTsaeDHbuCPzy6Pu9QnBDMzPK0e+bT/NsRnBDMzHLUNWEMPZMOaIgBak4IZmY5K/Z0OiGYmRn0Fjp4bMMWVjyZ7wA1JwQzs5w1ygpqTghmZjk7bspExrSNyL1h2QnBzCxno9tGcOLU9tyX1HRCMDNrAMVCB0tWbGDrjvwGqDkhmJk1gGKhk207dnH3yqdyi8EJwcysAZRmPs1zojsnBDOzBjCl/QCmtI/NdaI7JwQzswZRLHTk2tPICcHMrEEUezpZvm4zq57aksvrOyGYmTWI3mnZCmo5DVBzQjAzaxDPPqydUSPlhGBmNtyNHTWS46dMzG2AmhOCmVkDKRY6uXP5k+zYuavur+2EYGbWQIqFDrZs38U9j9V/gJoTgplZA+ktzXyaw3iEqhKCpEmS5klamt139nHc7OyYpZJml+2/RdK9ku7IbodUE4+ZWbPr7jyAg8ePYVEOI5arLSFcBsyPiOnA/Gx7D5ImAVcApwGnAlfslTjeFBGnZLdVVcZjZtbUJKUBas1WQgDOBeZkP88BzqtwzNnAvIhYGxHrgHnAOVW+rplZyyoWOnjwiU2s27Strq9bbUKYHBErAbL7SlU+U4FlZdvLs30l386qiy6XpL5eSNLFkhZIWrB69eoqwzYza1y72xHqW23Ub0KQdLOkJRVu5w7wNSqd5CO7f1NEnAi8KLtd0NeTRMQ1ETEjImZ0dXUN8KXNzJrPSd3tjFD9Ryy39XdARJzV12OSHpc0JSJWSpoCVGoDWA6cXrbdDdySPfej2f1Tkr5HamP4zoCjNzNrQeNGt3HsoRPrnhCqrTKaC5R6Dc0GbqxwzE3ALEmdWWPyLOAmSW2SDgaQNAp4BbCkynjMzFpCsdDBHcueZOeu6P/gGqk2IVwNzJS0FJiZbSNphqRvAkTEWuAq4LbsdmW2bwwpMdwJ3AE8CnyjynjMzFpCb6GTjVt3cN+qjXV7zX6rjPYlItYAZ1bYvwC4qGz7WuDavY7ZBDynmtc3M2tVpRXUFj2yjmMOnVCX1/RIZTOzBnTEwQfSMW5UXSe6c0IwM2tAkij2dNS1YdkJwcysQRULnSxdtZH1m7fX5fWcEMzMGlSpHWFxnaaxcEIwM2tQJ/d0oDoOUHNCMDNrUBPHjmL6IePrNoWFE4KZWQMr9nSy6JEniRj6AWpOCGZmDax3WgfrN2/ngSc2DflrOSGYmTWwYmnm0zq0IzghmJk1sKO6xjNhTBuL6jBAzQnBzKyBjRghTu7pYKFLCGZm1lvo4N7HNrBp644hfR0nBDOzBlcsdLIr4M7l64f0dZwQzMwa3Ck9acTyUE9054RgZtbgOg8czZEHHzjkPY2cEMzMmsAphQ7uWLZuSAeoOSGYmTWBYqGTJzZuY9nazUP2Gk4IZmZNoLe0gtoQzmvkhGBm1gSOmTyBA0aNHNJ2BCcEM7Mm0DZyBCd1tw9pT6OqEoKkSZLmSVqa3Xf2cdzs7JilkmaX7R8t6RpJf5J0j6Tzq4nHzKyV9U7r5K4VG9iyfeeQPH+1JYTLgPkRMR2Yn23vQdIk4ArgNOBU4IqyxPFRYFVEHA0cD/y/KuMxM2tZxZ4OduwKljw6NAPUqk0I5wJzsp/nAOdVOOZsYF5ErI2IdcA84JzssbcB/wAQEbsi4okq4zEza1mnlBqWh6gdodqEMDkiVgJk94dUOGYqsKxsezkwVVJHtn2VpIWSfihpcpXxmJm1rEMmjKW784Aha0foNyFIulnSkgq3cwf4GqqwL4A2oBv4TUT0Ar8DPr+POC6WtEDSgtWrVw/wpc3MWktvoTO/EkJEnBURJ1S43Qg8LmkKQHa/qsJTLAd6yra7gRXAGuBp4MfZ/h8CvfuI45qImBERM7q6ugb0x5mZtZpioYPHNmxh5fraD1CrtspoLlDqNTQbuLHCMTcBsyR1Zo3Js4CbIo2//k/g9Oy4M4G7qozHzKyllVZQW/hw7UsJ1SaEq4GZkpYCM7NtJM2Q9E2AiFgLXAXclt2uzPYBfAj4hKQ7gQuA91UZj5lZSzt+ykRGt40YkhXU2qr55YhYQ7qy33v/AuCisu1rgWsrHPcw8OJqYjAzG05Gt43gxKntLFrWeCUEMzOrs2JPB394dD3bduyq6fM6IZiZNZneaZ1s27GLu1ZuqOnzOiGYmTWZ4jMD1GrbjuCEYGbWZKa0H8ChE8eysMbjEZwQzMyaUO+0DpcQzMwMij2dLF+3mVVPbanZczohmJk1oeIQTHTnhGBm1oROmNrOqJFyQjAzG+7GjhrJ8VMm1rQdwQnBzKxJFQud3Ll8PTt21maAmhOCmVmTKhY62Lx9J/c89lRNns8JwcysSfVmM5/Wal4jJwQzsybV3XkAB48fXbN2BCcEM7MmJYlTemq3gpoTgplZE+ud1sGDT2xi3aZtVT+XE4KZWRMr9qR2hDtq0I7ghGBm1sRO6m5nhGBhDdoRnBDMzJrYgWPaOPbQiTVpR3BCMDNrcsVCB3cse5Kdu6Kq53FCMDNrcsVCJxu37uC+VRureh4nBDOzJtdboxXUqkoIkiZJmidpaXbf2cdxs7Njlkqane2bIOmOstsTkv6pmnjMzIajIw4+kPYDRlXdjlBtCeEyYH5ETAfmZ9t7kDQJuAI4DTgVuEJSZ0Q8FRGnlG7Aw8ANVcZjZjbsSKJY6GDRshxLCMC5wJzs5znAeRWOORuYFxFrI2IdMA84p/wASdOBQ4D/rjIeM7NhqdjTydJVG9mwZfugn6PahDA5IlYCZPeHVDhmKrCsbHt5tq/cG4AfRESfTeSSLpa0QNKC1atXVxm2mVlr6Z3WQQQsrmKAWr8JQdLNkpZUuJ07wNdQhX17n/hfD1y3ryeJiGsiYkZEzOjq6hrgS5uZDQ8n93QgVbekZlt/B0TEWX09JulxSVMiYqWkKcCqCoctB04v2+4Gbil7jpOBtoi4faBBm5nZniaOHcVRXeOrGrFcbZXRXGB29vNs4MYKx9wEzJLUmfVCmpXtK3kD/ZQOzMysfy85uouJY0cN+vf7LSH042rgekkXAo8ArwGQNAO4JCIuioi1kq4Cbst+58qIWFv2HK8FXl5lHGZmw97HXnF8Vb+vfbTjNqwZM2bEggUL8g7DzKypSLo9Imb09bhHKpuZGdCkJQRJq0kD2fZ2MPBEncOpBcddX467vhx3fe0r7mkR0Wc3zaZMCH2RtGBfxaFG5bjry3HXl+Our2ridpWRmZkBTghmZpZptYRwTd4BDJLjri/HXV+Ou74GHXdLtSGYmdngtVoJwczMBskJwczMACcEq4KkSjPZmlUkyeebBud/kA2KpBP3tX5FM/AJaugpmSqpKyJ2lfblHZdVNmy/EI3+oZT0akkvzH5uqP+TpCOB30h6U/aFb6j49kXSAZJOAiidoGxIfRr4CvCwpC8DNPuFRKMq/x4O9jvZNF/kakkamd13SOpugg/lIcAZ0Hgnroh4ALgAeCEwpdHi64uko4EfA5+W9ICkc7L9DX1xsD8kjZXUlU01n3csx5CW0H0DcDRwiqQr842qdkonXUkTJb1c0nmSjsnrAqmsBHYp8AlJcyVVWsWyT8MiIUhSROyUNAb4d+CW7M16Tt6x7a3s5PRL4HRJX5d0QJ4x7S37wP8y25yfTXfeDK4G5kfEK4CPAX8F6Yq19L43Y3Iou9h5KXAt8Eng45LG5RpYeo/vjYinI2I58G7gcEmDn7C/gZRdCH0LeAHwZeCiiNhV76RQlpz+EvhL4AbgecBESaMHGs+wSAhlPkj6gB4FLAb+VdJnJBVyjqv8RNQBEBF3A+cBK4CX5BVXJRGxKyI2RMTfAN8Bzpc0HhqveqskW5lvUkR8Ltv1C6Ao6RXZ9jRJBzZByfHPRMTO7McvAP8IHARsjYinJR2VxwWFpAOBe4BVkj4i6RTgUuCOiNguqdq1WHJVdgJ+DtAeER8D7gf+IzvkVfVKyJLGlCWnd5AS7xnA/4mI+4DTgL8tXTjsS0N+eWstuwKcBBwD/Dbbdznwymzfp3IMD9ijXvVCSYslfQD4EHACcLWkPpcyrRdJF0j6rKSjJc2QNJpUP3wi6Yp0RANXH20AviRpTBbnE6QRnX+RPX4d8Io+f7vBZQnvdxFxG1AAPp899H7gpHrHExGbgH8gva8HAB8gVTHOyx7fUe+YaikrBYh0AXeLpE8DD0XEbyRNJf3t7XUK56OSpmc//4i0KuX5EfHObN+7gBFlFw59auosvZ9OAyYCF0naBfx3RDwEnCdpIqSsn8cJTdLYiNgi6WzgDmAhMB14FDgcWAq8TdLSiKg07Xc9YpwEjAEmk06kD5Le02uzmD8CrJH02Ua8yo6IB0kxl5fGfgM8T9IbgNUR8YO84hsMSZ0RUVpAdxkwRtL9wLcj4glJLwaeGxGX1DmuMcDxpM/LUxFxuaQXAWuB12Q/3xARj9UzrlqR9HHgFxHxP6Qq05eTLi7fnJUcLgf+I1trfkjPKZJmAi+IiI9LehXpnPExICQdAbwM6I6I1w3o+Rrwu1szWdtBlG13ABcC00hVMYtJVQeR15Vt1vj3ctI/slQH+bu9jpkMvBfYGBG5lGYkvZV0lfEtSe3AgcCRwKtI7+WbgO9HxBfyiK8v2cnpTFKCPRr41+wqmqwq5TfAscBLSvubhaRfkZau/WBErJB0GqlUeR8g4LnA1yPiOkkjB3KFWIOYJpKqrXqzOKYBf8riehp4PXAKcFXWrtBUsmqXy4FzgVuBy0gloM8CQbpgWgW8JWu31FBeIEk6HfgqqQT8m4i4NPtc/wOp9PsT4JaIuGVAz9fKCaFE0gWkN+cuUgPQ0cDrgNHARyJic46xCXgr6R+4inTy2hgRT+913AeAMyLiZTnEOIZUL3kE6SrvFuDWiNiYPV6Xk81gSPoGMIlUytoFvJ10EfCOiNgo6R3AURHxgRzDHBRJB5M+Ny8Fvg78C6mKcTqpKvTmiPjvOsf0NWArqapqAqk9492kasW3RcSDkp4VEffXM65aU+p6/SFSw+2XIuLarKpoB7AlItbXq8ZB0puAfwXuBD4XEd/P9o/b+zzS73O1akIo/TMkvZd0BX4tqXQwCfhbUjXH4RGxZKiz+D5ifD7pw7NI0ruAZ5OqiH5KSlxvIBUHL5Q0DdiZx1WVpBdmdaPHAecDU4CHSFckv613PAOVNWT+e0Q8u2zfRNLJcyrwamA96XuwJZ8oB6c8CUs6FfgM6Ur17yPiZ3sdW5fPt6Re4MsR8fy99h8IfI5UffShoY5jqFR6HyWdQUoMI0l/+4/r8X5LOgi4knQum0hqJxoFfIL0mf7koL6bEdGyN1Id5n8DvWX7LgRuzDu2LJbzSVdQrwBek+17Pqlh6PukIuk5Occ4A/ghcBVwHKkjwhmkE9AXSFVZ3Xm/l33EfilwZfbzWGBU2WM3kEpckF0YNcuNVHUHMJ7Uw6W0//WkC53/AXpyiOvrwN3lr83ui86XAN8Gxub9/tXg75wBXAKcWLbv7aQaiGPq8f8ntf8+h1Q193XgkOyxcaTelCuBN+7vc7dkL6NS96qI2Ar8mlRnSbbvW8BB2dVubpQGSd0QEWtIfZjPyKqFVkfEq0kn3LdGxM/zjJNU1fIN0hqtnwIuJhVN/x64nVQ1sT236PZtEfBipWkTtkTq7ljqCngbqdcLkX2TmkF29VmqhvgccL2k70h6cUR8PyJOAeYCT9Y5rgNJpfD/DXxF0iclHVr23o4BjogmK4mVaPdYj1eSenAdBiyS9H1J7XJtXf4AAA8cSURBVBHxDWBGRNxb1mlhqHRHxI6IuB1YTbow+LmkD0Qa8/FZ0oXlDfv7xC1XZaTUr/wgUgPn1qxIdzXwA+D3pKvb50YanJRXjO3Al0j/zO8CfwBeRBpQMi7bnhepZ0xeMSoiDdjK7s8Gvgb8H1IR9RfAjcCBEVHXk89AZG0Dt5OK1A8BP4qIxWWP/5ZU99tsPYtKVaEfAbqA64GbSD3T7iJVW/yx/Ng6xfUpUhL6Gam75StJVaA/i4ivSPoJ8J2IuL4e8QwVSb8j9fU/A3gWKTGcR2rYr0uHCklvJ7UXXRQR3832nU7q2jsB+GqkjgT7XXXViglhKvA48B5S0e79pLq1y0nFrA2kL829eTaGSjqT1AvkGNIV978BG0lfpL8ErsuzdCCpEBGPlG3fAPwXaeqH80iNaRtIX4SGGnsg6aOkovT/IvXJv4BUillLOml1k4r7f5lbkFXI2kF+Rurp8nlScr4T+H+kLqfvr3M8HwJOBv6mdHEg6dXAGtKJ8wLgvojIfSxNNZQGsL6R1Bvw1xFxctbN9MfALyPin+oYy0GkQaGHAu+MiN9n+y8hVYW+dlDP22oJAUBSD+lE8FJSdczPgGuyKqTSMXk1JJ9KKvLdoDSwaybpS9NO+mL/B2lEba59tLOrkC8Df03qNfLuiHhV9tgBpGS7MSIW5Rfln5N0KKma8HmRBp+VeoScQWpIPpPUTe/XEbEst0AHqazEdiKpV9p3gVdHxAZJ3wX+JSJurWMPl0OBXwEvjIjV2b73kC7AHiZ1QZ1ImjLkT0Mdz1BTGmHdTZq070tAJ+n9vyh7vK7nlazjxI9IpeFLImJdNRe6LZMQsqumYyPi95KuIVUTPUFqP3gNqavpDyPimzmGiaQiqaroeaQT/zVZtn8lqcRwIPCB0pcrT1lcc0i9tN4VEV/JOaR+SXob8PyIeHvWXrC59AWV1E2ajK+pxhvAHlVFoyNiW9n+T5Ouzp8kNZoP6sqwirjeQkoGb1eao2gUqVrxfaSBiy8BLo2ItfWMq1bK3vejSONubomIbZI+RirdPxv4fER8r55VdHvFKFKHgjmk8Q/fG/RztVBCKJB6EL0WeDwiTs/2twEHkz6YayLi5tyCzGRX2C8jxboN+FpE/DZr6D4lIq7LNcC9SDqBVHJZSOq/v66fX8mNpONJFwPnR8T2bN+4SPP6zAT+Bnhd+Um1mWR19aeRBhz9iNTV9EWkEvG3I+LhOrcdHEfqAFH+fp+WlVKmk0qZrywvnTeLstLYDNLJ9h7SeKZPRMRXs+rpEY1S0lQaLzShVDIejJbpZZTVd99IKp5ul/QFSSdHmjNlPLApz2QgaUR2xQ0pcT0AfJTU0P1BSV8gjUm4Lju+YWbdjIglpMFONwCPS3pjziFVlL1nDwI7gS9n1SrE7sE5l5Dqeps1GbyJ1HvkM8CLSVUWx5G6UX8iSwaqYzIQqcF+7/f71uyQT5KmcGi6ZAB79D57KfCPEXE+qav430haQmqHaohkAKlXZTXJAFqohFCS1RdvIzUonkw64b4MuCwifrmv3x3iuI4ALiKN9j0issE7StNpHEkahLYyIr6YV4wDUYurkKGWlRbfSboQWEvqfvps0pXq8/KMbX+V10krTXA4OiJ+mm2/hfSZWkyq0svly6w0aPISdr/fi9n9fp+aR0zVKtXDZ9/b55PaDb4Su0fnvw84LCLel2ectdb0CaHsH9dDuop9VqQ+waVRhK8EtkXOIySzBuQXkEYgP0ZqlPppViTtIH2Bbo2IHXk1eLcSpTmiZpIav19A6sX1m4j4Q66B7SftOeK+FziLVCdfmp5gAnB0RNye5+emwvv9XdL7vSSPeGpF0m3AJlLHiv8CFsReI4DzajsYCk2fEEqUJvq6njRKbxvwsbIvTelLlVejT+n130mq//0xqRFoO2m079tIX56m7qNttVVWhz2Z1APtcqCHVI+9CZgTdZ6raDgou8icCfxFRFwh6bWkhDySNBHlnEZuSxuspk4IZf+42aSeMG8kzV75HVI96yOkq5bHB9sNqwYxlr7UE4DvAW+PiMey+tZZpDEHYyLihXnEZ41P0oVAITsxjSVVMb6SVBX61shxAGOryko895LGA/1ttu9w0vs+ISI+nV90Q6dpE8JedauXkPpCv4LUrfDvJL2bdLI9LxpguLzSjKtvAc4qi/sA0uyIHRGxupr+w9aastLBXNJMrR8nNYrvyC4wDouIe3MNsAWVXWj+HWltgd8BF0bE49njEyON+2iZqqKSZu5l9P1sAAwR8TXSnOvbgKeVpoY4DfhMpIVn+l06bihlV3UdpEEscyS9BCAiNkfE9sjGHDgZ2N6yk9AsUv31R4APS5oeEU85GdSWdi//OiY7Z/wzcAhpvY8/SiqNw9kIe6yp3DKauYRwHqlnw2jSYhu/zHoYfQbYAhzZaNUwkk4CziHNgbKcNFDunnyjskamPae5PoE0u+zJpFGpt+caXItSWtNhB2lsx9XZGKHppLEIb46IB3INcAg1bUKAZ2YgfBcwG1hCGh25kzRackdErMmrGkbZiNKsNPASoAh8kbQQd5HU5vG7iPi3esdmjW3v3kKlMSllVY2zSJMfNu+Xt8GUtfVdSOrJ9UXSFPQvBtaR3v7N2bEtV1VU0nRVRqVindL00SMi4kuktoJ1pJ4Y7ydNDbEO8qmGyT5cpcFPXyHVAR9H6rGwIiL+i9RjpNQLqmEGoVn+yk78zySC7GQ1Itv+RbZdr0XcW5qk8WWbx5IWnnkRqQT/KKn6+eqy/0dLJgNowoRQ9s94F7BV0rsiYmXWE+AS0om3K+d/2pWSxmYjS39CGs25HvhHSWMkfZm0UMgOaK75+G3oSHqJpPdkVZ9/9rkodZ0u2/XhbHyLVed6UlsBpIkwv0Wqkrss23cpcG+WhFv64q3pEkJJRLwHOB24RNL9ks6IiFsj4q8i4vG9vjh1I+k1QDHr2TSVNCPlD0mLnW8hVR+dHBEr8ojPGloP6YLmQknna/dUJ39WipT0YdJiSk05DUejUJqGvpu0aNZXSSss3gz8XtI/S/oc0BnZxI6tfvHWNAmh9IVQmhPoeEljIuJXEXECaRTqzUqznAK5FutKPZ2+R2rLmESaafXuLFl8grQweqkNxAxJR2XtSZ8hTf/wUtLFzksljSpdnWalhC5Sf/gv5Rlzi1hImp31e8DESBP0fYk0ceAy0pKkb4Dh8X1tukZlSW8g9dT5MXBnqcU/y+S/jIif5jmEP4vlX4HXkWbd/DxpJPJLSROvPRARX84rNms8SnPa/wq4BvhBRNwm6bnAq0iTNT5MWk9gcXb810n127nP3NsKJL2eNJXMXFISmB8Rd+QbVT6aMSFMAv6KtODJfaQMfgJpces35xlbidJ0uceRGruPIzUgz4+ITWXHtGxPBds/SnPt/4i0dOoTpA4R3yF1TT6b9Hm/LiLmS5oCfCQi3p1XvK1GaZGfDtJymOeQVla8j3SBeXeesdVbUyUESceS6uXvJ31x3kHq038o8A/ZlVVDnWiV1iL+BOlK76+Apa1eD2n7Lyv5vo00AG0k6ULit6T2pxERsb7s2D0WybHayZLDLFJb3w8i4hc5h1RXDZ8QtHtiuFeR5le/lVR/+p/Au4GdsXthjoadJTTr3/ydUqxmsGdJMZveZCbwA9LKeS8gLe70vqyjRMN+vptNpbEee20fMxxHgjd0Qij/J0n6IvDzUsaW9G+kRW/ekWeM+6vRSjCWn+wi4XmkXi33AOOAaaS5rb4m6WTSpHb/mWOYLW0AiWFiRGzIJ7r6a8s7gH0pSwYXkgaHzCt7+BLgh5KOiCaa7dHJwOCZ9THeQ1oH40Wkni3HAucBYyXdHxHzJN2ZHe/SQQ1kMwecDPwkIh6oMNYj9nqvPyLp48Oliq6hE0KZRwABn5T0JGkiu+NJsz02TTIwK4k0rclpwKuBi0n11m8nVYv2kObcLz/eyaA2SmM9JktaCNwSEWtgj6QrIIbjWI+GrTKqdEWU1bFeTlpY5ibgGxFxd9ZP23Xz1hQkjSK1DTyWXZF2k6ZcOZM0gdoXXZKsvWysx31K6xqcT1rOdiWp8f7XEbG9dN7JxnrcCLy4NKPAcNCQCUG75yM/BvgAqSSzA/gGaX3cvwdeC3wb+Fw06SLeNjxJ+m724zHABaXGS0mnkj7vM4BzhmOj5lDxWI+BaciRyrF7QrqvkKqHfsjuiev+IiI+DPx1dmuIsQdmAyHpzaSpEt4P3AC8UNJVkl4REb+PiNcA73QyqLmNwAPAZOCNkj5OmgTz70nnluNJpTaysR7bhlsygAYtIQBIOgK4JiJmZtsTSCuOHQe8dzjV61lrkNQG3E66+l8p6VvAiaSqiYuzx17XDN2om5HHevSvIUsIAFlj8WhJl2XbTwE/J2Xyho3bbB+mkea1ukBpacyjgVdHxKdI9dkilR4ANyTXinZPG34daQR4L6mb7y+Ak4CvAmOzY0tTXA+7ZAAN1suorO3gHOAgUnXRRVld3y9Ji8r8NNKymO7Pb00lIu6XNJG0FOZK4KaIeCR7uJ1UWvBnuoZKYz0klcZ63EfqlNKTjfW4lzTW43FwEm6YKqOy1v3JwC2kMQcPAKeSuoo9TGrkuTG/KM1qQ1In8F3StCuvJbWFjYqIy32xUxvZWI/bSGM97mOvsR6k5TDnlZ17hn0VXSMmhL8DNmfZu4NURfRGUle8B8qPzTNes1rIRiP/DOgE2rPxCf5814iksewe67GGNNZDZGM9PCXInhomIQBImkqan3xhRLysbP/3gMUR8ZncgjMbIlm9dSEiHpbUNpz6vQ8Vj/UYnIZqnI20fukbgYmSbpP0lmyG0w5STwyvP2wtJ5KHs5+dDGrjWuCzwK3ZRHXLI+K9QGkanPuzcU5WpqFKCCVKKxO9mbTAzHbgnyLii/lGZWbNIBvrcSHweuCtpGVsjwBujYifZMecExE/zy/KxtSQCaEka3h7CzCbNO31uzxFhZn1xWM9qtNQVUZ7i4h1EfGPwJuAPzoZmFk/PNajCg2dEEoi4o8R8c95x2FmjS0i7ifNTXQAaazHRo/1GLiGrjIyMxssj/XYf04IZtbSPNZj4JwQzKzleazHwDghmJkZ0CSNymZmNvScEMzMDHBCMDOzjBOCmZkBTghmZpZxQjAzMwD+P3wZREuNvLS2AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Import Lasso\n", "from sklearn.linear_model import Lasso\n", "\n", "# Instantiate a lasso regressor: lasso\n", "lasso = Lasso(alpha=0.4, normalize=True)\n", "\n", "# Fit the regressor to the data\n", "lasso.fit(X, y)\n", "\n", "# Compute and print the coefficients\n", "lasso_coef = lasso.coef_\n", "print(lasso_coef)\n", "\n", "# Plot the coefficients\n", "df_columns = df.columns[:-1]\n", "plt.plot(range(len(df_columns)), lasso_coef)\n", "plt.xticks(range(len(df_columns)), df_columns.values, rotation=60)\n", "plt.margins(0.02)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Regularization II: Ridge\n", "Lasso is great for feature selection, but when building regression models, Ridge regression should be your first choice.\n", "\n", "Recall that lasso performs regularization by adding to the loss function a penalty term of the absolute value of each coefficient multiplied by some alpha. This is also known as L1 regularization because the regularization term is the L1 norm of the coefficients. This is not the only way to regularize, however.\n", "\n", "If instead you took the sum of the squared values of the coefficients multiplied by some alpha - like in Ridge regression - you would be computing the L2 norm. In this exercise, you will practice fitting ridge regression models over a range of different alphas, and plot cross-validated R2 scores for each." ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [], "source": [ "def display_plot(cv_scores, cv_scores_std):\n", " fig = plt.figure()\n", " ax = fig.add_subplot(1,1,1)\n", " ax.plot(alpha_space, cv_scores)\n", "\n", " std_error = cv_scores_std / np.sqrt(10)\n", "\n", " ax.fill_between(alpha_space, cv_scores + std_error, cv_scores - std_error, alpha=0.2)\n", " ax.set_ylabel('CV Score +/- Std Error')\n", " ax.set_xlabel('Alpha')\n", " ax.axhline(np.max(cv_scores), linestyle='--', color='.5')\n", " ax.set_xlim([alpha_space[0], alpha_space[-1]])\n", " ax.set_xscale('log')" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEKCAYAAADTgGjXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZRcZ33n//f31tJ7t9RStyRra3mVJS+yLdtgQ2xsDAZsMANJ7Ak/YvCBJL/ADITMxBCTOJzxDMmZCQkTmMRkGIF/BEeYzRgbQ7zxM9ggybJs5FWStdtaWmpJvdfynT+qulXqtbrq3urt8zqqU7ee57n3Pn1Pq75977OZuyMiIlKKYLIrICIi05eCiIiIlExBRERESqYgIiIiJVMQERGRkimIiIhIyeKTXYGwzJ8/39va2ia7GiIi08qmTZsOu3tLqfvPmCDS1tbGxo0bJ7saIiLTipntKmd/Pc4SEZGSKYiIiEjJFERERKRkCiIiIlIyBRERESmZgoiIiJRMQUREREo2Y8aJiMxW7k7WIetO1h138MLPgGfB8cF0J1fGcfL/GFhaKJc7OsMwAwPMctuBGUbuPQggFhiBGbHAiJkRBBbtRZBJoyAiEoHxvtizfmoZ95H3yZ5SpiA/e3K/6cAM4jEjHgQkYkY8FpAIcu/xmFEVD0jGAswUbKYbBRGZVgZW4vTBv5592F/Rg9t+8vMpZbMjp2f95F/lg1/sQDabO2DhF/vgfs4pgWA6fbFXkjuk0k6KDD2jlDGDRCzIBZR47r0qEaMmESOmO5kpa8YEkYw7hzv7RswL+z/1eLf7Y+xYTvaIP8dodRm57EjlvGC7uHMXnvPkl/cIxxu2r49Q/uQxC7/8h6aNVz+Z/tyhP52lP50dlpeMB9QmY1QnYtQkFVimkpkTRDLO6x29k10NEYnAyeCSGkyrTgTUV8epq4pTn4yr3WWSzJggIiKzS28qS2+qn8Mn+jGDmmSMhqo49dVxahIxta9UiIKIiEx77tDdl6G7L8OB433EY0ZTTYKmmgR1Vfqai5KurojMOOmM097ZT3tn/2BAmVOboDapr7yw6YqKyIxWGFAScaO5NklzXZJ4TGOtw6AgIiKzRirtHDjex8ETfTTVJJhXn9TdSZl09URk1nGHju4UHd0papIB8+qqaKpJqIdXCRRERGRW6+nPsre/h9eP9dLSUMW8uqSCyQTooaCICJDJOm8c6+XlAydo7+w7ZeCsjE5BRESkQDrj7O/IBZMjXf0KJuNQEBERGUEq7ew72sMrBzo51p0af4dZSkFERGQM/eksu49089rhLvrSmcmuzpQTaRAxs+vN7GUz22Zmt4+Qv8zMHjOzzWb2nJm9uyDvAjN7ysy2mtnzZlYdZV1FRMbS2Zvm1QOdHDzRq0dcBSILImYWA74CvAtYBdxiZquGFLsDWO/uFwE3A1/N7xsH/j/gD919NXA1hTOviYhMAnc4cKyPbQc76e5PT3Z1poQo70QuA7a5+w537wfuBd43pIwDjfntJmB/fvsdwHPuvgXA3dvdXfeRIjIl9KaybD/Yxb6OHjLZ2X1XEmUQWQzsKfi8N59W6E7gQ2a2F3gQ+GQ+/WzAzexhM3vGzP5zhPUUESnJkc5+th3spKd/9v6NG2UQGWm0ztCQfQuwzt2XAO8G7jGzgNwgyLcAv5d/f7+ZXTvsBGYfN7ONZrbxSPvhcGsvIlKE/nSW7Yc6OdLVP9lVmRRRBpG9wNKCz0s4+bhqwG3AegB3fwqoBubn933C3Q+7eze5u5SLh57A3e9297XuvrZ53vwIfgQRkfG5w76jPew50j24nPJsEWUQ2QCcZWYrzCxJruH8/iFldgPXApjZueSCyCHgYeACM6vNN7JfBbwQYV1FRMrW0Z1i+6FOelOz5/FWZEHE3dPAJ8gFhBfJ9cLaamZfMLP35ot9BviYmW0Bvg3c6jlHgb8lF4ieBZ5x9x9HVVcRkbD0prJsOzh7BijaTOnvfMGai/1bDzw22dUQERnU2ljFgsapPcTNzDa5+9pS99eIdRGRiBw83sf+jp7JrkakFERERCLU3tnPniPdM3aUu4KIiEjEOrpT7J6hPbcUREREKuB4T5qd7V0zLpAoiIiIVEhXX4Ydh7tIZ7KTXZXQKIiIiFRQT3+G12ZQIFEQERGpsN5Ull0zpLFdQUREZBJ092XYe3T6d/9VEBERmSQd3SkOHO+d7GqURUFERGQSHTzex9FpPAOwgoiIyCTb19FDZ9/0XClRQUREZJK5w672rmk5+6+CiIjIFJDNwq727mnX9VdBRERkiuhPZ9nZPr26/iqIiIhMIT39GQ6d6JvsahRNQUREZIo5eKKPnv7p0T6iICIiMsW4w56j02PWXwUREZEpqC+V5Y1pMBAxPtkVCMuxjiP8/MffOSVt8YqzOWPVhaTTKX758A+G7bP8rFUsP3s1fb09/OqRB4bln37uBSw5/Ry6O0+w8YmfDMs/6/xLWLTsdE50HGHzLx4Zlr9yzeW0Ll5GR/tBnnv6iWH5q9deybwFp9F+YD9bN/5iWP4Fb7qKOfNaObhvNy89+6th+RddeS0Nc5p5ffcOXn1+07D8tVddT219A3t3vMyOF58bln/5tTdQVV3Drle2suvVF4blX/HOm4jHE2x/YQv7XntlWP5vvee3AXjl+U28sXvHKXmxeJwr3/l+AF7c/CsO7d99Sn6yuoY3XXsDAL/Z8CRHDr5+Sn5NXQOXXn09AFuefpxj7YdOya9vmsvFb3k7AM88+W90Hjt6Sn7TvBYufNPVAGx4/Cf0dJ04Jb+5dRHnXfoWAJ5+5AH6e0+dfqLltGWce9HlAPzi4e+TSZ/ah3/hstM5+/xLAIb93sHM/9278MprSdQ2snfnDna9uJmsO1kHd8cd5q66EpK1dL3xGj37X2Xg72kzMGDe+VeRrKqh+43tdO3fTiyAIDBigREz0+9e/nfv/u/+KwlzYoEN5p999tlcccUVAKxbt27YtVm9ejWXXnopqVSKb33rW8Py16xZw5o1a+ju7mb9+vXD8idqxgQREZkYd6cvlaU/kyWTcdJZJ5PNvf/4udc5EZwgc+wNGjq6yeTzMu5ks84/fesZjns1S4MOVsdPDDv2PY9uo8uTrIgd4ZxY97D8b/zsFfpIcGbsMGfGOk/JM4Ov/Z9fU1NdxVnBARZlu0nGAhLxgEQsIBkL6O5PU5ucHV9ffeksNckYNn7RSWFjdSUzsxjwDXf/UOWqVJoL1lzs33rgscmuhsik6ktnONadoqMnxbGeFMe6UxzrTdHRneJYTz/HetL591x+KjPy/38D6qri1FfFqauKUZ/frq+KU5OMUZOIUZ2IUZPMvVcnYlTnv+Tj+TuKeMyIBbnPgRlZ98FglC4ISqlMlq6+NJ196fx7hs6+NJ29aY73pjh4vJdDnX3D6jqnJsGK+XWc3lLPGS11nD6/nkVzqglsqn7dlm5ObYKlzbWRHNvMNrn72lL3HzOUu3vGzFrMLOnu03dyF5Fpqi+d4XhPmmM9KY735ALCse58gCjczr96RhnxnIwHzKlJ0FSTYG5tkrZ5dTTmPzdWx2msSdBYPfA5QW1VbEp9GWfdOdad4uCJPg6e6OXgiT72Hu1mx+EufvjsPtL5BujqRMCKeXWcs7CRi5bOYdVpjVQnYpNc+/J1dKdorEnRVJOY7KoMU8z94E7gF2Z2P9A1kOjufxtVpURmmqw73fm/sE/0pjjRl+ZEb3674P14b4rjPbn3Yz0p+tIjj16OBUZTPgg01SRY1FSd265N5INFkjm1CRprcp+n+xdpYMbcuiRz65Kcs7DhlLxUJsueI93sONTF9sOd7DjUxQPP7ecHz+4jETNWLWrkomVzWbN0Divm102p4DgR+472UJeMEY9Nrf5QxQSR/flXADSMU1Zkxslknd5Uhp5Uhu7+DD39A9tpuvsydPWn6e7P0NWXf+9PDz6eGXh192UYq7NmbTJGQ3WchupcIFjWXEtjTZzG6kT+LiFOU22Spnx+XTKGTdMvw7AlYgGnt9Rzeks9b2cBAL2pDC/sP87mPUd5dk8H6365E4CmmgRvWtHM21a2smpR47S6hpmss7+jl2XzonmsVaox20ROKWjWALi7d45beBKoTWR2cc89V09lsqQyA+9Z+tO5z33pDKmM05/O0J9/70tnT75SmVPee9MZelNZelOZ/Cs7GDhGuxsYqiYRozYZo7YqTl3y1HaEuuqT2wPBoiG/XV8Vn3J/Xc40R7r6eXZPB5t2HeXXO9vpTWVZ2FjNNStbeds5rSxsqp7sKhbtjNa6UDsVlNsmMm4QMbPzgHuA5nzSYeDD7r611JNGIewgUnhdfDBt5DJ+Shp4PmWg/OB7QboX7Oj4ybT8cQfyveDzwLEL98/mD54tLOMny3hh18uBcn7y3d3JAtlsPh0nm82dJ5vN7Vu4T9Zz+ZnBbSfj5Mue2oMn4ww2pGb9ZGNqNuuks1kyWchks6QLGlpTmexgw2s6kx0MFOmMn7LdX+YkdYFBdSJGVTwgGQ8GG4pzryDXaJyIURXPBYaafCPy0O3aZJy6ZK6xubAbpkxdPf0ZntrRzqMvHeC5vcdwYNWiRq5Z2cpVZ7dM+Ud/tVUxzmipD+14lQgivwT+3N0fy3++Gviv7n5FqSeNQvWis3zJR/4eYNhjg9F+xsLkqT8udPoJLPcsOxZYrodOULA9kF7Ym2fgFQuIBUYiZsSDgHjMBnv9DL7newIlAiM5sJ0vVzXQFTQeUBWPkYjZ4PtA4NBf/gJw6EQfj79ykMdeOsieoz00Vsd535rFvOf8RdRVTd0uxMvm1YbWyF6JILLF3S8cL22yLTpztX/ov/3LqPmFzz5P+XvRhm8Ovo/wvHQgyYYk2ClpYPlPA4OrCotbrkBun4KdrLC8FX62wXLBsP2NYDAvV84K983nG/n3fF4Q5MoG+WOYQSx/8sAs/zp5zCAwAowgyJULglPLxPKBYGB74Fwi04G78+IbJ/jOxj1s3HWUumSMGy44jRsvPG1K9oiqSgSc1Vofyv+xSgSR7wPPkHukBfAhYK2731TqSaOgNhERCcP2Q52s37iHp7a3U5UIuH71It5/0WKa65KTXbVTLJ5bE0qdKhFE5gJ/Bbwln/Rz4K/c/ejoe1WegoiIhGn3kW6+s2kPP3/lEPEg4HfWLuHfXbyExBR5FBqPGecsaCAosy0u0iCSH7H+RXf/T6WeoFIUREQkCq8f6+GbT+3iyW2HWTK3hj+++kzOW9w02dUCYEFTFa0N5fUsKzeIjBlS3T0DXFLqwc3sejN72cy2mdntI+QvM7PHzGyzmT1nZu8eIb/TzP601DqIiJRjUVMNf3b9Sv7yxlX0p7N89vvP8+VHXuV4T2qyq8ahE32TvpxuMd0PNudHq3+HU0esf2+snfJ3MV8BrgP2AhvM7H53L5wu9g5gvbv/LzNbBTwItBXkfwl4qJgfREQkSmuXN/OVf9/EvRv28INn9/Gr19q57S0reNs5rZPWiSSbzS1gddqcmkk5PxS3nkgz0A5cA9yYf91QxH6XAdvcfUd+3q17gfcNKeNAY367idzIeADM7CZgBzClxqOIyOxVnYhx6xVt/N3vrOG0OTV86d9e5fM//A1HuyZvasEjXf30pSdvFcQxg0j+buI5d//IkNdHizj2YmBPwee9+bRCdwIfMrO95O5CPpk/bx3wZ+Qa9Meq38fNbKOZbTzSfriIKomIlK9tfh1//YEL+H+vPoMX3zjBp9c/y8tvDJ8SvxLc4eDxyVuTvZg2kfeWeOyR7u+GtuLfAqxz9yXAu4F7zCwgFzy+NN4UK+5+t7uvdfe1zfPml1hNEZGJC8x413mL+O8fvIBYYNz+vef42QtvTEpdOrpTk7YmezGPs35pZv9gZm81s4sHXkXstxdYWvB5CQWPq/JuA9YDuPtTQDUwH7gc+Bsz2wl8CvicmX2iiHOKiFTUivn1fOl31nDe4ia+/Og2vvr4NlKT0Nh98MTkLKVbTMP6wPQmXyhIc3JtJGPZAJxlZiuAfcDNwL8fUmY3cC2wzszOJRdEDrn7WwcKmNmdQKe7/0MRdRURqbjGmgR33riabz61k+9t3seu9m5uv34lcys4QPF4T5r+dJZkvLLjWMYNIu7+tlIO7O7p/N3Dw0AM+Lq7bzWzLwAb3f1+4DPA18zs0+QC061e7LTCIiJTSCwwPnLlCs5oqefvH32VT61/ls+969xh659E6UhXf8VnJB51sKGZ/Z27fyq//R/d/e8L8ta5+62VqWJxNNhQRKaK1w538l9+/CLHelLceePqig1OjAXGyoUTG8Ue5WDD3yrY/v0heReUekIRkZluxfx6/vtvX0hLQxV/9cBWXnz9eEXOm8k6HRUeBDlWELFRtkVEZBxza5PcddP5NNcm+cv7t/LKgcp0AT7SVdnuvmMFkcDM5prZvILtZjNrJtfGISIiY2iuS3LX+8+nqSbBX/zwN2w7GP3CsD39Wbr60pGfZ8BYQaQJ2ARsJDeq/Jn8501orXURkaLMr6/irpvOo64qzl/88De8djj6QNLeWbkR9KMGEXdvc/fT3X3FCK/TK1ZDEZFprrWxmrvefz5ViYA7fvAbdrV3jb9TGY73puhPV2asytSYGF9EZIZb2FjNXTedTzyWCyR7jnRHdi53ONpdmbsRBRERkQo5bU4Nd910HmZw54+20tkbXdtFe2c/2Wz0w+4UREREKmjJ3Fo+9+5zOdLVz9/+28tkIxpfnck6xyrQ3XfUIDLQE2u0V+Q1ExGZoVYubOSjV65gw86jfO+ZfZGdp70C3X3HmvZkE7mpSAxYBhzNb88hN+fVishrJyIyQ91wwSJeeP049zy9k3MW1HP+kjmhn2Ogu29dVTHTJJZmrN5ZA72wHgZudPf57j6P3IJUY65qKCIiYzMzPnnNmSxqquFvfvoyRyJa2Cqq4w4opk3kUnd/cOCDuz8EXBVdlUREZofaZJzPvmsl3f0Z/ubhl8hE0BB+rCcV6dT0xQSRw2Z2h5m1mdlyM/tzcsvliohImZbPq+OPrz6TrfuPc8/Tu0I/ftTdfYsJIrcALcD3868WcmuDiIhICK5Z2cr1qxfy3Wf28qvXwv8b/XiEvbSKCSLXuvt/dPeL3P3i/PTwb4+sRiIis9DH3no6Z7TU8aWfvcIbx8JdpbCnP0tfOprlc4sJIp8tMk1EREqUjAfc/q5zAfifj71K2OvzRTVmZNR+X2b2LuDdwGIz+3JBViNQuSkiRURmiYWN1fw/b27jH5/YzpPbDvPWs1pCO/bxnhStDeGvejjWnch+cjP49nJy9t5NwP3AO0OviYiIcP3qhZzeUsc/P/ka3f3h/b0e1SOtscaJbHH3bwBn5t//BXgOeNLdj4ZeExERIRYYf3TVGRzp6udfN+wJ9dhRPNIaa9qTfzSz1e6eMrMmYAvwTWCzmd0Sek1ERATITYty3aoF/HDLfnaHONtvFL20xnqc9VZ335rf/gjwirufD1wC/OfQayIiIoN+/81tVCcC/umJ7aE1svf0Z+lNhftIa6wgUjg65TrgBwDu/kaoNRARkWGaahJ8+E1tPLfvGE9uOxzaccO+GxlrVq4OM7sB2AdcCdwGYGZxoCbUWojIjBQLjETMiMcCYmYEQS4tFhgxy70HgWHk5pIC8tuQS4WsOxl3PJvbzr1yU52nMtn8K7cd0azqk+adqxfy0xfe4J+ffI1Lls+lNln+RIrHelK0NobXS2usGv0B8GVgIfCpgjuQa4Efh1YDEZl2zCAeMxKxgGQsIBELSMSMRDwgEQSDwWMgMFTKQFDpS2XpTmXo6c/Qm8pM2+CSa2Q/kz+9bwv3btjDR68sf/L03lTukVZ1IhZCDccIIu7+CnD9COkPk5vZV0RmsFhgJOMBVfGAZDwXLJLxkwGj0gGiGIl8QKtNwtx8mrvTm8rSk8rQ3Z+msy9NKj19oso5Cxt4x6oF3L9lP28/dwHLmmvLPubxnlT0QUREZr5E3AaDQzIeUBWLUZXIBYwgmHpBohRmRk0yRk0yRnNdEoCe/gzHe1Mc70nRm4puhtuwfPjNbfxyezv/9MR2/stN55UdwMN8pKUgIjLDDQSKqkRsMGBUxWdWoJiogaCyoLGavnSG4z1pjvem6O6LZn6pcjXVJPjwm5fz1ce388vt7Vx55vyyjhfmIy0FEZFpzoxTHjclCx8/zeJAUayqeIyWhhgtDVX09Gc43NnHsZ7UlGtHeceqhfzw2f2s37SHK86YV/bdSFiPtCYURMzsAXe/oeyzisiEJOInG7GnS/vEdFSTjLG0uZYF6SztXX0c6eonO0WedsUC4wMXL+bLj25j8+4OLl4+d/ydxhDWI62J3oksLvuMInKKIDjZIJyI2cneTvGTnxUkKisZD1jUVENrQzVHu/s53Nk3JRrjrz6nlW/9ajf3PbO37CAy8EirXBMNIpsnUtjMrgf+HogB/+zuXxySvwz4BjAnX+Z2d3/QzK4DvggkyQ16/E/u/ugE6yoyqcxywSEeMxJB7r1weyBwxPS4acqKBcb8+iqaa5Mc6uzj0Im+SX3MlYgF3HTRYv73k6/x0hvHWbmwsazjhTGX1lhTwd8NPAT8m7ufAHD3jxZ7YDOLAV8hN9p9L7DBzO539xcKit0BrHf3/2Vmq4AHgTbgMHCju+83s/PIdSnWXZBMqoGxEQOD5OIDgSE/eC4eC4gHlk9TcJhJgsBY0FhNU02CfR09k9oA/85VC1m/YQ/3bdrLHe9ZVdaxIg0iwNfJjRP5EzPrB34K/MTdtxR57MuAbe6+A8DM7gXeBxQGESe3PglAE7np53H3wjuerUC1mVW5e1+R5xYZxgyCfACIBSe3T6adul04qjqeH1kts1t1IsYZLfUc7ern9WO9ZLKVvy2pSca48cLT+Jdf72ZXexfL59WVfKy+ELo3jzXY8GngaeBOM5sHvAP4jJmdT+6x1k/cff0Yx14MFM5jvBe4fEiZO4GfmtkngTpGXnb3A8DmkQKImX0c+DjA4iVLx6iKTGUDj/sDs8Ev+tx7ro+/5fMG0s0Y/MIfKJ975crn8k5NjwVqfJbwzK1L0lAd5/VjvXR0R7d++Wjec/4ivrd5L/c9s5fPXHdOxc9fqKg2EXdvB76df2FmlzDCaPYhRvofOzRs3wKsc/f/YWZvBu4xs/PcPZs/z2rgr8kFsJHqdTdwN8AFay72sL8jxjqejfjjjb7f0LSh+xfmD939ZJ6NkHYytfBL0oaUGzjfsHrY8DmLBsrn5i86ecChaQNf8IPzHBmD+YV5gZ0898AxgsJ99eUu01A8FrC0uZam2hR7jnRXtBdXY02Cd65ayI+e28+HLl/OghDnwpqoohvWzew6d/8ZgLsPrHI4lr1A4e3BEvKPqwrcRj4YuftTZlYNzAcOmtkS4PvAh919+3j1S8YDzlvcVNTPIiISlsbqBGe01LOzvauiPbhuumgxP37+db6/eR9/eNUZFTvvUGNNBT/UX0/w2BuAs8xshZklgZvJLa1baDe5CR0xs3OBauCQmc0hN8njZ939FxM8r4hIRVUnYpzZUk9tVTjzURVjfn0Vb1vZys9eOMDR7v7xd4jIRILIhLh7GvgEuZ5VL5LrhbXVzL5gZu/NF/sM8DEz20LuUdmtnlt95RPAmcDnzezZ/Ks1qrqKiJQrHgs4fX4dc2oTFTvnBy5aQiqT5Udbhj7kqRwba8UsM/s/5NoxDLiRgjuJiXT3rYS1a9f6xo0bJ7saIiIcOtHHG8d6K3KuLz70Ipv3dPD137+UuqqJz2R1wdI5m9x9bannH++M6wq230JuYKCIiIyhpaGKZDxgz5HuyAcnfvCSpfxiezsP/eYNPnjJkmhPNoIxg4i7PzGwbWYnCj+LiMjommoSBPNq2dUebSA5s7Wei5bO4Ydb9nHjhYuoileuXQYm1iYyeS03IiLTUEN1giVzo19N/AMXL6GjO8VT29sjP9dQRQcRd39TlBUREZmJ5tQmWdgU7TiO85c00dpQxSMvHYz0PCOJrHeWiIjktDRUMb8hGdnxAzOuWdnKlj0dHO6s7OxQCiIiIhWwqKkm0u6/16xsxYHHKnw3oiAiIlIhS+bWUF8dzYKyi5pqWH1aI4+8dJCxhm6EraggYmbLzezt+e0aM2uItloiIjOPmbGsuZaaZDR/v1+zspV9HT28cqAzkuOPZNyfxMw+BtwH/FM+aQnwgygrJSIyU8UCo21eHfFY+BOPvuXM+STjAY+8dCD0Y4+mmHD4x8CVwHEAd38V0BQkIiIliseCSLr+1ibjXHH6PH7+6iH605WZVriYINLn7oNjRMwszvAp3UVEZAIaqhM014ffY+uala109WX49c4joR97JMUEkSfM7HNATX7t8+8AP4q2WiIiM9+ixmqS8XDbRy5YMod5dUkeebEyj7SKqf3twCHgeeAPyK2DfkeUlRIRmQ2CwFjaHO5jrVhgvO2cVp7ZfZSjXdFPNDJmEDGzGPBNd/+au/+2u38wv63HWSIiIahNxmltrAr1mNec20rW4fFXoh8zMmYQcfcM0JJfVEpERCLQ2lAVarffpXNrOWdBA49WYMxIMbXeCfzCzD5vZn8y8Iq0ViIis4iZsWRuLRZir99rVrays72bHYe7wjvoCIoJIvuBB/JlGwpeIiISkupEjAWN4U3U+Naz5hMPjEcjngZl3PH37v5XAPlR6u7ulRsKKSIyi7Q0VHGiN0VXX6bsYzVUJ7j89Hk8/vJBbr2ijUQsmlHyxYxYP8/MNgO/Abaa2SYzWx1JbUREZrnFc2tCe6x17cpWjvem2bTraDgHHEExoelu4E/cfbm7Lwc+A3wtshqJiMxiVfEYzXXh9GW6eNlc5tQmIn2kVUwQqXP3xwY+uPvjQF1kNRIRmeVaG6pCuRuJBcbVZ7ewYecROvvS5R9wBMUEkR35nllt+dcdwGuR1EZERIjHAlobwhk7csUZ80lnnc27o3mkVUwQ+SjQAnwv/5oPfCSS2oiICADz6quIBeXfjpy9oIGG6jgbIppLq5jeWUeB/xDJ2UVEZESxwGhtrOL1jt6yj3PJ8rls2nWUTNZDCUyFiumd9TMzm1Pwea6ZPRxqLUREZJh5dXHTQhgAAA24SURBVEkS8fK/9C9ra+Z4b5pXD5wIoVanKuZx1nx37xj4kL8z0XoiIiIRMzMWNJQ/APGiZXMJjEimhy8miGTNbNnABzNbjtYTERGpiDm1CaoT5Q0UrK+Ks2pRIxsjGC9STM3+HHjSzO4xs3uAnwOfDb0mIiIyjJmxoKn8u5FL25p57XAXh070hVCrk8YNIu7+E+Bi4F/zr0vcXW0iIiIV0lidoLYqVtYxLm1rBmDjrnAfaY0aRMxsuZk1Abj7YaALuA74sKaGFxGprIVlTs64ZG4NCxurQ+/qO9adyHryI9PNbA25ZXF3AxcCXy3m4GZ2vZm9bGbbzOz2EfKXmdljZrbZzJ4zs3cX5H02v9/LZvbOifxQIiIzTV1VnIbqcUdljMrMWNs2ly17j9GXLn+CxwFjBZEad9+f3/4Q8HV3/x/kBhpeNt6B86sifgV4F7AKuMXMVg0pdgew3t0vAm4mH5zy5W4GVgPXA1/NH09EZNYqdwXES9ua6U9neX7vsZBqNHYQKeycfA3wCIC7Z4s89mXANnff4e79wL3A+4aUcaAxv91Ebu0S8uXudfc+d38N2EYRgUtEZCarTcbLWgHx/MVNVCeCULv6jnVv9KiZrQdeB+YCjwKY2SKgmNXfFwN7Cj7vBS4fUuZO4Kdm9klyj87eXrDv00P2XTz0BGb2ceDjAMuWLRuaLSIy48yrq2Jvf09J+yZiAWuWzmHDzqO4OxbCLI9jhbRPkZsrayfwFndP5dMXkuv2O56Rajd0fMktwDp3XwK8G7jHzIIi98Xd73b3te6+tqWlpYgqiYhMb001ibKmLrm0rZnDnX3sau8OpT6j3ol4bnX3e0dI31zksfcCSws+L+Hk46oBt5Fr88DdnzKzanITPBazr4jIrBMExty6BIdPFPNAaLi1y3NdfTfsPELb/PJX9YhmvcScDcBZZrYi3yX4ZuD+IWV2A9cCmNm5QDVwKF/uZjOrMrMVwFnAryOsq4jItFHOolXNdUnObKlnQ0ij1yMLIu6eBj4BPAy8SK4X1lYz+4KZvTdf7DPAx8xsC/Bt4FbP2Uqui/ELwE+AP3b38PqkiYhMY1XxWFndfde2zeXlN45zrCc1fuFxjFoLM/tT4F/dfc9oZcbj7g8CDw5J+4uC7ReAK0fZ9y7grlLPLSIykzXXJznRW9pqhZe2NXPvhj08E8JCVWPdiSwGfmlmPzezPzKz+WWfTUREQtFQFScZL+1h0pmt9cypTbAxhK6+o9bA3T8NLAM+D1wAPGdmD5nZh82soewzi4hIycys5LaRwIy1y+eyKeI7EfLtE0+4+x+R6y31d8CngQNln1lERMoytzZBqUM9Lm1rpquv/KbmolpmzOx8cr2rfhdoBz5X9plFRKQs8VhAU02Cju6JN5CvWTqHeAhL5Y7VsH4WucGANwMZcmNG3uHuO8o+q4iIhGJefbKkIFKbjHPe4ia2l3n+sR5nPQxUAb/r7ue7+10KICIiU0tuPq3S5qe94YJFZZ9/rCDyTuAhd3++MNHM3mpmZ5R9ZhERCcW8EhvYL18xr+xzjxVEvgQcHyG9h1wDu4iITAHlzqdVjrGCSJu7Pzc00d03Am2R1UhERCZkYD6tSTn3GHljrcVYE3ZFRESkdE01Uy+IbDCzjw1NNLPbgE3RVUlERCaqNhknEa/8I62xxol8Cvi+mf0eJ4PGWiAJvD/qiomIyMQ01ZQ+RXypxlpP5ABwhZm9DTgvn/xjd3+0IjUTEZEJmVJBZIC7PwY8VoG6iIhIGQYeaaXSwxaCjUyUi1KJiEiFNVZXtoFdQUREZAapdC8tBRERkRmkripOPFa5XloKIiIiM0wl70YUREREZhgFERERKVltMlaxR1oKIiIiM4yZ0VihuxEFERGRGahSj7QUREREZqC6ZKwi08MriIiIzEBmRlNt9HcjCiIiIjNUY/W4M1uVTUFERGSGqq+KR/5IS0FERGSGyvXSivZuREFERGQGi7qXloKIiMgMVl8VJ4jwm15BRERkBjOzSKeHjzSImNn1ZvaymW0zs9tHyP+SmT2bf71iZh0FeX9jZlvN7EUz+7KZVX7xYBGRGaC+Krp2kciObGYx4CvAdcBeYIOZ3e/uLwyUcfdPF5T/JHBRfvsK4Erggnz2k8BVwONR1VdEZKaqizCIRHknchmwzd13uHs/cC/wvjHK3wJ8O7/tQDWQBKqABHAgwrqKiMxYyXhAVSKar/sog8hiYE/B5735tGHMbDmwAngUwN2fIreu++v518Pu/uII+33czDaa2cZDhw6FXH0RkZkjqruRKIPISG0Yo60efzNwn7tnAMzsTOBcYAm5wHONmf3WsIO53+3ua919bUtLS0jVFhGZeaJqF4kyiOwFlhZ8XgLsH6XszZx8lAXwfuBpd+90907gIeBNkdRSRGQWmI5BZANwlpmtMLMkuUBx/9BCZnYOMBd4qiB5N3CVmcXNLEGuUX3Y4ywRESlOLDBqkrHQjxtZEHH3NPAJ4GFyAWC9u281sy+Y2XsLit4C3OvuhY+67gO2A88DW4At7v6jqOoqIjIbRHE3EumkKu7+IPDgkLS/GPL5zhH2ywB/EGXdRERmm/rqOIdO9IV6TI1YFxGZJWoTMcIetq0gIiIySwSBURtyu4iCiIjILFIf8kJVCiIiIrNI2I3rCiIiIrNITSIW6tTwCiIiIrOImYV6N6IgIiIyyyiIiIhIycKcjFFBRERklqlOxEjEwxkwoiAiIjIL1SXDuRtREBERmYUaQhovoiAiIjILhdUuoiAiIjILJWLhLJmrICIiMkuF0dVXQUREZJYK45GWgoiIyCylOxERESlZLCh/rIiCiIiIlExBRERESqYgIiIiJVMQERGRkimIiIhIyRRERESkZAoiIiJSMgUREREpmYKIiIiUzNx9susQCjM7AbxcgVM1Acci3m+8sqPlTyR9aNrQz/OBw+PWtDylXsuJ7FtMOV3P8K7nWPnFXLdi0ipxLUerR9j7Tcb1HPr5HHdvGL+qo3D3GfECNlboPHdHvd94ZUfLn0j60LQRPkd+PUu9lhPZt5hyup7hXc+x8ou5bsWk6f96edcz7N9NPc6auB9VYL/xyo6WP5H0oWml/lzlKOecxe5bTDldz/Cu51j5xVy3iaRFbSr/Xx8tr+K/mzPpcdZGd1872fWYKXQ9w6XrGR5dy3CVez1n0p3I3ZNdgRlG1zNcup7h0bUMV1nXc8bciYiISOXNpDsRERGpMAUREREpmYKIiIiUbNYEETOrM7NNZnbDZNdlujOzc83sH83sPjP7o8muz3RnZjeZ2dfM7Idm9o7Jrs90Zmanm9n/NrP7Jrsu01X+u/Ib+d/J3xuv/JQPImb2dTM7aGa/GZJ+vZm9bGbbzOz2Ig71Z8D6aGo5fYRxPd39RXf/Q+B3gFnd1TKk6/kDd/8YcCvwuxFWd0oL6VrucPfboq3p9DPBa/vvgPvyv5PvHe/YUz6IAOuA6wsTzCwGfAV4F7AKuMXMVpnZ+Wb2wJBXq5m9HXgBOFDpyk9B6yjzeub3eS/wJPBIZas/5awjhOuZd0d+v9lqHeFdSznVOoq8tsASYE++WGa8A8dDrWYE3P3nZtY2JPkyYJu77wAws3uB97n7fwOGPa4ys7cBdeQuVI+ZPeju2UgrPkWFcT3zx7kfuN/Mfgz8S3Q1ntpC+v004IvAQ+7+TLQ1nrrC+t2U4SZybYG95ALJsxRxozHlg8goFnMyUkLuh758tMLu/ucAZnYrcHi2BpAxTOh6mtnV5G55q4AHI63Z9DSh6wl8Eng70GRmZ7r7P0ZZuWlmor+b84C7gIvM7LP5YCMjG+3afhn4BzN7D0VMkTJdg4iNkDbuqEl3Xxd+VWaECV1Pd38ceDyqyswAE72eXyb3H1eGm+i1bAf+MLrqzCgjXlt37wI+UuxBpkObyEj2AksLPi8B9k9SXWYCXc9w6XqGR9cyOqFc2+kaRDYAZ5nZCjNLAjcD909ynaYzXc9w6XqGR9cyOqFc2ykfRMzs28BTwDlmttfMbnP3NPAJ4GHgRWC9u2+dzHpOF7qe4dL1DI+uZXSivLaagFFEREo25e9ERERk6lIQERGRkimIiIhIyRRERESkZAoiIiJSMgUREREpmYKISAnM7P1m5ma2Mv+5beg02yPsM24ZkelGQUSkNLeQmwr/5smuiMhkUhARmSAzqweuBG5jhCBiZrdabpXCn+QX/PnLguyY5VaM22pmPzWzmvw+HzOzDWa2xcy+a2a1lflpRMqjICIycTcBP3H3V4AjZnbxCGUuA34PWAP8tpkNrAB5FvAVd18NdAAfyKd/z90vdfcLyU1BodX5ZFpQEBGZuFuAe/Pb9+Y/D/Uzd2939x7ge8Bb8umvufuz+e1NQFt++zwz+//N7HlywWd1JDUXCdl0XU9EZFLkFz26htyXvgMxcutbfHVI0aGT0g187itIywA1+e11wE3uviW/eNrV4dVaJDq6ExGZmA8C33T35e7e5u5LgdfIrcVQ6Doza863edwE/GKc4zYAr5tZgtydiMi0oCAiMjG3AN8fkvZd4HND0p4E7iG3TvV33X3jOMf9PPAr4GfASyHUU6QiNBW8SMjyj6PWuvsnJrsuIlHTnYiIiJRMdyIiIlIy3YmIiEjJFERERKRkCiIiIlIyBRERESmZgoiIiJRMQUREREr2fwEWV3VvqBLzTQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.linear_model import Ridge\n", "\n", "# Setup the array of alphas and lists to store scores\n", "alpha_space = np.logspace(-4, 0, 50)\n", "ridge_scores = []\n", "ridge_scores_std = []\n", "\n", "# Create a ridge regressor: ridge\n", "ridge = Ridge(normalize=True)\n", "\n", "# Compute scores over range of alphas\n", "for alpha in alpha_space:\n", " \n", " # Specify the alpha value to use: ridge.alhpa\n", " ridge.alpha = alpha\n", " \n", " # Perform 10-fold CV: ridge_cv_scores\n", " ridge_cv_scores = cross_val_score(ridge, X, y, cv=10)\n", " \n", " # Append the mean of ridge_cv_scores to ridge_scores\n", " ridge_scores.append(np.mean(ridge_cv_scores))\n", " \n", " # Append the std of ridge_cv_scores to ridge_scores_std\n", " ridge_scores_std.append(np.std(ridge_cv_scores))\n", " \n", "# Display the plot\n", "display_plot(ridge_scores, ridge_scores_std)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }