{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# *Predicting House Sale Prices*\n", "\n", "***In this project, we will predict a House's Sale Price using its attributes. The data set we will be working with contains housing data for the city of Ames, Iowa, United States from 2006 to 2010. For each house, we have information such as Lot Area, Garage Area, Year Built, and more.***\n", "\n", "[Download](https://dsserver-prod-resources-1.s3.amazonaws.com/235/AmesHousing.txt)
\n", "[Documentation](https://s3.amazonaws.com/dq-content/307/data_description.txt)\n", "\n", "***We will be using Linear Regression algorithm to predict a House's price accurately.***" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OrderPIDMS SubClassMS ZoningLot FrontageLot AreaStreetAlleyLot ShapeLand ContourUtilitiesLot ConfigLand SlopeNeighborhoodCondition 1Condition 2Bldg TypeHouse StyleOverall QualOverall CondYear BuiltYear Remod/AddRoof StyleRoof MatlExterior 1stExterior 2ndMas Vnr TypeMas Vnr AreaExter QualExter CondFoundationBsmt QualBsmt CondBsmt ExposureBsmtFin Type 1BsmtFin SF 1BsmtFin Type 2BsmtFin SF 2Bsmt Unf SFTotal Bsmt SFHeatingHeating QCCentral AirElectrical1st Flr SF2nd Flr SFLow Qual Fin SFGr Liv AreaBsmt Full BathBsmt Half BathFull BathHalf BathBedroom AbvGrKitchen AbvGrKitchen QualTotRms AbvGrdFunctionalFireplacesFireplace QuGarage TypeGarage Yr BltGarage FinishGarage CarsGarage AreaGarage QualGarage CondPaved DriveWood Deck SFOpen Porch SFEnclosed Porch3Ssn PorchScreen PorchPool AreaPool QCFenceMisc FeatureMisc ValMo SoldYr SoldSale TypeSale ConditionSalePrice
0152630110020RL141.031770PaveNaNIR1LvlAllPubCornerGtlNAmesNormNorm1Fam1Story6519601960HipCompShgBrkFacePlywoodStone112.0TATACBlockTAGdGdBLQ639.0Unf0.0441.01080.0GasAFaYSBrkr16560016561.00.01031TA7Typ2GdAttchd1960.0Fin2.0528.0TATAP210620000NaNNaNNaN052010WDNormal215000
1252635004020RH80.011622PaveNaNRegLvlAllPubInsideGtlNAmesFeedrNorm1Fam1Story5619611961GableCompShgVinylSdVinylSdNone0.0TATACBlockTATANoRec468.0LwQ144.0270.0882.0GasATAYSBrkr896008960.00.01021TA5Typ0NaNAttchd1961.0Unf1.0730.0TATAY1400001200NaNMnPrvNaN062010WDNormal105000
2352635101020RL81.014267PaveNaNIR1LvlAllPubCornerGtlNAmesNormNorm1Fam1Story6619581958HipCompShgWd SdngWd SdngBrkFace108.0TATACBlockTATANoALQ923.0Unf0.0406.01329.0GasATAYSBrkr13290013290.00.01131Gd6Typ0NaNAttchd1958.0Unf1.0312.0TATAY393360000NaNNaNGar21250062010WDNormal172000
3452635303020RL93.011160PaveNaNRegLvlAllPubCornerGtlNAmesNormNorm1Fam1Story7519681968HipCompShgBrkFaceBrkFaceNone0.0GdTACBlockTATANoALQ1065.0Unf0.01045.02110.0GasAExYSBrkr21100021101.00.02131Ex8Typ2TAAttchd1968.0Fin2.0522.0TATAY000000NaNNaNNaN042010WDNormal244000
4552710501060RL74.013830PaveNaNIR1LvlAllPubInsideGtlGilbertNormNorm1Fam2Story5519971998GableCompShgVinylSdVinylSdNone0.0TATAPConcGdTANoGLQ791.0Unf0.0137.0928.0GasAGdYSBrkr928701016290.00.02131TA6Typ1TAAttchd1997.0Fin2.0482.0TATAY212340000NaNMnPrvNaN032010WDNormal189900
\n", "
" ], "text/plain": [ " Order PID MS SubClass MS Zoning Lot Frontage Lot Area Street \\\n", "0 1 526301100 20 RL 141.0 31770 Pave \n", "1 2 526350040 20 RH 80.0 11622 Pave \n", "2 3 526351010 20 RL 81.0 14267 Pave \n", "3 4 526353030 20 RL 93.0 11160 Pave \n", "4 5 527105010 60 RL 74.0 13830 Pave \n", "\n", " Alley Lot Shape Land Contour Utilities Lot Config Land Slope Neighborhood \\\n", "0 NaN IR1 Lvl AllPub Corner Gtl NAmes \n", "1 NaN Reg Lvl AllPub Inside Gtl NAmes \n", "2 NaN IR1 Lvl AllPub Corner Gtl NAmes \n", "3 NaN Reg Lvl AllPub Corner Gtl NAmes \n", "4 NaN IR1 Lvl AllPub Inside Gtl Gilbert \n", "\n", " Condition 1 Condition 2 Bldg Type House Style Overall Qual Overall Cond \\\n", "0 Norm Norm 1Fam 1Story 6 5 \n", "1 Feedr Norm 1Fam 1Story 5 6 \n", "2 Norm Norm 1Fam 1Story 6 6 \n", "3 Norm Norm 1Fam 1Story 7 5 \n", "4 Norm Norm 1Fam 2Story 5 5 \n", "\n", " Year Built Year Remod/Add Roof Style Roof Matl Exterior 1st Exterior 2nd \\\n", "0 1960 1960 Hip CompShg BrkFace Plywood \n", "1 1961 1961 Gable CompShg VinylSd VinylSd \n", "2 1958 1958 Hip CompShg Wd Sdng Wd Sdng \n", "3 1968 1968 Hip CompShg BrkFace BrkFace \n", "4 1997 1998 Gable CompShg VinylSd VinylSd \n", "\n", " Mas Vnr Type Mas Vnr Area Exter Qual Exter Cond Foundation Bsmt Qual \\\n", "0 Stone 112.0 TA TA CBlock TA \n", "1 None 0.0 TA TA CBlock TA \n", "2 BrkFace 108.0 TA TA CBlock TA \n", "3 None 0.0 Gd TA CBlock TA \n", "4 None 0.0 TA TA PConc Gd \n", "\n", " Bsmt Cond Bsmt Exposure BsmtFin Type 1 BsmtFin SF 1 BsmtFin Type 2 \\\n", "0 Gd Gd BLQ 639.0 Unf \n", "1 TA No Rec 468.0 LwQ \n", "2 TA No ALQ 923.0 Unf \n", "3 TA No ALQ 1065.0 Unf \n", "4 TA No GLQ 791.0 Unf \n", "\n", " BsmtFin SF 2 Bsmt Unf SF Total Bsmt SF Heating Heating QC Central Air \\\n", "0 0.0 441.0 1080.0 GasA Fa Y \n", "1 144.0 270.0 882.0 GasA TA Y \n", "2 0.0 406.0 1329.0 GasA TA Y \n", "3 0.0 1045.0 2110.0 GasA Ex Y \n", "4 0.0 137.0 928.0 GasA Gd Y \n", "\n", " Electrical 1st Flr SF 2nd Flr SF Low Qual Fin SF Gr Liv Area \\\n", "0 SBrkr 1656 0 0 1656 \n", "1 SBrkr 896 0 0 896 \n", "2 SBrkr 1329 0 0 1329 \n", "3 SBrkr 2110 0 0 2110 \n", "4 SBrkr 928 701 0 1629 \n", "\n", " Bsmt Full Bath Bsmt Half Bath Full Bath Half Bath Bedroom AbvGr \\\n", "0 1.0 0.0 1 0 3 \n", "1 0.0 0.0 1 0 2 \n", "2 0.0 0.0 1 1 3 \n", "3 1.0 0.0 2 1 3 \n", "4 0.0 0.0 2 1 3 \n", "\n", " Kitchen AbvGr Kitchen Qual TotRms AbvGrd Functional Fireplaces \\\n", "0 1 TA 7 Typ 2 \n", "1 1 TA 5 Typ 0 \n", "2 1 Gd 6 Typ 0 \n", "3 1 Ex 8 Typ 2 \n", "4 1 TA 6 Typ 1 \n", "\n", " Fireplace Qu Garage Type Garage Yr Blt Garage Finish Garage Cars \\\n", "0 Gd Attchd 1960.0 Fin 2.0 \n", "1 NaN Attchd 1961.0 Unf 1.0 \n", "2 NaN Attchd 1958.0 Unf 1.0 \n", "3 TA Attchd 1968.0 Fin 2.0 \n", "4 TA Attchd 1997.0 Fin 2.0 \n", "\n", " Garage Area Garage Qual Garage Cond Paved Drive Wood Deck SF \\\n", "0 528.0 TA TA P 210 \n", "1 730.0 TA TA Y 140 \n", "2 312.0 TA TA Y 393 \n", "3 522.0 TA TA Y 0 \n", "4 482.0 TA TA Y 212 \n", "\n", " Open Porch SF Enclosed Porch 3Ssn Porch Screen Porch Pool Area Pool QC \\\n", "0 62 0 0 0 0 NaN \n", "1 0 0 0 120 0 NaN \n", "2 36 0 0 0 0 NaN \n", "3 0 0 0 0 0 NaN \n", "4 34 0 0 0 0 NaN \n", "\n", " Fence Misc Feature Misc Val Mo Sold Yr Sold Sale Type Sale Condition \\\n", "0 NaN NaN 0 5 2010 WD Normal \n", "1 MnPrv NaN 0 6 2010 WD Normal \n", "2 NaN Gar2 12500 6 2010 WD Normal \n", "3 NaN NaN 0 4 2010 WD Normal \n", "4 MnPrv NaN 0 3 2010 WD Normal \n", "\n", " SalePrice \n", "0 215000 \n", "1 105000 \n", "2 172000 \n", "3 244000 \n", "4 189900 " ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import seaborn as sns\n", "import matplotlib.pyplot as plt\n", "\n", "import matplotlib.style as style\n", "style.use('fivethirtyeight')\n", "\n", "from sklearn.linear_model import LinearRegression\n", "from sklearn.metrics import mean_squared_error\n", "%matplotlib inline\n", "\n", "pd.set_option('display.max_columns', 500)\n", "data = pd.read_csv('AmesHousing.txt', delimiter='\\t')\n", "\n", "\n", "data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Method to quickly get info related to a dataframe**" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape\n", "(2930, 82)\n", "----------\n", "\n", "Data Types\n", "object 43\n", "int64 28\n", "float64 11\n", "dtype: int64\n", "----------\n", "\n", "Columns : Null Counts\n", "Lot Frontage 490\n", "Alley 2732\n", "Mas Vnr Type 23\n", "Mas Vnr Area 23\n", "Bsmt Qual 80\n", "Bsmt Cond 80\n", "Bsmt Exposure 83\n", "BsmtFin Type 1 80\n", "BsmtFin SF 1 1\n", "BsmtFin Type 2 81\n", "BsmtFin SF 2 1\n", "Bsmt Unf SF 1\n", "Total Bsmt SF 1\n", "Electrical 1\n", "Bsmt Full Bath 2\n", "Bsmt Half Bath 2\n", "Fireplace Qu 1422\n", "Garage Type 157\n", "Garage Yr Blt 159\n", "Garage Finish 159\n", "Garage Cars 1\n", "Garage Area 1\n", "Garage Qual 159\n", "Garage Cond 159\n", "Pool QC 2917\n", "Fence 2358\n", "Misc Feature 2824\n", "dtype: int64\n" ] } ], "source": [ "def data_shape(data):\n", " print('Shape')\n", " print(data.shape)\n", " print(len(str(data.shape)) * \"-\")\n", " \n", " print('\\nData Types')\n", " print(data.dtypes.value_counts())\n", " print(len(str(data.shape)) * \"-\")\n", " \n", " print('\\nColumns : Null Counts')\n", " null_counts = data.isnull().sum()\n", " print(null_counts[null_counts > 0])\n", " \n", "\n", "data_shape(data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Transforming Features\n", "\n", "- Creating more useful columns from existing Columns
\n", "- Dropping columns which are not useful or leak information about the sale
\n", "- Dropping Columns which have more then 5% missing Values" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape\n", "(2927, 65)\n", "----------\n", "\n", "Data Types\n", "object 32\n", "int64 24\n", "float64 9\n", "dtype: int64\n", "----------\n", "\n", "Columns : Null Counts\n", "Mas Vnr Type 23\n", "Mas Vnr Area 23\n", "Bsmt Qual 80\n", "Bsmt Cond 80\n", "Bsmt Exposure 83\n", "BsmtFin Type 1 80\n", "BsmtFin SF 1 1\n", "BsmtFin Type 2 81\n", "BsmtFin SF 2 1\n", "Bsmt Unf SF 1\n", "Total Bsmt SF 1\n", "Electrical 1\n", "Bsmt Full Bath 2\n", "Bsmt Half Bath 2\n", "Garage Cars 1\n", "Garage Area 1\n", "dtype: int64\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MS SubClassMS ZoningLot AreaStreetLot ShapeLand ContourUtilitiesLot ConfigLand SlopeNeighborhoodCondition 1Condition 2Bldg TypeHouse StyleOverall QualOverall CondRoof StyleRoof MatlExterior 1stExterior 2ndMas Vnr TypeMas Vnr AreaExter QualExter CondFoundationBsmt QualBsmt CondBsmt ExposureBsmtFin Type 1BsmtFin SF 1BsmtFin Type 2BsmtFin SF 2Bsmt Unf SFTotal Bsmt SFHeatingHeating QCCentral AirElectrical1st Flr SF2nd Flr SFLow Qual Fin SFGr Liv AreaBsmt Full BathBsmt Half BathFull BathHalf BathBedroom AbvGrKitchen AbvGrKitchen QualTotRms AbvGrdFunctionalFireplacesGarage CarsGarage AreaPaved DriveWood Deck SFOpen Porch SFEnclosed Porch3Ssn PorchScreen PorchPool AreaMisc ValSalePriceYears_since_remodelledYears_since_sold
020RL31770PaveIR1LvlAllPubCornerGtlNAmesNormNorm1Fam1Story65HipCompShgBrkFacePlywoodStone112.0TATACBlockTAGdGdBLQ639.0Unf0.0441.01080.0GasAFaYSBrkr16560016561.00.01031TA7Typ22.0528.0P21062000002150005050
120RH11622PaveRegLvlAllPubInsideGtlNAmesFeedrNorm1Fam1Story56GableCompShgVinylSdVinylSdNone0.0TATACBlockTATANoRec468.0LwQ144.0270.0882.0GasATAYSBrkr896008960.00.01021TA5Typ01.0730.0Y140000120001050004949
220RL14267PaveIR1LvlAllPubCornerGtlNAmesNormNorm1Fam1Story66HipCompShgWd SdngWd SdngBrkFace108.0TATACBlockTATANoALQ923.0Unf0.0406.01329.0GasATAYSBrkr13290013290.00.01131Gd6Typ01.0312.0Y393360000125001720005252
320RL11160PaveRegLvlAllPubCornerGtlNAmesNormNorm1Fam1Story75HipCompShgBrkFaceBrkFaceNone0.0GdTACBlockTATANoALQ1065.0Unf0.01045.02110.0GasAExYSBrkr21100021101.00.02131Ex8Typ22.0522.0Y00000002440004242
460RL13830PaveIR1LvlAllPubInsideGtlGilbertNormNorm1Fam2Story55GableCompShgVinylSdVinylSdNone0.0TATAPConcGdTANoGLQ791.0Unf0.0137.0928.0GasAGdYSBrkr928701016290.00.02131TA6Typ12.0482.0Y21234000001899001213
\n", "
" ], "text/plain": [ " MS SubClass MS Zoning Lot Area Street Lot Shape Land Contour Utilities \\\n", "0 20 RL 31770 Pave IR1 Lvl AllPub \n", "1 20 RH 11622 Pave Reg Lvl AllPub \n", "2 20 RL 14267 Pave IR1 Lvl AllPub \n", "3 20 RL 11160 Pave Reg Lvl AllPub \n", "4 60 RL 13830 Pave IR1 Lvl AllPub \n", "\n", " Lot Config Land Slope Neighborhood Condition 1 Condition 2 Bldg Type \\\n", "0 Corner Gtl NAmes Norm Norm 1Fam \n", "1 Inside Gtl NAmes Feedr Norm 1Fam \n", "2 Corner Gtl NAmes Norm Norm 1Fam \n", "3 Corner Gtl NAmes Norm Norm 1Fam \n", "4 Inside Gtl Gilbert Norm Norm 1Fam \n", "\n", " House Style Overall Qual Overall Cond Roof Style Roof Matl Exterior 1st \\\n", "0 1Story 6 5 Hip CompShg BrkFace \n", "1 1Story 5 6 Gable CompShg VinylSd \n", "2 1Story 6 6 Hip CompShg Wd Sdng \n", "3 1Story 7 5 Hip CompShg BrkFace \n", "4 2Story 5 5 Gable CompShg VinylSd \n", "\n", " Exterior 2nd Mas Vnr Type Mas Vnr Area Exter Qual Exter Cond Foundation \\\n", "0 Plywood Stone 112.0 TA TA CBlock \n", "1 VinylSd None 0.0 TA TA CBlock \n", "2 Wd Sdng BrkFace 108.0 TA TA CBlock \n", "3 BrkFace None 0.0 Gd TA CBlock \n", "4 VinylSd None 0.0 TA TA PConc \n", "\n", " Bsmt Qual Bsmt Cond Bsmt Exposure BsmtFin Type 1 BsmtFin SF 1 \\\n", "0 TA Gd Gd BLQ 639.0 \n", "1 TA TA No Rec 468.0 \n", "2 TA TA No ALQ 923.0 \n", "3 TA TA No ALQ 1065.0 \n", "4 Gd TA No GLQ 791.0 \n", "\n", " BsmtFin Type 2 BsmtFin SF 2 Bsmt Unf SF Total Bsmt SF Heating Heating QC \\\n", "0 Unf 0.0 441.0 1080.0 GasA Fa \n", "1 LwQ 144.0 270.0 882.0 GasA TA \n", "2 Unf 0.0 406.0 1329.0 GasA TA \n", "3 Unf 0.0 1045.0 2110.0 GasA Ex \n", "4 Unf 0.0 137.0 928.0 GasA Gd \n", "\n", " Central Air Electrical 1st Flr SF 2nd Flr SF Low Qual Fin SF \\\n", "0 Y SBrkr 1656 0 0 \n", "1 Y SBrkr 896 0 0 \n", "2 Y SBrkr 1329 0 0 \n", "3 Y SBrkr 2110 0 0 \n", "4 Y SBrkr 928 701 0 \n", "\n", " Gr Liv Area Bsmt Full Bath Bsmt Half Bath Full Bath Half Bath \\\n", "0 1656 1.0 0.0 1 0 \n", "1 896 0.0 0.0 1 0 \n", "2 1329 0.0 0.0 1 1 \n", "3 2110 1.0 0.0 2 1 \n", "4 1629 0.0 0.0 2 1 \n", "\n", " Bedroom AbvGr Kitchen AbvGr Kitchen Qual TotRms AbvGrd Functional \\\n", "0 3 1 TA 7 Typ \n", "1 2 1 TA 5 Typ \n", "2 3 1 Gd 6 Typ \n", "3 3 1 Ex 8 Typ \n", "4 3 1 TA 6 Typ \n", "\n", " Fireplaces Garage Cars Garage Area Paved Drive Wood Deck SF \\\n", "0 2 2.0 528.0 P 210 \n", "1 0 1.0 730.0 Y 140 \n", "2 0 1.0 312.0 Y 393 \n", "3 2 2.0 522.0 Y 0 \n", "4 1 2.0 482.0 Y 212 \n", "\n", " Open Porch SF Enclosed Porch 3Ssn Porch Screen Porch Pool Area \\\n", "0 62 0 0 0 0 \n", "1 0 0 0 120 0 \n", "2 36 0 0 0 0 \n", "3 0 0 0 0 0 \n", "4 34 0 0 0 0 \n", "\n", " Misc Val SalePrice Years_since_remodelled Years_since_sold \n", "0 0 215000 50 50 \n", "1 0 105000 49 49 \n", "2 12500 172000 52 52 \n", "3 0 244000 42 42 \n", "4 0 189900 12 13 " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Creating more useful columns from existing Columns\n", "df = data.copy()\n", "df['Years_since_remodelled'] = df['Yr Sold'] - df['Year Remod/Add']\n", "df['Years_since_sold'] = df['Yr Sold'] - df['Year Built']\n", "df = df[(df['Years_since_remodelled'] >= 0)]\n", "df = df[(df['Years_since_sold'] >= 0)]\n", "\n", "# Dropping columns which are not useful or leak information about the sale\n", "df.drop(['Sale Condition', 'Sale Type', 'Mo Sold', 'Yr Sold', 'PID', 'Order', \n", " 'Year Remod/Add', 'Year Built'], axis=1, inplace=True)\n", "\n", "# Dropping Columns which have more then 5% missing Values\n", "null_counts = df.isnull().sum()\n", "df.drop(null_counts[null_counts > df.shape[0]*0.05].index, axis=1, inplace=True)\n", "\n", "data_shape(df)\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature Engineering \n", "\n", "- Imputing missing values with mode for numerical columns\n", "- Finding Highly correlated columns to target column\n", "- Dropping numerical columns with high collinearity\n", "- Dropping Text Columns with missing Values\n", "- Converting nominal columns into dummy variables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Numerical columns" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape\n", "(2927, 33)\n", "----------\n", "\n", "Data Types\n", "int64 24\n", "float64 9\n", "dtype: int64\n", "----------\n", "\n", "Columns : Null Counts\n", "Mas Vnr Area 23\n", "BsmtFin SF 1 1\n", "BsmtFin SF 2 1\n", "Bsmt Unf SF 1\n", "Total Bsmt SF 1\n", "Bsmt Full Bath 2\n", "Bsmt Half Bath 2\n", "Garage Cars 1\n", "Garage Area 1\n", "dtype: int64\n" ] } ], "source": [ "numerical_cols = df[df.dtypes[df.dtypes != 'object'].index].copy()\n", "data_shape(numerical_cols)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Imputing missing values with mode" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape\n", "(2927, 33)\n", "----------\n", "\n", "Data Types\n", "int64 24\n", "float64 9\n", "dtype: int64\n", "----------\n", "\n", "Columns : Null Counts\n", "Series([], dtype: int64)\n" ] } ], "source": [ "numerical_cols.fillna(numerical_cols.mode().iloc[0], inplace=True)\n", "data_shape(numerical_cols)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Finding Highly correlated columns to target Column\n", "- Dropping columns with high collinearilty\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "16\n", "Open Porch SF 0.316262\n", "Wood Deck SF 0.328183\n", "BsmtFin SF 1 0.439284\n", "Fireplaces 0.474831\n", "TotRms AbvGrd 0.498574\n", "Mas Vnr Area 0.506983\n", "Years_since_remodelled 0.534985\n", "Full Bath 0.546118\n", "Years_since_sold 0.558979\n", "1st Flr SF 0.635185\n", "Garage Area 0.641425\n", "Total Bsmt SF 0.644012\n", "Garage Cars 0.648361\n", "Gr Liv Area 0.717596\n", "Overall Qual 0.801206\n", "SalePrice 1.000000\n", "Name: SalePrice, dtype: float64\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFjCAYAAADCXlkLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnXecVdXVv58vHURA7IqK3SgqKhLFhgaNJbbYSyJv9KcmlsS81jdqLLEkmmLsxoLGGsUAJkYQFVFEAQFpdkVFjYqCClJn1u+PvS8cLufO3Jl7Bu6dux4/58M5++yz9r7XmVln7bK+MjMcx3Ecx6ksWqzsDjiO4ziO03DcgTuO4zhOBeIO3HEcx3EqEHfgjuM4jlOBuAN3HMdxnArEHbjjOI7jVCCtVnYHnPJh0cz3MtlTeP/2l2ZhBoA1amoys7WqLc7MVpd2CzKz9d/5HTKxc1e7eZnYAbiizaLMbF23KJvPB3BmTTb9atGiPLfPLlrcMjNb66z3TWa2Fs3Ppl+t2mb3+wyw4bhnlJWtYv/+tV5jk8zaLBV34I7jOI5Tm+3LxYrAHbjjOI7jWO3K7kGDafZz4JK6SRos6W1J70q6QVKbJm6zv6QvJE2UNE3S/8vI7nRJa9RTp4OkByRNljRF0ouSOsZ7NbFPuaN7Fv1yHMepdKxmcVFHOdGsI3BJAh4HbjWzQyW1BO4ArgLOa+LmHzGzMyWtBUyVNMTMPqvvIUmtzEqarP0l8JmZbRvtbQnkJg7nmVnPEmw7juM0T2o9Ai839gHmm9k9AGZWA5wD/CxGqv1jdP6UpDcl/Tb3oKQTJY2Jkert0fkjaY6kqyS9JullSWvX1QEz+xx4F9hIUldJgyRNis9uF21eJukOScOA+yS1lHR9jKInSTorYfIsSePjva1SmlwX+DjR/ptmlt2KK8dxnOaI1RZ3lBHN3YFvA7yaLDCzb4APgc1iUW/gBKAncJSkXpK+BxwD7BYj1ppYB2AV4GUz2x4YCdQ5PC5pE2AT4B3gcmCCmW0H/B9wX6LqTsChZnY8cCqwMbBDrPtAot5MM9sRuBU4N6XJu4ELJI2W9DtJmyfutU8Mn/+zrn47juNUFbU1xR1lRLMeQgcEpG0NSJY/bWZfAkh6HNgdWExwqGPDKDztgc9j/YXAv+L5q8C+Bdo+RtLuwALgNDP7Kl4fAWBmz0paXVLnWH+ImeX2AfUDbssNpZvZVwm7jyfa/nF+o2Y2Mb407BftjJW0q5m9jg+hO47jpFNm0XUxNHcHPpXoMHNI6gRsQBjW3onlHbwRHPy9ZnZRis1FtlSDtYbC3+EjZnZmXlna/sGcrbl59QrtScwNhxds28zmEBz945JqgQOB1wvYcxzHqXrKbYFaMTT3IfRngA6SfgoQ57H/CAwws+9inX3j3HR74DBgVHzuyLgAjXh/owz6M5I4FC+pL2E4PC3bwjDgdEmtcu0X24Ck3SStFs/bAFsDH5TYb8dxnOZNbW1xRxnRrB14jJQPJ8xtvw28BcwnzD/neBH4OzARGGhm48xsGnAxMEzSJOBpwuKwUrkM6BVtXgucVKDenYR5+kmSXgOOb0AbmwLPS5oMTADGAQMb3WPHcZxqoMRFbJL2j4uh35F0Ycr9jSQ9Excmj5DUrdQua+locPUhqT/QK2WouyrxVKrF46lUi8dTqRaPp1JtGFmmUl3wxvNF/VC03Wqv5dqMo7tvEdZEzQDGAsfFYDBX51HgX2Z2r6R9gP8xs5+U0udmHYE7juM4TlGUFoH3Bt4xs/fMbCHwMHBoXp2tCdOzAM+l3G8wzX0RW52Y2QBgwEruRtmQVeR84mtXZGIHYMqO52RmS8ou6vpqfrvMbHVQNlFJW7KL3mbNye5Pw/w22UVdyijeaplhBF5Tm522Rfu22Y18fP1l+8xstWyZzdxvmw5lvFCstPnt9YGPEtczgO/n1XmNsKj6BsLU7qqSVs/tgmoMHoE7juM4Ts3i4o506tphlONcYC9JE4C9CAm3SnqjqeoI3HEcx3EAQqLORjODsD05Rzfgk2Xt2yfE3B1Rn+IIM/u6lEY9Anccx3Gc0ubAxwKbS9o4bt89FhiSrCBpDUk5n3sRIWtmSTR7By7pz5J+lbgeKunOxPUfJf06g3YGSDqyQPn7MXf6W5Luk7R+I9voL+mmeuqsLelfsb1pkp6M5d0lzctTI2tSVTbHcZyKoYR94DFr5pnAUELSrH+Y2VRJV0g6JFbrC7wp6S1gbYKoVklUwxD6S8BRwF/i288aQKfE/T7Ar9IezJDzzOwxhbysvwKek9QjrlbMmisI6WFvAMgJpkTe9VSqjuM4KZSYStXMngSezCu7NHH+GPBYSY3k0ewjcEJmtT7xfBtgCvCtpNUktQW+B0xQ4DoFDe3Jko6BIElaR/lNMcr9N7BWfR2xwJ+B/wIHRDv7ReGR8ZIe1VLt7p0lvRQj6TGSVk3aknRQfC5fH3xdwnxMrs1JDf7GHMdxqg0XMyk/zOwTSYslbUhw5KMJS/53Bb4GJpnZQklHEBTJtidE6WMljYzPpJXvCmwJbEsYDplG8XMa44GtJI0iZHzrZ2ZzJV0A/FrStcAjwDFmNjbmb1+SpUPS4cCvgQPNbFae7ZuBRySdCQwH7omLJwA2lTQxno8yszOK7K/jOE7zpgJzoTd7Bx7JReF9gD8RHHgfggN/KdbZHXgoaoZ/Jul5YOc6yvdMlH8i6dkG9Ce35WAXwub+UWF0nTaEF4wtgU/NbCwskUAl1tkb6AXsl5ZH3cyGKqiR7U+I8idI6hFv+xC64zhOGhWoRlYNQ+gQnHQfQrQ8BXiZEEH3ITh3SN/HV1c5FFYMq48dCAsdRJiv7hmPrc3sZOpWI3sPWBXYomCnzL4yswdjmr6xhJcNx3EcpxAuZlK2jAJ+BHxlZjVRX7sLwYmPjnVGEjS8W0pak+D0xtRTfmwsX5cQGddJnDc/mzBP/RThRWI3SZvF+x0kbQG8AawnaedYvqqiMhlBWezHwH2StklpYx9JHXLPEcRNPmzQt+U4jlNtVKADr5Yh9MmE+esH88o6mtnMeP1PgkN/jRD9nm9m/5VUV/k+0c5bwPN1tH+dpEuADgSnvXdcgf6FgqDKQ3FBHcDFZvZWXCx3o4LM6TygX86Ymb0p6QTgUUkHm9m7ibZ2Am6StJjwgnZnnEfvXvS35TiOU2WUmMhlpVAVDjzOU3fKK+ufd23AefEotrxeFbP8dlLuP0uYU88vH0uYI08yIB6Y2QTC/Hn+c9cB16WUTwd65Jc7juM4+CI2p7LJSrozSwGSHuP/nJmtj/Y+PTNbExZ3qr9Skey52heZ2Ok6p0smdgA+zvBPw7elZYtcho13m5uJndmvZ/f51tgnO9GQj57Iboj267nZCe6s0SWb773jJuUp4wqU3fB4MbgDdxzHcZwKXIXuDtxxHMdxPAJ3HMdxnAqkAiPwatlGloqkmijq8VpMZdqn/qfqtdlT0oGJ6/6SvkgIiNwXy6+Q1K+wpeXsdpD0QEznOkXSi4m0qzVaVqSke8rzT0maLelfpX5Gx3GcZodvI6s45uUyk0n6IXANQWi9FHoSMqUlk9o/YmbLrFhPJrkvkl8Cn5nZtgCStgQWxXtLPkcdXEfYxnZaA9t1HMdp/lTgKvSqjsDz6ATMApC0rqSRMZqdImmPWD5H0u8lvSppuKTekkZIek/SIQrynFcQEr9MzAmfpKGE/Kik6ZIuj6MAkyVtlfLIusDHuQsze9PMFhT74czsGeDbYus7juNUFRUYgVe7A28fHe0bwJ3AlbH8eGBojGq3B3ICIKsAI8xsJ4Iz/B2wL3A4cEVMznIpIeLuaWaPxOdyDn2ipP8p0JeZZrYjcCtwbsr9u4ELogLZ7yRtnvI5JsYEM47jOE5DsNrijjLCh9CXDqHvSkhP2oOQP/xuSa2BQWaWc+ALCSlQIWRgW2BmiyRNBrrX0c5yQ+gpPB7/fZWQKnUZzGxiFCnZj5CVbaykXc3sdYobQnccx3EKUWbRdTFUewS+BDMbTUi3uqaZjSTkPP8Y+Lukn8Zqi2IGNoBaYEF8tpbSX4Zyw+E1hWyZ2Rwze9zMfgHcDxyYVs9xHMdpIBUYgbsDj8R555bAl5I2Aj43s78BdwE7NsDUtwS1sKz7t5uk1eJ5G0Ia1Q+ybsdxHKcqqcA58GofQm8vKTc8LuAkM6uR1Bc4T9IiYA7w00IGUngOuDDavSbDvm4K3KogCt4C+DcwsNiHJb0AbAV0lDQDONnMhmbYP8dxnMolo1TSK5KqduBm1rJA+b3AvSnlHRPnl6Xdi1Kl+eIkA1Js9U+cd0+cjwP6ptS/D7ivQH87ppXn1dmjvjqO4zhVS5lF18VQ1Q7ccRzHcQB34E5ls6plk8hAyk5xKEsFsQ2euy0zW31/kF0+nIULsvk1bIUysQOwZdtvMrPVjQ6Z2aqZk00qg9qa7L6r2aPmZGZrlc7Z9Wu9XYpOE1Evi2Zm87ehRYc2mdhpEspsgVoxuAN3HMdxHI/AHcdxHKcC8UVsjuM4jlOBVGAE7vvAG0maApikXpL+mmEb0yWtkZU9x3EcpwAVmMjFI/DGk5a+dDowLr+ipFZmGa0QcxzHcTLHarNbfLui8Ag8QyT1zeltS7pM0h2ShhFyrLeUdJ2ksZImSTot8cxISf+UNE3SbZKW+/8iaVBUQZsq6dRE+f5Rxew1Sc/EslUk3R3bmiDp0Fi+jaQxccRgUp4giuM4TvXimdiqimQWt/fN7PCUOjsBu5vZvOh0vzaznSW1BUZF5w7Qm6WpUZ8iiJk8lmfrZ2b2laT2BCGTgYQXsL8Be5rZ+5K6xrq/AZ41s59J6gKMkTQcOB24wcweiOlYUxPZOI7jVB1lNjxeDO7AG08xCmBDzGxePN8P2C6nAQ50BjYnKJyNMbP3ACQ9BOzO8g78bEm5l4QN4rNrAiPN7H1YkgUu19YhknKypO2ADYHRwG8kdQMeN7O3G/SJHcdxmiuLfRW6syxzE+cCzsrPPx7zrudPvlhKnX7Armb2naQRBKeslGdzbR1hZm/mlb8u6RXgIGCopFPM7NkGfSLHcZzmSJkNjxeDz4GvOIYCP48a40jaQtIq8V5vSRvHue9jgBfznu0MzIrOeytgl1g+GthL0sbRZm4IfShwVhQ+QdIO8d9NgPfM7K/AEGC7pvigjuM4FYdZcUcZ4Q58xXEnMA0YL2kKcDtLR0BGA9cCU4D3gX/mPfsU0ErSJOBK4GUAM/sCOBV4XNJrwCOx/pVAa2BSbOvKWH4MMCXO3W9FAXEUx3GcqqPERWxxQfGbkt6RdGGBOkfHxcpTJT1Yapd9CL2RpCmAmdkIYEQ8vyzvXi3wf/FYQgySvzOzY1LsdU9cHlCgH/8B/pNXNg9YLlm3mV1DthKnjuM4zYMStpFJagncDOwLzCAsNB5iZtMSdTYHLgJ2M7NZktYqscfuwJ2ldGmXjfjBV/PbZWIHYMLiTpnZylKApNszt2dm673dz8jEziwWZWIHYJMjsxuc6/RYdpsdaudnY2fVdRYwf3Y2f/7mft02EzsAndeaV3+lImnT//jMbC24+oFM7KhtGbuc0lah9wbeSSxGfhg4lDDqmuP/ATeb2SwAM/u8lAbBHfhKJxm1O46zYsjKeTvNByttFfr6wEeJ6xnA9/PqbAEgaRRhC+9lZvZUKY36T7HjOI7jlJaJLU0HNt9gK8L2375AN+AFST3MbHZjG/VFbI7jOI5TWi70GYT8HDm6AZ+k1BlsZoti7o43CQ690TQLBy5p9YSoyH8lfZy4Xk5BXlJXSacnrjeTNC/Wf13SAElNMjoh6WZJH+a2eMWy30n6VQPt7CLpeUlvx1SqT0japshnZ8QMbY7jOA6ECLyYI52xwOZxO3Ab4FjCVt0kg4C9AaJI1RbAe6V0uVk4cDP70sx6xsxotwF/zl2b2cKUR7oS0oomeTM+vy2wMXBE1v2MKxUPAT4FdivBzrrAQ8D5Zra5me0IXAdsmlLXp0kcx3Hqo4RtZFGs6kxCDo7XgX+Y2VRJV0g6JFYbCnwpaRrwHHCemX1ZSpeb/R93SecDP42Xt5vZjYQ911vG/dBPEfZoA+F/hKSxhEUJSDoFOBBoA2wD/AHoCBwPzAMONLPZks4hrDJcBEw2sxNTutMPmAAMBo5j2YQtO0h6jjD0co2Z3R3znd9uZsNiX+4HHiUkcrnbzF5J9Htk4jPfD3wG7EjYznAd8CCwOvAK6fM1juM41UtNaalUzexJ4Mm8sksT5wb8Oh6Z0Cwi8EJI6g2cQFjivyvwC0nbARcSI24zuzDvmfbAzoS3pRzbEJKg7AL8npAVbQfgVSDnqM8HeprZ9oQ3sTSOI0TOA4FD86LjbQl7vXcDrpC0NvBwbBdJ7YC9CC8c2wDj6/n4mwI/MLPzgcuB52Kk/hSwXj3POo7jVBelDaGvFJq1Awf2AAaa2Xdm9i1hDmL3AnVzEfmXhP18UxP3njWzuWb2GTAHeCKWTwa6x/OpwP2SToDlN+RGBbL9CAInswkO+AeJKoPMbH7cGziS8BLxb2DfmH71oNiP5TZrSxon6Q1Jf0wUPxqTxwDsCdwPYGaDgW8LfAeO4zhVidXWFnWUE83dgTdkqDg3B74ZIb/4gYl7SadZm7iuZek0xA8J8++9gXFxvjvJQYSc5lMlTSeMCByXuL+coImZfQeMImT3OYYQkUN4WdgxUbEXcFm0nyMppJJm33Ecx8nhEXjZMRI4XFJ7SR0JmXFeIESgq6Y9YGafENLdXVRsI9FZd4vKXucRZD475FU7DuhvZt1jitRNgAPi0DjAYZLaxtWJewDjYvnDwMkEhz88lt0InCJpF5aS316SkYSpBCQdTIHP7jiOU7VUoANv1ovYzGxM1NceG4tuNbPJsGTYeTJhmPrOvEcfAy6TtGuRTbUCHpS0KuGl6PdxyJ7YVkfCcPn/JPr2bULak9jH/xD2Ev42DtdDmLO+lzAkvig++4mk44DrJK0DfA7MJMx1p/Fb4CFJRxNWP35c5OdyHMepDkpLpbpSaHYOPEVE5A+EleP59fLFQ3om7hlhoRgEpbDkc90S50nHX3BbmJnNIWxdyy/PbS8YWMezC4Dl9myb2UuESD3tmRPzrr8grIDP8b+F2nMcx6lGbLE7cMdxHMepPMpseLwY3IE7S/jv/Lqm0Yung0rbT5lkz9W+yMzWwgXZ/bhnpSAGsMmLN2dip2Wv8zOxA/DR4OyUzSbUzMrMVlajnG06Ls7GENCuS3a2ahZkl6Jh7h+yURADaJWVKGCLMk5BUWYrzIvBHbjjOI7jeATuOI7jOBWIO3DHcRzHqTyspvKG0Jv7PvCCSDJJf09ct5L0haR/lWh3gKTT8soOk/RkoWcaYHuwpNH113Qcx3EaRAXuA69aB07IVNYj5j6HkO0si/3RDxGk5JIcG8uLIk1BLMp/7gh0kbRxsc85juM49WO1VtRRTlSzA4eQOCWXSCUnNAIEIRRJL0maEP/dMpZvI2lM1A6fJClfkH04sFWU/ERSB8Ie7EGSuke98b9JmippWO4FQtIISVdLeh74ZUpfjyDkYH+YxAtCjPj/FJXMfi9pFUl3Sxob+35orNdd0gtRO3y8pD4lf3uO4zjNBY/AK46HgWNjOtPtCFKbOd4A9oyqY5cCV8fy04EbYt70XsCMpEEzqwEeB46ORYcQlMBymdk2B242s22A2SyrO97FzPYys6QoSY7cC8ZDLJtDHYIwfD8z+1/gNwTRk50J4vHXSVqFkK1t36hIdgzw17q/GsdxnCqitsijjKjqIVczmySpO8Eh5s9RdwbujRG2Aa1j+WjgN5K6AY+b2dspph8CrgNuIETL9yXuvW9mE+P5qyxVMwN4JK2fUVp0M+BFMzNJiyX1MLMpscqj8cUBguLZIZLOjdftgA2BT4CbJPUEaghO33Ecx4GyGx4vhmqPwAGGANez/Bz1lYTIuQdwMMERYmYPEqLqecBQSfuk2BwFrCtpe6APy74cJJXNalj2JSpfQSzHMcBqwPtRyaw7y86zJ58TcETUOu9pZhua2evAOcBnwPaEkYM2BdpyHMepPhZbcUcZ4Q4c7gauyImcJOjM0kVt/XOFkjYB3jOzvxKc/3b5BmMu9X8QREieNLP5JfbxOGD/hJLZTiy/UC7HUOAsSYr93SHxeT6NGuE/AfLlTh3HcaoWX8RWgZjZDDO7IeXWH4BrJI1iWWd3DDBF0kRgK5YdHk/yECHafbjA/aKIQ/wbAi8n+vw+8I2k76c8ciVhuH+SpCnxGuAW4CRJLxOGzwtF+47jONWHz4FXDmbWMaVsBDAino9m2XniS2L5NcA1RdifQBjOTpZNB3okrq9PnPctYGc6sH5K+Y7x9JW88nnAaSn132bZ0YKi9c4dx3GaO+UWXRdD1Tpwx3Ecx1lCmUXXxeAO3FnCXe3mZWKnbYbT613nLCeF3mhakZ0S0iyyU+vKSkXs1nHLyd43mi23OqL+SkUy5YId6q9UJCfd9GUmdqYvyk4hrWvLVTKz9eGCbD4fwHptsvvdGTXzjUzstG3Vuv5KDeDrm7KzlZXS3YrEHbjjOI5T9Vh2qrArDHfgjuM4juMRuOM4juNUHpU4hF7nNjIFXpR0QKLsaElPNX3X6uzXBpJSs5Y1ByTNiOIl9daJKmqzG2j/d5J+VVovHcdxmg9WW9xRTtQZgce0nacDj0axjJbAVcD+pTQqqZVZ42cczOwjwn7sFUapfXYcx3HKl3JzzsVQbyKXmG/7CeAC4LfAfWb2rqSTEqpct0hqASDpDknjotrWpTk7MWK8JCZGOVzSOZKmSXpN0v2F2pe0T6wzMaporSJps5hIBUmnSHpM0lBJb0u6JvHsQfGZ1yQNi2Udo4LXmKjWdXAdbZ8i6WEFjfD/xLIL47OTcp8v9mdKVAGbKuk+ST+MKmZvSeoV660haUh89iVJPWL5mpKejn29lcT+8ULfcx19Xq5/sfxSSW9KepogqOI4juNErEZFHeVEsXPglwPjgYVAr+h4Dgf6mNliSXcQUns+CFxoZl8paFM/J+kxM5sW7cw1s90AJH0KbGRmC+sZLj4PONXMXpHUEUhLS7o9QSt7MfCWpBsJSxJuBfYwsw8kdY11LwWeMrP+klYDXpH0dB3pTncFeprZLEkHErKifZ/gZJ9UkOX8HNiSoED2RvyuFphZH0lHABcCRxKyor1iZodI2g8YQMhLfjkh7/rVCvKfp8fvqK7veTnq6N9igupZT0IO9IkEURbHcRwHsNrycs7FUJQDN7O5cc55jpktkNQP2BkYF1Nutwc+itWPk3RytL0esDWQc+DJeeupwP2SBgOD6mh+FPAXSQ8CA81sTmwzyfCcXKekNwhObF2CU/wgfoavYt39gAMkXRivc2pdbxVof5iZ5TaN7gccAEyI1x0J2do+B97JvahImkbQBQeYzNKsZ7sT9cfNbFgcCVgF2BM4MJYPlpSTHq3re06jUP/WIHx384B5kp6ow4bjOE7VUYlD6A1ZhZ7MBCvgbjO7JFlBQXrzl0BvM5sdh8bbJaok82//ENgLOBS4WEEes4Y8zOx3koYQHN9YSX0J8p5J0hS+lFIv1/fDzOzduj5sgT4L+J2Z3bWMQWmzvD7UJq5rWfo95795JK8L9XW577kOCvXv3AL2HcdxHMCs8iLwxoqZDAeOlrQGgKTVJW0IdAK+JQhtrEtw0sshqSXQzcyeJQyRrwl0KFB3UzObFHOQTyAMVRfDKGAfSRtFO7kh9KHA2Qn7DUkTNRQ4OUbNSOqW+w6KZCRwQny2HzDDzObmlR8MrBrrF/qeG9q/kcCPJbWT1An4UQP67DiO0+xpdqvQC2FmkyVdDgyPi6oWEeZtxxGGy6cA7xGcaKF2H5S0KuEl4ve5IfAUzpW0ByGSnQQMIwx519fHzyT9HBisMP78CWF4+XLCkPzk2PY7hFGAejGzJyVtBbwch7S/BY4v5tnIpcA9kiYBc4D/ieW/BR6SdDTwHFHGtI7v+cOG9M/Mxkj6J/AaMJ3g0B3HcZxIqXPgkvYHbiDs1rrTzK7Nu386cAZhlHgOYW3XtOUMNaTNIF3tOHDURodm8sOQaS50ZZc7uWxzoWfUL8+FXjyeC71hlG0u9DnvZvZL/cGO/Yr6+7fR+OHLtRlHld8C9gVmAGOB45IOWlInM/smnh8C/MLMStqS7ZnYnCVc0SYbpzRrTnY/Vh9n+CO6ZdtvMrO1yZGNnX1ano8GZ/O9Z+l033xjYGa2NtvysMxsje25WiZ2WnbI7mXuu/8uzMxW554Zivf02CQzW7UftKu/UjG0zdaBZ0mJEXhvwkLm9wAkPUwY2V3iwHPOO7IKGaxLKhsHLukU4My84pFmdnZa/YzbPhC4Oq/4HTM7sqnbdhzHcVY+JQ5Gr8+yO4RmELbzLoOkM4BfE7bz7lNSi5SRAzezO4E7V1LbTwJProy2HcdxnJVPiRF42sPLvRKY2c3AzZKOBy4GTiql0bJx4I7jOI6zsihxG9kMYIPEdTfCwulCPExINFYS2U3kOQBIqolpT3NH93rqT09sE5tTj83XYrrVPvXY7CLpF4nrvgrpYB3HcZwUStxGNhbYXNLGktoQMmYOSVaIeVJyHAS8XWqfPQLPnnlm1rOpbEr6IXANIQlOIboAvwBuybgfjuM4zZKa2sbHszHV9ZmEXBwtCQm4pkq6AhhnZkOAM2P+j0XALEocPgd34CsESf2BXmZ2Zrz+F3C9mY1ohLlOhP/5xNzwg4HVgNbAxWY2GLgW2FRB8OVp4N9AR0mPAT2AV4ETzfcQOo7jAKXvA09bS2VmlybOf1lSAym4A8+e9tFxArxvZodnaLMdIcd7bvXifOBwM/smDsO/HNPOXgj0SETtfYEdgG0I8zKjgN2AFzPom+M4TsVTieGMO/Dsaeoh9F2B+6JSmYCrJe1JyFS3PrB2ARtjzGxGtDER6I47cMdxHKAZq5E5JbOYZRcMNjorgpmNjtH2mgQFszWBncxskaTpddhOE3xxHMdxgNoqEjNxGsZ0oKekFpI2IGTtaRQx13lL4EugM/B5dN57AxvFat+yVBDFcRzHqYfaWhV1lBMeha0YRgHvE7TBpwDjG/h8cl7IFt6rAAAgAElEQVRdwElmViPpAeAJSeOAicAbAGb2paRRkqYA/yEsYnMcx3EKUIkRuDvwjDGzjillRpQLTbnXva5nY3mqOoiZzQR2LXAvXyVtROJefspax3GcqqYS9cDdgTuO4zhVj69Cdyqa6xZ1yMTO/DY1mdgB+Na+zsxWN7L5fACdHstOMnVCTTbSllnKdmapIPbOm4Mys3XADj/PxM4bcz/OxA5A1zbZLTf54PXPM7O1/+rZ/bwP/PTVTOy0bpmty5l3TXa2fAjdcRzHcSoQH0J3HMdxnAqkxh244ziO41QePoTuOI7jOBVIJQ6hV1QiFwVelHRAouxoSU+t5H5tIOmRldmHfCS1kjS7wL37JWW3SslxHKfCqS3yKCcqKgI3M5N0OvCopOcIGcmuAvYvxa6kVma2uIR+fQQcU0ofHMdxnJWH4RF4k2NmU4AngAuA3wL3mdm7kk6SNEbSREm3SGoBIOkOSeMkTZW0RNpN0gxJl0gaBRwu6RxJ0yS9Jun+Qu1L2ifWmShpvKRVJG2Wy5Qm6RRJj0kaKultSdcknj0oPvOapGGxrKOkAbHvEyQdXEfb20oaG9ueJGmTWH6+pCnxOCvluRbxO5km6QlgjYZ9647jOM2bWivuKCcqKgJPcDkhHelCoFdU5joc6BOF1e8AjgUeBC40s68ktQKek/SYmU2Lduaa2W4Akj4FNjKzhZK61NH2ecCpZvZK1OOen1Jne2BHgojJW5JuJIy+3ArsYWYfSOoa614KPGVm/SWtBrwi6WkzS7P7C4KO+COS2oZuqzchy1tvwojEGEnPA9MSzx0JbEzQAl8v3rutjs/oOI5TVdRUXjxbmQ7czObGOec5ZrZAUj9gZ2CcJID2wEex+nGSTiZ81vWArVnq3JLz1lOB+yUNBurKPDEK+IukB4GBZjYntplkuJl9CyDpDWBDgo73c2b2QfwMX8W6+wEHSLowXreL9d9Kafsl4GJJGwGPm9k7kvaI/fgutjcI2J1lHfiewENmVgvMkDSijs/nOI5TdZTb/HYxVKQDjyTXFAi428wuSVaQtDnwS6C3mc2OQ+NJuc25ifMfAnsBhxKcZA8zWy6lmJn9TtIQ4CBgrKS+QP7ASpp0p1Lq5fp+mJm9W9eHjW3/XdLo2PbTkk6KzxdDmQ3+OI7jlA8+B77yGA4cHXWykbS6pA2BTgRpzW8krUtw0sshqSXQzcyeJQyRrwnpeTclbWpmk8zsGmACsGWRfRwF7BOjZxJD6EOBsxP2C+bDlLSJmb1jZjcQFMa2A0YS5vDbxyH9Q4EX8h4dCRwb58LXJ7yoOI7jOBFfhb6SMLPJki4HhsfFa4uA04FxhKHkKcB7BCeaRivgQUmrEl5qfp8bAk/h3DhsXQtMAoYRhrzr6+Nnkn4ODFYYc/8EOIAwn/8XSZNj2+8QnHAax0s6Ln6+T4CL48jCQ8DYWOfW+H0k/98+BuxN+B7eJDh0x3EcJ1JuzrkYZJUoweI0CeM3ODSTH4bllwQ0no13+yYzWzVzsvsVrU1bYthILKNunfFOXWsvG8bNm6WmEGgUx7/Vrv5KRfKfCbdmZmveb7IRRml9RKH37Yaz4J5HM7PVqnvX+isVSe3n2fwettpu00zs5Ojwy9sy+2vzxDrHFfX37+D/PlQ2Y+3NIgJ3HMdpCFk5b6f5UFuBc+DuwAsg6RTgzLzikWZ2dlr9jNs+ELg6r/gdMzuyqdt2HMepRipxLNodeAHM7E7gzpXU9pPAkyujbcdxnGqkEufAm8sq9JWOpLslfS5pShF1+0rqU+Bef0lfxGxrEyXdF8sHSKo3Ape0paQR8dnXY1KbXJtfJ+wOb+hndBzHaa7USkUd5YRH4NkxALgJuK+Iun2BOYTELGk8Ymb5w/epSGqZt1/9r8CfzWxwvL9t4t4LZvajYuw6juNUE5U4hO4ReEaY2Ujgq/xySWfHHOSTJD0sqTthi9s5MRLeo6FtSZou6VJJLwJH5d1eF5iR6Nfkhtp3HMepNharuKOc8Ai86bkQ2DimfO0S923fRkgDe32BZ46RtHs8v8HM7kmpM9/Mdk8p/zPwrKSXCHvU7zGz3J6gPRRFV4BHzeyqRn4mx3GcZoWvQnfSmAQ8EHOU15VjPUkxQ+ip+uNmdo+koQSJ1UOB0yRtH2/7ELrjOE4KPoTupHEQcDOwE/BqXoa0Uphb6IaZfWJmd5vZoQRFtB4Ztek4jtMsqVVxRznhDrwJiWldNzCz54DzgS5AR0J+9lWbqM39JbWO5+sAqwMfN0VbjuM4zYVKzIXuDjwjYj7y0cCWkmZECdOWBInSyQThkz/H+egnCAIkjVrEVg/7AVMkvUYQSjnPzP6bcRuO4zjNCivyKEQMnt6U9E5CHjp5v62kR+L9V+KC5pLwOfCMMLPjCtxabqGZmb1FUBJLszOAsCUtv7x/4rx7Hf34NfDrlPIRwIhCzzmO41Qzpawwj4qWNwP7EnYBjZU0xMymJaqdDMwys80kHQv8Hjim8a16BO44juM4pQ6h9yaku37PzBYCD7O8quShwL3x/DHgB1GZstF4BO4soUWLbNZhtszIDsDs17P7Ea2tyW4FSut2NfVXKpI2HRdnYmf6olmZ2AFo2SG77+qNudktwchKhKT9VRmqml10ema22l+03Mhro1lw4x8zs9VyvdWyMdSifGNGK+1Hfn3go8T1DOD7heqY2WJJXxPWKM1sbKPuwB3HcZyqp8QFamnuPz+SKaZOgyjf1yHHcRzHWUGUOIQ+A9ggcd0N+KRQnbiduDMp2Tsbgjtwx3Ecp+qpUXFHAcYCm0vaWFIb4FhgSF6dIcBJ8fxI4Fkz8wg8CyStLelBSe9JelXSaEmHr+x+JZF0g6SP4/5yx3EcJyNKicDNbDFwJmHr7uvAP8xsqqQrJB0Sq90FrC7pHcJOoZIXPPgcOBBXAg4C7jWz42PZRsAhdT64rI18VbBMiU77cMIiiD1J2RImqVX8QXIcx3EaQKlJWszsSeDJvLJLE+fzWV58qiQ8kgvsAyw0s9tyBWb2gZndCCCpu6QXJI2PR59Y3lfSc5IeBCbHskExgp8q6dScPUknS3oranX/TdJNsXxNSQMljY3HbgX6uDcwBbgVOC5h9zJJd0gaBtwnqaWk66KtSZJOi/U6Snom9n+ypPwtDo7jOFVLqYlcVgYegQe2AcbXcf9zYF8zmy9pc+AhoFe81xvoYWbvx+ufmdlXktoTNvMPBNoClwA7EtKoPgu8FuvfQMjQ9qKkDQlDMN9L6cNxsd3BwNWSWpvZonhvJ2B3M5sXXxq+NrOdJbUFRkXn/hFwuJl9I2kN4OWYaKDcfiYdx3FWOOWW57wY3IGnIOlmQga1hWa2M9AauElST6AG2CJRfUzCeQOcnZg73wDYHFgHeN7Mvor2H03Y6AdsndjP30nSqmb2baI/bYADgXPM7FtJrxBSpv47VhliZvPi+X7AdpKOjNedYx9mEBz/noTRovWBtQFPs+o4TtVTbnnOi8EdeGAqcETuwszOiFHquFh0DvAZsD1h2mF+4tklqmCS+hIc8q5m9p2kEUA70vf/5WgR68+ro87+BEc8OTr6DsB3LHXgSWUyAWeZ2dCkAUn9gTWBncxskaTpsW+O4zhVT03ZDZDXj8+BB54F2klKpnnqkDjvDHxqZrXATwgiJWl0JuS6/U7SVsAusXwMsJek1eL+vyMSzwwjrF4EIEb5+RwHnGJm3WMe9I2B/SR1SKk7FPh5QpFsC0mrxL59Hp333sBGBT6D4zhO1eFqZBVKnAc+jOBk35c0hpCz9oJY5RbgJEkvE4a+C2lxPwW0kjQJuBJ4Odr/GLgaeAUYDkwDvo7PnA30igvOpgHL5GWMTvqHLI22MbO5wIvAwSl9uDPaHy9pCnA7YaTlgdjOOOAE4I0ivhrHcZyqwBexVTBm9ilh833avbdZVj3solg+gsR2LjNbABxQoIkHzeyOGIH/kxB5Y2YzqUORxsy+A7qmlP+4QP1a4P/ikc+uhdpxHMepZsotui4Gd+Arjssk9SPMOw8j7DtvltRkuJxz9qz2bHZE/fWKsjVqTjaGgLlft83MVrsu2Wzdf3IbceIbabMqDef4iXDnOgszsdW1zaqZ2AFofUQ2ux8Xj3+KRY9m9yvY/prb6q9UJHPPODkTO1abXbzYYo1sUlzYJ5/RYr21M7GVNb4K3SmImZ27svtQiWTlvKuBrJw3ZOe8y5Xm7rzLlXJ13lCZi9jcgTuO4zhVjw+hO47jOE4FUusReHkjaXXgmXi5DiEpyxfxureZLcyr3xU4OplitYDdVsBMM+uSUr6AmGY1tneGmb1c4ufYEVjLzJ5KudeRsBJ9G8Ke8FmEVewL8/oCcLCZfZRvw3Ecp9qoPPddZQ7czL4EekLIIQ7MMbPr63ikK2FbVymTXN+aWa7Ng4CrgB+UYA9CStYehG1r+ZwDfGhmx8Y2twJyKVeX9MVxHMdZSiUOofs+8Iik8yVNicdZsfhaYEtJEyVdK6mTpGejIMgkST9qYDOdCBExktaX9GK0PUVSH0mtJM2OYiTjJQ2V9H1JzyvInB4Yc6xfCpwQnz0yr411gY9zF2b2RiJnuuM4jpNCLVbUUU5UVQReCEm9CclNehOyrI2R9DxBr3WzRATdGjg05iNfCxgF/Kse86tKmkjYPrYOQVUM4ETgCTP7vaSWQPtY3hkYZmbnSXoCuIwQsW8P3G5mvSRdQRBQ+VVKe3cBT0k6hjBdcK+ZvZPXF4B3zCzf+TuO41QlTaYF3YS4Aw/sAQyMSVOQNIggZjIsr56A30vanTDiskHMmT67DtvJIfTdgfuAbYGxwO2S2gGDzOy1OGc+z8yejs9OJiiLLZY0Gehe3wcxs1clbUIQNekHjIsvKO/hQ+iO4zipWJlF18XgQ+iBYrfw/5QQIe8YHeFMGiAIYmYvAutJ6mpmzwJ9gU+BBySdEKslF9LVEhae5c6LeuEys2/NbKCZ/Rx4mMLZ4RzHcRw8F3olMxI4XFL7uIr7UOAFgnZ3Mo1UThBksaR9CZKcRSNpG8LPwCxJGwH/NbM7gAHADg0wld+vZBu7S+oSz9sStMU/aEg/Hcdxqg2fA69QzGyMpIcIw9oAt5rZZABJ4+Lw9b+BPwFPREGQ8cDbRZhPzjsD/NTMTNIPgF9LWgTMIcyJF8uzwHmSJgBXmdljiXubA7dG2dEWwBPAYAorqDmO41Q95eWai6NqHbiZXZZ3/QfgDyn18oVGvl/AZJf8AjNbTAHHaWZ3A3fXZcfMLs6z1SWefwH0KmD3HuCelFtLnnccx3GWpdyi62KoWgfuOI7jODk8F7pT0SxanM0oe/u22W07/+iJ7JaNrNI5O7mhzmvNy8xWzYJs+vXhgi8zsQPQuWd2gzUfvP55ZrYW3PNoJnbaX3RhJnYgWwGSVW6+KzNb8y/+RWa21KljNoZalO+yq3JboFYM7sAdx3GcqqcSt5G5A3ccx3GqHo/AHcdxHKcCqbXKi8DLd0JiBSNpbUkPxpzjr0oaLenwld2vHJJ6Sxop6U1Jb0i6U1KHld0vx3Gc5kANVtRRTngEDihsmh5EyBt+fCzbCDikATZamlmTpNOVtDbwKHCsmY2O/T2CkMzluyKebxW3oTmO4zgpVOIcuEfggX2AhUndbzP7wMxuBJDUXdILUSFsvKQ+sbyvpOckPUjU2ZY0KEbwUyWdmrMn6WRJb0kaIelvkm6K5WtKGihpbDx2S+nfGYSXi9Gxb2Zmj5nZZzEyf0nShPjvltFuf0mPRkGUYZLWjRF8Tv1sjyb5Jh3HcSqQSkyl6hF4YBtCZrVCfA7sa2bzJW0OPMTSRCq9Ccpg78frn5nZV1H2c6ykgUBb4BKCjve3hExqr8X6NwB/NrMXJW0IDCWkP03SA7i3QN/eAPaM6V37AVcTonOAXYHtYn/+FxhqZldF9TMffnccx4l4IpdmgqSbCWpkC81sZ6A1cJOkngTVuS0S1ccknDfA2Ym58w0IqU3XAZ43s6+i/UcTNvoBW8fUpwCdJK1qZt8W2d3OwL3xxcJiX3M8nWuTkCb27iiJOsjMJuI4juMAPoReyUwlRMcAmNkZBA3uNWPROcBnBE3uXkCbxLNzcyeS+hIc8q5mtj0wgaBWVlemjhaxfs94rJ/ivKcCOxV4/krgOTPrARzMsupoS/pmZiOBPYGPgb9L+mkdfXIcx6kqKnEI3R144FmgnaSfJ8qSQ8ydgU/NrBb4CYWFQToDs8zsO0lbAbvE8jHAXpJWi5rfRySeGQacmbuIUX4+NwEnSfp+ot6JktaJbX4ci/sX+oBxUd7nZvY34C4SLyyO4zjVTo3VFnWUE+7ACYvCgMMITvZ9SWMIc84XxCq3EBzoy4Sh77nplngKaCVpEiEyfjna/5gwN/0KMByYBnwdnzkb6CVpkqRpwOkp/fsMOBa4Pm4jex3YA/iGIMByjaRR1K041heYGBXMjiDMvTuO4zg0XQQuqaukpyW9Hf9dLaXORnHx88S4AHo5P5CGz4FHzOxTgpNMu/c2sF2i6KJYPgIYkai3ADigQBMPmtkdMQL/JyHyxsxmAvmKZ2l9GE1w2vmMZtk5+Uti/QEEnfHc8/dSeCGc4zhOVdOEc+AXAs+Y2bWSLozXF+TV+RToY2YLJHUEpkgaYmaf1GXYI/AVx2VRF3wK8D5h37njOI5TBtRiRR2N4FCWBk/3EkZ7l8HMFsYAEMKupaJ8s0fgKwgzO3dl96E+1lnvm0zsfP1l+0zsAHw9t139lYpkvV0W1F+pSNr0Pz4zW3P/8EAmdtZrk52CWKsem2Rma//Vs9ux2Kp7Nop5C278YyZ2AKw2u8gtSwWxdr+7JTNbC//6f5nYUffumdhpCqzpUqmuHUd4MbNPJa2VVknSBsC/gc2A8+qLvsEduOM4juOUtMJc0nDCduF8flOsDTP7CNhO0nrAIEmPxfVPBXEH7jiO41Q9NSW4cDPrV+iepM8krRuj73UJicHqsvWJpKmENU+P1VXX58Adx3GcqsfMijoawRDgpHh+EjA4v4KkbjF7J3GV+m7Am/UZdgdeBKUolUmak1J2ekMTqUhqJWmmpGsa8pzjOI5TP024iO1aYF9JbwP7xmsk9ZJ0Z6zzPeAVSa8BzwPXm9nk+gz7EHo9NESprFjVr6RoSgPYj/BGdrSk/7OUV8GmVERzHMdpzjTVNjIz+5KQ2TO/fBxwSjx/mmW3KheFR+D1U59S2TKqX8UYlHSZpHMlfS8mjcmVd49JYNI4jpB85UOWZnhD0nRJl0p6EThK0qaSnoojBS/EjHBIOljSK1G1bLiCRKnjOI4D1JoVdZQTHoHXT31KZZBQ/WqIYTN7XVIbSZuY2XuEhC7/yK8X50Z+AJwGdCE489GJKvPNbPdY9xngdDN7O6ZevYXwEvIisIuZmaRTgPOB/21Ifx3HcZorNRUoZuIOvIGkKJXBsqpfDeUfwNGEeZFjSM/K9iOCYMl3UZ70EknnJIbLH4l96wj0AR5NqJu1jf92Ax6JqyDbEJLJOI7jOFSmnKgPoddPfUplUDg3ejE8QpjX3iKYt7dT6hwH9JM0HXgVWB3YO6X9FsDshLJZTzPLaYvfCNxkZtsSIvnsMqQ4juNUOE24Cr3JcAdeP/UplZWEmb1L0Bi/hBhJJ5HUiRDxb2hm3c2sO3AGwann2/oGeF/SUfFZSdo+3k6qlp2U/6zjOE4104Sr0JsMd+D1UIRSWX10kDQjcfw6pc4jwImkzH8DPwaeTeTJhbCP8BBJbVPqnwCcHLcjTCXk4QW4jDC0/gIws8i+O47jVAVW5H/lhM+BF0E9SmUDSKh+pdyv9yXJzK4Hri/Wfpxvzw3hd8+79z6wf4qdwaQkEHAcx3GaNBd6k+EO3FnCovnZCEW0bJmd6P0aXUpZXrAsi2bWu0W/aBZcnY0ACUCrTtnYGTXzjWwMAbUfZLdEYuCnr2Zm62+fb52JnZbrLSfJ3GharJFd6gV16piZrawESADanH11JnYWvzY8EztNQY1l93drReEO3HEcx6l6ym1+uxjcgTuO4zhVT7nNbxeDO3DHcRyn6im3LGvF4KvQE0RFmMGS3pb0rqQbJLVZAe3Oif92lzSlQJ1tJD0r6a3Yt8slNfr/X0zBukZjn3ccx2lOVOIqdHfgkSha8jgwyMw2B7YAOgJXZWC7pJGOmEp1CHCtmW0BbAv0Bn5Zat8cx3GcysyF7g58KfsQcorfAxDTlJ4D/ExShygEsk2usqQRknaStIqkuyWNjUIhh8b7y4icSOoo6RlJ4yVNztUrkuOBUWY2LPbtO+BM4LzY1mWSzk30bYqk7vF8UBQ2mSrp1MZ/PY7jOM2XGqst6ignfA58KdsQ0pQuwcy+kfQhsBnwMCFn+W9jPvH1zOxVSVcTEq38TFIXYIyk3F6JJSInMQo/PNpcA3hZ0pA0WdAi+/aupPaxzbr4WWy/PTBW0sAob+c4juNEym14vBg8Al+KIPX/YK78H8BRsexo4NF4vh9woaSJwAhCjvEN472kyImAq6Nc6HBgfaBYSc+6+lYfZ8esbC8DGwCbF9mm4zhO1VCJQ+gegS9lKnBEsiDmId8AeDcqgX0paTuCYthpuWrAEWb2Zt6z32dZkZMTCNnTdjKzRVGYpNhsGVOBPfPsbwLMNLPZkhaz7MtYu1inL9AP2DX2f0QD2nQcx6kaPAKvbJ4h5C3/KYCklsAfgQFxzhnCMPr5QGczmxzLhgJnxUVwSNqhgP3OwOfRee8NbNSAvj0A7C6pX2yjPfBX4Lfx/nSiYpqkHYGNE23Ois57K2CXBrTpOI5TNZjVFnWUE+7AI3Eu+nDgKElvA28B84FkPsLHCDnRk6IjVwKtgUlxC9iVBZp4AOglaRwhGi8676WZzQMOAX4j6S2CGMkoM8vl8xwIdI3D+D+PfQd4CmgVh+2vJAyjO47jOHn4IrYKx8w+Ag6u4/5n5H1n0bmellJ3AAkREjObSVjUlma3Y/x3OtCjQJ0pRA1wSYcBf5L0oJl9EPuwX4FuH1DAXvcC9R3HcaoOT6XqrBDMbBAwaGX3w3Ecp7ngamRORdOqbTaqSm06ZKf61XGT7H6pWnTILqme2mb4q9OimM0E9dO2VetM7ARj2dlq3TK776rVdptmY6hFdrOH2fzfi2TZr+7dM7OVlYpYq+37ZWKnKSi3FebF4A7ccRzHqXoqcRW6O3DHcRyn6vEhdMdxHMepQMpthXkx+DayDJH0m5hzfJKkiTGZS6G6AyQdWY+9AZLej7bGS0pdxS7p9Nz+dcdxHKfheCa2KiY61x8BO5rZgpjvPItVU+eZ2WOS9gNuB7bLa7eVmd2WQTuO4zhViw+hVzfrElKbLoAl+76RdClhb3l74CXgtHwBE0k7AX8iyJfOBPqb2ad59kcSRFWIKVFfAnYDhkhaFZhjZtdL2gy4jZC2tQY4KgqfnEfI4d4W+KeZ/RbHcRwHqMx94D6Enh3DgA0kvSXpFkl7xfKbzGxnM+tBcOI/Sj4kqTVwI3Ckme0E3E26BvnBwOTEdRcz28vM/phX7wHgZjPbHugDfBqj980JGuI9gZ0k7YnjOI4DhAi8mKOc8Ag8I8xsToyk9yBkTHtE0oXAt5LOBzoAXQnCJE8kHt2SkH3t6ZhOvSWQjL6vk3Qx8AVwcqL8kfw+xEh8fTP7Z+zT/Fi+HyFT24RYtSPBoY8s5TM7juM0FypxEZs78AwxsxqCpOgISZMJKVa3A3qZ2UeSLmN5NTABU80sdYEacQ48pXxuSlmhnBICrjGz2+v5CI7jOFVJuS1QKwYfQs8ISVtKSmpt9wRyEqMzJXUE0ladvwmsmVthLqm1pG0a0wcz+waYEXOlI6mtpA4ExbSfxT4gaX1JazWmDcdxnOaID6FXNx2BGyV1ARYD7wCnArMJc9fTgbH5D5nZwrid7K+SOhP+n/yFMNTeGH4C3C7pCmARYRHbMEnfA0bHYfo5wInA541sw3Ecp1nRVJnYJHUlTHl2J/iBo81sVkq9DYE7gQ0AAw6MAleFbZfbG4Wz8viw1w8y+WFo0TK7n6lsc6Fn975ajrnQuw14s/5KRfLxWT0zs7XmX8ZlZuvLa1PF9RpOhjnHMyXLXOhdV8/MFmutn4mZrHOht15jk8xS0bdp262oPzYLF8xoUJuS/gB8ZWbXxnVRq5nZBSn1RgBXmdnTcbS01sy+q8t2mf4UO47jOM6KowmH0A8F7o3n9wKH5VeQtDXQysyejn2ZU5/zBo/AHcdxHKfJkDTbzLokrmeZ2Wp5dQ4DTgEWAhsDw4EL48LogvgcuOM4juOUgKThwDopt35TpIlWhC3IOwAfEubM+wN31feQ4ziO4ziNxMwKTu5L+kzSumb2qaR1SV88PAOYYGbvxWcGAbtQjwP3OXDHcRzHaTqGACfF85OAwSl1xgKrSVozXu8DTKvPsM+BO47jOE4TIWl14B/AhoTh8aPM7CtJvYDTzeyUWG9f4I+ExFuvAqea2cI6bbsDd9KQtKGZfZiRrVZmtjgLW/W0s29uFeeKRlKnuu7HJDsrHEln13XfzP7aCJt9CHtal0zBmdl9DXj+kHr6NKShfcqamOhoSdbEhvwuSFrfzD7OuD+7ARPNbK6kE4EdgRvM7IMs22lAf7rWdd/MvlpRfalmfA7cKcQgwh8JJA00syNKsDUmYetGMzsrg/6lcRfhLbfBSNoUmBGlYPsSUuDeZ2azizQxlZB8QcB6wLfxvCPwcUP7JWkXgsjN9wiytC2BuWZW54tCCrkhuZyYTS4P/4+A5xtoC0l/BzYFJhLU7iB87qIdOHBU/HcNguDOiHi9V+xT0Q5c0jAz2y+eX2Rm1zSgH2n2DiFEQesR5io3Al4HGpId8QmW/rz/w8yOLqVPkVuB7SVtD5xP+Fm/j/CdNYg4THsBsDXLvqTs0wAzr7L05z0fAzZpaL9i33YHNjeze2I/O5rZ+2miLEIAACAASURBVI2xVQ24A3cKkfzFbNQvYwFbu5VkSCr0x11AKZkrBgK9ohzrXQQn8iBwYDEPm9kGsX+3AE/lokhJBwONUX67CTgWeBToBfyUKCfbEMzsktiPoUDP3EiApEtIEcQpgl7A1vmSuA3s009iH4ZEWx/H6/WBho4IrJk4PwooyYEDVxIWD/3/9s473NGyWvu/e4ZeFSkWivQmneEAKgpS5CBdgaF9CqIIHIqfIFbEc+AgKPIJiKJYQEFBFJE6Ir23oYoeQDnSVKQjUgbu74/1ZCY7JNl53rx7svfs53dduSZ5M+/KSnbyPmWtda/LbK8laWNgcqaN5u/78h3/Vx7TbFvStsTK+zRJ/2fYs9rzU+JvvxWwLxGXfSLHgO2lK752RyQdQXy/VgR+CMwO/IQ+rxmzMmUAL3TCHe73a6tf3kvIwL7QclzECrMqr9ueJml74ATbJ0qaOuxZb2Q92/s1Htj+TbowZWP7AUkTUy3oDyVdX8VOYingpabHLxP1prncQ5TLtParr8IyLVvNjxEX7xzqjgG+avtJSRMkTbB9haSv9eFTXf49L+lzxHd/I0kTiQGuCm9JE4CDbF8FXCUpezemgaQ3ExOV5tV8lU6H2xNlVLcnG4+lDouFDpQBvNCJNSQ9RwyMczfdB3DmVu5Kku5K5y+b7pMe2/bqGbZuBF5MF54hSOpHS/RVSZOJ1cjW6ViVC+RTSS7xJ8TFe3fgDbrHPfCipDmAO5IU4+PAvBXsNDgTuEnSucmvHYiVWE9I+k06b37g95JuJiYBANjuGtfuwNWSLgTOSrZ3Ib/F7TJpJa+m+9Op4NczScbyGuCnkv5O9DbIYQ1JTyWf5k/3Ycb3vWv8uAM7A7sCe9v+a9LNPq6CHYgeCQCPS9qKmDgtXsWQpI8DB6Xz7yB2L24gsqhzeSXtMjjZ7uf7Pi4oSWyFEUfSUt2eH1QiTjNJynBf4AbbZ0laGtjZ9jGZdhYGjiS2zU0MSF+x/Y9MO0sBfyPi34cACwLftv1Ajp0Wm5OYsZ1/te03NNfpcm7XWGu7CVUPNkVse7+34RPwi5zt+br9SoPGv4gS292Iz/2ntp/MsDFxGJ+6qmt1sHdpt1rjTHsfIiYoSxB5FgsAR1ZJHlS0TZ4E3Gh7TUkrJVs7V7D1GWIlvxkRCtkLONP2ibm2xgtlAC+0JbUhfdX2q+nxikQ8+CHbv+rT9luIgeQvtm/r29makDQ3sKTtSiv5dKE9yvbho8GfNvbWB1awfXr6G8ybW2kg6WutjRjaHRsUkmYH3gU8artSt700eVre9mXpdzDR9vMZ589F/HZeS4+XA7Ykfju/6XpyZ5vnA3vYfrbK+SOFpFtsT5J0B/BvKQn0DtuVuuGkUqrNid2KSwdVVTJWKEIuhU5cQpQKNS5ANxDJbAdIyl2VXiDpXen+24g46l7AGZIOrtPpqqRkszuI942kNbskzLUlXbD7icPX6k+LvS8CRwBfTIfmIrbVc9mszbGsFmGSnpb0VJvb003bzb3a+o6kVdP9BYE7iezsqSkkkoWkfYBfAN9Nh95BVGTkcCmRqd+obriZyPj+v5KOzvUp8RJwt6TTJH2rcatiSNIKkn4n6Z70ePX0/ajCI4oWyucBv5X0a2JLvopfSwPX2D7U9meAayW9s6Jf44KyAi+0RdLdtldL9/8TWMj2/ikue1vjuR5t3Wu7cZH9PLCS7T1Tgsp1mTHwEUHSbUTc7krba6Vjd+e8z3TO14mJzjnAPxvHc7cnO/hzV9XPKq2Q1gJur2JP0qeA/Yj39mDTU/MD19veLcOX2raYW75bBwPvt72dpLcCFzfea4a9O4hJ2E1Vvwctv52vAgvb3k/SnMCtud+pZKdtxrntH7c7Poytq4BDge82vcd7bL8r11aL3fcRIYdLPIwASYfzbwU2bJybrjXX2Z7Uj1+zMiWJrdCJ5pndJqSEGduvSHo909arTfc/AHwv2Xo+15ZqFJhpYZrtZyMsO50qs9vFiIG7ufzMZNQ2d/GnH15uSRCaJ/P8M4GLidhkc4jgeWeKdjQP0Gln5j3p4dW2h5WPbKF5oNiMmDiREr0yTQHxOb3SOFfSbOR/D1p/O99IPr1c4bdDOjd7oO7CPLZvbvl8KgktpWS6Bo167bcSimO5zNY88Ke/wxxV/BovlAG80Im70mryUaL+eApA2i7L5WFJ/0EI9q/NjG3hucnP9K5TYKaZeyTtCkyUtDxwIJBdttWocR4t/jTxS0knAwtK+hiwN/CDXk9OsddnSTXRmqFUNp+k+apMqiQdQKzqG1vU50g62fa3M8w8k5KyHiXqhfdOtmcD5s71iSip+jxRebFZ8i83bn1vCjM9CqzAjN/OgrQXPhmW9B34b94ovlJFo+EfaWu/MZn7MNXLAi9khqDLXERp4h/JE75p8ISkbTxDQ2FbICv5c9zhHpuYl9v4uhEXv8OB/wes0XR8QyKZJsfWosB3CBH/zZuObwx8JtPW1Hb3a3i/8wBHEU0FbgH+C5gr4/yVgQ81PT4OODXd1pzZ/nSwuSXwTeAEYMuKNrYG7id2Gf4MvA7cW9HWXYTSVuPxfMBdmTZWICaEdwAfbTq+BfCNCj5NAPYhVvK/SPeVaWNeItfgZGDtpuPvbvYx0+a1xO7VXURN/1eIbO8qtpYh+k2/SEwyrgWW6ue71WR7bWJrvsq5yxJlon8BHiYmrMvV4deseisx8MKYQtLtttduvT9oUvLOcbavTY/vIy6y8wDb2N4+w9ZE4Bjbh46Er/0g6U5iW3iIUpntT1SwdTewru2X0+PKMeI6SJ/7j23vPojX74ak22yv0xJfv8b2e4c7t8XOBODDts9WlMxNcEaGfY+v0dfvUlGHr7r9mhUpW+iFsUadAjPTkfRbokvQM+nxm4Gf2d6iRxPvaAzeiRds/zzZ2ivHF9uvSVon55xOSHqa9jHcqqIidSiVNTgDuFEhLgOhxFVnrDeL9LkvImkOV0jCGmFeSoPv/Sn08Cixs5WF7dfT+Wfb/uewJwyDpE83PZxArMCzZFkl7W77Jy22aMTobR/fr5+zKmUAL4wpbHfNYO6Dhd3UuMT20ynO2ytDJB89NHM2+0JLlEGdzxuz2X+ZaWfhCq/djYZS2dVUVyoDwPaxkq4ghFxEtFbsWVxmhHgIuC599s2f+6AHkYOJ3ZwDCb32jZnRYzqX3ypEU37O0PdYpYNY8/d+GhETP7fD/+1EQ3GtyKZmUgbwwkxD0kKtFwlJSzuj25BGTmDm9eYMd4WYR0586XFJ69q+tcXf9QhFtVwWAp5kqCSlgdwBfDg5ytw2p9sSSmWHMEOp7KuZNgBINb532r5F0nuBSZL+6MzWq83bwlX8aOGxdJvAjAGlUpxR0g6tE652x4axMRcwf9PE5gXgY5IWI5IKq9DYEdq/6VilDmK2j1RqpZv7d2uy8d0UvnjO9jer2BivlBh4oSuSViBqRpdiaP/nbK1jSdcRyVONjlirEFt5PdefSrqa0IO+XyEwczOh6b0KcIsrqqBJ+iCRcNaQ3twI+ITtS3s8f32i1Oo0UjMGYB0iK3qy7Rur+NXyGpNyV6iSHqZL20fbldqvJtsLA0+64kUk1VxPIlqtTgEuApa2/aEKtq62XaXr23B25wK2tn1OhXPfEAtuxLIzbJxK1FW3TgR2A95j+1O5fnV4neywgaLu/lAi+1xExviXbf9M0hK2H860d4XtjXPOGe+UAbzQlZS09B2i/+/0+l1XkEBVNE44jGhjuCKhmLWb7TsybNQmMNPG9sJEMwYRmui5+uVvJbY4GyU09wIn2q7cuStNcnYhyreetb1uVVv9kCYoxwBPEVu4ZxDb8xOAPW1fUsHm7bbXlnQoUX/9LUlTnSm+kmx9idgZ6HtbOK0GNyc+882Ba21/OOP8LYAPEs1HmhvGLEBUdPQsTCLp97ZX6fDcdBGbKiiCzBsnP7e2vVjGuV8hBG8OsP2ndGwZomrlWmAf21ntbyUdRezotP4Nb+940jinbKEXhmOa7VPqMGT7QoVW9RRie3I72/fnmmm636/ATCuvAX8nVhSrSMIZbREd4iE3AEc0tvirkLbvJ6fbNGL3Y13bD1W1mezuQrTwPFrS4sBiGROxk4DPExfYy4mdlBsVzSvOItX2ZzJN0keAPYDt0rGqLTL73haWtBExmG1F7Oy8m/i8Xsz05e+EXPBLxCSuwfMMFcHpya0uz1WSwpb0b8T73J4I1exPrKRz2A1Yzfb0FrW2/yRpJyKJbdcKrm2Y/m0OyZhqnc3GBWUFXmiLpEZ28oHEBelXDG0f2fPKRtKJvHHg/RORMITtAzNs/QT4K5GFezix5fqiQmDmKttr9GqrxW7btoi5oQJJZxDb75cDPyPKrXKkQa8nBsmfEVnw90v6s+0qvbub7Z5EDI4b2V45/X0v7XU1qKYGFZLus71y03NVV83vIoRSrk9ZyEsDu9o+KtdWv0h6hKg/PgU4z6ES2NfnLmmu5gGuoo2rgENt39xyfBJR595z2CCtcHci3udZxG/61irvMeUqtO3d3u25Qr2UFXihE7cxNHbaPEPPTXi5teVxPx3I9iEG2ncSojCN1dEqwNf7sHsQM9oibpxWlkfmGrG9h6KeeStiVXiqpItt79ujiSeIScRiwCKEaEods+wN03b11OTnU8qTqWze3fhXy3OV/LN9DzGANx7/mRCvAUDS2bZ36mZD0g7DvEavCWPnErsAOwOvKer6+/3c35/CPI38kSqle4cCZ0v6ETN+N+sCexKhlRw+QaiknQJcYPslJWndCjwi6QO2f9d8UNImxOS6Z9KOwKmEkMvdwF6276vo17iirMALMw2FcMRLntFmcSIwZ4UtytpR/W0RG3HUvYFNci7aCsnNHYkt9OWANwFbtK7CMv25CdiAWHGtrWgnelmvK2dJrxFxSREqfY2/mQiFuKpb391ec9iVvaQfdnnatnuuwW+KCU8mKhsWIP5+F9l+oVc7TfYeIFa8d9M0AcrZkUl2FiW2uRvJnvcCJzmzXWpLbH8T4ApgU2AJ21mlgIoOcL8m4t2Nyf4kIuywjTM07RVNTD5HlCZuA3zcvesvjGvKAF7oiqT9gZ96qMDJZOfpVTds3Qhs2rgYKuqJp9jesPuZI4+kXwEfI+ptNwGeBma3/e9dT3yjnU2JldGmwHVEQk6l7kzJ3qLEqnAycaFdoqKdPYmY57qEBvpOhBTnz6rYmxm0y+Keia89O5GINpnY6cmup5d0JTF56zc3o3ZSdv2HiPf3HuB3trPi1snGrkTSpoiJxU9zwwatf+dB/t3HGmUAL3Sl3Sq0j5hnO1uVV7kjhfpoiyjpF0T8+kLbrVvN/fq1lO3/7eP8VYmJhYjV9z21OTcC9HIhl/Qj2x9N9/+P6+3a1XiNuav8LRUaAEcAVzI0f6RSH++RQlHHvf1IfHY9vv6fgM80Hfp68+OMMMi4owzgha5IuosofWl0LppINJzILl9R1IH/R6MsRCEXepLtDer0OdOnrlvbFcuQFiG2E01sWWdJS9ZJujh3xBXFN2YGPW6hT/8/o23lJuliopVu6xb6lwbm1CikzjDIeKMksRWGYwqRRPMdYkDal2olQxDb0+dIeiw9fhuxPZyN6hOYaU3WayZbnUrSR4k66auSze9I+vKgVjfEtmbj/b2dKGWCKON7lBBRGRgpkW5J2w+0efrzPZgYzSuQRZ0h2tILkuZ1DRrmownbHxu0D2OVsgIvdEUhU/kJZmy9TgG+n5uI02RvdkLERcAfqtZLq0aBmTqR9EdCIeuJ9Hhh4Lqcspq0y3Gga5SVlPRtIiTQ6LW8NVFSNrCOZwphn+OBOWwvLWlNooY+p3Pb34mQhYjJ4JCYfk6JYovdvgdKSccSn/nl/dhJtjYEvk+0X11S0hrAJ23vN8ypYwaFPOzRwNttb6kQMdrA9mkDdm3UUgbwQkdUc3tFhY75p4new/tIWh5Y0fYFFWxlSVL2aHMHIqHHwDW2z6tg43IiY7yh1T47cRH/QKadK22/P/f1u9i71S0qbu2OdTn/ebp3NcvuAifpNqLH9RVN2+DTlfZ6tNG1oUfuzkedA6WiE9yCRMb+K1QrI2vYugn4MHB+02d1jzNkiJts3UkkV/7c9oO55ycbd9P9+7B6BZsXAz8EvmB7DUmzAVNzvg/jjbKFXuiI62+v+ENixdyIeT9CdNvKHsCB30jajz4EZppJK9TlCIELgH0lbWZ7/y6nNZ/fWOn9BbhB0nnEBW47oEqHresU4it1yUo+Jelw4CfJr92JTPuesD0SnaJetf2MNCR6kbWiGIHQxDeBLYDzk/07FQptVai1E5zth1s+q0q7YESp1s5EaOx14jt2tlMjnx7J1qvvgYUdfco/B2B7WipfLHSgDOCF4XiI+torLmt7Z0mTk41/qeWKlEFj5dWPwEwz7wPe1ZSs92Mi+ahXFkn/Ppxuc6bHl1AtTlu3rOSuhDDNxenx1UQJUU+MRLIfcJ9CenOCQoXtIKDvpi/9UuNAOYlI+HwxfefXAr5FTFxzeTjtDjjlDRwIVBI7SZUMxwLHpl2wLwFfA3pu1dtPNUQX/qnQJ2j8Btenese1cUEZwAvD0a69YlVekTQ3M36gy9K0es7BfUqLtuGPREJX48K0BHBXhj9tM4vTxXarXGdcc1cmR2OW/VMYwxXKompN9kscAHyZyND+JXApvSWujSS1DZSEutgaklYn3tePiOYm76tga1+iUcg7iAnAFIbqvmehaOW6E0l5jmgylHN+7SEVIrx2PrBsqlhZhAgbFDpQYuCFnpA0P/HDzFakarKxGfBFQvZ0CqHa9FHbV2bY2MT25eogoVm1ZlShOT2JaGJBun8DSXHM9jYZtiYQsd3JwJbATba3637WG2zMSaixvZOhWfZVe2+vSgwgbyMuso8AH3OGYtZoRtK7bV833LEe7CxMDJTNSZsH2X6ygk+NbmtfAh63/f3RUOqW4umzE+Grnzt1ExsNpLh3I8n1j1WTXMcLZQAvdEXRcOIMomsRRM/fPW3f2/msrvbewoyWnTc6v2XnkbaP6FA7WrlmVCHe0hHbV3V7PtnYkNiq3gaYSrzPZatMeiRdQmwftmbZfyPXVrJ3LaG89tv0eFPgK7bfk2mnbTzYGV3bmmxdAuzioSp/P7GdvWPRbmAc9GAp6RpiRflx4P3A34A7qyRlSWon/vIsoTPw60xbK9n+Q64Pw9hclOjiB0BOPL3TZLzJVhFy6UAZwAtdUXTH+oLtK9Lj9wNHO1P+NM2stwRWSofuI7KzszSYRxJFG8/lbV+Wtvpns/38cOelc/+XCDWcCvzS9rPqo5tV1QzjLvbudEuntnbHerDzm6aHcxE9oW9zfv19W6GWdseGsbEBkS9wMJGA1qChLpb7/uocKN9OJAveYvsKSUsCH7DdTbikk61Tid/OOenQjkSN/xLAn2wf3ION3R1d3z7d7vkqeS2StgG+QWgM/J3QZbjPGUJPHSbjTW4VIZdOlBh4YTjmbQzeALavVDQl6Zl0IbsCeJxYmYrIYv2GpI1tP9bt/BZbIyKdKWkfot59IaIr0uJEnXmv5V8XAFsD2xLJOL+hP5GR6yWtZjsnka4bD6Xs3jPS492ZEe/vGdtbNz+WtASREFWF1yUtbvuRZKuKqMwcwHzEtaw5R+M5qsVP56L9QLl3+q4OO1A2SN/rYwEkvSmtSrMH78RyhK76tGTvFGJ7fzN6T7Zs/G7b5bJU/a7+J7HTdJnttSQ1msH0jIuQS2XKCrzQFUWTj9sZeuFfNyemq2iFeIftE1qOHwisY7trLW/LOSMinanoQrYeEa+uWpM8gYidTia6Pi1AZMtf4syOa5J+T1y0/0wk+lWur0323kJcbN+TbF0NfLlKbLfFrohM6yrbwlsB3yZ6p0N0AvuU7Ys7n9XR1lKNzOj0d5jPFWRiFXX8mzcNlLPRNFDaXqUHG+sRgiRPEe1RzyBWqK8Be9ieUsGvPwLr2X42PV6Q+K6ulLtr0cH+wa2/zx7Pu9X2uora8rVsvy7pZtvrVfRjK6I5SvN2fKW8j/FAWYEXhmMvovyoEYe6mujalcP6jVVzM7a/lS5MOYzUjPNl2680yofShTu3Jvl14mI/pSn7fDKxrZ5bE7xl5v8fzrcnaeq9XRVJJzLjc5kArAncWdGnC9NgtwExqfisM1tkNvHfkvYlBsnbgAUlHW/7uEw77yBWqo3ypXkJZbDXJPVaMXEy0cRkQWLnaWvb16VEwjOI70guxwJ3KDqcCdgIODrthl1WwV4rnwayB3DgGUVXwauBnyqU8SqFxRRyzfMQE7nvEzsolVvojgfKCrzQEUVTjqWABxqJRhXtdFwhVIh5jpR05rHAM8CewH8Qg93vbX+hir0W2z3Lckq6EDgTOK/Xc3q0uzZwOG/Mas/awdBQ5bNpwEO5md4t9t5KlO81+3R9BTt32F5T0m7AOsBnidh81o6FpL2JSokraRooCYGfr7gH6dmWXaL7bK/c7rlcJL2N2CUScHNO6KkH2w+7QqvaNIH4FzGZ242YtPy0Ytb+XbZXb/p3PiKfZPNcW+OFsgIvtEXSx4kL14PA0pI+4aSjXYEFO2SaithmzqH5AnprRX/acTiwNxFP/CRwEbEK6JvMgfhUop/4NyVdQQwcF7l/JbwziVrkIZ2xekXSkrb/UlfOQbJ5NBGSua/JJwNZPdgTsytka7cjOty9Kil7dWL7NEkXMWOg/HzTQNmrbnzz67bW2/ezYnqJyCOZC1hO0nJVsv87kO2XQmr517Y3Jf5+/X43Gp/Viylv5imgbr2HWYoygBc6cTCwqu0nJC1DCFBUHcCvIhK82pF1AWoMIJI+Yvuc5uckfaSKcxqq+f69KjbqImU6/zplwW9DxNC/kwaVsxplYBV4ss9ynPOAtQEknWt7xz5sNdgRWMH2SzXY+i6hGngncHWqKKjaKrXfgXINSU8RE4D5033S4/mqOJQm1AcRyZV3EIljN5ChzKfu4itz5/qUwgovSlqwEZvvkwskvYkIFzSaEtUyiZ5VKVvohba0JojVmTBWB+386cdHSZcSsco6NN8bNgXM0+9WuELJ68fA6rZ7lrtssbE5MWBexlDt+J4mZS3bwn0nTSU7lwA75Cb49WhbwERnlil2GihzyuTShLAjrtDJT9E8ZBKhnbCmpJWIuv5K7XjrQtLZxGf0W4ZKLfccypI0CXjY9l/T4z2JnZk/EGGLSv0NxgNlBV7oxOIaWhM75HHVWHO/SNqS2GJ9R4t/C1AxeSbxEDVovks6nZAInUZs8S8s6ZgKdhYjpC53IdTTziE/ebCZ3YDViRVg83Z1r7sq7nC/H54HpkpqnVS0rVPOwbYl7UF+2dZBzBgoN24MlJmvPRINOF6y/ZIkJM1p+w+Sem5RO4JcmG798F2ieqMhFHQMkYeyJhFSKnKqHSgDeKETrfG+gfbZbuIxYmDchqE+PQ8c0qfdOjTfV7P9nKRdiWzjwwh/exrAFfXokwk5yV8Ch/WTJNbEOu5PGGYNSc+RtlvTfaAv7etL0m2kOJL8AXy0DpSPpO3l84DfKlqV1pbEVhXbP07hniVt51aUNJjYtMreGTjV9rnAuYryzkIHygBeaEudyUp1YvtO4E5JZ7pGnWTbWausLsyRStC2BU5JpWk5SWMbEiuQy1JZWl3cJGnFqhfZqlv3w9g8rV8bkjo1nBGwWAWTo3Wg3D7d/UpKblyQkZ389ISkrYGvE4I6S0taE/iqM3oHABMlzZbCHR8gBJUalDGqC+XDKYw4HTLQp1MxuWoLSf9JlLnNRsWVoKQTbB+sDsppmRciiKSbvwD3AFcp1MV61kL3yKlSrQfcJekBhgrDzPS8Bkln2Z4saSrtP/McnxYj+ne39jYXkF2ONhoHSoUwzV2NHRT3oMvfxdZE4NKUOV4HXyG+W1cC2L5D0Ro2h7OI38o/iEz0a5Kvy1HaiXalDOCFmUEjA31RYoXZrLx1JTNEYnI4AdiBUMfqJybbUJj7eh82pmP7mzRpckt6mOo9vOskqxvaCNMIz9QR27yAUF17w1arQvSkZ+oaKNOqvVurza691VtxqJvd2Sjlq+JTk626M8enOXT/h7xMpk9HSfodkesxpen3PIGIhRc6UAbwwojTWFVKugBYxfbj6fHbCNWqKjwM3NPn4A3wRPKx8qqmGUkLEBm072To76vvxKx+sP2gpPWJsq3TFdKqWZr2NfrySLr7GBFztqI3/IpkqpTZ3rvLc7tm2qproMxV3euFtwH3SrqZoUmWuTtEEGVyd0uqnDnexD0p32OipOWJ/ulVdj5ubHPsfyr4M64oZWSFrijU2PbhjQpe2R2C1NJhq3XFk2lrEqHtfRVDM5hzs72nl57VUd+saNt5Oy2CKbnx3jSgPWL7ZUUHuNWB011REU/SF4n+68vaXkHSO4he0FntROtE0q2E0tmCwC1Eo5unbe85QJ8uZ0Zf+H4HyobNhRiq7Z0dU1eHdrdVJp4aqqbXbCs770XSPMAXCO1/gEuB/6qptr8wDGUFXhiOXxMxqcto6ktdkStTvfVZxDbbLoRWdBWOImLLcxEJNFVp3vtbpg87DeapqcTuXGDdFAc8jSj3OpNqKmUQ29VrEZMLbD+adgsGyQTbL0rai1BPO2YUZB3XlczYaMzxTaKm/ElCZ/1/mNFSt2fq2iFKtupMUF3RITfct+RwIZ8ygBeGYx7bn63DkO0DJG1PrLogykV+VdHcQq5HI7nu+uYzJX2MiM027wzkqoK9bnta+rxOsH1iSvqqystpq9owfeU0aCaknZRdmZF5XHu2ew51DpTEJPPdRFx3LUmbEWI62aTwx4nAysSEdSLwz5ykTUnbAovbPjk9vglYJD19mO1fVHDt+BQKOwf4me17K9goVGTCoB0ojHoukFR11deO64kktt8B/dQ3X5bUxfplDUnPKWQmV0/3qdtaxQAADn1JREFUn5P0fFOtcw4vEAl2U4k+0vcSGem5vCppMiGlekE6NnsFOw1+KelkQpf+Y0Ss+Qd92KuDQ4gV74W271FI9l4zSIckrS/pFkkvSHpF0msVvwcQCV5PEBMVOWRwq2b9n0ToA9xPyJ5+PB3L4TCGCvfMSYQL3g98qopTtjdO5z8BnCrp7hSuKcwESgy80JU0sM0LvJJulYU7JO0EHMeMTk/vBQ6tMvNv8utl4NV+/KoTSQ8CG7h6W8yGnVWAfQkZz7NSac7Oto/pw+aWRKxSRClRdt/tmUmjxG8mv+atRGjnHGBdojvd8rY/X8HW7wjBoWMJpcC/A++2vX4Vvxx9t+9y6rAm6XrbG2bYuMX2pKbHJ9k+IN2/sYpfLfZXIyYJO9vuJ6xV6JEygBdmGpLuBDZrDG4pQe4y22sM1rP6SPXkHxlNSTyp9vci21sM2pccNAD9/ToGyiZb8wMvEjudexLJeqfb/kcFW1cTcqPfB/5KNFv5aM5vR9IDtpfr8NyDtpet4NfKhHraR4B/EO19z+13AlvojbKFXuiKgt0lfSk9XkLSehXNTWj5YT9Jxe+gpHcrehGT/DteIZoyaF4h9L1PTj4dLykrMx5A0vKSfiHp95L+1LhVccihzf3KKEhaGwu8KGkO4A5Jx0o6hOrldp+z/ZrtV22fliokqpYT7kH8Vg4gsuOXID+efpNCqncIkj5JZN1X4YeEiM5mtt9n+5QyeM88ygq80BVJpxDlUJvYXlnSm4mknEnDnNrO1nFEOdRZ6dDORBlZdpKcQkJzjWTvDCJTewfbbcttZhaS2tYlVygjuxY4gshi3ppoZCLbR1T06yyia9QUhpZHDbQ+vRsDWoEvBfyNSBQ7hFg1f9v2AxVsteuYd+egdpwkLUpIxL5MqkYA1iFi4dvZ/lsFm3MDyxIJoA+Opp2n8UAZwAtdaVyENLSdZOWLkEJW9T1EHPbqqlnoTX59GXjU9mmDuOD3gqR/s31T5jm32V5H0t22V0vHrrH93oo+1DKxmJmopralM5u0ot0XWAFo1p6fH7jV9uQMW7VnjkvaBFg1PbzX9uXd/n8HG7MBRxMTy78QuwOLEyvyL7jGPgWFzpQyssJwvJpiqI3yo0VoEiipwHVE0pmpvm0H8LykzxGqZxslH/vJ0u4LhSjNjkSt76W275P0QeDzwJuB1TJNvpRs3i/pAOBRQoq2EsMN1JLOtr1TVfv9IrXtnZ6bZd3P69c5UJ5NVFn8N3B40/HnK2wvH0Yk1TVoZI7PSwyW2QN4GrCzB+0WjiMmJMvYfh6mqxB+Pd0O6tN+oQdKDLwwHN8CfgUsJuko4Fpi5p1NykK/mRAV2YmIyVXVw96Z2Arc2/ZfiYHzuIq26uD7wP7Jj1MkfY+o2/1WYwWdycHAPIQ05TpEDLStglZNLD+Cttsi6XRJCyhq0u8F/ixp+pb+TN4dqK3EyvbTth+w/RGi5GuzdFuk+5ltmcP2w02Pr7X9pEPqdSBSuIkPAfs0Bm+YrnXwKaqLDRUyKVvohWGRtBLR5g/gctv3VbQzIlnokhYGnvQAv8yS7gVWdzSLmJvIyF3OSfd9tDOgePPUJHCyK9HR6jBii3n1melH8qX2EitJ+xOTuvPSoW2Bk21/O8NG7ZnjdSDpf2yvkPtcoV7KFnqhF+YhlJ9MrCiq0ncWukKR6hjgKUIL/QyiecQESXvaHlTrx5dTtje2/yXpj1UGb0nnd3vefWhyj0L67Z1eJ29uftAYvBNVVs4AnwTWs/0CgKSjCSGjngdwUua47e81H+wzc7wOfp9+b6c3H5S0O/CHAfk07igDeKErKUnsI4Q2t4AfSjrH9n9VMHeJZmihQ2yDX5Rp4yQirrwgEcfb0vaNaZfgLAbXu3klSY3MXgErpse5fbc3IDqtnQXcxFCt9pFkZr1OM9+jj97pNTMSA6WIfI8GDcGhHA4Bzku7FG/IHK/oVx3sT6j77QXcRkzuJxET/O27nVioj7KFXuiKpPuAtRrlIWl7+HbbK1e0tyOhD10pC13SHbbXbPjW7Mcgs5YV3cM6YvvBHu1MJOKlk4kSuQuBs1yDxnSqb16yXUmUpC09E5XZ0vvczva5TccmALPZfmVm+dH02rWVWEmazaFjfxjxd2y8x+2Jv2V27/k6MsdHgia/RPj1uwG7NK4oA3ihK5IuBiY7tbGU9CbgJ7Y/lGHjYCL7fKrtaX3609z+c0jcdjSUkUk62i2ym+2O9WhrTmIAOA74qu0T+/BrK+B4IilqaUlrAkfYHthqqZ+yuJGiphKr5u/oJEIyuDFhvaU2ZwvjnjKAF7oi6Txia+y3xDbZZkQm+t8B3EPrTElfBzYk2ijeRcQBryN0vp/K9Oc1QohExHbdi42ngLlsD6yUDOoR70gD91bE4P1OIjv6B7Yf7cOv24hExCua6vmn15gPAkXTixeAnzNUXKZq85BRwVitXy+MPcoAXuiKpK6lS87oLZy2cNclBvMN0u0Z26v05eQooC7xDkk/Bt4FXEy0Z6zSyayd3Rttr98iyDNd73sQSHq4zWHbHg2SuJWR9Aix29EWh6RqodA3JYmtMBw/B5ajHqnEuYmuTAum22PA3X17ODqoS7xjD2I1ugJwYOibAPTdbe2+VIc/QdHZ7CDgxoq2asH2EoN8/RFkIjAfg0kMLIwjygq80JYmqcS9gP+lD6lESacSccXniczqG4EbbT9dt9+jAUnvIuRiAa6pIwGtXxSNX75MtBMFuJSIq7/Y+ayRJ1UPrALM1Thm+8zBedQ/oyEXozA+KEpshU4cBywELG17nbTtuizwJkIqMYcliWzevxKSoI8Az9To66ghiXecTbznJYGzJe03WK/A9j9tf9b2Wul2+CgYvL8InAp8B9gSOIFQ6RvrlJV3YaZQVuCFtki6H1ihVd0slf/8wXaW9GbSul6ViH9vSMR5nyIS2Sp12BqNKLqkbdgk3jEfcP0gY83Jj0uAXZqqCd5MVBNsNUCf7gbWJMoS15D0NuC7Y12sRtJCucmZhUIVSgy80Am3kyZNUqHZs75k6x5JzwDPptuHCAnNWWYApx7xjpFgscbgDaHXLentg3QI+Ff6Pk2TND+xQ7PMgH3qmzJ4F2YWZQAvdKI2qURJBxKr7ncTA9p1wA3AD5hFktga4h2EtOuNkprFO3rO1B9BXpe0uO1HAJLq2aCZmnQFfgDcCjzHDBGVQqEwDGULvdAWSe8Afgn8izZSiTk1yZKOJ9V+V9EHHwuMdvGOJOTybWa0kdwY+NTMVF/rhqTlgAVslwG8UOiRMoAXulKkEntjLIh3SFqMqL0XMZnK7U1dO5J2AZa1fZSkJYBFbd82aL8KhbFAGcALhRoYC+Idkt5KZMZPD53Zvn6A/pwEzA5sZHtlSQsBl7qprWehUOhMiYEXCvUwqsU7UivL3YH7gEbLTgP/PjCnIlt/bUlTIZK/klpfoVDogTKAFwr18Ljtrw7aiS7sSJQF9qOkVzevpg5kBpD0FmZMLgqFwjAUIZdCoR5G5cq7iT8z+n7vJxOtNheRdCTRJOdrg3WpUBg7lBh4oVADo128Q9I5RH/xy4ie1wDY/vQAfLkI2M/2Q5JWBTYlJkCX1dW8pVAYD5Qt9EKhBkbz4J24JN1GAz8CpqTOa8eOBq34QmEsUlbghUJhptPUXOWDhPjN9Nj3aMjYLxTGAmUFXijMwkg6y/bklOndThp3UF2zXiXaps5J9EwvyWuFQiZlAC8UZm0OTf+Omi5fkj5I1MyfD6w96K5ohcJYpWyhFwrjAElzAy/ZtqRlgRWBKUm/fWb7cg2wb4l9Fwr9UQbwQmEcIOlWYCNgQeAWYCrwtO09B+pYoVCozGirCy0UCiPDhLRVvSNwku2tibKyQqEwRikDeKEwPpiQuqTtClyQjk0coD+FQqFPygBeKIwPDgGOBC60fY+kZYBrBuxToVDogxIDLxQKSDrB9sGD9qNQKPROWYEXCgWIBLdCoTCGKAN4oVAoFApjkDKAFwqFQqEwBikDeKFQgNHfDrVQKLRQBvBCYZyhYN6WwycNxJlCoVCZMoAXCuMASadLWkDSPMC9wJ8lTe8Fbvu0wXlXKBSqUAbwQmF8sJrt54DtgCnA4sBHB+pRoVDoizKAFwrjgzkkzQZsC5xn+xVKC89CYUxTBvBCYXzwPeAvwJuBqyQtCbwwWJcKhUI/lH7ghcIsjqSJwMO239507BFgk8F5VSgU+qVIqRYK4wBJ19h+76D9KBQK9VEG8EJhHCDpi8SW+c+BfzaOp8S2QqEwBikDeKEwDpD0cJvDtr3kTHemUCjUQhnAC4VCoVAYg5QktkJhnCBpJWAVYK7GMdtnDs6jQqHQD2UFXiiMA1IMfHNgJeBSYAvgWts7DNSxQqFQmVIHXiiMD3YGNgYet70HsAZlB65QGNOUAbxQGB/8y/ZrwDRJ8wN/BZYZsE+FQqEPygy8UBgfTJX0JuAHwK3Ac8Dtg3WpUCj0Q4mBFwrjDEnLAQvYLgN4oTCGKVvohcI4QdIukr5g+wHgCUnrDNqnQqFQnbICLxTGAZJOAmYHNrK9sqSFgEttTxqwa4VCoSIlBl4ojA82tL22pKkAtp+SNMegnSoUCtUpW+iFwvjgVUkTAANIegulH3ihMKYpA3ihMD44GTgXWETSkcC1wNcG61KhUOiHEgMvFGZhJF0E7Gf7IUmrApsCAi6zfc9gvSsUCv1QYuCFwqzNj4Apkn4MHGv73gH7UygUaqKswAuFWRxJ8wJfBj4InEFT7Nv28YPyq1Ao9EdZgRcKsz6vAv8E5gTmpySvFQqzBGUALxRmYSR9EDgeOB9Y2/aLA3apUCjURNlCLxRmYSRdA+xbYt+FwqxHGcALhUKhUBiDlDrwQqFQKBTGIGUALxQKhUJhDFIG8EKhUCgUxiBlAC8UCoVCYQxSBvBCoVAoFMYgZQAvFAqFQmEM8v8Bo0cmo/wbhzUAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "correlation_matrix = numerical_cols.corr().abs()\n", "sorted_corrs = correlation_matrix.SalePrice.sort_values()\n", "strong_corrs = sorted_corrs[sorted_corrs > 0.3]\n", "\n", "print(len(strong_corrs))\n", "print(strong_corrs)\n", "\n", "sns.heatmap(numerical_cols[strong_corrs.index].corr())\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape\n", "(2927, 13)\n", "----------\n", "\n", "Data Types\n", "int64 10\n", "float64 3\n", "dtype: int64\n", "----------\n", "\n", "Columns : Null Counts\n", "Series([], dtype: int64)\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Open Porch SFWood Deck SFBsmtFin SF 1FireplacesMas Vnr AreaYears_since_remodelledFull BathYears_since_sold1st Flr SFGarage AreaGr Liv AreaOverall QualSalePrice
062210639.02112.0501501656528.016566215000
10140468.000.049149896730.08965105000
236393923.00108.0521521329312.013296172000
3001065.020.0422422110522.021107244000
434212791.010.012213928482.016295189900
\n", "
" ], "text/plain": [ " Open Porch SF Wood Deck SF BsmtFin SF 1 Fireplaces Mas Vnr Area \\\n", "0 62 210 639.0 2 112.0 \n", "1 0 140 468.0 0 0.0 \n", "2 36 393 923.0 0 108.0 \n", "3 0 0 1065.0 2 0.0 \n", "4 34 212 791.0 1 0.0 \n", "\n", " Years_since_remodelled Full Bath Years_since_sold 1st Flr SF \\\n", "0 50 1 50 1656 \n", "1 49 1 49 896 \n", "2 52 1 52 1329 \n", "3 42 2 42 2110 \n", "4 12 2 13 928 \n", "\n", " Garage Area Gr Liv Area Overall Qual SalePrice \n", "0 528.0 1656 6 215000 \n", "1 730.0 896 5 105000 \n", "2 312.0 1329 6 172000 \n", "3 522.0 2110 7 244000 \n", "4 482.0 1629 5 189900 " ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "strong_corrs.drop(['Garage Cars', 'Total Bsmt SF', 'TotRms AbvGrd'], inplace=True)\n", "high_corr_num_cols = numerical_cols[strong_corrs.index]\n", "\n", "data_shape(high_corr_num_cols)\n", "high_corr_num_cols.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Text Columns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Dropping Columns with missing values" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape\n", "(2927, 25)\n", "----------\n", "\n", "Data Types\n", "object 25\n", "dtype: int64\n", "----------\n", "\n", "Columns : Null Counts\n", "Series([], dtype: int64)\n" ] } ], "source": [ "text_cols = df[df.dtypes[df.dtypes == 'object'].index].copy()\n", "\n", "# Dropping Text Columns with missing values\n", "null_cols = text_cols.isnull().sum()\n", "text_cols.drop(null_cols[null_cols > 0].index, axis=1, inplace=True)\n", "data_shape(text_cols)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Converting nominal columns to dummy variables" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "## Nominal columns from the documentation\n", "nominal_cols = ['MS Zoning', 'Street', 'Alley', 'Land Contour', 'Lot Config', 'Neighborhood', 'Condition 1', \n", " 'Condition 2', 'Bldg Type', 'House Style', 'Overall Qual', 'Roof Style', 'Roof Mat1', \n", " 'Exterior 1st', 'Exterior 2nd', 'Mas Vnr Type', 'Foundation', 'Heating', 'Central Air'] \n", "\n", "nominal_num_col = ['MS SubClass']\n", "\n", "## Categorical Text Data\n", "common_columns = [x for x in text_cols if x in nominal_cols]\n", "categories_per_col = {x:len(text_cols[x].value_counts()) for x in common_columns}\n", "col_less_then_thresh = [x for x in categories_per_col if categories_per_col[x] <= 10]\n", "\n", "text_cols = text_cols[col_less_then_thresh].astype('category') \n", "categorical_text_data = pd.get_dummies(text_cols)\n", "\n", "## Categorical Numerical Data\n", "common_columns = [x for x in numerical_cols if x in nominal_num_col]\n", "numerical_cols[common_columns] = numerical_cols[common_columns].astype('category') \n", "categorical_num_data = pd.get_dummies(numerical_cols.select_dtypes(include=['category']))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Concatenating all the dataframes together" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape\n", "(2927, 97)\n", "----------\n", "\n", "Data Types\n", "uint8 84\n", "int64 10\n", "float64 3\n", "dtype: int64\n", "----------\n", "\n", "Columns : Null Counts\n", "Series([], dtype: int64)\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Open Porch SFWood Deck SFBsmtFin SF 1FireplacesMas Vnr AreaYears_since_remodelledFull BathYears_since_sold1st Flr SFGarage AreaGr Liv AreaOverall QualSalePriceMS Zoning_A (agr)MS Zoning_C (all)MS Zoning_FVMS Zoning_I (all)MS Zoning_RHMS Zoning_RLMS Zoning_RMStreet_GrvlStreet_PaveLand Contour_BnkLand Contour_HLSLand Contour_LowLand Contour_LvlLot Config_CornerLot Config_CulDSacLot Config_FR2Lot Config_FR3Lot Config_InsideCondition 1_ArteryCondition 1_FeedrCondition 1_NormCondition 1_PosACondition 1_PosNCondition 1_RRAeCondition 1_RRAnCondition 1_RRNeCondition 1_RRNnCondition 2_ArteryCondition 2_FeedrCondition 2_NormCondition 2_PosACondition 2_PosNCondition 2_RRAeCondition 2_RRAnCondition 2_RRNnBldg Type_1FamBldg Type_2fmConBldg Type_DuplexBldg Type_TwnhsBldg Type_TwnhsEHouse Style_1.5FinHouse Style_1.5UnfHouse Style_1StoryHouse Style_2.5FinHouse Style_2.5UnfHouse Style_2StoryHouse Style_SFoyerHouse Style_SLvlRoof Style_FlatRoof Style_GableRoof Style_GambrelRoof Style_HipRoof Style_MansardRoof Style_ShedFoundation_BrkTilFoundation_CBlockFoundation_PConcFoundation_SlabFoundation_StoneFoundation_WoodHeating_FloorHeating_GasAHeating_GasWHeating_GravHeating_OthWHeating_WallCentral Air_NCentral Air_YMS SubClass_20MS SubClass_30MS SubClass_40MS SubClass_45MS SubClass_50MS SubClass_60MS SubClass_70MS SubClass_75MS SubClass_80MS SubClass_85MS SubClass_90MS SubClass_120MS SubClass_150MS SubClass_160MS SubClass_180MS SubClass_190
062210639.02112.0501501656528.016566215000000001001000110000001000000001000001000000100000000100010000010000011000000000000000
10140468.000.049149896730.08965105000000010001000100001010000000001000001000000100000010000010000010000011000000000000000
236393923.00108.0521521329312.013296172000000001001000110000001000000001000001000000100000000100010000010000011000000000000000
3001065.020.0422422110522.021107244000000001001000110000001000000001000001000000100000000100010000010000011000000000000000
434212791.010.012213928482.016295189900000001001000100001001000000001000001000000000100010000001000010000010000010000000000
\n", "
" ], "text/plain": [ " Open Porch SF Wood Deck SF BsmtFin SF 1 Fireplaces Mas Vnr Area \\\n", "0 62 210 639.0 2 112.0 \n", "1 0 140 468.0 0 0.0 \n", "2 36 393 923.0 0 108.0 \n", "3 0 0 1065.0 2 0.0 \n", "4 34 212 791.0 1 0.0 \n", "\n", " Years_since_remodelled Full Bath Years_since_sold 1st Flr SF \\\n", "0 50 1 50 1656 \n", "1 49 1 49 896 \n", "2 52 1 52 1329 \n", "3 42 2 42 2110 \n", "4 12 2 13 928 \n", "\n", " Garage Area Gr Liv Area Overall Qual SalePrice MS Zoning_A (agr) \\\n", "0 528.0 1656 6 215000 0 \n", "1 730.0 896 5 105000 0 \n", "2 312.0 1329 6 172000 0 \n", "3 522.0 2110 7 244000 0 \n", "4 482.0 1629 5 189900 0 \n", "\n", " MS Zoning_C (all) MS Zoning_FV MS Zoning_I (all) MS Zoning_RH \\\n", "0 0 0 0 0 \n", "1 0 0 0 1 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " MS Zoning_RL MS Zoning_RM Street_Grvl Street_Pave Land Contour_Bnk \\\n", "0 1 0 0 1 0 \n", "1 0 0 0 1 0 \n", "2 1 0 0 1 0 \n", "3 1 0 0 1 0 \n", "4 1 0 0 1 0 \n", "\n", " Land Contour_HLS Land Contour_Low Land Contour_Lvl Lot Config_Corner \\\n", "0 0 0 1 1 \n", "1 0 0 1 0 \n", "2 0 0 1 1 \n", "3 0 0 1 1 \n", "4 0 0 1 0 \n", "\n", " Lot Config_CulDSac Lot Config_FR2 Lot Config_FR3 Lot Config_Inside \\\n", "0 0 0 0 0 \n", "1 0 0 0 1 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 1 \n", "\n", " Condition 1_Artery Condition 1_Feedr Condition 1_Norm Condition 1_PosA \\\n", "0 0 0 1 0 \n", "1 0 1 0 0 \n", "2 0 0 1 0 \n", "3 0 0 1 0 \n", "4 0 0 1 0 \n", "\n", " Condition 1_PosN Condition 1_RRAe Condition 1_RRAn Condition 1_RRNe \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " Condition 1_RRNn Condition 2_Artery Condition 2_Feedr Condition 2_Norm \\\n", "0 0 0 0 1 \n", "1 0 0 0 1 \n", "2 0 0 0 1 \n", "3 0 0 0 1 \n", "4 0 0 0 1 \n", "\n", " Condition 2_PosA Condition 2_PosN Condition 2_RRAe Condition 2_RRAn \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " Condition 2_RRNn Bldg Type_1Fam Bldg Type_2fmCon Bldg Type_Duplex \\\n", "0 0 1 0 0 \n", "1 0 1 0 0 \n", "2 0 1 0 0 \n", "3 0 1 0 0 \n", "4 0 1 0 0 \n", "\n", " Bldg Type_Twnhs Bldg Type_TwnhsE House Style_1.5Fin House Style_1.5Unf \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " House Style_1Story House Style_2.5Fin House Style_2.5Unf \\\n", "0 1 0 0 \n", "1 1 0 0 \n", "2 1 0 0 \n", "3 1 0 0 \n", "4 0 0 0 \n", "\n", " House Style_2Story House Style_SFoyer House Style_SLvl Roof Style_Flat \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 1 0 0 0 \n", "\n", " Roof Style_Gable Roof Style_Gambrel Roof Style_Hip Roof Style_Mansard \\\n", "0 0 0 1 0 \n", "1 1 0 0 0 \n", "2 0 0 1 0 \n", "3 0 0 1 0 \n", "4 1 0 0 0 \n", "\n", " Roof Style_Shed Foundation_BrkTil Foundation_CBlock Foundation_PConc \\\n", "0 0 0 1 0 \n", "1 0 0 1 0 \n", "2 0 0 1 0 \n", "3 0 0 1 0 \n", "4 0 0 0 1 \n", "\n", " Foundation_Slab Foundation_Stone Foundation_Wood Heating_Floor \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " Heating_GasA Heating_GasW Heating_Grav Heating_OthW Heating_Wall \\\n", "0 1 0 0 0 0 \n", "1 1 0 0 0 0 \n", "2 1 0 0 0 0 \n", "3 1 0 0 0 0 \n", "4 1 0 0 0 0 \n", "\n", " Central Air_N Central Air_Y MS SubClass_20 MS SubClass_30 \\\n", "0 0 1 1 0 \n", "1 0 1 1 0 \n", "2 0 1 1 0 \n", "3 0 1 1 0 \n", "4 0 1 0 0 \n", "\n", " MS SubClass_40 MS SubClass_45 MS SubClass_50 MS SubClass_60 \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 1 \n", "\n", " MS SubClass_70 MS SubClass_75 MS SubClass_80 MS SubClass_85 \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " MS SubClass_90 MS SubClass_120 MS SubClass_150 MS SubClass_160 \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " MS SubClass_180 MS SubClass_190 \n", "0 0 0 \n", "1 0 0 \n", "2 0 0 \n", "3 0 0 \n", "4 0 0 " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "final_data = pd.concat([high_corr_num_cols, categorical_text_data, categorical_num_data], axis=1)\n", "data_shape(final_data)\n", "final_data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Creating a Pipeline with adjustable parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When we did our data cleaning, we decided to remove columns that had more than 5% missing values. We can incorporate our this into a function as an adjustable parameter. In addition, this function will perform all the data cleaning operations done above." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "def transform_features(df, percent_missing=0.05):\n", " \n", " # Adding relevant features\n", " df['Years_since_remodelled'] = df['Yr Sold'] - df['Year Remod/Add']\n", " df['Years_since_sold'] = df['Yr Sold'] - df['Year Built']\n", " df = df[(df['Years_since_remodelled'] >= 0)]\n", " df = df[(df['Years_since_sold'] >= 0)]\n", "\n", " # Dropping columns which are not useful or leak information about the sale\n", " df.drop(['Sale Condition', 'Sale Type', 'Mo Sold', 'Yr Sold', 'PID', 'Order', \n", " 'Year Remod/Add', 'Year Built'], axis=1, inplace=True)\n", "\n", " # Dropping Columns which have more then given percentage missing Values\n", " null_counts = df.isnull().sum()\n", " df.drop(null_counts[null_counts > df.shape[0]*percent_missing].index, axis=1, inplace=True)\n", " \n", " return df" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the feature engineering, we chose columns that had more than 0.4 correlation with 'SalePrice' and removed any columns with more than 10 categories.\n", "\n", "Once again, we will combine all the work done previously into a function with adjustable parameters." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "def select_features(df, correlation_threshold = 0.4, cat_threshold = 10):\n", " ## Imputing missing values with mode\n", " numerical_cols = df[df.dtypes[df.dtypes != 'object'].index].copy()\n", " numerical_cols.fillna(numerical_cols.mode().iloc[0], inplace=True)\n", " \n", " ## Dropping columns less then given correlation threshold\n", " correlation_matrix = numerical_cols.corr().abs()\n", " sorted_corrs = correlation_matrix.SalePrice.sort_values()\n", " strong_corrs = sorted_corrs[sorted_corrs > correlation_threshold]\n", " \n", " ## Comment below line if correlation threshold > 0.4\n", "# strong_corrs.drop(['Garage Cars', 'Total Bsmt SF', 'TotRms AbvGrd'], inplace=True)\n", " high_corr_num_cols = numerical_cols[strong_corrs.index]\n", " \n", " \n", " ## Text Columns\n", " text_cols = df[df.dtypes[df.dtypes == 'object'].index].copy()\n", " null_cols = text_cols.isnull().sum()\n", " text_cols.drop(null_cols[null_cols > 0].index, axis=1, inplace=True)\n", " \n", " #Nominal columns from the documentation\n", " nominal_cols = ['MS Zoning', 'Street', 'Alley', 'Land Contour', 'Lot Config', 'Neighborhood', 'Condition 1', \n", " 'Condition 2', 'Bldg Type', 'House Style', 'Overall Qual', 'Roof Style', 'Roof Mat1', \n", " 'Exterior 1st', 'Exterior 2nd', 'Mas Vnr Type', 'Foundation', 'Heating', 'Central Air'] \n", " \n", " nominal_num_col = ['MS SubClass']\n", " \n", " ## Categorical Text Data\n", " common_columns = [x for x in text_cols if x in nominal_cols]\n", " categories_per_col = {x:len(text_cols[x].value_counts()) for x in common_columns}\n", " col_less_then_thresh = [x for x in categories_per_col if categories_per_col[x] <= cat_threshold]\n", " \n", " text_cols = text_cols[col_less_then_thresh].astype('category') \n", " categorical_text_data = pd.get_dummies(text_cols)\n", " \n", " # Categorical Numerical Data\n", " common_columns = [x for x in numerical_cols if x in nominal_num_col]\n", " numerical_cols[common_columns] = numerical_cols[common_columns].astype('category') \n", " categorical_num_data = pd.get_dummies(numerical_cols.select_dtypes(include=['category']))\n", " \n", " ## Concatenating all the dataframes together\n", " final_data = pd.concat([high_corr_num_cols, categorical_text_data, categorical_num_data], axis=1)\n", " return final_data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Applying Linear Regression\n", "Now we are ready to apply machine learning, we'll use the linear regression model from scikit-learn. Linear regression should work well here since our target column 'SalePrice' is a continuous value. We'll evaluate this model with RMSE as an error metric." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "34917.70149125682\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
BsmtFin SF 1FireplacesTotRms AbvGrdMas Vnr AreaYears_since_remodelledFull BathYears_since_sold1st Flr SFGarage AreaTotal Bsmt SFGarage CarsGr Liv AreaOverall QualSalePriceMS Zoning_A (agr)MS Zoning_C (all)MS Zoning_FVMS Zoning_I (all)MS Zoning_RHMS Zoning_RLMS Zoning_RMStreet_GrvlStreet_PaveLand Contour_BnkLand Contour_HLSLand Contour_LowLand Contour_LvlLot Config_CornerLot Config_CulDSacLot Config_FR2Lot Config_FR3Lot Config_InsideCondition 1_ArteryCondition 1_FeedrCondition 1_NormCondition 1_PosACondition 1_PosNCondition 1_RRAeCondition 1_RRAnCondition 1_RRNeCondition 1_RRNnCondition 2_ArteryCondition 2_FeedrCondition 2_NormCondition 2_PosACondition 2_PosNCondition 2_RRAeCondition 2_RRAnCondition 2_RRNnBldg Type_1FamBldg Type_2fmConBldg Type_DuplexBldg Type_TwnhsBldg Type_TwnhsEHouse Style_1.5FinHouse Style_1.5UnfHouse Style_1StoryHouse Style_2.5FinHouse Style_2.5UnfHouse Style_2StoryHouse Style_SFoyerHouse Style_SLvlRoof Style_FlatRoof Style_GableRoof Style_GambrelRoof Style_HipRoof Style_MansardRoof Style_ShedFoundation_BrkTilFoundation_CBlockFoundation_PConcFoundation_SlabFoundation_StoneFoundation_WoodHeating_FloorHeating_GasAHeating_GasWHeating_GravHeating_OthWHeating_WallCentral Air_NCentral Air_YMS SubClass_20MS SubClass_30MS SubClass_40MS SubClass_45MS SubClass_50MS SubClass_60MS SubClass_70MS SubClass_75MS SubClass_80MS SubClass_85MS SubClass_90MS SubClass_120MS SubClass_150MS SubClass_160MS SubClass_180MS SubClass_190
0639.027112.0501501656528.01080.02.016566215000000001001000110000001000000001000001000000100000000100010000010000011000000000000000
1468.0050.049149896730.0882.01.08965105000000010001000100001010000000001000001000000100000010000010000010000011000000000000000
2923.006108.0521521329312.01329.01.013296172000000001001000110000001000000001000001000000100000000100010000010000011000000000000000
31065.0280.0422422110522.02110.02.021107244000000001001000110000001000000001000001000000100000000100010000010000011000000000000000
4791.0160.012213928482.0928.02.016295189900000001001000100001001000000001000001000000000100010000001000010000010000010000000000
\n", "
" ], "text/plain": [ " BsmtFin SF 1 Fireplaces TotRms AbvGrd Mas Vnr Area \\\n", "0 639.0 2 7 112.0 \n", "1 468.0 0 5 0.0 \n", "2 923.0 0 6 108.0 \n", "3 1065.0 2 8 0.0 \n", "4 791.0 1 6 0.0 \n", "\n", " Years_since_remodelled Full Bath Years_since_sold 1st Flr SF \\\n", "0 50 1 50 1656 \n", "1 49 1 49 896 \n", "2 52 1 52 1329 \n", "3 42 2 42 2110 \n", "4 12 2 13 928 \n", "\n", " Garage Area Total Bsmt SF Garage Cars Gr Liv Area Overall Qual \\\n", "0 528.0 1080.0 2.0 1656 6 \n", "1 730.0 882.0 1.0 896 5 \n", "2 312.0 1329.0 1.0 1329 6 \n", "3 522.0 2110.0 2.0 2110 7 \n", "4 482.0 928.0 2.0 1629 5 \n", "\n", " SalePrice MS Zoning_A (agr) MS Zoning_C (all) MS Zoning_FV \\\n", "0 215000 0 0 0 \n", "1 105000 0 0 0 \n", "2 172000 0 0 0 \n", "3 244000 0 0 0 \n", "4 189900 0 0 0 \n", "\n", " MS Zoning_I (all) MS Zoning_RH MS Zoning_RL MS Zoning_RM Street_Grvl \\\n", "0 0 0 1 0 0 \n", "1 0 1 0 0 0 \n", "2 0 0 1 0 0 \n", "3 0 0 1 0 0 \n", "4 0 0 1 0 0 \n", "\n", " Street_Pave Land Contour_Bnk Land Contour_HLS Land Contour_Low \\\n", "0 1 0 0 0 \n", "1 1 0 0 0 \n", "2 1 0 0 0 \n", "3 1 0 0 0 \n", "4 1 0 0 0 \n", "\n", " Land Contour_Lvl Lot Config_Corner Lot Config_CulDSac Lot Config_FR2 \\\n", "0 1 1 0 0 \n", "1 1 0 0 0 \n", "2 1 1 0 0 \n", "3 1 1 0 0 \n", "4 1 0 0 0 \n", "\n", " Lot Config_FR3 Lot Config_Inside Condition 1_Artery Condition 1_Feedr \\\n", "0 0 0 0 0 \n", "1 0 1 0 1 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 1 0 0 \n", "\n", " Condition 1_Norm Condition 1_PosA Condition 1_PosN Condition 1_RRAe \\\n", "0 1 0 0 0 \n", "1 0 0 0 0 \n", "2 1 0 0 0 \n", "3 1 0 0 0 \n", "4 1 0 0 0 \n", "\n", " Condition 1_RRAn Condition 1_RRNe Condition 1_RRNn Condition 2_Artery \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " Condition 2_Feedr Condition 2_Norm Condition 2_PosA Condition 2_PosN \\\n", "0 0 1 0 0 \n", "1 0 1 0 0 \n", "2 0 1 0 0 \n", "3 0 1 0 0 \n", "4 0 1 0 0 \n", "\n", " Condition 2_RRAe Condition 2_RRAn Condition 2_RRNn Bldg Type_1Fam \\\n", "0 0 0 0 1 \n", "1 0 0 0 1 \n", "2 0 0 0 1 \n", "3 0 0 0 1 \n", "4 0 0 0 1 \n", "\n", " Bldg Type_2fmCon Bldg Type_Duplex Bldg Type_Twnhs Bldg Type_TwnhsE \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " House Style_1.5Fin House Style_1.5Unf House Style_1Story \\\n", "0 0 0 1 \n", "1 0 0 1 \n", "2 0 0 1 \n", "3 0 0 1 \n", "4 0 0 0 \n", "\n", " House Style_2.5Fin House Style_2.5Unf House Style_2Story \\\n", "0 0 0 0 \n", "1 0 0 0 \n", "2 0 0 0 \n", "3 0 0 0 \n", "4 0 0 1 \n", "\n", " House Style_SFoyer House Style_SLvl Roof Style_Flat Roof Style_Gable \\\n", "0 0 0 0 0 \n", "1 0 0 0 1 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 1 \n", "\n", " Roof Style_Gambrel Roof Style_Hip Roof Style_Mansard Roof Style_Shed \\\n", "0 0 1 0 0 \n", "1 0 0 0 0 \n", "2 0 1 0 0 \n", "3 0 1 0 0 \n", "4 0 0 0 0 \n", "\n", " Foundation_BrkTil Foundation_CBlock Foundation_PConc Foundation_Slab \\\n", "0 0 1 0 0 \n", "1 0 1 0 0 \n", "2 0 1 0 0 \n", "3 0 1 0 0 \n", "4 0 0 1 0 \n", "\n", " Foundation_Stone Foundation_Wood Heating_Floor Heating_GasA \\\n", "0 0 0 0 1 \n", "1 0 0 0 1 \n", "2 0 0 0 1 \n", "3 0 0 0 1 \n", "4 0 0 0 1 \n", "\n", " Heating_GasW Heating_Grav Heating_OthW Heating_Wall Central Air_N \\\n", "0 0 0 0 0 0 \n", "1 0 0 0 0 0 \n", "2 0 0 0 0 0 \n", "3 0 0 0 0 0 \n", "4 0 0 0 0 0 \n", "\n", " Central Air_Y MS SubClass_20 MS SubClass_30 MS SubClass_40 \\\n", "0 1 1 0 0 \n", "1 1 1 0 0 \n", "2 1 1 0 0 \n", "3 1 1 0 0 \n", "4 1 0 0 0 \n", "\n", " MS SubClass_45 MS SubClass_50 MS SubClass_60 MS SubClass_70 \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 1 0 \n", "\n", " MS SubClass_75 MS SubClass_80 MS SubClass_85 MS SubClass_90 \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " MS SubClass_120 MS SubClass_150 MS SubClass_160 MS SubClass_180 \\\n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 0 0 0 \n", "4 0 0 0 0 \n", "\n", " MS SubClass_190 \n", "0 0 \n", "1 0 \n", "2 0 \n", "3 0 \n", "4 0 " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def train_and_test(data):\n", " train = data[0:1460]\n", " test = data[1460:]\n", " features = data.columns.drop(['SalePrice'])\n", " \n", " #train\n", " lr = LinearRegression()\n", " lr.fit(train[features], train['SalePrice'])\n", " #predict\n", " \n", " predictions = lr.predict(test[features])\n", " rmse = np.sqrt(mean_squared_error(test['SalePrice'], predictions))\n", " return rmse\n", "\n", "data = pd.read_csv('AmesHousing.txt', delimiter='\\t')\n", "\n", "transformed_data = transform_features(data)\n", "final_data = select_features(transformed_data)\n", "result = train_and_test(final_data)\n", "\n", "print(result)\n", "\n", "final_data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Cross Validation\n", "\n", "We've selected the first 1460 rows as the training set, and the remaining data as the testing set. This is not really a good way to evaluate a model's performance because the error will change as soon as we shuffle the data.\n", "\n", "We can use KFold cross validation to split the data in K number of folds. Using the KFold function from scikit learn, we can get the indices for the testing and training sets." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import KFold\n", "\n", "def train_and_test2(data, k=2): \n", " rf = LinearRegression()\n", " if k == 0:\n", " train = data[0:1460]\n", " test = data[1460:]\n", " features = data.columns.drop(['SalePrice'])\n", " \n", " #train\n", " rf.fit(train[features], train['SalePrice'])\n", " \n", " #predict \n", " predictions = rf.predict(test[features])\n", " rmse = mean_squared_error(test['SalePrice'], predictions)**0.5\n", " return rmse\n", " \n", " elif k == 1:\n", " train = data[:1460]\n", " test = data[1460:]\n", " features = data.columns.drop(['SalePrice'])\n", " \n", " rf.fit(train[features], train[\"SalePrice\"])\n", " predictions_one = rf.predict(test[features]) \n", " \n", " mse_one = mean_squared_error(test[\"SalePrice\"], predictions_one)\n", " rmse_one = np.sqrt(mse_one)\n", " \n", " rf.fit(test[features], test[\"SalePrice\"])\n", " predictions_two = rf.predict(train[features]) \n", " \n", " mse_two = mean_squared_error(train[\"SalePrice\"], predictions_two)\n", " rmse_two = np.sqrt(mse_two)\n", " return np.mean([rmse_one, rmse_two]) \n", " \n", " else:\n", " kf = KFold(n_splits=k, shuffle=True, random_state = 2)\n", " rmse_list = []\n", " for train_index, test_index in kf.split(data):\n", " train = data.iloc[train_index]\n", " test = data.iloc[test_index]\n", " features = data.columns.drop(['SalePrice'])\n", " \n", " #train\n", " rf.fit(train[features], train['SalePrice'])\n", " \n", " #predict \n", " predictions = rf.predict(test[features])\n", " \n", " rmse = mean_squared_error(test['SalePrice'], predictions)**0.5\n", " rmse_list.append(rmse)\n", " return np.mean(rmse_list)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "99 29215.144646883306\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAboAAAEXCAYAAADbdYG1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XecXFX9//HXZ2Z7NrubThopZCmhJQEhSu+hhi58FVCxg4g/FVFUQOX7FRuCAn4VUGJDBPmCtFBCKFJCTyOwCYFU0rYk23dnP78/7t3N7MzsZjZltr2fj8c82HvuuXfOHCb72VPuOebuiIiI9FWR7i6AiIjIrqRAJyIifZoCnYiI9GkKdCIi0qcp0ImISJ+mQCciIn2aAp2IZIyZXWdmS7eR52gzczMbk6lySd+mQCf9mpn9Kfyl6mYWM7NVZjbLzEZ3kO/+FPc4MzzXnJB+hpm9YGblZlZjZkvN7K9mVhSeHx/33omvb+3aT570Geaa2R0JaSeZ2RYzu9XMImb2mQ7K+rtMllWkq7K6uwAiPcDzwPlAFNgDuBX4J/CJhHwrgNPNbIS7r4tL/yLwIdDWAjGzY4F/AT8OzzcAk4AzgdyE+84E5iWkbd6Bz7PDzOxi4A7gene/IUwDiBH3OUO1mS2dSNeoRScCje7+kbuvdvfngN8DH29tecUpA14GPtOaYGa7AycAf0zIewbwlrtf7+6L3X2Zu89296+4+4aEvOXh+8e/UgYPMzshbHmOTUj/pJnVm1lJePw9M3vfzBrMbIOZzTaz/HQqw8yuJghyX2gNcvFSlHVz3LWnmNnr4fuuN7PbzGzANt7va2FLutbMZgO7J5wvMrM/mtlH4X1Xmtmv0vksIqBAJ9KOmY0CziVoucRSZPk98HkLmzfA54GnCVp08dYCk8zskJ1cxKfDe386If0i4EF3rzSzs4Grga8DpQSB+LE07h0xs98A1wBnuPvdXSmYmR0APAQ8B0wBLgFOAzrs2jSzmcBNwK/Ca+4Ffp6Q7SfANIKWbynwSeCdrpRN+jd1XYrA0WZWTfCHX2ur55fuXpMi733AzeE1zwGfA64AElt/vwEOA14xs4+AV4BngL+4+6aEvE+YWUtC2gnu/lLim7t7i5n9hSCw/Q+AmQ0HTiLoFgUYB3wEPO7uTQRdrm91VgGhTwE5wFFhyzaVaFhX8aa5+3vAt4E33P0bYfo7ZvY14AEz+767J/4xQHjNP9y9tYX2npntA3wzLs844E13fyU8XgG8mMbnEQHUohOBIAhNAQ4hGFN7GfhBqozuXg/8GfgCcCrBH4v/TpGv1t3PACYA3wXWhP99N/xFHu+z4fvHv97spLx3A/uY2cfC4wuBTcDs8PheIBv4MJxEc5GZDezkfq1eJmgt/iRFt22rWIqyLg/P7UvQmov3LGDA5A7uN5nkoPVCwvFtwLlmttDMbjazk81Mv7skbfqyiECduy9194Xu/kOCbshbO8n/v8DZwFXAH8NWU0ru/oG7/8ndvwrsA3h4XbzV4fvHv+o7uec7wGvAxWHSxcDf3L05PL8a2JugtbmeIGi/mziul0IZcATBGNlTZjaog/dPLGv85+9oO5TOtknpdAsVd28dt7sByAP+Aswxs2hn14m0UqATSXYdcImZHZzqZBhoXiWYlXlHqjwdXFdB0KU4fCeUcRZwgZkdSDB+1W48zd0b3P1xd78K2B8oYGvXZmdlXEYQ7EqAuWG3aLoWAUclpB1FEMgWd3DNYoIu3niJx7h7ubv/3d2/RNCSPoqOW4ki7WiMTiSBuy8xs4cJxsBO6CDbSUCeu5enOmlm1wGFwCPAB+HPlwD7EYzxxRtsZrslpNW4+5ZOivl34JfAn4D57v523HtfSvBH7DygEjgOGEjHwaYdd19pZkcATwHPmtlx7r4mjUt/DrwRzoj8PTCeYKzyr+6+ooNrfgn808zmAY8ChxOMP7YxsxuA1wkCaQvBWGI1wVidyDapRSeS2s+A483suFQnwzG4lEEu9CwwluCxg3cIJqJ8HPi0uye2Ah8kGBuLf/1PZ4Vz940EQXQKQesuXgXBuN/c8L3/H/BFd3+6s3sm3H8dQaupGnjezMancc18gscqjgLeJhjLfAT4cifXPEAw8eQqYD5BEPtOQrZ64EcEwe414ADgZHevSvfzSP9m2mFcRET6MrXoRESkT1OgExGRPk2BTkRE+jQFOhER6dP6zeMFVVVVmnUjItLHFRcXW2KaWnQiItKnKdCJiEifpkDXRWVlZd1dhB5HdZJMdZJMdZJMdZJsV9RJRgKdmeWZ2Twze9vMFpnZ9WH6n8xsuZm9Fb6mhOlmZreY2VIzm29m0+LudYmZlYWvS+LSDzKzBeE1t8TtFyYiIv1YpiajNADHunu1mWUDL5hZ60aQ33b3+xLyn0ywwWIpcChwO3ComQ0GrgUOJlgo9nUzeyhcLPd24IsEW408Cswgvc0mRUSkD8tIi84DrZs1ZoevzmZBzgRmhde9DJSY2UiChXSfDFcyrwCeBGaE54rc/SUP1jSbRRortYuISN+XsTE6M4ua2VsE+2M9Gbdb8A1h9+RNZpYbpo0GVsZdvipM6yx9VYp0ERHp5zL2HJ27x4ApZlYCPGBm+xHsuPwRkEOwrcd3CFYpTzW+5tuRntKODnZqADmZ6iSZ6iSZ6iSZ6iRZV+uktLS00/MZf2Dc3SvNbC4ww91/ESY3mNkfgW+Fx6sItjhpNQZYE6YfnZA+N0wfkyJ/StuqlM6UlZXt0PV9keokmeokmeokmeok2a6ok0zNuhwWtuQws3zgeGBJOLZGOEPyTGBheMlDwMXh7MvpQJW7rwVmAyea2SAzGwScCMwOz20xs+nhvS4m2ONrp3l/czOvrGvglcoIj66o46Pa2M68vYiI7CKZatGNBO42syhBcL3X3R82szlmNoyg6/Ettm7Q+ChwCrAUqCXYRBJ3LzezHwOvhvl+FLf55VcIdlvOJ5htuVNnXF4zr4rHVtYDebCwnL8cO5jTxuXvzLcQEZFdICOBLtx5eGqK9GM7yO/AZR2cuwu4K0X6a8B+O1bSjhVktR8GrGvW0pkiIr2BVkZJU35ioIsp0ImI9AYKdGnKj6pFJyLSGynQpSmpRadAJyLSKyjQpSkx0NWq61JEpFdQoEtT4mSUerXoRER6BQW6NGmMTkSkd1KgS1Oeui5FRHolBbo06Tk6EZHeSYEuTYldl7UKdCIivYICXZqSJqOo61JEpFdQoEtT4hidui5FRHoHBbo0qetSRKR3UqBLkyajiIj0Tgp0adISYCIivZMCXZqSWnSajCIi0iso0KUpTyujiIj0Sgp0aUq1H12LK9iJiPR0CnRpipiRF22fpmfpRER6PgW6Lkhs1WkHAxGRnk+BrgsKou2rS8/SiYj0fAp0XZCX1f5YMy9FRHo+BbouyM9Si05EpLdRoOuCAj1iICLS6yjQdUHSZBR1XYqI9HgKdF2QGOjUdSki0vMp0HVB4g4G6roUEen5MhLozCzPzOaZ2dtmtsjMrk84/xszq447zjWzf5jZUjN7xczGx537bpj+rpmdFJc+I0xbamZX74rPoRadiEjvk6kWXQNwrLsfCEwBZpjZdAAzOxgoSch/KVDh7pOAm4Abw7yTgQuAfYEZwG1mFjWzKHArcDIwGbgwzLtTaaseEZHeJyOBzgOtLbbs8OVhgPo5cFXCJTOBu8Of7wOOMzML0+9x9wZ3Xw4sBQ4JX0vd/X13bwTuCfPuVJqMIiLS+2RsjC5seb0FrAeedPdXgMuBh9x9bUL20cBKAHdvBqqAIfHpoVVhWkfpO1XiDgbquhQR6fmytp1l53D3GDDFzEqAB8zsSOA84OgU2S1FmneSnipgdxiFysrKtlneVGors4CctuM1G8opK1u3Xffqa7a3Tvsy1Uky1Uky1UmyrtZJaWlpp+czFuhauXulmc0FjgEmAUuDXkkKzGxpOC63ChgLrDKzLKAYKI9LbzUGWBP+3FF6km1VSkfGNlXDh1Vtx7lFJZSWJg4v9j9lZWXbXad9leokmeokmeok2a6ok0zNuhwWtuQws3zgeOB1d9/N3ce7+3igNgxyAA8Bl4Q/nwvMcXcP0y8IZ2VOAEqBecCrQKmZTTCzHIIJKw/t7M+ROBlFuxeIiPR8mWrRjQTuDiefRIB73f3hTvLfCfzZzJYStOQuAHD3RWZ2L7AYaAYuC7tEMbPLgdlAFLjL3Rft7A+R9BydJqOIiPR4GQl07j4fmLqNPIVxP9cTjN+lyncDcEOK9EeBR3espJ3L03N0IiK9jlZG6QI9Ryci0vso0HWBlgATEel9FOi6IKlFpzE6EZEeT4GuCxJXRlGLTkSk51Og6wKtjCIi0vso0HWBJqOIiPQ+CnRdkNR1qTE6EZEeT4GuC1LNugwWbBERkZ5Kga4LohEj27YGNgcaYt1XHhER2TYFui7KS6gxdV+KiPRsCnRdlBdtH9g081JEpGdToOuixBaddjAQEenZFOi6KDHQ1arrUkSkR1Og66LchK7LuuaWbiqJiIikQ4Gui5Imo6jrUkSkR1Og66KkrksFOhGRHk2BrosSZ13Wa4xORKRHU6Droly16EREehUFui7KiyRORlGgExHpyRTouiixRadAJyLSsynQdVFetP2xnqMTEenZFOi6KLHrUiujiIj0bAp0XaTHC0REehcFui5KWhlFXZciIj2aAl0XaWUUEZHeRYGuixToRER6l4wEOjPLM7N5Zva2mS0ys+vD9DvDtPlmdp+ZFYbpuWb2DzNbamavmNn4uHt9N0x/18xOikufEaYtNbOrd9VnSVwZRV2XIiI9W6ZadA3Ase5+IDAFmGFm04FvuPuB7n4AsAK4PMx/KVDh7pOAm4AbAcxsMnABsC8wA7jNzKJmFgVuBU4GJgMXhnl3Oq2MIiLSu2Qk0HmgOjzMDl/u7psBzMyAfKA1aswE7g5/vg84LswzE7jH3RvcfTmwFDgkfC119/fdvRG4J8y706nrUkSkd8nK1BuFra7XgUnAre7+Spj+R+AUYDHwzTD7aGAlgLs3m1kVMCRMfznutqvCNFrzx6Uf2lFZysrKtvtz5EWs3XFVbcMO3a+vUB0kU50kU50kU50k62qdlJaWdno+Y4HO3WPAFDMrAR4ws/3cfaG7fzYMgr8BPgn8EbBUt+gkPVXLtMOm1rYqpTOrFyxtdxyLZlNaOna779cXlJWV7VCd9kWqk2Sqk2Sqk2S7ok4yPuvS3SuBuQRjbK1pMeAfwDlh0ipgLICZZQHFQHl8emgMsKaT9J1OizqLiPQumZp1OSxsyWFm+cDxwLtmNilMM+B0YEl4yUPAJeHP5wJz3N3D9AvCWZkTgFJgHvAqUGpmE8wsh2DCykO74rNoUWcRkd4lU12XI4G7wy7KCHAv8AjwvJkVEXRJvg18Jcx/J/BnM1tK0JK7AMDdF5nZvQTjec3AZWFrEDO7HJgNRIG73H3RrvggSUuAxRx3J4jVIiLS02Qk0Ln7fGBqilOHdZC/Hjivg3M3ADekSH8UeHQHipmWrAhkR6CpJThucWhsgdxo59eJiEj30Moo2yE/q33rTd2XIiI9lwLddiiIJgQ6rY4iItJjKdBthzy16EREeg0Fuu2Q2KLTMmAiIj2XAt120BidiEjvoUC3HZICncboRER6LAW67ZCfOBlFLToRkR5LgW47qOtSRKT3UKDbDomBrra5pZtKIiIi26JAtx0KEgJdfaybCiIiItukQLcd1KITEek9FOi2gyajiIj0Hgp02yE/q3216fECEZGeS4FuOyR3XSrQiYj0VNsMdGZ2S8LxpQnH9+/sQvV0iUuA1SvQiYj0WOm06D6TcPzzhOMTdk5Reo+kFp26LkVEeqx0Al3i1tn9fivtxEA3d00Dl84t56dvbmZ1jZ41EBHpSdLZYTyxudLvmy+Jsy7X17Vw//I6AP6+tJYXzhzOwGwNf4qI9ATpBLosMzuGrS25xOPoLilZDzZuYMcf+cPqGL9dWM13pxZlsEQiItKRdALdeuCuuONNCcfrd2qJeoE9i7O4fN9C/vedappSPCv+24XVfHavAexW0O/+BhAR6XG2GejcfXwGytGrmBk/OaSY704dyOqaGKtrYnzhuQo21gdRr6bZufGtzdz0iUHdXFIREdmugSQz28vMzjKzcTu7QL3JgOwIe5Zkc8zoPL4zZWC7c7Peq+XdyqZuKpmIiLRK5zm6X5rZp+OOLwYWAb8HlpjZybuwfL3GZ/YawMS4sbuYw/Wvb+7GEomICKTXojsTeC7u+L+BK9x9GPBl4NpdUbDeJjtiXHtwcbu0R1fU89K6hm4qkYiIQHqBbpi7rwAws/2AIcCd4bm/AHvuorL1OmeMy+PgYdnt0m5bVN1NpREREUgv0FWZ2Yjw5yOA19y9tZmSTRoPkJtZnpnNM7O3zWyRmV0fpv/VzN41s4VmdpeZZYfpZma3mNlSM5tvZtPi7nWJmZWFr0vi0g8yswXhNbeYWcYfbDczfnhQ+1bd7JX1VDZoGx8Rke6STqC7F7jHzK4Argb+FnfuUGBZGvdoAI519wOBKcAMM5sO/BXYG9gfyAc+H+Y/GSgNX18Ebgcws8EEXaWHAocA15pZ69TG28O8rdfNSKNcO90Ru+UwPm6srrEFHvqwrjuKIiIipBforgbmEqxp+Xvgf+POTQnTOuWB1j687PDl7v5oeM6BecCYMM9MYFZ46mWgxMxGAicBT7p7ubtXAE8SBM2RQJG7vxTeaxbB2GLGmRnnTixol/bPZbXdURQRESG95+iagOs7OHdzum9kZlHgdWAScKu7vxJ3Lhu4CPh6mDQaWBl3+aowrbP0VSnSu8X5E/P5xdtb2o5f+KiRNTUxRg3QA+QiIpm2zUAXPk7QKXeflUaeGDDFzEqAB8xsP3dfGJ6+DXjO3Z9vfdtUt9iO9JTKysq2VdxObet6A/YpzOWd6mhbQX43bwUXjWneofftyXa0Tvsi1Uky1Uky1UmyrtZJaWlpp+fTWQLsT8BS4CM6DijbDHRtmd0rzWwuwRjaQjO7FhgGfCku2ypgbNzxGGBNmH50QvrcMH1MivwpbatSOlNWVpbW9Z9urOaaeVVtx89sHsCPSodv9/v2ZOnWSX+iOkmmOkmmOkm2K+oknTG6W4ACYAvwW+B4dz8i7nXktm5gZsPClhxmlg8cT/Cw+ecJxt0udPf4qYkPAReHsy+nA1XuvhaYDZxoZoPCSSgnArPDc1vMbHo42/Ji4ME062CXOGdCPpG4PwsWlDexRCuliIhk3DYDnbtfCYwj6F48G/jAzP5gZod34X1GAs+Y2XzgVYIJJQ8DvwNGAC+Z2Vtm9sMw/6PA+wQtyT8AXw3LUg78OLzHq8CPwjSArwB3hNcsAx7rQvl2ut0Kohw5MrddmialiIhkXjpdl63ja48Aj5hZEfB9YK6ZneDuz6Rx/Xxgaor0lO8fzpy8rINzd9F+94TW9NeA/bZVlkw6b2I+c9dsXRnln+/Xcc20IiKZf8RPRKTfSntRZzMrNrMvAY8DZxG0rN7aVQXrC04fl09e3ETLFdUx5qzWkmAiIpmUzqLOp5nZP4F3CFpl33b3Une/PnyWTTpQlBPhjPH57dL+sKSmm0ojItI/pdN1+RDwLsEqJnXASWZ2UnwGd/9hqgsFvrB3Ifcu27oyyhMr6/lgSzPjB6bVaywiIjsona7LWcDLwFCCKf+JrzEdXyoHD8tmypCtCz07cKdadSIiGZPOyiif6eicmR1IMDFFOmBmfGGfAVz2QmVb2p/fq+G7UwdSkLVd+96KiEgXpDNGV2BmPzazf5vZr8ysyMwmmtkDwAvA+l1fzN7t7AkFDMrdOtOystG5/30t9CwikgnpNCluBU4HFhM86H0/8CzBLuMT3D3lYwCyVX6WcXHpgHZpv3+nhuApiq3e39zMvctq2VQfy2TxRET6tHRmRJwETHH39Wb2G2AFcFTcupSShs/tPYBbFla3LcC5oLyJeesbOXRE8FD5wvImTnxkA7XNTmGW8dfjBnPUqLzuK7CISB+RTouu0N3XA7j7KqBaQa7rxg3M4qSx7QPXrLKtK6X87K3N1DYHYbC62fnkU5uYs7o+o2UUEemL0gl0WWZ2jJkda2bHAsQft6bJtl26d/vuyweX11HT1EJFQwuPr2wf1OpjcOHTm3hylYKdiMiOSKfrcj3tl9zalHDswMSdWai+6thRuYwsiLC2Nli/urrZ+feH9dQ0t9DYkpy/IQafenoT9xw/hGNHqxtTRGR7pPN4wfgMlKNfiEaM8ycWcPPC6ra0vy+tpbY5RZQLNbbAl56rYN7ZIxiUq8cRRES6Sr85M+zC0oJ2x8+ubeDVDe237/nmAYXtjjfUt/D9V6sQEZGuU6DLsL1Lspk2NLvD84ftlsMPDirmqikD26X/tayWZ9dovE5EpKsU6LrBhZMKtnnuWwcMZJ+S9j3LV75YSW1zC00tzqrqZuqaPdUtREQkjgJdNzhnQj45KWo+P2qcMS7Y7SAnatx8WAnxO9ct3xJjyn3rGDlrDfv9cx37/GOtHkEQEdkGBbpuMDgvyoyxybMoTx+XR1FcBDxkeC6f36f9Iwnr61pobchVNjoXzSnn7U2Nu7S8IiK9mQJdN0nVfXlBirQfHlTE6IJoUnqrmmbngqc2saZGy4aJiKSiQNdNjh+Tx5gBWwPY+IFRjhqZm5RvYHaE248cREGWJZ1rtba2hU8+tYm3NjbyyroG/vNRA9VNHT+yICLSn2j3z26SHTFmHTOYa16tImJww8eKiUZSB7MjR+byylnDeaeimeH5EfYozuK/39jM7Yu37mu3oLyJo/+9oe24IMuYfeow9h/c8QxPEZH+QIGuG00blsNjpwxLK+/YwizGFm793/WTjxWzfEssaemwVrXNzg9ereL/Thq6U8oqItJbqeuyl4pGjDuOGsQBnbTYnl/boC1/RKTfU6DrxQqzI9x34hDOGp/PHkVRJg/Koihna/dnzOGRFXr8QET6NwW6Xm54fpQ/HjOY18/ZjRfPHMFXJ7dfPuz/lmsncxHp3xTo+pgzJ+S3O352bQPl6r4UkX5Mga6P2bskm73jlg6L7758dk0DV/yngr+U1eCu5cNEpH/ISKAzszwzm2dmb5vZIjO7Pky/3MyWmpmb2dC4/GZmt4Tn5pvZtLhzl5hZWfi6JC79IDNbEF5zi5l1/OBZH3fG+Patugc/qOOepbXMnL2RWe/VcvkLldy0oLqDq0VE+pZMtegagGPd/UBgCjDDzKYD/wGOBz5MyH8yUBq+vgjcDmBmg4FrgUOBQ4BrzWxQeM3tYd7W62bsyg/Uk52ZEOieWdPA5S9UtEu74Y3NzFvfkMliiYh0i4wEOg+0NiGyw5e7+5vu/kGKS2YCs8LrXgZKzGwkcBLwpLuXu3sF8CRB0BwJFLn7Sx70yc0CztzVn6un2qckiz2L23dfJm50EHO49NkKKhu0goqI9G0ZG6Mzs6iZvQWsJwhWr3SSfTSwMu54VZjWWfqqFOn9kpkldV+msrI6xv97qVLjdSLSp2VsZRR3jwFTzKwEeMDM9nP3hR1kTzW+5tuRnlJZWdm2itupHb0+E6ZFDEgOdvsUxnineusam/9aXsei9TXsltvCiFzn2CExDirpeiuvN9RJpqlOkqlOkqlOknW1TkpLSzs9n/ElwNy90szmEoyhdRToVgFj447HAGvC9KMT0ueG6WNS5E9pW5XSmbKysh26PlMmubPn++t5r6q5Le3r+xXyvWlFHP/wBhaUN7Wlv1sT4d2aoHF/79psbjmshIv3HJB0z470ljrJJNVJMtVJMtVJsl1RJ5madTksbMlhZvkEE1CWdHLJQ8DF4ezL6UCVu68FZgMnmtmgcBLKicDs8NwWM5sezra8GHhwV36mns7M+OmhxW0rpXx58gCuPbiI3Khx51Gd74bw9f9Ucu+y2kwVVURkl8pUi24kcLeZRQmC673u/rCZXQFcBewGzDezR93988CjwCnAUqAW+CyAu5eb2Y+BV8P7/sjdy8OfvwL8iaC/7rHw1a8dOzqPxefvRlMLDMrd+jfNniXZ/OHIQVz2QgWVjck9vA585fkKcqPGzDTG+kREerKMBDp3nw9MTZF+C3BLinQHLuvgXncBd6VIfw3Yb4cL28cUZqdutJ86Lp8Tx+axuibG2toY8zc1cc28qrbZmTGHS+eW87vhOexVnMXeg7I5ZlQue5Vo2x8R6V20TU8/lh0xxg/MYvzALD4+Ipfh+REufbaCljDYNTu8tK6Rl9Y1tl2z76AszplYwDkT8hk3UF8fEen59JtK2pw1oYD6WNBt2ZFFFc0sen0zP359MyeOzePL+wxgtJ5OEJEeTGtdSjsXTirgrqMGMaqg86+GA7NX1nPWE5v45Bt5/PtD7ZIgIj2TWnSS5OyJBcwcn8/KmhhLKpt4t7KZuWsaeHZtQ1u3ZrzldREumlPOtw4YyPemDSTSf5cZFZEeSIFOUorGjd/NGAtf338g6+tiPPhBHX8tq+WtTU1J1/xi/hbeq2ri9iMGMaCDSTAiIpmm30aStuH5Ub6wTyHPnD6M2acM5ewJ+UQTGm8PfVjPCQ9v4DcLtrCgvImWFMuLVTa08P15VXzthQo+2NKcdF5EZGdSi066zMw4dEQuh47I5ZV1DVzw5AYqmrZGvMWVzfzgtc3w2mZG5Ef43tQiLtkrWGllc2MLpz62gUUVQYB7fGU9L581nCF50ZTvJSKyo9Sikx1y6Ihc7j6wnsklqf9mWlfXwtdfrOS616poanE+O7e8LcgBbKhv4XvzqjJVXBHphxToZIeNzHNmnzaMmePzOszz6wXVTH9gHU+vTt4D7x/L6pizun5XFlFE+jEFOtkpBmZHuPuYIbx69nBuPLSYk8fmkdgbuWxzrMPrr3yxkpqmFuqanefWNvDyugZtHyQiO4XG6GSnKi3OprQ4my9NLuTNjY2c+8QmNqXY3HVkQYSPalva9lJaUR3juIc38OGWGHWxIPXksXnccZRmcIrIjtFvENllpg7N4fFThzJmQPumXVGO8cBJQ/ny5PZbAS2pbG4LcgCPrazntMc3sr4uxpamFn63uJqZj2/kwqc28f5mzdYg1h7eAAAbc0lEQVQUkfSoRSe7VGlxNrNPHcaFT21ifnkTg3KNWccMYe+SbK6ZVsTDK+pZWd1xl+abG5s4+qH11DQ7VXE7Lby1qZFnTh/ObgWarSkinVOLTna50QOizD1jGM/PHM6b5+zGESNzgWBnhV9/ooRIwrN4uQmxa01tS7sgB7C2toWL5myiIWwBvr+5mZvmb+FvZTVtaSIioBadZEjEjP0HJ2/xc9zoPB44cSiPraxjWF6Uk8bmsXthlM/OLU85QzPeqxua+Np/KhiUE+HOJTVtWwz9dmE1tx4xiKlDc3bFR2FNTYz/fNTAu1XNxFqcFg/W/hyUG6G0OIu9SrJoTh6WFJFuokAn3e6oUbkcNSq3Xdo9xw/hGy9W8peyrTudD86NMGpAlIXlW5cfu3dZ8mLSiyubOf7hDVyxXyEnjMkjP8soyDImFWURTWw+pml9XYxfvL2FJ1fVs3xLx12trXIj+VxaVclPPlastT9FupkCnfRI2RHjN4eVcOyoXJ5d28C0oTmct0c+jTE47uH1nT6qAMHGsTctqOamBdVtaaMKItw4vYTTx3Vt1/QPtjRz2mMbWVWz7QDXqqHFuG1RDUNyo3zzwIFdej8R2bk0Ric9lplx9sQCbj5sEJfsNYCCrAgluRH+dtwQBmYnt5KG5Hb+dV5T28JFc8r5wavBKi3pWFHdzBmPdy3IxfvJG5v1MLxIN1OLTnqdvUqyuevowVw0ZxP1MSjKNr554EC+tE8hz61t4MoXK1hT2/Eg2W8WVvPahkZ+Pr2EfQdlYR10La4Kg9yKhFmhWQYHDcvh0OE5lORGaL16dU2MdyubeH1jE7XhgKEDlz5bztzTh2tHdpFuon950iudMCaPV84awZLK5raAA3Di2DxePHMEv1tczcvrG6lrduqancUVTW2TVQBeWtfI4Q+uZ4+iKGeMy2e3gii1zU5Ns/PhlmYWlTfxXlVzu2sAzpuYz68/UdLpQ+zPrK7nnCc20hKGwIoG59Nzyvny5AFMGJjFniVZDE2xiPX6uhhVjS1MKuo4+IpI1ynQSa81bmBWylZSSW6Eq6cWtUubt76BzzxTntTSW7Y51m4crzNnT8jn9iMGkbWNCS3HjM7jy+OauO3DrbM+F5Q3cdkLlW3Hx43O5aoDB3LoiFw+2NLMDW9s5r7363Dg4GHZXH9wMYftFkzQaYg58zc1kR2BKbtoJqlIX6ZAJ/3CIcNzeW7mcD7/bAVz13T+2EIqM8fn8fsjtx3kWl0yppkPvIhHV6Qen3t6dQNPr25gypBsFpa3b22+tqGJUx/byIljcjEzXljbQE2Y4VOlBfzmsJIuz+RsiDk3vLGZJZVNfG7vAcwY27UJOSK9mSajSL8xNC/K/ScM4Q9HDuLEMbmks4TmmAFRrpk6kDuOGpx2kAOIGNx+xKAOty9q9dampqTu0VZPrGpg9sr6tiAH8NeyWn721pa24weW13L4g+s5/uH1/GNZbYcLYf/o9c3csrCaJ1Y1cOFT5fzno64He5HeSi066VeiEeO8PQo4b48CqhpbmL2ynnnrGwEoCJ+3G5IXYfKgbPYdlN029rc9inMizD5tGA9/WM+7lU0s39LMe5XNvFO5Y+t0/vStLZQWZ/HGxiZuXbS12/W1DRXc+U4NN04vbvew/Ia6GHcu2ZrPga88X8ELM4dTlKO/daXvU6CTfqs4J8L5exRw/h4Fu+w9BmZHuHBS+/u/8FEDN765mec/amxLG1cY5ZppRYwtjHLda5t5ZX1j4q3aufTZipTp8zY0cuy/N/DlyQO44ZDgYfX/XVxDfcLTESuqY3xvXhW/PXzQ9n0wkV5EgU4kww7fLZfDTx7Gy+saeGJVPZOKsjh3YgE50aBr9PFThvL4ynqeWt3A+MIox47Oo7KxhTNnb6QpjaXFHLh9cQ2DcyN8aXIhf1iSerLNX8pqOXlsHvsOzmZxRRNNLcEkmUJtiyR9TEYCnZnlAc8BueF73ufu15rZBOAeYDDwBnCRuzeaWS4wCzgI2AR80t0/CO/1XeBSIAZc4e6zw/QZwM1AFLjD3X+aic8msr2mj8hl+ojcpHQz4+Td8zl59/YTRn758RKu+E9lUv5RBRH2H5zN7FXtx93+560tzC9vSloQO96n5pQn3etn00s4rYurx4j0ZJn6060BONbdDwSmADPMbDpwI3CTu5cCFQQBjPC/Fe4+CbgpzIeZTQYuAPYFZgC3mVnUzKLArcDJwGTgwjCvSJ9x8Z4D+Oq+7ffw+/iIHJ45fTj/OGEo9584pN3qMC0O//6w/azPmePzyOpkTs2a2hY+Paec/3p6E6uqteef9A0ZCXQeaO0/yQ5fDhwL3Bem3w2cGf48MzwmPH+cBU/QzgTucfcGd18OLAUOCV9L3f19d28kaCXO3MUfSyTjfnxwMd+bOpBDh+fwvakDefCkoYwI9+Q7bnSwI3tHcWxAlnHTx0v49pRtr7356Ip6PvF/6zudnfnkqnoOuv8jpt33EXPXJD9G0dEMUJFMs0x9GcNW1+vAJILW18+Bl8NWG2Y2FnjM3fczs4XADHdfFZ5bBhwKXBde85cw/U7gsfAtZrj758P0i4BD3f3y1vevqqpq+6BlZWW78qOKdKs/rMji9yuSHyz/r1FNfGNi8DjDNUtymLMpGLkojDoTClpYXB0h5u3D5ICoc+t+Dew7sP3g4LzKCF9flEtzmD8v4syaUs+EguCf2T/XZnHrB9lkGRw+OMbJw5o5qKSl09akyPYqLS1t+7m4uDjpW5axySjuHgOmmFkJ8ACwT6ps4X9T/XPwTtJTtUw7jODxldJVZWVlO3R9X6Q6SdaddfLTSc6yJze1288vOwLXHD6W0QOC1t99pc7K6hgRC54VNDMWlTfxjRcrmbdh64zPmpjxjSX5PHLyMCYPCvYTnL+pkatf2Uhz3B/J9S3GdcsH8vRpw/lrWQ0/W1bVdu6R9Vk8sj6LwdnODw4exGf3bt/92p/p306yXVEnGZ9e5e6VwFxgOlBiZq3BdgywJvx5FTAWIDxfDJTHpydc01G6SL8TMeP3Rw5iXOHW9TQv27ewLci15hk3MIuxhVvX1dx3cDaPnzqUqxO6NisanDNnb+R3i6v5v+V1nPfkJrY0Jf8dubiimbOf2Mi3X65KOgdQ3mR846VKbl+U3pJrIjtLpmZdDgOa3L3SzPKB4wkmmDwDnEswpnYJ8GB4yUPh8Uvh+Tnu7mb2EPA3M/sVMAooBeYRtPRKw1mcqwkmrPxXJj6bSE80JC/KczOHM+u9GobnRzl/YnqzKCNmXD21iMYW51fztwak9XUtXP1K6gAW76V1nT//B/C9eVXsXhjl1DRmdm6oi1HR0EJpsRa6lu2Xqa7LkcDd4ThdBLjX3R82s8XAPWb2E+BN4M4w/53An81sKUFL7gIAd19kZvcCi4Fm4LKwSxQzuxyYTfB4wV3uvihDn02kRyrOifC1/bZv09cfTCtiS5Pzh3dqOs33+b0HMGd1Pe+n2HU9y+BHHytmYXkTDyyvoy62deuizz9bwe1HwOLKJh5dUc+Wxhau2L+QS/cubLv+wQ/q+NzccmIOF+wRLKitYCfbIyOBzt3nA1NTpL9PMGMyMb0eOK+De90A3JAi/VHg0R0urIhgZtx4aDGNMefu92pT5jlnQj4/m17M/E0FnPjIBhoTHma//YhBnBeuOnP2hHw++eRGYuEwe13M+czc9s/wffOlKkYVRDl593zW1cb42gsVhLGRe5bVcd4eBRw3Om/nflDpF7QEgoikFDHj5sMG8eSpw/j+tCIuKi3gyJG57D84m6/tV8htRwwiYsaUoTn8+GPF7a698dDitiAHcPyYPK7ao2mb7/nNlyrZ3NjC91+tYnPCOOD/vLlZjyzIdtESYCLSqY8Nz+FjwzvfB+9LkwsZkhfhubUNnLp7PieNTW55nT2ymer8wfxmYceTUdbUtvDJpzalHOt7bUMTT61u4IQxatVJ1yjQichOce7EAs6d2PkC2dcfXER9LBj7y47AMaNyyYsaD8Wt4NLZhJb/fnMzx4/OTRqrq2lq4b2qZvYoymq3I8Oi8iZ+vWAL8zc1cdq4PK6aUkRuVON8/Y0CnYhkTMSMn08v4ZqpQcDJzzLqmp1FD65j2ebkCS2J3tzYxOxV9W0bxy6rauYPS6r5W1ktm5ucnEiww/tpu+fx/NoG/hnu2g7w7vxqnlnTwKxjBjOmMPlX39KqJv70bi1FOcZJY/M4YHC2Jr/0EQp0IpJx8fv85WcFY4GnPbYxKd8lexawsb6FR+J2ar9mXhUPflDPB1uak1p/jS0we2U9s1em3tn9jY1NHP3vDdx19GCOHLl1Qe2X1zVwzhOb2ja5/e83t7B7YZRTds/j4yNyOWhoNqPDB+ul91GgE5Fud/huuXxmzwL+FDfDc0huhOsOLmZVTaxdoFu2OcayzalngqZjY32w5dFl+xbynSkDWVLZzHlPbmq3kzsEe/b9bnENv1scPGIxPD/CBXsUcNWUgdrKqJfR/y0R6RGu/1gx+w8OlhnLjcLNh5UwKDfYguiMcduegJIbTZ0+uSSLY0a13w6pxeE3C6s59F/rOeeJjSlXekm0vq6FWxZWc9GcchpjHedvavFOz3eFu/Pr+VvY5x9rOf2xDayt3Xb3riRTi05EeoTinAhPnDqM/3zUwLiBUUqLs9vOXTOtiGfWNKQMSNOGZvOlyYWcOT6fdyubePCDOp5c1UB2BL48uZCzJ+RjBj9/ews/fXNLu0VwV6cIHIcMy+Ht8kYaOogpz6xp4Ir/VLQ9wN7izlsbm3hmTQPPrKln3vpGomZ8qrSAK/cvTDkemK4fvb6ZmxYEs1TX1jbyubnlPHLyUCLqQu0SBToR6THys4zjUzw+sFdJNo+eMoyHP6wDYFRBlN0KouxRFGVSXEA8YEgOBwzJ4QcHJd/7O1OKOGhoDle+WMmqmtRR7FsHDuT704qobmphzuoGXlrXwJsbm3h7U1Pbyi4QPMBelBMhP2rc935dioDp3LGkhlnv1XDRngP4RkLA21Qf45YF1cxdkUvR0g2U5EYoyYmwV0kWZ00oYPSAKL+ev6UtyLV6aV0jf1tay6dLtTB2V2Rsm57uFr9Nz47QauPJVCfJVCfJekqd1DS18PO3t/DbhdXED8t9fb9Crju4KOWEkw11MU58ZAPLUyx1lo7sCPzXpAKu3H8gz3/UwLWvVVHRkPpXkgEHDcvmtQ2pH7AfnBvhtbOHMzivg77aXm5HvyeptunRGJ2I9CsDsoNJLi+cOZxzJ+az76Asfj69uMMgBzAsP8r9Jw5tt4N7VzS1wN3v1TL1/nVc8Z/KDoMcBGuBdhTkAMobWrj2tc3bVY7+Sl2XItIv7V2SzR1HDU47/8SiLP5xwhBOf2xju25MgIIs47jRuRwzKo8jR+bw4rpGfvH2FlZU7/jkkbwonDYun/ver2tL+3NZLTPG5nH0qFwGZEdwdyoaWlhd28Lg3AijCiKdPgqxqT7GwvImVlTHWFkTY01NjBH5Eb66byFD+mBLUYFORCRNBw/L4e/HD+arz1ewsb6FI0fmcv4eBZyye167Rw4mFWdz4aQC/r60ll/N38IHKbo886PGZ8c0cOLeI6loaGFldYx/La/jrU1bW3NZBncfM4RjRuWyYFMT71Y1t5371JxgUewR+RG2NDm1cf2wnxiRw3UHF3HI8GC26Ue1MZ5YVc/L6xqZt76RpZu33iferPdqufmwEk7ZPfUWSq1DXb3teUKN0XVRTxln6ElUJ8lUJ8n6Up24e9q/7JtbnPuX1/HLt7fwXhioThqTy8+ml9D40fKkOllc0cS/ltexoS7GJXsOYNqwYJ3R59c2cPrjyQ/Vd+b40blUNLTw+sZtL6gd74I98pk6NId1dTE+qm1hdU2MVTXNrK6J4cAnRuRy1oR8Th+Xz6Dt7M7tyK4Yo1OLTkSki7rSosmKGJ/co4BzJ+Tz9qYm8rKMfUqCjWTLPkrOP3lQNpMHZSelHzEyl0+VFvDXsvQfln9qdUPaeePds6yOe5bVdXg+eJSigf/3YiWnj8vnp4cWM6Kg53Z5KtCJiGRANGJtrbPt9auPl1BalMWL6xooq2rmw+oYLWFfVUGWMSQvwso0xgWjBvsOymbPkizGDojS0OL8fnENzV3s92p2eOCDOl5e38CsY4Zsc5eL7qJAJyLSS+RGjSsPGMiVBDvHN8acNbUxinMilOQYZsbzaxu4/vWqpJmb04Zmc9LYrWt3DkhYxuz8iQV8+fkKllSmHr/rzNraFk55bAM/n17ChZMK2naIKK+PsaC8iUUVzRSHi2UP7YbJLgp0IiK9VE7UGD+w/a/xI0bm8uSpw5i9qp7n1jawZ3EQ4EZuo2txytAc5p4+nNsXV/PWpkYG50YYnh9lt/wouxVEGFuYxZgBUdbVxXhgeR33L6+jLG5yTFMLXPliJVe+WMmArGBnio317bedH5BlfGGfAXxtv8zO7lSgExHpY8yMGWPz27YzSldelvGNAwZ2mqckN8LVU7P59oED+fEbm/n1guSNdGuaPWmR7Nb0Xy+o5o53avj0ngWcN7GAaUN3/XZIemBcRES6LBoxrju4mD8ePYiCrK4Fqupm53eLazju4Q1Mu38d//6w44kvO4MCnYiIbLezJhTw1GnDOG33PIbmRYjfwD3LYL/B2Zw/MZ/h+anDzfItMQZ0MVB2lbouRURkh0welM1fjhsCQIs7mxudLU0tDM+Ptk1MqW1u4a4lNdy8oJoNcWN3Q/Mi7TbB3RXUohMRkZ0mYkZJbjB5JTeueVeQFeHy/QYy/7zd+NPRgzlt9zxyo3Dm+HyyImrRiYhIH5GfZZw5IZ8zJ+RT1dhCfVcf3tsOCnQiItItinMiFGfgGfOMdF2a2Vgze8bM3jGzRWb29TD9QDN7ycwWmNm/zawo7prvmtlSM3vXzE6KS58Rpi01s6vj0ieY2StmVmZm/zCznvmIvoiIZFSmxuiagW+6+z7AdOAyM5sM3AFc7e77Aw8A3wYIz10A7AvMAG4zs6iZRYFbgZOBycCFYV6AG4Gb3L0UqAAuzdBnExGRHiwjgc7d17r7G+HPW4B3gNHAXsBzYbYngXPCn2cC97h7g7svB5YCh4Svpe7+vrs3AvcAMy142vBY4L7w+ruBM3f9JxMRkZ4u47MuzWw8MBV4BVgInBGeOg8YG/48GlgZd9mqMK2j9CFApbs3J6SLiEg/l9HJKGZWCNwPXOnum83sc8AtZvZD4CGgsTVrisud1IHZO8mfUllZWZfKvbOv74tUJ8lUJ8lUJ8lUJ8m6Wifb2r8uY4HOzLIJgtxf3f1fAO6+BDgxPL8ncGqYfRVbW3cAY4A14c+p0jcCJWaWFbbq4vMn6SubP4qIyLZlatalAXcC77j7r+LSh4f/jQDfB34XnnoIuMDMcs1sAlAKzANeBUrDGZY5BBNWHvJgm/RngHPD6y8BHtz1n0xERHq6TI3RHQZcBBxrZm+Fr1MIZk2+BywhaIH9EcDdFwH3AouBx4HL3D0WttYuB2YTTGi5N8wL8B3g/5nZUoIxuzsz9NlERKQHs6AxJCIi0jdprUsREenTFOi6oKNVWfqTTla5GWxmT4Yr0zxpZoO6u6yZFi5q8KaZPRwe9+vVesysxMzuM7Ml4ffl4/39e2Jm3wj/3Sw0s7+bWV5/+56Y2V1mtt7MFsalpfxeWOCW8HfufDObtj3vqUCXpm2sytKfdLTKzdXA0+HKNE+Hx/3N1wnGjlv199V6bgYed/e9gQMJ6qbffk/MbDRwBXCwu+8HRAkm1PW378mfCFa8itfR9+JkgsmIpcAXgdu35w0V6NKXclWWbi5TxnWyys1MghVpoB+uTGNmYwgej7kjPO7Xq/WE69YeSTgpzN0b3b2Sfv49IXikK9/MsoACYC397Hvi7s8B5QnJHX0vZgKzPPAywWNkI7v6ngp06etoVZZ+K2GVmxHuvhaCYAgM776SdYtfA1cBrTtK9vfVeiYCG4A/ht25d5jZAPrx98TdVwO/AFYQBLgq4HX69/ekVUffi53ye1eBLn1dWn2lr0tc5aa7y9OdzOw0YL27vx6fnCJrf/q+ZAHTgNvdfSpQQz/qpkwlHHeaCUwARgEDCLrmEvWn78m27JR/Rwp06etstZZ+JdUqN8C61i6F8L/ru6t83eAw4Awz+4CgS/tYghZeSdhFBf3v+7IKWOXur4TH9xEEvv78PTkeWO7uG9y9CfgX8An69/ekVUffi53ye1eBLn0pV2Xp5jJlXEer3BDUxSXhz/1qZRp3/667j3H38QTfiznu/in68Wo97v4RsNLM9gqTjiNYAKLffk8Iuiynm1lB+O+otU767fckTkffi4eAi8PZl9OBqtYuzq7QA+NdEK7m8muC2VJ3ufsN3VykjDOzw4HngQVsHY/6HsE43b3A7gT/oM9z98QB5z7PzI4GvuXup5nZRIIW3mDgTeDT7t7QneXLJDObQjA5Jwd4H/gswR/X/fZ7YmbXA58kmL38JvB5gjGnfvM9MbO/A0cDQ4F1wLXA/5HiexH+QfBbglmatcBn3f21Lr+nAp2IiPRl6roUEZE+TYFORET6NAU6ERHp0xToRESkT1OgExGRPk2BTqSXMbOzzGylmVWb2dRt5P2Tmf2kk/NuZpN2filFeg4FOpEeyMw+MLPj444vMLMKMzuKYL3Ey9290N3f7L5SivQOCnQiPZyZXUKwRdSp7v4sMA5Y1L2lEuk9FOhEejAz+yLwS+Ak4HUzqyZYmedtM1sW5tnHzOaaWWW4qecZndzv22a21szWmNnnEs6dYmaLzWyLma02s2/two8mkjFZ284iIt3kK8DhwHHu/naYVmhmDhzo7kvDBbb/DdwFnBjmf9DMDnb3d+NvZmYzgG8RrLG4HPhDwvvdCZzv7s+HK+1P2FUfTCST1KIT6blOAF4mWFe0I9OBQuCn4eamc4CHgQtT5D0f+KO7L3T3GuC6hPNNwGQzK3L3itYNdkV6OwU6kZ7ry8CewB3h4rapjAJWuntLXNqHpN6cchTtN7H8MOH8OcApwIdm9qyZfXz7ii3SsyjQifRc6wm6GY8AbusgzxpgrJnF/1veHVidIu9a2u/ttXv8SXd/1d1nEuzu3LqavEivp0An0oO5+xqCjVxnmNlNKbK8QrB791Vmlh1uE3Q6wbYvie4FPmNmk82sgGB7FADMLMfMPmVmxeGmoJuB2E7+OCLdQoFOpIdz95UEwe5cM/ufhHONwBnAycBGgpbfxe6+JMV9HiPYT3EOsDT8b7yLgA/MbDNBt+mnd/JHEekW2o9ORET6NLXoRESkT1OgExGRPk2BTkRE+jQFOhER6dMU6EREpE9ToBMRkT5NgU5ERPo0BToREenTFOhERKRP+/+jO3Msp3KXFwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "data = pd.read_csv(\"AmesHousing.txt\", delimiter='\\t')\n", "\n", "transformed_data = transform_features(data, percent_missing=0.05)\n", "final_data = select_features(transformed_data, 0.4, 10)\n", "\n", "results = []\n", "for i in range(100):\n", " result = train_and_test2(final_data, k=i)\n", " results.append(result)\n", " \n", "x = [i for i in range(100)]\n", "y = results \n", "plt.plot(x, y)\n", "plt.xlabel('Kfolds')\n", "plt.ylabel('RMSE')\n", "plt.title('RMSE vs KFolds')\n", "plt.savefig('rmse.png')\n", "\n", "min_rmse = min(results)\n", "\n", "print(results.index(min_rmse), min_rmse)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our error is actually the lowest, when k = 99. We can see the average RMSE goes down as we increase the number of folds. This makes sense as the RMSE shown on the graph above is an average of the cross validation tests. A larger K means we have less bias towards overestimating the model's true error. As a trade off, this requires a lot more computation time.\n", "\n", "***That is it for now though, the goal of this project is to explore the fundamentals of Linear Regression.***" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.8" } }, "nbformat": 4, "nbformat_minor": 2 }