"
]
},
{
"cell_type": "markdown",
"id": "2dad407d",
"metadata": {},
"source": [
"## Objective\n",
"- learn how to select variables for linear regression model through correlation analysis \n",
"\n",
"Correlation analysis is a bivariate analysis, which measures \n",
"- the strength of association between two variables and \n",
"- the direction of the relationship."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "af3b5d83",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
prov
\n",
"
gdpr
\n",
"
year
\n",
"
gdp
\n",
"
pop
\n",
"
finv
\n",
"
trade
\n",
"
fexpen
\n",
"
uinc
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
Guangdong
\n",
"
First
\n",
"
2000
\n",
"
1.074125
\n",
"
8.650000
\n",
"
0.314513
\n",
"
1.408147
\n",
"
0.108032
\n",
"
0.976157
\n",
"
\n",
"
\n",
"
1
\n",
"
Guangdong
\n",
"
First
\n",
"
2001
\n",
"
1.203925
\n",
"
8.733000
\n",
"
0.348443
\n",
"
1.501391
\n",
"
0.132133
\n",
"
1.041519
\n",
"
\n",
"
\n",
"
2
\n",
"
Guangdong
\n",
"
First
\n",
"
2002
\n",
"
1.350242
\n",
"
8.842000
\n",
"
0.385078
\n",
"
1.830169
\n",
"
0.152108
\n",
"
1.113720
\n",
"
\n",
"
\n",
"
3
\n",
"
Guangdong
\n",
"
First
\n",
"
2003
\n",
"
1.584464
\n",
"
8.963000
\n",
"
0.481320
\n",
"
2.346735
\n",
"
0.169563
\n",
"
1.238043
\n",
"
\n",
"
\n",
"
4
\n",
"
Guangdong
\n",
"
First
\n",
"
2004
\n",
"
1.886462
\n",
"
9.052298
\n",
"
0.587002
\n",
"
2.955899
\n",
"
0.185295
\n",
"
1.362765
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" prov gdpr year gdp pop finv trade fexpen \\\n",
"0 Guangdong First 2000 1.074125 8.650000 0.314513 1.408147 0.108032 \n",
"1 Guangdong First 2001 1.203925 8.733000 0.348443 1.501391 0.132133 \n",
"2 Guangdong First 2002 1.350242 8.842000 0.385078 1.830169 0.152108 \n",
"3 Guangdong First 2003 1.584464 8.963000 0.481320 2.346735 0.169563 \n",
"4 Guangdong First 2004 1.886462 9.052298 0.587002 2.955899 0.185295 \n",
"\n",
" uinc \n",
"0 0.976157 \n",
"1 1.041519 \n",
"2 1.113720 \n",
"3 1.238043 \n",
"4 1.362765 "
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# import required packages\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns\n",
"\n",
"# read data\n",
"df = pd.read_csv('./data/gdp_china_outlier_cl.csv')\n",
"\n",
"# display the first 5 rows\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"id": "f2fe14a2",
"metadata": {},
"source": [
"## 1. Visualize correlation using matrix scatterplot\n",
" - a great method to roughly determine if there is a linear correlation between multiple variables"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "f01d2ad7",
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# Increase the size of the heatmap.\n",
"plt.figure(figsize=(16,6))\n",
"# Store heatmap object in a variable to easily access it \n",
"# Set the range of values to be displayed on the colormap from -1 to 1, and \n",
"# set the annotation to True to display the correlation values on the heatmap.\n",
"heatmap = sns.heatmap(corr,vmin=-1,vmax=1,annot=True)\n",
"\n",
"# Give a title to the heatmap. \n",
"# Pad defines the distance of the title from the top of the heatmap.\n",
"heatmap.set_title('Correlation Heatmap',fontdict={'fontsize':12},pad=12)"
]
},
{
"cell_type": "markdown",
"id": "43de09e0",
"metadata": {},
"source": [
"
Table: Interpreting of correlation coefficient (Hinkle et al., 2003)
\n",
" 
\n",
" 
\n",
" ![](\"./images/shouke_logo.png\"\n",){kind=logo}
\n",
"