{
"cells": [
{
"cell_type": "code",
"execution_count": 171,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import seaborn as sns\n",
"import statsmodels.formula.api as smf\n",
"import sklearn.datasets as datasets\n",
"import sklearn.model_selection as model_selection\n",
"import statsmodels.stats.api as sms"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"5. In Chapter 4, we used logistic regression to predict the probability of default using income and balance on the Default data set. We will now estimate the test error of this logistic regression model using the validation set approach. Do not forget to set a random seed before beginning your analysis."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>default</th>\n",
" <th>student</th>\n",
" <th>balance</th>\n",
" <th>income</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>729.526495</td>\n",
" <td>44361.625074</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>817.180407</td>\n",
" <td>12106.134700</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1073.549164</td>\n",
" <td>31767.138947</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>529.250605</td>\n",
" <td>35704.493935</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>785.655883</td>\n",
" <td>38463.495879</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" default student balance income\n",
"0 0 0 729.526495 44361.625074\n",
"1 0 1 817.180407 12106.134700\n",
"2 0 0 1073.549164 31767.138947\n",
"3 0 0 529.250605 35704.493935\n",
"4 0 0 785.655883 38463.495879"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"default_df = pd.read_csv('default.csv')\n",
"default_df = default_df.drop(default_df.columns[0], axis=1)\n",
"default_df['default'] = default_df['default'].map({'No' : 0, 'Yes' : 1})\n",
"default_df['student'] = default_df['student'].map({'No' : 0, 'Yes' : 1})\n",
"default_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(a) Fit a logistic regression model that uses income and balance to predict default."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.078948\n",
" Iterations 10\n"
]
},
{
"data": {
"text/html": [
"<table class=\"simpletable\">\n",
"<caption>Logit Regression Results</caption>\n",
"<tr>\n",
" <th>Dep. Variable:</th> <td>default</td> <th> No. Observations: </th> <td> 10000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Model:</th> <td>Logit</td> <th> Df Residuals: </th> <td> 9997</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Method:</th> <td>MLE</td> <th> Df Model: </th> <td> 2</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Date:</th> <td>Mon, 24 Sep 2018</td> <th> Pseudo R-squ.: </th> <td>0.4594</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Time:</th> <td>19:11:48</td> <th> Log-Likelihood: </th> <td> -789.48</td> \n",
"</tr>\n",
"<tr>\n",
" <th>converged:</th> <td>True</td> <th> LL-Null: </th> <td> -1460.3</td> \n",
"</tr>\n",
"<tr>\n",
" <th> </th> <td> </td> <th> LLR p-value: </th> <td>4.541e-292</td>\n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <td></td> <th>coef</th> <th>std err</th> <th>z</th> <th>P>|z|</th> <th>[0.025</th> <th>0.975]</th> \n",
"</tr>\n",
"<tr>\n",
" <th>Intercept</th> <td> -11.5405</td> <td> 0.435</td> <td> -26.544</td> <td> 0.000</td> <td> -12.393</td> <td> -10.688</td>\n",
"</tr>\n",
"<tr>\n",
" <th>income</th> <td> 2.081e-05</td> <td> 4.99e-06</td> <td> 4.174</td> <td> 0.000</td> <td> 1.1e-05</td> <td> 3.06e-05</td>\n",
"</tr>\n",
"<tr>\n",
" <th>balance</th> <td> 0.0056</td> <td> 0.000</td> <td> 24.835</td> <td> 0.000</td> <td> 0.005</td> <td> 0.006</td>\n",
"</tr>\n",
"</table><br/><br/>Possibly complete quasi-separation: A fraction 0.14 of observations can be<br/>perfectly predicted. This might indicate that there is complete<br/>quasi-separation. In this case some parameters will not be identified."
],
"text/plain": [
"<class 'statsmodels.iolib.summary.Summary'>\n",
"\"\"\"\n",
" Logit Regression Results \n",
"==============================================================================\n",
"Dep. Variable: default No. Observations: 10000\n",
"Model: Logit Df Residuals: 9997\n",
"Method: MLE Df Model: 2\n",
"Date: Mon, 24 Sep 2018 Pseudo R-squ.: 0.4594\n",
"Time: 19:11:48 Log-Likelihood: -789.48\n",
"converged: True LL-Null: -1460.3\n",
" LLR p-value: 4.541e-292\n",
"==============================================================================\n",
" coef std err z P>|z| [0.025 0.975]\n",
"------------------------------------------------------------------------------\n",
"Intercept -11.5405 0.435 -26.544 0.000 -12.393 -10.688\n",
"income 2.081e-05 4.99e-06 4.174 0.000 1.1e-05 3.06e-05\n",
"balance 0.0056 0.000 24.835 0.000 0.005 0.006\n",
"==============================================================================\n",
"\n",
"Possibly complete quasi-separation: A fraction 0.14 of observations can be\n",
"perfectly predicted. This might indicate that there is complete\n",
"quasi-separation. In this case some parameters will not be identified.\n",
"\"\"\""
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"smf.logit(formula='default ~ income + balance', data = default_df).fit().summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(b) Using the validation set approach, estimate the test error of this model. In order to do this, you must perform the following steps:\n",
"\n",
"i. Split the sample set into a training set and a validation set\n",
"\n",
"ii. Fit a multiple logistic regression model using only the training observations\n",
"\n",
"iii. Obtain a prediction of default status for each individual in the validation set by computing the posterior probability of default for that individual, and classifying the individual to the default category if the posterior probability is greater than 0.5\n",
"\n",
"iv. Compute the validation set error, which is the fraction of the observations in the validation set that are misclassified."
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"data": {
"text/plain": [
"0.971"
]
},
"execution_count": 47,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train, test = model_selection.train_test_split(default_df, test_size=0.5, random_state=0)\n",
"logit_model = smf.logit(formula='default ~ income + balance', data= train).fit()\n",
"((logit_model.predict(test) > 0.5) == test['default']).mean()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(c) Repeat the process in (b) three times, using three different splits of the observations into a training set and a validation set. Comment on the results obtained.\n",
"\n",
"This is an ambiguous question, do they want different ratios of `train : test` or different random seeds?\n",
"\n",
"**random seeds**\n",
"\n",
"`random_state=0` : `0.971`\n",
"\n",
"`random_state=1` : `0.975`\n",
"\n",
"`random_state=2` : `0.9752`\n",
"\n",
"The estimates have low variance.\n",
"\n",
"**ratios**\n",
"\n",
"`test_size=0.25` : `0.9744`\n",
"\n",
"`test_size=0.5` : `0.9752`\n",
"\n",
"`test_size=0.75` : `0.9744`\n",
"\n",
"The estimates have low variance.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(d) Now consider a logistic regression model that predicts the probability of default using income, balance, and a dummy variable for student. Estimate the test error for this model using the validation set approach. Comment on whether or not including a dummy variable for student leads to a reduction in the test error rate.\n",
"\n",
"- Test error without including student: `0.971`\n",
"- Test error including student: `0.9708`\n",
"- There is not a significant change in the test error rate."
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072293\n",
" Iterations 10\n",
"0.9708\n"
]
}
],
"source": [
"train, test = model_selection.train_test_split(default_df, test_size=0.5, random_state=0)\n",
"logit_model = smf.logit(formula='default ~ income + balance + student', data= train).fit()\n",
"print(((logit_model.predict(test) > 0.5) == test['default']).mean())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"6. We continue to consider the use of a logistic regression model to predict the probability of default using income and balance on the Default data set. In particular, we will now compute estimates for the standard errors of the income and balance logistic regression co- efficients in two different ways: (1) using the bootstrap, and (2) using the standard formula for computing the standard errors in the glm() function. Do not forget to set a random seed before beginning your analysis."
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"data": {
"text/html": [
"<table class=\"simpletable\">\n",
"<caption>Logit Regression Results</caption>\n",
"<tr>\n",
" <th>Dep. Variable:</th> <td>default</td> <th> No. Observations: </th> <td> 5000</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Model:</th> <td>Logit</td> <th> Df Residuals: </th> <td> 4997</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Method:</th> <td>MLE</td> <th> Df Model: </th> <td> 2</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Date:</th> <td>Mon, 24 Sep 2018</td> <th> Pseudo R-squ.: </th> <td>0.4500</td> \n",
"</tr>\n",
"<tr>\n",
" <th>Time:</th> <td>19:52:15</td> <th> Log-Likelihood: </th> <td> -364.78</td> \n",
"</tr>\n",
"<tr>\n",
" <th>converged:</th> <td>True</td> <th> LL-Null: </th> <td> -663.25</td> \n",
"</tr>\n",
"<tr>\n",
" <th> </th> <td> </td> <th> LLR p-value: </th> <td>2.369e-130</td>\n",
"</tr>\n",
"</table>\n",
"<table class=\"simpletable\">\n",
"<tr>\n",
" <td></td> <th>coef</th> <th>std err</th> <th>z</th> <th>P>|z|</th> <th>[0.025</th> <th>0.975]</th> \n",
"</tr>\n",
"<tr>\n",
" <th>Intercept</th> <td> -11.3896</td> <td> 0.635</td> <td> -17.935</td> <td> 0.000</td> <td> -12.634</td> <td> -10.145</td>\n",
"</tr>\n",
"<tr>\n",
" <th>income</th> <td> 1.58e-05</td> <td> 7.34e-06</td> <td> 2.151</td> <td> 0.031</td> <td> 1.4e-06</td> <td> 3.02e-05</td>\n",
"</tr>\n",
"<tr>\n",
" <th>balance</th> <td> 0.0056</td> <td> 0.000</td> <td> 16.792</td> <td> 0.000</td> <td> 0.005</td> <td> 0.006</td>\n",
"</tr>\n",
"</table><br/><br/>Possibly complete quasi-separation: A fraction 0.14 of observations can be<br/>perfectly predicted. This might indicate that there is complete<br/>quasi-separation. In this case some parameters will not be identified."
],
"text/plain": [
"<class 'statsmodels.iolib.summary.Summary'>\n",
"\"\"\"\n",
" Logit Regression Results \n",
"==============================================================================\n",
"Dep. Variable: default No. Observations: 5000\n",
"Model: Logit Df Residuals: 4997\n",
"Method: MLE Df Model: 2\n",
"Date: Mon, 24 Sep 2018 Pseudo R-squ.: 0.4500\n",
"Time: 19:52:15 Log-Likelihood: -364.78\n",
"converged: True LL-Null: -663.25\n",
" LLR p-value: 2.369e-130\n",
"==============================================================================\n",
" coef std err z P>|z| [0.025 0.975]\n",
"------------------------------------------------------------------------------\n",
"Intercept -11.3896 0.635 -17.935 0.000 -12.634 -10.145\n",
"income 1.58e-05 7.34e-06 2.151 0.031 1.4e-06 3.02e-05\n",
"balance 0.0056 0.000 16.792 0.000 0.005 0.006\n",
"==============================================================================\n",
"\n",
"Possibly complete quasi-separation: A fraction 0.14 of observations can be\n",
"perfectly predicted. This might indicate that there is complete\n",
"quasi-separation. In this case some parameters will not be identified.\n",
"\"\"\""
]
},
"execution_count": 53,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train, test = model_selection.train_test_split(default_df, test_size=0.5, random_state=0)\n",
"logit_model = smf.logit(formula='default ~ income + balance', data= train).fit()\n",
"logit_model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(b) Write a function, boot.fn(), that takes as input the Default data set as well as an index of the observations, and that outputs the coefficient estimates for income and balance in the multiple logistic regression model."
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {},
"outputs": [],
"source": [
"def boot(data, func, r):\n",
" i = 0\n",
" stats = pd.DataFrame()\n",
" while i < r:\n",
" sample = data.sample(data.shape[0], replace=True)\n",
" value = func(sample)\n",
" stats[i] = value\n",
" i+=1\n",
" std = stats.std(axis=1)\n",
" return pd.DataFrame(data={'original': func(data), 'std_err' : std})\n",
"\n",
"def logit_coeff(data):\n",
" logit_model = smf.logit(formula='default ~ income + balance', data= train).fit()\n",
" return logit_model.params"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(c) Use the boot() function together with your boot.fn() function to estimate the standard errors of the logistic regression coefficients for income and balance."
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n",
"Optimization terminated successfully.\n",
" Current function value: 0.072956\n",
" Iterations 10\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>original</th>\n",
" <th>std_err</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>Intercept</th>\n",
" <td>-11.389575</td>\n",
" <td>0.000000e+00</td>\n",
" </tr>\n",
" <tr>\n",
" <th>income</th>\n",
" <td>0.000016</td>\n",
" <td>0.000000e+00</td>\n",
" </tr>\n",
" <tr>\n",
" <th>balance</th>\n",
" <td>0.005565</td>\n",
" <td>8.677957e-19</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" original std_err\n",
"Intercept -11.389575 0.000000e+00\n",
"income 0.000016 0.000000e+00\n",
"balance 0.005565 8.677957e-19"
]
},
"execution_count": 57,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boot(default_df, logit_coeff, 1000)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(d) Comment on the estimated standard errors obtained using the glm() function and using your bootstrap function.\n",
"\n",
"- They are almost identical"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"7. In Sections `5.3.2` and `5.3.3`, we saw that the `cv.glm()` function can be used in order to compute the LOOCV test error estimate. Alternatively, one could compute those quantities using just the `glm()` and `predict.glm()` functions, and a for loop. You will now take this approach in order to compute the LOOCV error for a simple logistic regression model on the `Weekly` data set. Recall that in the context of classification problems, the LOOCV error is given in `(5.4)`."
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Year</th>\n",
" <th>Lag1</th>\n",
" <th>Lag2</th>\n",
" <th>Lag3</th>\n",
" <th>Lag4</th>\n",
" <th>Lag5</th>\n",
" <th>Volume</th>\n",
" <th>Today</th>\n",
" <th>Direction</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1990</td>\n",
" <td>0.816</td>\n",
" <td>1.572</td>\n",
" <td>-3.936</td>\n",
" <td>-0.229</td>\n",
" <td>-3.484</td>\n",
" <td>0.154976</td>\n",
" <td>-0.270</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>1990</td>\n",
" <td>-0.270</td>\n",
" <td>0.816</td>\n",
" <td>1.572</td>\n",
" <td>-3.936</td>\n",
" <td>-0.229</td>\n",
" <td>0.148574</td>\n",
" <td>-2.576</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1990</td>\n",
" <td>-2.576</td>\n",
" <td>-0.270</td>\n",
" <td>0.816</td>\n",
" <td>1.572</td>\n",
" <td>-3.936</td>\n",
" <td>0.159837</td>\n",
" <td>3.514</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1990</td>\n",
" <td>3.514</td>\n",
" <td>-2.576</td>\n",
" <td>-0.270</td>\n",
" <td>0.816</td>\n",
" <td>1.572</td>\n",
" <td>0.161630</td>\n",
" <td>0.712</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>1990</td>\n",
" <td>0.712</td>\n",
" <td>3.514</td>\n",
" <td>-2.576</td>\n",
" <td>-0.270</td>\n",
" <td>0.816</td>\n",
" <td>0.153728</td>\n",
" <td>1.178</td>\n",
" <td>1</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Year Lag1 Lag2 Lag3 Lag4 Lag5 Volume Today Direction\n",
"0 1990 0.816 1.572 -3.936 -0.229 -3.484 0.154976 -0.270 0\n",
"1 1990 -0.270 0.816 1.572 -3.936 -0.229 0.148574 -2.576 0\n",
"2 1990 -2.576 -0.270 0.816 1.572 -3.936 0.159837 3.514 1\n",
"3 1990 3.514 -2.576 -0.270 0.816 1.572 0.161630 0.712 1\n",
"4 1990 0.712 3.514 -2.576 -0.270 0.816 0.153728 1.178 1"
]
},
"execution_count": 69,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weekly_df = pd.read_csv('weekly.csv')\n",
"weekly_df = weekly_df.drop(weekly_df.columns[0], axis=1)\n",
"weekly_df['Direction'] = weekly_df['Direction'].map({'Up' : 1, 'Down' : 0})\n",
"weekly_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(a) Fit a logistic regression model that predicts Direction using Lag1 and Lag2."
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.682882\n",
" Iterations 4\n"
]
}
],
"source": [
"logit_model = smf.logit(formula='Direction ~ Year + Lag1 + Lag2', data=weekly_df[1:]).fit()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(c) Use the model from (b) to predict the direction of the first observation. You can do this by predicting that the first observation will go up if P(Direction=\"Up\"|Lag1, Lag2) > 0.5. Was this observation correctly classified?\n",
"\n",
"- No"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0 False\n",
"dtype: bool"
]
},
"execution_count": 85,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(logit_model.predict(weekly_df.iloc[0]) > 0.5) == weekly_df['Direction'].iloc[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(d) Write a for loop from `i=1` to `i=n`, where `n` is the number of observations in the data set, that performs each of the following steps:\n",
"\n",
"i. Fit a logistic regression model using all but the `ith` observation to predict `Direction` using `Lag1` and `Lag2`.\n",
"\n",
"ii. Compute the posterior probability of the market moving up for the `ith` observation.\n",
"\n",
"iii. Use the posterior probability for the `ith` observation in order to predict whether or not the market moves up.\n",
"\n",
"iv. Determine whether or not an error was made in predicting the direction for the `ith` observation. If an error was made, then indicate this as a 1, and otherwise indicate it as a 0."
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683147\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683149\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683416\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683253\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683454\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683181\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683432\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683335\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683402\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683203\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683084\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683344\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683288\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683388\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683064\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683402\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683227\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683458\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683239\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683442\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683318\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683399\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683224\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683067\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683182\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683200\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683199\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683216\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683403\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683179\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683070\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683130\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683207\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683367\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683263\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683478\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683357\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683511\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683277\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683430\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683182\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683419\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683330\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683143\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683429\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683218\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683113\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683161\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683353\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683239\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683488\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683379\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683357\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683080\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683460\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683338\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683163\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683135\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683400\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683306\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683433\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683363\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683158\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683428\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683229\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683138\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683364\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683338\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683201\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683495\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683272\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683132\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683381\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683326\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683376\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683149\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683422\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683237\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683128\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683389\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683337\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683116\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683137\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683366\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683244\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683134\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683389\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683352\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683234\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683480\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683297\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683129\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683113\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683225\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683344\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683361\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683386\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683407\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683316\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683081\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683471\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683228\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683135\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683389\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683333\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683169\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683396\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683194\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683421\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683326\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683196\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683098\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683343\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683360\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683215\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683475\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683326\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683170\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683426\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683331\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683156\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683161\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683176\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683178\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683347\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683348\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683422\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683163\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683404\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683266\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683472\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683222\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683148\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683353\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683375\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683387\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683177\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683113\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683222\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683390\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683296\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683428\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683184\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683414\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683358\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683396\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683395\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683397\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683388\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683202\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683118\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683172\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683385\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683261\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683432\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683368\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683206\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683085\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683396\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683286\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683424\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683385\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683155\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683167\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683393\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683346\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683208\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683460\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683302\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683167\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683405\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683344\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683155\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683149\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683168\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683381\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683221\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683411\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683202\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683404\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683365\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683387\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683387\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683381\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683370\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683401\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683404\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683176\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683165\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683372\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683203\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683406\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683225\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683450\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683238\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683434\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683256\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683423\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683367\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683185\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683396\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683370\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683168\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683392\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683367\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683178\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683410\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683198\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683432\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683237\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683167\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683189\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683379\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683370\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683188\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683102\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683378\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683279\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683394\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683373\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683184\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683143\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683379\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683322\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683433\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683176\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683154\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683189\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683434\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683280\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683390\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683177\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683412\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683209\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683415\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683360\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683403\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683211\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683098\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683378\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683210\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683436\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683264\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683424\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683285\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683468\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683243\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683464\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683252\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683176\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683409\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683243\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683410\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683302\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683100\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683391\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683376\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683193\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683415\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683357\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683385\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683416\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683396\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683202\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683422\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683356\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683385\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683164\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683411\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683364\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683155\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683399\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683360\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683397\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683168\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683427\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683339\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683187\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683437\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683324\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683146\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683440\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683325\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683125\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683416\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683245\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683112\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683377\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683354\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683399\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683377\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683375\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683168\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683432\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683197\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683400\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683370\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683179\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683416\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683318\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683423\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683193\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683414\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683364\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683178\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683121\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683390\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683355\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683162\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683423\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683292\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683399\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683386\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683172\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683476\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683250\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683078\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683354\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683319\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683169\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683430\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683234\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683464\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683284\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683165\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683441\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683309\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683382\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683127\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683438\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683233\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683416\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683353\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683187\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683122\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683205\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683209\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683362\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683271\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683479\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683374\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683171\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683424\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683316\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683335\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683076\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683408\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683226\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683438\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683211\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683439\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683341\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683330\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683458\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683382\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683155\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683097\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683349\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683298\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683113\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683426\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683328\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683175\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683446\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683331\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683425\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683213\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683086\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683384\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683254\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683092\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683202\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683189\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683178\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683374\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683326\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683474\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683272\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683501\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683409\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683365\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683431\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683369\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683344\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683058\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683418\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683279\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683071\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683385\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683337\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683116\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683122\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683402\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683325\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683479\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683315\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683467\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683251\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683459\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683335\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683117\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683125\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683245\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683421\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683284\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683416\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683248\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683480\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683259\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683015\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683257\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683383\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683288\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683539\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683378\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683473\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683334\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683389\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683449\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683380\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683394\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683409\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683379\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683198\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683455\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683237\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683457\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683233\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683429\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683242\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683429\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683354\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683181\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683414\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683271\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683453\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683347\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683339\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683435\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683387\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683169\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683034\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683259\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683178\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683352\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683284\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683014\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683337\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683171\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683451\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683193\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683013\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683308\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683200\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683523\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683375\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683174\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683490\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683291\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683122\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683073\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683365\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683331\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683377\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683455\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683243\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683002\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683343\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683303\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682960\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683314\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683217\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683403\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683332\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683431\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683140\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683412\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683337\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683234\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683508\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683296\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683473\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683242\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683146\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683185\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683403\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683280\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683467\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683322\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683498\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683224\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683499\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683175\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683051\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683267\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683365\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683292\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683423\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683386\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683166\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683151\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683169\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683420\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683259\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683245\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683560\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683107\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683413\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683482\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683366\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683158\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683433\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683214\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683429\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683350\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683348\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683111\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683430\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683266\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683094\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683429\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683412\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682963\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683200\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683322\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683310\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683522\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683271\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683105\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683506\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683258\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683550\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682929\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683013\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683112\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683205\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683185\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683391\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683364\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683555\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683231\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683420\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683328\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683379\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683169\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683071\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683398\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683218\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683445\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683377\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683401\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683168\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683114\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683194\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683215\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683199\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683167\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683375\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683283\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683441\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683280\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683519\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683304\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683137\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683367\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683313\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682945\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683279\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683215\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683435\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683315\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683393\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683141\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683143\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683140\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683237\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683123\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683286\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683207\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683036\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683212\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683242\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683466\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683221\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683420\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683485\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683176\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683434\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683296\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683034\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683378\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683227\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683451\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683311\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683150\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683424\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683299\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683104\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683380\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683206\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683117\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683371\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683286\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683021\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683225\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683443\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682855\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683512\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683125\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683453\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683282\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683473\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683290\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683430\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683149\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683423\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683240\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683467\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683269\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683408\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683156\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683101\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683350\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683237\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683144\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683395\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683262\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683421\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683283\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683430\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683091\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683145\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683223\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683140\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683346\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683236\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683046\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683299\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683404\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683318\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682970\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683212\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683152\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683251\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683362\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683235\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683176\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683034\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683331\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683141\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683377\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683331\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683468\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683136\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683083\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683211\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683243\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683089\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683283\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683381\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683217\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683394\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683493\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683140\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683403\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683338\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683388\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683120\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683107\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683358\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683266\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683435\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683257\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683100\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683053\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683174\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683296\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683414\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683293\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683192\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683112\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683386\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683338\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683300\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683585\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683315\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683443\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683303\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683443\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683341\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683459\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683192\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683429\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683298\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683422\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683441\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683191\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683430\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683317\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683175\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683421\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683205\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683125\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683340\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683355\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683412\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683365\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683159\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683419\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683221\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683479\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683220\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683449\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683158\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683402\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683322\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683117\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683167\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683399\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683309\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683431\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683372\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683391\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683406\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683382\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683392\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683388\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683132\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683401\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683343\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683153\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683392\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683379\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683198\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683081\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683254\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683362\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683242\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683077\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683385\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683203\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683102\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683251\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683198\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683380\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683338\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683439\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683199\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683138\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683183\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683166\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683182\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683194\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683380\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683248\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683467\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683292\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683333\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683444\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683396\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683382\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683159\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683418\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683264\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683101\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683180\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683370\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683301\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683409\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683109\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683438\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683336\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683162\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683405\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683368\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683375\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683141\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683132\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683240\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683402\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683343\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683347\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683097\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683393\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683363\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683172\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683115\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683217\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683386\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683342\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683190\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683449\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683283\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683397\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683192\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683436\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683293\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683098\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683406\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683375\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683205\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683451\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683326\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683366\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683398\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683408\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683166\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683394\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683205\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683152\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683381\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683335\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683185\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683113\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683405\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683255\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683088\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683244\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683373\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683342\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683394\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683408\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683395\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683177\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683124\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683384\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683362\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683162\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683411\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683291\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683087\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683419\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683272\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683448\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683341\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683363\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683133\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683171\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683386\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683232\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683109\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683385\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683200\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683392\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683228\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683425\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683204\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683089\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683347\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683318\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683160\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683102\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683261\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683391\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683236\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683106\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683410\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683318\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683341\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683081\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683400\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683262\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683457\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683220\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683427\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683238\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683427\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683348\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683406\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683390\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683409\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683186\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683413\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683339\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683171\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683133\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683388\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683361\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683175\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683430\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683225\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683406\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683227\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683133\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683392\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683233\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683438\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683225\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683136\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683442\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683325\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683428\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683279\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683480\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683330\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683412\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683364\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683425\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683387\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683402\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683201\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683417\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683220\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683444\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683266\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683452\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683332\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683355\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683152\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683110\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683115\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683282\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683259\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683426\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683240\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683098\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683396\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683312\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683390\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683448\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683352\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683121\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683448\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683333\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683070\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683400\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683293\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683411\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683304\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683123\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683457\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683272\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683136\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683098\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683296\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683446\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683227\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683259\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683550\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683233\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683140\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683139\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683174\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683317\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683253\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683473\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683289\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683515\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683233\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683474\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683186\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683438\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683283\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683494\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683312\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683063\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683262\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683117\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683213\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683245\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683383\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683267\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683428\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683374\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683342\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683091\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683166\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683180\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683414\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683287\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683162\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683102\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683081\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683412\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683642\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683570\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683514\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682693\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683171\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683355\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683562\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683642\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683318\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683185\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683431\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683348\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682838\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683222\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683270\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683341\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683296\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682891\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683184\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683289\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683378\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.682939\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683581\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683320\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683475\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683435\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683158\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683426\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683352\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683251\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683573\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683239\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683335\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683434\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683121\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683105\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683253\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683138\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683352\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683204\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683479\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683466\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683199\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683448\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683331\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683116\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683409\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683308\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683147\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683053\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683356\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683255\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683063\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683117\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683426\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683216\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683123\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683440\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683379\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683362\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683139\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683390\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683232\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683450\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683260\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683077\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683124\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683262\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683353\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683351\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683213\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683462\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683321\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683441\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683394\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683395\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683382\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683177\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683419\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683226\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683058\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683419\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683392\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683494\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683304\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683422\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683281\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683148\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683023\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683370\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683409\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683519\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683281\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683467\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683218\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683040\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683276\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683377\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683303\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683463\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683372\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683175\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683435\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683351\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683408\n",
" Iterations 4\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Optimization terminated successfully.\n",
" Current function value: 0.683397\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683397\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683247\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683496\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683245\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683398\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683311\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683434\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683410\n",
" Iterations 4\n",
"Optimization terminated successfully.\n",
" Current function value: 0.683374\n",
" Iterations 4\n"
]
}
],
"source": [
"errs = []\n",
"weekly_df\n",
"for train_idx, test_idx in model_selection.KFold(n_splits=weekly_df.shape[0]).split(weekly_df):\n",
" train_dat = weekly_df.iloc[train_idx]\n",
" test_dat = weekly_df.iloc[test_idx]\n",
" logit_model = smf.logit(formula='Direction ~ Lag1 + Lag2', data=train_dat).fit()\n",
" errs.append(((logit_model.predict(test_dat) > 0.5) == test_dat['Direction']).mean())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(e) Take the average of the `n` numbers obtained in (d)iv in order to obtain the LOOCV estimate for the test error. Comment on the results.\n",
"\n",
"- Prediction accuracy is 55%"
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.5500459136822773"
]
},
"execution_count": 89,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pd.DataFrame(errs).mean()[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"8. We will now perform cross-validation on a simulated data set.\n",
"\n",
"(a) Generate a simulated data set as follows:\n",
"\n",
"```\n",
"set.seed(1)\n",
"x=rnorm(100)\n",
"y=x-2*x^2+rnorm(100)\n",
"```\n",
"\n",
"In this data set, what is `n` and what is `p`? Write out the model used to generate the data in equation form.\n",
"\n",
"- `n = 100`\n",
"- `p = 1`\n",
"- `x = N ~ (0, 1)`\n",
"- `y = -2x^2 + epsilon`"
]
},
{
"cell_type": "code",
"execution_count": 119,
"metadata": {},
"outputs": [],
"source": [
"n = 100\n",
"x = np.random.normal(size=n)\n",
"y = x - 2 * x ** 2 + np.random.normal(size=n)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(b) Create a scatterplot of `X` against `Y` . Comment on what you find.\n",
"\n",
"- As we might expect from the formula in (a), the data follows the form of an inverted parabola with some noise."
]
},
{
"cell_type": "code",
"execution_count": 120,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.axes._subplots.AxesSubplot at 0x10f74d978>"
]
},
"execution_count": 120,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X90VOWdP/D3Z2YyYQiRxMgvCSlWWbbRYmmmWMr322r9UdeDS/lha11AaytQlj1t122x66F6lu45IljO6bEU8NcWa1ctLpWDVkS/+nWXWmsiSCGYCv4oAYSQBghhzDCZz/ePzMx3bnLv/MjMnXtn5v06h+Nk5ubmkzFzP/d5ns/zPKKqICIiivM4HQAREbkLEwMRERkwMRARkQETAxERGTAxEBGRARMDEREZMDEQEZEBEwMRERkwMRARkYHP6QCG4oILLtCJEyc6HQYRUVFpaWk5oaqj0h1XlIlh4sSJaG5udjoMIqKiIiIfZnIcu5KIiMjA1sQgIhNE5BURaRWRfSLyXZNjrhSRUyKyO/bvx3bGREREqdndlRQBcKeqviUi1QBaRGSHqrYOOO6/VXWmzbEQEVEGbG0xqOpRVX0r9rgbwH4A4+38mURElJuCjTGIyEQAUwG8YfLydBF5W0R+JyKXFiomIiIarCBVSSIyAsAzAL6nqqcHvPwWgE+o6hkRuQHAbwFMMjnHIgCLAKChocHmiImo2ESjis6eMMKRPvh9XtRV+eHxiNNhFSWxewc3EakAsA3AdlX9aQbHfwAgqKonrI4JBoPKclUiiotGFW3HunHHpma0d4VQXxvAQwuDmDymmskhiYi0qGow3XF2VyUJgEcA7LdKCiIyNnYcRGRaLKZOO+MiotLS2RNOJAUAaO8K4Y5NzejsCTscWXGyuytpBoAFAP4kIrtjz/0rgAYAUNX1AOYB+I6IRACEANys3IiaiLIQjvQlkkJce1cI4UifQxEVN1sTg6r+D4CU7ThVfRDAg3bGQeQE9nkXjt/nRX1twJAc6msD8Pu8DkZVvDjzmcgG8T7v2et2YsaqVzB73U60HetGNMrGsB3qqvx4aGEQ9bUBAEiMMdRV+R2OrDjZPvhsBw4+k9t1dPdi9rqdg+5gtyydgVHVlQ5GVrrYQksv08HnolxEj8jt2OddeB6PMOnmCbuSiGwQ7/NOxj5vKhZMDEQ2YJ83FTN2JRHZwOMRTB5TjS1LZxRNnzf76CmOiYHIJsXU582Zw5SMXUlElPPM4WhU0dHdi8NdZ9HR3cuy3CLHFgMR5VRF5YbWBrvB8ostBnK9YrwbLbaYM6misvqdnF6niJMJ84+JgVytGD/0xRhzuiqqVL/TwNbG1Ak1WDGzEWfDkYIkRacTUyliYiBXK8YPfTHGnFxFtXP5VdiydIahKyjV75Tc2pg6oQb/8pXJWLmtFV9a/WpBkiInE+YfEwO5WjF+6Icas9PdT/EqqvG1wzGqutLQR5/qd0pubSy58mIsf2ZPQZMiJxPmHxMDuVqxfeijUYWIYPOS6diwoAlTJ9QASB+z27ufrP4/9MXii7c2/nZsdcETOScT5h8TA7maWz/0Znf38Yv71za8jnnrX8fKba34l69MxnWNo9PG7Obup2hU4fUA6+c3Gf4/rJo7BT95rhWdPeFEa2O431fwRJ6uG4yyx3JVcjU3ziC2Ks+sG+EfdHFf/swePL14OsaeNyxlzG7tMotEomg73o3Fj7dg1IhKrJx1GRrqhuPoyRDWbG/DrkMncc+NfYly0Wg0ig0LmrD48Rbje2NzIi+myYTFgImBXM9tH3qru/tf33GF6cVdVdMmMjduNBONKo6cCiUu8u1dIXzzP95EfW0AK2Y2Ytehk6ivDaDC5zEkyusaR+PX374CXo/kLZFznkJhsSuJKEtWd/dekSF3o7ixy6yzJ4zj3b2mv2tNoCIRo88jhkT5Yutx3PLwG/D7vImEHu92O3IyhGOnQlkNrrt9/KUU2Z4YROR6EWkTkQMicpfJ65Ui8lTs9TdEZKLdMRHlwmogNuD3DvniHo0qRgYq8MS3r8DO5Vdh2z85308ejvShsyds+rvGNx2aPKYaobB1N9jAi/rXNryOAx09uHvLnowv7m4efylVtiYGEfEC+DmAvwPQCOAbItI44LBvAehS1UsArAWwys6YiHJldXdfE/APaRA0Eonindig9ZdWv4qvb/wDDnV9jL6+aCF+HUt+nxfPtBzCqrlTDL/rhgVNGDcykChpTVU5ZnZRX/7MHsxtmpDxxd2t4y+lzO4xhmkADqjqewAgIk8CmAWgNemYWQDujT3eDOBBEREtxj1HqSykGxDPdjzk+JleLPlVi+Hi+Z1fteDJRZ/HhSMDhsSSbV97Ln3zdVV+fP/ayVi7ow0rZjairsqP0dWVuHBkfxLo6O5FONKHCp8Hm26fhoWP/jEx4LxhQROi0Sh6+6KWXVGZXtzdOP5S6uxODOMBHEr6uh3AFVbHqGpERE4BqANwIvkgEVkEYBEANDQ02BUvUUbiA+LxC+/RU6EhD4qes7h4hiNRdPaEUVflT1zc+6KKnzzXihdbj6ddrC7d4nbpkkY8Af777CmGYwCYnnfrshkIhY0xPnbb50wv6idD57IefxlUBcZ5CrYpmqokVd0IYCMABINBtibIcZmsKprJHXuF12N68eyLrUM08GesmjsFHd1h7Dp0EndsasaWpTNMWylWffNbls5AXZU/oxVRPR4xJKbOnjC8Hlie1+/zYva6nYnXfvbyu1g9bwp+sHmPIf5f/v79jC/ubixZLnV2J4bDACYkfV0fe87smHYR8QEYCaDT5riIchKNKj46/TF6eiNYMbMR6189OOhCnSpxAEhcbCu8Hjz2zc/hm4+9mThu3T98Fpub/4JbZ3zStI/+vjmfRk+4DzWBisQg78BkdDYcseybT5U0kpNMJBLFkVMhHO/uRWdPGM+0HMJ3r/4bfOGTdbi6cQxqAhU4GTqH9a8eTHQLJf/MXYdO4v4X2vDUos8DAEQEIsA9N16KqkovTvT04lwkmvZi77aS5VJnd2J4E8AkEbkI/QngZgC3DDhmK4BbAbwOYB6A/8PxBXIzswv+qrlTEhO+4hdIq4vv1mUzcOx0rzFhLAhi85LpOBvritnc/Bd89bMT4BWYXtzH1QRwa1Kf/kMLg5g0agT+GgrjbG8f3j/Rg2EV5i0Rv8+b0YBuNKqJyW3Jv+fW3e2YP/0TWPrEW4nnV8+bgoDfi74oBv3MjjO9iQt//H0bNaISP7x+sqElwR3j3MPWqiRVjQBYBmA7gP0AnlbVfSLybyLy97HDHgFQJyIHAPwzgEElrURDEV+24tipEI6cDOVtcTqrSpslV15s6De3uviGwn2DE8bjzYhGFcMqvBju9+LbX7wkdpH0mFb8/KXzrOH71+5oQ9vxbsxZ93tcueZVrHh2LzwiePCWqabls5msQXWipzeRFJJ/z3nBhkRSiD//g817EIlqyvkYye/bkisvTiSFxHvAElTXsH2MQVWfB/D8gOd+nPT4YwA32R0HlZf4Xf3aHW249QsXJVb8zPTONNXYgNUFP35RjPebB/xePHbb5zDc7010t3Sc6UUkqqbff6grhDt/83YsvkCif3/gwOv6+U1Y8du9hu+f2zRh0EX8zt+8jfvmfBorZjbiU2OrEfD7Er9HugHdaFRxttdiIp9HTJ8/F4mmHA9Ift/iVUkDz8ESVHcomsFnomzE705XzGw0XQbaasAWSD+obFU+eWFNILEmUjSqOHa6Fyue3Wvobhnu96Kju9eyUmdgfGYXWq+nv3tm6oQaLLnyYtQEKlA3otL0Qlvh9WDltj8Zxj3iZaZ1I/yJSqKBya+zJ4z3T/RYdEVZd1EB1uMBye9bvCqJJajuxCUxqCTF706HcmdqNTbw0emPEU3RXZK8UJ7ZOX6weQ/O9Sl8XsHqecZJYw/cdDnWv3rQNL6B+yTUBPzYdPs0/PD6/g1xvr7xDzj017OmXUNnw32JlsDAWchz1v0ex073GiarJb9/P3v53cGT2+Y3YcyIyiHN8E5+39a/enDQe8ASVPdgi4FKUvzuNJs70+RqnuRKI6D/Yn3kZAinQucweUx12vJJq+6mmuEVuO2xNzFqRCVWzGxETaACZ8N9iKomfla6O2ePRzBimA8LH/3/LSGzstAN85swrmYYagL9sXV092ZUiRR//zrO9GLN9jZDnONqhqGiwmv6+wPGSW8+jwxqjSR/X8DvxX8t/UJGVUlUWEwMVJLid6drd7Rh1dwpg8YYBt6Zpqs0qq8NoLMnjO89tTtxIU1VPmnV3eQVSaxUuvjxlsRrm5dMTxyTyZ3zuYhxUtzAstBskpVZ6yl5DGLx4y2GZT+Awd1FZu/fL/7hszjTG8FjO9/H96+dnOiKY9mp+zExUElKnrUbjUbx9OLpUFXLO1OrSqMVMxuxcltrIklkOkBqNWh84kzYcnxi5/KrMr5zNks88bJQqwtvNktLZDup7ETP4NbId554CytnXYZbv3AR1u5ow7/PnpJVUuBS285hYqCSlc3dqdXd9KTRI7BiZqOh5ZDJAKnZhbU2UIHTveewYX4TFv/KuJFNuo18BhrKMhHZfk+m71+qCqbhfi/u/M3bWDGzMauKo0xmlZN9mBio5GVy52l1N+3zCFZuax3SGj1mF9bzff2Dx6n65zO5Ox7KMhF2LS2RqoIpXmkVnzuRzTkzHQ+h/GNioJKW6Z2n1d30hSMDeb+QZtI/n8nd8VD66+3o40+uYEoey4l3v9XXBjC6ujKriiMute0sJgYqaVZ3nk8t+rzhQp/qbtruO9RivztOrmC6b86nMa6mf2b2mu1t6DjTiw0LmgYtH57JOTnPwTmcx0AlzerOs70rNGiLyIHzBQrVl13sd8fx1lbHmV7Mf+SPWPW7/bh4VBUevGUqtiydgU+NPQ8+X3aXGjdudVpO2GKgkmZ152k2y9htMRbL3XG+xi4GjgVNGjWCS207hC0GKqj4kgz5WtAuHbM7z1Vzp1jOMnZCbaAC6+c3GWJcP78JtYEKR+PKRq6trYGzsmev24l3O86grspvewuu0H+TxYAtBioYJ0oQk+9mQ+f6cPD4mUTpKeCOO/Ou0Dn87OU/J2YYn4x9nW3dfzFzapzF7G9y0+3TMKLSh48jffCKIOD3JmaPlwsmBsqbdGWhhfzwm8UCAD29EXSc6QXgnn7rcKQPL7Yex4utxw3P33NjcYwx5INT4ywD/yZHjajEsdMfY2HS0iKr503BmPOGYWJdVdkkByYGyotMWgOF+vCnisWNW0QW+xhDPjj1Hgz8mzTbJ+IHm/dg5azLUD2somxacBxjoLywag0kb7ySyeYw2bDqG04Vi1OVR6mwAse592Dg36TVarzD/V7Hx6IKiS0GyotMWgNDWcbBSqpWQbGVf3Kze+feg7qq/iXMP+w8i+F+L+pGVJq2XM7GVoktF7YlBhFZDeBGAGEABwF8U1VPmhz3AYBuAH0AIqoatCsmsk8mXQH5/PCnGq8oxq4Zrjrq3HvQG4kmNlS6rnE01s9vwpKktaziYwzl1IKzs8WwA8CPVDUiIqsA/AjAcotjr1LVEzbGQjbLtDWQrw9/qlbBuJGBvLVMqLQNvMGIFwA8vXg6IlGFV8CqpHxS1ReTvvwDgHl2/SxyXqG7AlK1Ctg1Q2bMKtXMbjBebD2Oe25UNJw/3KFInVeowefbAfzO4jUF8KKItIjIogLFQzYo5MBuusHKeCzjRva/fvRUiJOXypjZBLq2Y92oiO1fnczt3Y6FIKpD/6CIyEsAxpq8dLeqPhs75m4AQQBz1OSHich4VT0sIqPR3/30T6r6mslxiwAsAoCGhoamDz/8cMhxU2lIN2+Ca/pTXEd3L2av2zmohbl12QwcO91bNn8jItKSyThuTokhgyBuA7AYwNWqejaD4+8FcEZV16Q6LhgManNzc15ipNJldTFwem0ksmbXrm2Hu85ixqpXBj2/c/lVGDcyUDY7xWWaGOysSroewA8BfMkqKYhIFQCPqnbHHl8H4N/sionKQ/zicjYcKaqy1XJnZwsv3mU08CahwudhRZgJO8cYHgRQDWCHiOwWkfUAICIXisjzsWPGAPgfEXkbwB8BPKeqL9gYE5W45L7kdz7qZv9xEclkkuRQ+TyC1fOmGMakVs+bAl+JtgxyZWdV0iUWzx8BcEPs8XsALrcrBio/yReX9a8eHLSrGMtW3cvOiYmhcB/uf6HNsFDh/S+04cFbpgJVOZ++5HDmMznGjv7k5IvLrkMnsWZ7/8XgU2OrEfD7Srr/uNjZOTExvsvc4sdb8nJuu8ZC3IJrJZEjrMoHzcpJs1kvf+DaN7sOncTKba0I+H2uWRuJzNm5XlI+z53N326xsrUqyS6sSip+mVYMZTsgyRLV4mbnnXi+zp3PardCtzwcr0oiSiXT/uRs93DgrOfiZmeFUCGWY8mGm29i2JVEBReNKjwiphVDFQM2jR/Kh9CNS2tT6cjX8vF2VmHliomBCip+l9TeFcqofDDfezgQ5SrdeEWmY2JuXh6eXUlUUPG7pAduujyj8sF87uFAlC+VPg9WzroMw/1enA33oTLW0s2me8jNy8Nz8JkKInk28jsfdeO8YT7DFoqA9QBeqZcGUnFJNfgMIOOBaSfGGDj4TK5h9gF44KbL8eAtU7Hs17vStgS4ZAG5SbouoEy7h9xcKMHEQLYzG2S78zdv4745n8bKWZfhoguqMLzSiwuq+i/+Hd29rvugEMVZdQEF/F58fC6aVfeQW296OPhMtrO6w2o4fzguGz8SDecPx+jqYQBQ8hOHqPiZDT5vun0ajp3uxb1b92LV3CmWA9PFgi0Gsp31HZbPcLeU7ZwFIieYdQEpFAvX/R7tXSF0dIexYmYj6qr8uLAmgLHnDSu6Vi9bDGS7TJcjcHP5HlGygXNlzkWihjW6Fj/egnnrX4eqFl1SANhioALIdJDNzeV7RKmU2t8uWwxUEJnMRrZzETUiO5Xa3y7nMZCrcM4CFati+NvlPAbKmhv+sN1avkeUTin97TIxEAD7Z2G6IekQUWZsG2MQkXtF5HBsv+fdInKDxXHXi0ibiBwQkbvsiodSs3Olx3LY2IQoW9lsQJXNsflgd4thraqusXpRRLwAfg7gWgDtAN4Uka2q2mpzXDSAnaWinJ9AZJRNC92JNZWcrkqaBuCAqr6nqmEATwKY5XBMZcnO5a05P4HIKJsWuhP7NtidGJaJyB4ReVREak1eHw/gUNLX7bHnBhGRRSLSLCLNHR0ddsRa1uwst+OeCkRG2dwsOXFjlVNiEJGXRGSvyb9ZAH4B4GIAnwFwFMADufwsVd2oqkFVDY4aNSqXU5GJ5EloO5dfhS1LZ+StqVpqNd5EucrmZsmJG6ucxhhU9ZpMjhORhwBsM3npMIAJSV/Xx54jB9hVbufm5YWJnJDNBlRObFZl2wQ3ERmnqkdjj78P4ApVvXnAMT4AfwZwNfoTwpsAblHVfanOzQluRFTssinhzle5txsmuN0vIp8BoAA+ALA4FtiFAB5W1RtUNSIiywBsB+AF8Gi6pEDZ4xwCIvfJpoVe6MlztiUGVV1g8fwRADckff08gOftiqPcWZW6TRo1Al2hc0wWRDSI0+WqZDOzUre1O9rQdpwTzojIHBNDiTMrdZvbNAGLH28paF00ERUPJoYSZ1bqVlfl54QzIrLExFDizOYQjK6u5IQzIrLE1VVLnNkcgtpARcHroomoeDAxlAGzUjdOOCMiK0wMZaqUNhUhovziGAMRERkwMRARkQETAxERGTAxEBGRARMDEREZMDEQEZEBEwMRERlwHkMZ4v4MRJQKE0MJM0sAAEz3Z8jX/s5EVPzYlVSi4hv0DNxz4URP76D9GbjkNhElsy0xiMhTIrI79u8DEdltcdwHIvKn2HHcyDlPzDbouWNTMz4+N3h/Bi65TUTJ7Nza8+vxxyLyAIBTKQ6/SlVP2BVLKcl0fMBsg572rhC8IqivDRhe45LbRJTM9q4kEREAXwPwn3b/rFJn1T1ktiWn2QY99bUBBPzeQfszcMltIkomqvbu8ysiXwTwU1UNWrz+PoAuAApgg6puTHfOYDCozc3l1+vU0d2L2et2Drrb37J0xqCVUuNJxGyQGQCrkojKkIi0WF2Lk+XUlSQiLwEYa/LS3ar6bOzxN5C6tfC/VPWwiIwGsENE3lHV10x+1iIAiwCgoaEhl7CLllX3kNn4gNkGPckJgEtuE5GVnBKDql6T6nUR8QGYA6ApxTkOx/57XES2AJgGYFBiiLUkNgL9LYYcwi5a8e6hTMcHuOcCEQ2F3WMM1wB4R1XbzV4UkSoRqY4/BnAdgL02x1S0zPZv5vgAEeWb3RPcbsaAbiQRuRDAw6p6A4AxALb0j0/DB+DXqvqCzTEVrXTdQ0RE+WBrYlDV20yeOwLghtjj9wBcbmcMpYbdQ0RkN858JiIiAyYGIiIyYGIgIiIDJgYiIjLgsttFgnsoEFGhMDEUgVTLWzA5EFG+sSupCFgtoc09FIjIDkwMRSCbNZKIiHLFxFAErJbQ7ouq6ZLbRES5YGIoAnVVfmxY0GRYI2nV3Cn4yXOt7E4iorzj4HOe2VE95PEILqjyY8XMRtQEKnAydA5rtrdh16GTuOdGdicRUX4xMeSRndVDHo8HK7e1cktOIrIdu5LyyM7qIS65TUSFwhZDHtlZPcQlt4moUJgY8ijbHdayxSW3iagQ2JWUR+zuIaJSwBZDHrG7h4hKARNDnrG7h4iKXc5dSSJyk4jsE5GoiAQHvPYjETkgIm0i8hWL779IRN6IHfeUiLDfhYjIQfkYY9gLYA6A15KfFJFGADcDuBTA9QDWiYjZKOwqAGtV9RIAXQC+lYeYiIhoiHJODKq6X1XbTF6aBeBJVe1V1fcBHAAwLfkAEREAXwawOfbULwF8NdeYiIho6OysShoP4FDS1+2x55LVATipqpEUxwAARGSRiDSLSHNHR0fegyUion4ZDT6LyEsAxpq8dLeqPpvfkMyp6kYAGwEgGAxySVEiIptklBhU9ZohnPswgAlJX9fHnkvWCaBGRHyxVoPZMUREVEB2diVtBXCziFSKyEUAJgH4Y/IBqqoAXgEwL/bUrQAK0gIhIiJz+ShXnS0i7QCmA3hORLYDgKruA/A0gFYALwD4R1Xti33P8yJyYewUywH8s4gcQP+YwyO5xkREREMn/TftxSUYDGpzc7PTYRARFRURaVHVYLrjuFYSEREZMDEQEZEBEwMRERkwMRARkQETAxERGTAxEBGRARMDEREZMDEQEZEBEwMRERkwMRARkQETAxERGTAxEBGRARMDEREZMDEQEZFBRju4latoVNHZE0Y40ge/z4u6Kj88HnE6LCIiWzExWIhGFW3HunHHpma0d4VQXxvAQwuDmDymmsmBiEoau5IsdPaEE0kBANq7QrhjUzM6e8IOR0ZEZK+cEoOI3CQi+0QkKiLBpOevFZEWEflT7L9ftvj+e0XksIjsjv27IZd48ikc6Uskhbj2rhDCkT6HIiIiKoxcu5L2ApgDYMOA508AuFFVj4jIZQC2AxhvcY61qromxzjyzu/zor42YEgO9bUB+H1eB6MiIrJfTi0GVd2vqm0mz+9S1SOxL/cBCIhIZS4/q9Dqqvx4aGEQ9bUBAEiMMdRV+R2OjIjIXoUYfJ4L4C1V7bV4fZmILATQDOBOVe0qQExpeTyCyWOqsWXpDFYlEVFZSdtiEJGXRGSvyb9ZGXzvpQBWAVhsccgvAFwM4DMAjgJ4IMW5FolIs4g0d3R0pPvReeHxCEZVV2J87XCMqq5kUiCispC2xaCq1wzlxCJSD2ALgIWqetDi3MeSjn8IwLYUcWwEsBEAgsGgDiUmIiJKz5ZyVRGpAfAcgLtUdWeK48YlfTkb/YPZRETkoFzLVWeLSDuA6QCeE5HtsZeWAbgEwI+TSlFHx77n4aTS1vtjJa17AFwF4Pu5xENERLkT1eLrlQkGg9rc3Ox0GERERUVEWlQ1mO44znwmIiIDJgYiIjJgYiAiIgMmBiIiMmBiICIiAyYGIiIyYGIgIiIDJgYiIjJgYiAiIgMmBiIiMmBiICIiAyYGIiIyYGIgIiIDJgYiIjJgYiAiIgMmBiIiMmBiICIig1y39rxJRPaJSDRpu06IyEQRCSVt67ne4vvPF5EdIvJu7L+1ucRDRES5y7XFsBfAHACvmbx2UFU/E/u3xOL77wLwsqpOAvBy7GsiInJQTolBVferalsOp5gF4Jexx78E8NVc4iEiotzZOcZwkYjsEpH/KyL/2+KYMap6NPb4IwBjbIyHiIgy4Et3gIi8BGCsyUt3q+qzFt92FECDqnaKSBOA34rIpap62urnqKqKiKaIYxGARQDQ0NCQLmwiIhqitIlBVa/J9qSq2gugN/a4RUQOAvgbAM0DDj0mIuNU9aiIjANwPMU5NwLYCADBYNAygRARUW5s6UoSkVEi4o09/iSASQDeMzl0K4BbY49vBWDVAiEiogLJtVx1toi0A5gO4DkR2R576YsA9ojIbgCbASxR1b/GvufhpNLW+wBcKyLvArgm9jURETlIVIuvVyYYDGpz88BeKSIiSkVEWlQ1mO44znwmIiIDJgYiIjJgYiAiIgMmBiIiMmBiICIiAyYGIiIyYGIgIiIDJgYiIjJIu1ZSqYhGFZ09YYQjffD7vKir8sPjEafDIiJynbJIDNGoou1YN+7Y1Iz2rhDqawN4aGEQk8dUMzkQEQ1QFl1JnT3hRFIAgPauEO7Y1IzOnrDDkRERuU9ZJIZwpC+RFOLau0IIR/ocioiIyL3KIjH4fV7U1wYMz9XXBuD3eR2KiIjIvcoiMdRV+fHQwmAiOcTHGOqq/A5HRkTkPmUx+OzxCCaPqcaWpTNYlURElEZZJAagPzmMqq50OgwiItcri64kIiLKHBMDEREZMDEQEZEBEwMRERkwMRARkYGoqtMxZE1EOgB86HQcOboAwAmng3Ahvi/m+L4MxvfEXKr35ROqOirdCYoyMZQCEWlW1aDTcbgN3xdzfF8G43tiLh/vC7uSiIjIgImBiIgMmBics9HpAFyK74s5vi+D8T0xl/P7wjEGIiKxs5UfAAACGklEQVQyYIuBiIgMmBgcJCKrReQdEdkjIltEpMbpmNxARG4SkX0iEhWRsq46EZHrRaRNRA6IyF1Ox+MGIvKoiBwXkb1Ox+IWIjJBRF4RkdbYZ+e7uZyPicFZOwBcpqpTAPwZwI8cjsct9gKYA+A1pwNxkoh4AfwcwN8BaATwDRFpdDYqV/gPANc7HYTLRADcqaqNAD4P4B9z+VthYnCQqr6oqpHYl38AUO9kPG6hqvtVtc3pOFxgGoADqvqeqoYBPAlglsMxOU5VXwPwV6fjcBNVPaqqb8UedwPYD2D8UM/HxOAetwP4ndNBkKuMB3Ao6et25PBhp/IgIhMBTAXwxlDPUTYb9ThFRF4CMNbkpbtV9dnYMXejvyn4RCFjc1Im7wsRZUdERgB4BsD3VPX0UM/DxGAzVb0m1esichuAmQCu1jKqHU73vhAA4DCACUlf18eeIxpERCrQnxSeUNX/yuVc7EpykIhcD+CHAP5eVc86HQ+5zpsAJonIRSLiB3AzgK0Ox0QuJCIC4BEA+1X1p7mej4nBWQ8CqAawQ0R2i8h6pwNyAxGZLSLtAKYDeE5EtjsdkxNihQnLAGxH/2Di06q6z9monCci/wngdQCTRaRdRL7ldEwuMAPAAgBfjl1LdovIDUM9GWc+ExGRAVsMRERkwMRAREQGTAxERGTAxEBERAZMDEREZMDEQEREBkwMRERkwMRAREQG/w9SRn7s9yzClwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"sns.scatterplot(x, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(c) Set a random seed, and then compute the LOOCV errors that result from fitting the following four models using least squares:\n",
"\n",
"i. `Y = β0 + β1X + ε`\n",
"\n",
"ii. `Y = β0 + β1X + β2X2 + ε`\n",
"\n",
"iii. `Y = β0 +β1X +β2X2 +β3X3 +ε`\n",
"\n",
"iv. `Y = β0 +β1X +β2X2 +β3X3 +β4X4 +ε`"
]
},
{
"cell_type": "code",
"execution_count": 130,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[6.794905844691184, 1.0245281466670826, 1.0620845270746653, 1.089129915670262]"
]
},
"execution_count": 130,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def k_cv_err(k, df, seed, formula):\n",
" errs = []\n",
" for train_idx, test_idx in model_selection.KFold(n_splits=k, random_state=seed).split(df):\n",
" train_dat = df.iloc[train_idx]\n",
" test_dat = df.iloc[test_idx]\n",
" lin_reg = smf.ols(formula=formula, data=train_dat).fit()\n",
" errs.append(((test_dat['y'] - lin_reg.predict(test_dat)) ** 2).mean())\n",
" return pd.DataFrame(errs).mean()[0]\n",
"\n",
"seed = 0\n",
"formulae = ['y ~ x', \\\n",
" 'y ~ x + np.power(x, 2)', \\\n",
" 'y ~ x + np.power(x, 2) + np.power(x, 3)',\n",
" 'y ~ x + np.power(x, 2) + np.power(x, 3) + np.power(x, 4)']\n",
"df = pd.DataFrame({'x' : x, 'y' : y})\n",
"[k_cv_err(df.shape[0], df, seed, f) for f in formulae]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(d) Repeat (c) using another random seed, and report your results.\n",
"Are your results the same as what you got in (c)? Why?\n",
"\n",
"- The test error estimates are identical.\n",
"- This is to be expected as LOOCV is deterministic."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(e) Which of the models in (c) had the smallest LOOCV error? Is this what you expected? Explain your answer.\n",
"\n",
"- `'y ~ x + np.power(x, 2)'` had the lowest test error \n",
"- This is to be expected as the data were generated from a quadratic model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(f) Comment on the statistical significance of the coefficient estimates that results from fitting each of the models in (c) using least squares. Do these results agree with the conclusions drawn based on the cross-validation results?\n",
"\n",
"- Yes: the coefficients for polynomial terms greater than 2 are not statistically significant."
]
},
{
"cell_type": "code",
"execution_count": 133,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: y R-squared: 0.282\n",
"Model: OLS Adj. R-squared: 0.275\n",
"Method: Least Squares F-statistic: 38.46\n",
"Date: Mon, 24 Sep 2018 Prob (F-statistic): 1.33e-08\n",
"Time: 21:07:40 Log-Likelihood: -233.66\n",
"No. Observations: 100 AIC: 471.3\n",
"Df Residuals: 98 BIC: 476.5\n",
"Df Model: 1 \n",
"Covariance Type: nonrobust \n",
"==============================================================================\n",
" coef std err t P>|t| [0.025 0.975]\n",
"------------------------------------------------------------------------------\n",
"Intercept -1.9088 0.253 -7.547 0.000 -2.411 -1.407\n",
"x 1.5987 0.258 6.202 0.000 1.087 2.110\n",
"==============================================================================\n",
"Omnibus: 32.535 Durbin-Watson: 1.964\n",
"Prob(Omnibus): 0.000 Jarque-Bera (JB): 55.882\n",
"Skew: -1.391 Prob(JB): 7.34e-13\n",
"Kurtosis: 5.381 Cond. No. 1.03\n",
"==============================================================================\n",
"\n",
"Warnings:\n",
"[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: y R-squared: 0.891\n",
"Model: OLS Adj. R-squared: 0.889\n",
"Method: Least Squares F-statistic: 396.2\n",
"Date: Mon, 24 Sep 2018 Prob (F-statistic): 2.13e-47\n",
"Time: 21:07:40 Log-Likelihood: -139.43\n",
"No. Observations: 100 AIC: 284.9\n",
"Df Residuals: 97 BIC: 292.7\n",
"Df Model: 2 \n",
"Covariance Type: nonrobust \n",
"==================================================================================\n",
" coef std err t P>|t| [0.025 0.975]\n",
"----------------------------------------------------------------------------------\n",
"Intercept 0.0021 0.129 0.016 0.987 -0.253 0.257\n",
"x 0.9236 0.105 8.791 0.000 0.715 1.132\n",
"np.power(x, 2) -1.9959 0.086 -23.273 0.000 -2.166 -1.826\n",
"==============================================================================\n",
"Omnibus: 19.878 Durbin-Watson: 1.841\n",
"Prob(Omnibus): 0.000 Jarque-Bera (JB): 27.877\n",
"Skew: 0.940 Prob(JB): 8.84e-07\n",
"Kurtosis: 4.776 Cond. No. 2.52\n",
"==============================================================================\n",
"\n",
"Warnings:\n",
"[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: y R-squared: 0.891\n",
"Model: OLS Adj. R-squared: 0.888\n",
"Method: Least Squares F-statistic: 262.7\n",
"Date: Mon, 24 Sep 2018 Prob (F-statistic): 3.90e-46\n",
"Time: 21:07:40 Log-Likelihood: -139.21\n",
"No. Observations: 100 AIC: 286.4\n",
"Df Residuals: 96 BIC: 296.8\n",
"Df Model: 3 \n",
"Covariance Type: nonrobust \n",
"==================================================================================\n",
" coef std err t P>|t| [0.025 0.975]\n",
"----------------------------------------------------------------------------------\n",
"Intercept -0.0146 0.132 -0.111 0.912 -0.276 0.247\n",
"x 0.8183 0.193 4.240 0.000 0.435 1.201\n",
"np.power(x, 2) -1.9635 0.099 -19.766 0.000 -2.161 -1.766\n",
"np.power(x, 3) 0.0473 0.073 0.651 0.517 -0.097 0.191\n",
"==============================================================================\n",
"Omnibus: 20.095 Durbin-Watson: 1.817\n",
"Prob(Omnibus): 0.000 Jarque-Bera (JB): 27.764\n",
"Skew: 0.961 Prob(JB): 9.35e-07\n",
"Kurtosis: 4.723 Cond. No. 6.51\n",
"==============================================================================\n",
"\n",
"Warnings:\n",
"[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
" OLS Regression Results \n",
"==============================================================================\n",
"Dep. Variable: y R-squared: 0.893\n",
"Model: OLS Adj. R-squared: 0.888\n",
"Method: Least Squares F-statistic: 197.4\n",
"Date: Mon, 24 Sep 2018 Prob (F-statistic): 4.09e-45\n",
"Time: 21:07:40 Log-Likelihood: -138.66\n",
"No. Observations: 100 AIC: 287.3\n",
"Df Residuals: 95 BIC: 300.3\n",
"Df Model: 4 \n",
"Covariance Type: nonrobust \n",
"==================================================================================\n",
" coef std err t P>|t| [0.025 0.975]\n",
"----------------------------------------------------------------------------------\n",
"Intercept -0.0940 0.153 -0.615 0.540 -0.397 0.209\n",
"x 0.9288 0.221 4.200 0.000 0.490 1.368\n",
"np.power(x, 2) -1.7423 0.238 -7.321 0.000 -2.215 -1.270\n",
"np.power(x, 3) -0.0229 0.100 -0.229 0.819 -0.221 0.175\n",
"np.power(x, 4) -0.0657 0.064 -1.023 0.309 -0.193 0.062\n",
"==============================================================================\n",
"Omnibus: 18.005 Durbin-Watson: 1.839\n",
"Prob(Omnibus): 0.000 Jarque-Bera (JB): 23.510\n",
"Skew: 0.899 Prob(JB): 7.85e-06\n",
"Kurtosis: 4.553 Cond. No. 19.8\n",
"==============================================================================\n",
"\n",
"Warnings:\n",
"[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n"
]
}
],
"source": [
"for f in formulae:\n",
" print(smf.ols(formula=f, data=df).fit().summary())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"9. We will now consider the Boston housing data set, from the MASS library."
]
},
{
"cell_type": "code",
"execution_count": 165,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>CRIM</th>\n",
" <th>ZN</th>\n",
" <th>INDUS</th>\n",
" <th>CHAS</th>\n",
" <th>NOX</th>\n",
" <th>RM</th>\n",
" <th>AGE</th>\n",
" <th>DIS</th>\n",
" <th>RAD</th>\n",
" <th>TAX</th>\n",
" <th>PTRATIO</th>\n",
" <th>B</th>\n",
" <th>LSTAT</th>\n",
" <th>MEDV</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>0.00632</td>\n",
" <td>18.0</td>\n",
" <td>2.31</td>\n",
" <td>0.0</td>\n",
" <td>0.538</td>\n",
" <td>6.575</td>\n",
" <td>65.2</td>\n",
" <td>4.0900</td>\n",
" <td>1.0</td>\n",
" <td>296.0</td>\n",
" <td>15.3</td>\n",
" <td>396.90</td>\n",
" <td>4.98</td>\n",
" <td>24.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>0.02731</td>\n",
" <td>0.0</td>\n",
" <td>7.07</td>\n",
" <td>0.0</td>\n",
" <td>0.469</td>\n",
" <td>6.421</td>\n",
" <td>78.9</td>\n",
" <td>4.9671</td>\n",
" <td>2.0</td>\n",
" <td>242.0</td>\n",
" <td>17.8</td>\n",
" <td>396.90</td>\n",
" <td>9.14</td>\n",
" <td>21.6</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>0.02729</td>\n",
" <td>0.0</td>\n",
" <td>7.07</td>\n",
" <td>0.0</td>\n",
" <td>0.469</td>\n",
" <td>7.185</td>\n",
" <td>61.1</td>\n",
" <td>4.9671</td>\n",
" <td>2.0</td>\n",
" <td>242.0</td>\n",
" <td>17.8</td>\n",
" <td>392.83</td>\n",
" <td>4.03</td>\n",
" <td>34.7</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>0.03237</td>\n",
" <td>0.0</td>\n",
" <td>2.18</td>\n",
" <td>0.0</td>\n",
" <td>0.458</td>\n",
" <td>6.998</td>\n",
" <td>45.8</td>\n",
" <td>6.0622</td>\n",
" <td>3.0</td>\n",
" <td>222.0</td>\n",
" <td>18.7</td>\n",
" <td>394.63</td>\n",
" <td>2.94</td>\n",
" <td>33.4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>0.06905</td>\n",
" <td>0.0</td>\n",
" <td>2.18</td>\n",
" <td>0.0</td>\n",
" <td>0.458</td>\n",
" <td>7.147</td>\n",
" <td>54.2</td>\n",
" <td>6.0622</td>\n",
" <td>3.0</td>\n",
" <td>222.0</td>\n",
" <td>18.7</td>\n",
" <td>396.90</td>\n",
" <td>5.33</td>\n",
" <td>36.2</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX \\\n",
"0 0.00632 18.0 2.31 0.0 0.538 6.575 65.2 4.0900 1.0 296.0 \n",
"1 0.02731 0.0 7.07 0.0 0.469 6.421 78.9 4.9671 2.0 242.0 \n",
"2 0.02729 0.0 7.07 0.0 0.469 7.185 61.1 4.9671 2.0 242.0 \n",
"3 0.03237 0.0 2.18 0.0 0.458 6.998 45.8 6.0622 3.0 222.0 \n",
"4 0.06905 0.0 2.18 0.0 0.458 7.147 54.2 6.0622 3.0 222.0 \n",
"\n",
" PTRATIO B LSTAT MEDV \n",
"0 15.3 396.90 4.98 24.0 \n",
"1 17.8 396.90 9.14 21.6 \n",
"2 17.8 392.83 4.03 34.7 \n",
"3 18.7 394.63 2.94 33.4 \n",
"4 18.7 396.90 5.33 36.2 "
]
},
"execution_count": 165,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boston = datasets.load_boston()\n",
"boston = pd.DataFrame(data=np.c_[boston['data'], boston['target']], columns= [c for c in boston['feature_names']] + ['MEDV'])\n",
"boston.head()"
]
},
{
"cell_type": "code",
"execution_count": 168,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"count 506.000000\n",
"mean 22.532806\n",
"std 9.197104\n",
"min 5.000000\n",
"25% 17.025000\n",
"50% 21.200000\n",
"75% 25.000000\n",
"max 50.000000\n",
"Name: MEDV, dtype: float64"
]
},
"execution_count": 168,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boston['MEDV'].describe()"
]
},
{
"cell_type": "code",
"execution_count": 145,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"22.532806324110677"
]
},
"execution_count": 145,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boston['MEDV'].mean()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(a) Based on this data set, provide an estimate for the population mean of `medv`. Call this estimate `μ_hat`.\n",
"\n",
"*Hint: We can compute the standard error of the sample mean by dividing the sample standard deviation by the square root of the number of observations.*"
]
},
{
"cell_type": "code",
"execution_count": 147,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.40886114749753505"
]
},
"execution_count": 147,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boston['MEDV'].std() / boston.shape[0] ** 0.5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(c) Now estimate the standard error of μ_hat using the bootstrap. How does this compare to your answer from (b)?\n",
"\n",
"- It is lower"
]
},
{
"cell_type": "code",
"execution_count": 179,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.2616310836264143\n"
]
}
],
"source": [
"rho = 10 ^ 4\n",
"i = 0\n",
"means = []\n",
"while i < rho:\n",
" sample = boston.sample(n=boston.shape[0], replace=True)\n",
" means.append(sample['MEDV'].mean())\n",
" i += 1\n",
"std_err = np.array(means).std()\n",
"#mean_of_means = np.array(means).mean()\n",
"print(std_err)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(d) Based on your bootstrap estimate from (c), provide a `95%` confidence interval for the mean of `medv`. Compare it to the results obtained using `t.test(Boston$medv)`.\n",
"\n",
"*Hint: You can approximate a `95%` confidence interval using the formula:*\n",
"\n",
"```\n",
"[μ_hat − 2SE(μ_hat), μ_hat + 2SE(μ_hat)]\n",
"```\n",
"\n",
"- The CIs calculated from the bootstrap SE estimate are narrower"
]
},
{
"cell_type": "code",
"execution_count": 191,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(22.00954415685785, 23.056068491363504)"
]
},
"execution_count": 191,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(boston['MEDV'].mean() - (2 * std_err), boston['MEDV'].mean() + 2 * std_err)"
]
},
{
"cell_type": "code",
"execution_count": 193,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(21.72952801457859, 23.336084633642756)"
]
},
"execution_count": 193,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sms.DescrStatsW(boston['MEDV']).tconfint_mean()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(e) Based on this dataset, provide an estimate, `μ_med_hat`, for the median value of `medv` in the population."
]
},
{
"cell_type": "code",
"execution_count": 176,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"21.2"
]
},
"execution_count": 176,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boston['MEDV'].median()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(f) We now would like to estimate the standard error of `μ_med_hat`. Unfortunately, there is no simple formula for computing the standard error of the median. Instead, estimate the standard error of the median using the bootstrap.\n",
"\n",
"- The estimate for the median has higher SE than the estimate for the mean"
]
},
{
"cell_type": "code",
"execution_count": 177,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.3624604130390076\n"
]
}
],
"source": [
"rho = 10 ^ 4\n",
"i = 0\n",
"meds = []\n",
"while i < rho:\n",
" sample = boston.sample(n=boston.shape[0], replace=True)\n",
" meds.append(sample['MEDV'].median())\n",
" i += 1\n",
"std_err = np.array(meds).std()\n",
"print(std_err)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(g) Based on this data set, provide an estimate for the tenth percentile of `medv` in Boston suburbs. Call this quantity `μ_hat_0.1` (You can use the `quantile()` function.)"
]
},
{
"cell_type": "code",
"execution_count": 198,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"12.75"
]
},
"execution_count": 198,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boston['MEDV'].quantile(0.1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(h) Use the bootstrap to estimate the standard error of `μ_hat_0.1`. Comment on your findings.\n",
"\n",
"- SE of tenth quantile estimate is 0.5397112795802241\n",
"- ?other comments"
]
},
{
"cell_type": "code",
"execution_count": 200,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.5397112795802241\n"
]
}
],
"source": [
"rho = 10 ^ 4\n",
"i = 0\n",
"quants = []\n",
"while i < rho:\n",
" sample = boston.sample(n=boston.shape[0], replace=True)\n",
" quants.append(sample['MEDV'].quantile(0.1))\n",
" i += 1\n",
"std_err = np.array(quants).std()\n",
"print(std_err)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}