{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Ordinary Least Squares and Ridge Regression Variance\nDue to the few points in each dimension and the straight\nline that linear regression uses to follow these points\nas well as it can, noise on the observations will cause\ngreat variance as shown in the first plot. Every line's slope\ncan vary quite a bit for each prediction due to the noise\ninduced in the observations.\n\nRidge regression is basically minimizing a penalised version\nof the least-squared function. The penalising `shrinks` the\nvalue of the regression coefficients.\nDespite the few data points in each dimension, the slope\nof the prediction is much more stable and the variance\nin the line itself is greatly reduced, in comparison to that\nof the standard linear regression\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Authors: The scikit-learn developers\n# SPDX-License-Identifier: BSD-3-Clause\n\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfrom sklearn import linear_model\n\nX_train = np.c_[0.5, 1].T\ny_train = [0.5, 1]\nX_test = np.c_[0, 2].T\n\nnp.random.seed(0)\n\nclassifiers = dict(\n    ols=linear_model.LinearRegression(), ridge=linear_model.Ridge(alpha=0.1)\n)\n\nfor name, clf in classifiers.items():\n    fig, ax = plt.subplots(figsize=(4, 3))\n\n    for _ in range(6):\n        this_X = 0.1 * np.random.normal(size=(2, 1)) + X_train\n        clf.fit(this_X, y_train)\n\n        ax.plot(X_test, clf.predict(X_test), color=\"gray\")\n        ax.scatter(this_X, y_train, s=3, c=\"gray\", marker=\"o\", zorder=10)\n\n    clf.fit(X_train, y_train)\n    ax.plot(X_test, clf.predict(X_test), linewidth=2, color=\"blue\")\n    ax.scatter(X_train, y_train, s=30, c=\"red\", marker=\"+\", zorder=10)\n\n    ax.set_title(name)\n    ax.set_xlim(0, 2)\n    ax.set_ylim((0, 1.6))\n    ax.set_xlabel(\"X\")\n    ax.set_ylabel(\"y\")\n\n    fig.tight_layout()\n\nplt.show()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.21"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}