{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Plotting Cross-Validated Predictions\n\nThis example shows how to use\n:func:`~sklearn.model_selection.cross_val_predict` together with\n:class:`~sklearn.metrics.PredictionErrorDisplay` to visualize prediction\nerrors.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Authors: The scikit-learn developers\n# SPDX-License-Identifier: BSD-3-Clause"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We will load the diabetes dataset and create an instance of a linear\nregression model.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from sklearn.datasets import load_diabetes\nfrom sklearn.linear_model import LinearRegression\n\nX, y = load_diabetes(return_X_y=True)\nlr = LinearRegression()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        ":func:`~sklearn.model_selection.cross_val_predict` returns an array of the\nsame size of `y` where each entry is a prediction obtained by cross\nvalidation.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from sklearn.model_selection import cross_val_predict\n\ny_pred = cross_val_predict(lr, X, y, cv=10)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Since `cv=10`, it means that we trained 10 models and each model was\nused to predict on one of the 10 folds. We can now use the\n:class:`~sklearn.metrics.PredictionErrorDisplay` to visualize the\nprediction errors.\n\nOn the left axis, we plot the observed values $y$ vs. the predicted\nvalues $\\hat{y}$ given by the models. On the right axis, we plot the\nresiduals (i.e. the difference between the observed values and the predicted\nvalues) vs. the predicted values.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n\nfrom sklearn.metrics import PredictionErrorDisplay\n\nfig, axs = plt.subplots(ncols=2, figsize=(8, 4))\nPredictionErrorDisplay.from_predictions(\n    y,\n    y_pred=y_pred,\n    kind=\"actual_vs_predicted\",\n    subsample=100,\n    ax=axs[0],\n    random_state=0,\n)\naxs[0].set_title(\"Actual vs. Predicted values\")\nPredictionErrorDisplay.from_predictions(\n    y,\n    y_pred=y_pred,\n    kind=\"residual_vs_predicted\",\n    subsample=100,\n    ax=axs[1],\n    random_state=0,\n)\naxs[1].set_title(\"Residuals vs. Predicted Values\")\nfig.suptitle(\"Plotting cross-validated predictions\")\nplt.tight_layout()\nplt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "It is important to note that we used\n:func:`~sklearn.model_selection.cross_val_predict` for visualization\npurpose only in this example.\n\nIt would be problematic to\nquantitatively assess the model performance by computing a single\nperformance metric from the concatenated predictions returned by\n:func:`~sklearn.model_selection.cross_val_predict`\nwhen the different CV folds vary by size and distributions.\n\nIt is recommended to compute per-fold performance metrics using:\n:func:`~sklearn.model_selection.cross_val_score` or\n:func:`~sklearn.model_selection.cross_validate` instead.\n\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.21"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}