{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Outlier detection on a real data set\n\nThis example illustrates the need for robust covariance estimation\non a real data set. It is useful both for outlier detection and for\na better understanding of the data structure.\n\nWe selected two sets of two variables from the Wine data set\nas an illustration of what kind of analysis can be done with several\noutlier detection tools. For the purpose of visualization, we are working\nwith two-dimensional examples, but one should be aware that things are\nnot so trivial in high-dimension, as it will be pointed out.\n\nIn both examples below, the main result is that the empirical covariance\nestimate, as a non-robust one, is highly influenced by the heterogeneous\nstructure of the observations. Although the robust covariance estimate is\nable to focus on the main mode of the data distribution, it sticks to the\nassumption that the data should be Gaussian distributed, yielding some biased\nestimation of the data structure, but yet accurate to some extent.\nThe One-Class SVM does not assume any parametric form of the data distribution\nand can therefore model the complex shape of the data much better.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Authors: The scikit-learn developers\n# SPDX-License-Identifier: BSD-3-Clause"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## First example\n\nThe first example illustrates how the Minimum Covariance Determinant\nrobust estimator can help concentrate on a relevant cluster when outlying\npoints exist. Here the empirical covariance estimation is skewed by points\noutside of the main cluster. Of course, some screening tools would have pointed\nout the presence of two clusters (Support Vector Machines, Gaussian Mixture\nModels, univariate outlier detection, ...). But had it been a high-dimensional\nexample, none of these could be applied that easily.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from sklearn.covariance import EllipticEnvelope\nfrom sklearn.inspection import DecisionBoundaryDisplay\nfrom sklearn.svm import OneClassSVM\n\nestimators = {\n    \"Empirical Covariance\": EllipticEnvelope(support_fraction=1.0, contamination=0.25),\n    \"Robust Covariance (Minimum Covariance Determinant)\": EllipticEnvelope(\n        contamination=0.25\n    ),\n    \"OCSVM\": OneClassSVM(nu=0.25, gamma=0.35),\n}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import matplotlib.lines as mlines\nimport matplotlib.pyplot as plt\n\nfrom sklearn.datasets import load_wine\n\nX = load_wine()[\"data\"][:, [1, 2]]  # two clusters\n\nfig, ax = plt.subplots()\ncolors = [\"tab:blue\", \"tab:orange\", \"tab:red\"]\n# Learn a frontier for outlier detection with several classifiers\nlegend_lines = []\nfor color, (name, estimator) in zip(colors, estimators.items()):\n    estimator.fit(X)\n    DecisionBoundaryDisplay.from_estimator(\n        estimator,\n        X,\n        response_method=\"decision_function\",\n        plot_method=\"contour\",\n        levels=[0],\n        colors=color,\n        ax=ax,\n    )\n    legend_lines.append(mlines.Line2D([], [], color=color, label=name))\n\n\nax.scatter(X[:, 0], X[:, 1], color=\"black\")\nbbox_args = dict(boxstyle=\"round\", fc=\"0.8\")\narrow_args = dict(arrowstyle=\"->\")\nax.annotate(\n    \"outlying points\",\n    xy=(4, 2),\n    xycoords=\"data\",\n    textcoords=\"data\",\n    xytext=(3, 1.25),\n    bbox=bbox_args,\n    arrowprops=arrow_args,\n)\nax.legend(handles=legend_lines, loc=\"upper center\")\n_ = ax.set(\n    xlabel=\"ash\",\n    ylabel=\"malic_acid\",\n    title=\"Outlier detection on a real data set (wine recognition)\",\n)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Second example\n\nThe second example shows the ability of the Minimum Covariance Determinant\nrobust estimator of covariance to concentrate on the main mode of the data\ndistribution: the location seems to be well estimated, although the\ncovariance is hard to estimate due to the banana-shaped distribution. Anyway,\nwe can get rid of some outlying observations. The One-Class SVM is able to\ncapture the real data structure, but the difficulty is to adjust its kernel\nbandwidth parameter so as to obtain a good compromise between the shape of\nthe data scatter matrix and the risk of over-fitting the data.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "X = load_wine()[\"data\"][:, [6, 9]]  # \"banana\"-shaped\n\nfig, ax = plt.subplots()\ncolors = [\"tab:blue\", \"tab:orange\", \"tab:red\"]\n# Learn a frontier for outlier detection with several classifiers\nlegend_lines = []\nfor color, (name, estimator) in zip(colors, estimators.items()):\n    estimator.fit(X)\n    DecisionBoundaryDisplay.from_estimator(\n        estimator,\n        X,\n        response_method=\"decision_function\",\n        plot_method=\"contour\",\n        levels=[0],\n        colors=color,\n        ax=ax,\n    )\n    legend_lines.append(mlines.Line2D([], [], color=color, label=name))\n\n\nax.scatter(X[:, 0], X[:, 1], color=\"black\")\nax.legend(handles=legend_lines, loc=\"upper center\")\nax.set(\n    xlabel=\"flavanoids\",\n    ylabel=\"color_intensity\",\n    title=\"Outlier detection on a real data set (wine recognition)\",\n)\n\nplt.show()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.21"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}