{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions for generating confusion matrices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `from mlxtend.evaluate import confusion_matrix`    \n",
    "> `from mlxtend.plotting import plot_confusion_matrix`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Confusion Matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The *confusion matrix* (or *error matrix*) is one way to summarize the performance of a classifier for binary classification tasks. This square matrix consists of columns and rows that list the number of instances as absolute or relative \"actual class\" vs. \"predicted class\" ratios.\n",
    "\n",
    "\n",
    "Let $P$ be the label of class 1 and $N$ be the label of a second class or the label of all classes that are *not class 1* in a multi-class setting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./confusion_matrix_files/confusion_matrix_1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "\n",
    "- -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Binary classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[3, 1],\n",
       "       [2, 2]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from mlxtend.evaluate import confusion_matrix\n",
    "\n",
    "y_target =    [0, 0, 1, 0, 0, 1, 1, 1]\n",
    "y_predicted = [1, 0, 1, 0, 0, 0, 0, 1]\n",
    "\n",
    "cm = confusion_matrix(y_target=y_target, \n",
    "                      y_predicted=y_predicted)\n",
    "cm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To visualize the confusion matrix using matplotlib, see the utility function [`mlxtend.plotting.plot_confusion_matrix`](../plotting/plot_confusion_matrix):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPkAAAD+CAYAAAD4SqMoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAEhxJREFUeJzt3XmUXGWdxvHvQxKMSQSMSNhCFiAgowSCmhMHCAhziFEC\njCICQ9SIG/t+hOBkBESEmAzLMAgIRIkDDiJkzkRAMCooy2ELMIEAJgFExBDD1knI9ps/7m2sFJ3Q\nXX2rbvVbz+ecOl33vW/d+nVVP31vvVX1XkUEZpaujcouwMzqyyE3S5xDbpY4h9wscQ65WeIccrPE\nOeRmiXPIzRLXu+wCukrSB4ADgEXAinKrMStNX2AocHtELNlQxx4XcrKAzyy7CLMmcSTw0w116Ikh\nXwRw2tnfY/CQYSWX0rrG7LZj2SW0tKeeepKJR/0L5HnYkJ4Y8hUAg4cMY4cRu5RdS8saNeojZZdg\nmXd9yeqBN7PEOeRmiXPIzRLnkJslziE3S5xDbpY4h9wscQ65WeIccrPEOeRmiXPIzRLnkJslziE3\nS5xDbpY4h9wscQ65WeIccrPEOeRmiXPIzRLnkJslziE3S5xDbpY4h9wscQ65WeIccrPEOeRmiXPI\nzRLnkJslziE3S5xDbpY4h9wscQ65WeIccrPEOeRmiXPIzRLnkJslziE3S5xDbpY4h9wscU0TcknH\nSlooabmk+yR9rOyaijT71p9x3KTPcej4T3Do+E9w6jFH8eD995RdVsu5++67OfigCWw3eBv69N6I\n/5k1q+yS6q4pQi7pMOAHwBRgd2AucLukzUstrECbb7ElX/r6SVxy1Y1cfOUNjBz1cc6dfCLPL1pQ\ndmktpa2tjZG77cZll12OpLLLaYjeZReQOxn4YUT8GEDSN4BPA5OAC8ssrCgfH7P3OssTjz6e2bf+\njPnzHmO7ocNLqqr1jBs3jnHjxgEQESVX0xil78kl9QH2AO5qb4vs0b8TGFNWXfW0du1afnvXL3lr\nxQp2/oeRZZdjiWuGPfnmQC/g5ar2l4GdGl9O/Sxa8AynHXMUK1eu5L39+jH5vOkMHjKs7LIscaXv\nyVvJ4O2Gcek1NzH9ipmMP+jzTDt/Mi88t7DssixxzbAnfwVYAwyqah8E/GV9N7rqsovo13/AOm1j\n9/sU++w/vvACi9Krd2+22npbALYf8SGefvIJZt10Pcee+u2SK7OUlR7yiFgl6SFgP2AWgLJhz/2A\nS9Z3u68edzo7jNilMUXWScRaVq1aVXYZlrjSQ56bBlyXh/0BstH2fsB1ZRZVpBlXXsweo/dki0Fb\nsWxZG7+5czaPz32I86Z+tezSWkpbWxvPPvvs2yPrCxYuYO7cuQwcOJDBgweXXF19NEXII+Jn+Xvi\n55Adpj8KHBARi8utrDivvvo3pn3vbJYueYV+AwYwbPgIzpt6BSNHjS67tJby4IMPsv9++yIJSZx+\n2qkATJz4Ra7+0TUlV1cfTRFygIi4HLi87Drq5cQzvlN2CQaMHTuWVavXll1GQ3l03SxxDrlZ4hxy\ns8Q55GaJc8jNEueQmyXOITdLnENuljiH3CxxDrlZ4hxys8Q55GaJ69QXVCRN6OwGIyL9OW7NepDO\nfgvtlk72C7L52sysSXQq5BHhw3qzHqpb4ZXUt6hCzKw+uhxySb0kfVvSi8Cbkobn7edK+krhFZpZ\nt9SyJ58MfAk4A1hZ0f4EcHQBNZlZgWoJ+UTgaxExk2wq5XZzgZ0LqcrMClNLyLcBnl3Ptvp0rxwz\nK1otIZ8H7NVB++eAR7pXjpkVrZbZWs8BZkjahuyfxD9L2onsMP4zRRZnZt3X5T15RNwKHAjsD7SR\nhf5DwIER8atiyzOz7qpp3vWIuBv4p4JrMbM6qPnkCpI+SrYHB5gXEQ8VU5KZFanLIZe0LfBfwD8C\nr+bNm0n6A/CFiPhTgfWZWTfVMrp+NdlbZR+KiIERMZBsj75Rvs7Mmkgth+tjgU9ExPz2hoiYL+l4\n4O7CKjOzQtSyJ3+Bjj/00gv4c/fKMbOi1RLy04FL84E34O1BuIuB04oqzMyK0dmZYZaSTQjRrj9w\nv6TVFdtZDVxD5yeYMLMG6Oxr8pPqWoWZ1U1nZ4aZUe9CzKw+av4wDLw9M8zGlW0R8Xq3KjKzQtUy\nM0x/SZdJ+ivZZ9eXVl3MrInUMrp+IfBJ4JvAW2SzwUwhe/tsYnGlmVkRajlcPxCYGBG/kXQtcHdE\nPCvpOeBIYGahFZpZt9SyJx8ILMivv54vA9wD7F1EUWZWnFpCvgAYll9/Cvh8fv1A/v6FFTNrErWE\n/FpgZH79AuBYSSuA6cBFRRVmZsXo8mvyiJhecf1OSTsDewDPRsRjRRZnZt3XrffJASLiOeC5Amox\nszro7GfXT+jsBiPiktrLMbOidXZPfnIn+wXQkJAPGrw1g3cc2oi7sg7Me/mNsktoaQuWtHW6b2c/\nuz7s3XuZWTPyKYnNEueQmyXOITdLnENuljiH3CxxNYVc0l6Srpd0b37iQyQdJWnPYsszs+6qZdKI\nzwK3A8uB3YH35Ks2Bc4qrjQzK0Ite/KzgW9ExFeBVRXtvwdGFVKVmRWmlpDvBPyug/bXgM26V46Z\nFa2WkP8F2KGD9j35+2QSZtYkagn5VcDFkkaTfVZ9a0lHAlOB/yyyODPrvlq+anoB2T+Hu4B+ZIfu\nbwFTI+LSAmszswLUMmlEAN+VdBHZYfsAYF5EvFl0cWbWfTVPGhERK4F5BdZiZnXQ5ZBLmsO6Jz9c\nR0R8slsVmVmhatmTP1q13AfYDfgw4HOmmTWZWl6TdzhLjKR/I3t9bmZNpMgvqFwPTCpwe2ZWgCJD\nPgZYUeD2zKwAtQy83VzdBGwFfBQ4t4iizKw4tQy8vVa1vBaYD/xrRNzR/ZLMrEhdCrmkXmSnSXo8\nInwucrMeoEuvySNiDXAH/raZWY9Ry8DbE8Dwogsxs/qoddKIqZI+I2krSZtUXoou0My6p5aBt9n5\nz1ms+/FW5cu9uluUmRWnlpDvW3gVZlY3tYR8IfBC/pXTt0kSMLiQqsysMLW8Jl8IfLCD9oH5OjNr\nIrWEvP21d7UB+GOtZk2n04frkqblVwM4V9KyitW9gNG882uoZlayrrwm3z3/KeAjwMqKdSuBuWST\nOZpZE+l0yCNiXwBJ1wInRsTrdavKzApTy6QRX65HIWZWHz6rqVniHHKzxDnkZolzyM0S55CbJc4h\nN0ucQ26WuKYIuaS9JM2S9KKktZImlF1T0a6+9AccMX4fxozYhn123Z6TJh3Boj8+U3ZZLaVVn4Om\nCDnQn+xz78ewgfOs9WQP3/8HDp/0dWb+76+58sZZrF69im8cfggrli8vu7SW0arPQc1nNS1SRNwG\n3AZvfy89OZdf//N1ls+dfgX77DqceY89yqjRY0qqqrW06nPQLHvylvPG668iiU3f//6yS2lZrfIc\nOOQliAgunPItdv/4GLYfsXPZ5bSkVnoOmuJwvdWcd+Yp/PHp+fz4Vp9wpiyt9Bz02JBfNOVbDNhk\n03XaPnXw5xh/yKElVdQ55591KvfcdQfX/eI2Pjhoy7LLaUmt9hz02JCf/p0L2GXX3couo0vOP+tU\n5tw+m2tvns1W23rOyzK04nPQFCGX1B/YgWzWGYDhkkYCf4uIF8qrrDjnnXkyv7zl51xy3Q28t19/\nliz+KwAD3rcJ7+nbt+TqWkOrPgeqmlm5nCKkscAc3vke+YyImFTVdxTw0A23/a5H7clHbrMpHb07\neM70y5lw6BElVNR6UnoO5j32KF8YtzfAHhHx8Ib6NsWePCJ+S+Ij/XNfrD7jszVaqz4HSQfLzBxy\ns+Q55GaJc8jNEueQmyXOITdLnENuljiH3CxxDrlZ4hxys8Q55GaJc8jNEueQmyXOITdLnENuljiH\n3CxxDrlZ4hxys8Q55GaJc8jNEueQmyXOITdLnENuljiH3CxxDrlZ4hxys8Q55GaJc8jNEueQmyXO\nITdLnENuljiH3CxxDrlZ4hxys8Q55GaJc8jNEueQmyXOITdLnENektm/+O+yS2h5rfIcOOQl+eUt\nN5VdQstrlefAITdLnENuljiH3CxxvcsuoAZ9AVa/8jyrXupfdi01i7faWPXSM2WX0dJ68nOw+pXn\n26/2fbe+ioj6VlMwSUcAM8uuw6xJHBkRP91Qh54Y8g8ABwCLgBXlVmNWmr7AUOD2iFiyoY49LuRm\n1jUeeDNLnENuljiH3CxxDnmDSTpW0kJJyyXdJ+ljZdfUSiTtJWmWpBclrZU0oeya6s0hbyBJhwE/\nAKYAuwNzgdslbV5qYa2lP/AocAzQEqPOHl1vIEn3AfdHxIn5soAXgEsi4sJSi2tBktYCB0fErLJr\nqSfvyRtEUh9gD+Cu9rbI/sPeCYwpqy5Ln0PeOJsDvYCXq9pfBrZsfDnWKhxys8Q55I3zCrAGGFTV\nPgj4S+PLsVbhkDdIRKwCHgL2a2/LB972A/5QVl2Wvp74VdOebBpwnaSHgAeAk4F+wHVlFtVKJPUH\ndgCUNw2XNBL4W0S8UF5l9eO30BpM0jHAGWSH6Y8Cx0fEg+VW1TokjQXm8M73yGdExKQSSqo7h9ws\ncX5NbpY4h9wscQ65WeIccrPEOeRmiXPIzRLnkJslziE3S5xDbpY4hzxhkq6VdHPF8hxJ00qoY2w+\nn9omG+jTpfnWJE2R9Eg36xqS3++u3dlOs/MXVFrLIcCqznSs+Iz3ZhHxegH3XY/PTxexzeQ/1+2Q\nNzlJffKvqXZbRLzalbsmC4DerWMPl/rv58P1RsoPly/NL69KWizpnKo+CyWdLWmGpNeAH+bt20q6\nUdJSSUsk3SJpSMXtNpI0LV+/WNL3qfoDrj5cl7SxpO9Lel7SCklPS/pyvt1f592WSloj6Zr8NpJ0\npqQFkpZJekTSZ6vuZ7yk+fn6u8jO2dXVx+qCfBttkv4o6RxJvTro97W8/rb88Xlf1fqjJc3Lp8Ce\nJ+mbXa2lx4sIXxp0ITv8fY3se+U7AocDbwJfqeizEFhK9l3zYfmlN/B/wJXALsBOwE+AJ4He+e3O\nIJt95qB8/VX5fd1cdf/TKpZvJDtx5ASyIO4NHEr2z+EQsplstge2AN6X32ZyXsv++W0mAsuAvfL1\ng4HlwIUVv+NL+bY22cBjsxaYULF8FjAa2A74NPBn4LSK9VOAN4BfAR8B9gSeBn5S0edI4E/5YzIE\nOBhYDByVrx+S3++uZf9t1PXvruwCWumSh+yJqrbvVbblIb+pqs+RwLyqto2BNmD/fPlF4JSK9b2A\n59cXcmBE/ge+73pqHVsdzPw+3wRGV/W9Crg+v34+8HgHv2OXQt7B+lOBByqWpwArgS0r2g4AVgNb\n5MvPAIdVbWcy8Pv8ekuE3K/JG+++quV7gVMkKfK/PLJpoiqNBHaU9EZV+3uA7SU9AGxFNtsMABGx\nRtKGJqMYSRaI33Wh9h3IZrL5VT51Vbs+wMP59Z2B+6tud28X7gN4+0QUx5MdSQwgO5p5rarb8xFR\nOT/evWQvQXeS9GZ+2x9JurqiTy+gK2MTPZ5D3pzaqpYHAA8CR/DOgaLFHbR1xvIabjMg/zme7PC5\n0ls1bK9DksYA1wPfBu4gC/fhwCld2Ex7rUdT8c8vt6a7NfYkDnnjja5aHgM8U7EX78jDwOeBxRHx\nZkcdJL2Ub/uefLkX2ckcqo8K2j1Ottcby98H2SqtzH9WDnbNIwvzkIi4Zz3bfRI4sKqtqyePGAMs\niogL2hskDe2g33aStqzYm48hC/BTEbFY0p+B7SPihg3cV/JvoXl0vfG2kzRV0ghJhwPHAf/+LreZ\nSTaodqukPSUNlbSPpIslbZ33uRj4lqSDJO0EXA5str4NRsRzwI+Ba/LbDM0/tHJo3uU5sgAcKGlz\nSf3zfzBTgemSJkoaLml3ScdJOiq/3RVkLy0uzH/HI4AvdvExeiZ/nA7L7+MEskGzam8BMyTtKmmv\n/DG4MSIW5+unAGdKOl7SjpI+LOlLkk6q2Ebyb6GVPijQSheyga9Lgf8ge134CnBOVZ8FwAkd3HYL\n4FqyM64sIwvCFcCAfH0vslH7pcAS4KK8f+XA269Zd3R9Y7LQ/ons8H0+8MWK9ZPJDstXA9dUtB9P\ntldfQTZn/Gxgz4r14/NtLQN+Qxbydxt4W8O6o+sXAH8lO1T/KXAC2Yyq7eunkB3hfD2vvw24Adi0\nartfyPstzx/vOcBB+boh+f0mPfDmiRwbSNIc4JGI6MprS7Nu8eG6WeIc8sbyYZM1nA/XzRLnPblZ\n4hxys8Q55GaJc8jNEueQmyXOITdLnENuljiH3CxxDrlZ4v4fUA+dJCME/lMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x106072278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.plotting import plot_confusion_matrix\n",
    "\n",
    "fig, ax = plot_confusion_matrix(conf_mat=cm)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Multi-class classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 1, 0, 0],\n",
       "       [1, 2, 0, 0],\n",
       "       [0, 0, 1, 0],\n",
       "       [0, 0, 0, 1]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from mlxtend.evaluate import confusion_matrix\n",
    "\n",
    "y_target =    [1, 1, 1, 0, 0, 2, 0, 3]\n",
    "y_predicted = [1, 0, 1, 0, 0, 2, 1, 3]\n",
    "\n",
    "cm = confusion_matrix(y_target=y_target, \n",
    "                      y_predicted=y_predicted, \n",
    "                      binary=False)\n",
    "cm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To visualize the confusion matrix using matplotlib, see the utility function [`mlxtend.plotting.plot_confusion_matrix`](../plotting/plot_confusion matrix):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPkAAAD+CAYAAAD4SqMoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAGnlJREFUeJzt3X14FPW99/H3xzSKgoYqECwi+FCxWAgmFSTHJwinKN6g\n9gCC3BWKPae2FQUFPRaPPNRSqwIBNfbWVtSjVey52uofcNOW0NNy5MkE0UqltoCg1cUgoiaIEH/n\nj5nQzTaG3dmH2R2+r+vai+zs7MxnST67szO7v5FzDmNMdB0VdgBjTHZZyY2JOCu5MRFnJTcm4qzk\nxkScldyYiLOSGxNxVnJjIu5zYQdIlaSTgOHAduDjcNMYE5oOQG9ghXNud3szFlzJ8Qr+VNghjMkT\nE4CftTdDIZZ8O8D0O35Iz16nhRwluNKeXwg7QlpOP6lj2BGOaK+99ieu/fr/Bb8P7SnEkn8M0LPX\naZx5Vt+wswTW84u9w46Qlr6lx4cdwXgO+5bVdrwZE3FWcmMizkpuTMRZyY2JOCu5MRFnJTcm4qzk\nxkScldyYiLOSGxNxVnJjIs5KbkzEWcmNiTgruTERZyU3JuKs5MZEXN6UXNJ3JW2TtE/SWknn5XL9\nzz75E6Z96xpGXzaYCVdewl0zp/LWzu25jJC2unUvMGXi1Qwr70NZjxJWrVgWdqRAamoe5MwzTqNT\nx2OprDyfDRs2hB0pJfmWPy9KLulqYD4wCzgX2ASskNQlVxlefbmekV8bz4KHnuIH8x/mYPNB7ph+\nPfv3F84wcvuaGjn7nH7MnLcASWHHCeTZpUuZMf0WZs2aw4t1GynrX8aIy4bT0NAQdrSk5GN+5cNZ\nTSWtBdY5527yrwvYCSx2zt2TMG85ULfokWeyOjLM3vf3MOHKS/jR4iWc078848vP9sgwZT1KqH70\naYYMH5GV5WdrZJjKyvMZNHAQC6sXAeCco3evntww5UZmzLg1K+vMpFzlr6+vZ+B5FQAVzrn69uYN\n/ZVcUjFQAaxsmea8Z57fAoPDytX40YdI4vgTSsKKcMQ5cOAA9XV1DB1adWiaJKqqhrF2zZoQkyUn\nX/OHXnKgC1AExBKmx4DuuY/jPfs+/MA99O13Lqf2PiOMCEekhoYGmpub6VZa2mp6t9JS3om9E1Kq\n5OVr/kIcyDHrahbcxY7tf+W+B58IO4oxacuHkjcAzUBpwvRS4DOf/h554F6O69ip1bSLqy7jkmHp\nvQd9qHoeG9at5p77H+PEk7qmtSyTmi5dulBUVMSuWOuNul2xGN1LQ9moS0m+5g+95M65A5LqgCrg\neTi0460KWPxZ9/vXG2ZkfMfbQ9XzWLt6FXcvXkK30pMzumxzeMXFxZRXVFBbu5KRo0YB3lun2tqV\n3DDlxpDTHV6+5g+95L4FwGN+2dcD04DjgMdyFeDBBXfx3yuXc+e8xXTocCx73vPOPNOxYyeOPuaY\nXMVIS1NTIzu3baXliMmbO7ax5dVXKOn8ebr3OCXkdMmZNvVmJk+eRHl5BecNHMii6oU0NTUxceKk\nsKMlJR/z50XJnXPP+sfE5+Jtpr8EDHfOvZurDMuf/zmSuH3qda2mT71tLlWXjspVjLRs3rSR60Zf\njiQkMX/OTABGjbmGuQtrQk6XnDFjx9Kwu4HZs+8kFotRNmAAy5avoGvXwnjrlI/58+I4eSpydZw8\n2+wMKiYdBXWc3BiTXVZyYyLOSm5MxFnJjYk4K7kxEWclNybirOTGRJyV3JiIs5IbE3FWcmMizkpu\nTMRZyY2JOCu5MRFnJTcm4qzkxkScldyYiLOSGxNxVnJjIi4vxngLorTnFwp6CKWdr28PO0Ja+pb2\nCzuCSZK9khsTcVZyYyLOSm5MxFnJjYk4K7kxEWclNybirOTGRJyV3JiIs5IbE3FWcmMizkpuTMRZ\nyY2JOCu5MRFnJTcm4qzkxkScldyYiLOSGxNxSY0MI2lUsgt0zj0fPI4xJtOSHf7pV0nO54CigFmM\nMVmQ1Oa6c+6oJC+BCi7pQknPS3pL0qepbDlkUt26F5gy8WqGlfehrEcJq1YsCyNGYM8++ROmfesa\nRl82mAlXXsJdM6fy1s7tYcdKWU3Ng5x5xml06ngslZXns2HDhrAjpSTf8qf1nlxShwzl6Ai8BHwH\nb2sgFPuaGjn7nH7MnLcASWHFCOzVl+sZ+bXxLHjoKX4w/2EONh/kjunXs3//x2FHS9qzS5cyY/ot\nzJo1hxfrNlLWv4wRlw2noaEh7GhJycf8ci61TkkqAr4HXA+UAmc557ZK+j6w3Tn307QCSZ8CV37W\ne3tJ5UDdM///9/TtPyCdVbWrrEcJ1Y8+zZDhI7Ky/FyM1rr3/T1MuPISfrR4Cef0L8/osodfkJ3R\nWisrz2fQwEEsrF4EgHOO3r16csOUG5kx49asrDOTcpW/vr6egedVAFQ45+rbmzfIK/lMYBJwK/BJ\n3PQ/At8MsDyTJY0ffYgkjj+hJOwoSTlw4AD1dXUMHVp1aJokqqqGsXbNmhCTJSdf8wcp+bXAvznn\nngKa46ZvAs7OSCqTNuccDz9wD337ncupvc8IO05SGhoaaG5upltpaavp3UpLeSf2Tkipkpev+YOc\nXKEH8Jc2ph8FFKcXJ3n3zvp3OiW8Ql125WhGXDUmVxHyWs2Cu9ix/a/c9+ATYUcxIQtS8s3AhcAb\nCdNHAxvTTpSkGXPuzup78kL2UPU8NqxbzT33P8aJJ3UNO07SunTpQlFREbtisVbTd8VidC/tHlKq\n5OVr/iCb63OBByTd5t//a5IewXuvPjeT4UzqHqqex9rVq/hh9U/pVnpy2HFSUlxcTHlFBbW1Kw9N\nc85RW7uSwZWVISZLTr7mT/mV3Dn3nKSRwJ1AI16x64GRzrnfBAkhqSNwJtBy3Op0SWXAe865nUGW\nGURTUyM7t22l5YjDmzu2seXVVyjp/Hm69zglVzECe3DBXfz3yuXcOW8xHTocy573dgPQsWMnjj7m\nmJDTJWfa1JuZPHkS5eUVnDdwIIuqF9LU1MTEiZPCjpaUfMwf6ISHzrk/AP+cwRxfAVbhHSN3wHx/\n+uPA5Ayup12bN23kutGXIwlJzJ8zE4BRY65h7sKaXMUIbPnzP0cSt0+9rtX0qbfNperSUD5flLIx\nY8fSsLuB2bPvJBaLUTZgAMuWr6Br18J425GP+VM+Tn7ojtJXgC/5Vzc75+oylqr99ebkOHm2FfpZ\nTbN1nNwkJ5Xj5Cm/kks6BXga+CfgfX9yZ0kvAOOcc2+mukxjTPYE2fH2E7xDZV9yzp3onDsR7xX9\nKP82Y0weCfKe/GKg0jm3pWWCc26LpCnAHzKWzBiTEUFeyXfS9odeioC/pRfHGJNpQUo+A7jf3/EG\nHNoJtwiYnqlgxpjMSHZkmD20/gpoR2CdpINxyzkIPEryA0wYY3Ig2ffkU7OawhiTNUmV3Dn3eLaD\nGGOyI9An3lr4I8McHT/NOfdBWomMMRmV8o43SR0lPSBpF95n1/ckXIwxeSTI3vV7gKHAt4H9eKPB\nzMI7fHZt5qIZYzIhyOb6SOBa59zvJC0B/uCc+4ukN4AJwFMZTWiMSUuQV/ITga3+zx/41wFWAxdl\nIpQxJnOClHwrcJr/82vAWP/nkfz9CyvGmDwRpORLgDL/57uB70r6GFgI3JupYMaYzAgyMszCuJ9/\nK+lsoAL4i3Pu5UyGM8akL63j5ADOuTf4x0EdjTF5ItnPrt+Y7AKdc4uDxzHGZFqyr+TTkpzPATkp\n+ekndaRv6fG5WFVW9C0t7OGTNsc+DDtC2gr57ycVyX52/bTDz2WMyUdpndXUGJP/rOTGRJyV3JiI\ns5IbE3FWcmMiLlDJJV0o6UlJayT18Kd9XdIFmY1njElXkEEj/gVYAewDzgVazqRXAnwvc9GMMZkQ\n5JX8DuB659y/Agfipv8PUJ6RVMaYjAlS8j7A79uYvhfonF4cY0ymBSn5O3jnEk90AX8fTMIYkyeC\nlPwRYJGkQXifVf+CpAnAfcBDmQxnjElfkK+a3o335LASOA5v030/cJ9z7v4MZjPGZECQQSMc8ANJ\n9+JttncCNjvnPsp0OGNM+gIPGuGc+wTYnMEsxpgsSLnkklbR+uSHrTjnhqaVyBiTUUFeyV9KuF4M\nDAC+DNg504zJM0Hek7c5Soyk2Xjvz40xeSSTX1B5Epic6p0k3S5pvaQPJMUk/VLSWRnMlZKamgc5\n84zT6NTxWCorz2fDhg1hRQmk0PPXrXuBKROvZlh5H8p6lLBqxbKwI6Us334HmSz5YODjAPe7ELgf\nGAQMw9v8/7WkYzOYLSnPLl3KjOm3MGvWHF6s20hZ/zJGXDachoaGXEcJpNDzA+xrauTsc/oxc94C\nJIUdJ2X5+DuQd0QshTtIv0icBJwMfAX4vnNuTlqBpC7ALuAi59zqNm4vB+rWb6ijvDyzH5WvrDyf\nQQMHsbB6EQDOOXr36skNU25kxoxbM7qubMhl/lwM5FjWo4TqR59myPARWVl+NgZyzNXvoL6+noHn\nVQBUOOfq25s3yCv53oTLe8DvgBHpFtzXGW/v/XsZWFbSDhw4QH1dHUOHVh2aJomqqmGsXbMml1EC\nKfT8UZCvv4OUdrxJKsI7TdIrzrmMn4tc3vZZNbDaOZfTY/ANDQ00NzfTrbS01fRupaVs+fOWXEYJ\npNDzR0G+/g5SeiV3zjUDvyZ73zarAfoC47K0fGOOOEGOk/8ROB3Ylskgkh4ARgAXOufePtz8t9w8\njZKSklbTxo0bz7jx4wOtv0uXLhQVFbErFms1fVcsRvfS7oGWmUuFnj8K8vV3EHTQiPsk/R9JJ0s6\nIf4SJIRf8CuAIc65HcncZ/6ChfzquedbXYIWHKC4uJjyigpqa1cemuaco7Z2JYMrKwMvN1cKPX8U\n5OvvIMgrecuBy+dp/fFW+deLUlmYpBpgPDAKaJTU8oZmr3MuyCG5wKZNvZnJkydRXl7BeQMHsqh6\nIU1NTUycOCmXMQIr9PwATU2N7Ny2lZajPm/u2MaWV1+hpPPn6d7jlJDTHV4+/g6ClHxIhjNcj/fk\n8LuE6d8Ansjwuto1ZuxYGnY3MHv2ncRiMcoGDGDZ8hV07do1lzECK/T8AJs3beS60ZcjCUnMnzMT\ngFFjrmHuwpqQ0x1ePv4OghwnPxXY6RLu6O8Z75ns5nZQ2TxObpJnJzwMV7aPk28D2npaOpEM74wz\nxqQvSMlb3nsn6kSwj7UaY7Io6ffkkhb4Pzrg+5Ka4m4uwvvseeLXUI0xIUtlx9u5/r8C+gGfxN32\nCbAJbzBHY0weSbrkzrkhAJKWADc55z7IWipjTMYEGTTiG9kIYozJDjurqTERZyU3JuKs5MZEnJXc\nmIizkhsTcVZyYyLOSm5MxFnJjYk4K7kxEWclNybirOTGRJyV3JiICzLGmzEFPXRSi0Iewmrr7sak\n57VXcmMizkpuTMRZyY2JOCu5MRFnJTcm4qzkxkScldyYiLOSGxNxVnJjIs5KbkzEWcmNiTgruTER\nZyU3JuKs5MZEnJXcmIizkhsTcVZyYyLOSm5MxFnJjYm40Esu6XpJmyTt9S8vSLo0rDw1NQ9y5hmn\n0anjsVRWns+GDRvCihJIoeeHwn4MdeteYMrEqxlW3oeyHiWsWrEs7EjhlxzYCdwGlAMVQC3wnKQv\n5TrIs0uXMmP6LcyaNYcX6zZS1r+MEZcNp6GhIddRAin0/FD4j2FfUyNnn9OPmfMWICnsOADIORd2\nhn8gaTcw3Tm3pI3byoG69RvqKC8vz+h6KyvPZ9DAQSysXgSAc47evXpyw5QbmTHj1oyuKxsKPT/k\n9jFke7TWsh4lVD/6NEOGj8j4sje//BLjLr0IoMI5V9/evPnwSn6IpKMkjQOOA9bkct0HDhygvq6O\noUOr4vNQVTWMtWtyGiWQQs8P0XgM+SgvSi7py5I+BPYDNcBVzrnXcpmhoaGB5uZmupWWtprerbSU\nd2Lv5DJKIIWeH6LxGPJRvpxc4TWgDCgBRgNPSLqovaLfcvM0SkpKWk0bN24848aPz2pQYwpNXpTc\nOXcQ2Opf3ShpIHAT8O3Pus/8BQsz+p68S5cuFBUVsSsWazV9VyxG99LuGVtPthR6fojGY8hHebG5\n3oajgGNyucLi4mLKKyqorV15aJpzjtralQyurMxllEAKPT9E4zHko9BfySXNA5YDO4DjgQnAxcBX\nc51l2tSbmTx5EuXlFZw3cCCLqhfS1NTExImTch0lkELPD4X/GJqaGtm5bSstR63e3LGNLa++Qknn\nz9O9xymhZAq95EA34HHgZGAv8DLwVedcba6DjBk7lobdDcyefSexWIyyAQNYtnwFXbt2zXWUQAo9\nPxT+Y9i8aSPXjb4cSUhi/pyZAIwacw1zF9aEkikvj5O3J5vHyc2RpZDPalqwx8mNMZlnJTcm4qzk\nxkScldyYiLOSGxNxVnJjIs5KbkzEWcmNiTgruTERZyU3JuKs5MZEnJXcmIizkhsTcVbyz/DM00+H\nHSEtlj98y37587AjAFbyz/TMM4X9R2b5w7f8V/8VdgTASm5M5FnJjYk4K7kxEZcPY7ylqgPAn7f8\niaIsPkV98MFeNr3U7qg6ec3yh8/tb+TA269nZdkHG3a0/NjhcPMW4hhv1wBPhZ3DmDwxwTn3s/Zm\nKMSSnwQMB7YDH4ebxpjQdAB6Ayucc7vbm7HgSm6MSY3teDMm4qzkxkScldyYiLOSJ5D0XUnbJO2T\ntFbSeWFnSpakCyU9L+ktSZ9KGhV2plRIul3SekkfSIpJ+qWks8LOlQpJ10vaJGmvf3lB0qVhZrKS\nx5F0NTAfmAWcC2wCVkjqEmqw5HUEXgK+AxTiHtULgfuBQcAwoBj4taRjQ02Vmp3AbUA5UAHUAs9J\n+lJYgWzvehxJa4F1zrmb/OvC+6Utds7dE2q4FEn6FLjSOfd82FmC8p9cdwEXOedWh50nKEm7genO\nuSVhrN9eyX2SivGeeQ+dHNt5z4C/BQaHlesI1xlvi+S9sIMEIekoSeOA44A1YeUoxI+1ZksXoAiI\nJUyPAX1yH+fI5m9FVQOrnXObw86TCklfxit1B+BD4Crn3Gth5bGSm3xVA/QF/insIAG8BpQBJcBo\n4AlJF4VVdCv53zUAzUBpwvRS4J3cxzlySXoAGAFc6Jx7O+w8qXLOHQS2+lc3ShoI3AR8O4w89p7c\n55w7ANQBVS3T/E3GKuCFsHIdafyCXwEMcc7tONz8BeIo4JiwVm6v5K0tAB6TVAesB6bh7TR5LMxQ\nyZLUETgTkD/pdEllwHvOuZ3hJUuOpBpgPDAKaJTUslW11zlXEF9GkjQPWA7sAI4HJgAXA18NLZMd\nQmtN0neAW/E2018CpjjnXgw3VXIkXQys4h+PkT/unJscQqSU+If92vqD/IZz7olc5wlC0k+AocDJ\nwF7gZeBu51xtaJms5MZEm70nNybirOTGRJyV3JiIs5IbE3FWcmMizkpuTMRZyY2JOCu5MRFnJTcm\n4qzkESZpiaRfxF1fJWlBCDku9secO6GdeVIak07SLEkb08zVy19v/3SWk+/sCypHlquAA8nMGPc5\n+M7OuQ8ysO5sfH46E8uM/Oe6reR5TlKx/zXYtDnn3k9l1XgF0OFmLHBRf3y2uZ5L/uby/f7lfUnv\nSpqbMM82SXdIelzSXuD/+dNPkbRU0h5JuyX9SlKvuPsdJWmBf/u7kn5Ewh9w4ua6pKMl/UjSDkkf\nS/qzpG/4y2351tQeSc2SHvXvI3/o5K2SmiRtlPQvCesZIWmLf/tKvHN2pfp/dbe/jEZJf5U0V1JR\nG/P9m5+/0f//OT7h9m9K2uwPsb1ZUigDN4TKOWeXHF3wNn/34n1v/Yt4353+CLgubp5twB6877Kf\n5l8+B7wKPIw3JFIf4D+BPwGf8+93K97oNlf4tz/ir+sXCetfEHd9Kd6JI0fhFfEiYAzek8NVeCPl\nnAF0A4737zPTzzLMv8+1QBPeKC4APYF9wD1xj/Ftf1kntPN/8ykwKu769/CGZj4VuBz4G96Ipy23\nz8IbP+03QD/gAuDPwH/GzTMBeNP/P+kFXAm8C3zdv72Xv97+Yf9tZPXvLuwAR9LFL9kfE6b9MH6a\nX/L/SphnArA5YdrRQCMwzL/+FnBz3O1FeAMXtFly4Cz/D3zIZ2S9OLGY/jo/AgYlzPsI8KT/8zzg\nlTYeY0olb+P2W4D1cddnAZ8A3eOmDQcOAt38668DVycsZybwP/7PR0TJ7T157q1NuL4GuFmSnP+X\nhzcMVbwy4IuSPkyYfgxwhqT1eIMUrG+5wTnXLKm9wS7K8Arx+xSyn4k3Us5v/KGxWhQD9f7PZwPr\nEu6X8nDE/okupuBtSXTC25rZmzDbDudc/Ph7a/DegvaR9JF/35/6Azm0KAJS2TdR8Kzk+akx4Xon\n4EXgGv5xR9G7bUxLxr4A9+nk/zsCb/M53v4Ay2uTpMHAk8B/AL/GK/d44OYUFtOS9ZvEPfn5mtPN\nWEis5Lk3KOH6YOD1uFfxttQDY4F3nXMftTWDpLf9Za/2rxfhnSwicaugxSt4r3oX8/edbPE+8f+N\n39m1Ga/Mvdxnn9HkT8DIhGmpnpxiMLDdOXd3ywRJvduY71RJ3eNezQfjFfg159y7kv4GnOGce6ad\ndUX+EJrtXc+9UyXdJ+ksSeOBG/BOItCep/B2qj0n6QJJvSVdImmRpC/48ywC/l3SFZL64I1b3vmz\nFuicewN4AnjUv09v/0MrY/xZ3sArwEhJXSR19J9g7gMWSrpW0umSzpV0g6Sv+/f7Md5bi3v8x3gN\nMDHF/6PX/f+nq/113Ii30yzRfuBxSf0lXej/Hyx1zr3r3z4LuF3SFElflPRlSZMkTY1bRuQPoYW+\nU+BIuuDt+LofeBDvfWEDMDdhnq3AjW3ctxuwBO+MLk14Rfgx0Mm/vQhvr/0eYDdwrz9//I63Wlrv\nXT8ar7Rv4m2+bwEmxt0+E2+z/CDwaNz0KXiv6h/jjUm/DLgg7vYR/rKagN/hlfxwO96aab13/W68\n86DtBX4G3Ig36mzL7bPwtnC+5edvBJ4BShKWO86fb5///70KuMK/rZe/3kjveLOBHHNI0ipgo3Mu\nlfeWxqTFNteNiTgreW7ZZpPJOdtcNybi7JXcmIizkhsTcVZyYyLOSm5MxFnJjYk4K7kxEWclNybi\nrOTGRJyV3JiI+19MzjwBUnxJJwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10eabf8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.evaluate import confusion_matrix\n",
    "\n",
    "fig, ax = plot_confusion_matrix(conf_mat=cm)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - Multi-class to binary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By setting `binary=True`, all class labels that are not the positive class label are being summarized to class 0. The positive class label becomes class 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4, 1],\n",
       "       [1, 2]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.evaluate import confusion_matrix\n",
    "\n",
    "y_target =    [1, 1, 1, 0, 0, 2, 0, 3]\n",
    "y_predicted = [1, 0, 1, 0, 0, 2, 1, 3]\n",
    "\n",
    "cm = confusion_matrix(y_target=y_target, \n",
    "                      y_predicted=y_predicted, \n",
    "                      binary=True, \n",
    "                      positive_label=1)\n",
    "cm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To visualize the confusion matrix using matplotlib, see the utility function [`mlxtend.plotting.plot_confusion_matrix`](../plotting/plot_confusion matrix):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPkAAAD+CAYAAAD4SqMoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAEidJREFUeJzt3XmUVOWdxvHvY4MSQHGI4xLTLIqiniiIiQxG0yrOkXFB\nnbghI+5O4oYbnjHqkKPH0bjghiZxAYn7EoNojLgmonE5uBANiCK4R0VF0AaU5Z0/7m1SlA10V9+q\nW/3W8zmnTtd971K/rtNP33vfqvtehRAws3itlXcBZlZeDrlZ5Bxys8g55GaRc8jNIueQm0XOITeL\nnENuFrkOeRfQWpK+C+wJvA0szrcas9x0AnoBk0MIn61uwXYXcpKA35Z3EWZVYjhw++oWaI8hfxvg\nzHMvor5n75xLqV2D+m+Rdwk17fXXZzDi8P+CNA+r0x5Dvhigvmdv+my5Td611KwBA7bNuwRLrPGU\n1R1vZpFzyM0i55CbRc4hN4ucQ24WOYfcLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4hN4ucQ24WOYfc\nLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4hN4ucQ24WOYfcLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4h\nN4ucQ24WOYfcLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4hN4ucQ24WOYfcLHIOuVnkqibkkk6UNEfS\nIknPSfpR3jWV09233cQ+u/bjhrGX5l1KTZkyZQr77zeUHvWb0rHDWjwwaVLeJZVdVYRc0iHA5cBo\nYHtgGjBZ0ga5FlYmb8x4jYcfuJfem/fNu5Sa09jYSL/+/Rk79jok5V1ORXTIu4DUacBvQwi/A5D0\nM2Bv4GjgkjwLy9qihQu57MKzGXnWL7ljwvV5l1NzhgwZwpAhQwAIIeRcTWXkvieX1BHYAXi8qS0k\n7/5jwKC86iqXX195ITvu1EC/AQPzLsVqRO4hBzYA6oCPi9o/BjaufDnl85fH/8TsWTM58viReZdi\nNaRaDtej9+knH3H92Eu48PLr6dChY97lWA2phpB/CiwDNipq3wj4aFUr3TD2Ujp36bpSW8Pg/2DX\nPfbKvMAsvPnGDBZ8MY+Rxx1C06ng8uXL+PvfXuTBP9zBxMderJmOIKus3EMeQlgi6UVgMDAJQMlf\n+2Dg6lWtd9xJo+iz5TaVKTID2+8wkGvH/36ltisuOo/6nr058LBjHHArm9xDnhoD3JyG/QWS3vbO\nwM15FpWlTt/pTI9em6/Utk6n77DueuvTo9dmOVVVexobG5k1a9aKnvXZc2Yzbdo0unfvTn19fc7V\nlUdVhDyEcHf6mfj5JIfprwB7hhDm5ltZeXnnXXlTp05lj8G7IQlJjDrzDABGjDiCG28al3N15VEV\nIQcIIVwHXJd3HZV00ZU35V1CzWloaGDJ0uV5l1FR1fARmpmVkUNuFjmH3CxyDrlZ5Bxys8g55GaR\nc8jNIueQm0XOITeLnENuFjmH3CxyDrlZ5Fp0gYqkoS3dYAgh/jFuzdqRll6FNrGFywWS8drMrEq0\nKOQhBB/Wm7VTbQqvpE5ZFWJm5dHqkEuqk3SepA+AryRtlrZfIOmYzCs0szYpZU9+DnAkcBbwTUH7\na8CxGdRkZhkqJeQjgONDCLeRDKXcZBqwVSZVmVlmSgn5psCsVWzLdw0wqzKlhHw6sEsz7QcCL7et\nHDPLWimjtZ4PTJC0Kck/if+U1JfkMH6fLIszs7Zr9Z48hHA/sC+wB9BIEvqtgX1DCI9mW56ZtVVJ\n466HEKYA/55xLWZWBiXfXEHSD0n24ADTQwgvZlOSmWWp1SGX9H3gDuDHwBdp8/qS/gocGkJ4P8P6\nzKyNSuldv5Hko7KtQwjdQwjdSfboa6XzzKyKlHK43gDsFEKY2dQQQpgp6WRgSmaVmVkmStmTv0fz\nX3qpAz5sWzlmlrVSQj4KuCbteANWdMJdBZyZVWFmlo2Wjgwzj2RAiCZdgOclLS3YzlJgHC0fYMLM\nKqCl5+SnlrUKMyublo4MM6HchZhZeZT8ZRhYMTLM2oVtIYQFbarIzDJVysgwXSSNlfQJyXfX5xU9\nzKyKlNK7fgmwO/Bz4GuS0WBGk3x8NiK70swsC6Ucru8LjAgh/FnSeGBKCGGWpHeA4cBtmVZoZm1S\nyp68OzA7fb4gnQZ4GvhJFkWZWXZKCflsoHf6/HXg4PT5vvzzghUzqxKlhHw80C99fjFwoqTFwBXA\npVkVZmbZaPU5eQjhioLnj0naCtgBmBVC+FuWxZlZ27Xpc3KAEMI7wDsZ1GJmZdDS766f0tINhhCu\nLr0cM8taS/fkp7VwuQBUJOSD+m/BgAHbVuKlrBkff7k47xJq2ueNX7d42ZZ+d733mpcys2rkWxKb\nRc4hN4ucQ24WOYfcLHIOuVnkSgq5pF0k3Srp2fTGh0g6XNLO2ZZnZm1VyqARPwUmA4uA7YF10lnd\ngF9kV5qZZaGUPfm5wM9CCMcBSwranwEGZFKVmWWmlJD3BZ5qpn0+sH7byjGzrJUS8o+APs2078w/\nB5MwsypRSshvAK6SNJDku+rfkzQcuAz4dZbFmVnblXKp6cUk/xweBzqTHLp/DVwWQrgmw9rMLAOl\nDBoRgAslXUpy2N4VmB5C+Crr4sys7UoeNCKE8A0wPcNazKwMWh1ySU+y8s0PVxJC2L1NFZlZpkrZ\nk79SNN0R6A/8APA908yqTCnn5M2OEiPplyTn52ZWRbK8QOVW4OgMt2dmGcgy5IMAD/xlVmVK6Xi7\nr7gJ2AT4IXBBFkWZWXZK6XibXzS9HJgJ/G8I4ZG2l2RmWWpVyCXVkdwm6dUQgu9FbtYOtOqcPISw\nDHgEX21m1m6U0vH2GrBZ1oWYWXmUOmjEZZL2kbSJpPUKH1kXaGZtU0rH20Ppz0ms/PVWpdN1bS3K\nzLJTSsh3y7wKMyubUkI+B3gvveR0BUkC6jOpyswyU8o5+RzgX5tp757OM7MqUkrIm869i3XFX2s1\nqzotPlyXNCZ9GoALJC0smF0HDOTbl6GaWc5ac06+ffpTwLbANwXzvgGmkQzmaGZVpMUhDyHsBiBp\nPDAyhLCgbFWZWWZKGTTiqHIUYmbl4buamkXOITeLnENuFjmH3CxyDrlZ5Bxys8g55GaRq4qQS9pF\n0iRJH0haLmlo3jWVw5QpU9h/v6H0qN+Ujh3W4oFJk/IuqaZcM+ZS9t59Z/rWb0i/LXpyzPCDeWvW\nm3mXVXZVEXKgC8n33k9gNfdZa+8aGxvp178/Y8deR3JlrlXSC88+w1HHn8CDjz3FnRP/yJKlSzns\ngH1YtGhR3qWVVcl3Nc1SCOFh4GFYcV16lIYMGcKQIUMAKLoc3yrglnsmrjR95XXXs12fHrz6ykvs\nOOjHOVVVftWyJzeruPnz5yOJ9f+le96llJVDbjUphMDos0ex47/txJZbbZ13OWVVFYfrZpV29hkj\nefP1GUyc/ETepZRduw35GaefRrdu3VZqO/TQYRw6bFhOFVl7cc6oU3nikYf5w58eY6ONN8m7nLJr\ntyG/fMwVDBgwIO8yrJ05Z9SpTH7oQX7/x0fZtL5H3uVURFWEXFIXoA/JqDMAm0nqB3weQngvv8qy\n1djYyKxZs1b0rM+eM5tp06bRvXt36us90G25nX3GSO6/927G33EPnTt3Zu4nHwOw7nrd6NSpU87V\nlU9VhJzktsdPknxGHoDL0/YJwNF5FZW1qVOnssfg3ZCEJEadeQYAI0YcwY03jcu5uvjdMu4GJHHg\nPnuu1D7m2us5aNjwnKoqv6oIeQjhL9RAT39DQwNLli7Pu4ya9f68hWteKELRB8us1jnkZpFzyM0i\n55CbRc4hN4ucQ24WOYfcLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4hN4ucQ24WOYfcLHIOuVnkHHKz\nyDnkZpFzyM0i55CbRc4hN4ucQ24WOYfcLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4hN4ucQ24WOYfc\nLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4hN4ucQ24WOYc8J3fecUfeJdS8iffelXcJFeGQ5+TOOx3y\nvE289568S6gIh9wscg65WeQccrPIdci7gBJ0Anhj5gzq2vG/qAUL5jPtlZfyLqOmfb3wSz56a3re\nZZTk0/dnNz3ttKZlFUIobzUZk3QYcFvedZhVieEhhNtXt0B7DPl3gT2Bt4HF+VZjlptOQC9gcgjh\ns9Ut2O5Cbmat047Pas2sJRxys8g55GaRc8grTNKJkuZIWiTpOUk/yrumWiJpF0mTJH0gabmkoXnX\nVG4OeQVJOgS4HBgNbA9MAyZL2iDXwmpLF+AV4ASgJnqd3bteQZKeA54PIYxMpwW8B1wdQrgk1+Jq\nkKTlwP4hhEl511JO3pNXiKSOwA7A401tIfkP+xgwKK+6LH4OeeVsANQBHxe1fwxsXPlyrFY45GaR\nc8gr51NgGbBRUftGwEeVL8dqhUNeISGEJcCLwOCmtrTjbTDw17zqsvi1x0tN27MxwM2SXgReAE4D\nOgM351lULZHUBegDKG3aTFI/4PMQwnv5VVY+/gitwiSdAJxFcpj+CnByCGFqvlXVDkkNwJN8+zPy\nCSGEo3MoqewccrPI+ZzcLHIOuVnkHHKzyDnkZpFzyM0i55CbRc4hN4ucQ24WOYfcLHIOecQkjZd0\nX8H0k5LG5FBHQzqe2nqrWaZV461JGi3p5TbW1TN93e3asp1q5wtUassBwJKWLFjwHe/1QwgLMnjt\ncnx/OottRv+9boe8yknqmF6m2mYhhC9a89IkAdCaFmznYv/9fLheSenh8jXp4wtJcyWdX7TMHEnn\nSpogaT7w27T9+5LukjRP0meSJkrqWbDeWpLGpPPnSvoVRX/AxYfrktaW9CtJ70paLOkNSUel230i\nXWyepGWSxqXrSNLZkmZLWijpZUk/LXqdvSTNTOc/TnLPrta+Vxen22iU9Jak8yXVNbPc8Wn9jen7\ns27R/GMlTU+HwJ4u6eetraXdCyH4UaEHyeHvfJLryrcAhgFfAccULDMHmEdyrXnv9NEB+DtwPbAN\n0Be4BZgBdEjXO4tk9Jn90vk3pK91X9HrjymYvovkxpFDSYL4E+Agkn8OB5CMZLM5sCGwbrrOOWkt\ne6TrjAAWAruk8+uBRcAlBb/jP9Jtrbea92Y5MLRg+hfAQKAHsDfwIXBmwfzRwJfAo8C2wM7AG8At\nBcsMB95P35OewP7AXODwdH7P9HW3y/tvo6x/d3kXUEuPNGSvFbVdVNiWhvzeomWGA9OL2tYGGoE9\n0ukPgNML5tcB764q5MCW6R/4bquotaE4mOlrfgUMLFr2BuDW9Pn/Aa828zu2KuTNzD8DeKFgejTw\nDbBxQduewFJgw3T6TeCQou2cAzyTPq+JkPucvPKeK5p+FjhdkkL6l0cyTFShfsAWkr4sal8H2FzS\nC8AmJKPNABBCWCZpdYNR9CMJxFOtqL0PyUg2j6ZDVzXpCLyUPt8KeL5ovWdb8RrAihtRnExyJNGV\n5GhmftFi74YQCsfHe5bkFLSvpK/SdW+SdGPBMnVAa/om2j2HvDo1Fk13BaYCh/HtjqK5zbS1xKIS\n1uma/tyL5PC50NclbK9ZkgYBtwLnAY+QhHsYcHorNtNU67EU/PNLLWtrje2JQ155A4umBwFvFuzF\nm/MScDAwN4TwVXMLSPpHuu2n0+k6kps5FB8VNHmVZK/XwD872Qp9k/4s7OyaThLmniGEp1ex3RnA\nvkVtrb15xCDg7RDCxU0Nkno1s1wPSRsX7M0HkQT49RDCXEkfApuHEO5czWtF/xGae9crr4ekyyRt\nKWkYcBJw5RrWuY2kU+1+STtL6iVpV0lXSfpeusxVwP9I2k9SX+A6YP1VbTCE8A7wO2Bcuk6v9Esr\nB6WLvEMSgH0lbSCpS/oP5jLgCkkjJG0maXtJJ0k6PF3vNySnFpekv+NhwBGtfI/eTN+nQ9LXOIWk\n06zY18AESdtJ2iV9D+4KIcxN548GzpZ0sqQtJP1A0pGSTi3YRvQfoeXeKVBLD5KOr2uAa0nOCz8F\nzi9aZjZwSjPrbgiMJ7njykKSIPwG6JrOryPptZ8HfAZcmi5f2PH2BCv3rq9NEtr3SQ7fZwJHFMw/\nh+SwfCkwrqD9ZJK9+mKSMeMfAnYumL9Xuq2FwJ9JQr6mjrdlrNy7fjHwCcmh+u3AKSQjqjbNH01y\nhPPfaf2NwJ1At6LtHpoutyh9v58E9kvn9UxfN+qONw/kWEGSngReDiG05tzSrE18uG4WOYe8snzY\nZBXnw3WzyHlPbhY5h9wscg65WeQccrPIOeRmkXPIzSLnkJtFziE3i5xDbha5/wc7H1Q1d1OMswAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10eb8d6d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.plotting import plot_confusion_matrix\n",
    "\n",
    "fig, ax = plot_confusion_matrix(conf_mat=cm)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## confusion_matrix\n",
      "\n",
      "*confusion_matrix(y_target, y_predicted, binary=False, positive_label=1)*\n",
      "\n",
      "Compute a confusion matrix/contingency table.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `y_target` : array-like, shape=[n_samples]\n",
      "\n",
      "    True class labels.\n",
      "\n",
      "- `y_predicted` : array-like, shape=[n_samples]\n",
      "\n",
      "    Predicted class labels.\n",
      "\n",
      "- `binary` : bool (default: False)\n",
      "\n",
      "    Maps a multi-class problem onto a\n",
      "    binary confusion matrix, where\n",
      "    the positive class is 1 and\n",
      "    all other classes are 0.\n",
      "\n",
      "- `positive_label` : int (default: 1)\n",
      "\n",
      "    Class label of the positive class.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `mat` : array-like, shape=[n_classes, n_classes]\n",
      "\n",
      "\n",
      "**Examples**\n",
      "\n",
      "For usage examples, please see\n",
      "    [http://rasbt.github.io/mlxtend/user_guide/evaluate/confusion_matrix/](http://rasbt.github.io/mlxtend/user_guide/evaluate/confusion_matrix/)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.evaluate/confusion_matrix.md', 'r') as f:\n",
    "    s = f.read() \n",
    "print(s)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}