{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "이 노트북의 코드에 대한 설명은 [분류기 체인: ClassifierChain](https://tensorflow.blog/2018/02/16/%EB%B6%84%EB%A5%98%EA%B8%B0-%EC%B2%B4%EC%9D%B8-classifierchain/) 글을 참고하세요." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from sklearn.datasets import fetch_openml" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "fetch yeast dataset (https://www.openml.org/d/40597)\n", "\n", "사이킷런 0.20 버전에서 `fetch_openml`이 추가되었습니다. `fetch_mldata`는 0.22 버전에서 삭제될 예정입니다." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "yeast = fetch_openml('yeast', version=4)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "X = yeast['data']\n", "Y = yeast['target']" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((2417, 103), (2417, 14))" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X.shape, Y.shape" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([['FALSE', 'FALSE', 'FALSE', ..., 'TRUE', 'TRUE', 'FALSE'],\n", " ['FALSE', 'FALSE', 'TRUE', ..., 'FALSE', 'FALSE', 'FALSE'],\n", " ['FALSE', 'TRUE', 'TRUE', ..., 'TRUE', 'TRUE', 'FALSE'],\n", " ...,\n", " ['FALSE', 'FALSE', 'FALSE', ..., 'TRUE', 'TRUE', 'FALSE'],\n", " ['FALSE', 'FALSE', 'FALSE', ..., 'TRUE', 'TRUE', 'FALSE'],\n", " ['FALSE', 'TRUE', 'TRUE', ..., 'TRUE', 'TRUE', 'FALSE']],\n", " dtype=object)" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Y" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "Y = Y == 'TRUE'" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[False, False, False, ..., True, True, False],\n", " [False, False, True, ..., False, False, False],\n", " [False, True, True, ..., True, True, False],\n", " ...,\n", " [False, False, False, ..., True, True, False],\n", " [False, False, False, ..., True, True, False],\n", " [False, True, True, ..., True, True, False]])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Y" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=42)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "from sklearn.linear_model import LogisticRegression\n", "from sklearn.multiclass import OneVsRestClassifier" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "for OVR model" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "ovr = OneVsRestClassifier(LogisticRegression(solver='liblinear'))\n", "ovr.fit(X_train, Y_train)\n", "pred_ovr = ovr.predict(X_test)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.5081742752197298" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.metrics import jaccard_similarity_score\n", "ovr_score = jaccard_similarity_score(Y_test, pred_ovr)\n", "ovr_score" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "for CC model (https://www.cs.waikato.ac.nz/ml/publications/2009/chains.pdf)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "from sklearn.multioutput import ClassifierChain" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.5118178538633084" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cc = ClassifierChain(LogisticRegression(solver='liblinear'), order='random', random_state=42)\n", "cc.fit(X_train, Y_train)\n", "pred_cc = cc.predict(X_test)\n", "cc_score = jaccard_similarity_score(Y_test, pred_cc)\n", "cc_score" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "for CC ensemble" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "chains = [ClassifierChain(LogisticRegression(solver='liblinear'), order='random', random_state=42+i)\n", " for i in range(10)]\n", "for chain in chains:\n", " chain.fit(X_train, Y_train)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0.5118178538633084,\n", " 0.5261753455451803,\n", " 0.5298506016481224,\n", " 0.5150574517310055,\n", " 0.5076409758744469,\n", " 0.49889807162534433,\n", " 0.5104657108272811,\n", " 0.49708858061130784,\n", " 0.5161265846183202,\n", " 0.4871433490751672]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pred_chains = np.array([chain.predict(X_test) for chain in chains])\n", "chain_scores = [jaccard_similarity_score(Y_test, pred_chain)\n", " for pred_chain in pred_chains]\n", "chain_scores" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAADXBJREFUeJzt3H+s3Xddx/Hni9aq/PDnrgbbjlZt0AZ/zFwrSoJkzNgF05IISZdAwMw0JJRNIdGiZibzH34YkMTGUMeUKFDmJPGK1WkAY/yDpXdjAbq6UOtcL0N3+eEwGhgNb/+4B3K8u+v93vacnt73no9/dr7f89n3vE/WPPft95zvSVUhSerlGbMeQJI0ecZdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDW2f1wtdcc03t2rVrVi8vSZvSfffd9/mqmltv3czivmvXLhYXF2f18pK0KSX59yHrvCwjSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDc3sDlVdXXYd/ZupHPfht7xsKseVdHHG/SplbCVdDi/LSFJDxl2SGjLuktSQ19w1E9P6TAH8XKErP4faGM/cJakhz9wH8CxTm4Fntho3KO5J9gPvArYAd1TVW1Y9/1rg7cBnR7v+sKrumOCc/4+x1Ub5Z0ZPN+vGPckW4BjwC8AScCrJQlU9uGrpB6vqyBRmlCRt0JBr7vuAs1V1rqqeAE4AB6c7liTpcgyJ+3bg/Nj20mjfar+c5JNJ7k6yc60DJTmcZDHJ4vLy8iWMK0kaYsg196yxr1Zt/zXwgar6apLXAe8Frn/Sv1R1HDgOMD8/v/oYUhte49esDYn7EjB+Jr4DeHR8QVV9YWzzj4G3Xv5okq5mfjvn6jbksswpYE+S3Um2AYeAhfEFSZ47tnkAODO5ESVJG7XumXtVXUhyBLiHla9C3llVp5PcDixW1QJwS5IDwAXgi8BrpzizJGkdg77nXlUngZOr9t029vjNwJsnO5okzc5m/9zEnx+QpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDg+KeZH+Sh5KcTXL0IutekaSSzE9uREnSRq0b9yRbgGPAjcBe4KYke9dY9xzgFuDeSQ8pSdqYIWfu+4CzVXWuqp4ATgAH11j3e8DbgK9McD5J0iUYEvftwPmx7aXRvm9Kch2ws6o+fLEDJTmcZDHJ4vLy8oaHlSQNMyTuWWNfffPJ5BnAO4E3rXegqjpeVfNVNT83Nzd8SknShgyJ+xKwc2x7B/Do2PZzgBcA/5jkYeCFwIIfqkrS7AyJ+ylgT5LdSbYBh4CFbzxZVY9X1TVVtauqdgEfBw5U1eJUJpYkrWvduFfVBeAIcA9wBrirqk4nuT3JgWkPKEnauK1DFlXVSeDkqn23PcXal1z+WJKky+EdqpLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJamhQ3JPsT/JQkrNJjq7x/OuSfCrJA0n+OcneyY8qSRpq3bgn2QIcA24E9gI3rRHv91fVj1XVTwJvA94x8UklSYMNOXPfB5ytqnNV9QRwAjg4vqCqvjy2+SygJjeiJGmjtg5Ysx04P7a9BPzM6kVJXg+8EdgGXD+R6SRJl2TImXvW2PekM/OqOlZVPwT8JvA7ax4oOZxkMcni8vLyxiaVJA02JO5LwM6x7R3AoxdZfwJ4+VpPVNXxqpqvqvm5ubnhU0qSNmRI3E8Be5LsTrINOAQsjC9Ismds82XAZyY3oiRpo9a95l5VF5IcAe4BtgB3VtXpJLcDi1W1ABxJcgPwNeBLwGumObQk6eKGfKBKVZ0ETq7ad9vY41snPJck6TJ4h6okNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8ZdkhoaFPck+5M8lORskqNrPP/GJA8m+WSSjyR53uRHlSQNtW7ck2wBjgE3AnuBm5LsXbXsE8B8Vf04cDfwtkkPKkkabsiZ+z7gbFWdq6ongBPAwfEFVfWxqvrf0ebHgR2THVOStBFD4r4dOD+2vTTa91RuBv72coaSJF2erQPWZI19tebC5FXAPPDzT/H8YeAwwLXXXjtwREnSRg05c18Cdo5t7wAeXb0oyQ3AbwMHquqrax2oqo5X1XxVzc/NzV3KvJKkAYbE/RSwJ8nuJNuAQ8DC+IIk1wHvZiXsj01+TEnSRqwb96q6ABwB7gHOAHdV1ekktyc5MFr2duDZwF8keSDJwlMcTpJ0BQy55k5VnQROrtp329jjGyY8lyTpMniHqiQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGhoU9yT7kzyU5GySo2s8/+Ik9ye5kOQVkx9TkrQR68Y9yRbgGHAjsBe4KcneVcseAV4LvH/SA0qSNm7rgDX7gLNVdQ4gyQngIPDgNxZU1cOj574+hRklSRs05LLMduD82PbSaJ8k6So1JO5ZY19dyoslOZxkMcni8vLypRxCkjTAkLgvATvHtncAj17Ki1XV8aqar6r5ubm5SzmEJGmAIXE/BexJsjvJNuAQsDDdsSRJl2PduFfVBeAIcA9wBrirqk4nuT3JAYAkP51kCXgl8O4kp6c5tCTp4oZ8W4aqOgmcXLXvtrHHp1i5XCNJugp4h6okNWTcJakh4y5JDRl3SWrIuEtSQ8Zdkhoy7pLUkHGXpIaMuyQ1ZNwlqSHjLkkNGXdJasi4S1JDxl2SGjLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkPGXZIaMu6S1JBxl6SGjLskNWTcJakh4y5JDRl3SWrIuEtSQ8ZdkhoaFPck+5M8lORskqNrPP+tST44ev7eJLsmPagkabh1455kC3AMuBHYC9yUZO+qZTcDX6qqHwbeCbx10oNKkoYbcua+DzhbVeeq6gngBHBw1ZqDwHtHj+8GXpokkxtTkrQRQ+K+HTg/tr002rfmmqq6ADwOfO8kBpQkbVyq6uILklcCv1hVvzrafjWwr6reMLbm9GjN0mj7X0drvrDqWIeBw6PN5wMPTeqNrOMa4PNX6LVmwfe3+XV/j76/yXleVc2tt2jrgAMtATvHtncAjz7FmqUkW4HvBL64+kBVdRw4PuA1JyrJYlXNX+nXvVJ8f5tf9/fo+7vyhlyWOQXsSbI7yTbgELCwas0C8JrR41cAH631/kogSZqadc/cq+pCkiPAPcAW4M6qOp3kdmCxqhaA9wB/luQsK2fsh6Y5tCTp4oZclqGqTgInV+27bezxV4BXTna0ibril4KuMN/f5tf9Pfr+rrB1P1CVJG0+/vyAJDXUOu7r/WzCZpZkZ5KPJTmT5HSSW2c907Qk2ZLkE0k+POtZJi3JdyW5O8m/jP5b/uysZ5q0JL8++jP66SQfSPJts57pciS5M8ljST49tu97kvxDks+M/vnds5wRGsd94M8mbGYXgDdV1Y8CLwRe3+z9jbsVODPrIabkXcDfVdWPAD9Bs/eZZDtwCzBfVS9g5UsZm/0LF38K7F+17yjwkaraA3xktD1TbePOsJ9N2LSq6nNVdf/o8X+zEoXVdw5vekl2AC8D7pj1LJOW5DuAF7PybTOq6omq+q/ZTjUVW4FvH90D80yefJ/MplJV/8ST7+MZ/wmW9wIvv6JDraFz3If8bEILo1/hvA64d7aTTMUfAL8BfH3Wg0zBDwLLwJ+MLjvdkeRZsx5qkqrqs8DvA48AnwMer6q/n+1UU/H9VfU5WDnxAr5vxvO0jvtaP1zW7qtBSZ4N/CXwa1X15VnPM0lJfgl4rKrum/UsU7IV+Cngj6rqOuB/uAr+Oj9Jo2vPB4HdwA8Az0ryqtlO9fTQOe5DfjZhU0vyLayE/X1V9aFZzzMFLwIOJHmYlctq1yf589mONFFLwFJVfeNvXHezEvtObgD+raqWq+prwIeAn5vxTNPwn0meCzD652Mznqd13If8bMKmNfpJ5fcAZ6rqHbOeZxqq6s1VtaOqdrHy3++jVdXmrK+q/gM4n+T5o10vBR6c4UjT8AjwwiTPHP2ZfSnNPjQeGf8JltcAfzXDWYCBd6huRk/1swkzHmuSXgS8GvhUkgdG+35rdDexNo83AO8bnYCcA35lxvNMVFXdm+Ru4H5WvuH1Ca7Cuzk3IskHgJcA1yRZAn4XeAtwV5KbWfkf2szv2PcOVUlqqPNlGUl62jLuktSQcZekhoy7JDVk3CWpIeMuSQ0Zd0lqyLhLUkP/B7VkSzMSiLyuAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.bar(np.arange(11), [ovr_score]+chain_scores)\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.5180391578118851" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "proba_chains = np.array([chain.predict_proba(X_test) for chain in chains])\n", "proba_ensemble = proba_chains.mean(axis=0)\n", "ensemble_score = jaccard_similarity_score(Y_test, proba_ensemble >= 0.5)\n", "ensemble_score" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAADfpJREFUeJzt3H+s3Xddx/Hny9aqwBR11WBbbNVm2iAyc60oCRoYsRPSkghJFyEjYhoSCigk0gWzP2Zi+GFAEhqljglRoMyJsWJxEMAYEyG9AzLo6qTOSS9DV34IRgKj4e0f92wc7257v/eec3t63z4fydLz/Z7PPff9Te+e+fZ7z/mmqpAk9fUdsx5AkrS+DL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOY2z+obX3311bVz585ZfXtJ2pDuuuuuL1TV1tV8zcxCv3PnTubn52f17SVpQ0ry76v9Gi/dSFJzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnMz+2SsLr+dR/52qq93/2ufPdXXk7Q+DP0VYNoBBiMs6du8dCNJzRl6SWrOSzeSdAkdLq0aek1Vh/8pNBl/Bq48XrqRpOY8o78Ez0x0ufjWV62nQaFPsg94M7AJuLWqXrvk+RcBbwA+N9r1lqq6dYpz/h8GWP4MSMOtGPokm4CjwLOABeBUkhNVdc+Spe+pqsPrMKMkaQJDrtHvBc5W1X1V9RBwHDiwvmNJkqZlSOi3AefGthdG+5b6tSR3J7kjyY7lXijJoSTzSebPnz+/hnElSas15Bp9ltlXS7b/Bnh3VX0jyUuAdwDPeNQXVR0DjgHMzc0tfQ3piuPvAtTBkNAvAONn6NuBB8YXVNUXxzb/BHjd5KNJmjbf3fP/05BLN6eA3Ul2JdkCHAROjC9I8oSxzf3AmemNKEmaxIpn9FV1Iclh4E4W3155W1WdTnILMF9VJ4CXJ9kPXAC+BLxoHWeWJK3CoPfRV9VJ4OSSfTePPb4JuGm6o0nSxfn7k+G8BYIkNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4ZekpobFPok+5Lcm+RskiOXWPe8JJVkbnojSpImsWLok2wCjgLXA3uAG5LsWWbdVcDLgY9Ne0hJ0toNOaPfC5ytqvuq6iHgOHBgmXW/B7we+PoU55MkTWhI6LcB58a2F0b7HpHkWmBHVb3vUi+U5FCS+STz58+fX/WwkqTVGxL6LLOvHnky+Q7gTcCrVnqhqjpWVXNVNbd169bhU0qS1mxI6BeAHWPb24EHxravAp4E/H2S+4GnAif8hawkXRmGhP4UsDvJriRbgIPAiYefrKqvVNXVVbWzqnYCHwX2V9X8ukwsSVqVFUNfVReAw8CdwBng9qo6neSWJPvXe0BJ0mQ2D1lUVSeBk0v23XyRtb88+ViSpGnxk7GS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWpuUOiT7Etyb5KzSY4s8/xLknwqySeT/GOSPdMfVZK0FiuGPskm4ChwPbAHuGGZkL+rqn66qp4CvB5449QnlSStyZAz+r3A2aq6r6oeAo4DB8YXVNVXxzYfC9T0RpQkTWLzgDXbgHNj2wvAzy9dlOSlwCuBLcAzpjKdJGliQ87os8y+R52xV9XRqvpx4NXA7y77QsmhJPNJ5s+fP7+6SSVJazIk9AvAjrHt7cADl1h/HHjuck9U1bGqmququa1btw6fUpK0ZkNCfwrYnWRXki3AQeDE+IIku8c2nw18ZnojSpImseI1+qq6kOQwcCewCbitqk4nuQWYr6oTwOEk1wHfBL4M3LieQ0uShhvyy1iq6iRwcsm+m8cev2LKc0mSpsRPxkpSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqblBoU+yL8m9Sc4mObLM869Mck+Su5N8KMmPTn9USdJarBj6JJuAo8D1wB7ghiR7liz7BDBXVU8G7gBeP+1BJUlrM+SMfi9wtqruq6qHgOPAgfEFVfWRqvraaPOjwPbpjilJWqshod8GnBvbXhjtu5gXA++fZChJ0vRsHrAmy+yrZRcmLwDmgF+6yPOHgEMAT3ziEweOKEmaxJAz+gVgx9j2duCBpYuSXAe8BthfVd9Y7oWq6lhVzVXV3NatW9cyryRplYaE/hSwO8muJFuAg8CJ8QVJrgXeymLkH5z+mJKktVox9FV1ATgM3AmcAW6vqtNJbkmyf7TsDcDjgL9I8skkJy7ycpKky2zINXqq6iRwcsm+m8ceXzfluSRJU+InYyWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Nyg0CfZl+TeJGeTHFnm+acn+XiSC0meN/0xJUlrtWLok2wCjgLXA3uAG5LsWbLss8CLgHdNe0BJ0mQ2D1izFzhbVfcBJDkOHADueXhBVd0/eu5b6zCjJGkCQy7dbAPOjW0vjPZJkjaAIaHPMvtqLd8syaEk80nmz58/v5aXkCSt0pDQLwA7xra3Aw+s5ZtV1bGqmququa1bt67lJSRJqzQk9KeA3Ul2JdkCHAROrO9YkqRpWTH0VXUBOAzcCZwBbq+q00luSbIfIMnPJVkAng+8Ncnp9RxakjTckHfdUFUngZNL9t089vgUi5d0JElXGD8ZK0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDVn6CWpOUMvSc0ZeklqztBLUnOGXpKaM/SS1Jyhl6TmDL0kNWfoJak5Qy9JzRl6SWrO0EtSc4Zekpoz9JLUnKGXpOYMvSQ1Z+glqTlDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gaFPsm+JPcmOZvkyDLPf1eS94ye/1iSndMeVJK0NiuGPskm4ChwPbAHuCHJniXLXgx8uap+AngT8LppDypJWpshZ/R7gbNVdV9VPQQcBw4sWXMAeMfo8R3AM5NkemNKktZqSOi3AefGthdG+5ZdU1UXgK8APziNASVJk0lVXXpB8nzgV6rqN0fbLwT2VtXLxtacHq1ZGG3/62jNF5e81iHg0GjzGuDeaR3IJVwNfOEyfJ/LodOxQK/j6XQs4PFcya6pqqtW8wWbB6xZAHaMbW8HHrjImoUkm4HvA7609IWq6hhwbDUDTirJfFXNXc7vuV46HQv0Op5OxwIez5Usyfxqv2bIpZtTwO4ku5JsAQ4CJ5asOQHcOHr8PODDtdI/FSRJl8WKZ/RVdSHJYeBOYBNwW1WdTnILMF9VJ4C3AX+W5CyLZ/IH13NoSdJwQy7dUFUngZNL9t089vjrwPOnO9rUXNZLReus07FAr+PpdCzg8VzJVn0sK/4yVpK0sXkLBElqrm3oV7ptw0aSZEeSjyQ5k+R0klfMeqZJJdmU5BNJ3jfrWSaV5PFJ7kjyz6O/o1+Y9UxrleS3Rz9jn07y7iTfPeuZViPJbUkeTPLpsX0/kOSDST4z+vP7ZznjalzkeN4w+lm7O8lfJXn8Sq/TMvQDb9uwkVwAXlVVPwU8FXjpBj8egFcAZ2Y9xJS8Gfi7qvpJ4GfYoMeVZBvwcmCuqp7E4psvNtobK94O7Fuy7wjwoaraDXxotL1RvJ1HH88HgSdV1ZOBfwFuWulFWoaeYbdt2DCq6vNV9fHR4/9mMSRLP528YSTZDjwbuHXWs0wqyfcCT2fxnWdU1UNV9V+znWoim4HvGX0e5jE8+jMzV7Sq+gce/Rme8Vu0vAN47mUdagLLHU9VfWB0BwKAj7L42aZL6hr6Ibdt2JBGdwa9FvjYbCeZyB8CvwN8a9aDTMGPAeeBPx1diro1yWNnPdRaVNXngD8APgt8HvhKVX1gtlNNxQ9X1edh8aQJ+KEZzzNNvwG8f6VFXUO/3A3VNvzbi5I8DvhL4Leq6quznmctkjwHeLCq7pr1LFOyGfhZ4I+q6lrgf9hYlwYeMbp2fQDYBfwI8NgkL5jtVLqYJK9h8bLuO1da2zX0Q27bsKEk+U4WI//OqnrvrOeZwNOA/UnuZ/GS2jOS/PlsR5rIArBQVQ//C+sOFsO/EV0H/FtVna+qbwLvBX5xxjNNw38meQLA6M8HZzzPxJLcCDwH+PUhdyHoGvoht23YMEa3fH4bcKaq3jjreSZRVTdV1faq2sni38uHq2rDnjVW1X8A55JcM9r1TOCeGY40ic8CT03ymNHP3DPZoL9YXmL8Fi03An89w1kmlmQf8Gpgf1V9bcjXtAz96BcVD9+24Qxwe1Wdnu1UE3ka8EIWz34/OfrvV2c9lB7xMuCdSe4GngL8/oznWZPRv0ruAD4OfIrFPmyoT5QmeTfwT8A1SRaSvBh4LfCsJJ8BnjXa3hAucjxvAa4CPjhqwR+v+Dp+MlaSemt5Ri9J+jZDL0nNGXpJas7QS1Jzhl6SmjP0ktScoZek5gy9JDX3v9nEeDRmp8MGAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.bar(np.arange(12), [ovr_score]+chain_scores+[ensemble_score])\n", "plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.6" } }, "nbformat": 4, "nbformat_minor": 2 }