{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Clustering in Real World\n", "> A Summary of lecture \"Cluster Analysis in Python\", via datacamp\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- author: Chanseok Kang\n", "- categories: [Python, Datacamp, Machine_Learning]\n", "- image: images/batman.jpg" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Dominant colors in images\n", "- Dominant colors in images\n", " - All images consist of pixels\n", " - Each pixel has three values: R, G, B\n", " - Pixel Color: combination of these RGB values\n", " - Perform k-means on standardized RGB values to find cluster centers\n", " - Uses: Identifying features in statelite images" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Extract RGB values from image\n", "There are broadly three steps to find the dominant colors in an image:\n", "\n", "- Extract RGB values into three lists.\n", "- Perform k-means clustering on scaled RGB values.\n", "- Display the colors of cluster centers.\n", "![batman](dataset/batman.jpg)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(169, 269, 3)\n" ] } ], "source": [ "import matplotlib.image as img\n", "\n", "r = []\n", "g = []\n", "b = []\n", "\n", "# Read batman image and print dimensions\n", "batman_image = img.imread('./dataset/batman.jpg')\n", "print(batman_image.shape)\n", "\n", "# Store RGB values of all pixels in lists r, g, and b\n", "for row in batman_image:\n", " for temp_r, temp_g, temp_b in row: \n", " r.append(temp_r)\n", " g.append(temp_g)\n", " b.append(temp_b)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### How many dominant colors?\n", "Construct an elbow plot with the data frame. How many dominant colors are present?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Preprocess" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from scipy.cluster.vq import whiten\n", "\n", "batman_df = pd.DataFrame({'red':r, 'blue':b, 'green':g})\n", "batman_df['scaled_red'] = whiten(batman_df['red'])\n", "batman_df['scaled_blue'] = whiten(batman_df['blue'])\n", "batman_df['scaled_green'] = whiten(batman_df['green'])" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEHCAYAAACjh0HiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3Rc5bX+8e9WsSTbKi5yQZJ7ww0XhZaEDjbNxoQkOJRLLolxcikJJQESIIaEm8ANgRDqj1AChBJ6Nx0ntCC5VyC2seUqMJJsY8mWtH9/zMgIIUtjW0dnpHk+a82amTNnZvawkJ8575n9vubuiIhI4koKuwAREQmXgkBEJMEpCEREEpyCQEQkwSkIREQSXErYBeyu7t27e79+/cIuQ0SkTSkuLv7U3XMbe6zNBUG/fv0oKioKuwwRkTbFzD7Z1WMaGhIRSXAKAhGRBKcgEBFJcAoCEZEEpyAQEUlwCgIRkQSnIBARSXAJEwRlX2zn1jc/prZW026LiNSXMEHwxrKNXPfSMh76YFXYpYiIxJWECYKTxuRx0IBu/P7FpWzcXBl2OSIicSNhgsDM+N2UkVTtqOWa55aEXY6ISNxImCAAGJDbmf85fBDPzlvLm8s2hl2OiEhcSKggAJh+2AAG5nbiiqcXsm17TdjliIiELuGCIC0lmWunjGL1pm3c9NpHYZcjIhK6wILAzO42s41mtrCJfQ4zs7lmtsjM3gqqloYOGNCN7xXmc9c/l7N0fUVrva2ISFwK8ojgXmDirh40sxzgVmCSu48AvhtgLV9z2bH7kpWRymVPLFBvgYgktMCCwN1nAZua2OUHwBPuviq6f6ueve3SqQO/Pn5f5qwq4+//Vm+BiCSuMM8RDAG6mNmbZlZsZme2dgFTxuZx8MBu/OGlpWysUG+BiCSmMIMgBRgPHA9MAK4wsyGN7Whm08ysyMyKSktLW6yASG/BKKqqa5nx3OIWe10RkbYkzCAoAV5y963u/ikwC9ivsR3d/U53L3T3wtzcRtde3mP9u3fivMMH8fz8dbyxVL0FIpJ4wgyCp4Fvm1mKmXUEDgBCafmddugABvXozK+fWsgX26vDKEFEJDRB/nz0IeBdYKiZlZjZ2WY23cymA7j7EuAlYD7wb+Aud9/lT02DlJaSzO9OGsmaMvUWiEjiSQnqhd19agz7XA9cH1QNu+OAAd34fmEBd/1zBSeNyWPf3llhlyQi0ioSrrO4KZcdN4wc9RaISIJRENST07EDvz5hX+auLuPB9z8JuxwRkVahIGjgpDF5fGtQd657aRkb1FsgIglAQdCAmfHbk0ZSVVPL1c+qt0BE2j8FQSP6de/E+UcM4vkF63h96YawyxERCZSCYBemHTKQQT06c8VTi9RbICLtmoJgFzqkJHHtlFGR3oJX1VsgIu2XgqAJ+/fvyqnfKOCuf61g0drysMsREQmEgqAZlx47jC4dU7n8yYXUqLdARNohBUEzcjp24IoThjNPvQUi0k4pCGIwab99+PZg9RaISPukIIhBXW/BjppaZjy7KOxyRERalIIgRn27deL8IwfzwoL1vLZEvQUi0n4oCHbDj789gME9OnPl0+otEJH2Q0GwGzqkJHHtyZHegj+98mHY5YiItAgFwW76Rr+uTN2/D3e/vZKFa9RbICJtn4JgD1w6MdJb8KsnF6i3QETaPAXBHsjumBrpLSgp54H31FsgIm1bkGsW321mG82syXWIzewbZlZjZqcEVUsQ6noLrp+5jPXl6i0QkbYryCOCe4GJTe1gZsnAH4CZAdYRCPUWiEh7EVgQuPssYFMzu50HPA5sDKqOINX1Fry4cD2vLlZvgYi0TaGdIzCzPGAKcHtYNbSEaYcMYGjPTK58eiFbq9RbICJtT5gni28EfunuNc3taGbTzKzIzIpKS0tbobTYpSYnce3JI1lbXqneAhFpk8IMgkLgYTNbCZwC3GpmJzW2o7vf6e6F7l6Ym5vbmjXGZHzfrvzggD7c/fYK9RaISJsTWhC4e3937+fu/YDHgJ+6+1Nh1bO3fjlhGF07pXG5egtEpI0J8uejDwHvAkPNrMTMzjaz6WY2Paj3DFN2x1SuPHE480vKuf/dlWGXIyISs5SgXtjdp+7GvmcFVUdrOnF0bx4rLuH6mcuYMLIXvbMzwi5JRKRZ6ixuQWbGbyePpMad3zyj3gIRaRsUBC2sT7eOXHDkEGYu2sDLi9aHXY6ISLMUBAH40bf7M7RnJlc9s4gt6i0QkTinIAhApLdgFOvUWyAibYCCICDj+3bhtAP6cI96C0QkzikIAvSLiZHegsueWEB1TW3Y5YiINEpBEKDsjFSuOnE4C9aU87d3tW6BiMQnBUHAThjdm8OG5vLHl5extmxb2OWIiHyNgiBgZsY16i0QkTimIGgFBV078rOjhvDy4g3MVG+BiMQZBUErOftb/RnWK5PfqLdAROKMgqCVpCYn8bspo1hfUckNL6u3QETih4KgFdX1Ftz7zgoWlKi3QETig4KglV0yYRjdOqdx2ZPz1VsgInFBQdDKsjNS+c2JI1i4poL71FsgInFAQRCC40b14nD1FohInFAQhMDMuHrySGrduUq9BSISMgVBSAq6duTnRw3hFfUWiEjIglyz+G4z22hmC3fx+GlmNj96ecfM9guqlnj139HegqueXsTmyh1hlyMiCSrII4J7gYlNPL4CONTdRwPXAHcGWEtcSk1O4n9PHsWGzZX8Ub0FIhKSwILA3WcBm5p4/B13/zx69z0gP6ha4tnYPl0448C+3PfuSuatLgu7HBFJQPFyjuBs4MWwiwjLxROGkts5jcuf1LoFItL6Qg8CMzucSBD8sol9pplZkZkVlZaWtl5xrSQrPZXfTBrBorUV3PvOyrDLEZEEE2oQmNlo4C5gsrt/tqv93P1Ody9098Lc3NzWK7AVHTuyF0cM68ENr3zIGvUWiEgrCi0IzKwP8ARwhrsn/JlSM2PGpBG4w1VPL8Tdwy5JRBJEkD8ffQh4FxhqZiVmdraZTTez6dFdrgS6Abea2VwzKwqqlraioGtHfn70YF5dslG9BSLSaqytffMsLCz0oqL2mxnVNbWc+Je32bS1ilcvPJTM9NSwSxKRdsDMit29sLHHQj9ZLF+VEu0t2Li5Sr0FItIqFARxaExBDmeqt0BEWomCIE5dNGEoPTLTuOwJ9RaISLAUBHEqKz2ybsHidRXc8/bKsMsRkXZMQRDHJo7sxZHR3oKSz78IuxwRaacUBHHMzJgxeQQAVz69SL0FIhKImILAzDqZWVL09hAzm2Rm+l1jK8jv0pGLjhnC60s38tJC9RaISMuL9YhgFpBuZnnAa8APiUwzLa3grIP7Mbx3Flc9s4gKrVsgIi0s1iAwd/8COBm42d2nAMODK0vqq+stKN1SxR9nLgu7HBFpZ2IOAjM7CDgNeD66LSWYkqQx+xXk8F8H9eNv733CXPUWiEgLijUILgAuA55090VmNgB4I7iypDEXHTNkZ2/BDvUWiEgLiSkI3H2Wu09y9z9E7y939/ODLU0aykxPZcakESxZV8E9b68IuxwRaSdiGt4xsyHAxUC/+s9x9yOCKUt2ZcKIXhy1b0/+9MpHHDuyNwVdO4Zdkoi0cbEODf0DmAP8Grik3kVaWV1vgRlcqXULRKQFxBoE1e5+m7v/292L6y6BVia7lJeTwYVHD+GNZaW8qN4CEdlLsQbBs2b2UzPrbWZd6y6BViZNOuvgfozYJ4vfqLdARPZSrEHwX0SGgt4BiqOX9rs6TBtQ11vw6ZYqrn9JvQUisudiOlns7v2DLkR23+j8HM48qB/3vbuSKePyGNenS9gliUgbFOtcQ6lmdr6ZPRa9nNvcXENmdreZbTSzhbt43Mzsz2b2sZnNN7Nxe/IBEt3FE4bSMzOdy9VbICJ7KNahoduA8cCt0cv46Lam3AtMbOLxY4HB0cu0GF5PGtE5LYUZk0ewdP1m7v6XegtEZPfFOk3EN9x9v3r3XzezeU09wd1nmVm/JnaZDPzNI79/fM/Mcsyst7uvi7EmiZowohdHD+/Jn179kONGqbdARHZPrEcENWY2sO5OdIqJmr187zxgdb37JdFtsgdmTBpBkpl6C0Rkt8UaBJcAb5jZm2b2FvA6cNFevrc1sq3Rf8HMbJqZFZlZUWlp6V6+bfu0T04GFx0zlDeWlfL8Ah1UiUjsYp1r6DUiY/nnRy9D3X1vJ50rAQrq3c8H1u7i/e9090J3L8zNzd3Lt22//uugvozMy2LGs4sp36beAhGJTZNBYGZHRK9PBo4HBgEDgeOj2/bGM8CZ0V8PHQiU6/zA3klJTuJ/p4zmsy1VXD9zadjliEgb0dzJ4kOJDAOd2MhjDjyxqyea2UPAYUB3MysBrgJSAdz9duAF4DjgY+ALIqueyV4alZ/NWQf35553VjBlbD7j+6q3QESaZrGcWDSz/u6+orltraGwsNCLitTU3JQtVdUcfcNbZGek8ux53yI1OdZTQSLSXplZsbsXNvZYrP9CPN7Itsf2vCQJUue0FGZMivQW/FW9BSLSjCaHhsxsGDACyG5wTiALSA+yMNk7x4zoxTHDe/KnVz7k0CG57Ns7K+ySRCRONXdEMBQ4Acghcp6g7jIO+HGwpcne+u2UkeR0TOWc+4sp+2J72OWISJxqMgjc/WngR8Af3f2H9S7nu/s7rVOi7Kkemencetp41pVv44KH51JTq0YzEfm6Zs8RuHsNcHQr1CIBGN+3C1edOIK3Pizlxlc/DLscEYlDsc419I6Z/QV4BNhat9HdZwdSlbSo0w7ow/ySMm5+/WNG5mUzYUSvsEsSkTgSaxAcHL2+ut42B7R4fRtgZlw9eSRL12/mokfnMejczgzM7Rx2WSISJ2KdYuLwRi4KgTYkPTWZ204fT4eUJM65v5gtVdVhlyQicSLWhWmyzeyGuonfzOyPZpYddHHSsvJyMvjLD8ayvHQLFz86T7OUiggQe0PZ3cBm4HvRSwVwT1BFSXAOHtidy47dl5cWree2t/4TdjkiEgdiPUcw0N2/U+/+DDObG0RBErwffbs/89eU838zlzFyn2wOGaIZXUUSWaxHBNvM7Ft1d8zsm8C2YEqSoJkZf/jOKAb3yOT8h+ewetMXYZckIiGKNQimA7eY2UozWwn8BTgnsKokcB07pHDHGeOprXXOub+Ybdv3dsE5EWmrYg2CiuiaxaOB0e4+lsg5A2nD+nXvxE2njmXJ+gouf3KBTh6LJKjdmn3U3SvcvSK6TbOPtgOHD+vBz44cwpNz1nDfOyvDLkdEQqDZR4XzjhjEgjXl/Pb5JQzfJ5v9+3cNuyQRaUWafVRISjJu+P5+FHTtyE8fnM368sqwSxKRVhTrCmUHufu7rVBPs7RCWXA+3LCZk255m6G9Mnl42oGkpSSHXZKItJCWWKFsipllmVmqmb1mZp+a2ekxvPFEM1tmZh+b2aWNPN7HzN4wszlmNt/MjouxHgnAkJ6Z/N9392POqjJmPLs47HJEpJXEGgTHRE8SnwCUAEOAS5p6gpklA7cAxwLDgalmNrzBbr8GHo3+CulU4NbdqF0CcNyo3kw/dCB/f38Vj3ywKuxyRKQVxBoEqdHr44CH3H1TDM/ZH/jY3Ze7+3bgYWByg32cyIlngGxgbYz1SIAuPmYI3xrUnSueWsS81WVhlyMiAYs1CJ41s6VAIfCameUCzZ1RzANW17tfEt1W32+A082sBHgBOC/GeiRAKclJ/HnqWHIz05j+QDGfbqkKuyQRCVCs01BfChwEFLr7DiKL0zT8dt+QNfZSDe5PBe5193wiRxv3m9nXajKzaXUzn5aWlsZSsuylrp06cMcZ49m0dTvn/n021TW1YZckIgFpMgjM7Ijo9cnA4cDk6O2JfLlYza6UAAX17ufz9aGfs4FHAaK/SkoHujd8IXe/090L3b0wN1cTpLWWkXnZXDtlFO8t38TvX1wadjkiEpDmZh89BHidSO+AE/mWX//6iSae+wEw2Mz6A2uInAz+QYN9VgFHAvea2b5EgkBf+ePId8bnM7+kjLv+tYLRBTlM2m+fsEsSkRbWXBBsNrMLgYV8GQDw9SGer3H3ajM7F5gJJAN3u/siM7saKHL3Z4CLgP9nZj+PvuZZrglv4s6vjh/OorUV/PKx+Qzu0Zl9e2c1/yQRaTOabCgzs6uiN4cC3wCeJhIGJwKz3P1HgVfYgBrKwrGxopITbv4X6anJPHvut8jumNr8k0QkbuxxQ5m7z3D3GUTG7ce5+8XufhEwnsiYvySIHlnp3Hb6ONaVb+OCR+ZQW6sDN5H2Itafj/YBtte7vx3o1+LVSFwb37crV544gjeXlXLjqx+GXY6ItJBYl6q8H/i3mT1JZCx/CnBfYFVJ3Dr9gD7MX13Gn1//mFH5ORw9vGfYJYnIXoq1j+B3wA+Bz4Ey4Ifu/r9BFibxycy45qSRjMrL5sJH5vKf0i1hlyQieynWoSHcfba73xS9zAmyKIlv6anJ3H7GeFJTkjjn/mK2VFWHXZKI7IWYg0CkvrycDP4ydSzLS7dwyT/maZlLkTZMQSB77OBB3bn02GG8uHA9t7+1POxyRGQPKQhkr/z42wM4fnRvrp+5lH9+pKZwkbZIQSB7xcy47jujGdwjk/MemsPqTV+EXZKI7CYFgey1Tmkp3HHGeGpqnekPFFO5oybskkRkNygIpEX0696Jm04dw6K1FVz+xAKdPBZpQxQE0mKOGNaTnx01mCfmrOFv734SdjkiEiMFgbSo848YzJHDenDNc4v5YGUsK5qKSNgUBNKikpKMG74/hoKuHfnpg7PZUNHciqYiEjYFgbS47IxU7jhjPFurqvnJA8Vsr9YylyLxTEEggRjSM5PrT9mP2avKuPq5RWGXIyJNUBBIYI4f3ZtzDhnAA++t4tGi1WGXIyK7oCCQQF0yYSjfHNSNXz+1kPklZWGXIyKNCDQIzGyimS0zs4/N7NJd7PM9M1tsZovM7O9B1iOtLyU5iZunjiO3cxrT7y/msy1VYZckIg0EFgRmlgzcAhwLDAemmtnwBvsMBi4DvunuI4CfBVWPhKdrpw7cfvp4Pt26nfMemkN1jU4ei8STII8I9gc+dvfl7r4deBiY3GCfHwO3uPvnAO6+McB6JESj8rO5dsoo3vnPZ1w3c1nY5YhIPUEGQR5Q/wxhSXRbfUOAIWb2tpm9Z2YTG3shM5tmZkVmVlRaqhku26pTxudzxoF9uXPWcp6dtzbsckQkKsggsEa2NZyAJgUYDBwGTAXuMrOcrz3J/U53L3T3wtzc3BYvVFrPFScMp7BvF37x2HyWrq8IuxwRIdggKAEK6t3PBxp+DSwBnnb3He6+AlhGJBikneqQksStp42jc3oK59xfTPm2HWGXJJLwggyCD4DBZtbfzDoApwLPNNjnKeBwADPrTmSoSEtdtXM9stK57bRxrPl8Gz97eA61tZqpVCRMgQWBu1cD5wIzgSXAo+6+yMyuNrNJ0d1mAp+Z2WLgDeASd/8sqJokfhT268pVJw7njWWl3PTaR2GXI5LQrK3NG19YWOhFRUVhlyEtwN25+B/zeXx2CXedWchRw3uGXZJIu2Vmxe5e2Nhj6iyW0JgZv5sykpF5Wfz8kbksL90SdkkiCUlBIKFKT03m9tPHk5JsnHN/MVurqsMuSSThKAgkdPldOnLz1HH8p3QLlzw2T8tcirQyBYHEhW8N7s4vJw7jhQXruWOWfjgm0poUBBI3ph0ygONH9ea6l5byr48+DbsckYShIJC4YWZcd8poBvXozHkPzWb1pi/CLkkkISgIJK50SkvhjjMKqa5xfvJgMZU7asIuSaTdUxBI3OnfvRM3njqGhWsq+NWTC3XyWCRgCgKJS0fu25MLjhzM47NLuP+9T8IuR6RdUxBI3LrgyMEcOawHVz+7mKKVm8IuR6TdUhBI3EpKMm74/hjyu2Twkwdns6GiMuySRNolBYHEteyMVO44o5AtldX89MHZbK/WMpciLU1BIHFvaK9Mrv/uaIo/+Zxrnlscdjki7U5K2AWIxOKE0fswv6ScO2ctZ3R+Nt8tLGj+SSISEx0RSJvxiwlDOXhgN3711EIWlJSHXY5Iu6EgkDYjJTmJm6eOJbdzGtMfKOazLVVhlyTSLigIpE3p1jmN204fR+mWKs57aA7VNTp5LLK3FATS5ozOz+G3J43knf98xvUzl4VdjkibF2gQmNlEM1tmZh+b2aVN7HeKmbmZNbqMmkhD3yss4PQD+3DHrOU8N39t2OWItGmBBYGZJQO3AMcCw4GpZja8kf0ygfOB94OqRdqnK08Ywbg+OfzisfksW7857HJE2qwgjwj2Bz529+Xuvh14GJjcyH7XANcBahuV3dIhJYnbTh9Pp7QUzrm/iPJtO8IuSaRNCjII8oDV9e6XRLftZGZjgQJ3f66pFzKzaWZWZGZFpaWlLV+ptFk9s9K59bRxlHy+jQsfmUttrWYqFdldQQaBNbJt51+pmSUBfwIuau6F3P1Ody9098Lc3NwWLFHag2/068oVJwzntaUb+fPrH4VdjkibE2QQlAD12z/zgfpn9TKBkcCbZrYSOBB4RieMZU+ceVBfTh6Xx42vfsSLC9ZpDQOR3RDkFBMfAIPNrD+wBjgV+EHdg+5eDnSvu29mbwIXu3tRgDVJO2VmXDtlFMvWb+YnD84mp2Mq++XnMKYghzF9chiTn0OXTh3CLlMkLgUWBO5ebWbnAjOBZOBud19kZlcDRe7+TFDvLYkpPTWZv//oQF5YuI65q8qYu7qMP3/0EXUHB/27d4oEQ/Syb+8sOqSolUbE2tohdGFhoRcV6aBBYrOlqpr5JZFQmBMNh9LNkakpOqQkMXKfLMYUdGFMnxzGFuSQ3yUDs8ZOb4m0bWZW7O6NDr0rCCShuDtryyujRwyfM2dVGQvWlFMVXeege+cO9Y4aujC6IJus9NSQqxbZe00FgaahloRiZuTlZJCXk8Hxo3sDsKOmlmXrNzNndRlzV5UxZ/XnvLpkY3R/GJTb+ctzDQU5DO2ZSUqyhpSk/dARgUgjyr/YwbzokFLdZdPW7QBkpCYzKj+bsQVfnozunZ0RcsUiTdMRgchuyu6YyiFDcjlkSKRvxd1ZtemLr5xruOftlWyPzn7aKyv9K0cNo/Oz6dhBf17SNuj/VJEYmBl9u3Wib7dOTB4TaZCvqq5h8dqKrxw1vLRoPQBJBkN7ZTGmIHISekyfHAbldiYpSSeiJf4oCET2UFpKMmP7dGFsny47t322pSoypLSqjDmry3h+/loe+vcqADqnpbBfQfbOE9FjCnLIzUwLq3yRnRQEIi2oW+c0jhjWkyOG9QSgttZZ8dnW6HDS58xdXcYdby2nOjonUl5OBmOjw0lj++QwYp9s0lOTw/wIkoAUBCIBSkoyBuZ2ZmBuZ04Znw9A5Y4aFq4p33muYc6qMp6bvw6AlCRj395ZO8NhTEEO/bt3Um+DBEq/GhKJAxsrKiOhEP0J6/ySMrZurwEgOyOV/eqda9B0GbIn1FAm0sbU1Dofb9yys+lt7uoyPtywmbpZtvt16xgdToqcaxjaK1NDStIkBYFIO7C1qpr5JeXRXyhFAmJjdLoMgC4dU+mVnUGvrLTodTq9suvfTicrPUXDTAlKfQQi7UCntBQOGtiNgwZ2AyK9DevKI0NKy0u3sL6ikvXllayvqGTBmnI+3bL9a6+RkZpM7+x0emalR66z079yv1dWOt06p5Gsn7kmFAWBSBtlZuyTk8E+OY13NW+vrmVDRSUbKipZV/7ldV1gvL9iExs3V7Kj5qujAslJRs/MtK+FROQ6cnTRIytNQ1HtiIJApJ3qkJJEQdeOFHTtuMt9amudz7ZubxAS21hfXsX6im0sW7+Zt5aV7jxxXV/XTh0ahET6ziGouktmmoai2gIFgUgCS0oycjPTyM1MY2Re9i7321y5Y+ew07rySjaUV7KuInK9vqKSeavL+Gzr14eiOnZIjoRCXUA0vM5Op3unNHVch0xBICLNykxPJTM9lcE9M3e5T1V1DRsrqr4aFjuHpLbx/vJNbKio3NlMVyclyeiZlU7PrDR6Z2d87fxF3VBUWoqGooKiIBCRFpGWkhzTUNSnW6siRxf1zlfU3V6yroI3lm3ki0aGorrVH4rKTqd7pw5kZaSSlZ5KVkZK9PrL+53TUjRdeIwUBCLSapKSjB6Z6fTITGd0fuP7uDsVldVsaBASdUcXa8srmb3qc8q27aC5X793TkshKz1lF4Gxq+2JFySBBoGZTQRuIrJm8V3u/vsGj18I/AioBkqB/3b3T4KsSUTim5mRnZFKdkYqQ5oYiqqtdbZsr6Zi2w4qtlVTUbkjcrsyuq2y4fYdrC2rZGnlZiq27WBzVbWCJCqwIDCzZOAW4GigBPjAzJ5x98X1dpsDFLr7F2b2E+A64PtB1SQi7UdSkkX+0U1PhS7N79/Q7gTJ5ujtdeWVLNuweef25oKkU4fkPQqRrPRUMtNbL0iCPCLYH/jY3ZcDmNnDwGRgZxC4+xv19n8POD3AekREdmqJINm6vfrL4GjmaKRiWzXrKyr5cOOeB8mUsfn84IA+e/aBmxBkEOQBq+vdLwEOaGL/s4EXG3vAzKYB0wD69Gn5/wgiIrsrKcl2/poqbxdNfU3ZkyBxgpkSKMggaOyHwY1+CjM7HSgEDm3scXe/E7gTInMNtVSBIiJh2dsgaUlBBkEJUFDvfj6wtuFOZnYU8CvgUHevavi4iIgEK8gzER8Ag82sv5l1AE4Fnqm/g5mNBe4AJrn7xgBrERGRXQgsCNy9GjgXmAksAR5190VmdrWZTYrudj3QGfiHmc01s2d28XIiIhKQQPsI3P0F4IUG266sd/uoIN9fRESa1za6HUREJDAKAhGRBKcgEBFJcAoCEZEE1+YWrzezUmBPJ6brDnzaguW0BfrMiUGfOTHszWfu6+65jT3Q5oJgb5hZkbsXhl1Ha9JnTgz6zIkhqM+soSERkQSnIBARSXCJFgR3hl1ACPSZE4M+c2II5DMn1DkCERH5ukQ7IhARkQYUBCIiCS4hgsDM7jazjWa2MOxaWouZFZjZG2a2xMwWmdkFYdcUNDNLN7N/m9m86AzGknIAAAUmSURBVGeeEXZNrcHMks1sjpk9F3YtrcXMVprZguisxUVh1xM0M8sxs8fMbGn0b/qgFn39RDhHYGaHAFuAv7n7yLDraQ1m1hvo7e6zzSwTKAZOcvfFzTy1zTIzAzq5+xYzSwX+BVzg7u+FXFqgzOxCIiv8Zbn7CWHX0xrMbCVQ6O4J0VBmZvcB/3T3u6Lru3R097KWev2EOCJw91nAprDraE3uvs7dZ0dvbyayJkReuFUFyyO2RO+mRi/t+puOmeUDxwN3hV2LBMPMsoBDgL8CuPv2lgwBSJAgSHRm1g8YC7wfbiXBiw6TzAU2Aq+4e3v/zDcCvwBqwy6klTnwspkVm9m0sIsJ2ACgFLgnOgR4l5l1ask3UBC0c2bWGXgc+Jm7V4RdT9DcvcbdxxBZI3t/M2u3Q4FmdgKw0d2Lw64lBN9093HAscD/RId/26sUYBxwm7uPBbYCl7bkGygI2rHoOPnjwIPu/kTY9bSm6KHzm8DEkEsJ0jeBSdHx8oeBI8zsgXBLah3uvjZ6vRF4Etg/3IoCVQKU1Du6fYxIMLQYBUE7FT1x+ldgibvfEHY9rcHMcs0sJ3o7AzgKWBpuVcFx98vcPd/d+wGnAq+7++khlxU4M+sU/QEE0SGSY4B2+4tAd18PrDazodFNRwIt+qOPQNcsjhdm9hBwGNDdzEqAq9z9r+FWFbhvAmcAC6Jj5gCXR9eRbq96A/eZWTKRLzmPunvC/KQygfQEnox81yEF+Lu7vxRuSYE7D3gw+ouh5cAPW/LFE+LnoyIismsaGhIRSXAKAhGRBKcgEBFJcAoCEZEEpyAQEUlwCgIRkQSnIBBpIWbWb0+nOjezs8xsn5auSSQWCgKR+HAWsFtBYGYJ0RAqwVMQSLsR/Ua+xMz+X3RhmpfNLMPM3jSzwug+3aNz89R9C3/KzJ41sxVmdq6ZXRid4fE9M+vaxHsNMrNXo4vgzDazgQ0eP8vM/lLv/nNmdlh0dtR7zWxhdGGVn5vZKUTWE3gwutBKhpmNN7O3orNrzoyuL0H0s1xrZm8BF5jZd6OvNc/MZrX8f1VJBPpGIe3NYGCqu//YzB4FvtPM/iOJTNGdDnwM/NLdx5rZn4AziUzz3JgHgd+7+5Nmlk7kS1WPGOobA+TVLZBkZjnuXmZm5wIXu3tRdLLAm4HJ7l5qZt8Hfgf8d/Q1ctz90OjzFwAT3H1N3TxLIrtLQSDtzQp3r5tbqRjo18z+b0QX7tlsZuXAs9HtC4DRjT0hOuFZnrs/CeDuldHtsdS3HBhgZjcDzwMvN7LPUCIB9Ur0NZOBdfUef6Te7beBe6Ohl1AzzErLURBIe1NV73YNkAFU8+UwaHoT+9fWu1/Lrv8+YvkXv/577nxfd//czPYDJgD/A3yPL7/p13/9Re6+q3Vpt9bdcPfpZnYAkVXK5prZGHf/LIb6RHbSOQJJBCuB8dHbp+zti0UX+Ckxs5MAzCzNzDo28p5jzCzJzAqIzpdvZt2BJHd/HLiCL+eV3wxkRm8vA3LrFig3s1QzG9FYLWY20N3fd/crgU+Bgr39fJJ4dEQgieD/gEfN7Azg9RZ6zTOAO8zsamAH8F2+ulzk28AKIkNMC4HZ0e15RJYcrPsSdln0+l7gdjPbBhxEJLD+bGbZRP5ObwQWNVLH9WY2mMhRxGvAvBb5dJJQNA21iEiC09CQiEiC09CQSBPM7BYiq73Vd5O73xNGPSJB0NCQiEiC09CQiEiCUxCIiCQ4BYGISIJTEIiIJLj/DxKiwZW6srHlAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from scipy.cluster.vq import kmeans\n", "\n", "distortions = []\n", "num_clusters = range(1, 7)\n", "\n", "# Create a list of distortions from the kmeans function\n", "for i in num_clusters:\n", " cluster_centers, distortion = kmeans(batman_df[['scaled_red', 'scaled_blue', 'scaled_green']], i)\n", " distortions.append(distortion)\n", " \n", "# Create a data frame with two lists, num_clusters and distortions\n", "elbow_plot = pd.DataFrame({'num_clusters': num_clusters, 'distortions': distortions})\n", "\n", "# Create a line plot of num_clusters and distortions\n", "sns.lineplot(x='num_clusters', y='distortions', data=elbow_plot);\n", "plt.xticks(num_clusters);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Display dominant colors\n", "To display the dominant colors, convert the colors of the cluster centers to their raw values and then converted them to the range of 0-1, using the following formula: \n", "```python\n", "converted_pixel = standardized_pixel * pixel_std / 255\n", "```" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAABbCAYAAABwOT7wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAHeUlEQVR4nO3db6jWZx3H8fdnxzNXrpZuukQtV4k0Fqw6CCFEbNlcDR3VhkLDweQQZFv0oJRg1J5kPaie7MnYpL/kwvXnsEmboSO22uZx6ZaazWQwUbBlq86ghuvTg/MbHE6385zz+93nd+5zfV5wc//+XN7X90L8nIvL330d2SYiIma/i9ouICIipkcCPyKiEAn8iIhCJPAjIgqRwI+IKEQCPyKiELUCX9ICSXskvVC9zz9Pu9clHaxeQ3X6jIiIqVGd5/AlfRs4a3u7pK3AfNtf7dBuxPalNeqMiIia6gb+MeBjtk9LWgw8bntlh3YJ/IiIltVdw7/S9mmA6n3RedpdImlY0lOSbq7ZZ0RETMGcCzWQ9BvgnR1ufW0S/bzL9ilJ7wH2Snre9l869DUIDFanH9YkOug1i/rntl1CV7264Hw/+3tf//wFbZfQVSMnjrZdQlet6H9r2yV01ZFXX3nZ9sJO96ZlSWfcn/k+8LDtXW/W7iLJ/XP6plzbTLdl8XvbLqGrDmy8s+0SumbRp29tu4SuenLDqrZL6KrHllzbdglddc2Tvzxge6DTvbpLOkPApup4E/Cr8Q0kzZc0tzq+AlgNHKnZb0RETFLdwN8OrJH0ArCmOkfSgKT7qzbvB4YlHQL2AdttJ/AjIqbZBdfw34ztvwHXd7g+DGyujn8HfKBOPxERUV++aRsRUYgEfkREIRL4ERGFSOBHRBQigR8RUYgEfkREIRL4ERGFSOBHRBQigR8RUYgEfkREIRL4ERGFSOBHRBQigR8RUYgEfkREIRL4ERGFSOBHRBQigR8RUYgEfkREIRL4ERGFaCTwJa2VdEzScUlbO9yfK+nB6v7TkpY30W9ERExc7cCX1AfcC9wIXA1slHT1uGZ3AH+3/T7gu8C36vYbERGT08QMfxVw3PYJ268BO4H149qsB35QHe8CrpekBvqOiIgJaiLwlwAvjTk/WV3r2Mb2OeAfwOUN9B0RERM0p4HP6DRT9xTaIGkQGGygpoiIGKeJGf5JYNmY86XAqfO1kTQHuAw4O/6DbN9ne8D2QNZ7IiKa1UTg7wdWSLpK0sXABmBoXJshYFN1/Flgr+3/m+FHRET31F7SsX1O0hbgUaAP2GH7sKR7gGHbQ8ADwI8kHWd0Zr+hbr8RETE5TazhY3s3sHvctbvHHP8buKWJviIiYmryTduIiEIk8CMiCpHAj4goRAI/IqIQCfyIiEIk8CMiCpHAj4goRAI/IqIQCfyIiEIk8CMiCpHAj4goRAI/IqIQCfyIiEIk8CMiCpHAj4goRAI/IqIQCfyIiEIk8CMiCpHAj4goRCOBL2mtpGOSjkva2uH+7ZL+Kulg9drcRL8RETFxtX+JuaQ+4F5gDXAS2C9pyPaRcU0ftL2lbn8RETE1TczwVwHHbZ+w/RqwE1jfwOdGRESDmgj8JcBLY85PVtfG+4yk5yTtkrSsgX4jImISZLveB0i3ADfY3lyd3wassv3FMW0uB0Zs/0fS54FbbV/X4bMGgcHqdCVwrFZxk3MF8PI09jfdMr7elvH1ruke27ttL+x0o4nA/wjwdds3VOfbAGx/8zzt+4Czti+r1XHDJA3bHmi7jm7J+Hpbxte7ZtLYmljS2Q+skHSVpIuBDcDQ2AaSFo85XQccbaDfiIiYhNpP6dg+J2kL8CjQB+ywfVjSPcCw7SHgTknrgHPAWeD2uv1GRMTk1A58ANu7gd3jrt095ngbsK2JvrrovrYL6LKMr7dlfL1rxoyt9hp+RET0hmytEBFRiAQ+F94aopdJ2iHpjKQ/tl1LN0haJmmfpKOSDku6q+2amiLpEknPSDpUje0bbdfUDZL6JP1B0sNt19I0SS9Ker7aUma49XpKX9KpHhP9M2O2hgA2dtgaoidJ+igwAvzQ9jVt19O06gmwxbaflfQ24ABw82z4+5MkYJ7tEUn9wBPAXbafarm0Rkn6MjAAvN32TW3X0yRJLwIDtmfEdwwyw5/lW0PY/i2jT0bNSrZP2362Ov4Xo4/8dvqmd8/xqJHqtL96zaoZmqSlwKeA+9uupQQJ/IlvDREznKTlwAeBp9utpDnVcsdB4Aywx/asGVvle8BXgP+2XUiXGHhM0oFqJ4FWJfBBHa7NqllUCSRdCjwEfMn2P9uupym2X7d9LbAUWCVp1izLSboJOGP7QNu1dNFq2x8CbgS+UC2xtiaBPzqjH7uZ21LgVEu1xBRU69sPAT+x/fO26+kG268AjwNrWy6lSauBddU6907gOkk/brekZtk+Vb2fAX7B6BJyaxL4E9gaImau6j82HwCO2v5O2/U0SdJCSe+ojt8CfBz4U7tVNcf2NttLbS9n9N/dXtufa7msxkiaVz1IgKR5wCeAVp+WKz7wbZ8D3tga4ijwM9uH262qOZJ+CvweWCnppKQ72q6pYauB2xidHb7xG9U+2XZRDVkM7JP0HKMTkz22Z92ji7PYlcATkg4BzwCP2P51mwUV/1hmREQpip/hR0SUIoEfEVGIBH5ERCES+BERhUjgR0QUIoEfEVGIBH5ERCES+BERhfgfDcE5kV85BhcAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "colors = []\n", "\n", "# Get standard deviations of each color\n", "r_std, g_std, b_std = batman_df[['red', 'green', 'blue']].std()\n", "\n", "for cluster_center in cluster_centers:\n", " scaled_r, scaled_g, scaled_b = cluster_center\n", " # Convert each standardized value to scaled value\n", " colors.append((\n", " scaled_r * r_std / 255.0,\n", " scaled_g * g_std / 255.0,\n", " scaled_b * b_std / 255.0\n", " )\n", " )\n", " \n", "# Display colors of cluster centers\n", "plt.imshow([colors])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Document clustering\n", "- Document clustering: concepts\n", " - 1. Clean data before processing\n", " - 2. Determine the importance of the terms in a document (in tf-idf matrix)\n", " - 3. Cluster the tf-idf matrix\n", " - 4. Find top terms, documents in each cluster\n", "- TF-IDF (Term Frequency - Inverse Document Frequency)\n", " - A weighted measure: evaluate how important a word is to a document in a collection\n", "- Top terms per cluster\n", " - Cluster centers: lists with a size equal to the number of terms\n", " - Each value in the cluster center is its importance\n", "- More considerations\n", " - Work with hyperlinks, emoticons etc.\n", " - Normalize words (e.g. run, ran, running -> run)\n", " - ```.todense()``` may not work with large datasets" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TF-IDF of movie plots\n", "Let us use the plots of randomly selected movies to perform document clustering on. Before performing clustering on documents, they need to be cleaned of any unwanted noise (such as special characters and stop words) and converted into a sparse matrix through TF-IDF of the documents.\n", "\n", "Use the ```TfidfVectorizer``` class to perform the TF-IDF of movie plots stored in the list ```plots```. The ```remove_noise()``` function is available to use as a ```tokenizer``` in the ```TfidfVectorizer``` class. The ```.fit_transform()``` method fits the data into the ```TfidfVectorizer``` objects and then generates the TF-IDF sparse matrix.\n", "\n", "**Note: It takes a few seconds to run the ```.fit_transform()``` method.**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Preprocess" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TitlePlot
0The Ballad of Cable HogueCable Hogue is isolated in the desert, awaitin...
1Monsters vs. AliensIn the far reaches of space, a planet explodes...
2The Bandit QueenZarra Montalvo is the daughter of an American ...
3Broken ArrowMajor Vic Deakins (John Travolta) and Captain ...
4DolemiteDolemite is a pimp and nightclub owner who is ...
\n", "
" ], "text/plain": [ " Title \\\n", "0 The Ballad of Cable Hogue \n", "1 Monsters vs. Aliens \n", "2 The Bandit Queen \n", "3 Broken Arrow \n", "4 Dolemite \n", "\n", " Plot \n", "0 Cable Hogue is isolated in the desert, awaitin... \n", "1 In the far reaches of space, a planet explodes... \n", "2 Zarra Montalvo is the daughter of an American ... \n", "3 Major Vic Deakins (John Travolta) and Captain ... \n", "4 Dolemite is a pimp and nightclub owner who is ... " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "movie = pd.read_csv('./dataset/movies_plot.csv')\n", "movie.head()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "plots = movie['Plot'].values" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[nltk_data] Downloading package punkt to\n", "[nltk_data] C:\\Users\\kcsgo\\AppData\\Roaming\\nltk_data...\n", "[nltk_data] Package punkt is already up-to-date!\n" ] } ], "source": [ "from nltk.tokenize import word_tokenize\n", "import re\n", "\n", "import nltk\n", "nltk.download('punkt')\n", "\n", "def remove_noise(text, stop_words = []):\n", " tokens = word_tokenize(text)\n", " cleaned_tokens = []\n", " for token in tokens:\n", " token = re.sub('[^A-Za-z0-9]+', '', token)\n", " if len(token) > 1 and token.lower() not in stop_words:\n", " # Get lowercase\n", " cleaned_tokens.append(token.lower())\n", " return cleaned_tokens" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "from sklearn.feature_extraction.text import TfidfVectorizer\n", "\n", "# Initialize TfidfVectorizer\n", "tfidf_vectorizer = TfidfVectorizer(min_df=0.1, max_df=0.75, max_features=50, tokenizer=remove_noise)\n", "\n", "# Use the .fit_transform() on the list plots\n", "tfidf_matrix = tfidf_vectorizer.fit_transform(plots)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Top terms in movie clusters\n", "Now that you have created a sparse matrix, generate cluster centers and print the top three terms in each cluster. Use the ```.todense()``` method to convert the sparse matrix, ```tfidf_matrix``` to a normal matrix for the ```kmeans()``` function to process. Then, use the ```.get_feature_names()``` method to get a list of terms in the ```tfidf_vectorizer``` object. The zip() function in Python joins two lists.\n", "\n", "With a higher number of data points, the clusters formed would be defined more clearly. However, this requires some computational power, making it difficult to accomplish in an exercise here.\n", "\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['her', 'she', 'him']\n", "['him', 'they', 'who']\n" ] } ], "source": [ "num_clusters = 2\n", "\n", "# Generate cluster centers through the kmeans function\n", "cluster_centers, distortion = kmeans(tfidf_matrix.todense(), num_clusters)\n", "\n", "# Generate terms from the tfidf_vectorizer object\n", "terms = tfidf_vectorizer.get_feature_names()\n", "\n", "for i in range(num_clusters):\n", " # Sort the terms and print top 3 terms\n", " center_terms = dict(zip(terms, list(cluster_centers[i])))\n", " sorted_terms = sorted(center_terms, key=center_terms.get, reverse=True)\n", " print(sorted_terms[:3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clustering with multiple features\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Basic checks on clusters\n", "In the FIFA 18 dataset, we have concentrated on defenders in previous exercises. Let us try to focus on attacking attributes of a player. Pace (```pac```), Dribbling (```dri```) and Shooting (```sho```) are features that are present in attack minded players. In this exercise, k-means clustering has already been applied on the data using the scaled values of these three attributes. Try some basic checks on the clusters so formed.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Preprocess" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
IDnamefull_nameclubclub_logospecialageleaguebirth_dateheight_cm...prefers_cbprefers_lbprefers_lwbprefers_lsprefers_lfprefers_lamprefers_lcmprefers_ldmprefers_lcbprefers_gk
020801Cristiano RonaldoC. Ronaldo dos Santos AveiroReal Madrid CFhttps://cdn.sofifa.org/18/teams/243.png222832Spanish Primera División1985-02-05185.0...FalseFalseFalseFalseFalseFalseFalseFalseFalseFalse
1158023L. MessiLionel MessiFC Barcelonahttps://cdn.sofifa.org/18/teams/241.png215830Spanish Primera División1987-06-24170.0...FalseFalseFalseFalseFalseFalseFalseFalseFalseFalse
2190871NeymarNeymar da Silva Santos Jr.Paris Saint-Germainhttps://cdn.sofifa.org/18/teams/73.png210025French Ligue 11992-02-05175.0...FalseFalseFalseFalseFalseFalseFalseFalseFalseFalse
3176580L. SuárezLuis SuárezFC Barcelonahttps://cdn.sofifa.org/18/teams/241.png229130Spanish Primera División1987-01-24182.0...FalseFalseFalseFalseFalseFalseFalseFalseFalseFalse
4167495M. NeuerManuel NeuerFC Bayern Munichhttps://cdn.sofifa.org/18/teams/21.png149331German Bundesliga1986-03-27193.0...FalseFalseFalseFalseFalseFalseFalseFalseFalseTrue
\n", "

5 rows × 185 columns

\n", "
" ], "text/plain": [ " ID name full_name \\\n", "0 20801 Cristiano Ronaldo C. Ronaldo dos Santos Aveiro \n", "1 158023 L. Messi Lionel Messi \n", "2 190871 Neymar Neymar da Silva Santos Jr. \n", "3 176580 L. Suárez Luis Suárez \n", "4 167495 M. Neuer Manuel Neuer \n", "\n", " club club_logo special age \\\n", "0 Real Madrid CF https://cdn.sofifa.org/18/teams/243.png 2228 32 \n", "1 FC Barcelona https://cdn.sofifa.org/18/teams/241.png 2158 30 \n", "2 Paris Saint-Germain https://cdn.sofifa.org/18/teams/73.png 2100 25 \n", "3 FC Barcelona https://cdn.sofifa.org/18/teams/241.png 2291 30 \n", "4 FC Bayern Munich https://cdn.sofifa.org/18/teams/21.png 1493 31 \n", "\n", " league birth_date height_cm ... prefers_cb \\\n", "0 Spanish Primera División 1985-02-05 185.0 ... False \n", "1 Spanish Primera División 1987-06-24 170.0 ... False \n", "2 French Ligue 1 1992-02-05 175.0 ... False \n", "3 Spanish Primera División 1987-01-24 182.0 ... False \n", "4 German Bundesliga 1986-03-27 193.0 ... False \n", "\n", " prefers_lb prefers_lwb prefers_ls prefers_lf prefers_lam prefers_lcm \\\n", "0 False False False False False False \n", "1 False False False False False False \n", "2 False False False False False False \n", "3 False False False False False False \n", "4 False False False False False False \n", "\n", " prefers_ldm prefers_lcb prefers_gk \n", "0 False False False \n", "1 False False False \n", "2 False False False \n", "3 False False False \n", "4 False False True \n", "\n", "[5 rows x 185 columns]" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fifa = pd.read_csv('./dataset/fifa_18_sample_data.csv')\n", "fifa.head()" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "fifa['scaled_pac'] = whiten(fifa['pac'])\n", "fifa['scaled_dri'] = whiten(fifa['dri'])\n", "fifa['scaled_sho'] = whiten(fifa['sho'])" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "from scipy.cluster.vq import vq\n", "\n", "cluster_centers, _ = kmeans(fifa[['scaled_pac', 'scaled_dri', 'scaled_sho']], 3)\n", "\n", "fifa['cluster_labels'], _ = vq(fifa[['scaled_pac', 'scaled_dri', 'scaled_sho']], cluster_centers)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cluster_labels\n", "0 182\n", "1 457\n", "2 361\n", "Name: ID, dtype: int64\n", "cluster_labels\n", "0 63225.274725\n", "1 77297.592998\n", "2 62603.878116\n", "Name: eur_wage, dtype: float64\n" ] } ], "source": [ "# Print the size of the clusters\n", "print(fifa.groupby('cluster_labels')['ID'].count())\n", "\n", "# Print the mean value of wages in each cluster\n", "print(fifa.groupby('cluster_labels')['eur_wage'].mean())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### FIFA 18: what makes a complete player?\n", "The overall level of a player in FIFA 18 is defined by six characteristics: pace (```pac```), shooting (```sho```), passing (```pas```), dribbling (```dri```), defending (```def```), physical (```phy```)." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "fifa['scaled_def'] = whiten(fifa['def'])\n", "fifa['scaled_phy'] = whiten(fifa['phy'])" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "scaled_features = ['scaled_pac', 'scaled_sho', 'scaled_pac', 'scaled_dri', 'scaled_def', 'scaled_phy']" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " scaled_pac scaled_sho scaled_pac scaled_dri scaled_def \\\n", "cluster_labels \n", "0 6.828114 5.475576 6.828114 8.579753 2.366369 \n", "1 5.491062 3.998012 5.491062 7.059324 3.858428 \n", "\n", " scaled_phy \n", "cluster_labels \n", "0 8.274933 \n", "1 9.109248 \n", "0 ['Cristiano Ronaldo' 'L. Messi' 'Neymar' 'L. Suárez' 'M. Neuer']\n", "1 ['T. Kroos' 'Sergio Ramos' 'G. Chiellini' 'L. Bonucci' 'J. Boateng']\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEECAYAAAABJn7JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAZkklEQVR4nO3df3TU9Z3v8eebxAroAt6SIqalwRaFyPBrQwJIoXU5wS05KAqXzRZ7xQMcj4QuHGGDu/bCvR5/lCsXW+164VpvifVCKqBFdGmaUha4iBBqJAoRqYQsa0SkgIAFAnzuHzMZ8wsyGWYyn0lej3M8mfnO9/vJm++Jr/M5n+/3+x5zziEiIv7qlOgCRETkyhTUIiKeU1CLiHhOQS0i4jkFtYiI5xTUIiKeS43HoD179nQZGRnxGFpEpF3avXv3Z865tOY+i0tQZ2RkUFZWFo+hRUTaJTM7dLnPtPQhIuI5BbWIiOcU1CIinlNQi4h4TkEtIuI5BbWIiOcU1CIinlNQi4h4Li4PvIiIxMPSqXlRHfdw8YYYV9K2NKMWEfGcglpExHMKahERzymoRUQ8p6AWEfGcglpExHMKahERzymoRUQ8p6AWEfGcglpExHMKahERz6nXh+f29R8Q1XEDKvfFuBIRSRTNqEVEPKegFhHxnIJaRMRzCmoREc8pqEVEPKegFhHxnG7PkwY66lcdifhMM2oREc8pqEVEPBdRUJvZPDN738zeM7NVZtY53oWJiEhQi0FtZunAj4As59xAIAX4u3gXJiIiQZEufaQCXcwsFegKfBy/kkREpL4Wg9o59x/A00A1UAOcdM6VxLswEREJimTp4wbgLqAvcBNwnZlNa2a/WWZWZmZlR48ejX2lIiIdVCRLH+OAg865o865WmAdMKrxTs65Fc65LOdcVlpaWqzrFBHpsCIJ6mpghJl1NTMD/gZQs2MRkTYSyRr128Aa4I9AReiYFXGuS0REQiJ6hNw5twhYFOdaRESkGXoyUUTEcwpqERHPKahFRDynoBYR8ZyCWkTEcwpqERHPKahFRDynoBYR8ZyCWkTEcwpqERHPKahFRDynoBYR8ZyCWkTEcwpqERHPKahFRDynoBYR8VxEXxwgItKcff0HRHXcgEp9m19raEYtIuI5BbWIiOcU1CIinlNQi4h4rsNeTMxY+EZUx1U9NSGq4wIrA1Ed9+uojhKR9kQzahERzymoRUQ8p6AWEfGcglpExHMKahERzymoRUQ8p6AWEfGcglpExHMKahERzymoRUQ8p6AWEfGcglpExHMKahERzymoRUQ8F1FQm1kPM1tjZpVmts/MRsa7MBERCYq0H/VPgY3Ouclm9hWgaxxrEhGReloMajPrBowB7gdwzp0Hzse3LBERqRPJ0sfNwFHg/5jZO2b2gpldF+e6REQkJJKgTgWGAc8754YCZ4CFjXcys1lmVmZmZUePHo1xmSIiHVckQX0YOOycezv0fg3B4G7AObfCOZflnMtKS0uLZY0iIh1ai0HtnPsE+HczuzW06W+AvXGtSkREwiK962MO8HLojo+PgOnxK0lEROqLKKidc+VAVpxrERGRZujJRBERzymoRUQ8p6AWEfGcglpExHMKahERzymoRUQ8p6AWEfFcpA+8SJ3F3aM7rm+f2NYh7VrGwjeiOq7qqQlRHRdYGYjquF9HdZS0lmbUIiKeU1CLiHhOQS0i4jkFtYiI5xTUIiKeU1CLiHhOQS0i4jkFtYiI5xTUIiKeU1CLiHhOQS0i4jkFtYiI5xTUIiKeU1CLiHhOQS0i4jkFtYiI5xTUIiKeU1CLiHhOQS0i4jkFtYiI5/TlthKVa7peR+aEe7g+rRdmxr59+xJdUrvyvyf2BsDhOHSilmffPs7n5y4luCpJFAW1RCVzwj1k9M/kus7XYmbc+K1+iS6pXak9fAIA5xxf/ernzAEe33IssUVJwmjpQ6JyfVqvcEhL/JgZqV278c0e1yS6FEkgBbVExcwU0m3EzDB0rjsyBbWIiOe0Ri0xkbHwjZiOV/XUhJiO1+zvqKoiLy+P9957L+Jj7r//fvLy8pg8eXIcKxNpSDNqERHPKaglaZ05c4YJEyYwePBgBg4cSHFxMbt27WLUqFEMHjyY7OxsTp06RVVVFd/5zncYNmwYw4YNY/v27U3GunjxIgsWLGD48OEMGjSI5cuXA8G7LgoKCsjMzGTChAl8+umnV6wpIyODwsJCsrOzyc7O5sCBAwC8/vrr5OTkMHToUMaNG8eRI0cAOH36NNOnTycQCDBo0CDWrl0b47Mk7YGWPiRpbdy4kZtuuok33gguu5w8eZKhQ4dSXFzM8OHD+fzzz+nSpQtf+9rX+N3vfkfnzp358MMPyc/Pp6ysrMFYv/jFL+jevTu7du3i3Llz3H777eTm5vLOO+/wwQcfUFFRwZEjR8jMzOSBBx64Yl3dunVj586dFBUVMXfuXDZs2MDo0aPZsWMHZsYLL7zAkiVLWLp0KY899hjdu3enoqICgOPHj8fnZElSiziozSwFKAP+wzmXF7+SRCITCASYP38+hYWF5OXl0aNHD3r37s3w4cOBYGBCcOZdUFBAeXk5KSkp7N+/v8lYJSUl7NmzhzVr1gDB0P/www/ZsmUL+fn5pKSkcNNNN3HHHXe0WFd+fn7457x58wA4fPgwU6dOpaamhvPnz9O3b18ASktLWb16dfjYG2644SrOiLRXrVn6+AdAj5+JN2655RZ2795NIBDgkUce4dVXX232lsFly5bRq1cv3n33XcrKyjh//nyTfZxzPPvss5SXl1NeXs7BgwfJzc0FaPVtiPX3r3s9Z84cCgoKqKioYPny5Zw9ezb8e3Wbo7QkoqA2s68DE4AX4luOSOQ+/vhjunbtyrRp05g/fz47duzg448/ZteuXQCcOnWKCxcucPLkSXr37k2nTp146aWXuHjxYpOxxo8fz/PPP09tbS0A+/fv58yZM4wZM4bVq1dz8eJFampq+MMf/tBiXcXFxeGfI0eOBIIz9PT0dABWrlwZ3jc3N5fnnnsu/F5LH9KcSJc+ngH+EfirONYiSawtbqdrrKKiggULFtCpUyeuueYann/+eZxzzJkzh7/85S906dKF0tJSHnroIe69915eeeUVvve973Hdddc1GWvGjBlUVVUxbNgwnHOkpaXx2muvMWnSJDZt2kQgEOCWW25h7NixLdZ17tw5cnJyuHTpEqtWrQJg8eLFTJkyhfT0dEaMGMHBgwcBePTRR5k9ezYDBw4kJSWFRYsWcc8998T2REnSM+fclXcwywO+75x7yMy+C8xvbo3azGYBswD69Onz14cOHYpDubET7X2/VZ3/PqrjAn37RHXcr5+8ENVxAyqjW6VaOjWyyw+jZy/gm+k3hd+r10dQRkYGZWVl9OzZ86rG2RPq9VHnSPVHzFxf0+Jx+vts3sPFG6I6ri2Z2W7nXFZzn0Wy9HE7MNHMqoDVwB1m9qvGOznnVjjnspxzWWlpaVdVsIiIfKnFpQ/n3CPAIwD1ZtTT4lyXiNcmTZoUXr6o85Of/ISqqqrEFCTtmu6jFonCq6++mugSktrPH9yU6BKSSquC2jm3Gdgcl0pERKRZeoRcRMRzCmoREc8pqEVEPKeLiRIbi7vHeLyTsR2vGW3Vj1o9rOVqaUYtIuI5BbUkLR/7US9cuJDMzEwGDRrE/Pnzw9u3bNnCqFGjuPnmm8Md+pxzLFiwgIEDBxIIBMI9QkQa09KHJC3f+lH/+c9/5tVXX6WyshIz48SJLx8Dr6mpYdu2bVRWVjJx4kQmT57MunXrKC8v59133+Wzzz5j+PDhjBkzht69e8fvpElS0oxaklYgEKC0tJTCwkK2bt1KdXV1k37Uqamp1NbWMnPmTAKBAFOmTGHv3r1NxiopKaGoqIghQ4aQk5PDsWPHWt2Pulu3bnTu3JkZM2awbt06unbtGv7s7rvvplOnTmRmZoa/3WXbtm3hsXv16sXYsWPDnf9E6lNQS9LyrR91amoqO3fu5N577+W1117jzjvvDH927bXXNvhd9X+KtERBLUnLt37Up0+f5uTJk3z/+9/nmWeeoby8/Ir1jxkzhuLiYi5evMjRo0fZsmUL2dnZV3FGpL3SGrXERhvcTteYb/2oT506xV133cXZs2dxzrFs2bIr1j9p0iTeeustBg8ejJmxZMkSbrzxxqs+L9L+tNiPOhpZWVmu8cUa36gfdfPUj9oP7b0f9abv/jyq484e/59RHdcR+lGLiEgCaelDJAqX60c9fvz4BFUk7ZmCWiQK6kctbUlLHyIinlNQi4h4TkEtIuI5BbWIiOd0MVFiIrAyENPxKv5LRUzHa05b9aMWuVqaUYuIeE5BLUnLx37UGRkZFBYWkp2dTXZ2NgcOHADg9ddfJycnh6FDhzJu3LhwB73Tp08zffp0AoEAgwYNYu3atTE+S9IeaOlDkpZv/ajrdOvWjZ07d1JUVMTcuXPZsGEDo0ePZseOHZgZL7zwAkuWLGHp0qU89thjdO/enYqK4FLP8ePH43OyJKkpqCVpBQIB5s+fT2FhIXl5efTo0aNJP2oIzrwLCgooLy8nJSWF/fv3NxmrpKSEPXv2hL995eTJk63uR10nPz8//HPevHkAHD58mKlTp1JTU8P58+fp27cvAKWlpaxevTp87A033HAVZ0TaKy19SNLyrR91nfr7172eM2cOBQUFVFRUsHz5cs6ePRv+va0dXzoeBbUkLd/6Udep++7D4uJiRo4cCQRn6Onp6QCsXLkyvG9ubi7PPfdc+L2WPqQ5WvqQmGiL2+ma/E7P+lHXOXfuHDk5OVy6dIlVq1YBsHjxYqZMmUJ6ejojRowIN3R69NFHmT17NgMHDiQlJYVFixZxzz33xPZESdJTP+pWSpZ+v+pHnRgZGRmUlZXRs2fPqxpH/aibp37UIiLiJS19iEThcv2oq6qqElOQtGsKapEoqB+1tCUtfYiIeE4z6nbq5w9uSnQJIhIjmlGLiHhOQS0i4jktfUhM7Os/IKbjRXsfeGvEux/15s2befrpp9mwoek9vGVlZRQVFfGzn/2sVTVLx9RiUJvZN4Ai4EbgErDCOffTeBcm0l5duHCBrKwssrKafbZBpIlIlj4uAA875wYAI4DZZpYZ37JEWuZjP+qNGzfSv39/Ro8ezbp168LbFy9ezKxZs8jNzeWHP/whmzdvJi8vsqdARVqcUTvnaoCa0OtTZrYPSAf2xrk2kSvyrR/12bNnmTlzJps2beLb3/42U6dObfD57t272bZtG126dGHz5s1xOSfSPrXqYqKZZQBDgbfjUYxIawQCAUpLSyksLGTr1q1UV1c36UedmppKbW0tM2fOJBAIMGXKFPbubTrHKCkpoaioiCFDhpCTk8OxY8da3Y+6srKSvn370q9fP8yMadOmNfh84sSJdOnSJbYnQTqEiC8mmtn1wFpgrnPu82Y+nwXMAujTJ7oGLyKtUdeP+s033+SRRx4hNze3xX7Uly5donPnzk32qetHPX78+Abb33zzzVb1i77Svs117ROJREQzajO7hmBIv+ycW9fcPs65Fc65LOdcVlpaWixrFGmWb/2o+/fvz8GDB/nTn/4EEG5xKnK1Irnrw4BfAPucc9H1GJR2ry1up2vMt37UnTt3ZsWKFUyYMIGePXsyevToVt36J3I5LfajNrPRwFagguDteQD/5Jx783LHqB91U+2t36/6UceX+lE3r6P2o47kro9tgL7UTUQkQfRkokgULtePuvHFSJFYUFCLREH9qKUtqSmTiIjnFNQiIp5TUIuIeE5BLSLiOV1MlJiI9Vd/zf5fl++pESuJ7EddX35+Pu+//z7Tp09n3rx5EdciHYeCWiSBPvnkE7Zv386hQ4cSXYp4TEsfkrSSqR/1mTNneOCBBxg+fDhDhw7lN7/5DQC5ubl8+umnDBkyhK1bt8bw7Eh7ohm1JK1k6kf9+OOPc8cdd/Diiy9y4sQJsrOzGTduHOvXrycvL4/y8vL4nShJegpqSVqBQID58+dTWFhIXl4ePXr0aNKPGoKz2YKCAsrLy0lJSWH//v1NxiopKWHPnj2sWbMGCIb+1fSjBpg2bRorVqwIj79+/XqefvppIBjq1dXV6k8tEVFQS9JKpn7UzjnWrl3Lrbfe2mB7VVVVxGNLx6U1aklaydSPevz48Tz77LPUdat85513YnYepP3TjFpioi1up2ssmfpR//jHP2bu3LkMGjQI5xwZGRkt3rYnUkdBLUlr/PjxzXar27FjR4P3/fr1Y8+ePeH3Tz75JAAZGRnhIO3UqRNPPPEETzzxRJPxnnvuuYhruvPOO6msrGyyvUuXLuE7SeqrX4PI5WjpQ0TEc5pRi0RB/ailLSmoRaKgftTSlrT0ISLiOQW1iIjnFNQiIp5TUIuIeE4XEyUmlk7Ni+l4DxfH/2GQePejjsVxIqAZtYiI9xTUkrR87EedkZFBYWEh2dnZZGdnc+DAgfBnW7ZsYdSoUdx8883hLn333XdfuDc1wA9+8APWr18fi9Mj7YiWPiRp+daPuk63bt3YuXMnRUVFzJ07N9zTo6amhm3btlFZWcnEiROZPHkyM2bMYNmyZdx1112cPHmS7du3s3LlSvZ+cjo+J02SkmbUkrQCgQClpaUUFhaydetWqqurm/SjTk1Npba2lpkzZxIIBJgyZQp79+5tMlZJSQlFRUUMGTKEnJwcjh071up+1HXy8/PDP996663w9rvvvptOnTqRmZnJkSNHABg7diwHDhzg008/ZdWqVdx7772kpmr+JA3pL0KSlo/9qKFhT+r6r6+99toGv6/Offfdx8svv8zq1at58cUXW/W7pGPQjFqSlm/9qOsUFxeHf44cObLF/e+//36eeeYZAG677baI//3ScWhGLTHRFrfTNeZbP+o6586dIycnh0uXLjX48oDL6dWrFwMGDODuu++O6jxI+6eglqTlYz9qgNmzZ7No0aIG2375y182eH/69JcXC7/44ovwRU6R5mjpQySBSktL6d+/P3PmzKF79+6JLkc8pRm1SBQu14+6tV9WO27cOKqrq2NYmbRHCmqRKKgftbQlLX1IVJxzDW4xk/hxzuHQue7IFNQSldNHj3Dm7DmFdZw557jwxeccOlGb6FIkgbT0IVHZ+8Y6AK5P64WZcfz8hQRX1L4cOf4XAByOQydqefbt4wmuSBIpoqA2szuBnwIpwAvOuafiWpV4r/aLM7z7ykvh94m4j7o9+9uFbyS6BPFIi0sfZpYC/Bz4WyATyDezzHgXJiIiQZGsUWcDB5xzHznnzgOrgbviW5aIiNSJJKjTgX+v9/5waJuIiLQBa+mqvZlNAcY752aE3t8HZDvn5jTabxYwK/T2VuCD2JfbIfUEPkt0ESKXob/P2Pmmcy6tuQ8iuZh4GPhGvfdfBz5uvJNzbgWwIqry5LLMrMw5l5XoOkSao7/PthHJ0scuoJ+Z9TWzrwB/B+i7gkRE2kiLM2rn3AUzKwB+S/D2vBedc+/HvTIREQEivI/aOfcm8Gaca5HmaTlJfKa/zzbQ4sVEERFJLPX6EBHxnIJaRMRzasrkGTPrT/DJz3TAEbwVcr1zbl9CCxORhNGM2iNmVkjwEX0DdhK8NdKAVWa2MJG1iVyJmU1PdA3tmS4mesTM9gO3OedqG23/CvC+c65fYioTuTIzq3bO9Ul0He2Vlj78cgm4CTjUaHvv0GciCWNmey73EdCrLWvpaBTUfpkL/N7MPuTLRlh9gG8DBQmrSiSoFzAeaPwtBgZsb/tyOg4FtUeccxvN7BaCrWXTCf4PcBjY5Zy7mNDiRGADcL1zrrzxB2a2ue3L6Ti0Ri0i4jnd9SEi4jkFtYiI5xTUIiKeU1BLwpjZYjObH8VxPczsoRjWUWVmPVvY53Qrx4zq3ybSHAW1JKMeQKuC2oL09y5JSX+40mbM7IdmtsfM3jWzlxp9ttnMskKve5pZVej1bWa208zKQ8f2A54CvhXa9j9C+y0ws12hff5baFuGme0zs38B/kjDr5S7XI2vmdluM3s/9D2g9T9bamZ/NLPfm1laaNu3zGxj6JitoV4tjcf8kZntDdW2OopTJx2c7qOWNmFmtwH/DNzunPvMzP4T8KMIDn0Q+Klz7uXQo/QpwEJgoHNuSGjsXKAfwfvPDVhvZmOAaoJftDzdORfpDPwB59yfzawLsMvM1jrnjgHXAX90zj1sZv8VWETwIaQVwIPOuQ/NLAf4F+CORmMuBPo6586ZWY8I6xAJU1BLW7kDWOOc+wwgFIaRHPcW8M9m9nVgXSgQG++TG/rvndD76wkGdzVwyDm3oxV1/sjMJoVefyM0zjGCj/AXh7b/ClhnZtcDo4BX6tV0bTNj7gFeNrPXgNdaUYsIoKCWtmME27ZezgW+XIrrXLfROfd/zextYALwWzObAXzUzNhPOueWN9holgGcibhAs+8C44CRzrkvQk/bdb7M7i5U74m6mf0VTADGABOBH5vZbc65C5HWJaI1amkrvwf+s5l9FSC09FFfFfDXodeT6zaa2c3AR865nwHrgUHAKeCv6h37W+CB0AwXM0s3s69FUWN34HgopPsDI+p91qleXX8PbHPOfQ4cNLMpod9rZja4/oChC5jfcM79AfhHghdCr4+iNunANKOWNuGce9/MHgf+zcwuElymqKq3y9PAr83sPmBTve1TgWlmVgt8Avz30LLJ/zOz94B/dc4tMLMBwFuhJYjTwDSgtf1RNgIPhrrEfQDUXzI5A9xmZruBk6G6AH4APG9mjwLXEOwn/m6941KAX5lZd4Iz/2XOuROtrEs6OPX6EBHxnJY+REQ8p6UP6TBCFyUb35Vxn3OuIhH1iERKSx8iIp7T0oeIiOcU1CIinlNQi4h4TkEtIuI5BbWIiOf+P9FHlS0IhuCyAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Create centroids with kmeans for 2 clusters\n", "cluster_centers, _ = kmeans(fifa[scaled_features], 2)\n", "\n", "# Assign cluster labels and print cluster centers\n", "fifa['cluster_labels'], _ = vq(fifa[scaled_features], cluster_centers)\n", "print(fifa.groupby('cluster_labels')[scaled_features].mean())\n", "\n", "# Plot cluster centers to visualize clusters\n", "fifa.groupby('cluster_labels')[scaled_features].mean().plot(legend=True, kind='bar')\n", "\n", "# Get the name column of first 5 players in each cluster\n", "for cluster in fifa['cluster_labels'].unique():\n", " print(cluster, fifa[fifa['cluster_labels'] == cluster]['name'].values[:5])" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }