{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Comparative Evaluation and Combination of Audio Tempo Estimation Approaches (2011)\n", "### Results of the conference paper:\n", "\n", "### **\"Comparative Evaluation and Combination of Audio Tempo Estimation Approaches\"** , Zapata, Jose R.; Gómez, Emilia. AES 42nd International Conference: Semantic Audio, Ilmenau, Germany. pp. 198 - 207, Jul 2011.\n", "\n", "#### http://www.aes.org/e-lib/browse.cfm?elib=15964" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Index\n", "- [Tempo Ground truth histogram](#TH)\n", "- [Defining evaluation functions](#DEF)\n", "- [General Evaluation Results](#GR)\n", " - [Sorted General Evaluation results (by Metric 1)](#SM1)\n", " - [Sorted General Evaluation results (by Metric 2)](#SM2)\n", " - [Error relations](#ER)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/vnd.plotly.v1+html": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Importing libraries\n", "% matplotlib inline\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import plotly.offline as pyo # offline plotly plots\n", "from plotly.offline import init_notebook_mode #Run plotly in jupyter\n", "import plotly.graph_objs as go # Plotly graphs objects\n", "init_notebook_mode() # Init plotly inside jupyter notebook" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SONGGROUNDTRUTHEssentiaSonicBasicqmtempoAubioBeatrootEllisTzanetakisMIRTempo...Alo_specDix_aucoDix_induDix_tracKlapuriScheirerTzan_histTzan_mmulTzan_msumUhle
0Abba\\Gold-GreatestHits\\20sec\\01-DancingQueen.wav101.010084201.0100.3103.36101.332721100.000000135.135135100.0100.748146...80.2881152202.772201.439101101.819019810010067
1Abba\\Gold-GreatestHits\\20sec\\02-KnowingMe,Know...105.755105213.0105.7107.67107.666016109.090909108.695652106.5107.416010...106.7432216106.400212.720108108.0350107107107107
2Abba\\Gold-GreatestHits\\20sec\\03-TakeAChanceOnM...107.142857213.0106.6107.67107.666016105.263158107.142857107.5107.171263...80.2881214106.751214.826108220.0000106105107106
3Abba\\Gold-GreatestHits\\20sec\\04-MammaMia.wav136.617092136.0136.9139.67135.999178136.363636136.363636137.0137.089537...68.3679140136.428137.33513891.7879135135135135
4Abba\\Gold-GreatestHits\\20sec\\05-LayAllYourLove...133.636421132.0132.5136.00132.512019136.36363666.371681132.5132.863812...133.1982133132.893132.921133132.9380132132132132
\n", "

5 rows × 25 columns

\n", "
" ], "text/plain": [ " SONG GROUNDTRUTH Essentia \\\n", "0 Abba\\Gold-GreatestHits\\20sec\\01-DancingQueen.wav 101.010084 201.0 \n", "1 Abba\\Gold-GreatestHits\\20sec\\02-KnowingMe,Know... 105.755105 213.0 \n", "2 Abba\\Gold-GreatestHits\\20sec\\03-TakeAChanceOnM... 107.142857 213.0 \n", "3 Abba\\Gold-GreatestHits\\20sec\\04-MammaMia.wav 136.617092 136.0 \n", "4 Abba\\Gold-GreatestHits\\20sec\\05-LayAllYourLove... 133.636421 132.0 \n", "\n", " SonicBasic qmtempo Aubio Beatroot Ellis Tzanetakis \\\n", "0 100.3 103.36 101.332721 100.000000 135.135135 100.0 \n", "1 105.7 107.67 107.666016 109.090909 108.695652 106.5 \n", "2 106.6 107.67 107.666016 105.263158 107.142857 107.5 \n", "3 136.9 139.67 135.999178 136.363636 136.363636 137.0 \n", "4 132.5 136.00 132.512019 136.363636 66.371681 132.5 \n", "\n", " MIRTempo ... Alo_spec Dix_auco Dix_indu Dix_trac Klapuri \\\n", "0 100.748146 ... 80.2881 152 202.772 201.439 101 \n", "1 107.416010 ... 106.7432 216 106.400 212.720 108 \n", "2 107.171263 ... 80.2881 214 106.751 214.826 108 \n", "3 137.089537 ... 68.3679 140 136.428 137.335 138 \n", "4 132.863812 ... 133.1982 133 132.893 132.921 133 \n", "\n", " Scheirer Tzan_hist Tzan_mmul Tzan_msum Uhle \n", "0 101.8190 198 100 100 67 \n", "1 108.0350 107 107 107 107 \n", "2 220.0000 106 105 107 106 \n", "3 91.7879 135 135 135 135 \n", "4 132.9380 132 132 132 132 \n", "\n", "[5 rows x 25 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Reading tempo estimations results\n", "df = pd.read_excel(\"data/AES42_Tempo_2011.xlsx\")\n", "number_songs = len(df)\n", "# only the first 5 rows\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Tempo Ground truth histogram " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAGeCAYAAABSL10jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X+4XVV95/H3B5IoGiwoGgIhgEKtVCyaSLV2bKKtP6lSW63WcdBSsVNrbctU1LFV+0tTH7Rjp9p21Jp2qpGqKM1g/UGNWketuVQNyKAUCT8TqgISRCDhO3+cfcvN5d6bvZOz7zk3eb+e5z73nH3OWft7Thb7flhn7bVTVUiSJElq76BRFyBJkiQtNIZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREvSiCV5b5I/nOd9rkly7ZDb/FiSM4bZpiSNK0O0pH2S5CeT/N8ktyT5bpLPJ3nsqOuaKsnqJBuT3JTk5iRfT/JHSQ4fdW17kuTFSf55CO1UkhP24fWbkvzKtG27BfGqenpVre+7FkkaB4ZoSXstyQOAjcCfAQ8EjgbeCNwxyrqmSvITwCbg88CPVNVhwNOAncCPzfKaRfNW4BAkOXjUNYyLhfZvJ2nhMkRL2hc/DFBV76+qXVV1e1V9oqq+BpDkoCSvS7I1yY1J/ibJDzWPHdeMSJ6R5Ook307y3ycbTnJIkvXN6PFlSV41ddQzyTlJrktya5LLkzx5lhr/BPjrqnpTVW1v6r26ql5fVZuatl7cjKC/Lcl3gTfsofZ7TYVIclWSn25uvyHJec1rbk1yaZLVU5776CQXN499ALjvTIUneQTwF8Djk+xIcnOz/b1J3pnkwiS3AWunjxRPHcFO8tlm81ebdn5xyvPObt7fDUleMstn2MrUGpKckOQzzTcU327e56y1JHlpkiuabzMuSHLUlHaf0vwb35LkHU27k/uZ6d/uYUn+Kcl3mn3/XZLDprR3VZLfSfK1JLcleXeSZRlMR7k1yacWwrcUkkbLEC1pX3wD2NWE3afPEDxe3PysBR4KLAX+57Tn/CTwcODJwO81wRHg9cBxzet+BvjPky9I8nDg14HHVtWhwFOBq6YXl+T+wOOBD7V4Lz8OXAk8BPijlrXP5VnABuAw4ILJ1yZZAnwE+FsGo/d/D/z8TA1U1WXArwJfqKqlzSj6pF9q6jwUmHO6R1U9sbn5Y007H2juHwn8EINvEM4E/nyI4fEPgE8AhwMrGHxbMWMtSZ4EvAl4HrAc2MrgsyPJEcAHgdcADwIuB35i2r6m/9ulae8o4BHAMcAbpr3m5xn0qx8Gfhb4GPBa4AgGfxt/Yx/fv6T9nCFa0l6rqu8xCMEF/C/g35tRxGXNU14IvLWqrqyqHQyC0POnfeX+xmYE+6vAV7lnisXzgD+uqpuq6lrg7VNeswu4D3BSksVVdVVV/dsMJR7O4Di3bXJDkj/JYF70bUleN+W511fVn1XVzqq6vWXtc/nnqrqwqnYxCMyT7+txwGLgT6vqrqr6IPDllm1O9dGq+nxV3V1VP9iL1wPcBfx+U8eFwA4G/0Mzm7c3n93Nzaj4xj20fSxwVFX9oKrmCvovBN5TVRdX1R0MPuvHJzkOeAZwaVV9uKp2MugH26a9frd/u6q6oqo+WVV3VNW/A28Ffmraa/6sqrZX1XXA54AvVdW/Nvs/H3j0HPVKkiFa0r6pqsuq6sVVtQJ4JIPRvz9tHj6KwajipK3AImDZlG1TA9H3GYz4Tr72mimP/cftqroC+E0Go4s3Jtkw9ev/KW4C7mYwujn52lc1I7rnN7Xcq/0Otc9l+vu6bxPAjwKuq6qa1nZX0+vdG99pgumkqZ//TH6jqg6b/AFOm+O5r2IwIvwvzXSWX57jubt91s3/tHyHwQj5bv2g+dymryqy22eR5CFNn7guyfeA/81ghHmq7VNu3z7D/bk+B0kyREsanqr6f8B7GYRpgOsZjEZOWsnghL7t7NkNDKYBTDpm2r7eV1U/2bRfwLoZ6rkN+BLwnDblT7s/V+23AfebfCCDE/se3GIfMHhfRyfJtLbb1jXb9t1qYjBVY2SqaltVvbSqjgJeBrwjs6/Isdtn3UzDeRBwHdP6QfO5rZj2+umfxZuabY+qqgcwmAoUJGmIDNGS9lqSH2lOTFvR3D8GeAHwxeYp7wd+K8nxSZYCfwx8YNro52zOA16T5PAkRzOYAz2534cneVKS+wA/YDByuGuWdl4F/HKSVyd5SPP6FcDxe9j/XLV/g8HI8jOTLAZex2B6SRtfYBDGfyPJoiTPAU6d4/nbgRXNXOq5fAV4TpL7NWH1zBnaeWjLGvdZkudO9gsG3wgU9/wbTa/lfcBLkpzS/Jv+MYPpFVcB/wc4OcnpzUj+y9nz/yAcymBqys1N3/mdYbwnSZrKEC1pX9zK4KSuL2WwSsQXgUuAs5vH38NgPvBngW8xCLyvaNn27zP42v5bwKcYnFw2uXTefYA3A99mMG3iIQxOCruXZi7uk4AnAt9o5vL+I4Nl7/5sjv3PWntV3QL8GvAuBqOlt3HvKQYzqqo7GYyMv5hBuPxF4MNzvOSfgEuBbUm+Pcfz3gbcySCgrgf+btrjbwDWN/OZn9em1n30WAb9YgeDEytfWVXfmqmWqroI+F0GJ4DeADwMeD5AVX0beC6DVVa+A5wEbGbuZRTfCDwGuIVBCJ/r85WkvZLdp+VJ0nhK8l+B51fV9BPEdABJchCD/2F5YVV9etT1SDpwORItaSwlWZ7kCRms1/xwBqPb54+6Ls2/JE9Nclgz1eO1DOY3f3EPL5OkXnllJ0njagnwlwzmLt/MYN3gd4y0Io3K4xnMm14CfB04vVmGUJJGxukckiRJUkdO55AkSZI6MkRLkiRJHRmiJUmSpI4M0ZIkSVJHhmhJkiSpI0O0JEmS1JEhWpIkSerIEC1JkiR1ZIiWJEmSOjJES5IkSR0ZoiVJkqSODNGSJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREuSJEkdGaIlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJYkSZI6MkRLkiRJHRmiJUmSpI4M0ZIkSVJHhmhJkiSpo0WjLqCNI444oo477rhRl6E53Hbbbdz//vcfdRlaAOwrasN+orbsK2qrTV+ZmJj4dlU9uE17CyJEH3fccWzevHnUZWgOmzZtYs2aNaMuQwuAfUVt2E/Uln1FbbXpK0m2tm3P6RySJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREuSJEkdGaIlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJaGbPmKlSQZ6s/yFStH/bYkSdIUi0ZdgLS/2XbdNRx7zsahtrl13WlDbU+SJO0bR6IlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUkSFa0j7rsiLJxMSEq5FIkhY8V+eQtM+6rEiy5Mide3yuq5FIksadI9GSJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREuSJEkdGaIlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJYkSZI6MkRLkiRJHRmiJUmSpI4M0ZIkSVJHi/psPMlVwK3ALmBnVa1O8kDgA8BxwFXA86rqpj7rkCRJkoZpPkai11bVKVW1urn/auCiqjoRuKi5L0mSJC0Yo5jO8WxgfXN7PXD6CGqQJEmS9lqqqr/Gk28BNwEF/GVV/VWSm6vqsCnPuamqDp/htWcBZwEsW7Zs1YYNG3qrU/tux44dLF26dNRljIWJiQmWHHnCUNu8c9sVrFq1aqhtDlOX97zsENh++9zPGff3q/55TFFb9hW11aavrF27dmLK7Ik59R2ij6qq65M8BPgk8ArggjYheqrVq1fX5s2be6tT+27Tpk2sWbNm1GWMhSQce87Goba5dd1p9Pnf6r7q8p7PPnkn526Z+3SMcX+/6p/HFLVlX1FbbfpKktYhutfpHFV1ffP7RuB84FRge5LlAM3vG/usQZIkSRq23kJ0kvsnOXTyNvAU4BLgAuCM5mlnAB/tqwZJkiSpD30ucbcMOD/J5H7eV1X/mOTLwHlJzgSuBp7bYw2SJEnS0PUWoqvqSuDHZtj+HeDJfe1XkiRJ6ptXLNSCs3zFSpIM7Wf5ipWjfkuSJGmB6fWKhVIftl13zVBXv9i67rShtSVJkg4MjkRLkiRJHRmiJUmSpI4M0ZIkSVJHhmhJkiSpI0O0JEmS1JEhWpIkSerIEC1JkiR1ZIiWJEmSOjJES5IkSR0ZoiVJkqSODNGSJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIlhaCgxeTZGg/y1esHPU7kiRpQVs06gIktbDrLo49Z+PQmtu67rShtSVJ0oHIkWhJkiSpI0O0JEmS1JEhWpIkSerIEC1JkiR1ZIiWJEmSOjJES5IkSR0ZoiVJkqSODNGSJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREuSJEkdGaIlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJYkSZI6MkRLkiRJHRmiJUmSpI4M0ZIkSVJHhmhJkiSpI0O0JEmS1JEhWpIkSerIEC1JkiR1ZIiWJEmSOuo9RCc5OMm/JtnY3D8+yZeSfDPJB5Is6bsGSZIkaZjmYyT6lcBlU+6vA95WVScCNwFnzkMNkiRJ0tD0GqKTrACeCbyruR/gScAHm6esB07vswZJkiRp2FJV/TWefBB4E3Ao8N+AFwNfrKoTmsePAT5WVY+c4bVnAWcBLFu2bNWGDRt6q1P7bseOHSxdunRe9jUxMcGSI08YWnt3bruCVatWDa29YdcHgxr3l/e87BDYfvvcz7lz278Bwz02LV68hEc96uShtqn+zOcxRQubfUVttekra9eunaiq1W3a6y1EJzkNeEZV/VqSNQxC9EuAL0wL0RdW1Zx/2VavXl2bN2/upU4Nx6ZNm1izZs287CsJx56zcWjtbV13GsP872DY9cGgxv3lPZ998k7O3bJozucM+/1OttnnoIGGaz6PKVrY7Ctqq01fSdI6RM/9l2zfPAF4VpJnAPcFHgD8KXBYkkVVtRNYAVzfYw2SJEnS0PU2J7qqXlNVK6rqOOD5wD9V1QuBTwO/0DztDOCjfdUgSZIk9WEU60SfA/x2kiuABwHvHkENkiRJ0l7rczrHf6iqTcCm5vaVwKnzsV9JkiSpD16xUJIkSerIEC1JkiR1ZIiWJEmSOjJES5IkSR0ZoiVJkqSO5mV1Dklj5uDFJBl1FZIkLVidQnSSg4ClVfW9nuqRNB923TX0y4hLknQg2eN0jiTvS/KAJPcHvg5cnuR3+i9NkiRJGk9t5kSf1Iw8nw5cCKwEXtRrVZIkSdIYaxOiFydZzCBEf7Sq7gKq37IkSZKk8dUmRP8lcBVwf+CzSY4FnBMtSZKkA9YeTyysqrcDb5+yaWuStf2VJEmSJI23NicWLkvy7iQfa+6fBJzRe2WSJEnSmGozneO9wMeBo5r73wB+s6+CJEmSpHHXJkQfUVXnAXcDVNVOYFevVUmSJEljrE2Ivi3Jg2hW5EjyOOCWXquSJEmSxlibKxb+NnAB8LAknwceDPxCr1Vpv7F8xUq2XXfNqMuYm5fAliRJHbVZnePiJD8FPBwIcHmzVrS0R9uuu2aol5eGHi4x7SWwJUlSR21W53g5sLSqLq2qS4ClSX6t/9IkSZKk8dRmTvRLq+rmyTtVdRPw0v5KkiRJksZbmxB9UKZMGE1yMLCkv5IkSZKk8dbmxMKPA+cl+QsGK3T8KvCPvVYlSZIkjbE2Ifoc4GXAf2VwYuEngHf1WZQkSZI0ztqsznE38M7mR5IkSTrg7TFEJ3kC8Abg2Ob5AaqqHtpvaZIkSdJ4ajOd493AbwETeLlvSZIkqVWIvqWqPtZ7JZIkSdIC0SZEfzrJW4APA3dMbqyqi3urSpIkSRpjbUL0jze/V0/ZVsCThl+OJEmSNP7arM6xdj4KkSRJkhaKNiPRJHkm8KPAfSe3VdXv91WUJEmSNM72eNnv5kqFvwi8gsHyds9lsNydJEmSdEDaY4gGfqKq/gtwU1W9EXg8cEy/ZUmSJEnjq02Ivr35/f0kRwF3Acf3V5IkSZI03trMid6Y5DDgLcDFDFbmeFevVUmSJEljrE2I/pOqugP4UJKNDE4u/EG/ZUmSJEnjq810ji9M3qiqO6rqlqnbJEmSpAPNrCPRSY4EjgYOSfJoBitzADwAuN881CZJkiSNpbmmczwVeDGwAjiXe0L0rcBr+y1LkiRJGl+zhuiqWg+sT/LzVfWheaxJkiRJGmtt5kSvSPKADLwrycVJntJ7ZZIkSdKYahOif7mqvgc8BXgI8BLgzb1WJUmSJI2xNiF6ci70M4C/rqqvTtkmSZIkHXDahOiJJJ9gEKI/nuRQ4O5+y5IkSZLGV5uLrZwJnAJcWVXfT/IgBlM6JEmSpAPSHkN0Vd2dZDtwUpI2oVuSJEnar+0xFCdZB/wi8HVgV7O5gM/2WJckSZI0ttqMLJ8OPLyq7ujScJL7Mgja92n288Gqen2S44ENwAOBi4EXVdWd3cqWJEmSRqfNiYVXAov3ou07gCdV1Y8xmFP9tCSPA9YBb6uqE4GbGMy5liRJkhaMNiPR3we+kuQiBsEYgKr6jbleVFUF7GjuLm5+CngS8EvN9vXAG4B3dqpakiRJGqEMsu4cT0jOmGl7c1nwPb32YGACOAH4c+AtwBer6oTm8WOAj1XVI2d47VnAWQDLli1btWHDhj3tTiO0Y8cOli5deq/tExMTLDnyhKHu685tVwy1zXFvr482R9neskNg++3Da6+tO7ddwapVq4bapvoz2zFFms6+orba9JW1a9dOVNXqNu3tMUQPQ5LDgPOB32NwwZapIfrCqjp5rtevXr26Nm/e3Hud2nubNm1izZo199qehGPP2TjUfW1dd9pQ2xz39vpoc5TtnX3yTs7dMveXYH19hvNxvNNwzHZMkaazr6itNn0lSesQPetfsiTnVdXzkmxhMA1jN1X1qDY7aJ57c5JNwOOAw5IsqqqdwArg+rbtSJIkSeNgruGgVza/T9ubhpM8GLirCdCHAD/N4KTCTwO/wGCFjjOAj+5N+5IkSdKozBqiq+qG5vfWvWx7ObC+mRd9EHBeVW1M8nVgQ5I/BP4VePdeti9JkiSNRG9XIKyqrwGPnmH7lcCpfe1XkiRJ6lubdaIlSZIkTTFriG7WhZ687LckSZKkxlzTOZYn+SngWUk2AJn6YFVd3GtlkiRJ0piaK0T/HvBqBsvQvXXaY5NXHpQkSZIOOHOtzvFB4INJfreq/mAea5IkSZLG2h5X56iqP0jyLOCJzaZNVTXcS4lJkiRJC8geV+dI8iYGF175evPzymabJEmSdEBqs070M4FTqupugCTrGVwk5TV9FiZJkiSNq7brRB825fYP9VGIJEmStFC0GYl+E/CvST7NYJm7J+IotCRJkg5gbU4sfH+STcBjGYToc6pqW9+FSZIkSeOqzUg0VXUDcEHPtUiSJEkLQts50ZIkSZIahmhJkiSpozlDdJKDklwyX8VIkiRJC8GcIbpZG/qrSVbOUz2SJEnS2GtzYuFy4NIk/wLcNrmxqp7VW1WSJEnSGGsTot/YexWSJEnSAtJmnejPJDkWOLGqPpXkfsDB/ZcmSZIkjac9rs6R5KXAB4G/bDYdDXykz6IkSZKkcdZmibuXA08AvgdQVd8EHtJnUZIkSdI4axOi76iqOyfvJFkEVH8lSZIkSeOtTYj+TJLXAock+Rng74F/6LcsSZIkaXy1CdGvBv4d2AK8DLgQeF2fRUmSJEnjrM3qHHcnWQ98icE0jsuryukckiRJOmDtMUQneSbwF8C/AQGOT/KyqvpY38VJkiRJ46jNxVbOBdZW1RUASR4G/B/AEC1JkqQDUps50TdOBujGlcCNPdUjSZIkjb1ZR6KTPKe5eWmSC4HzGMyJfi7w5XmoTZKG5+DFJBlac0cefQw3XHv10NqTJC0sc03n+Nkpt7cDP9Xc/nfg8N4qkqQ+7LqLY8/ZOLTmtq47bWhtSZIWnllDdFW9ZD4LkSRJkhaKNqtzHA+8Ajhu6vOr6ln9lSVJkiSNrzarc3wEeDeDqxTe3W85kiRJ0vhrE6J/UFVv770SSZIkaYFoE6L/R5LXA58A7pjcWFUX91aVJEmSNMbahOiTgRcBT+Ke6RzV3JckSZIOOG1C9M8BD62qO/suRpIkSVoI2lyx8KvAYX0XIkmSJC0UbUailwH/L8mX2X1OtEvcSZIk6YDUJkS/vvcqJEmSpAVkjyG6qj4zH4VIkiRJC0WbKxbeymA1DoAlwGLgtqp6QJ+FSZIkSeOqzUj0oVPvJzkdOLW3iiRJkqQx12Z1jt1U1UdwjWhJkiQdwNpM53jOlLsHAau5Z3qHJEmSdMBpszrHz065vRO4Cnh2L9VIkiRJC0CbOdEvmY9CJEmSpIVi1hCd5PfmeF1V1R/0UI8kSZI09uY6sfC2GX4AzgTO2VPDSY5J8ukklyW5NMkrm+0PTPLJJN9sfh++j+9BkiRJmlezhuiqOnfyB/gr4BDgJcAG4KEt2t4JnF1VjwAeB7w8yUnAq4GLqupE4KLmviRJkrRgzLnEXTNq/IfA1xhM/XhMVZ1TVTfuqeGquqGqLm5u3wpcBhzN4KTE9c3T1gOn70P9kiRJ0rxL1cyr1SV5C/AcBqPQf15VO/Z6J8lxwGeBRwJXV9VhUx67qaruNaUjyVnAWQDLli1btWHDhr3dvebBjh07WLp06b22T0xMsOTIE4a6rzu3XTHUNse9vT7aHGV7yw6B7bcPr722+njPq1atGlp72t1sxxRpOvuK2mrTV9auXTtRVavbtDdXiL4buIPBtIypTwqDEwtbXfY7yVLgM8AfVdWHk9zcJkRPtXr16tq8eXOb3WlENm3axJo1a+61PQnHnrNxqPvauu60obY57u310eYo2zv75J2cu2XuhYEWymc42/FT+262Y4o0nX1FbbXpK0lah+hZ/5JVVeerGc5QyGLgQ8DfVdWHm83bkyyvqhuSLAf2ODVEkiRJGif7HJRnkyTAu4HLquqtUx66ADijuX0G8NG+apAkSZL60OaKhXvrCcCLgC1JvtJsey3wZuC8JGcCVwPP7bEGSZIkaeh6C9FV9c8M5k/P5Ml97VeS5sXBixl84TYcRx59DDdce/XQ2pMk9avPkWhJ2n/tumvoJypKkhaO3uZES5IkSfsrQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJYkSZI6MkRLkiRJHRmiJUmSpI4M0ZIkSVJHhmhJkiSpI0O0JEmS1JEhWpIkSerIEC1JkiR1ZIiWJEmSOjJES5IkSR0ZoiVJkqSODNGSJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREvSODh4MUmG+rN8xcpRvytJ2m8tGnUBkiRg110ce87GoTa5dd1pQ21PknQPR6IlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJYkSZI6MkRLkiRJHRmiJUmSpI4M0drN8hUr9+rywhMTEzNulyRJ2h952W/tZtt11+zVpYeXHLlzxtd52WFJkrQ/ciRakiRJ6sgQLUmSJHVkiJYkSZI6MkRLkiRJHRmiJWl/dfDivVptZ7af5StWjvodSdLYcHUOSdpf7bprr1bbmY2r7UjSPRyJliRJkjoyREuSJEkdGaIlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUUW8hOsl7ktyY5JIp2x6Y5JNJvtn8Pryv/UuSJEl96XMk+r3A06ZtezVwUVWdCFzU3JckSZIWlN5CdFV9FvjutM3PBtY3t9cDp/e1f0mSJKkvqar+Gk+OAzZW1SOb+zdX1WFTHr+pqmac0pHkLOAsgGXLlq3asGFDb3XqHhMTEyw58oTOr1t2CGy//d7b79x2xV61N5dhtznu7fXR5ijbm62v7G17be1Pn+Go2rxz2xWsWrVqaO3NZceOHSxdunRe9qWFzb6ittr0lbVr105U1eo27Y1tiJ5q9erVtXnz5t7q1D2S7NVlgs8+eSfnbrn3VeS3rjttqJcd7qPNcW+vjzZH2d5sfWVv22trf/oMR9Xm1nWn0effjKk2bdrEmjVr5mVfWtjsK2qrTV9J0jpEz/fqHNuTLAdoft84z/uXJEmS9tl8h+gLgDOa22cAH53n/UuSJEn7rM8l7t4PfAF4eJJrk5wJvBn4mSTfBH6muS9JkiQtKHNPTNwHVfWCWR56cl/7lCRJkuaDVyyUJEmSOjJES5IkSR0ZoiVJkqSODNGSJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREuSJEkdGaIlSZKkjgzRkiRJUkeGaEmSJKkjQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJYkSZI6MkQvcMtXrCTJ0H4kab7MdfyamJjofPxavmLlqN+SpAPIolEXoH2z7bprOPacjUNrb+u604bWliTNZa7j15Ijd3Y+tnn8kjSfHImWJEmSOjJES5IkSR0ZoiVJkqSODNGSJElSR4ZoSZIkqSNDtCRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjryioWSpHYOXkySUVcxux7qO/LoY7jh2quH2qak/YMhWpLUzq67Ol+Key5Dv0z3kOsDLyUuaXZO55AkSZI6MkRLkiRJHRmiJUmSpI4M0ZIkSVJHhmhJkiSpI0O0JEmS1JEhWpIkSerIEC1JkiR1ZIiWJEmSOjJES5IkSR0ZoufZ8hUrSTK0H0lSjw5ePNRj9vIVK0f9jiQNyaJRF3Cg2XbdNRx7zsahtbd13WlDa0uSNM2uuzxmS5qRI9GSJElSR4ZoSZIkqSNDtCRJktSRIXoOwz4J0BMBJUnjbth/+zyZcvz0kW8CXEmAAAAG/UlEQVQW3eeQA67feGLhHIZ9EiB4Uokkabx5Avz+r698c6D1G0eiJUmSpI5GEqKTPC3J5UmuSPLqUdQgSZIk7a15D9FJDgb+HHg6cBLwgiQnzXcdkiRJ0t4axUj0qcAVVXVlVd0JbACePYI6JEmSpL2SqprfHSa/ADytqn6luf8i4Mer6tenPe8s4Kzm7sOBy+e1UHV1BPDtURehBcG+ojbsJ2rLvqK22vSVY6vqwW0aG8XqHDOt83avJF9VfwX8Vf/laBiSbK6q1aOuQ+PPvqI27Cdqy76itobdV0YxneNa4Jgp91cA14+gDkmSJGmvjCJEfxk4McnxSZYAzwcuGEEdkiRJ0l6Z9+kcVbUzya8DHwcOBt5TVZfOdx0aOqfeqC37itqwn6gt+4raGmpfmfcTCyVJkqSFzisWSpIkSR0ZoiVJkqSODNHqLMlVSbYk+UqSzc22Byb5ZJJvNr8PH3Wdmn9J3pPkxiSXTNk2Y9/IwNuTXJHka0keM7rKNd9m6StvSHJdc2z5SpJnTHnsNU1fuTzJU0dTteZbkmOSfDrJZUkuTfLKZrvHFe1mjr7S23HFEK29tbaqTpmy3uKrgYuq6kTgoua+DjzvBZ42bdtsfePpwInNz1nAO+epRo2H93LvvgLwtubYckpVXQiQ5CQGKzn9aPOadyQ5eN4q1SjtBM6uqkcAjwNe3vQHjyuabra+Aj0dVwzRGpZnA+ub2+uB00dYi0akqj4LfHfa5tn6xrOBv6mBLwKHJVk+P5Vq1GbpK7N5NrChqu6oqm8BVwCn9lacxkZV3VBVFze3bwUuA47G44qmmaOvzGafjyuGaO2NAj6RZKK5PDvAsqq6AQYdGXjIyKrTuJmtbxwNXDPledcy9wFPB4Zfb76Gf8+UaWH2FZHkOODRwJfwuKI5TOsr0NNxxRCtvfGEqnoMg6/NXp7kiaMuSAtSZtjmmpsHtncCDwNOAW4Azm2221cOcEmWAh8CfrOqvjfXU2fYZl85gMzQV3o7rhii1VlVXd/8vhE4n8HXH9snvzJrft84ugo1ZmbrG9cCx0x53grg+nmuTWOkqrZX1a6quhv4X9zz1ap95QCWZDGDUPR3VfXhZrPHFd3LTH2lz+OKIVqdJLl/kkMnbwNPAS5hcOn2M5qnnQF8dDQVagzN1jcuAP5Lczb944BbJr+e1YFp2tzVn2NwbIFBX3l+kvskOZ7BSWP/Mt/1af4lCfBu4LKqeuuUhzyuaDez9ZU+jyvzftlvLXjLgPMHfZVFwPuq6h+TfBk4L8mZwNXAc0dYo0YkyfuBNcARSa4FXg+8mZn7xoXAMxiczPF94CXzXrBGZpa+sibJKQy+Ur0KeBlAVV2a5Dzg6wzOwH95Ve0aRd2ad08AXgRsSfKVZttr8biie5utr7ygr+OKl/2WJEmSOnI6hyRJktSRIVqSJEnqyBAtSZIkdWSIliRJkjoyREuSJEkdGaIlqWdJHpTkK83PtiTXTbm/ZIR1XZtkS1PHliQ/22xflGRXs/2SJB9IckizvZL89ZQ2liT5bpKPjOp9SNIoGKIlqWdV9Z2qOqWqTgH+Anjb5P2qunPE5f2npq7nA386ZfutzfaTm/svbX5/D3hMkvs095/KYJ1eSTqgGKIlaYSSnJHkX5pR33ckOagZ8b05yVuSXJzk40l+PMlnklyZ5BnNa38lyfnN45cned2Udl/VjCJfkuQVLUp5AHDT9I01uJjA54ATJjcBHwee3tx/AfD+ffgIJGlBMkRL0ogkeSSDy9D+RDPqu4jBiDDADwGfqKrHAHcCbwCezODKbL8/pZlTm9c8BvilJKckORV4YfPY44FfS/KoWcr4XJJLgYuA101/MMli4GnAlimbNzC4XO79gEcAEx3fuiQteF72W5JG56eBxwKbkwAcAlzTPHZ7VX2yub0FuKWqdibZAhw3pY2PV9VNAM285J8E7gN8qKq+P23712ao4T9V1c1Jfhj4eJIfZRDaD51y6dzPAO+dfEFVXdw8/wXAP+zD+5ekBcsQLUmjE+A9VfW7u21MFjEIspPuBu6YcnvqsbumtVlNu51U1TeSfBf4EQZhe3JO9PS6Jm0E/oRBOD+66/4kaaFzOockjc6ngOclOQL+YxWPlR3beEqSw5qpFc8GPg98Fvi5ZkWNpc32z83VSJIjgZW0P0nwXcDrq+qyjvVK0n7BkWhJGpGq2pLkjcCnkhwE3AX8KnB9h2b+GXgf8DDgb6vqKwBJ3g98uXnOO6tqyyyv/1ySXcBi4L9V1benjTjPVvvVwP/sUKck7VcyOPFakrTQJPkV4JFV9ZujrkWSDjRO55AkSZI6ciRakiRJ6siRaEmSJKkjQ7QkSZLUkSFakiRJ6sgQLUmSJHVkiJYkSZI6+v/UyHbP28l98AAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df[\"GROUNDTRUTH\"].hist(bins=30,ec=\"black\",figsize=(12,6))\n", "plt.suptitle(\"Songs Ground truth Histogram\")\n", "plt.xlabel(\"Tempo BPM\")\n", "plt.ylabel(\"Number of instances\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Defining evaluation functions" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# Tempo evaluation measure 1 \n", "def matchingFunction_1(computed,truth,accPerc):\n", " #normally accPerc = 4\n", " if ((computed >= truth*(1-accPerc/100))&(computed <= truth*(1+accPerc/100))):\n", " MF_1 = 1\n", " else:\n", " MF_1 = 0;\n", " return MF_1" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# Tempo evaluation measure 2 \n", "def matchingFunction_2(computed,truth,accPerc):\n", " # normally accuracy 4%\n", " if ( ((computed>=truth*(1-accPerc/100))&(computed<=truth*(1+accPerc/100))) |\n", " ((computed>=2*truth*(1-accPerc/100))&(computed<=2*truth*(1+accPerc/100))) |\n", " ((computed>=0.5*truth*(1-accPerc/100))&(computed<=0.5*truth*(1+accPerc/100))) |\n", " ((computed>=3*truth*(1-accPerc/100))&(computed<=3*truth*(1+accPerc/100))) |\n", " ((computed>=(1/3)*truth*(1-accPerc/100))&(computed<=(1/3)*truth*(1+accPerc/100)))):\n", " MF_2 = 1\n", " else:\n", " MF_2 = 0\n", " return MF_2 " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "cols_names = list(df)\n", "del cols_names[0:2] # only Tempo estimator systems\n", "evaluation_1 = df.loc[:, cols_names]\n", "evaluation_2 = df.loc[:, cols_names]" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "for n in cols_names:\n", " evaluation_1[n] = [matchingFunction_1(df.loc[idx,n],df.loc[idx,\"GROUNDTRUTH\"],4) for idx in range(len(df))]\n", " evaluation_2[n] = [matchingFunction_2(df.loc[idx,n],df.loc[idx,\"GROUNDTRUTH\"],4) for idx in range(len(df))]\n", "metric_1 = (evaluation_1.sum())*100/number_songs\n", "metric_2 = (evaluation_2.sum())*100/number_songs " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## General Evaluation Results" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Metric 1Metric 2
Essentia24.516183.4409
SonicBasic24.731250.7527
qmtempo43.225880.4301
Aubio39.354867.3118
Beatroot23.225867.957
Ellis45.591480.6452
Tzanetakis25.591466.4516
MIRTempo30.967765.5914
IBT35.91479.7849
BeatIt60.430178.2796
Mpeg7-xm48.387170.5376
jAudio5.1612932.2581
Alo_corr23.440958.2796
Alo_spec37.419468.6022
Dix_auco16.989276.9892
Dix_indu28.602262.5806
Dix_trac19.354868.8172
Klapuri58.494691.1828
Scheirer37.849569.4624
Tzan_hist21.290347.7419
Tzan_mmul18.709741.0753
Tzan_msum27.526952.4731
Uhle41.935571.828
" ], "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "final_evaluation = pd.concat([metric_1,metric_2],axis=1)\n", "final_evaluation.columns = ['Metric 1','Metric 2']\n", "final_evaluation.style.highlight_max() # High lighting the maximun value per column" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "data": [ { "name": "Metric 1", "type": "bar", "x": [ "Essentia", "SonicBasic", "qmtempo", "Aubio", "Beatroot", "Ellis", "Tzanetakis", "MIRTempo", "IBT", "BeatIt", "Mpeg7-xm", "jAudio", "Alo_corr", "Alo_spec", "Dix_auco", "Dix_indu", "Dix_trac", "Klapuri", "Scheirer", "Tzan_hist", "Tzan_mmul", "Tzan_msum", "Uhle" ], "y": [ 24.516129032258064, 24.731182795698924, 43.225806451612904, 39.354838709677416, 23.225806451612904, 45.59139784946237, 25.591397849462364, 30.967741935483872, 35.913978494623656, 60.43010752688172, 48.38709677419355, 5.161290322580645, 23.440860215053764, 37.41935483870968, 16.989247311827956, 28.602150537634408, 19.35483870967742, 58.494623655913976, 37.8494623655914, 21.29032258064516, 18.70967741935484, 27.526881720430108, 41.935483870967744 ] }, { "name": "Metric 2", "type": "bar", "x": [ "Essentia", "SonicBasic", "qmtempo", "Aubio", "Beatroot", "Ellis", "Tzanetakis", "MIRTempo", "IBT", "BeatIt", "Mpeg7-xm", "jAudio", "Alo_corr", "Alo_spec", "Dix_auco", "Dix_indu", "Dix_trac", "Klapuri", "Scheirer", "Tzan_hist", "Tzan_mmul", "Tzan_msum", "Uhle" ], "y": [ 83.44086021505376, 50.75268817204301, 80.43010752688173, 67.31182795698925, 67.95698924731182, 80.64516129032258, 66.45161290322581, 65.59139784946237, 79.78494623655914, 78.27956989247312, 70.53763440860214, 32.25806451612903, 58.27956989247312, 68.60215053763442, 76.98924731182795, 62.58064516129032, 68.81720430107526, 91.18279569892474, 69.46236559139786, 47.74193548387097, 41.075268817204304, 52.473118279569896, 71.82795698924731 ] } ], "layout": { "title": "Tempo Evaluation results" } }, "text/html": [ "
" ], "text/vnd.plotly.v1+html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "trace1 = go.Bar(\n", " x=final_evaluation.index, # Algorithms names\n", " y=final_evaluation['Metric 1'],\n", " name = 'Metric 1'\n", ")\n", "trace2 = go.Bar(\n", " x=final_evaluation.index, # Algorithms names\n", " y=final_evaluation['Metric 2'],\n", " name='Metric 2'\n", ") \n", "data = [trace1, trace2]\n", "layout = go.Layout(title='Tempo Evaluation results')\n", "fig = go.Figure(data=data, layout=layout)\n", "pyo.iplot(fig, filename='Bar-plot')" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# final_evaluation.plot(kind='bar',title =\"Evaluation Results\", figsize=(16,8),grid = True,fontsize=12);\n", "# Simple plot but not interactive" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Sorted General Evaluation results (by Metric 1)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Metric 1Metric 2
BeatIt60.430178.2796
Klapuri58.494691.1828
Mpeg7-xm48.387170.5376
Ellis45.591480.6452
qmtempo43.225880.4301
Uhle41.935571.828
Aubio39.354867.3118
Scheirer37.849569.4624
Alo_spec37.419468.6022
IBT35.91479.7849
MIRTempo30.967765.5914
Dix_indu28.602262.5806
Tzan_msum27.526952.4731
Tzanetakis25.591466.4516
SonicBasic24.731250.7527
Essentia24.516183.4409
Alo_corr23.440958.2796
Beatroot23.225867.957
Tzan_hist21.290347.7419
Dix_trac19.354868.8172
Tzan_mmul18.709741.0753
Dix_auco16.989276.9892
jAudio5.1612932.2581
" ], "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted_final_evaluation1 = final_evaluation.sort_values(by=['Metric 1'], ascending= False)\n", "sorted_final_evaluation1.style.highlight_max() # High lighting the maximun value per column" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "data": [ { "name": "Metric 1", "type": "bar", "x": [ "BeatIt", "Klapuri", "Mpeg7-xm", "Ellis", "qmtempo", "Uhle", "Aubio", "Scheirer", "Alo_spec", "IBT", "MIRTempo", "Dix_indu", "Tzan_msum", "Tzanetakis", "SonicBasic", "Essentia", "Alo_corr", "Beatroot", "Tzan_hist", "Dix_trac", "Tzan_mmul", "Dix_auco", "jAudio" ], "y": [ 60.43010752688172, 58.494623655913976, 48.38709677419355, 45.59139784946237, 43.225806451612904, 41.935483870967744, 39.354838709677416, 37.8494623655914, 37.41935483870968, 35.913978494623656, 30.967741935483872, 28.602150537634408, 27.526881720430108, 25.591397849462364, 24.731182795698924, 24.516129032258064, 23.440860215053764, 23.225806451612904, 21.29032258064516, 19.35483870967742, 18.70967741935484, 16.989247311827956, 5.161290322580645 ] }, { "name": "Metric 2", "type": "bar", "x": [ "BeatIt", "Klapuri", "Mpeg7-xm", "Ellis", "qmtempo", "Uhle", "Aubio", "Scheirer", "Alo_spec", "IBT", "MIRTempo", "Dix_indu", "Tzan_msum", "Tzanetakis", "SonicBasic", "Essentia", "Alo_corr", "Beatroot", "Tzan_hist", "Dix_trac", "Tzan_mmul", "Dix_auco", "jAudio" ], "y": [ 78.27956989247312, 91.18279569892474, 70.53763440860214, 80.64516129032258, 80.43010752688173, 71.82795698924731, 67.31182795698925, 69.46236559139786, 68.60215053763442, 79.78494623655914, 65.59139784946237, 62.58064516129032, 52.473118279569896, 66.45161290322581, 50.75268817204301, 83.44086021505376, 58.27956989247312, 67.95698924731182, 47.74193548387097, 68.81720430107526, 41.075268817204304, 76.98924731182795, 32.25806451612903 ] } ], "layout": { "title": "Tempo Evaluation results (Sorted by Metric 1)" } }, "text/html": [ "
" ], "text/vnd.plotly.v1+html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "trace1 = go.Bar(\n", " x=sorted_final_evaluation1.index, # Algorithms names\n", " y=sorted_final_evaluation1['Metric 1'],\n", " name = 'Metric 1'\n", ")\n", "trace2 = go.Bar(\n", " x=sorted_final_evaluation1.index, # Algorithms names\n", " y=sorted_final_evaluation1['Metric 2'],\n", " name='Metric 2'\n", ") \n", "data = [trace1, trace2]\n", "layout = go.Layout(\n", " title='Tempo Evaluation results (Sorted by Metric 1)'\n", ")\n", "fig = go.Figure(data=data, layout=layout)\n", "pyo.iplot(fig, filename='Bar-plot')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Sorted General Evaluation results (by Metric 2)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Metric 1Metric 2
Klapuri58.494691.1828
Essentia24.516183.4409
Ellis45.591480.6452
qmtempo43.225880.4301
IBT35.91479.7849
BeatIt60.430178.2796
Dix_auco16.989276.9892
Uhle41.935571.828
Mpeg7-xm48.387170.5376
Scheirer37.849569.4624
Dix_trac19.354868.8172
Alo_spec37.419468.6022
Beatroot23.225867.957
Aubio39.354867.3118
Tzanetakis25.591466.4516
MIRTempo30.967765.5914
Dix_indu28.602262.5806
Alo_corr23.440958.2796
Tzan_msum27.526952.4731
SonicBasic24.731250.7527
Tzan_hist21.290347.7419
Tzan_mmul18.709741.0753
jAudio5.1612932.2581
" ], "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted_final_evaluation2 = final_evaluation.sort_values(by=['Metric 2'], ascending= False)\n", "sorted_final_evaluation2.style.highlight_max() # High lighting the maximun value per column" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "data": [ { "name": "Metric 1", "type": "bar", "x": [ "Klapuri", "Essentia", "Ellis", "qmtempo", "IBT", "BeatIt", "Dix_auco", "Uhle", "Mpeg7-xm", "Scheirer", "Dix_trac", "Alo_spec", "Beatroot", "Aubio", "Tzanetakis", "MIRTempo", "Dix_indu", "Alo_corr", "Tzan_msum", "SonicBasic", "Tzan_hist", "Tzan_mmul", "jAudio" ], "y": [ 58.494623655913976, 24.516129032258064, 45.59139784946237, 43.225806451612904, 35.913978494623656, 60.43010752688172, 16.989247311827956, 41.935483870967744, 48.38709677419355, 37.8494623655914, 19.35483870967742, 37.41935483870968, 23.225806451612904, 39.354838709677416, 25.591397849462364, 30.967741935483872, 28.602150537634408, 23.440860215053764, 27.526881720430108, 24.731182795698924, 21.29032258064516, 18.70967741935484, 5.161290322580645 ] }, { "name": "Metric 2", "type": "bar", "x": [ "Klapuri", "Essentia", "Ellis", "qmtempo", "IBT", "BeatIt", "Dix_auco", "Uhle", "Mpeg7-xm", "Scheirer", "Dix_trac", "Alo_spec", "Beatroot", "Aubio", "Tzanetakis", "MIRTempo", "Dix_indu", "Alo_corr", "Tzan_msum", "SonicBasic", "Tzan_hist", "Tzan_mmul", "jAudio" ], "y": [ 91.18279569892474, 83.44086021505376, 80.64516129032258, 80.43010752688173, 79.78494623655914, 78.27956989247312, 76.98924731182795, 71.82795698924731, 70.53763440860214, 69.46236559139786, 68.81720430107526, 68.60215053763442, 67.95698924731182, 67.31182795698925, 66.45161290322581, 65.59139784946237, 62.58064516129032, 58.27956989247312, 52.473118279569896, 50.75268817204301, 47.74193548387097, 41.075268817204304, 32.25806451612903 ] } ], "layout": { "title": "Tempo Evaluation results (Sorted by Metric 2)" } }, "text/html": [ "
" ], "text/vnd.plotly.v1+html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "trace1 = go.Bar(\n", " x=sorted_final_evaluation2.index, # Algorithms names\n", " y=sorted_final_evaluation2['Metric 1'],\n", " name = 'Metric 1'\n", ")\n", "trace2 = go.Bar(\n", " x=sorted_final_evaluation2.index, # Algorithms names\n", " y=sorted_final_evaluation2['Metric 2'],\n", " name='Metric 2'\n", ") \n", "data = [trace1, trace2]\n", "layout = go.Layout(title='Tempo Evaluation results (Sorted by Metric 2)')\n", "fig = go.Figure(data=data, layout=layout)\n", "pyo.iplot(fig, filename='Bar-plot')" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "#final_evaluation.sort_values(by=['Metric 2']).plot(kind='bar',title =\"Evaluation Results\", figsize=(16,8),grid = True,fontsize=12);\n", "# Simple plot but not interactive" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Error relations" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SONGGROUNDTRUTHEssentiaSonic BasicqmtempoAubioBeatrootEllisTzanetakisMIRTempo...AnonDix_aucoDix_induDix_tracKlapluriScheirerTzan_histTzan_mmulTzan_msumUhle
0Abba\\Gold - Greatest Hits\\20 sec\\01-Dancing Qu...101.0100842GoodGoodGoodGood4/3GoodGood...23/222GoodGood2GoodGood2/3
1Abba\\Gold - Greatest Hits\\20 sec\\02-Knowing Me...105.7551052GoodGoodGoodGoodGoodGoodGood...Good2Good2GoodGoodGoodGoodGoodGood
2Abba\\Gold - Greatest Hits\\20 sec\\03-Take A Cha...107.1428572GoodGoodGoodGoodGoodGoodGood...Good2Good2Good2GoodGoodGoodGood
3Abba\\Gold - Greatest Hits\\20 sec\\04-Mamma Mia.wa136.617092GoodGoodGoodGoodGoodGoodGoodGood...GoodGoodGoodGoodGood2/3GoodGoodGoodGood
4Abba\\Gold - Greatest Hits\\20 sec\\05-Lay All Yo...133.636421GoodGoodGoodGoodGood1/2GoodGood...GoodGoodGoodGoodGoodGoodGoodGoodGoodGood
\n", "

5 rows × 26 columns

\n", "
" ], "text/plain": [ " SONG GROUNDTRUTH Essentia \\\n", "0 Abba\\Gold - Greatest Hits\\20 sec\\01-Dancing Qu... 101.010084 2 \n", "1 Abba\\Gold - Greatest Hits\\20 sec\\02-Knowing Me... 105.755105 2 \n", "2 Abba\\Gold - Greatest Hits\\20 sec\\03-Take A Cha... 107.142857 2 \n", "3 Abba\\Gold - Greatest Hits\\20 sec\\04-Mamma Mia.wa 136.617092 Good \n", "4 Abba\\Gold - Greatest Hits\\20 sec\\05-Lay All Yo... 133.636421 Good \n", "\n", " Sonic Basic qmtempo Aubio Beatroot Ellis Tzanetakis MIRTempo ... Anon \\\n", "0 Good Good Good Good 4/3 Good Good ... 2 \n", "1 Good Good Good Good Good Good Good ... Good \n", "2 Good Good Good Good Good Good Good ... Good \n", "3 Good Good Good Good Good Good Good ... Good \n", "4 Good Good Good Good 1/2 Good Good ... Good \n", "\n", " Dix_auco Dix_indu Dix_trac Klapluri Scheirer Tzan_hist Tzan_mmul Tzan_msum \\\n", "0 3/2 2 2 Good Good 2 Good Good \n", "1 2 Good 2 Good Good Good Good Good \n", "2 2 Good 2 Good 2 Good Good Good \n", "3 Good Good Good Good 2/3 Good Good Good \n", "4 Good Good Good Good Good Good Good Good \n", "\n", " Uhle \n", "0 2/3 \n", "1 Good \n", "2 Good \n", "3 Good \n", "4 Good \n", "\n", "[5 rows x 26 columns]" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Relation between estimated tempo and the ground truth\n", "df2 = pd.read_excel(\"data/AES42_Tempo_Error_2011.xls\")\n", "df2.head(5)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "Allerrors = df2.iloc[:,2:].values.copy()\n", "Allerrors.resize(Allerrors.size,1)\n", "ErrorSeries = pd.DataFrame(Allerrors)\n", "ErrorSeries[0]= ErrorSeries[0].astype(str)\n", "#y.plot(kind='bar',title = 'Examen Final');# Simple histogram plot" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "data": [ { "name": "Errors", "type": "bar", "x": [ "Good", "2", "Bad", "4/3", "1/2", "3/2", "2/3", "3", "9/4", "8/3", "4", "5/2", "5/4", "3/4", "6", "1/3", "16/3", "1/4" ], "y": [ "3635", "3271", "1842", "598", "396", "214", "206", "194", "179", "148", "136", "123", "113", "66", "17", "11", "8", "3" ] } ], "layout": { "title": "Error Relations in all results" } }, "text/html": [ "
" ], "text/vnd.plotly.v1+html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "hist = ErrorSeries[0].value_counts()\n", "datah = [\n", " go.Bar(\n", " y = hist.values.astype(str),\n", " x = hist.index,\n", " name = \"Errors\"\n", " )\n", "]\n", "layout = go.Layout(title='Error Relations in all results')\n", "fig = go.Figure(data=datah, layout=layout)\n", "pyo.iplot(fig, filename='ErrorsHistogram')" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 2 }