{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook contains some code snippets I used in writing the following blog post: https://dev.to/ericbonfadini/finding-my-new-favorite-song-on-spotify-4lgc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While I'm developing (or just while I'm commuting to work) I usually love to hear some rock music.\n",
    "\n",
    "I created some playlists on Spotify, but lately I'm stick to the same playlist, containing my favorite \"Indie Rock\" songs.\n",
    "This playlist is made up of more or less 45 songs I discovered through the years in several ways.\n",
    "\n",
    "Since I was starting to get bored about always listening to the same songs, last weekend I decided to analyze my playlist using Spotify APIs in order to discover insights and hopefully to find some new tunes I could add.\n",
    "\n",
    "Here's what I did in more or less 300 lines of Python 3 code (boilerplate included)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setting up the environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For my analysis I set up a Python 3 virtual environment with the following libraries:\n",
    "- Pandas for data analysis\n",
    "- Seaborn for data visualization\n",
    "- Spotipy for interaction with Spotify APIs\n",
    "\n",
    "In order to access the Spotify APIs I registered my app and then I provided the spotipy library with the client_id, the client_secret and a redirect url."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import spotipy\n",
    "from matplotlib import style\n",
    "from spotipy import util\n",
    "\n",
    "# Improve Pandas display settings\n",
    "pd.set_option(\"display.width\", 600)\n",
    "pd.set_option(\"display.max_columns\", 50)\n",
    "\n",
    "# Change Seaborn default settings\n",
    "sns.set_context('talk')\n",
    "style.use('ggplot')\n",
    "\n",
    "# Spotify API limit\n",
    "API_LIMIT = 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyzing my playlist tracks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After a first API call to get my playlist id, I got all the tracks of my playlist with another API call along with some basic information like: song id, song name, artist id, artist name, album name, song popularity.\n",
    "\n",
    "With another API call I got some extra information about the artists in my playlist, like genres and artist_popularity.\n",
    "\n",
    "Finally with another API call I got some insightful information about my tracks:\n",
    "- duration_ms: the duration of the track in milliseconds;\n",
    "- acousticness: describes the acousticness of a song (1 => high confidence the track is acoustic). It ranges from 0 to 1;\n",
    "- danceability: describes the danceability of a song (1 => high confidence the track is danceable). It ranges from 0 to 1;\n",
    "- energy: it's a perceptual measure of intensity and activity (e.g. death metal has high energy while classical music has low energy). It ranges from 0 to 1;\n",
    "- instrumentalness: predicts whether a track contains no vocals (1 => high confidence the track has no vocals). It ranges from 0 to 1;\n",
    "- liveness: detects the presence of an audience in the recording (1 => high confidence the track is live). It ranges from 0 to 1;\n",
    "- loudness: detects the overall loudness of a track in decibels. It ranges from -60dB to 0dB;\n",
    "- valence: describes the musical positiveness conveyed by a track (1 => more positive, 0 => more negative). It ranges from 0 to 1;\n",
    "- speechiness: detects the presence of spoken words in a track (1 => speech, 0 => non speech, just music). It ranges from 0 to 1;\n",
    "- key: describes the pitch class notation of the song. It ranges from 0 to 11;\n",
    "- mode: the modality of a track (0 => minor, 1 => major);\n",
    "- tempo: the overall estimated tempo of a track in beats per minute (BPM);\n",
    "- time_signature: An estimated overall time signature of a track (how many beats are in each bar or measure).\n",
    "\n",
    "The results of all these calls have been put inside Pandas dataframes in order to simplify the data analysis and then merged in a single dataframe using artist IDs and track IDs.\n",
    "Some values (like song/artist popularity and tempo) have been normalized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "username = \"eric.bonfadini\"\n",
    "playlist_name = \"Indie Rock\"\n",
    "\n",
    "# Get token\n",
    "scope = 'playlist-modify-public'\n",
    "spotify_token = util.prompt_for_user_token(username, scope)\n",
    "\n",
    "# Create client\n",
    "sp = spotipy.Spotify(auth=spotify_token)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get playlist ID\n",
    "playlists_results = sp.user_playlists(username)\n",
    "playlist_ids = [playlist['id'] for playlist in playlists_results['items'] if playlist['name'] == playlist_name]\n",
    "\n",
    "if not playlist_ids:\n",
    "    raise Exception(\"Cannot find playlist named: {}\".format(playlist_name))\n",
    "else:\n",
    "    print(playlist_ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get tracks\n",
    "tracks_results = sp.user_playlist(username, playlist_ids[0])\n",
    "\n",
    "df_tracks = pd.DataFrame([[t[\"track\"][\"id\"], t[\"track\"][\"name\"], t[\"track\"][\"artists\"][0][\"id\"], \n",
    "                           t[\"track\"][\"artists\"][0][\"name\"], t[\"track\"][\"album\"][\"name\"], t[\"track\"][\"popularity\"]] \n",
    "                          for t in tracks_results['tracks']['items']], \n",
    "                         columns=[\"id\", \"song_name\", \"artist_id\", \"artist_name\", \"album_name\", \"popularity\"]) \n",
    "# Normalize popularity\n",
    "df_tracks[\"popularity_norm\"] = df_tracks[\"popularity\"] / 100.\n",
    "\n",
    "df_tracks.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _get_artists_df(sp, artist_ids):\n",
    "    \"\"\"\n",
    "    This is an helper method to get artist's information with pagination from artist ids.\n",
    "    It returns a Pandas dataframe\n",
    "    \"\"\"\n",
    "\n",
    "    artist_list = []\n",
    "    i = 0\n",
    "\n",
    "    while artist_ids:\n",
    "        print(\"Call #{} for artists\".format(i + 1))\n",
    "        artists_results = sp.artists(artist_ids[:API_LIMIT])\n",
    "\n",
    "        artist_list += [[t[\"id\"], t[\"genres\"], t[\"popularity\"]] for t in artists_results[\"artists\"]]\n",
    "\n",
    "        artist_ids = artist_ids[API_LIMIT:]\n",
    "        i += 1\n",
    "\n",
    "    df_artists = pd.DataFrame(artist_list, columns=[\"artist_id\", \"artist_genres\", \"artist_popularity\"])\n",
    "\n",
    "    df_artists[\"artist_popularity_norm\"] = df_artists[\"artist_popularity\"] / 100.\n",
    "\n",
    "    return df_artists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "artist_ids = df_tracks[\"artist_id\"].unique().tolist()\n",
    "df_artists = _get_artists_df(sp, artist_ids)\n",
    "df_artists.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _get_features_df(sp, track_ids):\n",
    "    \"\"\"\n",
    "    This is an helper method to get track's features with pagination from track ids.\n",
    "    It returns a Pandas dataframe\n",
    "    \"\"\"\n",
    "\n",
    "    feature_list = []\n",
    "    i = 0\n",
    "    while track_ids:\n",
    "        print(\"Call #{} for audio features\".format(i + 1))\n",
    "        features_results = sp.audio_features(track_ids[:API_LIMIT])\n",
    "\n",
    "        feature_list += features_results\n",
    "        \n",
    "        track_ids = track_ids[API_LIMIT:]\n",
    "        i += 1\n",
    "\n",
    "    df_features = pd.DataFrame(feature_list)[[\"id\", \"analysis_url\", \"duration_ms\", \"acousticness\", \"danceability\", \n",
    "                                              \"energy\", \"instrumentalness\", \"liveness\", \"loudness\", \"valence\", \n",
    "                                              \"speechiness\", \"key\", \"mode\", \"tempo\", \"time_signature\"]]\n",
    "    # tempo is in range 24-200 ==> 0-176, normalize it\n",
    "    df_features[\"tempo_norm\"] = (df_features[\"tempo\"] - 24) / 176.\n",
    "    \n",
    "    return df_features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "track_ids = df_tracks[\"id\"].unique().tolist()\n",
    "df_features = _get_features_df(sp, track_ids)\n",
    "df_features.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a df for current playlist merging the dataframes\n",
    "df_cur = df_features.merge(df_tracks, on=\"id\")\n",
    "df_cur = df_cur.merge(df_artists, on=\"artist_id\")\n",
    "\n",
    "# Create a new column with full name of the song\n",
    "df_cur[\"full_name\"] = df_cur[\"artist_name\"] + \" -- \" + df_cur[\"song_name\"]\n",
    "\n",
    "# Sort by song popularity\n",
    "df_cur.sort_values(\"popularity\", inplace=True, ascending=False)\n",
    "\n",
    "df_cur.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explorative Data Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After ensuring that the artists in my playlist all contain \"Indie Rock\" as genre, I checked my playlist using shape, info and describe of the full dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(df_cur.info())\n",
    "\n",
    "df_cur.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert time_signature and key to category\n",
    "df_cur[\"time_signature\"] = df_cur[\"time_signature\"].astype(pd.api.types.CategoricalDtype(categories=[1, 2, 3, 4, 5]))\n",
    "df_cur[\"key\"] = df_cur[\"key\"].astype(pd.api.types.CategoricalDtype(categories=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _distplot(df, key, label, x_limits):\n",
    "    \"\"\"\n",
    "    This is an helper method to plot distribution charts\n",
    "    \"\"\"\n",
    "    ax = sns.distplot(df[[key]], bins=30, label=label)\n",
    "    if x_limits is not None:\n",
    "        ax.set_xlim(*x_limits)\n",
    "    plt.title(key)\n",
    "    plt.legend()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_limits = {\"duration_ms\": None, \"loudness\": (-60, 0), \"tempo\": (24, 200), \"popularity\": (0, 100), \n",
    "            \"artist_popularity\": (0, 100)}\n",
    "\n",
    "for key in [\"duration_ms\", \"acousticness\", \"danceability\", \"energy\", \"instrumentalness\", \"liveness\",\n",
    "                    \"loudness\", \"valence\", \"speechiness\", \"tempo\", \"popularity\", \"artist_popularity\"]:\n",
    "    _distplot(df_cur, key, label=\"My Playlist\", x_limits=x_limits.get(key, (0, 1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _countplot(df, key, label):\n",
    "    \"\"\"\n",
    "    This is an helper method to plot count charts\n",
    "    \"\"\"\n",
    "    ax = sns.countplot(data=df, x=key, palette=\"tab20\")\n",
    "    ax.set_title(label)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for key in [\"key\", \"time_signature\", \"mode\"]:\n",
    "    _countplot(df_cur, key, label=\"My Playlist\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = sns.boxplot(data=df_cur[[\"acousticness\", \"danceability\", \"energy\", \"instrumentalness\", \"liveness\", \n",
    "                              \"valence\", \"speechiness\", \"artist_popularity_norm\", \"popularity_norm\", \"tempo_norm\"]])\n",
    "ax.set_title(\"My Playlist\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All these graphs show that I like songs with low acousticness/instrumentalness/speechiness, high energy/loudness/tempo, high artist popularity and duration of more or less 200 seconds.\n",
    "Valence and song popularity span on a wide range, meaning that I have in my playlist both well-known and unknown songs, and both positive and negative ones.\n",
    "\n",
    "But how my playlist compare against the Indie Rock genre?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Comparing my playlist with a sample of the genre"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I used some calls to the search API with 'genre:\"Indie Rock\"' as a keyword and 'type=tracks' to get a sample of the Indie Rock genre (5000 songs in total).\n",
    "This API offers also some nice keywords like 'tag:hipster' (to get only albums with the lowest 10% popularity) or 'year:1980-2020' (to get only tracks released in a specific year range)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "number_of_tracks = 5000\n",
    "genre = \"indie rock\"\n",
    "\n",
    "search_runs = int(number_of_tracks / API_LIMIT)\n",
    "\n",
    "search_list = []\n",
    "for i in range(search_runs):\n",
    "    print(\"Call #{} for tracks\".format(i+1))\n",
    "    search_results = sp.search('genre:\"{}\"'.format(genre), type=\"track\", limit=API_LIMIT, offset=API_LIMIT*i)\n",
    "\n",
    "    search_list += [[t[\"id\"], t[\"name\"], t[\"artists\"][0][\"id\"], t[\"artists\"][0][\"name\"],\n",
    "                            t[\"album\"][\"name\"], t[\"popularity\"]]\n",
    "                           for t in search_results['tracks']['items']]\n",
    "\n",
    "df_search = pd.DataFrame(search_list, \n",
    "                         columns=[\"id\", \"song_name\", \"artist_id\", \"artist_name\", \"album_name\", \"popularity\"])\n",
    "df_search[\"popularity_norm\"] = df_search[\"popularity\"] / 100.\n",
    "df_search.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "track_ids = df_search[\"id\"].unique().tolist()\n",
    "df_features = _get_features_df(sp, track_ids)\n",
    "df_features.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "artist_ids = df_search[\"artist_id\"].unique().tolist()\n",
    "df_artists = _get_artists_df(sp, artist_ids)\n",
    "df_artists.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_sample = df_features.merge(df_search, on=\"id\")\n",
    "df_sample = df_sample.merge(df_artists, on=\"artist_id\")\n",
    "\n",
    "df_sample[\"full_name\"] = df_sample[\"artist_name\"] + \" -- \" + df_sample[\"song_name\"]\n",
    "df_sample.sort_values(\"popularity\", inplace=True, ascending=False)\n",
    "\n",
    "df_sample.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert time_signature and key to category\n",
    "df_sample[\"time_signature\"] = df_sample[\"time_signature\"].astype(pd.api.types.CategoricalDtype(categories=[1, 2, 3, 4, 5]))\n",
    "df_sample[\"key\"] = df_sample[\"key\"].astype(pd.api.types.CategoricalDtype(categories=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(df_sample.info())\n",
    "\n",
    "df_sample.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then I repeated the same analysis on both my current playlist and the Indie Rock sample and I got the following charts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " def _distplot2(df, df_other, key, labels, x_limits):\n",
    "    ax = sns.distplot(df[[key]], bins=30, label=labels[0])\n",
    "    if x_limits is not None:\n",
    "        ax.set_xlim(*x_limits)\n",
    "    ax = sns.distplot(df_other[[key]], bins=30, label=labels[1])\n",
    "    if x_limits is not None:\n",
    "        ax.set_xlim(*x_limits)\n",
    "    plt.title(key)\n",
    "    plt.legend()\n",
    "    plt.show()\n",
    "\n",
    "def _countplot2(df, df_other, key, labels):\n",
    "    fig, ax = plt.subplots(1, 2)\n",
    "    sns.countplot(data=df, x=key, ax=ax[0], palette=\"tab20\")\n",
    "    ax[0].set_title(labels[0])\n",
    "    sns.countplot(data=df_other, x=key, ax=ax[1], palette=\"tab20\")\n",
    "    ax[1].set_title(labels[1])\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for key in [\"duration_ms\", \"acousticness\", \"danceability\", \"energy\", \"instrumentalness\", \"liveness\",\n",
    "            \"loudness\", \"valence\", \"speechiness\", \"tempo\", \"popularity\", \"artist_popularity\"]:\n",
    "    _distplot2(df_cur, df_sample, key,\n",
    "              labels=[\"My Playlist\", \"5000 Indie rock songs\"],\n",
    "              x_limits=x_limits.get(key, (0, 1)))\n",
    "\n",
    "for key in [\"key\", \"time_signature\", \"mode\"]:\n",
    "    _countplot2(df_cur, df_sample, key, labels=[\"My Playlist\", \"5000 Indie rock songs\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(2, 1)\n",
    "sns.boxplot(data=df_cur[[\"acousticness\", \"danceability\", \"energy\", \"instrumentalness\", \"liveness\",\n",
    "                         \"valence\", \"speechiness\", \"artist_popularity_norm\", \"popularity_norm\",\n",
    "                         \"tempo_norm\"]], ax=ax[0])\n",
    "ax[0].set_title(\"My Playlist\")\n",
    "sns.boxplot(data=df_sample[[\"acousticness\", \"danceability\", \"energy\", \"instrumentalness\", \"liveness\",\n",
    "                           \"valence\", \"speechiness\", \"artist_popularity_norm\", \"popularity_norm\",\n",
    "                           \"tempo_norm\"]], ax=ax[1])\n",
    "ax[1].set_title(\"5000 Indie rock songs\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The graphs show that my playlist differs from the 5000 Indie Rock songs because:\n",
    "- I like shorter songs\n",
    "- I like songs with higher energy/loudness/tempo\n",
    "- I don't like songs with too negative mood (valence > 0.3)\n",
    "- I like songs mostly in key (0, 1, 6, 9)\n",
    "\n",
    "The boxplot confirms the same insights and I agree with the outcome of this analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a new playlist with songs I potentially like"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using these insights, I applied some filters to the 5000 songs dataframe in order to keep only tracks I potentially like; for each step I logged the dropped songs to double check the filter behavior.\n",
    "\n",
    "The first filter I applied was removing songs I already had in my original playlist, obviously.\n",
    "\n",
    "The other filters were:\n",
    "- acousticness < 0.1\n",
    "- energy > 0.75\n",
    "- loudness > -7dB\n",
    "- valence between 0.3 and 0.9\n",
    "- tempo > 120\n",
    "- key in (0, 1, 6, 9)\n",
    "- duration between 10% quartile of original playlist duration and 90% quartile (178s and 280s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _apply_condition(df, condition, label):\n",
    "    before = len(df)\n",
    "    dropped_songs = df[~condition][\"full_name\"].head().tolist()\n",
    "    df = df[condition]\n",
    "    print(\"\\ncondition [{}]: {}-{}={}\".format(label, before, before - len(df), len(df)))\n",
    "    print(\"first 10 dropped songs: {}\".format(dropped_songs))\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_new = df_sample.drop_duplicates([\"full_name\"], keep=\"first\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_new = _apply_condition(df_new,\n",
    "                          condition=~(df_new[\"full_name\"]).isin((df_cur[\"full_name\"]).tolist()),\n",
    "                          label=\"name\")\n",
    "\n",
    "df_new = _apply_condition(df_new, \n",
    "                          condition=(df_new[\"acousticness\"] < 0.1),\n",
    "                          label=\"acousticness\")\n",
    "\n",
    "df_new = _apply_condition(df_new,\n",
    "                          condition=(df_new[\"energy\"] > 0.75),\n",
    "                          label=\"energy\")\n",
    "\n",
    "df_new = _apply_condition(df_new,\n",
    "                          condition=(df_new[\"loudness\"] > -7),\n",
    "                          label=\"loudness\")\n",
    "\n",
    "df_new = _apply_condition(df_new,\n",
    "                          condition=(df_new[\"valence\"].between(0.3, 0.9)),\n",
    "                          label=\"valence\")\n",
    "\n",
    "df_new = _apply_condition(df_new,\n",
    "                          condition=(df_new[\"tempo\"] > 120),\n",
    "                          label=\"tempo\")\n",
    "\n",
    "df_new = _apply_condition(df_new,\n",
    "                          condition=(df_new[\"key\"].isin([9, 0, 1, 6])),\n",
    "                          label=\"key\")\n",
    "\n",
    "df_new = _apply_condition(df_new,\n",
    "                          condition=(df_new[\"duration_ms\"].between(*df_cur[\"duration_ms\"].quantile([0.1, 0.9]))),\n",
    "                          label=\"duraton_ms\")\n",
    "\n",
    "df_new.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the end I got a dataframe with 220 tracks and I created a new playlist using an API call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "playlist_name_new = \"Indie Rock Trial\"\n",
    "\n",
    "playlists = sp.user_playlists(username)\n",
    "\n",
    "playlist_ids = [playlist['id'] for playlist in playlists['items'] if playlist['name'] == playlist_name_new]\n",
    "\n",
    "if not playlist_ids:\n",
    "    playlists = sp.user_playlist_create(username, playlist_name_new)\n",
    "    playlist_id = playlists[\"id\"]\n",
    "\n",
    "else:\n",
    "    playlist_id = playlist_ids[0]\n",
    "\n",
    "    results = sp.user_playlist(username, playlist_id)\n",
    "    track_ids = [t[\"track\"][\"id\"] for t in results[\"tracks\"][\"items\"]]\n",
    "    results = sp.user_playlist_remove_all_occurrences_of_tracks(username, playlist_id, track_ids)\n",
    "    print(results)\n",
    "\n",
    "track_ids = df_new[\"id\"].unique().tolist()\n",
    "\n",
    "while track_ids:\n",
    "    results = sp.user_playlist_add_tracks(username, playlist_id, track_ids[:API_LIMIT])\n",
    "    print(results)\n",
    "    track_ids = track_ids[API_LIMIT:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusions and future steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After a few days listening to the new playlist, I'm quite happy with the results and I'm already promoting some tracks to my original playlist.\n",
    "\n",
    "This \"recommendation\" method is really simple and probably works well only in this specific use case (i.e. it's a well definable subset of a specific genre).\n",
    "The standard recommendation method of Spotify is obviously much better because, apart from audio analysis, it uses also a mix of Collaborative Filtering models (analyzing your behavior and others’ behavior) and Natural Language Processing (NLP) models (analyzing text of the songs).\n",
    "\n",
    "Next steps:\n",
    "- Run the analysis again after a few months, in order to take into account new entries in my playlist and new songs in the 5000 sample\n",
    "- Enrich the information I already got with something new using the Spotify APIs (e.g the Audio Analysis endpoint or other services. It would be nice, as an example, to detect musical instruments in a track (guitars anyone??) or the presence of some features like distortion, riffs, etc.\n",
    "- Use the preview sample from the API to tag manually what I like/dislike on a subset of the 5000 songs and then run some ML algorithms in order to classify the music I like\n",
    "- Analyze deeper my playlist using some ML algorithms (e.g. cluster my tracks)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}