![](\"https://github.com/JeffMacaluso/JeffMacaluso.github.io/blob/master/_posts/RestaurantRecommender_files/Reviews.png?raw=true\")
\n",
"\n",
"I highlighted the main points to stand out against the small font. Service, atmosphere, and apparently eggrolls were the most common and unifying factors. You see very little discussion on the quality of the actual food, and you can even see an instance where a reviewer rates the pizza place as 5/5 even after saying that it is expensive. I began to notice a disconnect in how I evaluate restaurants versus how the people of Minneapolis evaluate restaurants. If you have previously worked with recommender systems, you already know where I'm going with this. If not, here is a primer:\n",
"\n",
"### Recommender Systems Overview\n",
"\n",
"Before getting into the overview of recommender systems, I wanted to point out that I won't actually be building a legitimate recommender system in this notebook. There are some [great](https://turi.com/learn/userguide/recommender/introduction.html) [packages](https://github.com/lyst/lightfm) for doing so, but I'm going to stick with one of the main ideas behind recommender systems. This is for two reasons:\n",
"\n",
"**1)** Classes have started back up, and my available free time for side projects like this one is almost non-existant.\n",
"\n",
"**2)** My gastronomic adventures don't need the added benefits that a recommender system provides over what I'll be doing.\n",
"\n",
"Let's get back into it.\n",
"\n",
"In the world of recommender systems, there are three broad types:\n",
"\n",
"- **[Collaborative Filtering (user-user)](https://en.wikipedia.org/wiki/Collaborative_filtering)**: This is the most prevalent type of recommender systems that uses \"wisdom of the crowd\" for popularity among peers. This option is particularly popular because you don't need to know a lot about the item itself, you only need the ratings submitted by reviewers. The two primary restrictions are that it makes the assumption that peoples' tastes do not change over time, and new items run into the \"[cold start problem](https://en.wikipedia.org/wiki/Cold_start)\". This is when either a new item has not yet received any ratings and fails to appear on recommendation lists, or a new user has not reviewed anything so we don't know what their tastes are.\n",
" - **E.x.:** People who like item **X** also like item **Y**\n",
" - This is how Spotify selects songs for your recommended play list. Specifically, it will take songs from other play lists that contain songs you recently liked.\n",
" \n",
" \n",
"- **[Content-Based (item-item)](https://en.wikipedia.org/wiki/Recommender_system#Content-based_filtering)**: This method recommends items based off of their similarity to other items. This requires reliable information about the items themselves, which makes it difficult to implement in a lot of cases. Additionally, recommendations generated from this will option likely not deviate very far from the item being compared to, but there are tricks available to account for this.\n",
" - **E.x.:** Item **X** is similar to item **Y**\n",
" - This is how Pandora selects songs for your stations. Specifically, it assigns each song a list of characteristics (assigned through the [Music Genome Project](http://www.pandora.com/corporate/mgp.shtml)), and selects songs with similar characteristics as those that you liked.\n",
" \n",
"\n",
"- **[Hybrid](https://en.wikipedia.org/wiki/Recommender_system#Hybrid_recommender_systems)**: You probably guessed it - this is a combination of the above two types. The idea here is use what you have if you have it. [Here](http://www.math.uci.edu/icamp/courses/math77b/lecture_12w/pdfs/Chapter%2005%20-%20Hybrid%20recommendation%20approaches.pdf) are a few designs related to this that are worth looking into.\n",
"\n",
"\n",
"Those are the three main types, but there is one additional type that you may find if you are diving a little deeper into the subject material:\n",
"\n",
"\n",
"- **[Knowledge-Based](https://en.wikipedia.org/wiki/Knowledge-based_recommender_system)**: This is is the most rare type mainly because it requires explicit domain knowledge. It is often used for products that have a low number of available ratings, such as high luxury goods like hypercars. We won't delve any further into this type, but I recommend reading more about it if you're interested in the concept.\n",
"\n",
"\n",
"### Methodology\n",
"\n",
"Let's return to our problem. The previous way of selecting restaurants at the recommendation of locals and acquaintances (collaborative filtering) wasn't always successful, so we are going to use the idea behind content-based recommender systems to evaluate our options. However, we don't have a lot of content about the restaurants available, so we are going to primarily use the reviews people left for them. More specifically, we are going to determine similarity between restaurants based off of the similarity of the reviews that people have written for them. \n",
"\n",
"We're going to use cosine similarity since it's generally accepted as producing better results in item-to-item filtering:\n",
"\n",
"$\\hspace{8cm}sim(A, B) = \\cos(\\theta) = \\frac{A \\cdot B}{\\|A\\|\\|B\\|}$\n",
"\n",
"Before calculating this, we need to perform a couple of pre-processing steps on our reviews in order to make the data usable for our cosine similarity calculation. These will be common NLP (**n**atural **l**anguage **p**rocessing) techniques that you should be familiar with if you have worked with text before. These are the steps I took, but I am open to feedback and improvement if you have recommendations on other methods that may yield better results.\n",
"\n",
"**1) Normalizing**: This step converts our words into lower case so that when we map to our feature space, we don't end up with redundant features for the same words.\n",
"\n",
" Ex. \"Central Texas barbecue is the best smoked and the only barbecue that matters\"\n",
"\n",
"$\\hspace{1.75cm}$becomes\n",
"\n",
"```\n",
" \"central texas barbecue is the best smoked and the only barbecue that matters\"\n",
"```\n",
"\n",
"**2) Tokenizing**: This step breaks up a sentence into individual words, essentially turning our reviews into [bags of words](https://en.wikipedia.org/wiki/Bag-of-words_model), which makes it easier to perform other operations. Though we are going to perform many other preprocessing operations, this is more or less the beginning of mapping our reviews into the feature space.\n",
"\n",
" Ex. 'Central Texas barbecue is the best smoked and the only barbecue that matters'\n",
"\n",
"$\\hspace{1.75cm}$becomes\n",
"\n",
"```\n",
" ['Central', 'Texas', 'barbecue', 'is', 'the', 'best', 'smoked', 'and', 'the', 'only', 'barbecue', 'that', 'matters']\n",
"```\n",
"\n",
"**3) Removing Stopwords and Punctuation**: This step removes unnecessary words and punctuation often used in language that computers don't need such as *as*, *the*, *and*, and *of*.\n",
"\n",
" Ex. ['central', 'texas', 'barbecue', 'is', 'the', 'best', 'smoked', 'and', 'the', 'only', 'barbecue', 'that', 'matters']\n",
"\n",
"$\\hspace{1.75cm}$becomes\n",
"\n",
"```\n",
" ['central', 'texas', 'barbecue', 'best', 'smoked', 'only', 'barbecue', 'matters']\n",
"```\n",
"\n",
"**4) Lemmatizing (Stemming)**: Lemmatizing (which is very similar to stemming) removes variations at the end of a word to revert words to their root word.\n",
"\n",
" Ex. ['central', 'texas', 'barbecue', 'best', 'smoked', 'only', 'barbecue', 'matters']\n",
"\n",
"$\\hspace{1.75cm}$becomes\n",
"\n",
"```\n",
" ['central', 'texas', 'barbecue', 'best', 'smoke', 'only', 'barbecue', 'matter']\n",
"```\n",
"\n",
"**5) Term Frequency-Inverse Document Frequency (TF-IDF)**: This technique determines how important a word is to a document (which is a review in this case) within a corpus (the collection documents, or all reviews). This doesn't necessarily help establish context within our reviews themselves (for example, 'this Pad Kee Mao is bad ass' is technically a good thing, which wouldn't be accounted for unless we did [n-grams](https://en.wikipedia.org/wiki/N-gram) (which will give my laptop a much more difficult time)), but it does help with establishing the importance of the word.\n",
"\n",
"$\\hspace{8cm}TFIDF(t, d) = TF(t, d) \\cdot IDF(t)$\n",
"\n",
"$\\hspace{8cm}IDF(t) = 1 + \\log\\Big(\\frac{\\#\\ Documents}{\\#\\ Documents\\ Containing\\ t}\\Big)$\n",
"\n",
"$\\hspace{8cm}t:\\ \\text{Term}$\n",
"\n",
"$\\hspace{8cm}d:\\ \\text{Document}$\n",
"\n",
"On a side note, sarcasm, slang, misspellings, emoticons, and context are common problems in NLP, but we will be ignoring these due to time limitations. \n",
"\n",
"### Assumptions\n",
"\n",
"It's always important to state your assumptions in any analysis because a violation of them will often impact the reliability of the results. My assumptions in this case are as follows:\n",
"\n",
"- The reviews are indicative of the characteristics of the restaurant.\n",
"- The language used in the reviews does not directly impact the rating a user gives.\n",
" - E.g. Reviews contain a description of their experience, but ratings are the result of the user applying weights to specific things they value.\n",
" - Ex. \"The food was great, but the service was terrible.\" would be a 2/10 for one user, but a 7/10 for users like myself.\n",
"- The restaurants did not undergo significant changes in the time frame for the reviews being pulled.\n",
"- Sarcasm, slang, misspellings, and other common NLP problems will not have a significant impact on our results.\n",
"\n",
"---\n",
"# Restaurant Recommender\n",
"\n",
"\n",
"If you're still with us after all of that, let's get started!\n",
"\n",
"---\n",
"\n",
"In addition to the library imports, we have to specify our credentials to access the Foursquare API. I'm not keen on sharing mine, but you can get your own by [signing up](https://developer.foursquare.com/). I stored mine in a text file which is being read in as a variable before making the API calls."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:29:38.061295Z",
"start_time": "2017-07-02T15:29:22.852342Z"
},
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import nltk # Natural Language Processing\n",
"import re # Regex\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns\n",
"\n",
"# Functions for preparing and calculating similarity\n",
"from scipy import sparse\n",
"from sklearn.metrics.pairwise import cosine_similarity\n",
"from sklearn.preprocessing import MinMaxScaler, normalize\n",
"from sklearn.feature_extraction import text as sktext\n",
"\n",
"import foursquare\n",
"\n",
"# Controls aesthetics for plots\n",
"sns.set_context(\"notebook\", font_scale=1.1)\n",
"sns.set_style(\"ticks\")\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:29:38.109281Z",
"start_time": "2017-07-02T15:29:38.065251Z"
},
"collapsed": true
},
"outputs": [],
"source": [
"# Reading credentials for the API\n",
"with open('foursquareCredentials.txt') as f:\n",
" lines = [line.split('=')[1] for line in f.readlines()]\n",
" credentials = [line.replace(\"'\", '').replace('\\n', '') for line in lines]\n",
" f.close()\n",
"\n",
"# Assigning the credentials to an object\n",
"client = foursquare.Foursquare(\n",
" client_id=credentials[0], \n",
" client_secret=credentials[1], \n",
" redirect_uri=credentials[2])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Data\n",
"\n",
"Foursquare works similarly to Yelp where users will review restaurants. They can either leave a rating (1-10), or write a review for the restaurant. The reviews are what we're interested in here since I established above that the rating has less meaning due to the way people rate restaurants differently between the two cities.\n",
"\n",
"The [documentation](https://developer.foursquare.com/docs/) was fortunately fairly robust. I used the [foursquare categoryID tree](https://developer.foursquare.com/categorytree) in order to grab the venue category ID for the different types of restaurants. The [venue search](https://developer.foursquare.com/docs/venues/search) function grabs the actual restaurants, and the [tips](https://developer.foursquare.com/docs/venues/tips) function returns the reviews (and not what users left for a tip like you'd think).\n",
"\n",
"We're going to use the individual reviews, restaurant category, price tier, and the number of check-ins, reviews, and users.\n",
"\n",
"\n",
"### Restaurants\n",
"\n",
"Before pulling our restaurants, we have to first include a few parameters. We'll begin with the cities we want to include and the restaurant types we want to include.\n",
"\n",
"Since we're not interested in chains or fast food restaurants, we have to specify which venue category IDs we want. There are also a few grocery stores (such as Whole Foods and the magnificent [H-E-B](https://www.heb.com/)) that appear under the bakery and deli categories, so excluding this is easier than manually filtering them out. I left them commented out so there is the full list in case anyone else would like to include them for their own uses."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:29:38.245376Z",
"start_time": "2017-07-02T15:29:38.125291Z"
},
"collapsed": true
},
"outputs": [],
"source": [
"cities = ['Austin, TX', # Where I'm from\n",
" 'Minneapolis, MN'] # Where I'm going to grad school \n",
"\n",
"venueCategoryId = ['503288ae91d4c4b30a586d67' # Afghan\n",
" , '4bf58dd8d48988d1c8941735' # African\n",
" , '4bf58dd8d48988d14e941735' # American\n",
" , '4bf58dd8d48988d142941735' # Asian. Includes most Asian countries.\n",
" , '4bf58dd8d48988d169941735' # Australian\n",
" , '52e81612bcbc57f1066b7a01' # Austrian\n",
" , '4bf58dd8d48988d1df931735' # BBQ\n",
" , '4bf58dd8d48988d179941735' # Bagel\n",
"# , '4bf58dd8d48988d16a941735' # Bakery\n",
" , '52e81612bcbc57f1066b7a02' # Belgian\n",
" , '52e81612bcbc57f1066b79f1' # Bistro\n",
" , '4bf58dd8d48988d143941735' # Breakfast\n",
" , '52e81612bcbc57f1066b7a0c' # Bubble Tea\n",
" , '4bf58dd8d48988d16c941735' # Burgers. Does not include fast food.\n",
" , '4bf58dd8d48988d16d941735' # Cafe\n",
" , '4bf58dd8d48988d17a941735' # Cajun/Creole\n",
" , '4bf58dd8d48988d144941735' # Caribbean\n",
" , '5293a7d53cf9994f4e043a45' # Caucasian\n",
" , '4bf58dd8d48988d1e0931735' # Coffee Shop\n",
" , '52e81612bcbc57f1066b7a00' # Comfort Food\n",
" , '52e81612bcbc57f1066b79f2' # Creperie\n",
" , '52f2ae52bcbc57f1066b8b81' # Czech\n",
"# , '4bf58dd8d48988d146941735' # Deli\n",
" , '4bf58dd8d48988d1d0941735' # Dessert\n",
" , '4bf58dd8d48988d147941735' # Diner\n",
" , '4bf58dd8d48988d148941735' # Donuts\n",
" , '5744ccdfe4b0c0459246b4d0' # Dutch\n",
" , '4bf58dd8d48988d109941735' # Eastern Europe\n",
" , '52e81612bcbc57f1066b7a05' # English\n",
" , '4bf58dd8d48988d10b941735' # Falafel\n",
" , '4edd64a0c7ddd24ca188df1a' # Fish & Chips\n",
" , '52e81612bcbc57f1066b7a09' # Fondue\n",
" , '56aa371be4b08b9a8d57350b' # Food Stand\n",
" , '4bf58dd8d48988d1cb941735' # Food Truck\n",
" , '4bf58dd8d48988d10c941735' # French\n",
" , '4bf58dd8d48988d155941735' # Gastropub\n",
" , '4bf58dd8d48988d10d941735' # German\n",
" , '4bf58dd8d48988d10e941735' # Greek\n",
" , '52e81612bcbc57f1066b79ff' # Halal\n",
" , '52e81612bcbc57f1066b79fe' # Hawaiian\n",
" , '52e81612bcbc57f1066b79fa' # Hungarian\n",
" , '4bf58dd8d48988d10f941735' # Indian\n",
" , '52e81612bcbc57f1066b7a06' # Irish Pub\n",
" , '4bf58dd8d48988d110941735' # Italian\n",
" , '52e81612bcbc57f1066b79fd' # Jewish\n",
" , '4bf58dd8d48988d112941735' # Juice Bar\n",
" , '5283c7b4e4b094cb91ec88d7' # Kebab\n",
" , '4bf58dd8d48988d1be941735' # Latin American\n",
" , '4bf58dd8d48988d1c0941735' # Mediterranean\n",
" , '4bf58dd8d48988d1c1941735' # Mexican\n",
" , '4bf58dd8d48988d115941735' # Middle Eastern\n",
" , '52e81612bcbc57f1066b79f9' # Modern European\n",
" , '52e81612bcbc57f1066b79f8' # Pakistani\n",
" , '56aa371be4b08b9a8d573508' # Pet Cafe\n",
" , '4bf58dd8d48988d1ca941735' # Pizza\n",
" , '52e81612bcbc57f1066b7a04' # Polish\n",
" , '4def73e84765ae376e57713a' # Portuguese\n",
" , '5293a7563cf9994f4e043a44' # Russian\n",
" , '4bf58dd8d48988d1bd941735' # Salad\n",
" , '4bf58dd8d48988d1c6941735' # Scandinavian\n",
" , '5744ccdde4b0c0459246b4a3' # Scottish\n",
" , '4bf58dd8d48988d1ce941735' # Seafood\n",
" , '56aa371be4b08b9a8d57355a' # Slovak\n",
" , '4bf58dd8d48988d1dd931735' # Soup\n",
" , '4bf58dd8d48988d14f941735' # Southern\n",
" , '4bf58dd8d48988d150941735' # Spanish\n",
" , '5413605de4b0ae91d18581a9' # Sri Lankan\n",
" , '4bf58dd8d48988d1cc941735' # Steakhouse\n",
" , '4bf58dd8d48988d158941735' # Swiss\n",
" , '4bf58dd8d48988d1dc931735' # Tea Room\n",
" , '56aa371be4b08b9a8d573538' # Theme Restaurant\n",
" , '4f04af1f2fb6e1c99f3db0bb' # Turkish\n",
" , '52e928d0bcbc57f1066b7e96' # Ukranian\n",
" , '4bf58dd8d48988d1d3941735' # Vegetarian\n",
" ]\n",
"venueCategoryIdstr = ','.join(venueCategoryId) # Converts to string for API call"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With those specified, we'll go ahead and pull in the restaurants. This loop will go through each restaurant category, grab the restaurants that meet our criteria, and puts them into a data frame called dfRest (**d**ata **f**rame of **rest**aurants).\n",
"\n",
"Tasks like these can sometimes take a lot of time, but this fortunately runs in under a minute."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:30:49.368483Z",
"start_time": "2017-07-02T15:29:38.249378Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wall time: 1min 11s\n"
]
}
],
"source": [
"%%time\n",
"\n",
"# Empty data frame to be filled with restaurant information\n",
"dfRest = pd.DataFrame()\n",
"\n",
"for city in cities:\n",
" \n",
" # Run the API for each category\n",
" for category in venueCategoryId:\n",
" apiCall = client.venues.search(params = \n",
" {'near': city\n",
" , 'intent': 'browse' # Non-user based\n",
" , 'limit': '50' # Max is 50\n",
" , 'radius': '56372' # In meters. Converts to 35 miles\n",
" , 'categoryId': category})['venues']\n",
" if len(apiCall) == 0:\n",
" pass\n",
" else:\n",
" for restaurant in np.arange(len(apiCall)):\n",
" \n",
" # Error handling due to not every restaurant possessing\n",
" # a value for city/state\n",
" try:\n",
" apiCity = apiCall[restaurant]['location']['city']\n",
" apiState = apiCall[restaurant]['location']['state']\n",
" except:\n",
" apiCity = np.NaN\n",
" apiState = np.NaN\n",
" \n",
" # Appending a temporary data frame to dfRest to prevent overwriting\n",
" temp = pd.DataFrame({\n",
" 'name': apiCall[restaurant]['name'],\n",
" 'id': apiCall[restaurant]['id'],\n",
" 'category': apiCall[restaurant]['categories'][0]['name'],\n",
" 'shortCategory': apiCall[restaurant]['categories'][0]['shortName'],\n",
" 'city': apiCity,\n",
" 'state': apiState,\n",
" 'location': city,\n",
" 'checkinsCount': apiCall[restaurant]['stats']['checkinsCount'],\n",
" 'commentsCount': apiCall[restaurant]['stats']['tipCount'],\n",
" 'usersCount': apiCall[restaurant]['stats']['usersCount']\n",
" }, index=[0])\n",
" dfRest = pd.concat([dfRest, temp])"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:30:49.397504Z",
"start_time": "2017-07-02T15:30:49.377990Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"(3021, 10)"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dfRest.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we have pulled the restaurants, let's clean it up by removing any restaurants with:\n",
"- **Duplicates:** A small handful of these made it through, and we only need one of each.\n",
"- **$<$ 10 reviews:** These won't be too useful for our purposes.\n",
"- **Categories we don't want:** Some of these also got through, and I have no interest in seeing how the Burger King around the corner compares to a Whataburger back home."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:30:49.509083Z",
"start_time": "2017-07-02T15:30:49.408011Z"
},
"collapsed": true
},
"outputs": [],
"source": [
"# Cleaning the dataset before making more API calls\n",
"dfRest = dfRest.dropna().drop_duplicates()\n",
"dfRest = dfRest[dfRest['commentsCount'] >= 10]\n",
"\n",
"# Removing unwanted categories\n",
"excludeCategories = ['Fast Food', 'Gas Station', 'Golf Course', 'Bowling Alley']\n",
"dfRest = dfRest.loc[~dfRest.category.isin(excludeCategories)]"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:30:49.535602Z",
"start_time": "2017-07-02T15:30:49.515087Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"(1362, 10)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dfRest.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"While that removed more than half of our records, ~1,300 restaurants is still plenty enough to work with. This will also drastically reduce the number of reviews we need to grab, as well.\n",
"\n",
"Moving on, let's look at a few charts for exploratory analysis to see what restaurants we ended up with:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:30:50.344177Z",
"start_time": "2017-07-02T15:30:49.543609Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAfsAAAFECAYAAAAgF5dyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8XNWd///XFPViuci23Cn2wVSHTmjZYLJLsoHs0r4J\nC6nwZYGU3bTdBX4hhWw2CYGQngBL8iOksYRNARJKKCGhY4oxH8C4W7Zly6oz0tTvH/eOPBaSNZKm\nSKP38/EQmrn3zsy512Le95R7biCdTiMiIiLlK1jqAoiIiEhhKexFRETKnMJeRESkzCnsRUREypzC\nXkREpMwp7EVERMqcwl6kCJxz85xzVaUuR7E450LOuUWlLsdgzrkm59z0In3W/uX0OTK5hUtdAJka\nnHNXAykzu9Y59+9ApZl9fphtVwKfAY7COyF9AfiCmT1QtALnkXNuDvAqsAToL21pPM65DwA3A1F/\nURoIAC8CnzCzJ8b5Eb8AHge+Ps73ybfXgb8Dnh68wjn3EHACEPcXpfD+9r5oZn/wtzkZ+JWZzd3X\nhzjn3g1cCxw+zPoLgI+b2bH+v8WnzOzQ0e7M4M9xzq0GrjSzu0b7XlLeVLOXYvlb4N4hHu/F/+K7\nHfge0ALMAX4M/NY5d1rhi1kQNUBdqQsxhDVmVu//NOAd62eB/3HOhcb53rPGX7yCmDnC+iszxwSY\nDfwQuNM5dzqAmT06UtBnfc6w369m9lMzOzbXQuf6OWZ2iIJehqKavRSUc+554EC8wHvYORfwHz/g\nnDvZzJ7L2rYWuAH4oJn9OuttbnHOzQaW+68LAVcBHwIagCeBj5nZq865JcBLwL/521QCXwBiwJX+\nZ19rZteNZlu/fAcDN+K1OGwHvmRmt/nrHgL+jFdrdMDLwCVm9jzwvL8f651zZwAJ4Fv+cdkG3GJm\nXx3i2H0AeD/QCpwJbMarAf7OX7/Af5+TgS7gO8A3zCztnLsVqAaOA5LAwWYWG+7fCcDMep1zNwGX\nAdOBnSN8xglD7Ydz7tv+9ic45w40s0udc5cBlwKL/eP7EzP7pL8f6/39usN/fg1wtJn9vf/4GGAR\nXvgeARwEfNE/zrXAg8BFZtbh73c3cAhwLLAOr6XiAf9vEeAR59zFZvbTEY5HH/AT59xS4EvAfc65\ntwG/M7N6/+/1JuAd/j49Dvxf/3h8H6hwzvX4264H/gD8I/BH4D72rs1XOud+CJwNbAU+Y2b3+Mcj\nDRxjZk/7z28FeoCfDvM5nzKzO5xzzXgtK2fg/Q38Fvi0mXX6f1sXABv9z+wEvmlm39jXMZHJSzV7\nKSgzOwJYCTzs15ZWAo/5tafnBm3+VqAK+N0Q7/MVM/u2//TzwHuB0/Fq/08Bf3TOZWrPdcCheAFx\nOd4X3jHA/sAHgP9yzk0bzbbOuXq8L+g/4IXOhcB1zrlTs4r5Abwv0LnAFuC//OVH+L+XmNmjeLXF\nH5hZE3AucJV/IjGUtwGrgRnA1cAdzrn9/ROe3+J9WS/Aay25BLgo67WnAacAR40U9AB+X/ZngBfM\nbGcOnzHkfpjZFcCjeLXkS51zJ+KF8/lmNg14J/Ax59xxI5Upaz/+CViGF+R3Ad8ys9nAUn/5/83a\n/oN4J3AzgYfwTlAyf4sAp4wU9IP8HjjGD/ds/4z3t7DQL0cd8C9m9le8E5s1/t98xlK8k53LhviM\npYDh/e18Aa81YcG+CrWPz8n4Nd7J8DK8v/GFeF03GSvx/t+ZBXwW+OpInymTl8JeiuEkvFrv4MeD\nNQPtZhYfZn3GRXi16lfNrB/4HFCBFwoZ1/sB9yAQwqu19ON1H4SA+aPc9l1Ar5l9zcziZvYUcAte\n+GXcbp5e4Fd4X7JD6QT+0a/lvw40mdnLw2y7EfiymcXM7Fd4fc1nA0f77/8pM+szs9eAbwwqzyNm\ntsnMOod57+XOuQ7nXJdzrh9vXEE3XusEOXxGrvvxHLDCzNb4LTR1/ufMH2Lboaw2s+f9/ejDq/X/\n0jnXCMwD2ga9191m9qT/b3g7w/875GoX3niGxkHLO/FaEC7EOxk7w8z+Yx/v82sziwzz7/GGmV3n\n/239CngGrxVgTPxBeyfitXh1mNku4F+As7NOdNvM7PtmljCzn+O1OGmwX5lS2EtBOeeexRtA9C/O\nuQ7/8Uf9kFkxaPNWYIZzrmKI92l0zlX7T+cA6zPrzCwFbMKrnWe0+7+T/u8O/3fK/x0c5baLgf38\ncnf4+3IFXo03Y0fW4zjD//91Hl5T7U3+Z/+3H1xDecPMsu9WtRmv9rcYr5m+Las8X2Pv0Ns2zHtm\nrDGzJjNrBM7CC7Q/m1mrv36kz8h1P5LAfzjndgIPAxf7n5Xr98/AfphZEjjDb65eDVwDTBv0XoP/\nHQI5fs5wZuMNrNwxaPnNwFfxTn7WA0/7XRvD2de/x4ZBzzeT+8nQUOYACTPbnLVsvf97of978P4k\nUCaULf3DSkGZ2ZF4QbzMb+4deGxmqwZt/le8mts7h3irLwL3+483AvtlVjjnMmGc/WU6mts55rLt\nVmCVX+4mf1+WAeeP4nNwzoXxxh78s5nNx+tTPwb412FeMm/Q88V4x3ArsGtQefbD6yvPyPkYmNm9\neM3LNznnTvIXD/sZo9yPT+K1Eiw1s+V4TfLZAZxk7/FDgwfRDeyHH6ZfxqtFLzSzd5N14lcg7wKe\n8U8qsy0HfmtmR+OdEDwK/Gwf77Ovf4+h/p0zJwAp9n18hrIRCDvnFmYt298vw/YcXi9lRgP0pKD8\nml69mW3NfjzUtmbW75z7DPBDf1DS3XiD5j6MV3s609/0v/H6h5/A+0K8Gi88/kBuX4Rj8XvgG865\nD+NdHbAQr5n/53jdCPuSudyuEa9J+Bbg+865r+IFaspfPpRlzrmL8fb5POAwvP7x7XgD6D4HfMV/\n718Da/EG9Y2a3zT+HuDHzrnDgSf28RkfGGE/+tnT7N2IN4At5vd7X41XG8+04LwKnOuc+y3eycTZ\neFcFDGUa3slB1D/JOxuv2+G/c9zNGG9ujh+S35J0AV4LzplDbPJeYKV/+dtuoJe997/eORcc4iRh\nmI9z/4zXWnAe3onEHf66V4H3+q1kxwF/A9y6r88xsy3OufuAbzrnPoTXHXUdcK+ZtTnncjkEUkZU\ns5dCOxzvWuXBj4dkZj/C+3L9d7xmxi14fZdnmNl9/mZfwwvZPwI78a6NXmlm3Xkv/Z5y7cYbdf1P\neH3Ef8EbvPbFHF6+DW/Q4Yt4tcRz8MKjHVgDPIB3qeFQ3sAbpLcTb/Dc35vZZn9cw7vwrgzY7L/P\nq3iDDMfjo0A98J/7+gy/a2Ff+3Eb8Ann3O14IdPjH4e1eN0Qf8QLNIBP+cu2AT9g38H9B/+9n8P7\nd7jUf83yfbwm283A7/yrA4ZyrXOuxznXg3cieR7wbjN7aIhtv4y332vwun7eivf3AV53RR/Q4Zyb\nkUO5/oI35qQdr2/9nWaWaWa/BO9voB3v/4ufZL1uX59zARDBG/j3Gt7xvSCHskgZCqTTo2ntFJFi\nGc9kKyIi2VSzFxERKXMKexERkTKnZnwREZEyN6lH4/uX/ywANptZotTlERERmYgmddjjBf26Bx6Y\nlDdDExERGYtRTxSlPnsREZEyp7AXEREpcwp7ERGRMqewFxERKXMKexERkTKnsBcRESlzCnsREZEy\np7AXEREpcwp7ERGRMqewFxERKXMK+xJ74qVWXt/cUepiiIhIGVPYl1B3JMaXb32S//rJU6RSuvug\niIgUhsK+hF5au4tUGrbtirD6jV2lLo6IiJQphX0JvbR258Dj+57cUMKSiIhIOVPYl9CLa3dSGQ4y\nd2Ytj73QSm80XuoiiYhIGVLYl0h3JMb61i4OWjKDdxy3mFg8ySOrtpS6WCIiUoYU9iXy0tpdpNNw\n2IGzePvRCwkG4H415YuISAEo7Esk019/2AGzmDmthiMPmsOrGzvY0NpV4pKJiEi5CeeykXPufcBV\nQAVwg5l9Z9D6FcBNQCPwCHCpmSWccycC1wOVwC7gQ2a2wTnXBPwU2B9oA84zs2152qdJIdNfv2xR\nEwCnH7uIp9ds574nN/KRsw4tcelERKScjFizd87NB64FTgJWAJc45w4etNltwBVmtgwIABf7y38K\nfMTMVviPb/SXfwl41MyWAz8CvjneHZlMsvvrK8IhAI45eC6NdZU88tzmEpdORETKTS7N+CuBB82s\n3cx6gTuAczIrnXOLgRoze9xfdCtwrnOuCrjKzF7wl78ALPIfvwsv/AF+BpzhnKsY155MItn99RkV\n4SAHLZ7B7u5+unpjJSydiIiUm1zCfh7QmvW8FVgw0noz6zez2wCcc0HgGuCuwa8xswTQBTSPofyT\nUnZ/fbb5s+sB2LKjp+hlEhGR8pVLn30QyJ7LNQCkcl3vnKsEfux/1peztmG41wzFOXcN8Lkcyjvh\nDe6vz5jfXAfAlrZulu83oxRFExGRMpRL2G8GTs56PhfYOmh9y1DrnXP1wG/wBuedZWaZWWO2+Ntt\nds6FgQZ/m2GZ2TV4rQMDnHNLgHU57MOEkemvP+yAWQP99Rnzm/2afVtvKYomIiJlKpdm/PuB05xz\nzc65WuBs4N7MSjPbAPT5I+8BLgTu8R/fBrwOnG9m/VnveTdwkf/4fLzBelNi+riN27pJp2HpwqY3\nrdsT9mrGFxGR/Bkx7M1sC3Al8CdgFXC7mT3pnLvbOXe0v9kFwPXOuVeAeuBG59xbgLOAE4FnnXOr\nnHN3+9tfDRzvnFsNXAZcnte9msA6erxznhmN1W9a19RQRW11WGEvIiJ5ldN19mZ2O3D7oGXvzHr8\nPHDsoJc9x5v75jPbtwNnjqqkZaLTD/tp9VVvWhcIBJjXXM+G1i6SqTSh4JCHT0REZFQ0g16RdXR7\nYd/U8OawB1jQXE88kaJtd6SYxRIRkTKmsC+ykcJ+nvrtRUQkzxT2RZbps28aohkfvJo9KOxFRCR/\nFPZF1tHdTzAYoKG2csj18zLX2mtiHRERyROFfZF19PQzra6S4DCD7zLN+Ft1rb2IiOSJwr7IOnv6\nh+2vB6ipCjNzWjWb1YwvIiJ5orAvolg8SaQvMeRld9nmN9ezsyNKXyxRpJKJiEg5U9gX0Ugj8TMy\nM+m17lRTvoiIjJ/CvohGGomfMXD3OzXli4hIHijsiyjnsG/WrW5FRCR/FPZFNNpmfNXsRUQkHxT2\nRZRr2M+eXkM4FFDYi4hIXijsi2hfN8HJFgoFaZlVx5a2XtLpdDGKJiIiZUxhX0SZmv30EWr2APNm\n1dMbjdPZEyt0sUREpMwp7IsoM0CvsW7ksG+Z5U2bu61dl9+JiMj4KOyLqKOnn/qaCirCIx/22dNr\nAWhrjxa6WCIiUuYU9kXU0b3vqXKzzZnhhf123ddeRETGSWFfJMlkiu5ILOewn+2H/Y52hb2IiIyP\nwr5IOntjpNMjj8TPmD29BoAdqtmLiMg4KeyLJHPZ3fQcw762uoL6mgqFvYiIjJvCvkh25zihTrbZ\nM2rZ3h7VtfYiIjIuCvsiyVxjn2szPniD9GLxpK61FxGRcVHYF0mmGX9UNXv/8js15YuIyHgo7Isk\n13nxs2mQnoiI5IPCvkhyvb1tNl1+JyIi+aCwL5KxhP3AxDoKexERGQeFfZF0dPdTXRmiuiqc82ua\nB/rsNWWuiIiMncK+SDq6+0c1Eh+gvqaCuuqw+uxFRGRcFPZFkEql6ezJfV78bLNn1LKjPaJr7UVE\nZMwU9kXQ2xcnmUqPqr8+Y/b0WvpiSbp6da29iIiMjcK+CMZy2V1GZpBem/rtRURkjBT2RTAQ9mOo\n2WcG6elWtyIiMlYK+yLoGMPseRlzZvgT6+jyOxERGSOFfRFk+tsb6ypH/VpNmSsiIuOlsC+C3mgc\ngLqailG/ds8seuqzFxGRsVHYF0Gkb+xhX19TQU2VrrUXEZGxU9gXQU+mZl89+rAPBALMmVHLdl1r\nLyIiY6SwL4JIXwIYW80evH77aH9ioDtARERkNBT2RTCePnvYc6tb3RBHRETGQmFfBL3ROOFQgMrw\n2A73wCA99duLiMgYKOyLoLcvTl1NBYFAYEyvnz1wq1uNyBcRkdFT2BdBbzRO7RgG52XMmZ6ZMlc1\nexERGT2FfRH09iXG3F8P0Kw+exERGQeFfYHFEyli8SR11eExv0djXSXVlSH12YuIyJgo7AtsPBPq\nZAQCAe++9rrznYiIjIHCvsB6xzGhTrbZ02vpjcYHJugRERHJlcK+wHrzULOHPdfaa5CeiIiMlsK+\nwDI1+/GMxgeYM3D5ncJeRERGR2FfYL0DU+WOfYAeaGIdEREZO4V9geWzzx50q1sRERk9hX2B5WM0\nPmSFvWr2IiIySjm1LTvn3gdcBVQAN5jZdwatXwHcBDQCjwCXmlkia/0XgaSZXeM/PxW4E9jkb/Kc\nmX1wfLsyMY3n9rbZptVXUlkRUp+9iIiM2og1e+fcfOBa4CRgBXCJc+7gQZvdBlxhZsuAAHCx/9pp\nzrmbgU8O2v5o4OtmtsL/Kcugh/Hf3jbDu699jUbji4jIqOXSjL8SeNDM2s2sF7gDOCez0jm3GKgx\ns8f9RbcC5/qPzwJeA64b9J7HAO9wzr3gnPuNc27hOPZhQhvv7W2zNU+vpTsSH+gaEBERyUUuzfjz\ngNas563AsSOsXwBgZj8BcM5dM+g9O4BfmtmdzrlLgZ8DJ+6rEP57fC6H8k4oewbojW80Puy5Ic6O\n3VGWtIz/5EFERKaGXGr2QSCd9TwApEax/k3M7FIzu9N//H3gEOfctBFec42ZBbJ/gP1yKH9JZSbV\nqRlnnz3o8jsRERmbXMJ+M9CS9XwusHUU6/finAs65650zoUGrUoM+YJJrjcap6YqTCg4tnvZZxuo\n2WuQnoiIjEIuYX8/cJpzrtk5VwucDdybWWlmG4A+51ymGf5C4J7h3szMUsA/+O+Dc+4i4Al/PEDZ\nGe/tbbM1z9CtbkVEZPRGDHsz2wJcCfwJWAXcbmZPOufuds4d7W92AXC9c+4VoB64cYS3fT/wCefc\nauCDwEfGugMTXW80npf+esjus1fYi4hI7nJKITO7Hbh90LJ3Zj1+nr0H7Q1+/TWDnq8G3jqagk5G\nqVSaaF+curkNeXm/poYqKsNB3epWRERGRTPoFVBfLEEqPf6b4GQEAgGap9eqz15EREZFYV9AvVFv\nzGF9nvrswbvVbVdvjGh/WY5nFBGRAlDYF1DmsrvaPPXZgy6/ExGR0VPYF1A+Z8/LyNzXvk399iIi\nkiOFfQFlavbjvQlOtszd73T5nYiI5EphX0CRAtbsFfYiIpIrhX0B9ebp9rbZ5s6sA2DbrrKcg0hE\nRApAYV9AvXm6vW22afWV1FSFad2psBcRkdwo7AtozwC9/I3GDwQCtMyqY+vOXtLp9MgvEBGRKU9h\nX0B7Lr3L7+1oW2bVEYsnae/qy+v7iohIeVLYF1CmZp/PSXUA5s3y+u3VlC8iIrlQ2BdQJuxr8xz2\nLTMV9iIikjuFfQFF+hKEQ0Eqw/k9zHMzNXuNyBcRkRwo7AuoJxqnriZMIBDI6/uqGV9EREZDYV9A\nkb54Xq+xz5jeUE1lRUg1exERyYnCvoB6o/G899cDBIMBWmbW0qrL70REJAcK+wKJJ5LEEinqC1Cz\nB+/yu0hfgq7eWEHeX0REyofCvkAy97KvzeOEOtlaZtUD6rcXEZGRKewLJFKAO95la/EH6W1V2IuI\nyAgU9gXSU4A73mWbp2vtRUQkRwr7Ahmo2Rco7Ft0+Z2IiORIYV8gA3321YXps5/ZVEM4FKR1V09B\n3l9ERMqHwr5AMjfByfe8+BmhYIC5/uV3IiIi+6KwL5CBefELNEAPvKb87kicnoguvxMRkeEp7Auk\nt8B99pDVb6+Z9EREZB8U9gVSqNvbZtOIfBERyYXCvkAifZkBeoWs2WtiHRERGZnCvkB6C3ydPcDc\nWbWAJtYREZF9U9gXSGZSnZqqwlx6BzB7ei3hUIAtbbr8TkREhqewL5BIX5za6jChYH7vZZ8tHAoy\nr7meTdu7dfc7EREZlsK+QHqj8YL212csnNNApC/Bzo6+gn+WiIhMTgr7AuntSxR0JH7G4jkNAGza\n3l3wzxIRkclJYV8AqVR6oBm/0BbNbQRg4/augn+WiIhMTgr7AuiLJUinCzsSP2PRXK9mv3GbavYi\nIjI0hX0BDNzetgh99i2z6giHAgp7EREZlsK+ADIT6hSjZp8Zkb9RI/JFRGQYCvsC2HMTnML32QMs\nmtNAtD9BW0e0KJ8nIiKTi8K+AAp9e9vBMoP0NCJfRESGorAvgGLc3jabBumJiMi+KOwLIFKEefGz\nLZqjsBcRkeEp7Augpwj3ss82MCJf19qLiMgQFPYFEIn6o/GLNEAvHAoyX3Pki4jIMBT2BdBb5Jo9\neIP0ov1J2nZrRL6IiOxNYV8AvUWcVCdjYJCeRuSLiMggCvsC6C3yAD3QID0RERmewr4AevvihENB\nKitCRfvMhZmw1yA9EREZRGFfAL3R4tzeNtu8WXWEQ0HV7EVE5E0U9gXQW6Tb22YLhYIsmO3NkZ9M\naUS+iIjsobAvgEg0XtT++oylC5vojyXZrEF6IiKSRWGfZ/FEklgiVdSR+BkHLmwC4LVNu4v+2SIi\nMnHl1NbsnHsfcBVQAdxgZt8ZtH4FcBPQCDwCXGpmiaz1XwSSZnaN/7wJ+CmwP9AGnGdm28a9NxNA\nb7R4t7cdbKkf9q9u6mDlsYuL/vkiIjIxjVizd87NB64FTgJWAJc45w4etNltwBVmtgwIABf7r53m\nnLsZ+OSg7b8EPGpmy4EfAd8c115MIJkJdYrdZw+wpKWRcCjI65s6iv7ZIiIyceXSjL8SeNDM2s2s\nF7gDOCez0jm3GKgxs8f9RbcC5/qPzwJeA64b9J7vwqvZA/wMOMM5V/yqcAGU4hr7jIpwiCXzGlm3\ntYt4IlX0zxcRkYkpl+rnPKA163krcOwI6xcAmNlPAJxz1wz3nmaWcM51Ac3A1uEK4b/H53Iob0mV\nMuzBa8p/fVMH61s7WbpweknKICIiE0suYR8Esq/lCgCpUawfSmCI5/t8jd/ff032MufcEmDdCJ9V\nVJG+zE1wShT2C5q4B3h9U4fCXkREgNya8TcDLVnP57J3DXyk9UPZ4m+Hcy4MNAC7cijLhNdT6pr9\nIi/gX1O/vYiI+HIJ+/uB05xzzc65WuBs4N7MSjPbAPQ55070F10I3DPCe94NXOQ/Ph9vsF58VCWf\noCKZO96VYIAewMLZ9VRVhhT2IiIyYMSwN7MtwJXAn4BVwO1m9qRz7m7n3NH+ZhcA1zvnXgHqgRtH\neNurgeOdc6uBy4DLx7oDE02p++xDoSD7z5vGxm1d9MUSI79ARETKXk7VTzO7Hbh90LJ3Zj1+nr0H\n7Q1+/TWDnrcDZ46moJNFKe5lP9jSRU2sWd/OG1s6OXi/mSUrh4iITAyaQS/PSnEv+8EyA/PUlC8i\nIqCwz7vMaPzaUtbsM9PmblTYi4iIwj7veqJxAgGorSrNAD2Alpl11FWHeX2z5sgXERGFfd71RuPU\nVIUJBgdPJVA8wWCAAxc2saWtd+BSQBERmboU9nkW6SvN7W0HO2jxDADWrCuL6QtERGQcFPZ51huN\nl3RwXsZhB8wC4MW1CnsRkalOYZ9HqVSaSH9iQtTs3ZLphEMBXlq7s9RFERGRElPY51G0P0E6XZrb\n2w5WXRlm6cLprN3cMTCrn4iITE0K+zyaCBPqZDv0gJmk0vDyuvZSF0VEREpIYZ9HE2FCnWyZfns1\n5YuITG0K+zwauL3tBKnZL18yg1AwwIsKexGRKU1hn0d7aval77MHqK4Ks3RhE69v7lS/vYjIFKaw\nz6OJ1mcPcNiBs0il0qxZr357EZGpSmGfR6W+ve1QDs1cb/+6mvJFRKYqhX0eZWr2tRNkgB54/fbB\nYICX3tDkOiIiU5XCPo96o94AvfoJVLOv8fvtX9vUQbQ/UeriiIhICSjs8ygyULOfGAP0Mg47wOu3\nf1nz5IuITEkK+zzqmYB99gArljYD8KztKHFJRESkFBT2eTTRJtXJOHj/GVRVhnj2FYW9iMhUpLDP\no0hfnHAoSGVFqNRF2UtFOMThB85i844etrdHSl0cEREpMoV9HnX3xmmsm1i1+oyjDpoDwLOvbC9x\nSUREpNgU9nnUFYnRUFtZ6mIM6aiDZgPwjJryRUSmHIV9niSTKXqjcRrqJmbYz51Zx7xZdbzwehvx\nRKrUxRERkSJS2OdJd8QbnDdRa/YARy2fQ7Q/yZr1ugRPRGQqUdjnSXckBkDjBK3ZAxzpvKZ8jcoX\nEZlaFPZ50tU78cP+sANnURkOqt9eRGSKUdjnSaZmP5Gb8asqQhx6wCzWt3axqzNa6uKIiEiRKOzz\npLt34oc9wJEalS8iMuUo7PNkMvTZAxy93Lve/snV20pcEhERKRaFfZ50TZKa/fzmehbOqee5V9vo\ni+kueCIiU4HCPk8GLr2boDPoZTvukBZi8SSrXm0rdVFERKQIFPZ5MhkG6GUcf+hcAB5/qbXEJRER\nkWJQ2OdJV2+MQADqJ0HYL104nRmNVTy5ejvJpGbTExEpdwr7POmOxKirriAUDJS6KCMKBgMce0gL\n3ZEYa9a3l7o4IiJSYAr7POnujU3YefGHsqcpX6PyRUTKncI+D9LpNF29MRonQRN+xuEHzqKmKswT\nq1tJp9OlLo6IiBSQwj4Pov0Jkqn0pKrZV4RDHHXQbLbtirBhW3epiyMiIgWksM+DyTAv/lCOP7QF\ngCc0Kl9EpKwp7PNgMl12l+2o5XMIhwL8+fmtpS6KiIgUkMI+D7p7J8+EOtnqayo46qA5rG/tYkNr\nV6mLIyIiBaKwz4OuzLz4k6xmD3DqkQsAePi5zSUuiYiIFIrCPg8G7ng3yfrsAY49ZC41VWEefnYz\nqZRG5YuIlCOFfR5M1j578O5xf8JhLezYHdUEOyIiZUphnwfdk3Q0fsZAU/6zasoXESlHCvs86JrE\nNXuAIw6cRVNDFX9+fgvxhObKFxEpNwr7PJjMffYAoVCQU1bMpzsS57lXd5S6OCIikmcK+zzojsSo\nrAhRVRG91209AAAYqElEQVQqdVHGbKAp/xk15YuIlBuFfR50ReI01k6ua+wHW7qwifnNdfz1pVY6\ne/pLXRwREckjhX0edPf2T9om/IxAIMA7T9yPeCLFPX9dX+riiIhIHinsxymeSBHtT07awXnZVh6z\niNrqML9/bB3xRLLUxRERkTwJ57KRc+59wFVABXCDmX1n0PoVwE1AI/AIcKmZJZxzi4DbgNmAAReY\nWY9z7lTgTmCT/xbPmdkH87FDxZa5xn6yXnaXrba6gncct5i7Hl7Lo6u28PajF5W6SCIikgcj1uyd\nc/OBa4GTgBXAJc65gwdtdhtwhZktAwLAxf7y7wLfNbODgKeBq/3lRwNfN7MV/s+kDHqY/CPxB3v3\nSfsTDMD/PvyG7nMvIlImcmnGXwk8aGbtZtYL3AGck1npnFsM1JjZ4/6iW4FznXMVwCn+9gPL/cfH\nAO9wzr3gnPuNc27huPekRCbzvPhDmT2jlhMOm8cbWzt5ae2uUhdHRETyIJewnwdk3/C8FViQw/pZ\nQJeZJYZ4XQfwLTM7HLgb+PlIhXDOXeOcS2f/AOtyKH9BlVvNHuCsUw4A4H8fWVvikoiISD7kEvZB\nILs9NwCkclg/eDmZ15nZpWZ2p//4+8Ahzrlp+yqEmV1jZoHsH2C/HMpfUJN5XvzhHLRkOssWNfHk\ny9t4Y0tnqYsjIiLjlEvYbwZasp7PBbbmsH4HMM05l5lppgXY6pwLOueuzFqekWAS6prk8+IPJRAI\n8E9/t5x0Gn5414vquxcRmeRyCfv7gdOcc83OuVrgbODezEoz2wD0OedO9BddCNxjZnHgUeB8f/lF\n/vIU8A/+++Ccuwh4wh8PMOl0R+IANEzySXUGe4ubzfGHzmX1G7t4dNWWUhdHRETGYcSwN7MtwJXA\nn4BVwO1m9qRz7m7n3NH+ZhcA1zvnXgHqgRv95Zfhjd5/GTgZ7/I9gPcDn3DOrQY+CHwkXztUbOXY\nZ5/x4TMPpSIc5Jbfrqavf1I2vIiICBCYzE20zrklwLoHHniABQsWjLR5QXzplid4YvU2fvbFM6gv\no377jP//njX88v5XOW/lMi48Y3mpiyMiIt7YuFHRDHrj1NUbIxjwJqQpR+e+fSkzp1Xz64deZ0tb\nT6mLIyIiY6CwH6fuSIz62kqCwVGfaE0K1VVhLj7rMOKJFF/9ydPE4ppGV0RkslHYj1NXb6ysLrsb\nyolHzONvj1/MG1s7uel/Xyp1cUREZJQU9uOQTKboicSYVl/eYQ9w8XsOY0lLI/f8dT2PPKd73ouI\nTCYK+3Fo7+onlYZZTTWlLkrBVVWE+OxFR1NTFeLbv1ql/nsRkUlEYT8ObR0RAJqnQNgDLJjdwOXn\nrCDan+SLNz8+MHugiIhMbAr7cWjbHQWgeXptiUtSPKceuYCz/+ZAtrT18pUfP0U8kRr5RSIiUlIK\n+3Fo68iE/dSo2Wdc9M6DOeGwFl54fSff+5/nNZ2uiMgEp7Afh7bdU6sZPyMYDPCv7zuSAxdM474n\nN3LHg6+VukgiIrIPCvtx2FOznzrN+BnVlWGu+tBxzGqq4Sd3r+HBpzeWukgiIjIMhf04tO2OUlMV\npq46XOqilMTMaTVcc/Hx1NVUcOMvVvHMK9tLXSQRERmCwn4c2jqiNE+vIRAoz9nzcrF4biNXf+g4\nQsEAX/nxU7y2aXepiyQiIoMo7Mco0henNxqfcv31Qzlk/5l86p+OIhZP8rkfPs66rZ2lLpKIiGRR\n2I/RzincXz+UEw6bx+XnrqA7EuOq7/+FDa1dpS6SiIj4FPZjNDA4TzX7Ae84bjGXn3MEXb0xrvz+\nY2zYpsAXEZkIFPZjlJlQZypMlTsaf3fCEi475wg6e2Jc+b3HWP3GrlIXSURkylPYj9FUnVAnF2ec\nsIQrzj2C7kicq77/GH94fEOpiyQiMqUp7Mdoqk6ok6u/PX4JX7jkBGqqwnz7V6v4wZ0vEIsnS10s\nEZEpSWE/Rm0dUQIB71pzGdoRS5v5xidOZfHcBn732Dqu+NqfeHqNrsUXESk2hf0Yte2OMr2hmoqw\nDuG+zJ1Zx9c+dgpnnXIA23dH+PxNj/OlW57QaH0RkSKamlO/jVMylWZXZ5QDFjSVuiiTQk1VmI+c\ndSinH7uI7935Ak+s3sYTq7dxpJvNe049gBXLmqf0xEQiIoWmsB+Dju4+Esm0+utHaXFLI/952Yk8\nsXobdz28lmdtB8/aDpa0NPKeUw/glLfMpyIcKnUxRUTKjsJ+DKbyDXDGKxAIcPyhLRx/aAuvbtzN\n/z68lj+/sJUbfv4cP7n7Zf7hbQfy9yftTzik7hERkXzRN+oYZK6xV81+fJYtms6nLzyaH/37St5z\n6gFE+5Pc/JvVfOy6h3hx7c5SF09EpGwo7MdgIOx1jX1ezJ5Ry4fPPJSbrzqdM05YwuYd3fzHdx/j\n+p89S7Q/UeriiYhMegr7MdjZqZp9ITTUVnLZOUfw9Y+dwoELm3jw6U18+sZHaN3ZW+qiiYhMagr7\nMRiYUEd99gWxbNF0vvbRk3n3yfuzYVs3/3rDwzxrO0pdLBGRSUthPwZtHVGqKkM01FaUuihlKxwK\ncsl7DuPj57+F/niSz//or/zm0bWk0+lSF01EZNJR2I9B2+4ozU01uja8CFYeu4ivXH4S0+qr+NFd\nL/H9O18gmUyVulgiIpOKwn6U+mIJunpj6q8vomWLpvP1j5/CkpZG7v7Ler5wyxP0RGKlLpaIyKSh\nsB+lHe1ef71ubVtcs6fX8l9XnMRRB83m2Vd2cNlXH+SxF7aqWV9EJAeaVGeUXtvUAcD+86eVuCRT\nT211BVd/6DjufOh1fvZH4ys/forjD53L/zndsd+8aQSDb+5W2dUZ5dWNu9mwrZvO7n46e2NE+uIs\nmtvI8iUzOHi/GUyrryrB3oiIFI/CfpTWrG8H4KAlM0pckqkpFApy7mnLeOvh8/j2r1bx+EvbePyl\nbTTUVnDYgbOY0VBNZ2+Mzp5+trb1sLOzb8j3eeaVHfzaf3zswXP58JmHMK+5vng7IiJSRAr7UXpl\nfTtVlSH2a2ksdVGmtPnN9Vx76Yn89aVWnn55O8+/3sZfXmjda5vpDVUcd8hcli2azv7zpzFzWjWN\ndZVUVYZ5Y0sHa9a189Sa7Tz58jaete2cefIBnH/6MmqrdZWFiJQXhf0o9ETjbNzezWEHzCKkudtL\nLhgMcOLh8zjx8Hmk02m2t0eI9CWYVl9JY13VPm8/fPiBzRx+YDPnrVzGX15o5ZbfvsSdD73OYy9s\n5eoPHcdincyJSBlRYo2CbWgnnVYT/kQUCASYO7POr8HX7DPoB7/uxCPm8d3PnsY5b1/K9vYIn/7W\nIzz+UuvILxYRmSQU9qOQ6a9frrAvO1UVId7/roP57EVHk0rDtf/9JL+43zTaX0TKgsJ+FF7JDM5b\nPL3EJZFCOemI+Xz1ipNpnl7Dbfe8wtdue4a+mG7GIyKTm8I+R8lkilc37mbhnAbqaytLXRwpoP3n\nT+MbHz+V5Utm8OiqLfzbd/48cKdDEZHJSGGfow3buon2J9WEP0U0NVRx7T+fyOnHLmLt5k7+9YaH\neerlbaUulojImCjsc7Snv15N+FNFRTjIR89bwSXvOYyeaJwv3PwE3/rlKiJ98VIXTURkVBT2OXpF\nk+lMSYFAgHefvD/X/8up7DevkT8+sYGPXvcQDz2zSTfkEZFJQ2Gfo5fXt9NQW8F8zbI2JS1paeS6\nj5/KeSuXsbMjynW3P8sl/3k/v/vzG7opj4hMeJpUJwe7OqPsaI9wzMFzdFvbKawiHOTCM5Zz+rGL\nuOvhtdz3xAZ+8OsX+dFdL7J04XRWLGtm2aLpzJ1Zy5yZdVRVhEpdZBERQGGfk+df2wno+nrxzJ1Z\nx6X/eDjvfYfj3sfX88yaHdjG3djG3Xtt11hXSUNtJQ21FdT7vxtqK2moq6ShJrOsksb6Sprqq5hW\nX0lFWCcIIpJ/CvsRJJIpfnGfDUzNKpIxrb6K81c6zl/p6I3GeWntTjZu72bbrgjbdvWyq7OP3mic\nbbt6SaZym5ynrjpMY30VTfVVNNRWUl0VoqYqTE1VmOrKMDVVIWqqK5jRUMWMadXMaKxmekP1kHf8\nExHJUNiP4L4nNrB1Zy9nvHWJ7oomw6qrqeC4Q1s47tCWN61Lp9NE+xN0R+J0R2L0RGJ0R+L0RGJ0\nRWJ09cTo6Okf+N3Z04+1R0jleIJQVRlifnM9C2c3sGBOPQtm17NgdgPzZtVRqa4EEUFhv0/R/gS3\n/9Gorgzx3tNdqYsjk1QgEKC2uoLa6grmzKjN6TWplHeC0BdLEO33fvr6k0RjCXqjcXZ39dPe1cfO\nzihb23rYvL2bN7Z0DvpcmDOjlgWzG5jf7J0EzJ9dz4LmepoaqjT+RGQKUdjvw10Pr6Wju5//c7pj\nemN1qYsjU0gwGKCupoK6mtxut5tKpWnriLJ5Rzebd/T4P91s3t7D02u28/Sa7XttX1sdZn7znvCf\nP7ue+c31zGuu3+fAwmQyxY7dUTZs62LDti42bethZ2eUXZ1R2jv7SKW9loaqiiBN9dUsmtvAorkN\nLGlp5KAlM2jQ7JMiJaGwH0ZHdz+/fug1muqr+Ie3HVDq4ojsUzAYYM6MWubMqOWog+bsta47EmOL\nfwKwpc372byjh3Vbu3htU8de2wYC0NxUw/zmeqbVV5FMpUkkU0T64mxvj9C2O/qm8QfBgDfj4KK5\nDYSCQfrjSfrjSTa39fDG1r1bGxbOaeDg/WawfMkMlu83g5aZdWphECmCnMLeOfc+4CqgArjBzL4z\naP0K4CagEXgEuNTMEs65RcBtwGzAgAvMrMc51wT8FNgfaAPOM7MJMxfp65s6+NYvVxHtT/L+dx5M\nbXVutSuRiaihtpKDlsx404RQyVSaHe2RgROALTv2nAg892rbm95nekMVyxZNZ87MWhbNaWBxSyOL\n5zYya1o1odCbp+zIvP+GbV2s3dzJmvW7sA27+cP2bv7w+AbAO0lYvmTGwAnA/vOn6YoEkQIYMeyd\nc/OBa4GjgH7gL865P5nZy1mb3QZ8xMwed87dDFwMfA/4LvBdM/u5c+5q4Grgs8CXgEfN7F3OuQuB\nbwLn53PHxiLan+Cn977Cbx9dSyoNpx+7iL89YUmpiyVSEKFggJZZdbTMquPo5Xu3BkT64vRGE4RD\nAYLBAFWVIaorR9cQmP3+x/sDF5PJFOu2dvHy+l2sWdfOy+va+euLrfz1xVbAa6FYMLueJS2NLJzT\nwNyZdcybVUfz9Boa66oI5XjVQX88SU8kRk80Tk8kTm80Tk80Rk8k7i2Lxvda3xP1tkml0wQD3j4H\nA155AoEA4VDQ754I+cci8zhMVYX/PGv9wDr/cTgUIJlKk0qlSaXTJJN7fvfHkvTHE/TFkvTFkvTH\nkvTFvOeRvvjAmI1of4JIX4L+WJJUKu23sKQJh4JUVISoDAepr6mgoa6SxrpKGuuqaKitoLGuyn9e\nOXA5aH1tBeEhTtAmmmQqTTyRJJ5IEU+kiMW9fQ8GA4SCQUKhAKGg9xMMBqgIe8darUVvlsv/vSuB\nB82sHcA5dwdwDvAF//lioMbMHve3vxX4vHPuJuAU4D1Zyx/GC/t3+esAfgZ8xzlXYWYlnXT8B79+\ngQee2kTLrDouP+cIjljaXMriiJRMZkBhvoVCQQ5c2MSBC5s48+QDSKfT7NgdZc26XaxZ3866rV2s\nb+1k47buN702EID6mkoa6yqoCIeoCAcJh4IkkqmBMIj0ecEdT+Q+lXEwwMD4iFAwQCrNQCjHEynS\n6TQ9iTT98SSxeDKfh2NUqitDA5dgBv1wCwS8E6hYIkV3JEbrztwv86yuDFHvz/dQV1NBfU0FVZUh\nKsMhKiqCVIZDVFYEqQiHCPmfBQwEaQDYk6ne+uzn6bR3QpJMpUil8H97x7TPP6HpjyXp608MPO/L\neh5PJEkkc9uXbIEAVIS9k59M+TMnAN5JnP8T3HMylznBy+xn5nkgsOdEIhjw1oWCQSrCe37CIf93\nOLjnGBEgnkjRH08Sjyc55pC5HOlmj3pf8imXsJ8HtGY9bwWOHWH9AmAW0GVmiUHL93qN39zfBTQD\nW4crhHPuGuBzOZR3zP7myIUsaZnGGW9dotnPRIogENgz1uBtRy0EvKDdsdvrXti2s5fWXRHaOiJ0\n9sTo7OmnOxIjkegnnkiRSKYGvmwrwiFqqsPMaqoZCDHvtxdkdTWVA4+z19dUhXOepyCVShNLJP3a\nuP87+7FfQ997WZJEIkUo5AXGQHj4tdNM7b+6yms9yTyvqQpTW+3NsVBVGc6pVSOdThPpS9AdidHV\nm/npp6s37v+O+Zd/7mnhaNsdYX1rYsT3LqRgAKqrwlRXhqiuCtPUUEVVRcg7qasIesHtPw4GAl6r\nSCpNKrnnhCKRTJNIpIglksQSKe+xf4IWi3snb95J3J6TuVQqTTpreaHs6uqbFGEfBLIPQwBI5bB+\n8HKyXjf4r3bwe76JmV0DXJO9zDm3BFi3r9eNxhHLmjlimWrzIqUUDAaYO7OOuTPrSl2UNwkGA1RX\nhkfdpVEsgcCeqzhGc/ySqTSRvjixeHKguTyWSBGPe+GZTKZJkyad9Y2e9p+k/f9kP0+nGXRSk/nx\nmt4zoe4dS6+VptRN7+m0t38DJwWZEwL/5CBzUpBIpv3WJK/lIdPNkEimSKf3HI8Kv2WhsiLEgtkN\nJd03yC3sNwMnZz2fy9418M1AyxDrdwDTnHMhM0v622Ret8XfbrNzLgw0ALvGtAciIjIuoWBgyl8W\nGfCb6YMEoAwbdnMZoXE/cJpzrtk5VwucDdybWWlmG4A+59yJ/qILgXv8/vdH2TPw7iLgHv/x3f5z\n/PWPlrq/XkREpFyNGPZmtgW4EvgTsAq43cyedM7d7Zw72t/sAuB659wrQD1wo7/8MuAS59zLeK0D\nV/nLrwaOd86t9re5PF87JCIiInsLpNMFHJVQYJk++wceeIAFCxaMtLmIiEg5GPUAh4l/oaWIiIiM\ni8JeRESkzCnsRUREypzCXkREpMwp7EVERMqcwl5ERKTMTcw5H3MXAti2bcLcHVdERKSgTjvttCXA\n5qx7z4xosod9C8AFF1xQ6nKIiIgUyzpgP2B9ri+Y7GH/FN7MfK1Aqe49mTnoUng61sWjY108OtbF\nVS7He/NoNp7UM+hNBM65tJmV9nZNU4SOdfHoWBePjnVxTdXjrQF6IiIiZU5hLyIiUuYU9iIiImVO\nYT9+ny91AaYQHevi0bEuHh3r4pqSx1sD9ERERMqcavYiIiJlTmEvIiJS5hT2IiIiZU5hLyIiUuYU\n9iIiImVuss+NX1LOufcBVwEVwA1m9p0SF2nSc841An8B/t7M1jvnVgLfAGqAX5jZVf52K4CbgEbg\nEeDS0dwBaqpzzn0OOM9/+nsz+4yOdeE4574AnAOkgZvN7Bs63oXlnPs6MMvMPjDcMXXOLQJuA2YD\nBlxgZj0lK3QBqWY/Rs65+cC1wEnACuAS59zBpS3V5OacOw74M7DMf14D3AKcBSwHjnHOneFvfhtw\nhZktAwLAxcUv8eTkh8w7gLfg/e0e5Zx7LzrWBeGcOxV4O3A4cDTwUefcEeh4F4xz7jTg/VmLhjum\n3wW+a2YHAU8DVxe1oEWksB+7lcCDZtZuZr3AHXhn7jJ2FwOXA1v958cCr5nZOr9mcxtwrnNuMVBj\nZo/7290KnFvswk5ircAnzSxmZnFgDd4Jlo51AZjZw8Df+Md1Nl6LahM63gXhnJuBVxH7sv98yGPq\nnKsATsH77h5YXtTCFpHCfuzm4X1pZrQCC0pUlrJgZh8xs0ezFg13jHXsx8HMVme++JxzS/Ga81Po\nWBeMmcWdc58HXgYeQH/bhfQD4Epgt/98uGM6C+jK6iIp62OtsB+7IF7/W0YA7wtT8me4Y6xjnwfO\nuUOA+4BPA2+gY11QZvY5oBlYiNeSouOdZ865jwCbzOyBrMW5fo9AGR9rhf3YbQZasp7PZU/zs+TH\ncMdYx36cnHMn4tUw/83MfoyOdcE45w7yB4hhZhHgTuBt6HgXwvnAO5xzq4AvAGcCH2HoY7oDmOac\nC/nLWyjjY62wH7v7gdOcc83OuVrgbODeEpep3DwBOOfcgf7/kO8D7jGzDUCfH1gAFwL3lKqQk41z\nbiFwF/A+M/u5v1jHunD2B37knKtyzlXiDcr7ATreeWdmp5vZoWa2Avj/gN+Y2QcZ4pj641UexTtB\nALiIMj7WCvsxMrMteP1CfwJWAbeb2ZOlLVV5MbM+4APA/+D1db7CnsE0FwDXO+deAeqBG0tRxknq\nU0A18A3n3Cq/FvQBdKwLwszuBn4PPAc8A/zFP8n6ADrexTLcMb0M70qql4GT8S6lLku6652IiEiZ\nU81eRESkzCnsRUREypzCXkREpMwp7EVERMqcwl5ERKTMKexFRETKnMJeRESkzCnsRUREytz/A59Z\nuU/84pwRAAAAAElFTkSuQmCC\n",
"text/plain": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" id name \\\n",
"0 4e17b348b0fb8567c665ddaf Souper Salad \n",
"1 4aceefb7f964a52013d220e3 Aster's Ethiopian Restaurant \n",
"2 4b591015f964a520c17a28e3 Taste Of Ethiopia \n",
"3 4ead97ba4690615f26a8adfe Wasota African Cuisine \n",
"4 4c7efeba2042b1f76cd1c1ad Cazamance \n",
"\n",
" category shortCategory checkinsCount city state \\\n",
"0 Salad Place Salad 1769 Austin TX \n",
"1 Ethiopian Restaurant Ethiopian 1463 Austin TX \n",
"2 Ethiopian Restaurant Ethiopian 1047 Pflugerville TX \n",
"3 African Restaurant African 195 Austin TX \n",
"4 African Restaurant African 500 Austin TX \n",
"\n",
" location commentsCount usersCount priceTier numRatings rating \n",
"0 Austin, TX 17 683 1.0 41.0 6.9 \n",
"1 Austin, TX 34 1018 2.0 93.0 8.0 \n",
"2 Austin, TX 31 672 2.0 88.0 8.3 \n",
"3 Austin, TX 12 140 2.0 10.0 6.2 \n",
"4 Austin, TX 11 435 1.0 15.0 8.0 "
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Cleaning the dataset by dropping duplicates\n",
"dfRest = dfRest.dropna().drop_duplicates()\n",
"\n",
"dfRest.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Reviews\n",
"\n",
"These are the actual comments people write about the restaurants, and they are what we will be comparing for our \"recommender\" system. \n",
"comments\n",
"This loop grabs the actual comments per restaurant, and puts them into the data frame dfComments. It takes ~10 minutes to run.\n",
"\n",
"Since I'm using a free developer key, I'm currently limited to 30 comments per restaurant ID. This will impact the quality for some restaurants, but we saw that most of our restaurants had under 30 comments in the earlier chart."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:47:02.635009Z",
"start_time": "2017-07-02T15:37:42.280493Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wall time: 9min 20s\n"
]
}
],
"source": [
"%%time\n",
"\n",
"dfComments = pd.DataFrame()\n",
"\n",
"for rest in dfRest['id']:\n",
" apiCall = client.venues.tips(rest)['tips']['items']\n",
" numComments = len(apiCall)\n",
" \n",
" for idx in np.arange(numComments):\n",
" temp = pd.DataFrame({\n",
" 'id': rest,\n",
" 'comment': apiCall[idx]['text']\n",
" }, index=[0])\n",
" dfComments = pd.concat([dfComments, temp])\n",
" \n",
"dfComments = dfComments.reindex_axis(['id', 'comment'], axis = 1).reset_index(drop=True)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:47:02.666014Z",
"start_time": "2017-07-02T15:47:02.645000Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | id | \n", "name | \n", "category | \n", "shortCategory | \n", "checkinsCount | \n", "city | \n", "state | \n", "location | \n", "commentsCount | \n", "usersCount | \n", "priceTier | \n", "numRatings | \n", "rating | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "4e17b348b0fb8567c665ddaf | \n", "Souper Salad | \n", "Salad Place | \n", "Salad | \n", "1769 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "17 | \n", "683 | \n", "1.0 | \n", "41.0 | \n", "6.9 | \n", "
| 1 | \n", "4aceefb7f964a52013d220e3 | \n", "Aster's Ethiopian Restaurant | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1463 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "34 | \n", "1018 | \n", "2.0 | \n", "93.0 | \n", "8.0 | \n", "
| 2 | \n", "4b591015f964a520c17a28e3 | \n", "Taste Of Ethiopia | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1047 | \n", "Pflugerville | \n", "TX | \n", "Austin, TX | \n", "31 | \n", "672 | \n", "2.0 | \n", "88.0 | \n", "8.3 | \n", "
| 3 | \n", "4ead97ba4690615f26a8adfe | \n", "Wasota African Cuisine | \n", "African Restaurant | \n", "African | \n", "195 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "12 | \n", "140 | \n", "2.0 | \n", "10.0 | \n", "6.2 | \n", "
| 4 | \n", "4c7efeba2042b1f76cd1c1ad | \n", "Cazamance | \n", "African Restaurant | \n", "African | \n", "500 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "11 | \n", "435 | \n", "1.0 | \n", "15.0 | \n", "8.0 | \n", "
\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" id comment\n",
"0 4e17b348b0fb8567c665ddaf Healthy fresh salad plus baked potatoes, selec...\n",
"1 4e17b348b0fb8567c665ddaf Make sure to print out your coupons. These fol...\n",
"2 4e17b348b0fb8567c665ddaf First of all, this place uses CANNED chick pea...\n",
"3 4e17b348b0fb8567c665ddaf Great healthy food, good prices. Try the ginge...\n",
"4 4e17b348b0fb8567c665ddaf Love coming here. Makes getting lunch so easy ..."
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dfComments.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we need to group the comments together so we have one set of comments per restaurant before joining everything together."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:47:03.062796Z",
"start_time": "2017-07-02T15:47:02.673520Z"
},
"collapsed": true
},
"outputs": [],
"source": [
"# Grouping individual comments by restaurant ID\n",
"groupedComments = dfComments.groupby('id')['comment'].apply(\n",
" lambda x: \"{%s}\" % ''.join(x)).reset_index()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is where we merge everything into one data frame, df, which has a row for each restaurant that includes all of the comments. We'll also perform an additional sanitation step here by removing non-ASCII characters from the comments."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:47:03.245426Z",
"start_time": "2017-07-02T15:47:03.066298Z"
},
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | id | \n", "comment | \n", "
|---|---|---|
| 0 | \n", "4e17b348b0fb8567c665ddaf | \n", "Healthy fresh salad plus baked potatoes, selec... | \n", "
| 1 | \n", "4e17b348b0fb8567c665ddaf | \n", "Make sure to print out your coupons. These fol... | \n", "
| 2 | \n", "4e17b348b0fb8567c665ddaf | \n", "First of all, this place uses CANNED chick pea... | \n", "
| 3 | \n", "4e17b348b0fb8567c665ddaf | \n", "Great healthy food, good prices. Try the ginge... | \n", "
| 4 | \n", "4e17b348b0fb8567c665ddaf | \n", "Love coming here. Makes getting lunch so easy ... | \n", "
\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" id name \\\n",
"0 4e17b348b0fb8567c665ddaf Souper Salad \n",
"1 4aceefb7f964a52013d220e3 Aster's Ethiopian Restaurant \n",
"2 4b591015f964a520c17a28e3 Taste Of Ethiopia \n",
"3 4ead97ba4690615f26a8adfe Wasota African Cuisine \n",
"4 4c7efeba2042b1f76cd1c1ad Cazamance \n",
"\n",
" category shortCategory checkinsCount city state \\\n",
"0 Salad Place Salad 1769 Austin TX \n",
"1 Ethiopian Restaurant Ethiopian 1463 Austin TX \n",
"2 Ethiopian Restaurant Ethiopian 1047 Pflugerville TX \n",
"3 African Restaurant African 195 Austin TX \n",
"4 African Restaurant African 500 Austin TX \n",
"\n",
" location commentsCount usersCount priceTier numRatings rating \\\n",
"0 Austin, TX 17 683 1.0 41.0 6.9 \n",
"1 Austin, TX 34 1018 2.0 93.0 8.0 \n",
"2 Austin, TX 31 672 2.0 88.0 8.3 \n",
"3 Austin, TX 12 140 2.0 10.0 6.2 \n",
"4 Austin, TX 11 435 1.0 15.0 8.0 \n",
"\n",
" comments \n",
"0 {Healthy fresh salad plus baked potatoes, sele... \n",
"1 {The lunch buffet is wonderful; everything is ... \n",
"2 {Wonderful! Spicy lovers: Kitfo, was awesome! ... \n",
"3 {Obsessed with this place. One little nugget (... \n",
"4 {West African fusion reigns at this darling tr... "
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Merging everything together into one data frame\n",
"df = pd.merge(dfRest, groupedComments, on='id')\n",
"\n",
"# Removing non-ASCII characters\n",
"df['comments'] = df['comment'].apply(\n",
" lambda comment: re.sub(r'[^\\x00-\\x7f]', r'', comment))\n",
"\n",
"df.drop('comment', axis=1, inplace=True)\n",
"\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here's an example of the comments for one random restaurant to give a better idea of the data we're dealing with:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:47:03.279450Z",
"start_time": "2017-07-02T15:47:03.255935Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"'{Make sure you check your steaks, they don\\'t cook it the way you ask for them. The Outlaw Ribeye is excellent when cooked to order!Kyle is the best waiter here he should be a trainer or head waiter an train everyone at his level of customer service. I will wait to be seated when he\\'s working.Alex is a great waitress very polite and very conscious about refills and service. I give her 4.5 stars. Not quite a 5 yet.Great place awesome staff....but alas....their is a sign banning Cody G. Sangria is awesome!!!They make good food but if you are taking to go it takes awhile. They say 15 minutes but it\\'s like 25-30. Prepare for that.Lunch specials are great & service is always good.If you don\\'t like lemon juice in your water, make sure to ask for it without.The burgers here are awesome! Freshly ground sirloin! Yummy!Try the Wild West Shrimp. Plenty to share. Yum!!!Customer service here is at 110%. Best service of any longhorns that I have been to across Texas.I was enjoying my salad when I bit into something odd. It turned out to be a shard of clear hard plastic. I called the manager over. He replied, \"glad you didn\\'t swallow that\". Not even an apology.I ordered a pork chop dinner and they brought me ribs. Talk about disorganized, plus I had to wait an hour for the pork chops.This isn\\'t Ruth\\'s Chris; understanding that is key to having a good time.The Mula drink it\\'s excellentI can\\'t believe they don\\'t have Dr Pepper here; come on, this Texas, it\\'s a requirement!!!The broccoli cheese soup was good!The passion/pineapple vodka is yummy!!!!Excelente lugarBuen servicio.This place is trash, save ur money ! Go to Texas road house.... YeaaaahhhhhhhCody G. must live here.Casey spends a lot of time here.TERRIBLE!!! Mediocre chain food and RUDE SERVICE!!! NEVER GOING BACK!!!Expensive.}'"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['comments'][np.random.randint(df.shape[0])] # Selects one restaurant at random"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Data Processing\n",
"\n",
"When working with language, we have to process the text into something that a computer can handle more easily. Our end result will be a large number of numerical features for each restaurant that we can use to calculate the cosine similarity.\n",
"\n",
"The steps here are:\n",
"1. Normalizing\n",
"2. Tokenizing\n",
"3. Removing stopwords\n",
"4. Lemmatizing (Stemming)\n",
"5. Term Frequency-Inverse Document Frequency (TF-IDF)\n",
"\n",
"I'll explain a little more on what these are and why we are doing them below in case you aren't familiar with them."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1) Normalizing\n",
"\n",
"This section uses regex scripts that makes cases every word lower cased, removes punctuation, and removes digits.\n",
"\n",
"For example:\n",
"\n",
"**Before:**\n",
"\n",
"$\\hspace{1cm}$\"ThIs Is HoW mIdDlE sChOoLeRs TaLkEd 2 EaCh OtHeR oN AIM!!!!\"\n",
"\n",
"**After:**\n",
"\n",
"$\\hspace{1cm}$\"this is how middle schoolers talked each other on aim\"\n",
"\n",
"The benefit in this is that it vastly reduces our feature space. Our pre-processed example would have created an additional ~10 features from someone who doesn't know how to type like a regular human being."
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:47:04.242945Z",
"start_time": "2017-07-02T15:47:03.317477Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0 {healthy fresh salad plus baked potatoes, sele...\n",
"1 {the lunch buffet is wonderful; everything is ...\n",
"2 {wonderful! spicy lovers: kitfo, was awesome! ...\n",
"3 {obsessed with this place one little nugget (...\n",
"4 {west african fusion reigns at this darling tr...\n",
"Name: comments, dtype: object"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Converting all words to lower case and removing punctuation\n",
"df['comments'] = [re.sub(r'\\d+\\S*', '',\n",
" row.lower().replace('.', ' ').replace('_', '').replace('/', ''))\n",
" for row in df['comments']]\n",
"\n",
"df['comments'] = [re.sub(r'(?:^| )\\w(?:$| )', '', row)\n",
" for row in df['comments']]\n",
"\n",
"# Removing numbers\n",
"df['comments'] = [re.sub(r'\\d+', '', row) for row in df['comments']]\n",
"\n",
"df['comments'].head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2) Tokenizing\n",
"\n",
"Tokenizing a sentence is a way to map our words into a feature space. This is achieved by treating every word as an individual object.\n",
"\n",
"**Before:**\n",
"\n",
"$\\hspace{1cm}$'central texas barbecue is the best smoked and the only barbecue that matters'\n",
"\n",
"**After:**\n",
"\n",
"$\\hspace{1cm}$['central', 'texas', 'barbecue', 'is', 'the', 'best', 'smoked', 'and', 'the', 'only', 'barbecue', 'that', 'matters']\n"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:47:05.222643Z",
"start_time": "2017-07-02T15:47:04.504631Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0 [healthy, fresh, salad, plus, baked, potatoes,...\n",
"1 [the, lunch, buffet, is, wonderful, everything...\n",
"2 [wonderful, spicy, lovers, kitfo, was, awesome...\n",
"3 [obsessed, with, this, place, one, little, nug...\n",
"4 [west, african, fusion, reigns, at, this, darl...\n",
"Name: tokens, dtype: object"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Tokenizing comments and putting them into a new column\n",
"tokenizer = nltk.tokenize.RegexpTokenizer(r'\\w+') # by blank space\n",
"df['tokens'] = df['comments'].apply(tokenizer.tokenize)\n",
"\n",
"df['tokens'].head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3) Removing Stopwords & Punctuation\n",
"\n",
"Stopwords are unnecessary words like *as*, *the*, *and*, and *of* that aren't very useful for our purposes. Since they don't have any intrinsic value, removing them reduces our feature space which will speed up our computations.\n",
"\n",
"**Before:**\n",
"\n",
"$\\hspace{1cm}$['central', 'texas', 'barbecue', 'is', 'the', 'best', 'smoked', 'and', 'the', 'only', 'barbecue', 'that', 'matters']\n",
"\n",
"**After:**\n",
"\n",
"$\\hspace{1cm}$['central', 'texas', 'barbecue', 'best', 'smoked', 'only', 'barbecue', 'matters']\n",
"\n",
"This does take a bit longer to run at ~6 minutes"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T15:52:04.881831Z",
"start_time": "2017-07-02T15:47:05.230649Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wall time: 4min 59s\n"
]
}
],
"source": [
"%%time\n",
"\n",
"filtered_words = []\n",
"for row in df['tokens']:\n",
" filtered_words.append([\n",
" word.lower() for word in row\n",
" if word.lower() not in nltk.corpus.stopwords.words('english')\n",
" ])\n",
"\n",
"df['tokens'] = filtered_words"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4) Lemmatizing (Stemming)\n",
"\n",
"Stemming removes variations at the end of a word to revert words to their root in order to reduce our overall feature space (e.x. running $\\rightarrow$ run). This has the possibility to adversely impact our performance when the root word is different (e.x. university $\\rightarrow$ universe), but the net positives typically outweigh the net negatives.\n",
"\n",
"**Before:**\n",
"\n",
"$\\hspace{1cm}$['central', 'texas', 'barbecue', 'best', 'smoked', 'only', 'barbecue', 'matters']\n",
"\n",
"**After:**\n",
"\n",
"$\\hspace{1cm}$['central', 'texas', 'barbecue', 'best', 'smoke', 'only', 'barbecue', 'matter']\n",
"\n",
"One very important thing to note here is that we're actually doing something called **[Lemmatization](https://en.wikipedia.org/wiki/Lemmatisation)**, which is similar to [stemming](https://en.wikipedia.org/wiki/Stemming), but is a little different. Both seek to reduce inflectional forms and sometimes derivationally related forms of a word to a common base form, but they go about it in different ways. In order to illustrate the difference, here's a dictionary entry:\n",
"\n",
"| \n", " | id | \n", "name | \n", "category | \n", "shortCategory | \n", "checkinsCount | \n", "city | \n", "state | \n", "location | \n", "commentsCount | \n", "usersCount | \n", "priceTier | \n", "numRatings | \n", "rating | \n", "comments | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "4e17b348b0fb8567c665ddaf | \n", "Souper Salad | \n", "Salad Place | \n", "Salad | \n", "1769 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "17 | \n", "683 | \n", "1.0 | \n", "41.0 | \n", "6.9 | \n", "{Healthy fresh salad plus baked potatoes, sele... | \n", "
| 1 | \n", "4aceefb7f964a52013d220e3 | \n", "Aster's Ethiopian Restaurant | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1463 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "34 | \n", "1018 | \n", "2.0 | \n", "93.0 | \n", "8.0 | \n", "{The lunch buffet is wonderful; everything is ... | \n", "
| 2 | \n", "4b591015f964a520c17a28e3 | \n", "Taste Of Ethiopia | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1047 | \n", "Pflugerville | \n", "TX | \n", "Austin, TX | \n", "31 | \n", "672 | \n", "2.0 | \n", "88.0 | \n", "8.3 | \n", "{Wonderful! Spicy lovers: Kitfo, was awesome! ... | \n", "
| 3 | \n", "4ead97ba4690615f26a8adfe | \n", "Wasota African Cuisine | \n", "African Restaurant | \n", "African | \n", "195 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "12 | \n", "140 | \n", "2.0 | \n", "10.0 | \n", "6.2 | \n", "{Obsessed with this place. One little nugget (... | \n", "
| 4 | \n", "4c7efeba2042b1f76cd1c1ad | \n", "Cazamance | \n", "African Restaurant | \n", "African | \n", "500 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "11 | \n", "435 | \n", "1.0 | \n", "15.0 | \n", "8.0 | \n", "{West African fusion reigns at this darling tr... | \n", "
![](\"http://college.cengage.com/english/raimes/keys_writers/5e/assets/students/images/annotated_dictionary_html_5c444ce8.png\")
\n",
"\n",
"Lemmatization seeks to get the *lemma*, or the base dictionary form of the word. In our example above, that would be \"graduate\". It does this by using vocabulary and a morphological analysis of the words, rather than just chopping off the variations (the \"verb forms\" in the example above) like a traditional stemmer would.\n",
"\n",
"The advantage of lemmatization here is that we don't run into issues like our other example of *university* $\\rightarrow$ *universe* that can happen in conventional stemmers. It is also relatively quick on this data set!\n",
"\n",
"The disadvantage is that it is not able to infer if the word is a noun/verb/adjective/etc., so we have to specify which type it is. Since we're looking at, well, everything, we're going to lemmatize for nouns, verbs, and adjectives.\n",
"\n",
"[Here](https://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html) is an excerpt from the Stanford book *Introduction to Information Retrieval* if you wanted to read more stemming and lemmatization."
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:15:53.578714Z",
"start_time": "2017-07-02T17:11:00.982881Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wall time: 4min 52s\n"
]
}
],
"source": [
"%%time\n",
"\n",
"# Setting the Lemmatization object\n",
"lmtzr = nltk.stem.wordnet.WordNetLemmatizer()\n",
"\n",
"# Looping through the words and appending the lemmatized version to a list\n",
"stemmed_words = []\n",
"for row in df['tokens']:\n",
" stemmed_words.append([\n",
" # Verbs\n",
" lmtzr.lemmatize( \n",
" # Adjectives\n",
" lmtzr.lemmatize( \n",
" # Nouns\n",
" lmtzr.lemmatize(word.lower()), 'a'), 'v')\n",
" for word in row\n",
" if word.lower() not in nltk.corpus.stopwords.words('english')])\n",
"\n",
"# Adding the list as a column in the data frame\n",
"df['tokens'] = stemmed_words"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's take a look at how many unique words we now have and a few of the examples:"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:38.129948Z",
"start_time": "2017-07-02T17:30:36.895071Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of unique words: 36592 \n",
"\n",
"Previewing sample of unique words:\n",
" ['andhitachino' 'andhome' 'andhomemade' 'andhopping' 'andhot' 'andhouse'\n",
" 'andhuge' 'andiamo' 'andimmediatly' 'andis']\n"
]
}
],
"source": [
"# Appends all words to a list in order to find the unique words\n",
"allWords = []\n",
"for row in stemmed_words:\n",
" for word in row:\n",
" allWords.append(str(word))\n",
" \n",
"uniqueWords = np.unique(allWords)\n",
"\n",
"print('Number of unique words:', len(uniqueWords), '\\n')\n",
"print('Previewing sample of unique words:\\n', uniqueWords[1234:1244])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see a few of the challenges from slang or typos that I mentioned in the beginning. These will pose problems for what we're doing, but we'll just have to assume that the vast majority of words are spelled correctly.\n",
"\n",
"Before doing the TF-IDF transformation, we need to make sure that we have spaces in between each word in the comments:"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:42.993287Z",
"start_time": "2017-07-02T17:30:42.259265Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"' line ridiculous download app order get bowl min whilst person take place line wait wait hangry townsuper nice staff give extra chicken cost waswink speedy service even onsaturday order fajita sure let know mean pepper bean skimpy portion compare location especially compare portion employee give front pay customer try almost chipotle area one make best steak service really shitty everyone else say third row line least min getbowl andbunch chip best chip dip ever taste use app order advance get comical long line location staff oblivious tell cashier reguerilla getfree bag chip slow chipolte ever seriously come rush anywhere chance go late end withoutdinner get salad dress tangy place crazy busy matter go staff slow expect wait putlime wedge coke good linesridiculous worth wait chicken burrito bowl bestgetbowl healthy great burrito chipsits nice charge guacamolesteak soft taco withside guacamole mexican cuisineorder ahead online great staff line way behind fillingfax order good customer servicei desire earn back mayor ship quality food service takenserious turn since mgr ben sexe leave slow chipotle staff twin citiesthis place go hill lot staff fast mayor unhappyborrito bowlgreat place disfunctional officially bad chipotle go lyndale good service slow chipotle land prepare wait line always terrible management seem carekicks qdoba buttway toooooo slow tonight outta chip guaq areno brainerdamn soooo slow food good service suck'"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"stemmed_sentences = []\n",
"\n",
"# Spacing out the words in the reviews for each restaurant\n",
"for row in df['tokens']:\n",
" stemmed_string = ''\n",
" for word in row:\n",
" stemmed_string = stemmed_string + ' ' + word\n",
" stemmed_sentences.append(stemmed_string)\n",
" \n",
"df['tokens'] = stemmed_sentences\n",
"stemmed_sentences[np.random.randint(len(stemmed_sentences))]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5) Term Frequency-Inverse Document Frequency (TF-IDF)\n",
"\n",
"This determines how important a word is to a document (which is a review in this case) within a corpus (the collection documents). It is a number resulting from the following formula: \n",
"\n",
"$\\hspace{8cm}TFIDF(t, d) = TF(t, d) \\cdot IDF(t)$\n",
"\n",
"\n",
"$\\hspace{8cm}IDF(t) = 1 + \\log\\Big(\\frac{\\#\\ Documents}{\\#\\ Documents\\ Containing\\ t}\\Big)$\n",
"\n",
"\n",
"$\\hspace{8cm}t:\\ \\text{Term}$\n",
"\n",
"$\\hspace{8cm}d:\\ \\text{Document}$\n",
"\n",
"Scikit-learn has an [excellent function](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html) that is able to transform our processed text into a TF-IDF matrix very quickly. We'll convert it back to a data frame, and join it to our original data frame by the indexes."
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:44.482495Z",
"start_time": "2017-07-02T17:30:42.999792Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Wall time: 1.47 s\n"
]
}
],
"source": [
"%%time\n",
"\n",
"# Creating the sklearn object\n",
"tfidf = sktext.TfidfVectorizer(smooth_idf=False)\n",
"\n",
"# Transforming our 'tokens' column into a TF-IDF matrix and then a data frame\n",
"tfidf_df = pd.DataFrame(tfidf.fit_transform(df['tokens']).toarray(), \n",
" columns=tfidf.get_feature_names())"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:44.711154Z",
"start_time": "2017-07-02T17:30:44.489997Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(1335, 36571)\n"
]
},
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" aa aaa aaaaaaa aaaaaaamazing aaaaaamaaaaaaazing aaaaalll aaaaand \\\n",
"0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"\n",
" aaaammaazzing aaallllllll aaarrrggghhh ... zucca \\\n",
"0 0.0 0.0 0.0 ... 0.0 \n",
"1 0.0 0.0 0.0 ... 0.0 \n",
"2 0.0 0.0 0.0 ... 0.0 \n",
"3 0.0 0.0 0.0 ... 0.0 \n",
"4 0.0 0.0 0.0 ... 0.0 \n",
"\n",
" zucchini zuccini zuccotto zuchini zuke zuppa zur zushi \\\n",
"0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 \n",
"\n",
" zzzzzzzzzzzzoops \n",
"0 0.0 \n",
"1 0.0 \n",
"2 0.0 \n",
"3 0.0 \n",
"4 0.0 \n",
"\n",
"[5 rows x 36571 columns]"
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"print(tfidf_df.shape)\n",
"tfidf_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since we transformed *all* of the words, we have a [sparse matrix](https://en.wikipedia.org/wiki/Sparse_matrix). We don't care about things like typos or words specific to one particular restaurant, so we're going to remove columns that don't have a lot of contents."
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:45.359115Z",
"start_time": "2017-07-02T17:30:44.738673Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(1335, 976)\n"
]
},
{
"data": {
"text/html": [
"| \n", " | aa | \n", "aaa | \n", "aaaaaaa | \n", "aaaaaaamazing | \n", "aaaaaamaaaaaaazing | \n", "aaaaalll | \n", "aaaaand | \n", "aaaammaazzing | \n", "aaallllllll | \n", "aaarrrggghhh | \n", "... | \n", "zucca | \n", "zucchini | \n", "zuccini | \n", "zuccotto | \n", "zuchini | \n", "zuke | \n", "zuppa | \n", "zur | \n", "zushi | \n", "zzzzzzzzzzzzoops | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "
| 1 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "
| 2 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "
| 3 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "
| 4 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "
5 rows × 36571 columns
\n", "\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" absolute absolutely across actually add afternoon ahead ahi al \\\n",
"0 0.0 0.000000 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"1 0.0 0.000000 0.0 0.034778 0.0 0.0 0.0 0.0 0.0 \n",
"2 0.0 0.031268 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"3 0.0 0.000000 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"4 0.0 0.000000 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"\n",
" ale ... wow wrap wrong www year yes yet \\\n",
"0 0.0 ... 0.0000 0.000000 0.000000 0.074605 0.0 0.0 0.000000 \n",
"1 0.0 ... 0.0000 0.000000 0.024101 0.000000 0.0 0.0 0.000000 \n",
"2 0.0 ... 0.0376 0.000000 0.026175 0.000000 0.0 0.0 0.000000 \n",
"3 0.0 ... 0.0000 0.000000 0.000000 0.000000 0.0 0.0 0.000000 \n",
"4 0.0 ... 0.0000 0.060402 0.000000 0.000000 0.0 0.0 0.065236 \n",
"\n",
" york yum yummy \n",
"0 0.000000 0.000000 0.000000 \n",
"1 0.000000 0.023301 0.023143 \n",
"2 0.000000 0.000000 0.025135 \n",
"3 0.000000 0.000000 0.000000 \n",
"4 0.085197 0.000000 0.000000 \n",
"\n",
"[5 rows x 976 columns]"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Removing sparse columns\n",
"tfidf_df = tfidf_df[tfidf_df.columns[tfidf_df.sum() > 2.5]]\n",
"\n",
"# Removing any remaining digits\n",
"tfidf_df = tfidf_df.filter(regex=r'^((?!\\d).)*$')\n",
"\n",
"print(tfidf_df.shape)\n",
"tfidf_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This drastically reduced the dimensions of our data set, and we now have something usable to calculate similarity."
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:45.485703Z",
"start_time": "2017-07-02T17:30:45.366119Z"
},
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | absolute | \n", "absolutely | \n", "across | \n", "actually | \n", "add | \n", "afternoon | \n", "ahead | \n", "ahi | \n", "al | \n", "ale | \n", "... | \n", "wow | \n", "wrap | \n", "wrong | \n", "www | \n", "year | \n", "yes | \n", "yet | \n", "york | \n", "yum | \n", "yummy | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0000 | \n", "0.000000 | \n", "0.000000 | \n", "0.074605 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "
| 1 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.034778 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0000 | \n", "0.000000 | \n", "0.024101 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.023301 | \n", "0.023143 | \n", "
| 2 | \n", "0.0 | \n", "0.031268 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0376 | \n", "0.000000 | \n", "0.026175 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.025135 | \n", "
| 3 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0000 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "
| 4 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.0000 | \n", "0.060402 | \n", "0.000000 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.065236 | \n", "0.085197 | \n", "0.000000 | \n", "0.000000 | \n", "
5 rows × 976 columns
\n", "\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" id Name \\\n",
"0 4e17b348b0fb8567c665ddaf Souper Salad \n",
"1 4aceefb7f964a52013d220e3 Aster's Ethiopian Restaurant \n",
"2 4b591015f964a520c17a28e3 Taste Of Ethiopia \n",
"3 4ead97ba4690615f26a8adfe Wasota African Cuisine \n",
"4 4c7efeba2042b1f76cd1c1ad Cazamance \n",
"\n",
" category shortCategory checkinsCount City state \\\n",
"0 Salad Place Salad 1769 Austin TX \n",
"1 Ethiopian Restaurant Ethiopian 1463 Austin TX \n",
"2 Ethiopian Restaurant Ethiopian 1047 Pflugerville TX \n",
"3 African Restaurant African 195 Austin TX \n",
"4 African Restaurant African 500 Austin TX \n",
"\n",
" Location commentsCount usersCount ... wow wrap \\\n",
"0 Austin, TX 17 683 ... 0.0000 0.000000 \n",
"1 Austin, TX 34 1018 ... 0.0000 0.000000 \n",
"2 Austin, TX 31 672 ... 0.0376 0.000000 \n",
"3 Austin, TX 12 140 ... 0.0000 0.000000 \n",
"4 Austin, TX 11 435 ... 0.0000 0.060402 \n",
"\n",
" wrong www year yes yet york yum yummy \n",
"0 0.000000 0.074605 0.0 0.0 0.000000 0.000000 0.000000 0.000000 \n",
"1 0.024101 0.000000 0.0 0.0 0.000000 0.000000 0.023301 0.023143 \n",
"2 0.026175 0.000000 0.0 0.0 0.000000 0.000000 0.000000 0.025135 \n",
"3 0.000000 0.000000 0.0 0.0 0.000000 0.000000 0.000000 0.000000 \n",
"4 0.000000 0.000000 0.0 0.0 0.065236 0.085197 0.000000 0.000000 \n",
"\n",
"[5 rows x 991 columns]"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Storing the original data frame before the merge in case any changes are needed\n",
"df_orig = df.copy()\n",
"\n",
"# Renaming columns that conflict with column names in tfidfCore\n",
"df.rename(columns={'name': 'Name', \n",
" 'city': 'City', \n",
" 'location': 'Location'}, inplace=True)\n",
"\n",
"# Merging the data frames by index\n",
"df = pd.merge(df, tfidf_df, how='inner', left_index=True, right_index=True)\n",
"\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Lastly, we're going to add additional features for the category. This just puts a heavier weight on those with the same type, so for example a Mexican restaurant will be more likely to have Mexican restaurants show up as most similar instead of Brazilian restaurants."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:45.622801Z",
"start_time": "2017-07-02T17:30:45.490707Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | id | \n", "Name | \n", "category | \n", "shortCategory | \n", "checkinsCount | \n", "City | \n", "state | \n", "Location | \n", "commentsCount | \n", "usersCount | \n", "... | \n", "wow | \n", "wrap | \n", "wrong | \n", "www | \n", "year | \n", "yes | \n", "yet | \n", "york | \n", "yum | \n", "yummy | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "4e17b348b0fb8567c665ddaf | \n", "Souper Salad | \n", "Salad Place | \n", "Salad | \n", "1769 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "17 | \n", "683 | \n", "... | \n", "0.0000 | \n", "0.000000 | \n", "0.000000 | \n", "0.074605 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "
| 1 | \n", "4aceefb7f964a52013d220e3 | \n", "Aster's Ethiopian Restaurant | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1463 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "34 | \n", "1018 | \n", "... | \n", "0.0000 | \n", "0.000000 | \n", "0.024101 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.023301 | \n", "0.023143 | \n", "
| 2 | \n", "4b591015f964a520c17a28e3 | \n", "Taste Of Ethiopia | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1047 | \n", "Pflugerville | \n", "TX | \n", "Austin, TX | \n", "31 | \n", "672 | \n", "... | \n", "0.0376 | \n", "0.000000 | \n", "0.026175 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.025135 | \n", "
| 3 | \n", "4ead97ba4690615f26a8adfe | \n", "Wasota African Cuisine | \n", "African Restaurant | \n", "African | \n", "195 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "12 | \n", "140 | \n", "... | \n", "0.0000 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "
| 4 | \n", "4c7efeba2042b1f76cd1c1ad | \n", "Cazamance | \n", "African Restaurant | \n", "African | \n", "500 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "11 | \n", "435 | \n", "... | \n", "0.0000 | \n", "0.060402 | \n", "0.000000 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.065236 | \n", "0.085197 | \n", "0.000000 | \n", "0.000000 | \n", "
5 rows × 991 columns
\n", "\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" id Name \\\n",
"0 4e17b348b0fb8567c665ddaf Souper Salad \n",
"1 4aceefb7f964a52013d220e3 Aster's Ethiopian Restaurant \n",
"2 4b591015f964a520c17a28e3 Taste Of Ethiopia \n",
"3 4ead97ba4690615f26a8adfe Wasota African Cuisine \n",
"4 4c7efeba2042b1f76cd1c1ad Cazamance \n",
"\n",
" category shortCategory checkinsCount City state \\\n",
"0 Salad Place Salad 1769 Austin TX \n",
"1 Ethiopian Restaurant Ethiopian 1463 Austin TX \n",
"2 Ethiopian Restaurant Ethiopian 1047 Pflugerville TX \n",
"3 African Restaurant African 195 Austin TX \n",
"4 African Restaurant African 500 Austin TX \n",
"\n",
" Location commentsCount usersCount ... Tapas Tea Room Tex-Mex \\\n",
"0 Austin, TX 17 683 ... 0 0 0 \n",
"1 Austin, TX 34 1018 ... 0 0 0 \n",
"2 Austin, TX 31 672 ... 0 0 0 \n",
"3 Austin, TX 12 140 ... 0 0 0 \n",
"4 Austin, TX 11 435 ... 0 0 0 \n",
"\n",
" Thai Theme Restaurant Turkish Vegetarian / Vegan Vietnamese Wine Bar \\\n",
"0 0 0 0 0 0 0 \n",
"1 0 0 0 0 0 0 \n",
"2 0 0 0 0 0 0 \n",
"3 0 0 0 0 0 0 \n",
"4 0 0 0 0 0 0 \n",
"\n",
" Yogurt \n",
"0 0 \n",
"1 0 \n",
"2 0 \n",
"3 0 \n",
"4 0 \n",
"\n",
"[5 rows x 1090 columns]"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Creates dummy variables out of the restaurant category\n",
"df = pd.concat([df, pd.get_dummies(df['shortCategory'])], axis=1)\n",
"\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Because we introduced an additional type of feature, we'll have to check it's weight in comparison to the TF-IDF features:"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:45.709363Z",
"start_time": "2017-07-02T17:30:45.629806Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Max: 0.889350911658 \n",
" Mean: 0.005518928565159508 \n",
" Standard Deviation: 0.010229821082730768\n"
]
}
],
"source": [
"# Summary stats of TF-IDF\n",
"print('Max:', np.max(tfidf_df.max()), '\\n',\n",
" 'Mean:', np.mean(tfidf_df.mean()), '\\n',\n",
" 'Standard Deviation:', np.std(tfidf_df.std()))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The dummy variables for the restaurant type are quite a bit higher than the average word, but I'm comfortable with this since I think it has a benefit."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# \"Recommender System\"\n",
"\n",
"As a reminder, we are not using a conventional recommender system. Instead, we are using recommender system theory by calculating the cosine distance between comments in order to find restaurants with the most similar comments. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Loading in personal ratings\n",
"\n",
"In order to recommend restaurants with this approach, we have to identify the restaurants to which we want to find the most similarities. I took the data frame and assigned my own ratings to some of my favorites."
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:45.769906Z",
"start_time": "2017-07-02T17:30:45.716869Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | id | \n", "Name | \n", "category | \n", "shortCategory | \n", "checkinsCount | \n", "City | \n", "state | \n", "Location | \n", "commentsCount | \n", "usersCount | \n", "... | \n", "Tapas | \n", "Tea Room | \n", "Tex-Mex | \n", "Thai | \n", "Theme Restaurant | \n", "Turkish | \n", "Vegetarian / Vegan | \n", "Vietnamese | \n", "Wine Bar | \n", "Yogurt | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "4e17b348b0fb8567c665ddaf | \n", "Souper Salad | \n", "Salad Place | \n", "Salad | \n", "1769 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "17 | \n", "683 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "
| 1 | \n", "4aceefb7f964a52013d220e3 | \n", "Aster's Ethiopian Restaurant | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1463 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "34 | \n", "1018 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "
| 2 | \n", "4b591015f964a520c17a28e3 | \n", "Taste Of Ethiopia | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1047 | \n", "Pflugerville | \n", "TX | \n", "Austin, TX | \n", "31 | \n", "672 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "
| 3 | \n", "4ead97ba4690615f26a8adfe | \n", "Wasota African Cuisine | \n", "African Restaurant | \n", "African | \n", "195 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "12 | \n", "140 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "
| 4 | \n", "4c7efeba2042b1f76cd1c1ad | \n", "Cazamance | \n", "African Restaurant | \n", "African | \n", "500 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "11 | \n", "435 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "
5 rows × 1090 columns
\n", "\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" id selfRating\n",
"0 43c968a2f964a5209c2d1fe3 10.0\n",
"1 574481f8498e2cd16a0911a6 NaN\n",
"2 4cb5e045e262b60c46cb6ae0 9.0\n",
"3 49be75ccf964a520ad541fe3 9.0\n",
"4 4d8d295fc1b1721e798b1246 NaN"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Loading in self-ratings for restaurants in the data set\n",
"selfRatings = pd.read_csv('selfRatings.csv', usecols=[0, 4])\n",
"selfRatings.head()"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:45.873479Z",
"start_time": "2017-07-02T17:30:45.775911Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | id | \n", "selfRating | \n", "
|---|---|---|
| 0 | \n", "43c968a2f964a5209c2d1fe3 | \n", "10.0 | \n", "
| 1 | \n", "574481f8498e2cd16a0911a6 | \n", "NaN | \n", "
| 2 | \n", "4cb5e045e262b60c46cb6ae0 | \n", "9.0 | \n", "
| 3 | \n", "49be75ccf964a520ad541fe3 | \n", "9.0 | \n", "
| 4 | \n", "4d8d295fc1b1721e798b1246 | \n", "NaN | \n", "
\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" id Name \\\n",
"0 4e17b348b0fb8567c665ddaf Souper Salad \n",
"1 4aceefb7f964a52013d220e3 Aster's Ethiopian Restaurant \n",
"2 4b591015f964a520c17a28e3 Taste Of Ethiopia \n",
"3 4ead97ba4690615f26a8adfe Wasota African Cuisine \n",
"4 4c7efeba2042b1f76cd1c1ad Cazamance \n",
"\n",
" category shortCategory checkinsCount City state \\\n",
"0 Salad Place Salad 1769 Austin TX \n",
"1 Ethiopian Restaurant Ethiopian 1463 Austin TX \n",
"2 Ethiopian Restaurant Ethiopian 1047 Pflugerville TX \n",
"3 African Restaurant African 195 Austin TX \n",
"4 African Restaurant African 500 Austin TX \n",
"\n",
" Location commentsCount usersCount ... Tea Room Tex-Mex Thai \\\n",
"0 Austin, TX 17 683 ... 0 0 0 \n",
"1 Austin, TX 34 1018 ... 0 0 0 \n",
"2 Austin, TX 31 672 ... 0 0 0 \n",
"3 Austin, TX 12 140 ... 0 0 0 \n",
"4 Austin, TX 11 435 ... 0 0 0 \n",
"\n",
" Theme Restaurant Turkish Vegetarian / Vegan Vietnamese Wine Bar Yogurt \\\n",
"0 0 0 0 0 0 0 \n",
"1 0 0 0 0 0 0 \n",
"2 0 0 0 0 0 0 \n",
"3 0 0 0 0 0 0 \n",
"4 0 0 0 0 0 0 \n",
"\n",
" selfRating \n",
"0 NaN \n",
"1 NaN \n",
"2 NaN \n",
"3 NaN \n",
"4 NaN \n",
"\n",
"[5 rows x 1091 columns]"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Merging into df to add the column 'selfRating'\n",
"df = pd.merge(df, selfRatings)\n",
"\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Additional features & min-max scaling \n",
"\n",
"We're going to include a few additional features from the original data set to capture information that the comments may not have. Specifically:\n",
"\n",
"- **Popularity:** checkinsCount, commentsCount, usersCount, numRatings\n",
"- **Price:** priceTier\n",
"\n",
"We're also going to scale these down so they don't carry a huge advantage over everything else. I'm going to scale the popularity attributes to be between 0 and 0.5, and the price attribute to be between 0 and 1. I'll do this by first min-max scaling everything (to put it between 0 and 1), and then dividing the popularity features in half."
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.042100Z",
"start_time": "2017-07-02T17:30:45.881485Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | id | \n", "Name | \n", "category | \n", "shortCategory | \n", "checkinsCount | \n", "City | \n", "state | \n", "Location | \n", "commentsCount | \n", "usersCount | \n", "... | \n", "Tea Room | \n", "Tex-Mex | \n", "Thai | \n", "Theme Restaurant | \n", "Turkish | \n", "Vegetarian / Vegan | \n", "Vietnamese | \n", "Wine Bar | \n", "Yogurt | \n", "selfRating | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "4e17b348b0fb8567c665ddaf | \n", "Souper Salad | \n", "Salad Place | \n", "Salad | \n", "1769 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "17 | \n", "683 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "NaN | \n", "
| 1 | \n", "4aceefb7f964a52013d220e3 | \n", "Aster's Ethiopian Restaurant | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1463 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "34 | \n", "1018 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "NaN | \n", "
| 2 | \n", "4b591015f964a520c17a28e3 | \n", "Taste Of Ethiopia | \n", "Ethiopian Restaurant | \n", "Ethiopian | \n", "1047 | \n", "Pflugerville | \n", "TX | \n", "Austin, TX | \n", "31 | \n", "672 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "NaN | \n", "
| 3 | \n", "4ead97ba4690615f26a8adfe | \n", "Wasota African Cuisine | \n", "African Restaurant | \n", "African | \n", "195 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "12 | \n", "140 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "NaN | \n", "
| 4 | \n", "4c7efeba2042b1f76cd1c1ad | \n", "Cazamance | \n", "African Restaurant | \n", "African | \n", "500 | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "11 | \n", "435 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "NaN | \n", "
5 rows × 1091 columns
\n", "\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" absolute absolutely across actually add afternoon ahead ahi al \\\n",
"0 0.0 0.000000 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"1 0.0 0.000000 0.0 0.034778 0.0 0.0 0.0 0.0 0.0 \n",
"2 0.0 0.031268 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"3 0.0 0.000000 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"4 0.0 0.000000 0.0 0.000000 0.0 0.0 0.0 0.0 0.0 \n",
"\n",
" ale ... Turkish Vegetarian / Vegan Vietnamese Wine Bar Yogurt \\\n",
"0 0.0 ... 0 0 0 0 0 \n",
"1 0.0 ... 0 0 0 0 0 \n",
"2 0.0 ... 0 0 0 0 0 \n",
"3 0.0 ... 0 0 0 0 0 \n",
"4 0.0 ... 0 0 0 0 0 \n",
"\n",
" checkinsCount commentsCount usersCount priceTier numRatings \n",
"0 0.030682 0.017544 0.020429 0.000000 0.017494 \n",
"1 0.024844 0.060150 0.031648 0.333333 0.046840 \n",
"2 0.016906 0.052632 0.020060 0.333333 0.044018 \n",
"3 0.000649 0.005013 0.002244 0.333333 0.000000 \n",
"4 0.006468 0.002506 0.012123 0.000000 0.002822 \n",
"\n",
"[5 rows x 1080 columns]"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Removing everything that won't be used in the similarity calculation\n",
"df_item = df.drop(['id', 'category', 'Name', 'shortCategory', 'City', 'tokens',\n",
" 'comments', 'state', 'Location', 'selfRating', 'rating'],\n",
" axis=1)\n",
"\n",
"# Copying into a separate data frame to be normalized\n",
"df_item_norm = df_item.copy()\n",
"\n",
"columns_to_scale = ['checkinsCount', 'commentsCount',\n",
" 'usersCount', 'priceTier', 'numRatings']\n",
"\n",
"# Split\n",
"df_item_split = df_item[columns_to_scale]\n",
"df_item_norm.drop(columns_to_scale, axis=1, inplace=True)\n",
"\n",
"# Apply\n",
"df_item_split = pd.DataFrame(MinMaxScaler().fit_transform(df_item_split),\n",
" columns=df_item_split.columns)\n",
"df_item_split_half = df_item_split.drop('priceTier', axis=1)\n",
"df_item_split_half = df_item_split_half / 2\n",
"df_item_split_half['priceTier'] = df_item_split['priceTier']\n",
"\n",
"# Combine\n",
"df_item_norm = df_item_norm.merge(df_item_split,\n",
" left_index=True, right_index=True)\n",
"\n",
"df_item_norm.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Calculating cosine similarities\n",
"\n",
"Here's the moment that we've spent all of this time getting to: the similarity. \n",
"\n",
"This section calculates the cosine similarity and puts it into a matrix with the pairwise similarity:\n",
"\n",
"| | 0 | 1 | ... | n |\n",
"|------|------|------|------|------|\n",
"| 0 | 1.00 | 0.03 | ... | 0.15 |\n",
"| 1 | 0.31 | 1.00 | ... | 0.89 |\n",
"| ... | ... | ... | ... | ... |\n",
"| n | 0.05 | 0.13 | ... | 1.00 |\n",
"\n",
"As a reminder, we're using cosine similarity because it's generally accepted as producing better results in item-to-item filtering. For all you math folk, here's the formula again:\n",
"\n",
"$\\hspace{8cm}sim(A, B) = \\cos(\\theta) = \\frac{A \\cdot B}{\\|A\\|\\|B\\|}$"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.466401Z",
"start_time": "2017-07-02T17:30:46.048604Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | absolute | \n", "absolutely | \n", "across | \n", "actually | \n", "add | \n", "afternoon | \n", "ahead | \n", "ahi | \n", "al | \n", "ale | \n", "... | \n", "Turkish | \n", "Vegetarian / Vegan | \n", "Vietnamese | \n", "Wine Bar | \n", "Yogurt | \n", "checkinsCount | \n", "commentsCount | \n", "usersCount | \n", "priceTier | \n", "numRatings | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0.030682 | \n", "0.017544 | \n", "0.020429 | \n", "0.000000 | \n", "0.017494 | \n", "
| 1 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.034778 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0.024844 | \n", "0.060150 | \n", "0.031648 | \n", "0.333333 | \n", "0.046840 | \n", "
| 2 | \n", "0.0 | \n", "0.031268 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0.016906 | \n", "0.052632 | \n", "0.020060 | \n", "0.333333 | \n", "0.044018 | \n", "
| 3 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0.000649 | \n", "0.005013 | \n", "0.002244 | \n", "0.333333 | \n", "0.000000 | \n", "
| 4 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.000000 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0.006468 | \n", "0.002506 | \n", "0.012123 | \n", "0.000000 | \n", "0.002822 | \n", "
5 rows × 1080 columns
\n", "\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" 0 1 2 3 4 5 6 \\\n",
"0 1.000000 0.056252 0.067231 0.026230 0.024996 0.059179 0.051503 \n",
"1 0.056252 1.000000 0.881640 0.112950 0.036702 0.164534 0.195639 \n",
"2 0.067231 0.881640 1.000000 0.118254 0.045115 0.139096 0.179490 \n",
"3 0.026230 0.112950 0.118254 1.000000 0.767280 0.078028 0.151436 \n",
"4 0.024996 0.036702 0.045115 0.767280 1.000000 0.032617 0.030823 \n",
"\n",
" 7 8 9 ... 1113 1114 1115 \\\n",
"0 0.067922 0.079887 0.073811 ... 0.019677 0.051769 0.066944 \n",
"1 0.144018 0.158320 0.150945 ... 0.083064 0.125948 0.144452 \n",
"2 0.132280 0.153244 0.145920 ... 0.094744 0.120079 0.145357 \n",
"3 0.090713 0.086704 0.102522 ... 0.094147 0.071328 0.103130 \n",
"4 0.030906 0.041741 0.031845 ... 0.006193 0.026284 0.038349 \n",
"\n",
" 1116 1117 1118 1119 1120 1121 1122 \n",
"0 0.047500 0.055441 0.059609 0.032087 0.045675 0.064043 0.047003 \n",
"1 0.089293 0.128221 0.078654 0.113436 0.129943 0.146670 0.128873 \n",
"2 0.083324 0.126703 0.082119 0.102128 0.122537 0.151989 0.132252 \n",
"3 0.080002 0.100153 0.049730 0.093961 0.111084 0.109474 0.096562 \n",
"4 0.042255 0.046679 0.038977 0.011920 0.027661 0.044796 0.021876 \n",
"\n",
"[5 rows x 1123 columns]"
]
},
"execution_count": 39,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Calculating cosine similarity\n",
"df_item_norm_sparse = sparse.csr_matrix(df_item_norm)\n",
"similarities = cosine_similarity(df_item_norm_sparse)\n",
"\n",
"# Putting into a data frame\n",
"dfCos = pd.DataFrame(similarities)\n",
"\n",
"dfCos.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These are the some of the restaurants I rated very highly, and I'm pulling these up so we can use the index number in order to compare it to the others in our data set:"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.536451Z",
"start_time": "2017-07-02T17:30:46.474907Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | 0 | \n", "1 | \n", "2 | \n", "3 | \n", "4 | \n", "5 | \n", "6 | \n", "7 | \n", "8 | \n", "9 | \n", "... | \n", "1113 | \n", "1114 | \n", "1115 | \n", "1116 | \n", "1117 | \n", "1118 | \n", "1119 | \n", "1120 | \n", "1121 | \n", "1122 | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "1.000000 | \n", "0.056252 | \n", "0.067231 | \n", "0.026230 | \n", "0.024996 | \n", "0.059179 | \n", "0.051503 | \n", "0.067922 | \n", "0.079887 | \n", "0.073811 | \n", "... | \n", "0.019677 | \n", "0.051769 | \n", "0.066944 | \n", "0.047500 | \n", "0.055441 | \n", "0.059609 | \n", "0.032087 | \n", "0.045675 | \n", "0.064043 | \n", "0.047003 | \n", "
| 1 | \n", "0.056252 | \n", "1.000000 | \n", "0.881640 | \n", "0.112950 | \n", "0.036702 | \n", "0.164534 | \n", "0.195639 | \n", "0.144018 | \n", "0.158320 | \n", "0.150945 | \n", "... | \n", "0.083064 | \n", "0.125948 | \n", "0.144452 | \n", "0.089293 | \n", "0.128221 | \n", "0.078654 | \n", "0.113436 | \n", "0.129943 | \n", "0.146670 | \n", "0.128873 | \n", "
| 2 | \n", "0.067231 | \n", "0.881640 | \n", "1.000000 | \n", "0.118254 | \n", "0.045115 | \n", "0.139096 | \n", "0.179490 | \n", "0.132280 | \n", "0.153244 | \n", "0.145920 | \n", "... | \n", "0.094744 | \n", "0.120079 | \n", "0.145357 | \n", "0.083324 | \n", "0.126703 | \n", "0.082119 | \n", "0.102128 | \n", "0.122537 | \n", "0.151989 | \n", "0.132252 | \n", "
| 3 | \n", "0.026230 | \n", "0.112950 | \n", "0.118254 | \n", "1.000000 | \n", "0.767280 | \n", "0.078028 | \n", "0.151436 | \n", "0.090713 | \n", "0.086704 | \n", "0.102522 | \n", "... | \n", "0.094147 | \n", "0.071328 | \n", "0.103130 | \n", "0.080002 | \n", "0.100153 | \n", "0.049730 | \n", "0.093961 | \n", "0.111084 | \n", "0.109474 | \n", "0.096562 | \n", "
| 4 | \n", "0.024996 | \n", "0.036702 | \n", "0.045115 | \n", "0.767280 | \n", "1.000000 | \n", "0.032617 | \n", "0.030823 | \n", "0.030906 | \n", "0.041741 | \n", "0.031845 | \n", "... | \n", "0.006193 | \n", "0.026284 | \n", "0.038349 | \n", "0.042255 | \n", "0.046679 | \n", "0.038977 | \n", "0.011920 | \n", "0.027661 | \n", "0.044796 | \n", "0.021876 | \n", "
5 rows × 1123 columns
\n", "\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" Name category \\\n",
"14 Jack Allen's Kitchen American Restaurant \n",
"88 Black's Barbecue BBQ Joint \n",
"447 Cabo Bob's Burrito Place \n",
"420 Tacodeli Taco Place \n",
"228 Round Rock Donuts Donut Shop \n",
"27 Jack Allens on Capital of TX American Restaurant \n",
"124 Blue Dahlia Bistro Café \n",
"967 Chimborazo Latin American Restaurant \n",
"83 Black's BBQ, The Original BBQ Joint \n",
"66 The Salt Lick BBQ Joint \n",
"109 Torchy's Tacos Taco Place \n",
"338 Clay Pit Contemporary Indian Cuisine Indian Restaurant \n",
"637 Brasa Premium Rotisserie BBQ Joint \n",
"576 Afro Deli African Restaurant \n",
"441 Trudy's Texas Star Mexican Restaurant \n",
"426 Trudy's North Star Mexican Restaurant \n",
"421 Chuy's Mexican Restaurant \n",
"360 Mandola's Italian Market Italian Restaurant \n",
"72 La Barbecue Cuisine Texicana BBQ Joint \n",
"74 Franklin Barbecue BBQ Joint \n",
"154 Mighty Fine Burgers Burger Joint \n",
"151 Hopdoddy Burger Bar Burger Joint \n",
"171 P. Terry's Burger Stand Burger Joint \n",
"116 Juan in a Million Mexican Restaurant \n",
"216 Mozart's Coffee Coffee Shop \n",
"50 Uchi Japanese Restaurant \n",
"558 The Coffee Bean & Tea Leaf Coffee Shop \n",
"120 Texas Honey Ham Company Sandwich Place \n",
"832 Kramarczuk's East European Deli Eastern European Restaurant \n",
"\n",
" Location selfRating \n",
"14 Austin, TX 10.0 \n",
"88 Austin, TX 10.0 \n",
"447 Austin, TX 10.0 \n",
"420 Austin, TX 10.0 \n",
"228 Austin, TX 10.0 \n",
"27 Austin, TX 10.0 \n",
"124 Austin, TX 10.0 \n",
"967 Minneapolis, MN 10.0 \n",
"83 Austin, TX 10.0 \n",
"66 Austin, TX 10.0 \n",
"109 Austin, TX 9.0 \n",
"338 Austin, TX 9.0 \n",
"637 Minneapolis, MN 9.0 \n",
"576 Minneapolis, MN 9.0 \n",
"441 Austin, TX 9.0 \n",
"426 Austin, TX 9.0 \n",
"421 Austin, TX 9.0 \n",
"360 Austin, TX 9.0 \n",
"72 Austin, TX 9.0 \n",
"74 Austin, TX 9.0 \n",
"154 Austin, TX 9.0 \n",
"151 Austin, TX 9.0 \n",
"171 Austin, TX 9.0 \n",
"116 Austin, TX 8.0 \n",
"216 Austin, TX 8.0 \n",
"50 Austin, TX 8.0 \n",
"558 Austin, TX 8.0 \n",
"120 Austin, TX 8.0 \n",
"832 Minneapolis, MN 8.0 "
]
},
"execution_count": 40,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Filtering to those from my list with the highest ratings\n",
"topRated = df[df['selfRating'] >= 8].drop_duplicates('Name')\n",
"\n",
"# Preparing for display\n",
"topRated[['Name', 'category', 'Location', 'selfRating']].sort_values(\n",
" 'selfRating', ascending=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to speed things up, we'll make a function that formats the cosine similarity data frame and retrieves the top $n$ most similar restaurants for the given restaurant:"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.596994Z",
"start_time": "2017-07-02T17:30:46.542454Z"
},
"collapsed": true
},
"outputs": [],
"source": [
"def retrieve_recommendations(restaurant_index, num_recommendations=5):\n",
" \"\"\"\n",
" Retrieves the most similar restaurants for the index of a given restaurant \n",
" \n",
" Outputs a data frame showing similarity, name, location, category, and rating\n",
" \"\"\"\n",
" # Formatting the cosine similarity data frame for merging\n",
" similarity = pd.melt(dfCos[dfCos.index == restaurant_index])\n",
" similarity.columns = (['restIndex', 'cosineSimilarity'])\n",
"\n",
" # Merging the cosine similarity data frame to the original data frame\n",
" similarity = similarity.merge(\n",
" df[['Name', 'City', 'state', 'Location',\n",
" 'category', 'rating', 'selfRating']],\n",
" left_on=similarity['restIndex'],\n",
" right_index=True)\n",
" similarity.drop(['restIndex'], axis=1, inplace=True)\n",
"\n",
" # Ensuring that retrieved recommendations are for Minneapolis\n",
" similarity = similarity[(similarity['Location'] == 'Minneapolis, MN') | (\n",
" similarity.index == restaurant_index)]\n",
"\n",
" # Sorting by similarity\n",
" similarity = similarity.sort_values(\n",
" 'cosineSimilarity', ascending=False)[:num_recommendations + 1]\n",
" \n",
" return similarity"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alright, let's test it out!\n",
"\n",
"### Barbecue\n",
"\n",
"Let's start with the [Salt Lick](http://saltlickbbq.com/). This is a popular central Texas barbecue place featured on [various food shows](https://www.youtube.com/watch?v=vLnsXechOWc). They are well-known for their open smoke pit:\n",
"\n",
"| \n", " | Name | \n", "category | \n", "Location | \n", "selfRating | \n", "
|---|---|---|---|---|
| 14 | \n", "Jack Allen's Kitchen | \n", "American Restaurant | \n", "Austin, TX | \n", "10.0 | \n", "
| 88 | \n", "Black's Barbecue | \n", "BBQ Joint | \n", "Austin, TX | \n", "10.0 | \n", "
| 447 | \n", "Cabo Bob's | \n", "Burrito Place | \n", "Austin, TX | \n", "10.0 | \n", "
| 420 | \n", "Tacodeli | \n", "Taco Place | \n", "Austin, TX | \n", "10.0 | \n", "
| 228 | \n", "Round Rock Donuts | \n", "Donut Shop | \n", "Austin, TX | \n", "10.0 | \n", "
| 27 | \n", "Jack Allens on Capital of TX | \n", "American Restaurant | \n", "Austin, TX | \n", "10.0 | \n", "
| 124 | \n", "Blue Dahlia Bistro | \n", "Café | \n", "Austin, TX | \n", "10.0 | \n", "
| 967 | \n", "Chimborazo | \n", "Latin American Restaurant | \n", "Minneapolis, MN | \n", "10.0 | \n", "
| 83 | \n", "Black's BBQ, The Original | \n", "BBQ Joint | \n", "Austin, TX | \n", "10.0 | \n", "
| 66 | \n", "The Salt Lick | \n", "BBQ Joint | \n", "Austin, TX | \n", "10.0 | \n", "
| 109 | \n", "Torchy's Tacos | \n", "Taco Place | \n", "Austin, TX | \n", "9.0 | \n", "
| 338 | \n", "Clay Pit Contemporary Indian Cuisine | \n", "Indian Restaurant | \n", "Austin, TX | \n", "9.0 | \n", "
| 637 | \n", "Brasa Premium Rotisserie | \n", "BBQ Joint | \n", "Minneapolis, MN | \n", "9.0 | \n", "
| 576 | \n", "Afro Deli | \n", "African Restaurant | \n", "Minneapolis, MN | \n", "9.0 | \n", "
| 441 | \n", "Trudy's Texas Star | \n", "Mexican Restaurant | \n", "Austin, TX | \n", "9.0 | \n", "
| 426 | \n", "Trudy's North Star | \n", "Mexican Restaurant | \n", "Austin, TX | \n", "9.0 | \n", "
| 421 | \n", "Chuy's | \n", "Mexican Restaurant | \n", "Austin, TX | \n", "9.0 | \n", "
| 360 | \n", "Mandola's Italian Market | \n", "Italian Restaurant | \n", "Austin, TX | \n", "9.0 | \n", "
| 72 | \n", "La Barbecue Cuisine Texicana | \n", "BBQ Joint | \n", "Austin, TX | \n", "9.0 | \n", "
| 74 | \n", "Franklin Barbecue | \n", "BBQ Joint | \n", "Austin, TX | \n", "9.0 | \n", "
| 154 | \n", "Mighty Fine Burgers | \n", "Burger Joint | \n", "Austin, TX | \n", "9.0 | \n", "
| 151 | \n", "Hopdoddy Burger Bar | \n", "Burger Joint | \n", "Austin, TX | \n", "9.0 | \n", "
| 171 | \n", "P. Terry's Burger Stand | \n", "Burger Joint | \n", "Austin, TX | \n", "9.0 | \n", "
| 116 | \n", "Juan in a Million | \n", "Mexican Restaurant | \n", "Austin, TX | \n", "8.0 | \n", "
| 216 | \n", "Mozart's Coffee | \n", "Coffee Shop | \n", "Austin, TX | \n", "8.0 | \n", "
| 50 | \n", "Uchi | \n", "Japanese Restaurant | \n", "Austin, TX | \n", "8.0 | \n", "
| 558 | \n", "The Coffee Bean & Tea Leaf | \n", "Coffee Shop | \n", "Austin, TX | \n", "8.0 | \n", "
| 120 | \n", "Texas Honey Ham Company | \n", "Sandwich Place | \n", "Austin, TX | \n", "8.0 | \n", "
| 832 | \n", "Kramarczuk's East European Deli | \n", "Eastern European Restaurant | \n", "Minneapolis, MN | \n", "8.0 | \n", "
![](\"http://cdn.loc.gov/service/pnp/highsm/28500/28571v.jpg\")
\n",
"\n",
"In case you're not familiar with [central Texas barbecue](https://en.wikipedia.org/wiki/Barbecue_in_Texas), it primarily features smoked meats (especially brisket) with white bread, onions, pickles, potato salad, beans and cornbread on the side. Sweet tea is usually the drink of choice if you're not drinking a Shiner or a Lonestar."
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.682054Z",
"start_time": "2017-07-02T17:30:46.601496Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" cosineSimilarity Name City \\\n",
"66 1.000000 The Salt Lick Driftwood \n",
"637 0.696079 Brasa Premium Rotisserie Minneapolis \n",
"646 0.604501 Famous Dave's Minneapolis \n",
"1114 0.590742 Psycho Suzi's Motor Lounge & Tiki Garden Minneapolis \n",
"654 0.572947 Rack Shack BBQ Burnsville \n",
"838 0.567213 Brit's Pub & Eating Establishment Minneapolis \n",
"\n",
" state Location category rating selfRating \n",
"66 TX Austin, TX BBQ Joint 9.5 10.0 \n",
"637 MN Minneapolis, MN BBQ Joint 9.3 9.0 \n",
"646 MN Minneapolis, MN BBQ Joint 7.4 NaN \n",
"1114 MN Minneapolis, MN Theme Restaurant 8.5 2.0 \n",
"654 MN Minneapolis, MN BBQ Joint 8.0 NaN \n",
"838 MN Minneapolis, MN English Restaurant 8.8 NaN "
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Salt Lick\n",
"retrieve_recommendations(66)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Surprisingly, our top recommendation is one of my favorite restaurants I've found in Minneapolis - Brasa! They're actually a [Creole](https://en.wikipedia.org/wiki/Louisiana_Creole_cuisine) restaurant, but they have a lot of smoked meats, beans, and corn bread, and they're probably the only restaurant I've found so far that lists sweet tea on the menu:\n",
"\n",
"| \n", " | cosineSimilarity | \n", "Name | \n", "City | \n", "state | \n", "Location | \n", "category | \n", "rating | \n", "selfRating | \n", "
|---|---|---|---|---|---|---|---|---|
| 66 | \n", "1.000000 | \n", "The Salt Lick | \n", "Driftwood | \n", "TX | \n", "Austin, TX | \n", "BBQ Joint | \n", "9.5 | \n", "10.0 | \n", "
| 637 | \n", "0.696079 | \n", "Brasa Premium Rotisserie | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "BBQ Joint | \n", "9.3 | \n", "9.0 | \n", "
| 646 | \n", "0.604501 | \n", "Famous Dave's | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "BBQ Joint | \n", "7.4 | \n", "NaN | \n", "
| 1114 | \n", "0.590742 | \n", "Psycho Suzi's Motor Lounge & Tiki Garden | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Theme Restaurant | \n", "8.5 | \n", "2.0 | \n", "
| 654 | \n", "0.572947 | \n", "Rack Shack BBQ | \n", "Burnsville | \n", "MN | \n", "Minneapolis, MN | \n", "BBQ Joint | \n", "8.0 | \n", "NaN | \n", "
| 838 | \n", "0.567213 | \n", "Brit's Pub & Eating Establishment | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "English Restaurant | \n", "8.8 | \n", "NaN | \n", "
![](\"https://getbento.imgix.net/accounts/0430ef0cdee9c5699c71ec5ebdb2541c/media/images/press_thumbnail54270.jpeg?w=600&fit=max&auto=compress,format&h=600\")
\n",
"\n",
"Funny enough, Brasa was also in [Man vs Food](https://www.youtube.com/watch?v=gZmGAi5DKE4) with Andrew Zimmerman as a guest.\n",
"\n",
"Famous Dave's is a Midwestern barbecue chain that focuses more on ribs, which isn't generally considered a Texan specialty. Psycho Suzi's (a theme restaurant that serves pizza and cocktails) and Brit's Pub (an English pub with a lawn bowling field) don't seem very similar, but their cosine similarity scores reflect that.\n",
"\n",
"### Donuts\n",
"\n",
"Moving on, let's find some donuts. Before maple-bacon-cereal-whatever donuts become the craze (thanks for nothing, Portland), my home town was also famous for [Round Rock Donuts](http://roundrockdonuts.com/), a simple and delicious no-nonsense donut shop. And yes, Man vs. Food also did a segment here.\n",
"\n",
"![](\"http://www.fronteraridge.com/wp-content/uploads/2016/10/8092563177_48b5681eb7.jpg\")
"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.746601Z",
"start_time": "2017-07-02T17:30:46.689059Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" cosineSimilarity Name City state \\\n",
"222 1.000000 Summer Moon Wood-Fired Coffee Austin TX \n",
"749 0.865335 Spyhouse Coffee Minneapolis MN \n",
"742 0.857070 Five Watt Coffee Minneapolis MN \n",
"745 0.848061 Spyhouse Coffee Minneapolis MN \n",
"747 0.843742 Dunn Bros Coffee Minneapolis MN \n",
"740 0.834707 Spyhouse Coffee Minneapolis MN \n",
"\n",
" Location category rating selfRating \n",
"222 Austin, TX Coffee Shop 8.9 NaN \n",
"749 Minneapolis, MN Coffee Shop 8.1 NaN \n",
"742 Minneapolis, MN Coffee Shop 8.9 NaN \n",
"745 Minneapolis, MN Coffee Shop 8.3 NaN \n",
"747 Minneapolis, MN Coffee Shop 7.9 NaN \n",
"740 Minneapolis, MN Coffee Shop 9.1 6.0 "
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Round Rock Donuts\n",
"retrieve_recommendations(222)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Sadly, a lot of the most similar places our results returned were places I've tried and didn't like. For some reason, the donuts at most places up here are usually cold, cake-based, and covered in kitschy stuff like bacon. However, Granny donuts looks like it could be promising, as does Bogart's:\n",
"\n",
"| \n", " | cosineSimilarity | \n", "Name | \n", "City | \n", "state | \n", "Location | \n", "category | \n", "rating | \n", "selfRating | \n", "
|---|---|---|---|---|---|---|---|---|
| 222 | \n", "1.000000 | \n", "Summer Moon Wood-Fired Coffee | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "Coffee Shop | \n", "8.9 | \n", "NaN | \n", "
| 749 | \n", "0.865335 | \n", "Spyhouse Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "8.1 | \n", "NaN | \n", "
| 742 | \n", "0.857070 | \n", "Five Watt Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "8.9 | \n", "NaN | \n", "
| 745 | \n", "0.848061 | \n", "Spyhouse Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "8.3 | \n", "NaN | \n", "
| 747 | \n", "0.843742 | \n", "Dunn Bros Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "7.9 | \n", "NaN | \n", "
| 740 | \n", "0.834707 | \n", "Spyhouse Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "9.1 | \n", "6.0 | \n", "
![](\"https://s3-media3.fl.yelpcdn.com/bphoto/VVpoCxOqusvCGpr5zAW-ZQ/o.jpg\")
\n",
"\n",
"### Tacos\n",
"\n",
"This is another Austin specialty that likely won't give promising results, but let's try it anyway.\n",
"\n",
"[Tacodeli](http://www.tacodeli.com/) is my personal favorite taco place in Austin (yes, it's better than [Torchy's](http://torchystacos.com/)), and they're a little different than the traditional taco (corn tortilla, meat, onions, cilantro, and lime that you might find at traditional Mexican taquerias). They're typically on flour tortillas, and diversify their flavor profiles and toppings:\n",
"\n",
"![](\"https://cdn.vox-cdn.com/thumbor/TiugzPrR2vKWCPitTTV1oo9a1iE=/114x0:1934x1365/1200x900/filters:focal(114x0:1934x1365)/cdn.vox-cdn.com/uploads/chorus_image/image/53151105/16112919_10154883686894449_6136333774221690944_o.0.0.jpg\")
"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.800639Z",
"start_time": "2017-07-02T17:30:46.752103Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" cosineSimilarity Name City \\\n",
"420 1.000000 Tacodeli Austin \n",
"986 0.869268 Rusty Taco Minneapolis \n",
"849 0.826003 Taco Taxi Minneapolis \n",
"1114 0.270148 Psycho Suzi's Motor Lounge & Tiki Garden Minneapolis \n",
"579 0.259052 Hell's Kitchen Minneapolis \n",
"838 0.256581 Brit's Pub & Eating Establishment Minneapolis \n",
"\n",
" state Location category rating selfRating \n",
"420 TX Austin, TX Taco Place 9.2 10.0 \n",
"986 MN Minneapolis, MN Taco Place 8.1 4.0 \n",
"849 MN Minneapolis, MN Taco Place 8.4 NaN \n",
"1114 MN Minneapolis, MN Theme Restaurant 8.5 2.0 \n",
"579 MN Minneapolis, MN American Restaurant 8.9 4.0 \n",
"838 MN Minneapolis, MN English Restaurant 8.8 NaN "
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Tacodeli\n",
"retrieve_recommendations(420)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It looks like there's a pretty sharp drop-off in cosine similarity after our second recommendation (which makes sense when you look at the ratio of taco places in Austin vs. Minneapolis from when we pulled our data), so I'm going to discard the bottom three. I'm surprised again that Psycho Suzi's and Brit's Pub made a second appearance since neither of them serve tacos, but I won't into that too much since their cosine similarity is really low.\n",
"\n",
"I have tried Rusty Taco, and it does seem a lot like Tacodeli. They even sell breakfast tacos, which is a very Texan thing that can be rare in the rest of the country. The primary difference is in the diversity and freshness of ingredients, and subsequently for me, taste:\n",
"\n",
"| \n", " | cosineSimilarity | \n", "Name | \n", "City | \n", "state | \n", "Location | \n", "category | \n", "rating | \n", "selfRating | \n", "
|---|---|---|---|---|---|---|---|---|
| 420 | \n", "1.000000 | \n", "Tacodeli | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "Taco Place | \n", "9.2 | \n", "10.0 | \n", "
| 986 | \n", "0.869268 | \n", "Rusty Taco | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Taco Place | \n", "8.1 | \n", "4.0 | \n", "
| 849 | \n", "0.826003 | \n", "Taco Taxi | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Taco Place | \n", "8.4 | \n", "NaN | \n", "
| 1114 | \n", "0.270148 | \n", "Psycho Suzi's Motor Lounge & Tiki Garden | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Theme Restaurant | \n", "8.5 | \n", "2.0 | \n", "
| 579 | \n", "0.259052 | \n", "Hell's Kitchen | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "American Restaurant | \n", "8.9 | \n", "4.0 | \n", "
| 838 | \n", "0.256581 | \n", "Brit's Pub & Eating Establishment | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "English Restaurant | \n", "8.8 | \n", "NaN | \n", "
![](\"http://heavytable.com/wp-content/uploads/2011/05/rusty-taco_6-x-tacos.jpg\")
\n",
"\n",
"Taco Taxi looks like it could be promising, but they appear to be more of a traditional taqueria (delicious but dissimilar). To be fair, taquerias have had the best tacos I've found up here (though most of them aren't included in this list because they were outside of the search range).\n",
"\n",
"### Burritos\n",
"\n",
"I'm not actually going to run our similarity function for this part because the burrito place back home actually disappeared from our data pulling query in between me originally running this and finally having time to annotate everything and write this post. However, I wanted to include it because it was one of my other field tests.\n",
"\n",
"[Cabo Bob's](http://cabobobs.com/) is my favorite burrito place back home, and they made it to the semi-finals in the [538 best burrito in America search](https://fivethirtyeight.com/burrito/#brackets-view) losing to the overall champion. To anyone not familiar with non-chain burrito restaurants, they are similar to Chipotle, but are usually higher quality.\n",
"\n",
"![](\"http://5a41cb68e4f4b5f1172e-bf5f2072334b80bdf7b0bc4cd64c7593.r11.cf2.rackcdn.com/16c3f0f16a61fb6aeb081d8c0dd99467-b41587790ca364353f5a4b46d585058b.jpg\")
\n",
"\n",
"[El Burrito Mercado](http://elburritomercado.com/) returned as highly similar, so we tried it. It's tucked in the back of a mercado, and has both a sit-down section as well as a lunch line similar to Cabo Bob's. We decided to go for the sit-down section since we had come from the opposite side of the metropolitan area, so the experience was a little different. My burrito was more of a traditional Mexican burrito (as opposed to Tex-Mex), but it was still pretty darn good.\n",
"\n",
"![](\"https://media-cdn.tripadvisor.com/media/photo-s/06/a6/13/44/el-burrito-mercado.jpg\")
\n",
"\n",
"### Indian\n",
"\n",
"Next up is the the [Clay Pit](https://www.claypit.com/), a contemporary Indian restaurant in Austin. They focus mostly on curry dishes with naan, though some of my friends from grad school can tell you that India has way more cuisine diversity than curry dishes.\n",
"\n",
"![](\"http://www.funjunkie.com/wp-content/themes/GeoPlaces/timthumb.php?src=http://www.funjunkie.com/wp-content/uploads/2013/06/l-23.jpg&w=600&h=345&zc=1&q=70\")
"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.856178Z",
"start_time": "2017-07-02T17:30:46.806642Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" cosineSimilarity Name City \\\n",
"338 1.000000 Clay Pit Contemporary Indian Cuisine Austin \n",
"904 0.874870 India Palace Saint Paul \n",
"909 0.853726 Darbar India Grill Minneapolis \n",
"905 0.848635 Dancing Ganesha Minneapolis \n",
"916 0.847020 Gandhi Mahal Minneapolis \n",
"917 0.845153 Best of India Indian Restaurant Saint Louis Park \n",
"906 0.837536 Gorkha Palace Minneapolis \n",
"912 0.834911 India House Saint Paul \n",
"910 0.821674 Copper Pot Indian Grill Minneapolis \n",
"\n",
" state Location category rating selfRating \n",
"338 TX Austin, TX Indian Restaurant 8.9 9.0 \n",
"904 MN Minneapolis, MN Indian Restaurant 7.9 NaN \n",
"909 MN Minneapolis, MN Indian Restaurant 7.7 NaN \n",
"905 MN Minneapolis, MN Indian Restaurant 7.3 NaN \n",
"916 MN Minneapolis, MN Indian Restaurant 8.0 NaN \n",
"917 MN Minneapolis, MN Indian Restaurant 7.6 NaN \n",
"906 MN Minneapolis, MN Indian Restaurant 9.0 NaN \n",
"912 MN Minneapolis, MN Indian Restaurant 7.9 NaN \n",
"910 MN Minneapolis, MN Indian Restaurant 7.3 NaN "
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Clay Pit\n",
"retrieve_recommendations(338, 8)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This was actually the first place I did a field test on. When I originally looked this up, we ended up trying [Gorkha Palace](http://gorkhapalace.com/) since it was the closest one to our house with the best reviews. It has a more expanded offering including Nepali and Tibetan food (though I wasn't complaining because I love [momos](https://en.wikipedia.org/wiki/Momo_(food)). It was delicious, and was very similar to the Clay Pit. We'll be going back.\n",
"\n",
"| \n", " | cosineSimilarity | \n", "Name | \n", "City | \n", "state | \n", "Location | \n", "category | \n", "rating | \n", "selfRating | \n", "
|---|---|---|---|---|---|---|---|---|
| 338 | \n", "1.000000 | \n", "Clay Pit Contemporary Indian Cuisine | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "Indian Restaurant | \n", "8.9 | \n", "9.0 | \n", "
| 904 | \n", "0.874870 | \n", "India Palace | \n", "Saint Paul | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "7.9 | \n", "NaN | \n", "
| 909 | \n", "0.853726 | \n", "Darbar India Grill | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "7.7 | \n", "NaN | \n", "
| 905 | \n", "0.848635 | \n", "Dancing Ganesha | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "7.3 | \n", "NaN | \n", "
| 916 | \n", "0.847020 | \n", "Gandhi Mahal | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "8.0 | \n", "NaN | \n", "
| 917 | \n", "0.845153 | \n", "Best of India Indian Restaurant | \n", "Saint Louis Park | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "7.6 | \n", "NaN | \n", "
| 906 | \n", "0.837536 | \n", "Gorkha Palace | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "9.0 | \n", "NaN | \n", "
| 912 | \n", "0.834911 | \n", "India House | \n", "Saint Paul | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "7.9 | \n", "NaN | \n", "
| 910 | \n", "0.821674 | \n", "Copper Pot Indian Grill | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Indian Restaurant | \n", "7.3 | \n", "NaN | \n", "
![](\"https://s3-media4.fl.yelpcdn.com/bphoto/gueLSLa3LY6LGjYVrGdZkw/o.jpg\")
\n",
"\n",
"### French/Bistro\n",
"\n",
"One of our favorite places back home is [Blue Dahlia Bistro](http://www.bluedahliabistro.com/), a European-style bistro specializing in French fusion. They use a lot of fresh and simple ingredients, and it's a great place for a date night due to its cozy interior and decorated back patio.\n",
"\n",
"![](\"https://ferociousfoodiedotcom.files.wordpress.com/2016/03/blue-dahlia21.jpg\")
"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:46.926729Z",
"start_time": "2017-07-02T17:30:46.863183Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" cosineSimilarity Name City state \\\n",
"124 1.000000 Blue Dahlia Bistro Austin TX \n",
"584 0.827561 Wilde Roast Cafe Minneapolis MN \n",
"762 0.785928 Jensen's Cafe Burnsville MN \n",
"741 0.781564 Cafe Latte Saint Paul MN \n",
"759 0.777269 Peoples Organic Edina MN \n",
"686 0.772962 Black Dog, Lowertown Saint Paul MN \n",
"\n",
" Location category rating selfRating \n",
"124 Austin, TX Café 9.1 10.0 \n",
"584 Minneapolis, MN Café 8.6 NaN \n",
"762 Minneapolis, MN Café 8.2 NaN \n",
"741 Minneapolis, MN Café 9.3 NaN \n",
"759 Minneapolis, MN Café 8.1 NaN \n",
"686 Minneapolis, MN Café 8.6 NaN "
]
},
"execution_count": 46,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Blue Dahlia\n",
"retrieve_recommendations(124)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I think our heavier category weighting is hurting us here since Blue Dahlia is classified as a café. Most of the recommendations focus on American food (remember, American food is king in Minneapolis), but I'm guessing the [Wilde Roast Cafe](http://wildecafe.com/) was listed as the most similar restaurant due to the similarly cozy interior and various espresso drinks they offer. I've been to the Wilde Roast before beginning this project, and I can tell you that the food is completely different.\n",
"\n",
"| \n", " | cosineSimilarity | \n", "Name | \n", "City | \n", "state | \n", "Location | \n", "category | \n", "rating | \n", "selfRating | \n", "
|---|---|---|---|---|---|---|---|---|
| 124 | \n", "1.000000 | \n", "Blue Dahlia Bistro | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "Café | \n", "9.1 | \n", "10.0 | \n", "
| 584 | \n", "0.827561 | \n", "Wilde Roast Cafe | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Café | \n", "8.6 | \n", "NaN | \n", "
| 762 | \n", "0.785928 | \n", "Jensen's Cafe | \n", "Burnsville | \n", "MN | \n", "Minneapolis, MN | \n", "Café | \n", "8.2 | \n", "NaN | \n", "
| 741 | \n", "0.781564 | \n", "Cafe Latte | \n", "Saint Paul | \n", "MN | \n", "Minneapolis, MN | \n", "Café | \n", "9.3 | \n", "NaN | \n", "
| 759 | \n", "0.777269 | \n", "Peoples Organic | \n", "Edina | \n", "MN | \n", "Minneapolis, MN | \n", "Café | \n", "8.1 | \n", "NaN | \n", "
| 686 | \n", "0.772962 | \n", "Black Dog, Lowertown | \n", "Saint Paul | \n", "MN | \n", "Minneapolis, MN | \n", "Café | \n", "8.6 | \n", "NaN | \n", "
![](\"https://igx.4sqi.net/img/general/600x600/2736_Sy4oGIGsTbzfD5ykDTnPCUDgISnx9xQnDlCpqu6bGyU.jpg\")
\n",
"\n",
"\n",
"### Coffee\n",
"\n",
"Speaking of coffee, let's wrap this up with coffee recommendations. I still have a lot of places to find matches for, but since I did this project as a poor graduate student, most of them would be \"that looks promising, but I haven't tried it yet\". \n",
"\n",
"Sadly, a lot of my favorite coffee shops from in and around Austin didn't show up since Starbucks took up most of the space when searching for coffee places (remember, we're limited to 50 results per category). We ended up with [Mozart's](http://www.mozartscoffee.com/):\n",
"\n",
"![](\"http://www.mozartscoffee.com/images/photo9_5.jpg\")
\n",
"\n",
"and the [Coffee Bean & Tea Leaf](https://www.coffeebean.com/):\n",
"\n",
"![](\"http://dining.ucr.edu/images/foodshot-coffeebean.jpg\")
\n",
"\n",
"I ran the results for Mozart's and didn't get anything too similar back. To be fair, there aren't any coffee shops on a river up here, and I'm sure most comments for Mozart's are about the view. \n",
"\n",
"Let's go with The Coffee Bean & Tea Leaf instead. It's actually a small chain out of California that is, in my opinion, tastier than Starbucks."
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"ExecuteTime": {
"end_time": "2017-07-02T17:30:47.037307Z",
"start_time": "2017-07-02T17:30:46.935735Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" cosineSimilarity Name City state \\\n",
"558 1.000000 The Coffee Bean & Tea Leaf Austin TX \n",
"742 0.811790 Five Watt Coffee Minneapolis MN \n",
"747 0.805479 Dunn Bros Coffee Minneapolis MN \n",
"757 0.797197 Caribou Coffee Savage MN \n",
"760 0.791427 Dunn Bros Coffee Minneapolis MN \n",
"778 0.790609 Starbucks Bloomington MN \n",
"\n",
" Location category rating selfRating \n",
"558 Austin, TX Coffee Shop 8.1 8.0 \n",
"742 Minneapolis, MN Coffee Shop 8.9 NaN \n",
"747 Minneapolis, MN Coffee Shop 7.9 NaN \n",
"757 Minneapolis, MN Coffee Shop 7.7 NaN \n",
"760 Minneapolis, MN Coffee Shop 7.1 NaN \n",
"778 Minneapolis, MN Coffee Shop 8.0 NaN "
]
},
"execution_count": 47,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Coffee Bean & Tea Leaf\n",
"retrieve_recommendations(558)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These are perfectly logical results. [Caribou](https://www.cariboucoffee.com/) is a chain popular in the Midwest (I also describe it as 'like Starbucks but better' to friends back home), and [Dunn Bros](https://dunnbrothers.com/) is similar, but specific to Minnesota.\n",
"\n",
"This chart from [an article on FlowingData](http://flowingdata.com/2014/03/18/coffee-place-geography/) helps describe why I think these results make so much sense:\n",
"\n",
"| \n", " | cosineSimilarity | \n", "Name | \n", "City | \n", "state | \n", "Location | \n", "category | \n", "rating | \n", "selfRating | \n", "
|---|---|---|---|---|---|---|---|---|
| 558 | \n", "1.000000 | \n", "The Coffee Bean & Tea Leaf | \n", "Austin | \n", "TX | \n", "Austin, TX | \n", "Coffee Shop | \n", "8.1 | \n", "8.0 | \n", "
| 742 | \n", "0.811790 | \n", "Five Watt Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "8.9 | \n", "NaN | \n", "
| 747 | \n", "0.805479 | \n", "Dunn Bros Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "7.9 | \n", "NaN | \n", "
| 757 | \n", "0.797197 | \n", "Caribou Coffee | \n", "Savage | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "7.7 | \n", "NaN | \n", "
| 760 | \n", "0.791427 | \n", "Dunn Bros Coffee | \n", "Minneapolis | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "7.1 | \n", "NaN | \n", "
| 778 | \n", "0.790609 | \n", "Starbucks | \n", "Bloomington | \n", "MN | \n", "Minneapolis, MN | \n", "Coffee Shop | \n", "8.0 | \n", "NaN | \n", "
![](\"http://i1.wp.com/flowingdata.com/wp-content/uploads/2014/03/coffee-breakdowns-final.png?w=954\")
\n",
"\n",
"As for my verdict on Caribou, I actually like it better than the Coffee Bean and Tea Leaf. In fact, I actually have a gift card for them in my wallet right now. There also used to be a location for the Coffee Bean and Tea Leaf up here, but they closed it down shortly after I moved here (just like all of the Minnesotan [Schlotzsky's](https://www.schlotzskys.com/) locations...I'm still mad about that)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Summary\n",
"\n",
"Like any other tool, this method isn't perfect for every application, but it can work if we use it effectively. While there is room for improvement, I am pleased with how it has been working for me so far.\n",
"\n",
"I'm going to continue to use this when we can't decide on a restaurant and feeling homesick, and will likely update this post in the future with either more places I try out or when I move to a new city in the future. In the meantime, I hope you enjoyed reading, and feel free to use my code ([here is the github link](https://github.com/JeffMacaluso/Blog/blob/master/Restaurant%20Recommender.ipynb)) to try it out for your own purposes.\n",
"\n",
"Happy eating!"
]
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python [default]",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.3"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}