{ "cells": [ { "cell_type": "markdown", "id": "136200aa", "metadata": {}, "source": [ "## GGIS 527\n", "### Lab 7 Analyzing Geo-Text Data with Natural Language Processing (NLP) Tools\n", "#### Developed by Zhaonan Wang in Fall 2023\n", "In this lab, you will go through the data wrangling process with both types of geo-text data, with explicit geo-tags or implicit location mentions within text.\n", "- [**Explicit geo-text dataset**](#explicit): 2145 business located in Illinois side of St. Louis, derived from [Yelp Academic Dataset](https://www.yelp.com/dataset). Each data record also comes with a user review, in plain text. Refer [here for data format](https://www.yelp.com/dataset/documentation/main) of utilized business and review data. Your task is to perform sentiment analysis on each review and map the polarity score onto a map.\n", "- [**Implicit geo-text dataset**](#implicit): news reports usually mention various locations, like countries, states, and even local toponyms (place names). In this notebook, you will play with a toy corpus containing three chunks of online news about some dam failure events. Your task to extract location mentions buried in the unstructured text." ] }, { "cell_type": "markdown", "id": "f7fabf37", "metadata": {}, "source": [ "\n", "### Explicit Geo-Text Data Analysis" ] }, { "cell_type": "code", "execution_count": 1, "id": "f82f02e2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(2145, 25)\n" ] }, { "data": { "text/html": [ "
\n", " | Unnamed: 0 | \n", "Unnamed: 0_x | \n", "business_id | \n", "name | \n", "address | \n", "city | \n", "state | \n", "postal_code | \n", "latitude | \n", "longitude | \n", "... | \n", "hours | \n", "Unnamed: 0_y | \n", "review_id | \n", "user_id | \n", "stars_y | \n", "useful | \n", "funny | \n", "cool | \n", "text | \n", "date | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "0 | \n", "38 | \n", "LcAozWCMLGjwRbokaJAKMg | \n", "Edwardsville Children's Museum | \n", "722 Holyoake Rd | \n", "Edwardsville | \n", "IL | \n", "62025 | \n", "38.804395 | \n", "-89.949733 | \n", "... | \n", "{'Monday': '10:0-15:0', 'Tuesday': '9:30-14:0'... | \n", "313 | \n", "LfsU2lVUr1-pC802v0o32A | \n", "mRgAqvxz9jHYpm8ccIjZUQ | \n", "5.0 | \n", "0 | \n", "0 | \n", "0 | \n", "Place rocks excellent children's activities an... | \n", "2016-07-04 20:56:17 | \n", "
1 | \n", "13 | \n", "41 | \n", "ljxNT9p0y7YMPx0fcNBGig | \n", "Tony's Restaurant & 3rd Street Cafe | \n", "312 Piasa St | \n", "Alton | \n", "IL | \n", "62002 | \n", "38.896563 | \n", "-90.186203 | \n", "... | \n", "{'Monday': '0:0-0:0', 'Tuesday': '16:0-21:30',... | \n", "20 | \n", "uiqzlDEsUN_y1awEw_HHDA | \n", "qmQPWMV_YYmwV2DyvmIDYQ | \n", "5.0 | \n", "0 | \n", "0 | \n", "0 | \n", "We had been driving around for some time, on a... | \n", "2018-07-17 01:07:49 | \n", "
2 | \n", "118 | \n", "48 | \n", "bCBPXIVfVzBZBEpFu29dcg | \n", "All In Shipping | \n", "5343 Belleville Crossing St | \n", "Belleville | \n", "IL | \n", "62226 | \n", "38.517586 | \n", "-90.021929 | \n", "... | \n", "NaN | \n", "1378 | \n", "oZqb2LRrJFaEjTz9ETzpPA | \n", "BHrWZS0J0FuJuLqeNk6J7w | \n", "5.0 | \n", "0 | \n", "0 | \n", "0 | \n", "I love this little local business. They have e... | \n", "2017-01-20 14:13:47 | \n", "
3 | \n", "123 | \n", "86 | \n", "sE6jSnvMts_MAn-b4OkMAw | \n", "K-9 Groom Room | \n", "820 Industrial Dr | \n", "Troy | \n", "IL | \n", "62294 | \n", "38.716244 | \n", "-89.885830 | \n", "... | \n", "{'Monday': '8:0-16:0', 'Tuesday': '8:0-16:0', ... | \n", "194 | \n", "UjBwlySBW4iPpFWGOw5Xkw | \n", "SE85OT0FKxeL28izk-5POg | \n", "4.0 | \n", "3 | \n", "0 | \n", "0 | \n", "This is another great local business. Our two... | \n", "2011-03-25 17:36:39 | \n", "
4 | \n", "128 | \n", "102 | \n", "EuRGgOwJ0g1vTj2R04j37Q | \n", "Crafty Crab | \n", "51 Ludwig Dr | \n", "Fairview Heights | \n", "IL | \n", "62208 | \n", "38.601298 | \n", "-89.989683 | \n", "... | \n", "{'Monday': '12:0-22:0', 'Tuesday': '12:0-22:0'... | \n", "3261 | \n", "DrWMCBMRweRydBEk-OLKYg | \n", "h3o-SqWjDeMI2fCJI63-jg | \n", "1.0 | \n", "0 | \n", "0 | \n", "0 | \n", "Waiter was absolutely terrible ordered our foo... | \n", "2021-11-06 02:07:15 | \n", "
5 rows × 25 columns
\n", "\n", " | Unnamed: 0 | \n", "Unnamed: 0_x | \n", "business_id | \n", "name | \n", "address | \n", "city | \n", "state | \n", "postal_code | \n", "latitude | \n", "longitude | \n", "... | \n", "Unnamed: 0_y | \n", "review_id | \n", "user_id | \n", "stars_y | \n", "useful | \n", "funny | \n", "cool | \n", "text | \n", "date | \n", "sentiment | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "0 | \n", "38 | \n", "LcAozWCMLGjwRbokaJAKMg | \n", "Edwardsville Children's Museum | \n", "722 Holyoake Rd | \n", "Edwardsville | \n", "IL | \n", "62025 | \n", "38.804395 | \n", "-89.949733 | \n", "... | \n", "313 | \n", "LfsU2lVUr1-pC802v0o32A | \n", "mRgAqvxz9jHYpm8ccIjZUQ | \n", "5.0 | \n", "0 | \n", "0 | \n", "0 | \n", "Place rocks excellent children's activities an... | \n", "2016-07-04 20:56:17 | \n", "0.436623 | \n", "
1 | \n", "13 | \n", "41 | \n", "ljxNT9p0y7YMPx0fcNBGig | \n", "Tony's Restaurant & 3rd Street Cafe | \n", "312 Piasa St | \n", "Alton | \n", "IL | \n", "62002 | \n", "38.896563 | \n", "-90.186203 | \n", "... | \n", "20 | \n", "uiqzlDEsUN_y1awEw_HHDA | \n", "qmQPWMV_YYmwV2DyvmIDYQ | \n", "5.0 | \n", "0 | \n", "0 | \n", "0 | \n", "We had been driving around for some time, on a... | \n", "2018-07-17 01:07:49 | \n", "0.200250 | \n", "
2 | \n", "118 | \n", "48 | \n", "bCBPXIVfVzBZBEpFu29dcg | \n", "All In Shipping | \n", "5343 Belleville Crossing St | \n", "Belleville | \n", "IL | \n", "62226 | \n", "38.517586 | \n", "-90.021929 | \n", "... | \n", "1378 | \n", "oZqb2LRrJFaEjTz9ETzpPA | \n", "BHrWZS0J0FuJuLqeNk6J7w | \n", "5.0 | \n", "0 | \n", "0 | \n", "0 | \n", "I love this little local business. They have e... | \n", "2017-01-20 14:13:47 | \n", "0.266146 | \n", "
3 | \n", "123 | \n", "86 | \n", "sE6jSnvMts_MAn-b4OkMAw | \n", "K-9 Groom Room | \n", "820 Industrial Dr | \n", "Troy | \n", "IL | \n", "62294 | \n", "38.716244 | \n", "-89.885830 | \n", "... | \n", "194 | \n", "UjBwlySBW4iPpFWGOw5Xkw | \n", "SE85OT0FKxeL28izk-5POg | \n", "4.0 | \n", "3 | \n", "0 | \n", "0 | \n", "This is another great local business. Our two... | \n", "2011-03-25 17:36:39 | \n", "0.481250 | \n", "
4 | \n", "128 | \n", "102 | \n", "EuRGgOwJ0g1vTj2R04j37Q | \n", "Crafty Crab | \n", "51 Ludwig Dr | \n", "Fairview Heights | \n", "IL | \n", "62208 | \n", "38.601298 | \n", "-89.989683 | \n", "... | \n", "3261 | \n", "DrWMCBMRweRydBEk-OLKYg | \n", "h3o-SqWjDeMI2fCJI63-jg | \n", "1.0 | \n", "0 | \n", "0 | \n", "0 | \n", "Waiter was absolutely terrible ordered our foo... | \n", "2021-11-06 02:07:15 | \n", "-0.124603 | \n", "
5 rows × 26 columns
\n", "