{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Quickstart\n", "\n", "Hi! Glad to see you made it to the Quickstart guide! In this guide you are introduced to how CARTOframes can be used by data scientists in spatial analysis workflows. Using simulated Starbucks revenue data, this guide walks through some common steps a data scientist takes to answer the following question: which stores are performing better than others?\n", "\n", "Before you get started, we encourage you to have CARTOframes and Python 3 installed so you can get a feel for the library by using it:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# pip install cartoframes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For other ways to install CARTOframes, check out the [Installation guide](/developers/cartoframes/guides/Installation)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Spatial analysis scenario\n", "\n", "Let's say you are a data scientist working for Starbucks and you want to better understand why some stores in Brooklyn, New York, perform better than others.\n", "\n", "To begin, let's outline a workflow:\n", "\n", "- Get and explore your own data\n", "- Create areas of influence for your stores\n", "- Enrich your data\n", "- And finally, share the results of your analysis with your team\n", "\n", "Let's get started!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Get and explore your own data\n", "\n", "We are going to use a dataset that contains information about the location of Starbucks and each store's annual revenue.\n", "\n", "As a first exploratory step, read it into a Jupyter Notebook using [pandas](https://pandas.pydata.org/)." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
nameaddressrevenue
0Franklin Ave & Eastern Pkwy341 Eastern Pkwy,Brooklyn, NY 112381321040.772
1607 Brighton Beach Ave607 Brighton Beach Avenue,Brooklyn, NY 112351268080.418
265th St & 18th Ave6423 18th Avenue,Brooklyn, NY 112041248133.699
3Bay Ridge Pkwy & 3rd Ave7419 3rd Avenue,Brooklyn, NY 112091185702.676
4Caesar's Bay Shopping Center8973 Bay Parkway,Brooklyn, NY 112141148427.411
\n", "
" ], "text/plain": [ " name address \\\n", "0 Franklin Ave & Eastern Pkwy 341 Eastern Pkwy,Brooklyn, NY 11238 \n", "1 607 Brighton Beach Ave 607 Brighton Beach Avenue,Brooklyn, NY 11235 \n", "2 65th St & 18th Ave 6423 18th Avenue,Brooklyn, NY 11204 \n", "3 Bay Ridge Pkwy & 3rd Ave 7419 3rd Avenue,Brooklyn, NY 11209 \n", "4 Caesar's Bay Shopping Center 8973 Bay Parkway,Brooklyn, NY 11214 \n", "\n", " revenue \n", "0 1321040.772 \n", "1 1268080.418 \n", "2 1248133.699 \n", "3 1185702.676 \n", "4 1148427.411 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "stores_df = pd.read_csv('http://libs.cartocdn.com/cartoframes/files/starbucks_brooklyn.csv')\n", "stores_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To display your stores as points on a map, you first have to convert the `address` column into geometries. This process is called [geocoding](https://carto.com/help/working-with-data/geocoding-types/) and CARTO provides a straightforward way to do it (you can learn more about it in the [Data Services guide](/developers/cartoframes/guides/Data-Services)).\n", "\n", "In order to geocode, you have to set your CARTO credentials. If you aren't sure about your API key, check the [Authentication guide](/developers/cartoframes/guides/Authentication/) to learn how to get it. In case you want to see the geocoded result, without being logged in, [you can get it here](http://libs.cartocdn.com/cartoframes/files/starbucks_brooklyn_geocoded.csv).\n", "\n", "> Note: If you don't have an account yet, you can get a trial free account by [signing up here](https://carto.com/signup/)." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from cartoframes.auth import set_default_credentials\n", "\n", "set_default_credentials('creds.json')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that your credentials are set, we are ready to geocode the dataframe. The resulting data will be a [GeoDataFrame](http://geopandas.org/data_structures.html#geodataframe)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success! Data geocoded correctly\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
the_geomnameaddressrevenuegc_status_relcarto_geocode_hash
0POINT (-73.95746 40.67102)Franklin Ave & Eastern Pkwy341 Eastern Pkwy,Brooklyn, NY 112381321040.7720.919212e0e908d8c64d07c6a94827322397
1POINT (-73.96122 40.57796)607 Brighton Beach Ave607 Brighton Beach Avenue,Brooklyn, NY 112351268080.4180.97b1bbfe2893914a350193969a682dc1f5
2POINT (-73.98978 40.61944)65th St & 18th Ave6423 18th Avenue,Brooklyn, NY 112041248133.6990.94e47cf7b16d6c9b53c63e86a0418add1d
3POINT (-74.02750 40.63202)Bay Ridge Pkwy & 3rd Ave7419 3rd Avenue,Brooklyn, NY 112091185702.6760.942f21749c02f73116892eb3b6fd5d5738
4POINT (-74.00098 40.59321)Caesar's Bay Shopping Center8973 Bay Parkway,Brooklyn, NY 112141148427.4110.94134c23973313802448365db6235783f9
\n", "
" ], "text/plain": [ " the_geom name \\\n", "0 POINT (-73.95746 40.67102) Franklin Ave & Eastern Pkwy \n", "1 POINT (-73.96122 40.57796) 607 Brighton Beach Ave \n", "2 POINT (-73.98978 40.61944) 65th St & 18th Ave \n", "3 POINT (-74.02750 40.63202) Bay Ridge Pkwy & 3rd Ave \n", "4 POINT (-74.00098 40.59321) Caesar's Bay Shopping Center \n", "\n", " address revenue gc_status_rel \\\n", "0 341 Eastern Pkwy,Brooklyn, NY 11238 1321040.772 0.91 \n", "1 607 Brighton Beach Avenue,Brooklyn, NY 11235 1268080.418 0.97 \n", "2 6423 18th Avenue,Brooklyn, NY 11204 1248133.699 0.94 \n", "3 7419 3rd Avenue,Brooklyn, NY 11209 1185702.676 0.94 \n", "4 8973 Bay Parkway,Brooklyn, NY 11214 1148427.411 0.94 \n", "\n", " carto_geocode_hash \n", "0 9212e0e908d8c64d07c6a94827322397 \n", "1 b1bbfe2893914a350193969a682dc1f5 \n", "2 e47cf7b16d6c9b53c63e86a0418add1d \n", "3 2f21749c02f73116892eb3b6fd5d5738 \n", "4 134c23973313802448365db6235783f9 " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.data.services import Geocoding\n", "\n", "stores_gdf, _ = Geocoding().geocode(stores_df, street='address')\n", "stores_gdf.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Done! Now that the stores are geocoded, you will notice a new column named `geometry` has been added. This column stores the geographic location of each store and it's used to plot each location on the map.\n", "\n", "You can quickly visualize your geocoded dataframe using the Map and Layer classes. Check out the [Data Visualization guide](/developers/cartoframes/guides/Data-Visualization) to learn more about the visualization capabilities inside of CARTOframes." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " None\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", " Static map image\n", " \n", " \n", "
\n", "
\n", "
\n", " \n", " \n", "
\n", "
\n", "
\n", "\n", " \n", "\n", "
\n", "
\n", " :\n", "
\n", " \n", " \n", "
\n", "
\n", "\n", "
\n", " StackTrace\n", "
    \n", "
    \n", "
    \n", "\n", "\n", "\n", "\n", "\n", "\">\n", "\n", "" ], "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.viz import Map, Layer\n", "\n", "Map(Layer(stores_gdf))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Great! You have a map!\n", "\n", "With the stores plotted on the map, you now have a better sense about where each one is. To continue your exploration, you want to know which stores earn the most yearly revenue. To do this, you can use the [`size_continuous_style`](/developers/cartoframes/examples/#example-size-continuous-style) visualization layer:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " None\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", " Static map image\n", " \n", " \n", "
    \n", "
    \n", "
    \n", " \n", " \n", "
    \n", "
    \n", " \n", "\n", "
    \n", " \n", " \n", " \n", " \n", " \n", "
    \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
    \n", " \n", " \n", "
    \n", "
    \n", "
    \n", "
    \n", " \n", "
    \n", "
    \n", "
    \n", "\n", " \n", "\n", "
    \n", "
    \n", " :\n", "
    \n", " \n", " \n", "
    \n", "
    \n", "\n", "
    \n", " StackTrace\n", "
      \n", "
      \n", "
      \n", "\n", "\n", "\n", "\n", "\n", "\">\n", "\n", "" ], "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.viz import Map, Layer, size_continuous_style\n", "\n", "Map(Layer(stores_gdf, size_continuous_style('revenue', size_range=[10,40]), title='Annual Revenue ($)'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Good job! By using the [`size continuous visualization style`](/developers/cartoframes/examples/#example-size-continuous-style) you can see right away where the stores with higher revenue are. By default, visualization styles also provide a popup with the mapped value and an appropriate legend." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create your areas of influence\n", "\n", "Similar to geocoding, there is a straightforward method for creating isochrones to define areas of influence around each store. Isochrones are concentric polygons that display equally calculated levels over a given surface area measured by time.\n", "\n", "For this analysis, let's create isochrones for each store that cover the area within a 15 minute walk.\n", "\n", "To do this you will use the Isolines data service:\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success! Isolines created correctly\n" ] }, { "data": { "text/html": [ "
      \n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
      source_iddata_rangethe_geom
      00900MULTIPOLYGON (((-73.96743 40.67345, -73.96683 ...
      11900MULTIPOLYGON (((-73.97035 40.57749, -73.97009 ...
      22900MULTIPOLYGON (((-73.99781 40.62418, -73.99755 ...
      33900MULTIPOLYGON (((-74.03678 40.63431, -74.03618 ...
      44900MULTIPOLYGON (((-74.00451 40.59723, -74.00391 ...
      \n", "
      " ], "text/plain": [ " source_id data_range the_geom\n", "0 0 900 MULTIPOLYGON (((-73.96743 40.67345, -73.96683 ...\n", "1 1 900 MULTIPOLYGON (((-73.97035 40.57749, -73.97009 ...\n", "2 2 900 MULTIPOLYGON (((-73.99781 40.62418, -73.99755 ...\n", "3 3 900 MULTIPOLYGON (((-74.03678 40.63431, -74.03618 ...\n", "4 4 900 MULTIPOLYGON (((-74.00451 40.59723, -74.00391 ..." ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.data.services import Isolines\n", "\n", "isochrones_gdf, _ = Isolines().isochrones(stores_gdf, [15*60], mode='walk')\n", "isochrones_gdf.head()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " None\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", " Static map image\n", " \n", " \n", "
      \n", "
      \n", "
      \n", " \n", " \n", "
      \n", "
      \n", " \n", "\n", "
      \n", " \n", " \n", " \n", " \n", " \n", "
      \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
      \n", " \n", " \n", " \n", " \n", "
      \n", "
      \n", "
      \n", "
      \n", " \n", "
      \n", "
      \n", "
      \n", "\n", " \n", "\n", "
      \n", "
      \n", " :\n", "
      \n", " \n", " \n", "
      \n", "
      \n", "\n", "
      \n", " StackTrace\n", "
        \n", "
        \n", "
        \n", "\n", "\n", "\n", "\n", "\n", "\">\n", "\n", "" ], "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stores_map = Map([\n", " Layer(isochrones_gdf),\n", " Layer(stores_gdf, size_continuous_style('revenue', size_range=[10,40]), title='Annual Revenue ($)')\n", "])\n", "\n", "stores_map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There they are! To learn more about creating isochrones and isodistances check out the [Data Services guide](/developers/cartoframes/guides/Data-Services).\n", "\n", "> Note: You will see how to publish a map in the last section. If you already want to publish this map, you can do it by calling `stores_map.publish('starbucks_isochrones', password=None)`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Enrich your data\n", "\n", "Now that you have the area of influence calculated for each store, let's take a look at how to augment the result with population information to help better understand a store's average revenue per person.\n", "\n", "> Note: To be able to use the Enrichment functions you need an Enterprise CARTO account with Data Observatory 2.0 enabled. Please contact your Customer Success Manager or contact us at [sales@carto.com](mailto:sales@carto.com) for more information.\n", "\n", "\n", "First, let's find the demographic variable we need. We will use the `Catalog` class that can be filtered by country and category. In our case, we have to look for USA demographics datasets. Let's see which public ones are available." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "You can find more entities with the Global country filter. To apply that filter run:\n", "\tCatalog().country('glo')\n" ] }, { "data": { "text/html": [ "
        \n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
        slugnamedescriptioncategory_idcountry_iddata_source_idprovider_idgeography_namegeography_descriptiontemporal_aggregationtime_coverageupdate_frequencyis_public_datalangversioncategory_nameprovider_namegeography_idid
        4wp_population_704f6b75Population Mosaics - United States of America ...Mosaiced 1km resolution global datasets. The m...demographicsusapopulationworldpopGrid 1km - United States of AmericaGlobal grid at aprox. 1-kilometer resolution (...yearly[2020-01-01, 2021-01-01)yearlyTrueeng2020DemographicsWorldPopcarto-do-public-data.worldpop.geography_usa_gr...carto-do-public-data.worldpop.demographics_pop...
        5wp_population_22be8012Population Mosaics - United States of America ...Mosaiced 1km resolution global datasets. The m...demographicsusapopulationworldpopGrid 1km - United States of AmericaGlobal grid at aprox. 1-kilometer resolution (...yearly[2019-01-01, 2020-01-01)yearlyTrueeng2019DemographicsWorldPopcarto-do-public-data.worldpop.geography_usa_gr...carto-do-public-data.worldpop.demographics_pop...
        6wp_population_55b9b084Population Mosaics - United States of America ...Mosaiced 1km resolution global datasets. The m...demographicsusapopulationworldpopGrid 1km - United States of AmericaGlobal grid at aprox. 1-kilometer resolution (...yearly[2018-01-01, 2019-01-01)yearlyTrueeng2018DemographicsWorldPopcarto-do-public-data.worldpop.geography_usa_gr...carto-do-public-data.worldpop.demographics_pop...
        7wp_population_c506ad15Population Mosaics - United States of America ...Mosaiced 1km resolution global datasets. The m...demographicsusapopulationworldpopGrid 1km - United States of AmericaGlobal grid at aprox. 1-kilometer resolution (...yearly[2017-01-01, 2018-01-01)yearlyTrueeng2017DemographicsWorldPopcarto-do-public-data.worldpop.geography_usa_gr...carto-do-public-data.worldpop.demographics_pop...
        8wp_population_b2019d83Population Mosaics - United States of America ...Mosaiced 1km resolution global datasets. The m...demographicsusapopulationworldpopGrid 1km - United States of AmericaGlobal grid at aprox. 1-kilometer resolution (...yearly[2016-01-01, 2017-01-01)yearlyTrueeng2016DemographicsWorldPopcarto-do-public-data.worldpop.geography_usa_gr...carto-do-public-data.worldpop.demographics_pop...
        ............................................................
        530bls_labor_35c98360Labor - United States of America (County, 1992Q1)Extract from the Quarterly Census of Emploment...demographicsusalaborusa_blsCounty - United States of America (2018)Shoreline clipped TIGER/Line boundaries. More ...quarterly[1992-01-01, 1992-04-01)quarterlyTrueeng1992q1DemographicsBureau of Labor Statisticscarto-do-public-data.carto.geography_usa_count...carto-do-public-data.usa_bls.demographics_labo...
        531bls_labor_2c89d72eLabor - United States of America (County, 2003Q3)Extract from the Quarterly Census of Emploment...demographicsusalaborusa_blsCounty - United States of America (2018)Shoreline clipped TIGER/Line boundaries. More ...quarterly[2003-07-01, 2003-10-01)quarterlyTrueeng2003q3DemographicsBureau of Labor Statisticscarto-do-public-data.carto.geography_usa_count...carto-do-public-data.usa_bls.demographics_labo...
        532bls_labor_2d4bbd19Labor - United States of America (County, 2002Q3)Extract from the Quarterly Census of Emploment...demographicsusalaborusa_blsCounty - United States of America (2018)Shoreline clipped TIGER/Line boundaries. More ...quarterly[2002-07-01, 2002-10-01)quarterlyTrueeng2002q3DemographicsBureau of Labor Statisticscarto-do-public-data.carto.geography_usa_count...carto-do-public-data.usa_bls.demographics_labo...
        533bls_labor_933c1897Labor - United States of America (County, 2017Q3)Extract from the Quarterly Census of Emploment...demographicsusalaborusa_blsCounty - United States of America (2018)Shoreline clipped TIGER/Line boundaries. More ...quarterly[2017-07-01, 2017-10-01)quarterlyTrueeng2017q3DemographicsBureau of Labor Statisticscarto-do-public-data.carto.geography_usa_count...carto-do-public-data.usa_bls.demographics_labo...
        534acs_sociodemogr_4bcba275Sociodemographics - United States of America (...The American Community Survey (ACS) is an ongo...demographicsusasociodemographicsusa_acsCore-based Statistical Area - United States of...Shoreline clipped TIGER/Line boundaries. More ...5yrs[2009-01-01, 2014-01-01)yearlyTrueeng20092013DemographicsAmerican Community Surveycarto-do-public-data.carto.geography_usa_cbsa_...carto-do-public-data.usa_acs.demographics_soci...
        \n", "

        440 rows × 19 columns

        \n", "
        " ], "text/plain": [ " slug \\\n", "4 wp_population_704f6b75 \n", "5 wp_population_22be8012 \n", "6 wp_population_55b9b084 \n", "7 wp_population_c506ad15 \n", "8 wp_population_b2019d83 \n", ".. ... \n", "530 bls_labor_35c98360 \n", "531 bls_labor_2c89d72e \n", "532 bls_labor_2d4bbd19 \n", "533 bls_labor_933c1897 \n", "534 acs_sociodemogr_4bcba275 \n", "\n", " name \\\n", "4 Population Mosaics - United States of America ... \n", "5 Population Mosaics - United States of America ... \n", "6 Population Mosaics - United States of America ... \n", "7 Population Mosaics - United States of America ... \n", "8 Population Mosaics - United States of America ... \n", ".. ... \n", "530 Labor - United States of America (County, 1992Q1) \n", "531 Labor - United States of America (County, 2003Q3) \n", "532 Labor - United States of America (County, 2002Q3) \n", "533 Labor - United States of America (County, 2017Q3) \n", "534 Sociodemographics - United States of America (... \n", "\n", " description category_id \\\n", "4 Mosaiced 1km resolution global datasets. The m... demographics \n", "5 Mosaiced 1km resolution global datasets. The m... demographics \n", "6 Mosaiced 1km resolution global datasets. The m... demographics \n", "7 Mosaiced 1km resolution global datasets. The m... demographics \n", "8 Mosaiced 1km resolution global datasets. The m... demographics \n", ".. ... ... \n", "530 Extract from the Quarterly Census of Emploment... demographics \n", "531 Extract from the Quarterly Census of Emploment... demographics \n", "532 Extract from the Quarterly Census of Emploment... demographics \n", "533 Extract from the Quarterly Census of Emploment... demographics \n", "534 The American Community Survey (ACS) is an ongo... demographics \n", "\n", " country_id data_source_id provider_id \\\n", "4 usa population worldpop \n", "5 usa population worldpop \n", "6 usa population worldpop \n", "7 usa population worldpop \n", "8 usa population worldpop \n", ".. ... ... ... \n", "530 usa labor usa_bls \n", "531 usa labor usa_bls \n", "532 usa labor usa_bls \n", "533 usa labor usa_bls \n", "534 usa sociodemographics usa_acs \n", "\n", " geography_name \\\n", "4 Grid 1km - United States of America \n", "5 Grid 1km - United States of America \n", "6 Grid 1km - United States of America \n", "7 Grid 1km - United States of America \n", "8 Grid 1km - United States of America \n", ".. ... \n", "530 County - United States of America (2018) \n", "531 County - United States of America (2018) \n", "532 County - United States of America (2018) \n", "533 County - United States of America (2018) \n", "534 Core-based Statistical Area - United States of... \n", "\n", " geography_description temporal_aggregation \\\n", "4 Global grid at aprox. 1-kilometer resolution (... yearly \n", "5 Global grid at aprox. 1-kilometer resolution (... yearly \n", "6 Global grid at aprox. 1-kilometer resolution (... yearly \n", "7 Global grid at aprox. 1-kilometer resolution (... yearly \n", "8 Global grid at aprox. 1-kilometer resolution (... yearly \n", ".. ... ... \n", "530 Shoreline clipped TIGER/Line boundaries. More ... quarterly \n", "531 Shoreline clipped TIGER/Line boundaries. More ... quarterly \n", "532 Shoreline clipped TIGER/Line boundaries. More ... quarterly \n", "533 Shoreline clipped TIGER/Line boundaries. More ... quarterly \n", "534 Shoreline clipped TIGER/Line boundaries. More ... 5yrs \n", "\n", " time_coverage update_frequency is_public_data lang version \\\n", "4 [2020-01-01, 2021-01-01) yearly True eng 2020 \n", "5 [2019-01-01, 2020-01-01) yearly True eng 2019 \n", "6 [2018-01-01, 2019-01-01) yearly True eng 2018 \n", "7 [2017-01-01, 2018-01-01) yearly True eng 2017 \n", "8 [2016-01-01, 2017-01-01) yearly True eng 2016 \n", ".. ... ... ... ... ... \n", "530 [1992-01-01, 1992-04-01) quarterly True eng 1992q1 \n", "531 [2003-07-01, 2003-10-01) quarterly True eng 2003q3 \n", "532 [2002-07-01, 2002-10-01) quarterly True eng 2002q3 \n", "533 [2017-07-01, 2017-10-01) quarterly True eng 2017q3 \n", "534 [2009-01-01, 2014-01-01) yearly True eng 20092013 \n", "\n", " category_name provider_name \\\n", "4 Demographics WorldPop \n", "5 Demographics WorldPop \n", "6 Demographics WorldPop \n", "7 Demographics WorldPop \n", "8 Demographics WorldPop \n", ".. ... ... \n", "530 Demographics Bureau of Labor Statistics \n", "531 Demographics Bureau of Labor Statistics \n", "532 Demographics Bureau of Labor Statistics \n", "533 Demographics Bureau of Labor Statistics \n", "534 Demographics American Community Survey \n", "\n", " geography_id \\\n", "4 carto-do-public-data.worldpop.geography_usa_gr... \n", "5 carto-do-public-data.worldpop.geography_usa_gr... \n", "6 carto-do-public-data.worldpop.geography_usa_gr... \n", "7 carto-do-public-data.worldpop.geography_usa_gr... \n", "8 carto-do-public-data.worldpop.geography_usa_gr... \n", ".. ... \n", "530 carto-do-public-data.carto.geography_usa_count... \n", "531 carto-do-public-data.carto.geography_usa_count... \n", "532 carto-do-public-data.carto.geography_usa_count... \n", "533 carto-do-public-data.carto.geography_usa_count... \n", "534 carto-do-public-data.carto.geography_usa_cbsa_... \n", "\n", " id \n", "4 carto-do-public-data.worldpop.demographics_pop... \n", "5 carto-do-public-data.worldpop.demographics_pop... \n", "6 carto-do-public-data.worldpop.demographics_pop... \n", "7 carto-do-public-data.worldpop.demographics_pop... \n", "8 carto-do-public-data.worldpop.demographics_pop... \n", ".. ... \n", "530 carto-do-public-data.usa_bls.demographics_labo... \n", "531 carto-do-public-data.usa_bls.demographics_labo... \n", "532 carto-do-public-data.usa_bls.demographics_labo... \n", "533 carto-do-public-data.usa_bls.demographics_labo... \n", "534 carto-do-public-data.usa_acs.demographics_soci... \n", "\n", "[440 rows x 19 columns]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.data.observatory import Catalog\n", "\n", "datasets_df = Catalog().country('usa').category('demographics').datasets.to_dataframe()\n", "datasets_df[datasets_df['is_public_data'] == True]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Nice! Let's take the first one (`acs_sociodemogr_b758e778`) that has aggregated data from 2013 to 2018 and check which of its variables have data about the total population." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
        \n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
        slugnamedescriptiondb_typeagg_methodcolumn_namevariable_group_iddataset_idid
        2total_pop_3cf008b3Total PopulationTotal Population. The total number of all peop...FLOATSUMtotal_popNonecarto-do-public-data.usa_acs.demographics_soci...carto-do-public-data.usa_acs.demographics_soci...
        68income_per_capi_8a9352e0Income per capitaPer Capita Income in the past 12 Months. Per c...FLOATAVGincome_per_capitaNonecarto-do-public-data.usa_acs.demographics_soci...carto-do-public-data.usa_acs.demographics_soci...
        \n", "
        " ], "text/plain": [ " slug name \\\n", "2 total_pop_3cf008b3 Total Population \n", "68 income_per_capi_8a9352e0 Income per capita \n", "\n", " description db_type agg_method \\\n", "2 Total Population. The total number of all peop... FLOAT SUM \n", "68 Per Capita Income in the past 12 Months. Per c... FLOAT AVG \n", "\n", " column_name variable_group_id \\\n", "2 total_pop None \n", "68 income_per_capita None \n", "\n", " dataset_id \\\n", "2 carto-do-public-data.usa_acs.demographics_soci... \n", "68 carto-do-public-data.usa_acs.demographics_soci... \n", "\n", " id \n", "2 carto-do-public-data.usa_acs.demographics_soci... \n", "68 carto-do-public-data.usa_acs.demographics_soci... " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.data.observatory import Dataset\n", "\n", "dataset = Dataset.get('acs_sociodemogr_b758e778')\n", "variables_df = dataset.variables.to_dataframe()\n", "variables_df[variables_df['description'].str.contains('total population', case=False, na=False)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see the variable that contains the total population is the one with the slug `total_pop_3cf008b3`. Now we are ready to enrich our areas of influence with that variable." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
        \n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
        source_iddata_rangethe_geomtotal_pop
        00900MULTIPOLYGON (((-73.96743 40.67345, -73.96683 ...39565.324924
        11900MULTIPOLYGON (((-73.97035 40.57749, -73.97009 ...32622.336873
        22900MULTIPOLYGON (((-73.99781 40.62418, -73.99755 ...29685.123333
        33900MULTIPOLYGON (((-74.03678 40.63431, -74.03618 ...28988.433349
        44900MULTIPOLYGON (((-74.00451 40.59723, -74.00391 ...6086.711014
        \n", "
        " ], "text/plain": [ " source_id data_range the_geom \\\n", "0 0 900 MULTIPOLYGON (((-73.96743 40.67345, -73.96683 ... \n", "1 1 900 MULTIPOLYGON (((-73.97035 40.57749, -73.97009 ... \n", "2 2 900 MULTIPOLYGON (((-73.99781 40.62418, -73.99755 ... \n", "3 3 900 MULTIPOLYGON (((-74.03678 40.63431, -74.03618 ... \n", "4 4 900 MULTIPOLYGON (((-74.00451 40.59723, -74.00391 ... \n", "\n", " total_pop \n", "0 39565.324924 \n", "1 32622.336873 \n", "2 29685.123333 \n", "3 28988.433349 \n", "4 6086.711014 " ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.data.observatory import Variable\n", "from cartoframes.data.observatory import Enrichment\n", "\n", "variable = Variable.get('total_pop_3cf008b3')\n", "\n", "isochrones_gdf = Enrichment().enrich_polygons(isochrones_gdf, [variable])\n", "isochrones_gdf.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Great! Let's see the result on a map:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " None\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", " Static map image\n", " \n", " \n", "
        \n", "
        \n", "
        \n", " \n", " \n", "
        \n", "
        \n", " \n", "\n", "
        \n", " \n", " \n", " \n", " \n", " \n", "
        \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
        \n", " \n", " \n", " \n", " \n", " \n", "
        \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
        \n", " \n", " \n", "
        \n", "
        \n", "
        \n", "
        \n", " \n", "
        \n", "
        \n", "
        \n", "\n", " \n", "\n", "
        \n", "
        \n", " :\n", "
        \n", " \n", " \n", "
        \n", "
        \n", "\n", "
        \n", " StackTrace\n", "
          \n", "
          \n", "
          \n", "\n", "\n", "\n", "\n", "\n", "\">\n", "\n", "" ], "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes.viz import color_continuous_style\n", "\n", "Map([\n", " Layer(isochrones_gdf, color_continuous_style('total_pop'), title='Total Population'),\n", " Layer(stores_gdf, size_continuous_style('revenue', size_range=[10,40]), title='Annual Revenue ($)') \n", "])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "At this stage, we could say that the store on the right performs better than others because its area of influence is the one with the lowest population but the store is not the one with the lowest revenue. This insight will help us to focus on them in further analyses.\n", "\n", "To learn more about discovering and enriching your data with thousands of public and premium datasets, check out the [Data Observatory guide](/developers/cartoframes/guides/Data-Observatory)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Publish and share your results\n", "\n", "The final step of this workflow is to share this interactive map with your colleagues so they can explore the information on their own. Let's do it!\n", "\n", "First, let's add widgets to it so people are able to see some graphs of the information and filter it. To do this, we only have to add `default_widget=True` to the layers." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " None\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", " Static map image\n", " \n", " \n", " \n", "\n", "\n", " \n", "
          \n", "
          \n", "
          \n", " \n", " \n", "
          \n", "
          \n", " \n", "\n", "
          \n", " \n", " \n", " \n", " \n", " \n", "
          \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
          \n", " \n", " \n", " \n", " \n", " \n", "
          \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
          \n", " \n", " \n", "
          \n", "
          \n", "
          \n", "
          \n", " \n", "
          \n", "
          \n", "
          \n", "\n", " \n", "\n", "
          \n", "
          \n", " :\n", "
          \n", " \n", " \n", "
          \n", "
          \n", "\n", "
          \n", " StackTrace\n", "
            \n", "
            \n", "
            \n", "\n", "\n", "\n", "\n", "\n", "\">\n", "\n", "" ], "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "result_map = Map([\n", " Layer(\n", " isochrones_gdf,\n", " color_continuous_style('total_pop', stroke_width=0, opacity=0.7),\n", " title='Total Population',\n", " default_widget=True\n", " ),\n", " Layer(\n", " stores_gdf,\n", " size_continuous_style('revenue', size_range=[10,40], stroke_color='white'),\n", " title='Annual Revenue ($)',\n", " default_widget=True\n", " ) \n", "])\n", "result_map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Cool! Now that you have a small dashboard to play with, let's publish it on CARTO so you are able to share it with anyone. To do this, you just need to call the publish method from the Map class:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'id': '94e546ac-9aab-4857-a9b2-46874299c2ed',\n", " 'url': 'https://cartoframes-org.carto.com/u/cartoframes/kuviz/94e546ac-9aab-4857-a9b2-46874299c2ed',\n", " 'name': 'starbucks_analysis',\n", " 'privacy': 'public'}" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "result_map.publish('starbucks_analysis', password=None, if_exists='replace')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to improve the performance and reduce the size of your map, we recommend to upload the data to CARTO and use the table names in the layers instead. To upload your data, you just need to call `to_carto` with your GeoDataFrame:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Success! Data uploaded to table \"starbucks_stores\" correctly\n", "Success! Data uploaded to table \"starbucks_isochrones\" correctly\n" ] }, { "data": { "text/plain": [ "'starbucks_isochrones'" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cartoframes import to_carto\n", "\n", "to_carto(stores_gdf, 'starbucks_stores', if_exists='replace')\n", "to_carto(isochrones_gdf, 'starbucks_isochrones', if_exists='replace')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Congratulations!\n", "\n", "You have finished this guide and have a sense about how CARTOframes can speed up your workflow. To continue learning, you can check out other [Guides](/developers/cartoframes/guides), the [Reference](/developers/cartoframes/reference) to know everything about a class or a method or play with the [Examples](/developers/cartoframes/examples) to see CARTOframes in action." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.7" } }, "nbformat": 4, "nbformat_minor": 2 }