{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# This is a tutorial/demo on how to use the `Datamart` REST API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation\n",
    "\n",
    "This Jupyter notebook requires at least Python 3.3 with these packages installed:\n",
    "\n",
    "```\n",
    "pip install notebook\n",
    "pip install requests\n",
    "pip install pandas\n",
    "```\n",
    "\n",
    "To run change to the directory containing this notebook, and type\n",
    "\n",
    "```\n",
    "jupyter notebook\n",
    "```\n",
    "\n",
    "Then, open this page in the web browser: http://localhost:8888/notebooks/Datamart%20Data%20API%20Demo.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configuration\n",
    "\n",
    "By default the this notebook accesses the Datamart REST API server at ISI. Edit the cell below to choose a different server.\n",
    "\n",
    "To run you own server **locally** follow the instructions here: [README](README.md)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "## set datamart api url\n",
    "# The datamart server running at ISI\n",
    "# datamart_api_url = 'https://datamart:datamart-api-789@dsbox02.isi.edu:8888/datamart-api'\n",
    "\n",
    "# Datamart server running on localhost\n",
    "# datamart_api_url = 'http://localhost:14080'\n",
    "\n",
    "# Datamart server running on localhost in development mode\n",
    "datamart_api_url = 'http://localhost:5000'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import python modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from requests import get,post,put,delete\n",
    "import json\n",
    "import pandas as pd\n",
    "from io import StringIO\n",
    "from IPython.display import display, HTML"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get all datasets "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**GET `/metadata/datasets`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"name\": \"FSI dataset\",\n",
      "    \"description\": \"data downloaded from FSI\",\n",
      "    \"url\": \"https://fragilestatesindex.org\",\n",
      "    \"dataset_id\": \"FSI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"OECD dataset\",\n",
      "    \"description\": \"data downloaded from OECD\",\n",
      "    \"url\": \"https://data.oecd.org\",\n",
      "    \"dataset_id\": \"OECD\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"UAZ Indicators\",\n",
      "    \"description\": \"Collection of indicators, including indicators from FAO, WDI, FEWSNET, CLiMIS, UNICEF, ieconomics.com, UNHCR, DSSAT, WHO, IMF, WHP, ACLDE, World Bank and IOM-DTM\",\n",
      "    \"url\": \"https://github.com/ml4ai/delphi\",\n",
      "    \"dataset_id\": \"UAZ\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"WGI dataset\",\n",
      "    \"description\": \"Worldwide Governance Indicators\",\n",
      "    \"url\": \"https://databank.worldbank.org/source/worldwide-governance-indicators\",\n",
      "    \"dataset_id\": \"WGI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"WDI dataset\",\n",
      "    \"description\": \"World Development Indicators\",\n",
      "    \"url\": \"https://databank.worldbank.org/source/world-development-indicators\",\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Corruption Perceptions Index\",\n",
      "    \"description\": \"Transparency International  Corruption Perceptions Index The CPI scores and ranks countries/territories based on how corrupt a country\\u2019s public sector is perceived to be by experts and business executives. It is a composite index, a combination of 13 surveys and assessments of corruption, collected by a variety of reputable institutions. The CPI is the most widely used indicator of corruption worldwide.\",\n",
      "    \"url\": \"https://www.transparency.org/\",\n",
      "    \"dataset_id\": \"TICPI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"SIPRI Military Expenditure\",\n",
      "    \"description\": \"Military expenditure by country, in millions of US$ at current prices and exchange rates, 1949-2018 - SIPRI 2019\",\n",
      "    \"url\": \"https://sipri.org/databases/milex\",\n",
      "    \"dataset_id\": \"SIPRI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"economic fitness dataset\",\n",
      "    \"description\": \"EconomicFitness\",\n",
      "    \"url\": \"https://databank.banquemondiale.org/source/economic-fitness\",\n",
      "    \"dataset_id\": \"EconomicFitness\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Agricultural Market Information System (AMIS)\",\n",
      "    \"description\": \"The Agricultural Market Information System (AMIS) is an inter-agency platform to enhance food market transparency and policy response for food security. It was launched in 2011 by the G20 Ministers of Agriculture following the global food price hikes in 2007/08 and 2010. Bringing together the principal trading countries of agricultural commodities, AMIS assesses global food supplies (focusing on wheat, maize, rice and soybeans) and provides a platform to coordinate policy action in times of market uncertainty.\",\n",
      "    \"url\": \"http://www.amis-outlook.org\",\n",
      "    \"dataset_id\": \"AMIS\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"test test test\",\n",
      "    \"description\": \"testy test\",\n",
      "    \"url\": \"https://test.com\",\n",
      "    \"dataset_id\": \"TEST000\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"World Press Freedom Index\",\n",
      "    \"description\": \"Published every year since 2002 by Reporters Without Borders (RSF), the World Press Freedom Index is an important advocacy tool based on the principle of emulation between states. The Index ranks 180 countries and regions\\u00a0according to the level of freedom available to journalists.\",\n",
      "    \"url\": \"https://rsf.org/en\",\n",
      "    \"dataset_id\": \"WPFI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Poverty Rate Global DP\",\n",
      "    \"description\": \"Poverty Rate Global DP\",\n",
      "    \"url\": \"http://url\",\n",
      "    \"dataset_id\": \"DPPoverty\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/datasets')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As of June 25, 2020 there are 11 datasets in the database. More datasets will be added as they are processed. \n",
    "\n",
    "We can also get metadata about one dataset using the `dataset_id`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get metadata about one dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**GET `/metadata/datasets/{dataset_id}`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"name\": \"WDI dataset\",\n",
      "    \"description\": \"World Development Indicators\",\n",
      "    \"url\": \"https://databank.worldbank.org/source/world-development-indicators\",\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/datasets/WDI')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get all variables in a dataset "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**GET `/metadata/datasets/{dataset_id}/variables`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"name\": \"_2005 PPP conversion factor, GDP (LCU per international $)\",\n",
      "    \"variable_id\": \"_2005_ppp_conversion_factor_gdp_lcu_per_international\",\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"_2005 PPP conversion factor, private consumption (LCU per international $)\",\n",
      "    \"variable_id\": \"_2005_ppp_conversion_factor_private_consumption_lcu_per_international\",\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Access to clean fuels and technologies for cooking (% of population)\",\n",
      "    \"variable_id\": \"access_to_clean_fuels_and_technologies_for_cooking_of_population\",\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Access to electricity (% of population)\",\n",
      "    \"variable_id\": \"access_to_electricity_of_population\",\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/datasets/WDI/variables')\n",
    "print(json.dumps(response.json()[:4], indent=2)) # printing only 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of variables in dataset: WDI is 1429\n"
     ]
    }
   ],
   "source": [
    "print('Total number of variables in dataset: {} is {}'.format('WDI', len(response.json())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get metadata about one variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**GET `/metadata/datasets/{dataset_id}/variables/{variable_id}`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"name\": \"Access to electricity (% of population)\",\n",
      "  \"variable_id\": \"access_to_electricity_of_population\",\n",
      "  \"dataset_id\": \"WDI\",\n",
      "  \"description\": \"Access to electricity (% of population) in WDI\",\n",
      "  \"corresponds_to_property\": \"PWDI-005\",\n",
      "  \"qualifier\": [\n",
      "    {\n",
      "      \"identifier\": \"P585\",\n",
      "      \"name\": \"point in time\"\n",
      "    },\n",
      "    {\n",
      "      \"identifier\": \"P248\",\n",
      "      \"name\": \"stated in\"\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/datasets/WDI/variables/access_to_electricity_of_population')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find a variable using keyword search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**GET `/metadata/variables?keyword={keyword}`**\n",
    "\n",
    "Query for datasets related to: **road**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"variable_id\": \"road_fatalities\",\n",
      "    \"name\": \" Road Fatalities\",\n",
      "    \"rank\": 0.0759909,\n",
      "    \"dataset_id\": \"OECD\"\n",
      "  },\n",
      "  {\n",
      "    \"variable_id\": \"mortality_caused_by_road_traffic_injury_per_100_000_people\",\n",
      "    \"name\": \" Mortality caused by road traffic injury (per 100,000 people)\",\n",
      "    \"rank\": 0.0759909,\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  },\n",
      "  {\n",
      "    \"variable_id\": \"VUAZ-8054\",\n",
      "    \"name\": \" WDI: Mortality caused by road traffic injury[per 100,000 people]\",\n",
      "    \"rank\": 0.0607927,\n",
      "    \"dataset_id\": \"UAZ\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/variables?keyword=road')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Query datasets related to: **road AND fatalities**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"variable_id\": \"road_fatalities\",\n",
      "    \"name\": \" Road Fatalities\",\n",
      "    \"rank\": 0.334428,\n",
      "    \"dataset_id\": \"OECD\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/variables?keyword=road fatalities')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Query datasets related to: **road OR fatalities**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"variable_id\": \"road_fatalities\",\n",
      "    \"name\": \" Road Fatalities\",\n",
      "    \"rank\": 0.0759909,\n",
      "    \"dataset_id\": \"OECD\"\n",
      "  },\n",
      "  {\n",
      "    \"variable_id\": \"mortality_caused_by_road_traffic_injury_per_100_000_people\",\n",
      "    \"name\": \" Mortality caused by road traffic injury (per 100,000 people)\",\n",
      "    \"rank\": 0.0379954,\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  },\n",
      "  {\n",
      "    \"variable_id\": \"VUAZ-8054\",\n",
      "    \"name\": \" WDI: Mortality caused by road traffic injury[per 100,000 people]\",\n",
      "    \"rank\": 0.0303964,\n",
      "    \"dataset_id\": \"UAZ\"\n",
      "  },\n",
      "  {\n",
      "    \"variable_id\": \"VUAZ-8136\",\n",
      "    \"name\": \" Conflict fatalities[number of cases]\",\n",
      "    \"rank\": 0.0303964,\n",
      "    \"dataset_id\": \"UAZ\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/variables?keyword=road&keyword=fatalities')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get time series data for a variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**GET `/datasets/{dataset_id}/variables/{variable_id}`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>dataset_id</th>\n",
       "      <th>variable_id</th>\n",
       "      <th>variable</th>\n",
       "      <th>main_subject</th>\n",
       "      <th>main_subject_id</th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>time</th>\n",
       "      <th>time_precision</th>\n",
       "      <th>country</th>\n",
       "      <th>coordinate</th>\n",
       "      <th>stated_in</th>\n",
       "      <th>stated_in_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>73.600000</td>\n",
       "      <td></td>\n",
       "      <td>2000-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>76.344460</td>\n",
       "      <td></td>\n",
       "      <td>2001-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>77.307663</td>\n",
       "      <td></td>\n",
       "      <td>2002-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>78.251656</td>\n",
       "      <td></td>\n",
       "      <td>2003-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>79.171516</td>\n",
       "      <td></td>\n",
       "      <td>2004-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>81.600000</td>\n",
       "      <td></td>\n",
       "      <td>2005-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>80.943794</td>\n",
       "      <td></td>\n",
       "      <td>2006-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>81.820259</td>\n",
       "      <td></td>\n",
       "      <td>2007-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>82.708366</td>\n",
       "      <td></td>\n",
       "      <td>2008-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>83.621689</td>\n",
       "      <td></td>\n",
       "      <td>2009-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>92.389572</td>\n",
       "      <td></td>\n",
       "      <td>2010-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>90.631691</td>\n",
       "      <td></td>\n",
       "      <td>2011-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>89.300000</td>\n",
       "      <td></td>\n",
       "      <td>2012-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>86.400000</td>\n",
       "      <td></td>\n",
       "      <td>2013-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>88.803612</td>\n",
       "      <td></td>\n",
       "      <td>2014-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>89.926506</td>\n",
       "      <td></td>\n",
       "      <td>2015-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>91.058128</td>\n",
       "      <td></td>\n",
       "      <td>2016-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>92.191200</td>\n",
       "      <td></td>\n",
       "      <td>2017-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>The Gambia</td>\n",
       "      <td>Q1005</td>\n",
       "      <td>17.700000</td>\n",
       "      <td></td>\n",
       "      <td>1993-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>The Gambia</td>\n",
       "      <td>POINT(-15.5, 13.5)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>The Gambia</td>\n",
       "      <td>Q1005</td>\n",
       "      <td>18.708818</td>\n",
       "      <td></td>\n",
       "      <td>1994-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>The Gambia</td>\n",
       "      <td>POINT(-15.5, 13.5)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/datasets/WDI/variables/access_to_electricity_of_population')\n",
    "df = pd.read_csv(StringIO(response.text))\n",
    "display(HTML(df.fillna('').head(20).to_html(index=False)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get time series data for a variable for a country"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**GET `/datasets/{dataset_id}/variables/{variable_id}?country={country}`**\n",
    "\n",
    "Get data for **Gabon**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>dataset_id</th>\n",
       "      <th>variable_id</th>\n",
       "      <th>variable</th>\n",
       "      <th>main_subject</th>\n",
       "      <th>main_subject_id</th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>time</th>\n",
       "      <th>time_precision</th>\n",
       "      <th>country</th>\n",
       "      <th>coordinate</th>\n",
       "      <th>stated_in</th>\n",
       "      <th>stated_in_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>73.600000</td>\n",
       "      <td></td>\n",
       "      <td>2000-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>76.344460</td>\n",
       "      <td></td>\n",
       "      <td>2001-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>77.307663</td>\n",
       "      <td></td>\n",
       "      <td>2002-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>78.251656</td>\n",
       "      <td></td>\n",
       "      <td>2003-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>79.171516</td>\n",
       "      <td></td>\n",
       "      <td>2004-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>81.600000</td>\n",
       "      <td></td>\n",
       "      <td>2005-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>80.943794</td>\n",
       "      <td></td>\n",
       "      <td>2006-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>81.820259</td>\n",
       "      <td></td>\n",
       "      <td>2007-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>82.708366</td>\n",
       "      <td></td>\n",
       "      <td>2008-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>83.621689</td>\n",
       "      <td></td>\n",
       "      <td>2009-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>92.389572</td>\n",
       "      <td></td>\n",
       "      <td>2010-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>90.631691</td>\n",
       "      <td></td>\n",
       "      <td>2011-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>89.300000</td>\n",
       "      <td></td>\n",
       "      <td>2012-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>86.400000</td>\n",
       "      <td></td>\n",
       "      <td>2013-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>88.803612</td>\n",
       "      <td></td>\n",
       "      <td>2014-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>89.926506</td>\n",
       "      <td></td>\n",
       "      <td>2015-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>91.058128</td>\n",
       "      <td></td>\n",
       "      <td>2016-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>92.191200</td>\n",
       "      <td></td>\n",
       "      <td>2017-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/datasets/WDI/variables/access_to_electricity_of_population?country=Gabon')\n",
    "df = pd.read_csv(StringIO(response.text))\n",
    "display(HTML(df.fillna('').to_html(index=False)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get data for **Gabon OR Guinea**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>dataset_id</th>\n",
       "      <th>variable_id</th>\n",
       "      <th>variable</th>\n",
       "      <th>main_subject</th>\n",
       "      <th>main_subject_id</th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>time</th>\n",
       "      <th>time_precision</th>\n",
       "      <th>country</th>\n",
       "      <th>coordinate</th>\n",
       "      <th>stated_in</th>\n",
       "      <th>stated_in_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>73.600000</td>\n",
       "      <td></td>\n",
       "      <td>2000-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>76.344460</td>\n",
       "      <td></td>\n",
       "      <td>2001-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>77.307663</td>\n",
       "      <td></td>\n",
       "      <td>2002-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>78.251656</td>\n",
       "      <td></td>\n",
       "      <td>2003-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>79.171516</td>\n",
       "      <td></td>\n",
       "      <td>2004-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>81.600000</td>\n",
       "      <td></td>\n",
       "      <td>2005-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>80.943794</td>\n",
       "      <td></td>\n",
       "      <td>2006-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>81.820259</td>\n",
       "      <td></td>\n",
       "      <td>2007-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>82.708366</td>\n",
       "      <td></td>\n",
       "      <td>2008-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>83.621689</td>\n",
       "      <td></td>\n",
       "      <td>2009-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>92.389572</td>\n",
       "      <td></td>\n",
       "      <td>2010-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>90.631691</td>\n",
       "      <td></td>\n",
       "      <td>2011-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>89.300000</td>\n",
       "      <td></td>\n",
       "      <td>2012-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>86.400000</td>\n",
       "      <td></td>\n",
       "      <td>2013-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>88.803612</td>\n",
       "      <td></td>\n",
       "      <td>2014-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>89.926506</td>\n",
       "      <td></td>\n",
       "      <td>2015-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>91.058128</td>\n",
       "      <td></td>\n",
       "      <td>2016-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>Q1000</td>\n",
       "      <td>92.191200</td>\n",
       "      <td></td>\n",
       "      <td>2017-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Gabon</td>\n",
       "      <td>POINT(11.5, -0.68333055555556)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>16.400000</td>\n",
       "      <td></td>\n",
       "      <td>1999-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>16.503561</td>\n",
       "      <td></td>\n",
       "      <td>2000-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>17.478863</td>\n",
       "      <td></td>\n",
       "      <td>2001-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>18.439884</td>\n",
       "      <td></td>\n",
       "      <td>2002-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>19.381701</td>\n",
       "      <td></td>\n",
       "      <td>2003-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>20.299383</td>\n",
       "      <td></td>\n",
       "      <td>2004-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>20.200000</td>\n",
       "      <td></td>\n",
       "      <td>2005-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>22.067295</td>\n",
       "      <td></td>\n",
       "      <td>2006-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>22.941578</td>\n",
       "      <td></td>\n",
       "      <td>2007-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>23.827507</td>\n",
       "      <td></td>\n",
       "      <td>2008-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>24.738651</td>\n",
       "      <td></td>\n",
       "      <td>2009-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>25.688576</td>\n",
       "      <td></td>\n",
       "      <td>2010-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>26.687136</td>\n",
       "      <td></td>\n",
       "      <td>2011-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>26.200000</td>\n",
       "      <td></td>\n",
       "      <td>2012-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>28.806410</td>\n",
       "      <td></td>\n",
       "      <td>2013-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>29.909674</td>\n",
       "      <td></td>\n",
       "      <td>2014-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>31.618164</td>\n",
       "      <td></td>\n",
       "      <td>2015-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>33.500000</td>\n",
       "      <td></td>\n",
       "      <td>2016-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>WDI</td>\n",
       "      <td>access_to_electricity_of_population</td>\n",
       "      <td>Access to electricity (% of population)</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>Q1006</td>\n",
       "      <td>35.441216</td>\n",
       "      <td></td>\n",
       "      <td>2017-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Guinea</td>\n",
       "      <td>POINT(-11.0, 10.0)</td>\n",
       "      <td>WDI</td>\n",
       "      <td>Q8035640</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/datasets/WDI/variables/access_to_electricity_of_population?country=Gabon&country=Guinea')\n",
    "df = pd.read_csv(StringIO(response.text))\n",
    "display(HTML(df.fillna('').to_html(index=False)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a new dataset\n",
    "\n",
    "**NOTE: If the following POST methods have already been ran against the Datamart server, then server will respond with error messages.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**POST `/metadata/datasets`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a new dataset\n",
    "test_dataset = {\n",
    "    \"name\": \"Test Dataset 01\",\n",
    "    \"dataset_id\": \"TEST01\",\n",
    "    \"description\": \"Test Dataset 01\",\n",
    "    \"url\": \"http://test01.com/test\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"name\": \"Test Dataset 01\",\n",
      "  \"description\": \"Test Dataset 01\",\n",
      "  \"url\": \"http://test01.com/test\",\n",
      "  \"dataset_id\": \"TEST01\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# post it to the API\n",
    "td_response = post(f'{datamart_api_url}/metadata/datasets', json=test_dataset)\n",
    "print(json.dumps(td_response.json(), indent=2))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE: If the above POST method has already been ran against this Datamart server, then server will respond with:**\n",
    "\n",
    "```\n",
    "{\n",
    "  \"Error\": \"Dataset identifier TEST01 has already been used\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieve all datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"name\": \"FSI dataset\",\n",
      "    \"description\": \"data downloaded from FSI\",\n",
      "    \"url\": \"https://fragilestatesindex.org\",\n",
      "    \"dataset_id\": \"FSI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"OECD dataset\",\n",
      "    \"description\": \"data downloaded from OECD\",\n",
      "    \"url\": \"https://data.oecd.org\",\n",
      "    \"dataset_id\": \"OECD\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"UAZ Indicators\",\n",
      "    \"description\": \"Collection of indicators, including indicators from FAO, WDI, FEWSNET, CLiMIS, UNICEF, ieconomics.com, UNHCR, DSSAT, WHO, IMF, WHP, ACLDE, World Bank and IOM-DTM\",\n",
      "    \"url\": \"https://github.com/ml4ai/delphi\",\n",
      "    \"dataset_id\": \"UAZ\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"WGI dataset\",\n",
      "    \"description\": \"Worldwide Governance Indicators\",\n",
      "    \"url\": \"https://databank.worldbank.org/source/worldwide-governance-indicators\",\n",
      "    \"dataset_id\": \"WGI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"WDI dataset\",\n",
      "    \"description\": \"World Development Indicators\",\n",
      "    \"url\": \"https://databank.worldbank.org/source/world-development-indicators\",\n",
      "    \"dataset_id\": \"WDI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Corruption Perceptions Index\",\n",
      "    \"description\": \"Transparency International  Corruption Perceptions Index The CPI scores and ranks countries/territories based on how corrupt a country\\u2019s public sector is perceived to be by experts and business executives. It is a composite index, a combination of 13 surveys and assessments of corruption, collected by a variety of reputable institutions. The CPI is the most widely used indicator of corruption worldwide.\",\n",
      "    \"url\": \"https://www.transparency.org/\",\n",
      "    \"dataset_id\": \"TICPI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"SIPRI Military Expenditure\",\n",
      "    \"description\": \"Military expenditure by country, in millions of US$ at current prices and exchange rates, 1949-2018 - SIPRI 2019\",\n",
      "    \"url\": \"https://sipri.org/databases/milex\",\n",
      "    \"dataset_id\": \"SIPRI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"economic fitness dataset\",\n",
      "    \"description\": \"EconomicFitness\",\n",
      "    \"url\": \"https://databank.banquemondiale.org/source/economic-fitness\",\n",
      "    \"dataset_id\": \"EconomicFitness\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Agricultural Market Information System (AMIS)\",\n",
      "    \"description\": \"The Agricultural Market Information System (AMIS) is an inter-agency platform to enhance food market transparency and policy response for food security. It was launched in 2011 by the G20 Ministers of Agriculture following the global food price hikes in 2007/08 and 2010. Bringing together the principal trading countries of agricultural commodities, AMIS assesses global food supplies (focusing on wheat, maize, rice and soybeans) and provides a platform to coordinate policy action in times of market uncertainty.\",\n",
      "    \"url\": \"http://www.amis-outlook.org\",\n",
      "    \"dataset_id\": \"AMIS\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"test test test\",\n",
      "    \"description\": \"testy test\",\n",
      "    \"url\": \"https://test.com\",\n",
      "    \"dataset_id\": \"TEST000\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"World Press Freedom Index\",\n",
      "    \"description\": \"Published every year since 2002 by Reporters Without Borders (RSF), the World Press Freedom Index is an important advocacy tool based on the principle of emulation between states. The Index ranks 180 countries and regions\\u00a0according to the level of freedom available to journalists.\",\n",
      "    \"url\": \"https://rsf.org/en\",\n",
      "    \"dataset_id\": \"WPFI\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Poverty Rate Global DP\",\n",
      "    \"description\": \"Poverty Rate Global DP\",\n",
      "    \"url\": \"http://url\",\n",
      "    \"dataset_id\": \"DPPoverty\"\n",
      "  },\n",
      "  {\n",
      "    \"name\": \"Test Dataset 01\",\n",
      "    \"description\": \"Test Dataset 01\",\n",
      "    \"url\": \"http://test01.com/test\",\n",
      "    \"dataset_id\": \"TEST01\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/datasets')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The newly created dataset `TEST01` is returned"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a variable in the dataset `TEST01`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**POST `/metadata/datasets/{dataset_id}/variables`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a new variable\n",
    "test_variable = {\n",
    "    \"name\": \"test variable for test dataset\",\n",
    "    \"variable_id\": \"TEST01-01\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"name\": \"test variable for test dataset\",\n",
      "  \"variable_id\": \"TEST01-01\",\n",
      "  \"dataset_id\": \"TEST01\",\n",
      "  \"corresponds_to_property\": \"PTEST01-TEST01-01\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "tv_response = post(f'{datamart_api_url}/metadata/datasets/TEST01/variables', json=test_variable)\n",
    "print(json.dumps(tv_response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE: If the above POST method has already been ran against this Datamart server, then server will respond with:**\n",
    "\n",
    "```\n",
    "{\n",
    "  \"Error\": \"Variable TEST01-01 has already been defined in dataset TEST01\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieve all variables for the dataset `TEST01`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"name\": \"test variable for test dataset\",\n",
      "    \"variable_id\": \"TEST01-01\",\n",
      "    \"dataset_id\": \"TEST01\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/metadata/datasets/TEST01/variables')\n",
    "print(json.dumps(response.json(), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The variable `TEST01-01` is created in the dataset `TEST01`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload data to a variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets upload some data to the dataset: TEST01 and the variable TEST01-01. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**PUT `/datasets/{dataset_id}/variables/{variable_id}`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "def upload_data(file_path, url):\n",
    "    file_name = os.path.basename(file_path)\n",
    "    files = {\n",
    "        'file': (file_name, open(file_path, mode='rb'), 'application/octet-stream')\n",
    "    }\n",
    "    response = put(url, files=files)\n",
    "    if response.status_code == 400:\n",
    "        print(json.dumps(response.json(), indent=2))\n",
    "    else:\n",
    "        print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The upload data API validates the input file.\n",
    "\n",
    "In the example below, the file `test_sample_missing_header.csv` is missing a required column `main_subject`.\n",
    "\n",
    "All required columns are:\n",
    "- main_subject\n",
    "- value\n",
    "- time\n",
    "- time_precision\n",
    "- country"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>time</th>\n",
       "      <th>time_precision</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.8</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>2021-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>belllgium</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.9</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>2022-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>bellgium</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   value       value_unit                  time time_precision    country\n",
       "0    1.8  Annual growth %  2021-01-01T00:00:00Z           year  belllgium\n",
       "1    1.9  Annual growth %  2022-01-01T00:00:00Z           year   bellgium"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('test/test_data/test_sample_missing_header.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets try to upload this file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"Error\": \"Missing required column: 'main_subject'\",\n",
      "    \"Line Number\": 1,\n",
      "    \"Column\": \"main_subject\",\n",
      "    \"Description\": \"The uploaded file is missing a required column: main_subject. Please add the missing column and upload again.\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "url = f'{datamart_api_url}/datasets/TEST01/variables/TEST01-01'\n",
    "file_path = 'test/test_data/test_sample_missing_header.csv'\n",
    "upload_data(file_path, url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, the API throws an error about missing column `main_subject`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the example below, we have the file`test_sample_invalid.csv`\n",
    "This file contains some invalid values in the required columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>main_subject</th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>time</th>\n",
       "      <th>time_precision</th>\n",
       "      <th>country</th>\n",
       "      <th>source</th>\n",
       "      <th>dataset_id</th>\n",
       "      <th>variable_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>shdjshduihskdj</td>\n",
       "      <td>fifty</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>20-01-01T00:00:00Z</td>\n",
       "      <td>blah</td>\n",
       "      <td>belllgium</td>\n",
       "      <td>OECD</td>\n",
       "      <td>FAO</td>\n",
       "      <td>fake_gdp_growth</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>bellgium</td>\n",
       "      <td>1.9</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>2022-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>shdjshduihskdj</td>\n",
       "      <td>OECD</td>\n",
       "      <td>OECD</td>\n",
       "      <td>real_gdp_growth</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     main_subject  value       value_unit                  time  \\\n",
       "0  shdjshduihskdj  fifty  Annual growth %    20-01-01T00:00:00Z   \n",
       "1        bellgium    1.9  Annual growth %  2022-01-01T00:00:00Z   \n",
       "\n",
       "  time_precision         country source dataset_id      variable_id  \n",
       "0           blah       belllgium   OECD        FAO  fake_gdp_growth  \n",
       "1           year  shdjshduihskdj   OECD       OECD  real_gdp_growth  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('test/test_data/test_sample_invalid.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets try to upload this file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"Error\": \"Value Error: 'fifty'\",\n",
      "    \"Line Number\": 2,\n",
      "    \"Column\": \"value\",\n",
      "    \"Description\": \"'fifty' is not a valid number\"\n",
      "  },\n",
      "  {\n",
      "    \"Error\": \"Illegal precision value: 'blah'\",\n",
      "    \"Line Number\": 2,\n",
      "    \"Column\": \"time_precision\",\n",
      "    \"Description\": \"Legal precision values are: 'billion years,hundred million years,million years,hundred thousand years,ten thousand years,millennium,century,decade,year,month,day,hour,minute,second'\"\n",
      "  },\n",
      "  {\n",
      "    \"Error\": \"Could not wikify: 'shdjshduihskdj'\",\n",
      "    \"Line Number\": 2,\n",
      "    \"Column\": \"main_subject\",\n",
      "    \"Description\": \"Could not find a Wikidata Qnode for the main subject: 'shdjshduihskdj.' Please check for spelling mistakes in the country name.\"\n",
      "  },\n",
      "  {\n",
      "    \"Error\": \"Dataset ID in the file: 'FAO' is not same as Dataset ID in the url : 'TEST01'\",\n",
      "    \"Line Number\": 2,\n",
      "    \"Column\": \"dataset_id\",\n",
      "    \"Description\": \"Dataset IDs in the input file should match the Dataset Id in the API url\"\n",
      "  },\n",
      "  {\n",
      "    \"Error\": \"Variable ID in the file: 'fake_gdp_growth' is not same as Variable ID in the url : 'TEST01-01'\",\n",
      "    \"Line Number\": 2,\n",
      "    \"Column\": \"variable_id\",\n",
      "    \"Description\": \"Variable IDs in the input file should match the Variable Id in the API url\"\n",
      "  },\n",
      "  {\n",
      "    \"Error\": \"Invalid datetime format: '20-01-01T00:00:00Z'\",\n",
      "    \"Line Number\": 2,\n",
      "    \"Column\": \"time\",\n",
      "    \"Description\": \"Invalid format to specify time. Valid format: '%Y-%m-%dT%H:%M:%SZ' Explanation:  %Y - Year with century as a decimal number (2010, 2020 etc). %m - Month as a zero-padded decimal number(01, 02,..,12). %d - Day of the month as a zero-padded decimal number. (01,02,..,31). %H - Hour (24-hour clock) as a zero-padded decimal number. (00, 01,..,23). %M - Minute as a zero-padded decimal number.(00, 01,...,59). %S - Second as a zero-padded decimal number.(00, 01,...,59). A valid date: '2020-02-27T13:45:44Z'\"\n",
      "  },\n",
      "  {\n",
      "    \"Error\": \"Could not wikify: 'shdjshduihskdj'\",\n",
      "    \"Line Number\": 3,\n",
      "    \"Column\": \"country\",\n",
      "    \"Description\": \"Could not find a Wikidata Qnode for the country:  'shdjshduihskdj'. Please check for spelling mistakes in the country name.\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "url = f'{datamart_api_url}/datasets/TEST01/variables/TEST01-01'\n",
    "file_path = 'test/test_data/test_sample_invalid.csv'\n",
    "upload_data(file_path, url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The API will list all the errors in the file, which have to be fixed first before it can be uploaded!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will upload the contents of the file in `test_data/test_sample.csv`, which is a `valid` file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>main_subject</th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>time</th>\n",
       "      <th>time_precision</th>\n",
       "      <th>country</th>\n",
       "      <th>source</th>\n",
       "      <th>dataset_id</th>\n",
       "      <th>variable_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>belllgium</td>\n",
       "      <td>1.8</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>2019-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>belllgium</td>\n",
       "      <td>OECD</td>\n",
       "      <td>TEST01</td>\n",
       "      <td>TEST01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>bellgium</td>\n",
       "      <td>1.9</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>2020-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>bellgium</td>\n",
       "      <td>OECD</td>\n",
       "      <td>TEST01</td>\n",
       "      <td>TEST01-01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  main_subject  value       value_unit                  time time_precision  \\\n",
       "0    belllgium    1.8  Annual growth %  2019-01-01T00:00:00Z           year   \n",
       "1     bellgium    1.9  Annual growth %  2020-01-01T00:00:00Z           year   \n",
       "\n",
       "     country source dataset_id variable_id  \n",
       "0  belllgium   OECD     TEST01   TEST01-01  \n",
       "1   bellgium   OECD     TEST01   TEST01-01  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('test/test_data/test_sample.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 rows imported!\n"
     ]
    }
   ],
   "source": [
    "url = f'{datamart_api_url}/datasets/TEST01/variables/TEST01-01'\n",
    "file_path = 'test/test_data/test_sample.csv'\n",
    "upload_data(file_path, url)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the data for the variable `TEST01-01` to check if the was added"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dataset_id</th>\n",
       "      <th>variable_id</th>\n",
       "      <th>variable</th>\n",
       "      <th>main_subject</th>\n",
       "      <th>main_subject_id</th>\n",
       "      <th>value</th>\n",
       "      <th>value_unit</th>\n",
       "      <th>time</th>\n",
       "      <th>time_precision</th>\n",
       "      <th>country</th>\n",
       "      <th>coordinate</th>\n",
       "      <th>stated_in</th>\n",
       "      <th>stated_in_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>TEST01</td>\n",
       "      <td>TEST01-01</td>\n",
       "      <td>test variable for test dataset</td>\n",
       "      <td>Belgium</td>\n",
       "      <td>Q31</td>\n",
       "      <td>1.8</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>2019-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Belgium</td>\n",
       "      <td>POINT(4.6680555555556, 50.641111111111)</td>\n",
       "      <td>OECD</td>\n",
       "      <td>QTEST01Source-0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>TEST01</td>\n",
       "      <td>TEST01-01</td>\n",
       "      <td>test variable for test dataset</td>\n",
       "      <td>Belgium</td>\n",
       "      <td>Q31</td>\n",
       "      <td>1.9</td>\n",
       "      <td>Annual growth %</td>\n",
       "      <td>2020-01-01T00:00:00Z</td>\n",
       "      <td>year</td>\n",
       "      <td>Belgium</td>\n",
       "      <td>POINT(4.6680555555556, 50.641111111111)</td>\n",
       "      <td>OECD</td>\n",
       "      <td>QTEST01Source-0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response = get(f'{datamart_api_url}/datasets/TEST01/variables/TEST01-01')\n",
    "df = pd.read_csv(StringIO(response.text))\n",
    "display(HTML(df.to_html()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Success! The 2 rows from 2019 and 2020 were added "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Delete the rows added to the dataset for another run of this Jupyter Notebook**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = delete(f'{datamart_api_url}/datasets/TEST01/variables/TEST01-01')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The data has been deleted**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}