{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "986b02e2-779c-4c41-b000-00b83c616e0e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "repo & folder name: GreenhouseData\n",
      "Requirements imported!\n",
      "Imported COUNTRY_DICT & CODE_GENERATOR_ISO3\n"
     ]
    }
   ],
   "source": [
    "%run Requirements.ipynb\n",
    "%run Country_ISO_Codes.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c39cc569-5b18-447e-a47c-3c146324fa19",
   "metadata": {},
   "source": [
    "# Sources used in Greenhouse Data\n",
    "All of these inventories have been homogenized (as much as possible/recommended) by making sure they all include the following characteristics:\n",
    "- Country ISO codes (ISO-3). This is useful to easily create emission figures for all countries included in an inventory. \n",
    "- Emission units in CO2 or CO2-equivalent (CO2eq). Some inventories give native units of gases (for example kt of CH4), so columns were added for easy conversion using GWPs.\n",
    "\n",
    "Furthermore, thank you to the following people:\n",
    "- Mauricio Foronda, for all the coding help\n",
    "- Nicolás Guarín-Zapata (nicoguaro in github) for the neon style in matplotlib, and for inspiring my NEONIZE() function.\n",
    "- My thesis supervisors: Sonia Seneviratne, Anthony Patt, Jonas Schwabb\n",
    "- Open Climate Data, from which I took the UNFCCC submission inventories.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebc20a0c-1c28-4bae-b1a0-0be0adf5c873",
   "metadata": {},
   "source": [
    "# BP \n",
    "\n",
    "INFO:\n",
    " - BP. (2021). Methodology for calculating CO 2 emissions from energy use. https://www.ipcc-nggip.iges.or.jp/public/2006gl/vol2.html\n",
    " - Downloaded from \"https://www.bp.com/content/dam/bp/business-sites/en/global/corporate/xlsx/energy-economics/statistical-review/bp-stats-review-2021-all-data.xlsx\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "df13b7cb-0a97-45c1-bb5e-6e13c9e7f54c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> BP <<<<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 23 Missing codes (NaN) in this df\n",
      " ['Total North America', 'Central America', 'Other Caribbean', 'Other South America', 'Total S. & Cent. America', 'Other Europe', 'Total Europe', 'Other CIS', 'Total CIS', 'Other Middle East', 'Total Middle East', 'Eastern Africa', 'Middle Africa', 'Western Africa', 'Other Northern Africa', 'Other Southern Africa', 'Total Africa', 'Other Asia Pacific', 'Total Asia Pacific', 'Total World', 'of which: OECD', '                 Non-OECD', '                 European Union #']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>unit</th>\n",
       "      <th>1965</th>\n",
       "      <th>1966</th>\n",
       "      <th>1967</th>\n",
       "      <th>1968</th>\n",
       "      <th>1969</th>\n",
       "      <th>1970</th>\n",
       "      <th>1971</th>\n",
       "      <th>...</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "      <th>2020</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CAN</td>\n",
       "      <td>Canada</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>260.328494</td>\n",
       "      <td>271.695135</td>\n",
       "      <td>285.543065</td>\n",
       "      <td>308.282174</td>\n",
       "      <td>320.455802</td>\n",
       "      <td>344.911566</td>\n",
       "      <td>351.587904</td>\n",
       "      <td>...</td>\n",
       "      <td>554.684502</td>\n",
       "      <td>551.136751</td>\n",
       "      <td>564.581949</td>\n",
       "      <td>571.809431</td>\n",
       "      <td>570.155125</td>\n",
       "      <td>553.250422</td>\n",
       "      <td>565.912760</td>\n",
       "      <td>576.210989</td>\n",
       "      <td>577.997163</td>\n",
       "      <td>517.656723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MEX</td>\n",
       "      <td>Mexico</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>62.133329</td>\n",
       "      <td>65.055673</td>\n",
       "      <td>66.647498</td>\n",
       "      <td>72.170911</td>\n",
       "      <td>79.106716</td>\n",
       "      <td>84.172180</td>\n",
       "      <td>89.461650</td>\n",
       "      <td>...</td>\n",
       "      <td>472.973143</td>\n",
       "      <td>476.710063</td>\n",
       "      <td>483.185138</td>\n",
       "      <td>471.180127</td>\n",
       "      <td>475.225283</td>\n",
       "      <td>480.443636</td>\n",
       "      <td>486.075949</td>\n",
       "      <td>477.118539</td>\n",
       "      <td>459.758515</td>\n",
       "      <td>373.215914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>USA</td>\n",
       "      <td>US</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>3451.891855</td>\n",
       "      <td>3639.795004</td>\n",
       "      <td>3738.161319</td>\n",
       "      <td>3947.343459</td>\n",
       "      <td>4117.396247</td>\n",
       "      <td>4271.528132</td>\n",
       "      <td>4309.068601</td>\n",
       "      <td>...</td>\n",
       "      <td>5348.439351</td>\n",
       "      <td>5101.537959</td>\n",
       "      <td>5268.307567</td>\n",
       "      <td>5277.560073</td>\n",
       "      <td>5165.570150</td>\n",
       "      <td>5060.806233</td>\n",
       "      <td>5003.178942</td>\n",
       "      <td>5166.031776</td>\n",
       "      <td>5029.389363</td>\n",
       "      <td>4457.219800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Total North America</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>3774.353678</td>\n",
       "      <td>3976.545812</td>\n",
       "      <td>4090.351882</td>\n",
       "      <td>4327.796544</td>\n",
       "      <td>4516.958764</td>\n",
       "      <td>4700.611877</td>\n",
       "      <td>4750.118155</td>\n",
       "      <td>...</td>\n",
       "      <td>6376.096995</td>\n",
       "      <td>6129.384774</td>\n",
       "      <td>6316.074655</td>\n",
       "      <td>6320.549632</td>\n",
       "      <td>6210.950558</td>\n",
       "      <td>6094.500291</td>\n",
       "      <td>6055.167650</td>\n",
       "      <td>6219.361305</td>\n",
       "      <td>6067.145041</td>\n",
       "      <td>5348.092438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ARG</td>\n",
       "      <td>Argentina</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>82.175120</td>\n",
       "      <td>84.526808</td>\n",
       "      <td>86.852525</td>\n",
       "      <td>89.324243</td>\n",
       "      <td>92.183564</td>\n",
       "      <td>85.895493</td>\n",
       "      <td>90.804629</td>\n",
       "      <td>...</td>\n",
       "      <td>174.763500</td>\n",
       "      <td>183.748817</td>\n",
       "      <td>189.430178</td>\n",
       "      <td>189.544434</td>\n",
       "      <td>192.767155</td>\n",
       "      <td>191.450785</td>\n",
       "      <td>189.685376</td>\n",
       "      <td>187.460685</td>\n",
       "      <td>175.817913</td>\n",
       "      <td>164.133062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Total Asia Pacific</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>1431.313107</td>\n",
       "      <td>1549.491516</td>\n",
       "      <td>1615.940543</td>\n",
       "      <td>1721.616694</td>\n",
       "      <td>1958.204612</td>\n",
       "      <td>2234.981320</td>\n",
       "      <td>2445.576141</td>\n",
       "      <td>...</td>\n",
       "      <td>14869.453290</td>\n",
       "      <td>15309.823063</td>\n",
       "      <td>15666.971249</td>\n",
       "      <td>15850.095276</td>\n",
       "      <td>15988.325151</td>\n",
       "      <td>16148.466989</td>\n",
       "      <td>16501.051245</td>\n",
       "      <td>16917.645565</td>\n",
       "      <td>17203.330308</td>\n",
       "      <td>16812.478080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Total World</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>11189.710673</td>\n",
       "      <td>11694.894484</td>\n",
       "      <td>12055.547104</td>\n",
       "      <td>12701.480490</td>\n",
       "      <td>13483.695285</td>\n",
       "      <td>14291.709386</td>\n",
       "      <td>14762.395588</td>\n",
       "      <td>...</td>\n",
       "      <td>32172.508226</td>\n",
       "      <td>32503.997053</td>\n",
       "      <td>33071.152064</td>\n",
       "      <td>33140.667330</td>\n",
       "      <td>33206.145249</td>\n",
       "      <td>33361.880092</td>\n",
       "      <td>33726.868179</td>\n",
       "      <td>34351.098958</td>\n",
       "      <td>34356.611795</td>\n",
       "      <td>32318.644854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>NaN</td>\n",
       "      <td>of which: OECD</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>7701.282862</td>\n",
       "      <td>8003.175822</td>\n",
       "      <td>8270.866419</td>\n",
       "      <td>8782.764892</td>\n",
       "      <td>9306.691008</td>\n",
       "      <td>9791.451164</td>\n",
       "      <td>9934.096882</td>\n",
       "      <td>...</td>\n",
       "      <td>12857.597343</td>\n",
       "      <td>12667.868636</td>\n",
       "      <td>12767.240569</td>\n",
       "      <td>12553.922901</td>\n",
       "      <td>12473.073827</td>\n",
       "      <td>12377.713337</td>\n",
       "      <td>12396.274549</td>\n",
       "      <td>12494.836120</td>\n",
       "      <td>12140.099652</td>\n",
       "      <td>10778.102600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Non-OECD</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>3488.427811</td>\n",
       "      <td>3691.718662</td>\n",
       "      <td>3784.680685</td>\n",
       "      <td>3918.715598</td>\n",
       "      <td>4177.004277</td>\n",
       "      <td>4500.258222</td>\n",
       "      <td>4828.298706</td>\n",
       "      <td>...</td>\n",
       "      <td>19314.910883</td>\n",
       "      <td>19836.128418</td>\n",
       "      <td>20303.911495</td>\n",
       "      <td>20586.744430</td>\n",
       "      <td>20733.071422</td>\n",
       "      <td>20984.166754</td>\n",
       "      <td>21330.593631</td>\n",
       "      <td>21856.262837</td>\n",
       "      <td>22216.512143</td>\n",
       "      <td>21540.542254</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>NaN</td>\n",
       "      <td>European Union #</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>2616.385867</td>\n",
       "      <td>2663.180253</td>\n",
       "      <td>2742.047705</td>\n",
       "      <td>2923.833370</td>\n",
       "      <td>3148.076867</td>\n",
       "      <td>3332.435674</td>\n",
       "      <td>3409.412017</td>\n",
       "      <td>...</td>\n",
       "      <td>3300.253085</td>\n",
       "      <td>3218.738884</td>\n",
       "      <td>3147.046820</td>\n",
       "      <td>2982.410543</td>\n",
       "      <td>3045.619428</td>\n",
       "      <td>3077.175821</td>\n",
       "      <td>3115.282824</td>\n",
       "      <td>3070.528755</td>\n",
       "      <td>2936.705787</td>\n",
       "      <td>2550.939074</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>102 rows × 59 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    code                               name   unit          1965  \\\n",
       "0    CAN                             Canada  MtCO2    260.328494   \n",
       "1    MEX                             Mexico  MtCO2     62.133329   \n",
       "2    USA                                 US  MtCO2   3451.891855   \n",
       "3    NaN                Total North America  MtCO2   3774.353678   \n",
       "4    ARG                          Argentina  MtCO2     82.175120   \n",
       "..   ...                                ...    ...           ...   \n",
       "97   NaN                 Total Asia Pacific  MtCO2   1431.313107   \n",
       "98   NaN                        Total World  MtCO2  11189.710673   \n",
       "99   NaN                     of which: OECD  MtCO2   7701.282862   \n",
       "100  NaN                           Non-OECD  MtCO2   3488.427811   \n",
       "101  NaN                   European Union #  MtCO2   2616.385867   \n",
       "\n",
       "             1966          1967          1968          1969          1970  \\\n",
       "0      271.695135    285.543065    308.282174    320.455802    344.911566   \n",
       "1       65.055673     66.647498     72.170911     79.106716     84.172180   \n",
       "2     3639.795004   3738.161319   3947.343459   4117.396247   4271.528132   \n",
       "3     3976.545812   4090.351882   4327.796544   4516.958764   4700.611877   \n",
       "4       84.526808     86.852525     89.324243     92.183564     85.895493   \n",
       "..            ...           ...           ...           ...           ...   \n",
       "97    1549.491516   1615.940543   1721.616694   1958.204612   2234.981320   \n",
       "98   11694.894484  12055.547104  12701.480490  13483.695285  14291.709386   \n",
       "99    8003.175822   8270.866419   8782.764892   9306.691008   9791.451164   \n",
       "100   3691.718662   3784.680685   3918.715598   4177.004277   4500.258222   \n",
       "101   2663.180253   2742.047705   2923.833370   3148.076867   3332.435674   \n",
       "\n",
       "             1971  ...          2011          2012          2013  \\\n",
       "0      351.587904  ...    554.684502    551.136751    564.581949   \n",
       "1       89.461650  ...    472.973143    476.710063    483.185138   \n",
       "2     4309.068601  ...   5348.439351   5101.537959   5268.307567   \n",
       "3     4750.118155  ...   6376.096995   6129.384774   6316.074655   \n",
       "4       90.804629  ...    174.763500    183.748817    189.430178   \n",
       "..            ...  ...           ...           ...           ...   \n",
       "97    2445.576141  ...  14869.453290  15309.823063  15666.971249   \n",
       "98   14762.395588  ...  32172.508226  32503.997053  33071.152064   \n",
       "99    9934.096882  ...  12857.597343  12667.868636  12767.240569   \n",
       "100   4828.298706  ...  19314.910883  19836.128418  20303.911495   \n",
       "101   3409.412017  ...   3300.253085   3218.738884   3147.046820   \n",
       "\n",
       "             2014          2015          2016          2017          2018  \\\n",
       "0      571.809431    570.155125    553.250422    565.912760    576.210989   \n",
       "1      471.180127    475.225283    480.443636    486.075949    477.118539   \n",
       "2     5277.560073   5165.570150   5060.806233   5003.178942   5166.031776   \n",
       "3     6320.549632   6210.950558   6094.500291   6055.167650   6219.361305   \n",
       "4      189.544434    192.767155    191.450785    189.685376    187.460685   \n",
       "..            ...           ...           ...           ...           ...   \n",
       "97   15850.095276  15988.325151  16148.466989  16501.051245  16917.645565   \n",
       "98   33140.667330  33206.145249  33361.880092  33726.868179  34351.098958   \n",
       "99   12553.922901  12473.073827  12377.713337  12396.274549  12494.836120   \n",
       "100  20586.744430  20733.071422  20984.166754  21330.593631  21856.262837   \n",
       "101   2982.410543   3045.619428   3077.175821   3115.282824   3070.528755   \n",
       "\n",
       "             2019          2020  \n",
       "0      577.997163    517.656723  \n",
       "1      459.758515    373.215914  \n",
       "2     5029.389363   4457.219800  \n",
       "3     6067.145041   5348.092438  \n",
       "4      175.817913    164.133062  \n",
       "..            ...           ...  \n",
       "97   17203.330308  16812.478080  \n",
       "98   34356.611795  32318.644854  \n",
       "99   12140.099652  10778.102600  \n",
       "100  22216.512143  21540.542254  \n",
       "101   2936.705787   2550.939074  \n",
       "\n",
       "[102 rows x 59 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> BP <<<<<<<<\")\n",
    "\n",
    "# download file\n",
    "# link_bp = \"https://www.bp.com/content/dam/bp/business-sites/en/global/corporate/xlsx/energy-economics/statistical-review/bp-stats-review-2021-all-data.xlsx\"\n",
    "# resp = requests.get(link_bp)\n",
    "# output = open('raw_data/BP/bp-stats-review-2021-all-data.xlsx', \"wb\")\n",
    "# output.write(resp.content)\n",
    "# output.close()\n",
    "\n",
    "#read file\n",
    "df_bp = pd.read_excel(io=\"../raw_data/BP/bp-stats-review-2021-all-data.xlsx\",sheet_name=4,header=2)\n",
    "\n",
    "#clean\n",
    "df_bp = df_bp[df_bp[2020].notnull()]\n",
    "df_bp = df_bp.rename(columns = {\"Million tonnes of carbon dioxide\":\"name\"})\n",
    "df_bp = df_bp.drop([\"2020.1\",\"2009-19\",\"2020.2\"], axis=1)#deleting last three columns\n",
    "df_bp.loc[:,1965:] = df_bp.loc[:,1965:].astype(float)\n",
    "\n",
    "#add unit\n",
    "df_bp.insert(1, \"unit\", \"MtCO2\")\n",
    "\n",
    "#reindex\n",
    "df_bp = df_bp.reset_index().drop(\"index\", axis=1)\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_bp.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_bp, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "# -----------------------------------------------------------------------------------------------------\n",
    "\n",
    "#global?\n",
    "df_bp_global = df_bp[df_bp.name==\"Total World\"].iloc[0,:]\n",
    "df_bp_global.to_csv(\"../clean_data/global_subsets/BP_global.csv\")\n",
    "\n",
    "# ------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_bp.to_csv(\"../clean_data/BP_global.csv\", index=None)\n",
    "df_bp = pd.read_csv(\"../clean_data/BP.csv\")\n",
    "df_bp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f7def08-d6b6-4cd7-bffa-6b9db7908ab7",
   "metadata": {},
   "source": [
    "# CDIAC\n",
    "INFO:\n",
    "- Carbon Dioxide Information Analysis Center\n",
    "- CDIAC. (2017). Global, Regional, and National Fossil-Fuel CO2 Emissions. https://cdiac.ess-dive.lbl.gov/trends/emis/overview_2013.html\n",
    "- DOI: 10.3334/CDIAC/00001_V2017\n",
    "- Authors: Tom Boden and Bob Andres (Oak Ridge National Laboratory); Gregg Marland (Appalachian State University)\n",
    "\n",
    "NOTES:\n",
    "- Units: given in carbon units, not carbon dioxide (converted by multiplying values by 44/12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "2496f1fe-68ee-4f23-8255-0dc4386533cf",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> CDIAC <<<<<<<<\n",
      ">>> RHODESIA-NYASALAND made a list of (['MWI', 'ZWE']) but is now MWI (first item)\n",
      "\n",
      ">>> ST. KITTS-NEVIS-ANGUILLA made a list of (['AIA', 'KNA']) but is now AIA (first item)\n",
      "\n",
      "PATCH APPLIED\n",
      ">>> 37 Missing codes (NaN) in this df\n",
      " ['ANTARCTIC FISHERIES', 'CZECHOSLOVAKIA', 'DEMOCRATIC REPUBLIC OF VIETNAM', 'EAST & WEST PAKISTAN', 'FEDERAL REPUBLIC OF GERMANY', 'FEDERATION OF MALAYA-SINGAPORE', 'FORMER DEMOCRATIC YEMEN', 'FORMER GERMAN DEMOCRATIC REPUBLIC', 'FORMER PANAMA CANAL ZONE', 'FORMER YEMEN', 'FRENCH EQUATORIAL AFRICA', 'FRENCH INDO-CHINA', 'FRENCH WEST AFRICA', 'GUADELOUPE', 'JAPAN (EXCLUDING THE RUYUKU ISLANDS)', 'KUWAITI OIL FIRES', 'LEEWARD ISLANDS', 'MARTINIQUE', 'NETHERLAND ANTILLES', 'NETHERLAND ANTILLES AND ARUBA', 'PACIFIC ISLANDS (PALAU)', 'PENINSULAR MALAYSIA', 'REPUBLIC OF SOUTH VIETNAM', 'REPUBLIC OF SUDAN', 'RHODESIA-NYASALAND', 'RWANDA-URUNDI', 'RYUKYU ISLANDS', 'SABAH', 'SARAWAK', 'ST. KITTS-NEVIS-ANGUILLA', 'SUDAN', 'TANGANYIKA', 'UNITED KOREA', 'USSR', 'YUGOSLAVIA (FORMER SOCIALIST FEDERAL REPUBLIC)', 'YUGOSLAVIA (MONTENEGRO & SERBIA)', 'ZANZIBAR']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>unit</th>\n",
       "      <th>Total CO2 emissions from fossil-fuels and cement production (MtCO2)</th>\n",
       "      <th>Emissions from solid fuel consumption</th>\n",
       "      <th>Emissions from liquid fuel consumption</th>\n",
       "      <th>Emissions from gas fuel consumption</th>\n",
       "      <th>Emissions from cement production</th>\n",
       "      <th>Emissions from gas flaring</th>\n",
       "      <th>Per capita CO2 emissions (tCO2)</th>\n",
       "      <th>Emissions from bunker fuels (not included in the totals)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1751</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>9.357333</td>\n",
       "      <td>9.357333</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1752</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>9.361000</td>\n",
       "      <td>9.361000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1753</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>9.361000</td>\n",
       "      <td>9.361000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1754</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>9.364667</td>\n",
       "      <td>9.364667</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1755</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>9.368333</td>\n",
       "      <td>9.368333</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>259</th>\n",
       "      <td>2010</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>31905.100333</td>\n",
       "      <td>13772.781000</td>\n",
       "      <td>9991.120333</td>\n",
       "      <td>6260.899333</td>\n",
       "      <td>1634.831000</td>\n",
       "      <td>245.476000</td>\n",
       "      <td>1.119653</td>\n",
       "      <td>1118.076667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>260</th>\n",
       "      <td>2011</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>33156.196333</td>\n",
       "      <td>14752.862667</td>\n",
       "      <td>9956.906667</td>\n",
       "      <td>6400.636000</td>\n",
       "      <td>1810.537667</td>\n",
       "      <td>235.227667</td>\n",
       "      <td>1.110047</td>\n",
       "      <td>1145.705000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>261</th>\n",
       "      <td>2012</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>33767.770667</td>\n",
       "      <td>14974.094667</td>\n",
       "      <td>10144.544667</td>\n",
       "      <td>6507.746667</td>\n",
       "      <td>1904.147667</td>\n",
       "      <td>237.233333</td>\n",
       "      <td>1.141360</td>\n",
       "      <td>1091.170667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>262</th>\n",
       "      <td>2013</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>33860.959000</td>\n",
       "      <td>14777.051667</td>\n",
       "      <td>10184.390333</td>\n",
       "      <td>6619.715667</td>\n",
       "      <td>2029.511000</td>\n",
       "      <td>250.261000</td>\n",
       "      <td>1.124420</td>\n",
       "      <td>1120.885333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>263</th>\n",
       "      <td>2014</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>34122.733333</td>\n",
       "      <td>14778.734667</td>\n",
       "      <td>10385.591333</td>\n",
       "      <td>6628.457000</td>\n",
       "      <td>2082.091000</td>\n",
       "      <td>247.859333</td>\n",
       "      <td>1.129700</td>\n",
       "      <td>1130.917333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>264 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Year   unit  \\\n",
       "0    1751  MtCO2   \n",
       "1    1752  MtCO2   \n",
       "2    1753  MtCO2   \n",
       "3    1754  MtCO2   \n",
       "4    1755  MtCO2   \n",
       "..    ...    ...   \n",
       "259  2010  MtCO2   \n",
       "260  2011  MtCO2   \n",
       "261  2012  MtCO2   \n",
       "262  2013  MtCO2   \n",
       "263  2014  MtCO2   \n",
       "\n",
       "     Total CO2 emissions from fossil-fuels and cement production (MtCO2)  \\\n",
       "0                                             9.357333                     \n",
       "1                                             9.361000                     \n",
       "2                                             9.361000                     \n",
       "3                                             9.364667                     \n",
       "4                                             9.368333                     \n",
       "..                                                 ...                     \n",
       "259                                       31905.100333                     \n",
       "260                                       33156.196333                     \n",
       "261                                       33767.770667                     \n",
       "262                                       33860.959000                     \n",
       "263                                       34122.733333                     \n",
       "\n",
       "     Emissions from solid fuel consumption  \\\n",
       "0                                 9.357333   \n",
       "1                                 9.361000   \n",
       "2                                 9.361000   \n",
       "3                                 9.364667   \n",
       "4                                 9.368333   \n",
       "..                                     ...   \n",
       "259                           13772.781000   \n",
       "260                           14752.862667   \n",
       "261                           14974.094667   \n",
       "262                           14777.051667   \n",
       "263                           14778.734667   \n",
       "\n",
       "     Emissions from liquid fuel consumption  \\\n",
       "0                                  0.000000   \n",
       "1                                  0.000000   \n",
       "2                                  0.000000   \n",
       "3                                  0.000000   \n",
       "4                                  0.000000   \n",
       "..                                      ...   \n",
       "259                             9991.120333   \n",
       "260                             9956.906667   \n",
       "261                            10144.544667   \n",
       "262                            10184.390333   \n",
       "263                            10385.591333   \n",
       "\n",
       "     Emissions from gas fuel consumption  Emissions from cement production  \\\n",
       "0                               0.000000                          0.000000   \n",
       "1                               0.000000                          0.000000   \n",
       "2                               0.000000                          0.000000   \n",
       "3                               0.000000                          0.000000   \n",
       "4                               0.000000                          0.000000   \n",
       "..                                   ...                               ...   \n",
       "259                          6260.899333                       1634.831000   \n",
       "260                          6400.636000                       1810.537667   \n",
       "261                          6507.746667                       1904.147667   \n",
       "262                          6619.715667                       2029.511000   \n",
       "263                          6628.457000                       2082.091000   \n",
       "\n",
       "     Emissions from gas flaring  Per capita CO2 emissions (tCO2)  \\\n",
       "0                      0.000000                         0.000000   \n",
       "1                      0.000000                         0.000000   \n",
       "2                      0.000000                         0.000000   \n",
       "3                      0.000000                         0.000000   \n",
       "4                      0.000000                         0.000000   \n",
       "..                          ...                              ...   \n",
       "259                  245.476000                         1.119653   \n",
       "260                  235.227667                         1.110047   \n",
       "261                  237.233333                         1.141360   \n",
       "262                  250.261000                         1.124420   \n",
       "263                  247.859333                         1.129700   \n",
       "\n",
       "     Emissions from bunker fuels (not included in the totals)  \n",
       "0                                             0.000000         \n",
       "1                                             0.000000         \n",
       "2                                             0.000000         \n",
       "3                                             0.000000         \n",
       "4                                             0.000000         \n",
       "..                                                 ...         \n",
       "259                                        1118.076667         \n",
       "260                                        1145.705000         \n",
       "261                                        1091.170667         \n",
       "262                                        1120.885333         \n",
       "263                                        1130.917333         \n",
       "\n",
       "[264 rows x 10 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> CDIAC <<<<<<<<\")\n",
    "\n",
    "#download file\n",
    "link_cdiac = \"http://cdiac.ess-dive.lbl.gov/ftp/ndp030/CSV-FILES/nation.1751_2014.csv\"\n",
    "resp = requests.get(link_cdiac)\n",
    "output = open('../raw_data/CDIAC/nation.1751_2014.csv',\"wb\")\n",
    "output.write(resp.content)\n",
    "output.close()\n",
    "\n",
    "#read file\n",
    "df_cdiac = pd.read_csv(\"../raw_data/CDIAC/nation.1751_2014.csv\")\n",
    "\n",
    "# #clean\n",
    "df_cdiac = df_cdiac.drop([0,1,2])\n",
    "df_cdiac = df_cdiac.rename(columns = {\"Nation\":\"name\"})\n",
    "df_cdiac = df_cdiac.replace(\".\",np.nan) #remove empty cells marked with . to NaN or 0? NaN\n",
    "df_cdiac.iloc[:,2:] = df_cdiac.iloc[:,2:].astype(float, errors=\"raise\")\n",
    "\n",
    "#CONVERT Carbon units to Carbon Dioxide units\n",
    "convert_C_to_CO2=44/12\n",
    "df_cdiac.iloc[:,2:]  = df_cdiac.iloc[:,2:] * convert_C_to_CO2 / 1000\n",
    "\n",
    "\n",
    "#convert years to integers?\n",
    "df_cdiac.Year = df_cdiac.Year.astype(int)\n",
    "\n",
    "#add unit (and change column name, unit multiplied by 1000)\n",
    "df_cdiac.insert(2, \"unit\", \"MtCO2\")\n",
    "df_cdiac = df_cdiac.rename(columns={\"Total CO2 emissions from fossil-fuels and cement production (thousand metric tons of C)\":\"Total CO2 emissions from fossil-fuels and cement production (MtCO2)\",\n",
    "                                    \"Per capita CO2 emissions (metric tons of carbon)\": \"Per capita CO2 emissions (tCO2)\"})\n",
    "\n",
    "# reindex\n",
    "df_cdiac = df_cdiac.reset_index().drop(\"index\", axis=1)\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_cdiac.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_cdiac, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "# global\n",
    "df_cdiac_global = df_cdiac.groupby([\"Year\",\"unit\"]).sum().reset_index()\n",
    "df_cdiac_global.to_csv(\"../clean_data/global_subsets/CDIAC_global.csv\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_cdiac.to_csv(\"../clean_data/CDIAC.csv\", index=None)\n",
    "df_cdiac = pd.read_csv(\"../clean_data/CDIAC.csv\")\n",
    "df_cdiac_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d4c664e-74c6-4751-aeb7-b1dbfbf0cc8e",
   "metadata": {},
   "source": [
    "# EIA\n",
    "\n",
    "INFO:\n",
    "- Energy Information Administration of the United States of America\n",
    "- Data downloaded from website of eia.gov > international > other statistics > emissions by fuel type, and manually cleaned in excel, saved as EIA_raw_processed_all.csv\n",
    "- https://www.eia.gov/international/data/world/other-statistics/emissions-by-fuel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "4f357a97-9f72-42b6-a5b5-bf9031aef7f8",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> EIA <<<<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 13 Missing codes (NaN) in this df\n",
      " ['World', 'Former Czechoslovakia', 'Former Serbia and Montenegro', 'Former U.S.S.R.', 'Former Yugoslavia', 'Germany, East', 'Germany, West', 'Hawaiian Trade Zone', 'Kosovo', 'Micronesia', 'Netherlands Antilles', 'U.S. Pacific Islands', 'U.S. Territories']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>fuel</th>\n",
       "      <th>unit</th>\n",
       "      <th>1949</th>\n",
       "      <th>1950</th>\n",
       "      <th>1951</th>\n",
       "      <th>1952</th>\n",
       "      <th>1953</th>\n",
       "      <th>1954</th>\n",
       "      <th>...</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>Total CO2 emissions</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>2206.690829</td>\n",
       "      <td>2382.046176</td>\n",
       "      <td>2526.687327</td>\n",
       "      <td>2473.373964</td>\n",
       "      <td>2536.892888</td>\n",
       "      <td>2422.252560</td>\n",
       "      <td>...</td>\n",
       "      <td>33652.592480</td>\n",
       "      <td>34786.404150</td>\n",
       "      <td>35607.55121</td>\n",
       "      <td>36013.90151</td>\n",
       "      <td>36046.193730</td>\n",
       "      <td>35855.855040</td>\n",
       "      <td>35278.502260</td>\n",
       "      <td>36072.334790</td>\n",
       "      <td>36903.369940</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>Coal and coke</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>1117.538985</td>\n",
       "      <td>1151.600346</td>\n",
       "      <td>1166.858975</td>\n",
       "      <td>1052.386511</td>\n",
       "      <td>1057.069608</td>\n",
       "      <td>904.191799</td>\n",
       "      <td>...</td>\n",
       "      <td>15035.601790</td>\n",
       "      <td>15850.888010</td>\n",
       "      <td>16325.70577</td>\n",
       "      <td>16505.52204</td>\n",
       "      <td>16392.746650</td>\n",
       "      <td>15844.416120</td>\n",
       "      <td>14956.562250</td>\n",
       "      <td>15365.386070</td>\n",
       "      <td>15783.059180</td>\n",
       "      <td>15953.29648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>Consumed natural gas</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>269.537229</td>\n",
       "      <td>312.826758</td>\n",
       "      <td>369.725933</td>\n",
       "      <td>396.144693</td>\n",
       "      <td>414.852766</td>\n",
       "      <td>437.259069</td>\n",
       "      <td>...</td>\n",
       "      <td>6323.923968</td>\n",
       "      <td>6512.988366</td>\n",
       "      <td>6662.05692</td>\n",
       "      <td>6760.09430</td>\n",
       "      <td>6785.637172</td>\n",
       "      <td>6887.367588</td>\n",
       "      <td>7027.731893</td>\n",
       "      <td>7259.328867</td>\n",
       "      <td>7626.683356</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>Petroleum and other liquids</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>819.614615</td>\n",
       "      <td>917.619072</td>\n",
       "      <td>990.102419</td>\n",
       "      <td>1024.842760</td>\n",
       "      <td>1064.970514</td>\n",
       "      <td>1080.801692</td>\n",
       "      <td>...</td>\n",
       "      <td>12293.066720</td>\n",
       "      <td>12422.527770</td>\n",
       "      <td>12619.78852</td>\n",
       "      <td>12748.28517</td>\n",
       "      <td>12867.809900</td>\n",
       "      <td>13124.071330</td>\n",
       "      <td>13294.208110</td>\n",
       "      <td>13447.619860</td>\n",
       "      <td>13493.627400</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4 rows × 75 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  code   name                         fuel   unit         1949         1950  \\\n",
       "0  NaN  World          Total CO2 emissions  MtCO2  2206.690829  2382.046176   \n",
       "1  NaN  World                Coal and coke  MtCO2  1117.538985  1151.600346   \n",
       "2  NaN  World         Consumed natural gas  MtCO2   269.537229   312.826758   \n",
       "3  NaN  World  Petroleum and other liquids  MtCO2   819.614615   917.619072   \n",
       "\n",
       "          1951         1952         1953         1954  ...          2010  \\\n",
       "0  2526.687327  2473.373964  2536.892888  2422.252560  ...  33652.592480   \n",
       "1  1166.858975  1052.386511  1057.069608   904.191799  ...  15035.601790   \n",
       "2   369.725933   396.144693   414.852766   437.259069  ...   6323.923968   \n",
       "3   990.102419  1024.842760  1064.970514  1080.801692  ...  12293.066720   \n",
       "\n",
       "           2011         2012         2013          2014          2015  \\\n",
       "0  34786.404150  35607.55121  36013.90151  36046.193730  35855.855040   \n",
       "1  15850.888010  16325.70577  16505.52204  16392.746650  15844.416120   \n",
       "2   6512.988366   6662.05692   6760.09430   6785.637172   6887.367588   \n",
       "3  12422.527770  12619.78852  12748.28517  12867.809900  13124.071330   \n",
       "\n",
       "           2016          2017          2018         2019  \n",
       "0  35278.502260  36072.334790  36903.369940          NaN  \n",
       "1  14956.562250  15365.386070  15783.059180  15953.29648  \n",
       "2   7027.731893   7259.328867   7626.683356          NaN  \n",
       "3  13294.208110  13447.619860  13493.627400          NaN  \n",
       "\n",
       "[4 rows x 75 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> EIA <<<<<<<<\")\n",
    "\n",
    "#read file\n",
    "df_eia = pd.read_csv(\"../raw_data/EIA/EIA_raw_processed_all.csv\")\n",
    "\n",
    "#clean\n",
    "df_eia = df_eia.drop(\"Unnamed: 0\", axis=1)\n",
    "df_eia = df_eia.rename({'Unnamed: 1': 'name', 'Unnamed: 2': 'fuel'}, axis=1)  # new method\n",
    "df_eia = df_eia.replace(\"--\",np.nan)\n",
    "df_eia.loc[:,\"1949\":] = df_eia.loc[:,\"1949\":].astype(float)\n",
    "df_eia.loc[:,\"1949\":].columns = df_eia.loc[:,\"1949\":].columns.astype(int)\n",
    "df_eia.fuel = df_eia.fuel.replace({\"        CO2 emissions (MMtonnes CO2)\":\"Total CO2 emissions\",\n",
    "                           \"            Coal and coke (MMtonnes CO2)\":\"Coal and coke\",\n",
    "                           \"            Consumed natural gas (MMtonnes CO2)\": \"Consumed natural gas\",\n",
    "                           \"            Petroleum and other liquids (MMtonnes CO2)\": \"Petroleum and other liquids\"})\n",
    "#delete all empty unnecesary rows\n",
    "for x in range(231):\n",
    "    equation1 = 6*x\n",
    "    equation2 = 6*x+1\n",
    "    df_eia = df_eia.drop([equation1])\n",
    "    df_eia = df_eia.drop([equation2])\n",
    "\n",
    "# #add unit\n",
    "df_eia.insert(2, \"unit\", \"MtCO2\")\n",
    "\n",
    "# #reindex\n",
    "df_eia = df_eia.reset_index().drop(\"index\", axis=1)\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_eia.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_eia, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "# #global?\n",
    "year = 2018\n",
    "df_eia_global = df_eia[df_eia.name==\"World\"]\n",
    "df_eia_global.to_csv(\"../clean_data/global_subsets/EIA_global.csv\")\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_eia.to_csv(\"../clean_data/EIA.csv\", index=None)\n",
    "df_eia = pd.read_csv(\"../clean_data/EIA.csv\")\n",
    "\n",
    "df_eia_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4965a6c0-7fa9-4bc7-aad2-9914bef0694b",
   "metadata": {},
   "source": [
    "# IEA\n",
    "\n",
    "INFO:\n",
    "- International Energy Agency's GHG emissions from energy *highlights*\n",
    "- downloaded from https://www.iea.org/data-and-statistics/data-product/greenhouse-gas-emissions-from-energy-highlights\n",
    "\n",
    "NOTES:\n",
    "- this dataset includes all greenhouse gases, but only shows quantities in carbon dioxide equivalents\n",
    "- should I delete bunkers in the main df? (currently they are included)\n",
    "- Sectoral data by country is available in raw_data/IEA/cleaning_products folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7b5097c1-4707-4799-a540-454c431ff35c",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> IEA <<<<<<<<\n",
      ">>> China (incl. Hong Kong, China) made a list of (['CHN', 'HKG']) but is now CHN (first item)\n",
      "\n",
      "PATCH APPLIED\n",
      ">>> 38 Missing codes (NaN) in this df\n",
      " ['World', 'Annex I Parties ', '   Annex II Parties   ', '      North America      ', '      Europe      ', '      Asia Oceania', '   Annex I EIT   ', 'Non-Annex I Parties ', 'Annex B Kyoto Parties', 'OECD Total', 'Non-OECD Total', 'OECD Americas', 'Korea', 'OECD Asia Oceania', 'OECD Europe', 'Kosovo', 'Former Soviet Union (if no detail)', 'Former Yugoslavia (if no detail)', 'Non-OECD Europe and Eurasia', 'Other Africa', 'Africa', 'Other Asia', 'Asia (excl. China)', 'China (incl. Hong Kong, China)', 'Other Non-OECD Americas', 'Non-OECD Americas', 'Middle East', 'IEA/Accession/Association', 'European Union - 27', 'G7', 'G8', 'G20', 'Americas', 'Asia', 'Europe', 'Oceania', 'International marine bunkers', 'International aviation bunkers']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>unit</th>\n",
       "      <th>1971</th>\n",
       "      <th>1972</th>\n",
       "      <th>1973</th>\n",
       "      <th>1974</th>\n",
       "      <th>1975</th>\n",
       "      <th>1976</th>\n",
       "      <th>...</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "      <th>2020</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>energy</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>16120.824870</td>\n",
       "      <td>16840.204900</td>\n",
       "      <td>17770.991000</td>\n",
       "      <td>17722.368480</td>\n",
       "      <td>17767.906110</td>\n",
       "      <td>18761.579770</td>\n",
       "      <td>...</td>\n",
       "      <td>35147.956940</td>\n",
       "      <td>35556.986810</td>\n",
       "      <td>36178.556740</td>\n",
       "      <td>36273.153730</td>\n",
       "      <td>36167.304110</td>\n",
       "      <td>36182.452960</td>\n",
       "      <td>36771.194710</td>\n",
       "      <td>37596.786600</td>\n",
       "      <td>37629.669290</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>181</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>total_fuel_combustion</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>14284.288490</td>\n",
       "      <td>14967.536480</td>\n",
       "      <td>15815.826960</td>\n",
       "      <td>15786.664360</td>\n",
       "      <td>15852.705180</td>\n",
       "      <td>16687.245120</td>\n",
       "      <td>...</td>\n",
       "      <td>32034.068320</td>\n",
       "      <td>32384.933210</td>\n",
       "      <td>33000.345100</td>\n",
       "      <td>33060.846480</td>\n",
       "      <td>32962.678230</td>\n",
       "      <td>32986.788250</td>\n",
       "      <td>33534.777510</td>\n",
       "      <td>34248.773210</td>\n",
       "      <td>34233.900780</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>364</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>coal</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>5297.244895</td>\n",
       "      <td>5372.974981</td>\n",
       "      <td>5573.858828</td>\n",
       "      <td>5603.169177</td>\n",
       "      <td>5717.346241</td>\n",
       "      <td>5947.152890</td>\n",
       "      <td>...</td>\n",
       "      <td>14685.682110</td>\n",
       "      <td>14817.326130</td>\n",
       "      <td>15167.459170</td>\n",
       "      <td>15193.048480</td>\n",
       "      <td>14718.631140</td>\n",
       "      <td>14474.800500</td>\n",
       "      <td>14717.956060</td>\n",
       "      <td>15082.576110</td>\n",
       "      <td>14891.538660</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>547</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>oil</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>6764.667370</td>\n",
       "      <td>7258.563584</td>\n",
       "      <td>7823.021307</td>\n",
       "      <td>7705.750431</td>\n",
       "      <td>7689.601665</td>\n",
       "      <td>8169.091201</td>\n",
       "      <td>...</td>\n",
       "      <td>10732.520300</td>\n",
       "      <td>10828.834550</td>\n",
       "      <td>11000.550020</td>\n",
       "      <td>11038.235710</td>\n",
       "      <td>11311.187550</td>\n",
       "      <td>11379.850440</td>\n",
       "      <td>11523.130170</td>\n",
       "      <td>11540.009240</td>\n",
       "      <td>11541.869960</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>730</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>gas</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>2048.463652</td>\n",
       "      <td>2158.087347</td>\n",
       "      <td>2237.853518</td>\n",
       "      <td>2290.357773</td>\n",
       "      <td>2253.834552</td>\n",
       "      <td>2375.721994</td>\n",
       "      <td>...</td>\n",
       "      <td>6146.725564</td>\n",
       "      <td>6263.048097</td>\n",
       "      <td>6349.277867</td>\n",
       "      <td>6342.486626</td>\n",
       "      <td>6447.273823</td>\n",
       "      <td>6625.569660</td>\n",
       "      <td>6775.139570</td>\n",
       "      <td>7098.052606</td>\n",
       "      <td>7267.203157</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>913</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>marine_bunkers</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>354.049019</td>\n",
       "      <td>370.588804</td>\n",
       "      <td>391.541383</td>\n",
       "      <td>365.896176</td>\n",
       "      <td>341.212700</td>\n",
       "      <td>351.851792</td>\n",
       "      <td>...</td>\n",
       "      <td>662.463463</td>\n",
       "      <td>616.365331</td>\n",
       "      <td>614.885911</td>\n",
       "      <td>635.568373</td>\n",
       "      <td>659.841647</td>\n",
       "      <td>678.202256</td>\n",
       "      <td>701.873875</td>\n",
       "      <td>700.428265</td>\n",
       "      <td>684.843273</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1094</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>int_aviation_bunkers</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>169.300994</td>\n",
       "      <td>179.147485</td>\n",
       "      <td>186.892094</td>\n",
       "      <td>179.793336</td>\n",
       "      <td>173.866021</td>\n",
       "      <td>174.273795</td>\n",
       "      <td>...</td>\n",
       "      <td>472.112407</td>\n",
       "      <td>472.721116</td>\n",
       "      <td>480.909125</td>\n",
       "      <td>496.627698</td>\n",
       "      <td>524.654032</td>\n",
       "      <td>548.207782</td>\n",
       "      <td>582.230489</td>\n",
       "      <td>609.630469</td>\n",
       "      <td>617.763240</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1275</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>fugitive</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>1836.536378</td>\n",
       "      <td>1872.668420</td>\n",
       "      <td>1955.164040</td>\n",
       "      <td>1935.704120</td>\n",
       "      <td>1915.200933</td>\n",
       "      <td>2074.334648</td>\n",
       "      <td>...</td>\n",
       "      <td>3113.888620</td>\n",
       "      <td>3172.053598</td>\n",
       "      <td>3178.211640</td>\n",
       "      <td>3212.307248</td>\n",
       "      <td>3204.625880</td>\n",
       "      <td>3195.664710</td>\n",
       "      <td>3236.417200</td>\n",
       "      <td>3348.013385</td>\n",
       "      <td>3395.768510</td>\n",
       "      <td>3201.134588</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 54 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     code   name                   type     unit          1971          1972  \\\n",
       "0     NaN  World                 energy  MtCO2eq  16120.824870  16840.204900   \n",
       "181   NaN  World  total_fuel_combustion  MtCO2eq  14284.288490  14967.536480   \n",
       "364   NaN  World                   coal  MtCO2eq   5297.244895   5372.974981   \n",
       "547   NaN  World                    oil  MtCO2eq   6764.667370   7258.563584   \n",
       "730   NaN  World                    gas  MtCO2eq   2048.463652   2158.087347   \n",
       "913   NaN  World         marine_bunkers  MtCO2eq    354.049019    370.588804   \n",
       "1094  NaN  World   int_aviation_bunkers  MtCO2eq    169.300994    179.147485   \n",
       "1275  NaN  World               fugitive  MtCO2eq   1836.536378   1872.668420   \n",
       "\n",
       "              1973          1974          1975          1976  ...  \\\n",
       "0     17770.991000  17722.368480  17767.906110  18761.579770  ...   \n",
       "181   15815.826960  15786.664360  15852.705180  16687.245120  ...   \n",
       "364    5573.858828   5603.169177   5717.346241   5947.152890  ...   \n",
       "547    7823.021307   7705.750431   7689.601665   8169.091201  ...   \n",
       "730    2237.853518   2290.357773   2253.834552   2375.721994  ...   \n",
       "913     391.541383    365.896176    341.212700    351.851792  ...   \n",
       "1094    186.892094    179.793336    173.866021    174.273795  ...   \n",
       "1275   1955.164040   1935.704120   1915.200933   2074.334648  ...   \n",
       "\n",
       "              2011          2012          2013          2014          2015  \\\n",
       "0     35147.956940  35556.986810  36178.556740  36273.153730  36167.304110   \n",
       "181   32034.068320  32384.933210  33000.345100  33060.846480  32962.678230   \n",
       "364   14685.682110  14817.326130  15167.459170  15193.048480  14718.631140   \n",
       "547   10732.520300  10828.834550  11000.550020  11038.235710  11311.187550   \n",
       "730    6146.725564   6263.048097   6349.277867   6342.486626   6447.273823   \n",
       "913     662.463463    616.365331    614.885911    635.568373    659.841647   \n",
       "1094    472.112407    472.721116    480.909125    496.627698    524.654032   \n",
       "1275   3113.888620   3172.053598   3178.211640   3212.307248   3204.625880   \n",
       "\n",
       "              2016          2017          2018          2019         2020  \n",
       "0     36182.452960  36771.194710  37596.786600  37629.669290          NaN  \n",
       "181   32986.788250  33534.777510  34248.773210  34233.900780          NaN  \n",
       "364   14474.800500  14717.956060  15082.576110  14891.538660          NaN  \n",
       "547   11379.850440  11523.130170  11540.009240  11541.869960          NaN  \n",
       "730    6625.569660   6775.139570   7098.052606   7267.203157          NaN  \n",
       "913     678.202256    701.873875    700.428265    684.843273     0.000000  \n",
       "1094    548.207782    582.230489    609.630469    617.763240     0.000000  \n",
       "1275   3195.664710   3236.417200   3348.013385   3395.768510  3201.134588  \n",
       "\n",
       "[8 rows x 54 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> IEA <<<<<<<<\")\n",
    "\n",
    "# read excel file\n",
    "excel_df_dict = pd.read_excel(\"../raw_data/IEA/GHGHighlights.XLS\",sheet_name=None,header=3)\n",
    "\n",
    "# Cleaning 1: of the whole excel (wow), making individual dataframes\n",
    "sheet_list = excel_df_dict.keys()\n",
    "sheet_list = ['CONTENTS', 'GHG Energy', 'GHG Fugi', 'GHG FC', 'GHG FC - Coal', 'GHG FC - Oil', \n",
    "              'GHG FC - Gas', 'CO2 MARBUNK', 'CO2 AVBUNK', 'SECTOR', 'SECTOREH', 'CO2-TES', \n",
    "              'CO2-GDP', 'CO2-GDP PPP', 'CO2-POP', 'SECTPOP', 'KAYA', 'TIMEEXTENDED', 'GEO COVERAGE']\n",
    "country_list = excel_df_dict[\"GHG FC\"][20:189][\"million tonnes of CO2 eq\"].array\n",
    "sheet_list_24line = ['GHG FC', 'GHG FC - Coal', 'GHG FC - Oil', 'GHG FC - Gas']\n",
    "sheet_list_22line = ['GHG Energy', 'GHG Fugi','CO2 MARBUNK', 'CO2 AVBUNK', 'SECTOR', 'SECTOREH']\n",
    "df_list24 = [\"total_fuel_combustion\", \"coal\", \"oil\", \"gas\"]\n",
    "df_list22 = [\"energy\", \"fugitive\", \"marine_bunkers\", \"int_aviation_bunkers\", \"sectors_2019\", \"sectors_alt_2019\"]\n",
    "for i,j in enumerate(sheet_list_24line):\n",
    "    # print(j)\n",
    "    df = excel_df_dict[j]\n",
    "    df = df.drop([0,2,10,12,17,18,19])\n",
    "    df = df.iloc[:-3,:]\n",
    "    df.insert(1,\"type\", \"{}\".format(df_list24[i]), allow_duplicates=True)\n",
    "    df.insert(2,\"unit\",\"MtCO2eq\", allow_duplicates=True)\n",
    "    df = df.replace(\"..\",np.nan)\n",
    "    df = df.replace(\"x\",np.nan)\n",
    "    df.iloc[:,3:] = df.iloc[:,3:].astype(float)\n",
    "    df = df.rename(columns={\"million tonnes of CO2 eq\":\"name\"})\n",
    "    df = df[df.name.notna()]\n",
    "    df = df.reset_index().drop(\"index\", axis=1)\n",
    "    df.to_csv(\"../raw_data/IEA/Cleaning_products/IEA_{}.csv\".format(df_list24[i]))\n",
    "    # display(df)\n",
    "for i,j in enumerate(sheet_list_22line):\n",
    "    # print(j)\n",
    "    df = excel_df_dict[j]\n",
    "    df = df.drop([0,2,10,12,15,16,17,])\n",
    "    df = df.iloc[:-3,:]\n",
    "    df.insert(1,\"type\", \"{}\".format(df_list22[i]), allow_duplicates=True)\n",
    "    df.insert(2,\"unit\",\"MtCO2eq\", allow_duplicates=True)\n",
    "    df = df.replace(\"..\",np.nan)\n",
    "    df = df.replace(\"x\",np.nan)\n",
    "    df.iloc[:,3:] = df.iloc[:,3:].astype(float)\n",
    "    df = df.rename(columns={\"kilo tonnes of CO2 eq\":\"name\"}) #for fugitive!\n",
    "    df = df.rename(columns={\"million tonnes of CO2\":\"name\"}) #for bunkers\n",
    "    df = df.rename(columns={\"million tonnes of CO2 eq\":\"name\"})\n",
    "    df = df[df.name.notna()]\n",
    "    df = df.reset_index().drop(\"index\", axis=1)\n",
    "    df.to_csv(\"../raw_data/IEA/Cleaning_products/IEA_{}.csv\".format(df_list22[i]))\n",
    "    # display(df)\n",
    "\n",
    "#Cleaning 2: Stitch everything together!\n",
    "df_iea = pd.concat([\n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_energy.csv\"),\n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_total_fuel_combustion.csv\"),\n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_coal.csv\"),\n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_oil.csv\"),\n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_gas.csv\"), \n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_marine_bunkers.csv\"), #should i keep these last three?\n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_int_aviation_bunkers.csv\"), #should i keep these last three?\n",
    "    pd.read_csv(\"../raw_data/IEA/Cleaning_products/IEA_fugitive.csv\"), #should i keep these last three?\n",
    "                   ])\n",
    "\n",
    "#Cleaning 3: reset index and fix fugitive (/1000)\n",
    "df_iea = df_iea.drop(\"Unnamed: 0\", axis=1).reset_index().drop(\"index\", axis=1)\n",
    "df_iea.loc[df_iea.type==\"fugitive\", \"1971\":] = df_iea.loc[df_iea.type==\"fugitive\", \"1971\":]/1000\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_iea.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_iea, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "\n",
    "#global\n",
    "df_iea_global = df_iea[df_iea.name=='World']\n",
    "df_iea_global.to_csv(\"../clean_data/global_subsets/IEA_global.csv\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_iea.to_csv(\"../clean_data/IEA.csv\", index=None)\n",
    "df_iea = pd.read_csv(\"../clean_data/IEA.csv\")\n",
    "\n",
    "df_iea_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d793f994-107d-4269-9da1-c59b31706cf5",
   "metadata": {},
   "source": [
    "# GCP\n",
    "\n",
    "INFO:\n",
    "- Global Carbon Project's main dataset: production/territorial-based CO2 emissions\n",
    "- Reference of the full global carbon budget 2021: Pierre  Friedlingstein, Matthew W. Jones, Michael O'Sullivan, Robbie M. Andrew, Dorothee, C. E. Bakker, Judith  Hauck, Corinne  Le Quéré, Glen P. Peters, Wouter  Peters, Julia Pongratz, Stephen Sitch, Josep G. Canadell, Philippe Ciais, Rob B. Jackson, Simone R. Alin, Peter Anthoni, Nicholas R. Bates, Meike Becker, Nicolas Bellouin, Laurent Bopp, Thi Tuyet Trang Chau, Frédéric Chevallier, Louise P. Chini, Margot Cronin, Kim I. Currie, Bertrand Decharme, Laique M. Djeutchouang, Xinyu Dou, Wiley Evans, Richard A. Feely, Liang Feng, Thomas Gasser, Dennis Gilfillan, Thanos Gkritzalis, Giacomo Grassi, Luke Gregor, Nicolas Gruber, Özgür Gürses, Ian Harris, Richard A. Houghton, George C. Hurtt, Yosuke Iida, Tatiana Ilyina, Ingrid T. Luijkx, Atul Jain, Steve D. Jones, Etsushi Kato, Daniel Kennedy, Kees Klein Goldewijk, Jürgen Knauer, Jan Ivar Korsbakken, Arne Körtzinger, Peter Landschützer, Siv K. Lauvset, Nathalie Lefèvre, Sebastian Lienert, Junjie Liu, Gregg Marland, Patrick C. McGuire, Joe R. Melton, David R. Munro, Julia E.M.S Nabel Shin-Ichiro Nakaoka, Yosuke Niwa, Tsuneo Ono, Denis Pierrot, Benjamin Poulter, Gregor Rehder, Laure Resplandy, Eddy Robertson, Christian Rödenbeck, Thais M Rosan, Jörg Schwinger, Clemens Schwingshackl, Roland Séférian, Adrienne J. Sutton, Colm Sweeney, Toste Tanhua, Pieter P Tans, Hanqin Tian, Bronte Tilbrook, Francesco Tubiello, Guido van der Werf, Nicolas Vuichard, Chisato Wada Rik Wanninkhof, Andrew J. Watson, David Willis, Andrew J. Wiltshire, Wenping Yuan, Chao Yue, Xu Yue, Sönke Zaehle, Jiye Zeng. Global Carbon Budget 2021, Earth Syst. Sci. Data, 2021. https://doi.org/XXXXXXX\n",
    "\n",
    "NOTES:\n",
    "- Given in carbon units, converted to carbon dioxide"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f224aa21-f398-4c3b-b7ab-0c5912d4dc74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> GCP <<<<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 16 Missing codes (NaN) in this df\n",
      " ['Kosovo', 'KP Annex B', 'Non KP Annex B', 'OECD', 'Non-OECD', 'EU27', 'Africa', 'Asia', 'Central America', 'Europe', 'Middle East', 'North America', 'Oceania', 'South America', 'Bunkers', 'World']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>unit</th>\n",
       "      <th>1959</th>\n",
       "      <th>1960</th>\n",
       "      <th>1961</th>\n",
       "      <th>1962</th>\n",
       "      <th>1963</th>\n",
       "      <th>1964</th>\n",
       "      <th>1965</th>\n",
       "      <th>...</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "      <th>2020</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>234</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>8862.634223</td>\n",
       "      <td>9393.7655</td>\n",
       "      <td>9424.316685</td>\n",
       "      <td>9757.152775</td>\n",
       "      <td>10279.000031</td>\n",
       "      <td>10836.738839</td>\n",
       "      <td>11325.469625</td>\n",
       "      <td>...</td>\n",
       "      <td>34493.537255</td>\n",
       "      <td>34999.529349</td>\n",
       "      <td>35308.704501</td>\n",
       "      <td>35560.304992</td>\n",
       "      <td>35522.240696</td>\n",
       "      <td>35478.26167</td>\n",
       "      <td>35951.884512</td>\n",
       "      <td>36672.81092</td>\n",
       "      <td>36729.215078</td>\n",
       "      <td>34832.591912</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 65 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    code   name   unit         1959       1960         1961         1962  \\\n",
       "234  NaN  World  MtCO2  8862.634223  9393.7655  9424.316685  9757.152775   \n",
       "\n",
       "             1963          1964          1965  ...          2011  \\\n",
       "234  10279.000031  10836.738839  11325.469625  ...  34493.537255   \n",
       "\n",
       "             2012          2013          2014          2015         2016  \\\n",
       "234  34999.529349  35308.704501  35560.304992  35522.240696  35478.26167   \n",
       "\n",
       "             2017         2018          2019          2020  \n",
       "234  35951.884512  36672.81092  36729.215078  34832.591912  \n",
       "\n",
       "[1 rows x 65 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> GCP <<<<<<<<\")\n",
    "\n",
    "#read data\n",
    "df_gcp = pd.read_excel(\"../raw_data/GCP/National_Carbon_Emissions_2021v0.4.xlsx\",1, header=11, index_col=0)\n",
    "df_gcp = df_gcp.T\n",
    "\n",
    "#clean\n",
    "df_gcp = df_gcp.reset_index().rename({\"index\":\"name\"}, axis=1)\n",
    "\n",
    "#delete \"statistical difference\" row\n",
    "df_gcp = df_gcp.drop(df_gcp.loc[df_gcp.name==\"Statistical Difference\"].index[0])\n",
    "\n",
    "# convert from MtC to MtCO2\n",
    "df_gcp.loc[:,1959:] = df_gcp.loc[:,1959:]*44/12\n",
    "\n",
    "#add unit\n",
    "df_gcp.insert(1, \"unit\", \"MtCO2\")\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_gcp.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_gcp, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "#global\n",
    "df_gcp_global = df_gcp[df_gcp.name==\"World\"]\n",
    "df_gcp_global.to_csv(\"../clean_data/global_subsets/GCP_global.csv\")\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_gcp.to_csv(\"../clean_data/GCP.csv\", index=None)\n",
    "df_gcp = pd.read_csv(\"../clean_data/GCP.csv\")\n",
    "\n",
    "df_gcp_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5fd4c30-7a1a-4b58-85f9-5ca2add0985e",
   "metadata": {},
   "source": [
    "# GCP (consumption-based)\n",
    "\n",
    "INFO:\n",
    "- Consumption-based CO2 emissions\n",
    "- Updated from Peters, GP, Minx, JC, Weber, CL and Edenhofer, O 2011. Growth in emission transfers via international trade from 1990 to 2008. Proceedings of the National Academy of Sciences 108, 8903-8908. http://www.pnas.org/content/108/21/8903.abstract\n",
    "- “Growth in emission transfers via international trade from 1990 to 2008” (Peters & Hertwich, 2008).\n",
    "\n",
    "NOTES:\n",
    "- Given in carbon units, converted to carbon dioxide"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "06341f02-0093-4ae2-93ac-e968866f5d0b",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> GCP (consumption) <<<<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 16 Missing codes (NaN) in this df\n",
      " ['Kosovo', 'KP Annex B', 'Non KP Annex B', 'OECD', 'Non-OECD', 'EU27', 'Africa', 'Asia', 'Central America', 'Europe', 'Middle East', 'North America', 'Oceania', 'South America', 'Bunkers', 'World']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>unit</th>\n",
       "      <th>1959</th>\n",
       "      <th>1960</th>\n",
       "      <th>1961</th>\n",
       "      <th>1962</th>\n",
       "      <th>1963</th>\n",
       "      <th>1964</th>\n",
       "      <th>1965</th>\n",
       "      <th>...</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "      <th>2020</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ALB</td>\n",
       "      <td>Albania</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>6.317225</td>\n",
       "      <td>5.995611</td>\n",
       "      <td>5.997769</td>\n",
       "      <td>6.506155</td>\n",
       "      <td>5.645737</td>\n",
       "      <td>5.505851</td>\n",
       "      <td>6.162990</td>\n",
       "      <td>5.810145</td>\n",
       "      <td>5.855507</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>DZA</td>\n",
       "      <td>Algeria</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AND</td>\n",
       "      <td>Andorra</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AGO</td>\n",
       "      <td>Angola</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>229</th>\n",
       "      <td>NaN</td>\n",
       "      <td>North America</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>7027.401592</td>\n",
       "      <td>6865.431285</td>\n",
       "      <td>6951.050600</td>\n",
       "      <td>6955.268498</td>\n",
       "      <td>6794.297883</td>\n",
       "      <td>6637.812920</td>\n",
       "      <td>6605.988370</td>\n",
       "      <td>6788.294674</td>\n",
       "      <td>6661.377993</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>230</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Oceania</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>431.361483</td>\n",
       "      <td>450.070196</td>\n",
       "      <td>426.921526</td>\n",
       "      <td>436.005669</td>\n",
       "      <td>436.373495</td>\n",
       "      <td>427.209621</td>\n",
       "      <td>427.051713</td>\n",
       "      <td>418.082814</td>\n",
       "      <td>415.848778</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>231</th>\n",
       "      <td>NaN</td>\n",
       "      <td>South America</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>1159.746666</td>\n",
       "      <td>1235.436210</td>\n",
       "      <td>1295.095385</td>\n",
       "      <td>1319.141711</td>\n",
       "      <td>1289.884130</td>\n",
       "      <td>1240.998543</td>\n",
       "      <td>1239.521487</td>\n",
       "      <td>1174.705417</td>\n",
       "      <td>1140.566653</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>232</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Bunkers</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>213.524667</td>\n",
       "      <td>238.292845</td>\n",
       "      <td>263.440589</td>\n",
       "      <td>273.141804</td>\n",
       "      <td>285.944088</td>\n",
       "      <td>313.257358</td>\n",
       "      <td>323.585427</td>\n",
       "      <td>...</td>\n",
       "      <td>1112.771299</td>\n",
       "      <td>1102.538967</td>\n",
       "      <td>1118.908835</td>\n",
       "      <td>1129.513497</td>\n",
       "      <td>1172.553899</td>\n",
       "      <td>1186.190229</td>\n",
       "      <td>1232.700679</td>\n",
       "      <td>1266.183769</td>\n",
       "      <td>1258.682474</td>\n",
       "      <td>1004.963396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233</th>\n",
       "      <td>NaN</td>\n",
       "      <td>World</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>8862.634223</td>\n",
       "      <td>9393.765500</td>\n",
       "      <td>9424.316685</td>\n",
       "      <td>9757.152775</td>\n",
       "      <td>10279.000031</td>\n",
       "      <td>10836.738839</td>\n",
       "      <td>11325.469625</td>\n",
       "      <td>...</td>\n",
       "      <td>34493.537255</td>\n",
       "      <td>34999.529349</td>\n",
       "      <td>35308.704501</td>\n",
       "      <td>35560.304992</td>\n",
       "      <td>35522.240696</td>\n",
       "      <td>35478.261670</td>\n",
       "      <td>35951.884512</td>\n",
       "      <td>36672.810920</td>\n",
       "      <td>36729.215078</td>\n",
       "      <td>34832.591912</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>234 rows × 65 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    code           name   unit         1959         1960         1961  \\\n",
       "0    AFG    Afghanistan  MtCO2          NaN          NaN          NaN   \n",
       "1    ALB        Albania  MtCO2          NaN          NaN          NaN   \n",
       "2    DZA        Algeria  MtCO2          NaN          NaN          NaN   \n",
       "3    AND        Andorra  MtCO2          NaN          NaN          NaN   \n",
       "4    AGO         Angola  MtCO2          NaN          NaN          NaN   \n",
       "..   ...            ...    ...          ...          ...          ...   \n",
       "229  NaN  North America  MtCO2          NaN          NaN          NaN   \n",
       "230  NaN        Oceania  MtCO2          NaN          NaN          NaN   \n",
       "231  NaN  South America  MtCO2          NaN          NaN          NaN   \n",
       "232  NaN        Bunkers  MtCO2   213.524667   238.292845   263.440589   \n",
       "233  NaN          World  MtCO2  8862.634223  9393.765500  9424.316685   \n",
       "\n",
       "            1962          1963          1964          1965  ...          2011  \\\n",
       "0            NaN           NaN           NaN           NaN  ...           NaN   \n",
       "1            NaN           NaN           NaN           NaN  ...      6.317225   \n",
       "2            NaN           NaN           NaN           NaN  ...           NaN   \n",
       "3            NaN           NaN           NaN           NaN  ...           NaN   \n",
       "4            NaN           NaN           NaN           NaN  ...           NaN   \n",
       "..           ...           ...           ...           ...  ...           ...   \n",
       "229          NaN           NaN           NaN           NaN  ...   7027.401592   \n",
       "230          NaN           NaN           NaN           NaN  ...    431.361483   \n",
       "231          NaN           NaN           NaN           NaN  ...   1159.746666   \n",
       "232   273.141804    285.944088    313.257358    323.585427  ...   1112.771299   \n",
       "233  9757.152775  10279.000031  10836.738839  11325.469625  ...  34493.537255   \n",
       "\n",
       "             2012          2013          2014          2015          2016  \\\n",
       "0             NaN           NaN           NaN           NaN           NaN   \n",
       "1        5.995611      5.997769      6.506155      5.645737      5.505851   \n",
       "2             NaN           NaN           NaN           NaN           NaN   \n",
       "3             NaN           NaN           NaN           NaN           NaN   \n",
       "4             NaN           NaN           NaN           NaN           NaN   \n",
       "..            ...           ...           ...           ...           ...   \n",
       "229   6865.431285   6951.050600   6955.268498   6794.297883   6637.812920   \n",
       "230    450.070196    426.921526    436.005669    436.373495    427.209621   \n",
       "231   1235.436210   1295.095385   1319.141711   1289.884130   1240.998543   \n",
       "232   1102.538967   1118.908835   1129.513497   1172.553899   1186.190229   \n",
       "233  34999.529349  35308.704501  35560.304992  35522.240696  35478.261670   \n",
       "\n",
       "             2017          2018          2019          2020  \n",
       "0             NaN           NaN           NaN           NaN  \n",
       "1        6.162990      5.810145      5.855507           NaN  \n",
       "2             NaN           NaN           NaN           NaN  \n",
       "3             NaN           NaN           NaN           NaN  \n",
       "4             NaN           NaN           NaN           NaN  \n",
       "..            ...           ...           ...           ...  \n",
       "229   6605.988370   6788.294674   6661.377993           NaN  \n",
       "230    427.051713    418.082814    415.848778           NaN  \n",
       "231   1239.521487   1174.705417   1140.566653           NaN  \n",
       "232   1232.700679   1266.183769   1258.682474   1004.963396  \n",
       "233  35951.884512  36672.810920  36729.215078  34832.591912  \n",
       "\n",
       "[234 rows x 65 columns]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> GCP (consumption) <<<<<<<<\")\n",
    "\n",
    "#read data\n",
    "df_gcp_cons = pd.read_excel(\"../raw_data/GCP/National_Carbon_Emissions_2021v0.4.xlsx\",2, header=8, index_col=0)\n",
    "df_gcp_cons = df_gcp_cons.T\n",
    "\n",
    "#clean\n",
    "df_gcp_cons = df_gcp_cons.reset_index().rename({\"index\":\"name\"}, axis=1)\n",
    "\n",
    "#delete \"statistical difference\" row\n",
    "df_gcp_cons = df_gcp_cons.drop(df_gcp_cons.loc[df_gcp_cons.name==\"Statistical Difference\"].index[0])\n",
    "\n",
    "# convert from MtC to MtCO2\n",
    "df_gcp_cons.loc[:,1959:] = df_gcp_cons.loc[:,1959:]*44/12\n",
    "\n",
    "#add unit\n",
    "df_gcp_cons.insert(1, \"unit\", \"MtCO2\")\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_gcp_cons.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_gcp_cons, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "#-----------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "#global\n",
    "df_gcp_cons_global = df_gcp_cons[df_gcp_cons.name==\"World\"]\n",
    "df_gcp_cons_global.to_csv(\"../clean_data/global_subsets/GCP_consumption_global.csv\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_gcp_cons.to_csv(\"../clean_data/GCP_consumption.csv\", index=None)\n",
    "df_gcp_cons = pd.read_csv(\"../clean_data/GCP_consumption.csv\")\n",
    "df_gcp_cons"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad5618f5-5b92-4f9c-b4af-0490b31d4879",
   "metadata": {},
   "source": [
    "# EPA\n",
    "INFO:\n",
    "- Environmental Protection Agency of the United States of America\n",
    "- *Global Non-CO2 Greenhouse Gas Emission Projections & Mitigation Potential: 2015-2050*\n",
    "- from https://www.epa.gov/global-mitigation-non-co2-greenhouse-gases/global-non-co2-greenhouse-gas-emission-projections (zip, proj-data-annex_Sept2019.xlsx file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "84e7162d-7d81-4138-9094-0bcbf3024a9c",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> EPA <<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 1 Missing codes (NaN) in this df\n",
      " ['Kosovo']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Global non-CO2 emissions (2015): 12009.955309238629 MtCO2eq\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>unit</th>\n",
       "      <th>sector</th>\n",
       "      <th>source</th>\n",
       "      <th>subsource</th>\n",
       "      <th>gas</th>\n",
       "      <th>year</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Agriculture</td>\n",
       "      <td>AgSoils</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N2O</td>\n",
       "      <td>1990</td>\n",
       "      <td>4.433600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Agriculture</td>\n",
       "      <td>AgSoils</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N2O</td>\n",
       "      <td>1991</td>\n",
       "      <td>4.411430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Agriculture</td>\n",
       "      <td>AgSoils</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N2O</td>\n",
       "      <td>1992</td>\n",
       "      <td>4.389261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Agriculture</td>\n",
       "      <td>AgSoils</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N2O</td>\n",
       "      <td>1993</td>\n",
       "      <td>4.367091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Agriculture</td>\n",
       "      <td>AgSoils</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N2O</td>\n",
       "      <td>1994</td>\n",
       "      <td>4.344921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>618535</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Waste</td>\n",
       "      <td>Wastewater</td>\n",
       "      <td>Urban</td>\n",
       "      <td>N2O</td>\n",
       "      <td>2048</td>\n",
       "      <td>0.122433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>618536</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Waste</td>\n",
       "      <td>Wastewater</td>\n",
       "      <td>Rural</td>\n",
       "      <td>N2O</td>\n",
       "      <td>2049</td>\n",
       "      <td>0.165373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>618537</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Waste</td>\n",
       "      <td>Wastewater</td>\n",
       "      <td>Urban</td>\n",
       "      <td>N2O</td>\n",
       "      <td>2049</td>\n",
       "      <td>0.125733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>618538</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Waste</td>\n",
       "      <td>Wastewater</td>\n",
       "      <td>Rural</td>\n",
       "      <td>N2O</td>\n",
       "      <td>2050</td>\n",
       "      <td>0.166076</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>618539</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>Waste</td>\n",
       "      <td>Wastewater</td>\n",
       "      <td>Urban</td>\n",
       "      <td>N2O</td>\n",
       "      <td>2050</td>\n",
       "      <td>0.129034</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>618540 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       code         name     unit       sector      source subsource  gas  \\\n",
       "0       AFG  Afghanistan  MtCO2eq  Agriculture     AgSoils       NaN  N2O   \n",
       "1       AFG  Afghanistan  MtCO2eq  Agriculture     AgSoils       NaN  N2O   \n",
       "2       AFG  Afghanistan  MtCO2eq  Agriculture     AgSoils       NaN  N2O   \n",
       "3       AFG  Afghanistan  MtCO2eq  Agriculture     AgSoils       NaN  N2O   \n",
       "4       AFG  Afghanistan  MtCO2eq  Agriculture     AgSoils       NaN  N2O   \n",
       "...     ...          ...      ...          ...         ...       ...  ...   \n",
       "618535  ZWE     Zimbabwe  MtCO2eq        Waste  Wastewater     Urban  N2O   \n",
       "618536  ZWE     Zimbabwe  MtCO2eq        Waste  Wastewater     Rural  N2O   \n",
       "618537  ZWE     Zimbabwe  MtCO2eq        Waste  Wastewater     Urban  N2O   \n",
       "618538  ZWE     Zimbabwe  MtCO2eq        Waste  Wastewater     Rural  N2O   \n",
       "618539  ZWE     Zimbabwe  MtCO2eq        Waste  Wastewater     Urban  N2O   \n",
       "\n",
       "        year     value  \n",
       "0       1990  4.433600  \n",
       "1       1991  4.411430  \n",
       "2       1992  4.389261  \n",
       "3       1993  4.367091  \n",
       "4       1994  4.344921  \n",
       "...      ...       ...  \n",
       "618535  2048  0.122433  \n",
       "618536  2049  0.165373  \n",
       "618537  2049  0.125733  \n",
       "618538  2050  0.166076  \n",
       "618539  2050  0.129034  \n",
       "\n",
       "[618540 rows x 9 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> EPA <<<<<<\")\n",
    "\n",
    "# read file\n",
    "df_epa = pd.read_excel(\"../raw_data/EPA/proj-data-annex_Sept2019.xlsx\", 0)\n",
    "\n",
    "#clean\n",
    "df_epa = df_epa.rename(columns={\"country\":\"name\"})\n",
    "df_epa = df_epa.drop(\"unit\",axis=1)\n",
    "df_epa.insert(1,\"unit\",\"MtCO2eq\")\n",
    "df_epa.value.sum()\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_epa.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_epa, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "\n",
    "# #global - should I add it to the total dataframe?\n",
    "df_epa_global = df_epa.groupby([\"year\", \"unit\",\"sector\",\"source\",\"subsource\",\"gas\"],dropna=False).sum().reset_index()\n",
    "df_epa_global.insert(0, \"name\",\"World\")\n",
    "df_epa_global.insert(0, \"code\",np.nan)\n",
    "\n",
    "df_epa_global.to_csv(\"../clean_data/global_subsets/EPA_global.csv\")\n",
    "print(\"\\nGlobal non-CO2 emissions (2015): \" + str(df_epa_global.groupby(\"year\").sum().loc[2015,\"value\"]) + \" MtCO2eq\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_epa.to_csv(\"../clean_data/EPA.csv\", index=None)\n",
    "df_epa = pd.read_csv(\"../clean_data/EPA.csv\")\n",
    "df_epa"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23c06f36-9c61-4073-8a98-6553977fb284",
   "metadata": {},
   "source": [
    "# FAO / FAOSTAT\n",
    "INFO:\n",
    "- Food and Agriculture Organization of the United Nations\n",
    "- from FAOSTAT website > Emission totals > Bulk download > All data (zip file, using NOFLAG file here): https://www.fao.org/faostat/en/#data/GT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "71a6a299-1ea8-4eb3-aa02-a902d629b0fd",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> FAOSTAT <<<<<<<<\n",
      ">>> Australia and New Zealand made a list of (['AUS', 'NZL']) but is now AUS (first item)\n",
      "\n",
      "PATCH APPLIED\n",
      ">>> 49 Missing codes (NaN) in this df\n",
      " ['Belgium-Luxembourg', 'Channel Islands', 'China, mainland', 'Czechoslovakia', 'Ethiopia PDR', 'French Guyana', 'Netherlands Antilles (former)', 'Pacific Islands Trust Territory', 'Serbia and Montenegro', 'Sudan (former)', 'USSR', 'Yugoslav SFR', 'World', 'Africa', 'Eastern Africa', 'Middle Africa', 'Northern Africa', 'Southern Africa', 'Western Africa', 'Americas', 'Northern America', 'Central America', 'Caribbean', 'South America', 'Asia', 'Central Asia', 'Eastern Asia', 'Southern Asia', 'South-eastern Asia', 'Western Asia', 'Europe', 'Eastern Europe', 'Northern Europe', 'Southern Europe', 'Western Europe', 'Oceania', 'Australia and New Zealand', 'Melanesia', 'Micronesia', 'Polynesia', 'European Union (27)', 'Least Developed Countries', 'Land Locked Developing Countries', 'Small Island Developing States', 'Low Income Food Deficit Countries', 'Net Food Importing Developing Countries', 'Annex I countries', 'Non-Annex I countries', 'OECD']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>Area Code</th>\n",
       "      <th>name</th>\n",
       "      <th>Item Code</th>\n",
       "      <th>Item</th>\n",
       "      <th>Element Code</th>\n",
       "      <th>Element</th>\n",
       "      <th>Source Code</th>\n",
       "      <th>Source</th>\n",
       "      <th>Unit</th>\n",
       "      <th>...</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "      <th>2020</th>\n",
       "      <th>2030</th>\n",
       "      <th>2050</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>31181</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>5058</td>\n",
       "      <td>Enteric Fermentation</td>\n",
       "      <td>7225</td>\n",
       "      <td>Emissions (CH4)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>9.636930e+04</td>\n",
       "      <td>9.694660e+04</td>\n",
       "      <td>9.780942e+04</td>\n",
       "      <td>9.885354e+04</td>\n",
       "      <td>9.929557e+04</td>\n",
       "      <td>9.991874e+04</td>\n",
       "      <td>1.008342e+05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.099799e+05</td>\n",
       "      <td>1.221324e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31182</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>5058</td>\n",
       "      <td>Enteric Fermentation</td>\n",
       "      <td>7225</td>\n",
       "      <td>Emissions (CH4)</td>\n",
       "      <td>3051</td>\n",
       "      <td>UNFCCC</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>4.157057e+04</td>\n",
       "      <td>3.733662e+04</td>\n",
       "      <td>5.237421e+04</td>\n",
       "      <td>2.679629e+04</td>\n",
       "      <td>2.561591e+04</td>\n",
       "      <td>2.401546e+04</td>\n",
       "      <td>2.350277e+04</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31183</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>5058</td>\n",
       "      <td>Enteric Fermentation</td>\n",
       "      <td>724413</td>\n",
       "      <td>Emissions (CO2eq) from CH4 (AR5)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>2.698340e+06</td>\n",
       "      <td>2.714505e+06</td>\n",
       "      <td>2.738664e+06</td>\n",
       "      <td>2.767899e+06</td>\n",
       "      <td>2.780276e+06</td>\n",
       "      <td>2.797725e+06</td>\n",
       "      <td>2.823357e+06</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.079436e+06</td>\n",
       "      <td>3.419707e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31184</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>5058</td>\n",
       "      <td>Enteric Fermentation</td>\n",
       "      <td>724413</td>\n",
       "      <td>Emissions (CO2eq) from CH4 (AR5)</td>\n",
       "      <td>3051</td>\n",
       "      <td>UNFCCC</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>1.162406e+06</td>\n",
       "      <td>1.043867e+06</td>\n",
       "      <td>1.460253e+06</td>\n",
       "      <td>7.263198e+05</td>\n",
       "      <td>6.854012e+05</td>\n",
       "      <td>6.724328e+05</td>\n",
       "      <td>6.580774e+05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31185</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>5058</td>\n",
       "      <td>Enteric Fermentation</td>\n",
       "      <td>723113</td>\n",
       "      <td>Emissions (CO2eq) (AR5)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>2.698340e+06</td>\n",
       "      <td>2.714505e+06</td>\n",
       "      <td>2.738664e+06</td>\n",
       "      <td>2.767899e+06</td>\n",
       "      <td>2.780276e+06</td>\n",
       "      <td>2.797725e+06</td>\n",
       "      <td>2.823357e+06</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.079436e+06</td>\n",
       "      <td>3.419707e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31360</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>6516</td>\n",
       "      <td>Land Use change</td>\n",
       "      <td>7230</td>\n",
       "      <td>Emissions (N2O)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>1.765170e+02</td>\n",
       "      <td>1.813805e+02</td>\n",
       "      <td>2.143186e+02</td>\n",
       "      <td>2.101779e+02</td>\n",
       "      <td>2.088701e+02</td>\n",
       "      <td>1.639089e+02</td>\n",
       "      <td>1.970951e+02</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31361</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>6516</td>\n",
       "      <td>Land Use change</td>\n",
       "      <td>7273</td>\n",
       "      <td>Emissions (CO2)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>3.451477e+06</td>\n",
       "      <td>3.712828e+06</td>\n",
       "      <td>3.736617e+06</td>\n",
       "      <td>2.999283e+06</td>\n",
       "      <td>2.964426e+06</td>\n",
       "      <td>3.107372e+06</td>\n",
       "      <td>3.275819e+06</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31362</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>6516</td>\n",
       "      <td>Land Use change</td>\n",
       "      <td>724413</td>\n",
       "      <td>Emissions (CO2eq) from CH4 (AR5)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>1.057169e+05</td>\n",
       "      <td>1.973947e+05</td>\n",
       "      <td>2.157555e+05</td>\n",
       "      <td>8.317803e+04</td>\n",
       "      <td>7.074291e+04</td>\n",
       "      <td>1.063424e+05</td>\n",
       "      <td>1.746944e+05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31363</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>6516</td>\n",
       "      <td>Land Use change</td>\n",
       "      <td>724313</td>\n",
       "      <td>Emissions (CO2eq) from N2O (AR5)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>4.677701e+04</td>\n",
       "      <td>4.806582e+04</td>\n",
       "      <td>5.679442e+04</td>\n",
       "      <td>5.569714e+04</td>\n",
       "      <td>5.535058e+04</td>\n",
       "      <td>4.343587e+04</td>\n",
       "      <td>5.223020e+04</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31364</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5000</td>\n",
       "      <td>World</td>\n",
       "      <td>6516</td>\n",
       "      <td>Land Use change</td>\n",
       "      <td>723113</td>\n",
       "      <td>Emissions (CO2eq) (AR5)</td>\n",
       "      <td>3050</td>\n",
       "      <td>FAO TIER 1</td>\n",
       "      <td>kilotonnes</td>\n",
       "      <td>...</td>\n",
       "      <td>3.603971e+06</td>\n",
       "      <td>3.958288e+06</td>\n",
       "      <td>4.009167e+06</td>\n",
       "      <td>3.138158e+06</td>\n",
       "      <td>3.090520e+06</td>\n",
       "      <td>3.257151e+06</td>\n",
       "      <td>3.502744e+06</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>184 rows × 72 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      code  Area Code   name  Item Code                  Item  Element Code  \\\n",
       "31181  NaN       5000  World       5058  Enteric Fermentation          7225   \n",
       "31182  NaN       5000  World       5058  Enteric Fermentation          7225   \n",
       "31183  NaN       5000  World       5058  Enteric Fermentation        724413   \n",
       "31184  NaN       5000  World       5058  Enteric Fermentation        724413   \n",
       "31185  NaN       5000  World       5058  Enteric Fermentation        723113   \n",
       "...    ...        ...    ...        ...                   ...           ...   \n",
       "31360  NaN       5000  World       6516       Land Use change          7230   \n",
       "31361  NaN       5000  World       6516       Land Use change          7273   \n",
       "31362  NaN       5000  World       6516       Land Use change        724413   \n",
       "31363  NaN       5000  World       6516       Land Use change        724313   \n",
       "31364  NaN       5000  World       6516       Land Use change        723113   \n",
       "\n",
       "                                Element  Source Code      Source        Unit  \\\n",
       "31181                   Emissions (CH4)         3050  FAO TIER 1  kilotonnes   \n",
       "31182                   Emissions (CH4)         3051      UNFCCC  kilotonnes   \n",
       "31183  Emissions (CO2eq) from CH4 (AR5)         3050  FAO TIER 1  kilotonnes   \n",
       "31184  Emissions (CO2eq) from CH4 (AR5)         3051      UNFCCC  kilotonnes   \n",
       "31185           Emissions (CO2eq) (AR5)         3050  FAO TIER 1  kilotonnes   \n",
       "...                                 ...          ...         ...         ...   \n",
       "31360                   Emissions (N2O)         3050  FAO TIER 1  kilotonnes   \n",
       "31361                   Emissions (CO2)         3050  FAO TIER 1  kilotonnes   \n",
       "31362  Emissions (CO2eq) from CH4 (AR5)         3050  FAO TIER 1  kilotonnes   \n",
       "31363  Emissions (CO2eq) from N2O (AR5)         3050  FAO TIER 1  kilotonnes   \n",
       "31364           Emissions (CO2eq) (AR5)         3050  FAO TIER 1  kilotonnes   \n",
       "\n",
       "       ...          2013          2014          2015          2016  \\\n",
       "31181  ...  9.636930e+04  9.694660e+04  9.780942e+04  9.885354e+04   \n",
       "31182  ...  4.157057e+04  3.733662e+04  5.237421e+04  2.679629e+04   \n",
       "31183  ...  2.698340e+06  2.714505e+06  2.738664e+06  2.767899e+06   \n",
       "31184  ...  1.162406e+06  1.043867e+06  1.460253e+06  7.263198e+05   \n",
       "31185  ...  2.698340e+06  2.714505e+06  2.738664e+06  2.767899e+06   \n",
       "...    ...           ...           ...           ...           ...   \n",
       "31360  ...  1.765170e+02  1.813805e+02  2.143186e+02  2.101779e+02   \n",
       "31361  ...  3.451477e+06  3.712828e+06  3.736617e+06  2.999283e+06   \n",
       "31362  ...  1.057169e+05  1.973947e+05  2.157555e+05  8.317803e+04   \n",
       "31363  ...  4.677701e+04  4.806582e+04  5.679442e+04  5.569714e+04   \n",
       "31364  ...  3.603971e+06  3.958288e+06  4.009167e+06  3.138158e+06   \n",
       "\n",
       "               2017          2018          2019  2020          2030  \\\n",
       "31181  9.929557e+04  9.991874e+04  1.008342e+05   NaN  1.099799e+05   \n",
       "31182  2.561591e+04  2.401546e+04  2.350277e+04   NaN           NaN   \n",
       "31183  2.780276e+06  2.797725e+06  2.823357e+06   NaN  3.079436e+06   \n",
       "31184  6.854012e+05  6.724328e+05  6.580774e+05   NaN           NaN   \n",
       "31185  2.780276e+06  2.797725e+06  2.823357e+06   NaN  3.079436e+06   \n",
       "...             ...           ...           ...   ...           ...   \n",
       "31360  2.088701e+02  1.639089e+02  1.970951e+02   NaN           NaN   \n",
       "31361  2.964426e+06  3.107372e+06  3.275819e+06   NaN           NaN   \n",
       "31362  7.074291e+04  1.063424e+05  1.746944e+05   NaN           NaN   \n",
       "31363  5.535058e+04  4.343587e+04  5.223020e+04   NaN           NaN   \n",
       "31364  3.090520e+06  3.257151e+06  3.502744e+06   NaN           NaN   \n",
       "\n",
       "               2050  \n",
       "31181  1.221324e+05  \n",
       "31182           NaN  \n",
       "31183  3.419707e+06  \n",
       "31184           NaN  \n",
       "31185  3.419707e+06  \n",
       "...             ...  \n",
       "31360           NaN  \n",
       "31361           NaN  \n",
       "31362           NaN  \n",
       "31363           NaN  \n",
       "31364           NaN  \n",
       "\n",
       "[184 rows x 72 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> FAOSTAT <<<<<<<<\")\n",
    "\n",
    "df_fao = pd.read_csv(\"../raw_data/FAO/Emissions_Totals_E_All_Data_NOFLAG.csv\", encoding='latin-1')\n",
    "\n",
    "#clean\n",
    "df_fao.columns = df_fao.columns.str.replace(\"Y\",\"\")\n",
    "df_fao = df_fao.rename(columns={\"Area\":\"name\"})\n",
    "\n",
    "# some info\n",
    "    # print(\"Items    \", df.groupby(\"Item\").count().index)\n",
    "    # print(\"Elements  \",df.groupby(\"Element\").count().index)\n",
    "    # print(\"Source   \",df.groupby(\"Source\").count().index)\n",
    "    # print(\"Area   \",df.groupby(\"Area\").count().index)\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_fao.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_fao, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "#global\n",
    "df_fao_global = df_fao[df_fao.name==\"World\"]\n",
    "df_fao_global.to_csv(\"../clean_data/global_subsets/FAO_global.csv\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_fao.to_csv(\"../clean_data/FAO.csv\", index=None)\n",
    "df_fao = pd.read_csv(\"../clean_data/FAO.csv\", low_memory=False)\n",
    "\n",
    "df_fao_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cb30801-2211-4e22-9d1b-0cd66b320967",
   "metadata": {},
   "source": [
    "# CAIT \n",
    "INFO: \n",
    "- Climate Analysis IndicatorS Tool from the World Resources Institute (WRI)\n",
    "- Climate Watch Historical GHG Emissions. 2021. Washington, DC: World Resources Institute. Available online at: https://www.climatewatchdata.org/ghg-emissions\n",
    "- Downloaded from their website's data explorer >  \"Download bulk data\"> GHG emissions (zip file, only using CAIT csv file) https://www.climatewatchdata.org/data-explorer/historical-emissions?historical-emissions-data-sources=cait&historical-emissions-end_year=2018&historical-emissions-gases=all-ghg&historical-emissions-regions=All%20Selected&historical-emissions-sectors=total-including-lucf&historical-emissions-start_year=1990&page=1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "b7bbf069-d7e0-478d-84ec-1eb0c3f16b56",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> CAIT <<<<<<<<\n",
      "MISSING names:\n",
      " ['EUU', 'WORLD']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>Source</th>\n",
       "      <th>Sector</th>\n",
       "      <th>Gas</th>\n",
       "      <th>1990</th>\n",
       "      <th>1991</th>\n",
       "      <th>1992</th>\n",
       "      <th>1993</th>\n",
       "      <th>1994</th>\n",
       "      <th>...</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Total excluding LUCF</td>\n",
       "      <td>All GHG</td>\n",
       "      <td>15.182846</td>\n",
       "      <td>15.102011</td>\n",
       "      <td>13.634687</td>\n",
       "      <td>13.463994</td>\n",
       "      <td>13.271733</td>\n",
       "      <td>...</td>\n",
       "      <td>36.955463</td>\n",
       "      <td>44.906165</td>\n",
       "      <td>58.651861</td>\n",
       "      <td>66.749281</td>\n",
       "      <td>74.796108</td>\n",
       "      <td>84.619232</td>\n",
       "      <td>93.728621</td>\n",
       "      <td>95.372841</td>\n",
       "      <td>97.300113</td>\n",
       "      <td>98.920758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Total including LUCF</td>\n",
       "      <td>All GHG</td>\n",
       "      <td>12.794043</td>\n",
       "      <td>12.713208</td>\n",
       "      <td>11.245884</td>\n",
       "      <td>11.075191</td>\n",
       "      <td>10.882930</td>\n",
       "      <td>...</td>\n",
       "      <td>37.077364</td>\n",
       "      <td>45.028066</td>\n",
       "      <td>58.405642</td>\n",
       "      <td>66.503062</td>\n",
       "      <td>74.549889</td>\n",
       "      <td>84.373013</td>\n",
       "      <td>93.482402</td>\n",
       "      <td>95.527498</td>\n",
       "      <td>97.454770</td>\n",
       "      <td>99.075415</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Energy</td>\n",
       "      <td>All GHG</td>\n",
       "      <td>5.829497</td>\n",
       "      <td>5.334624</td>\n",
       "      <td>3.760858</td>\n",
       "      <td>3.422760</td>\n",
       "      <td>3.102594</td>\n",
       "      <td>...</td>\n",
       "      <td>20.140819</td>\n",
       "      <td>26.056618</td>\n",
       "      <td>39.529423</td>\n",
       "      <td>47.592229</td>\n",
       "      <td>55.527034</td>\n",
       "      <td>64.675839</td>\n",
       "      <td>74.741644</td>\n",
       "      <td>75.932911</td>\n",
       "      <td>77.718177</td>\n",
       "      <td>79.580444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Industrial Processes</td>\n",
       "      <td>All GHG</td>\n",
       "      <td>0.051879</td>\n",
       "      <td>0.054500</td>\n",
       "      <td>0.060111</td>\n",
       "      <td>0.062722</td>\n",
       "      <td>0.065343</td>\n",
       "      <td>...</td>\n",
       "      <td>0.222971</td>\n",
       "      <td>0.248895</td>\n",
       "      <td>0.313896</td>\n",
       "      <td>0.378967</td>\n",
       "      <td>0.449909</td>\n",
       "      <td>0.534630</td>\n",
       "      <td>0.592081</td>\n",
       "      <td>0.758807</td>\n",
       "      <td>0.911544</td>\n",
       "      <td>1.064280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Agriculture</td>\n",
       "      <td>All GHG</td>\n",
       "      <td>8.072853</td>\n",
       "      <td>8.396465</td>\n",
       "      <td>8.409491</td>\n",
       "      <td>8.486480</td>\n",
       "      <td>8.523959</td>\n",
       "      <td>...</td>\n",
       "      <td>13.857023</td>\n",
       "      <td>15.788381</td>\n",
       "      <td>15.902260</td>\n",
       "      <td>15.777792</td>\n",
       "      <td>15.724861</td>\n",
       "      <td>16.220447</td>\n",
       "      <td>15.112568</td>\n",
       "      <td>15.315738</td>\n",
       "      <td>15.221951</td>\n",
       "      <td>14.744536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9077</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Other Fuel Combustion</td>\n",
       "      <td>N2O</td>\n",
       "      <td>0.407812</td>\n",
       "      <td>0.417100</td>\n",
       "      <td>0.426387</td>\n",
       "      <td>0.435675</td>\n",
       "      <td>0.444962</td>\n",
       "      <td>...</td>\n",
       "      <td>0.876841</td>\n",
       "      <td>0.901256</td>\n",
       "      <td>1.023247</td>\n",
       "      <td>1.145238</td>\n",
       "      <td>1.267229</td>\n",
       "      <td>1.389220</td>\n",
       "      <td>1.511210</td>\n",
       "      <td>1.536870</td>\n",
       "      <td>1.562529</td>\n",
       "      <td>1.588188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9078</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Fugitive Emissions</td>\n",
       "      <td>N2O</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9079</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Total excluding LUCF</td>\n",
       "      <td>F-Gas</td>\n",
       "      <td>0.067604</td>\n",
       "      <td>0.064739</td>\n",
       "      <td>0.061875</td>\n",
       "      <td>0.059011</td>\n",
       "      <td>0.056146</td>\n",
       "      <td>...</td>\n",
       "      <td>0.300719</td>\n",
       "      <td>0.323994</td>\n",
       "      <td>0.379388</td>\n",
       "      <td>0.434782</td>\n",
       "      <td>0.490176</td>\n",
       "      <td>0.545570</td>\n",
       "      <td>0.600965</td>\n",
       "      <td>0.665562</td>\n",
       "      <td>0.730160</td>\n",
       "      <td>0.794758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9080</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Total including LUCF</td>\n",
       "      <td>F-Gas</td>\n",
       "      <td>0.067604</td>\n",
       "      <td>0.064739</td>\n",
       "      <td>0.061875</td>\n",
       "      <td>0.059011</td>\n",
       "      <td>0.056146</td>\n",
       "      <td>...</td>\n",
       "      <td>0.300719</td>\n",
       "      <td>0.323994</td>\n",
       "      <td>0.379388</td>\n",
       "      <td>0.434782</td>\n",
       "      <td>0.490176</td>\n",
       "      <td>0.545570</td>\n",
       "      <td>0.600965</td>\n",
       "      <td>0.665562</td>\n",
       "      <td>0.730160</td>\n",
       "      <td>0.794758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9081</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>CAIT</td>\n",
       "      <td>Industrial Processes</td>\n",
       "      <td>F-Gas</td>\n",
       "      <td>0.067604</td>\n",
       "      <td>0.064739</td>\n",
       "      <td>0.061875</td>\n",
       "      <td>0.059011</td>\n",
       "      <td>0.056146</td>\n",
       "      <td>...</td>\n",
       "      <td>0.300719</td>\n",
       "      <td>0.323994</td>\n",
       "      <td>0.379388</td>\n",
       "      <td>0.434782</td>\n",
       "      <td>0.490176</td>\n",
       "      <td>0.545570</td>\n",
       "      <td>0.600965</td>\n",
       "      <td>0.665562</td>\n",
       "      <td>0.730160</td>\n",
       "      <td>0.794758</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9082 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     code         name Source                  Sector      Gas       1990  \\\n",
       "0     AFG  Afghanistan   CAIT    Total excluding LUCF  All GHG  15.182846   \n",
       "1     AFG  Afghanistan   CAIT    Total including LUCF  All GHG  12.794043   \n",
       "2     AFG  Afghanistan   CAIT                 Energy   All GHG   5.829497   \n",
       "3     AFG  Afghanistan   CAIT   Industrial Processes   All GHG   0.051879   \n",
       "4     AFG  Afghanistan   CAIT            Agriculture   All GHG   8.072853   \n",
       "...   ...          ...    ...                     ...      ...        ...   \n",
       "9077  ZWE     Zimbabwe   CAIT  Other Fuel Combustion       N2O   0.407812   \n",
       "9078  ZWE     Zimbabwe   CAIT     Fugitive Emissions       N2O   0.000000   \n",
       "9079  ZWE     Zimbabwe   CAIT    Total excluding LUCF    F-Gas   0.067604   \n",
       "9080  ZWE     Zimbabwe   CAIT    Total including LUCF    F-Gas   0.067604   \n",
       "9081  ZWE     Zimbabwe   CAIT   Industrial Processes     F-Gas   0.067604   \n",
       "\n",
       "           1991       1992       1993       1994  ...       2009       2010  \\\n",
       "0     15.102011  13.634687  13.463994  13.271733  ...  36.955463  44.906165   \n",
       "1     12.713208  11.245884  11.075191  10.882930  ...  37.077364  45.028066   \n",
       "2      5.334624   3.760858   3.422760   3.102594  ...  20.140819  26.056618   \n",
       "3      0.054500   0.060111   0.062722   0.065343  ...   0.222971   0.248895   \n",
       "4      8.396465   8.409491   8.486480   8.523959  ...  13.857023  15.788381   \n",
       "...         ...        ...        ...        ...  ...        ...        ...   \n",
       "9077   0.417100   0.426387   0.435675   0.444962  ...   0.876841   0.901256   \n",
       "9078   0.000000   0.000000   0.000000   0.000000  ...   0.000000   0.000000   \n",
       "9079   0.064739   0.061875   0.059011   0.056146  ...   0.300719   0.323994   \n",
       "9080   0.064739   0.061875   0.059011   0.056146  ...   0.300719   0.323994   \n",
       "9081   0.064739   0.061875   0.059011   0.056146  ...   0.300719   0.323994   \n",
       "\n",
       "           2011       2012       2013       2014       2015       2016  \\\n",
       "0     58.651861  66.749281  74.796108  84.619232  93.728621  95.372841   \n",
       "1     58.405642  66.503062  74.549889  84.373013  93.482402  95.527498   \n",
       "2     39.529423  47.592229  55.527034  64.675839  74.741644  75.932911   \n",
       "3      0.313896   0.378967   0.449909   0.534630   0.592081   0.758807   \n",
       "4     15.902260  15.777792  15.724861  16.220447  15.112568  15.315738   \n",
       "...         ...        ...        ...        ...        ...        ...   \n",
       "9077   1.023247   1.145238   1.267229   1.389220   1.511210   1.536870   \n",
       "9078   0.000000   0.000000   0.000000   0.000000   0.000000   0.000000   \n",
       "9079   0.379388   0.434782   0.490176   0.545570   0.600965   0.665562   \n",
       "9080   0.379388   0.434782   0.490176   0.545570   0.600965   0.665562   \n",
       "9081   0.379388   0.434782   0.490176   0.545570   0.600965   0.665562   \n",
       "\n",
       "           2017       2018  \n",
       "0     97.300113  98.920758  \n",
       "1     97.454770  99.075415  \n",
       "2     77.718177  79.580444  \n",
       "3      0.911544   1.064280  \n",
       "4     15.221951  14.744536  \n",
       "...         ...        ...  \n",
       "9077   1.562529   1.588188  \n",
       "9078   0.000000   0.000000  \n",
       "9079   0.730160   0.794758  \n",
       "9080   0.730160   0.794758  \n",
       "9081   0.730160   0.794758  \n",
       "\n",
       "[9082 rows x 34 columns]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> CAIT <<<<<<<<\")\n",
    "\n",
    "df_cait = pd.read_csv(\"../raw_data/CAIT/CW_HistoricalEmissions_CAIT.csv\")\n",
    "\n",
    "#clean\n",
    "df_cait = df_cait.rename(columns={\"Country\":\"code\"})\n",
    "\n",
    "\n",
    "# COUNTRY_DICT NOT NECESSARY HERE, INVERSE (creating names out of ISO codes)\n",
    "df_cait.insert(1, \"name\", np.nan)\n",
    "COUNTRY_NAME_GENERATOR_FROM_ISO3(df_cait, show_missing = True, show_duplicates_nan = True)\n",
    "\n",
    "\n",
    "#Global (included in dataset!)\n",
    "df_cait_global = df_cait[df_cait.code==\"WORLD\"]\n",
    "df_cait_global.to_csv(\"../clean_data/global_subsets/CAIT_global.csv\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_cait.to_csv(\"../clean_data/CAIT.csv\", index=None)\n",
    "df_cait = pd.read_csv(\"../clean_data/CAIT.csv\")\n",
    "df_cait"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c432995-7cb9-46a9-9881-a6f345e0ee5e",
   "metadata": {},
   "source": [
    "# EDGAR \n",
    "INFO:\n",
    "- European Comission\n",
    "- Crippa, Monica; Guizzardi, Diego; Muntean, Marilena; Schaaf, Edwin; Lo Vullo, Eleonora; Solazzo, Efisio; Monforti-Ferrario, Fabio; Olivier, Jos; Vignati, Elisabetta (2021):  EDGAR v6.0 Greenhouse Gas Emissions. European Commission, Joint Research Centre (JRC) [Dataset] PID: http://data.europa.eu/89h/97a67d67-c62e-4826-b873-9d972c4f670b\n",
    "\n",
    "NOTES:\n",
    "- given in native units (must convert to CO2 equivalents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "c741f491-20a2-4acc-8604-e06946da966d",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> EDGAR <<<<<<<<\n",
      "NO DUPLICATES HERE!\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>IPCC_annex</th>\n",
       "      <th>ipcc_code_2006_for_standard_report</th>\n",
       "      <th>ipcc_code_2006_for_standard_report_name</th>\n",
       "      <th>fossil_bio</th>\n",
       "      <th>gas</th>\n",
       "      <th>unit</th>\n",
       "      <th>1970</th>\n",
       "      <th>1971</th>\n",
       "      <th>1972</th>\n",
       "      <th>1973</th>\n",
       "      <th>...</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.1.a</td>\n",
       "      <td>Main Activity Electricity and Heat Production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>3213.815550</td>\n",
       "      <td>3315.826787</td>\n",
       "      <td>3539.201951</td>\n",
       "      <td>3814.658085</td>\n",
       "      <td>...</td>\n",
       "      <td>5434.038774</td>\n",
       "      <td>5697.005749</td>\n",
       "      <td>5702.852710</td>\n",
       "      <td>5609.772496</td>\n",
       "      <td>5489.523630</td>\n",
       "      <td>5299.182358</td>\n",
       "      <td>5077.945208</td>\n",
       "      <td>4929.190635</td>\n",
       "      <td>4813.571128</td>\n",
       "      <td>4806.399160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.1.bc</td>\n",
       "      <td>Petroleum Refining - Manufacture of Solid Fuel...</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>569.719736</td>\n",
       "      <td>570.625564</td>\n",
       "      <td>595.947283</td>\n",
       "      <td>658.840504</td>\n",
       "      <td>...</td>\n",
       "      <td>681.302764</td>\n",
       "      <td>702.701907</td>\n",
       "      <td>663.974103</td>\n",
       "      <td>668.893197</td>\n",
       "      <td>674.105611</td>\n",
       "      <td>675.014980</td>\n",
       "      <td>687.200447</td>\n",
       "      <td>678.450499</td>\n",
       "      <td>675.199487</td>\n",
       "      <td>676.541685</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.2</td>\n",
       "      <td>Manufacturing Industries and Construction</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>2972.892721</td>\n",
       "      <td>2706.383463</td>\n",
       "      <td>2762.210279</td>\n",
       "      <td>2907.952202</td>\n",
       "      <td>...</td>\n",
       "      <td>1413.887302</td>\n",
       "      <td>1539.465763</td>\n",
       "      <td>1524.550049</td>\n",
       "      <td>1549.817713</td>\n",
       "      <td>1528.040393</td>\n",
       "      <td>1536.131137</td>\n",
       "      <td>1510.605271</td>\n",
       "      <td>1516.043312</td>\n",
       "      <td>1531.083444</td>\n",
       "      <td>1561.854007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.3.a</td>\n",
       "      <td>Civil Aviation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>158.235781</td>\n",
       "      <td>163.988506</td>\n",
       "      <td>168.030012</td>\n",
       "      <td>172.142863</td>\n",
       "      <td>...</td>\n",
       "      <td>208.858129</td>\n",
       "      <td>211.842706</td>\n",
       "      <td>216.147437</td>\n",
       "      <td>217.164181</td>\n",
       "      <td>221.185512</td>\n",
       "      <td>224.288113</td>\n",
       "      <td>230.979320</td>\n",
       "      <td>239.397098</td>\n",
       "      <td>250.956904</td>\n",
       "      <td>254.104536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.3.b_noRES</td>\n",
       "      <td>Road Transportation no resuspension</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>1412.217678</td>\n",
       "      <td>1489.615814</td>\n",
       "      <td>1599.277709</td>\n",
       "      <td>1704.497199</td>\n",
       "      <td>...</td>\n",
       "      <td>2931.487375</td>\n",
       "      <td>2987.419348</td>\n",
       "      <td>2914.213790</td>\n",
       "      <td>2856.974915</td>\n",
       "      <td>2902.279772</td>\n",
       "      <td>2909.505739</td>\n",
       "      <td>2983.026499</td>\n",
       "      <td>3010.393941</td>\n",
       "      <td>3032.812185</td>\n",
       "      <td>3042.232143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.3.c</td>\n",
       "      <td>Railways</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>150.252780</td>\n",
       "      <td>142.100079</td>\n",
       "      <td>145.504659</td>\n",
       "      <td>147.813581</td>\n",
       "      <td>...</td>\n",
       "      <td>48.905404</td>\n",
       "      <td>51.225435</td>\n",
       "      <td>61.246852</td>\n",
       "      <td>56.406584</td>\n",
       "      <td>57.184112</td>\n",
       "      <td>57.149435</td>\n",
       "      <td>59.446145</td>\n",
       "      <td>57.974854</td>\n",
       "      <td>58.674508</td>\n",
       "      <td>59.333129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.3.d</td>\n",
       "      <td>Water-borne Navigation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>78.824287</td>\n",
       "      <td>83.041373</td>\n",
       "      <td>80.029488</td>\n",
       "      <td>85.261555</td>\n",
       "      <td>...</td>\n",
       "      <td>59.801974</td>\n",
       "      <td>56.407708</td>\n",
       "      <td>66.789075</td>\n",
       "      <td>65.497970</td>\n",
       "      <td>62.884001</td>\n",
       "      <td>59.939346</td>\n",
       "      <td>61.688319</td>\n",
       "      <td>61.607988</td>\n",
       "      <td>61.152662</td>\n",
       "      <td>61.139501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.3.e</td>\n",
       "      <td>Other Transportation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>76.675159</td>\n",
       "      <td>77.660443</td>\n",
       "      <td>79.308104</td>\n",
       "      <td>76.992672</td>\n",
       "      <td>...</td>\n",
       "      <td>122.692584</td>\n",
       "      <td>139.784968</td>\n",
       "      <td>136.271811</td>\n",
       "      <td>128.427155</td>\n",
       "      <td>134.693367</td>\n",
       "      <td>125.875656</td>\n",
       "      <td>124.428583</td>\n",
       "      <td>131.610487</td>\n",
       "      <td>133.308594</td>\n",
       "      <td>142.276621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.4</td>\n",
       "      <td>Other Sectors</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>2202.055872</td>\n",
       "      <td>2206.685635</td>\n",
       "      <td>2285.567351</td>\n",
       "      <td>2305.021305</td>\n",
       "      <td>...</td>\n",
       "      <td>1799.485279</td>\n",
       "      <td>1832.718520</td>\n",
       "      <td>1730.958026</td>\n",
       "      <td>1656.426855</td>\n",
       "      <td>1740.985484</td>\n",
       "      <td>1674.202368</td>\n",
       "      <td>1667.363322</td>\n",
       "      <td>1669.706138</td>\n",
       "      <td>1698.416411</td>\n",
       "      <td>1743.825979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.A.5</td>\n",
       "      <td>Non-Specified</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>204.808594</td>\n",
       "      <td>213.331921</td>\n",
       "      <td>225.844137</td>\n",
       "      <td>239.613884</td>\n",
       "      <td>...</td>\n",
       "      <td>22.101318</td>\n",
       "      <td>22.339847</td>\n",
       "      <td>28.709985</td>\n",
       "      <td>30.122639</td>\n",
       "      <td>30.325389</td>\n",
       "      <td>29.962261</td>\n",
       "      <td>29.948848</td>\n",
       "      <td>29.505735</td>\n",
       "      <td>28.014742</td>\n",
       "      <td>27.777712</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.B.1</td>\n",
       "      <td>Solid Fuels</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>278.961408</td>\n",
       "      <td>253.984905</td>\n",
       "      <td>256.808946</td>\n",
       "      <td>272.801035</td>\n",
       "      <td>...</td>\n",
       "      <td>93.244887</td>\n",
       "      <td>104.790282</td>\n",
       "      <td>103.309120</td>\n",
       "      <td>109.526988</td>\n",
       "      <td>94.738056</td>\n",
       "      <td>90.175374</td>\n",
       "      <td>120.173820</td>\n",
       "      <td>100.622461</td>\n",
       "      <td>108.655472</td>\n",
       "      <td>107.413885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>1.B.2</td>\n",
       "      <td>Oil and Natural Gas</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>44.639152</td>\n",
       "      <td>45.309202</td>\n",
       "      <td>44.960362</td>\n",
       "      <td>49.613265</td>\n",
       "      <td>...</td>\n",
       "      <td>113.484430</td>\n",
       "      <td>94.738898</td>\n",
       "      <td>102.790347</td>\n",
       "      <td>77.567246</td>\n",
       "      <td>73.188266</td>\n",
       "      <td>77.043578</td>\n",
       "      <td>79.850279</td>\n",
       "      <td>78.059969</td>\n",
       "      <td>74.873738</td>\n",
       "      <td>86.814592</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>2.A.1</td>\n",
       "      <td>Cement production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>218.694788</td>\n",
       "      <td>226.591632</td>\n",
       "      <td>246.536747</td>\n",
       "      <td>252.978564</td>\n",
       "      <td>...</td>\n",
       "      <td>197.423490</td>\n",
       "      <td>203.470836</td>\n",
       "      <td>208.228500</td>\n",
       "      <td>208.341352</td>\n",
       "      <td>210.026684</td>\n",
       "      <td>215.595958</td>\n",
       "      <td>210.448760</td>\n",
       "      <td>211.044965</td>\n",
       "      <td>216.719631</td>\n",
       "      <td>217.435656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>2.A.2</td>\n",
       "      <td>Lime production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>69.836505</td>\n",
       "      <td>70.366975</td>\n",
       "      <td>73.739580</td>\n",
       "      <td>77.143060</td>\n",
       "      <td>...</td>\n",
       "      <td>55.619768</td>\n",
       "      <td>63.420261</td>\n",
       "      <td>62.790133</td>\n",
       "      <td>62.221542</td>\n",
       "      <td>60.837754</td>\n",
       "      <td>61.043851</td>\n",
       "      <td>58.408170</td>\n",
       "      <td>57.107250</td>\n",
       "      <td>58.403250</td>\n",
       "      <td>59.041500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>2.A.3</td>\n",
       "      <td>Glass Production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>5.997217</td>\n",
       "      <td>6.049004</td>\n",
       "      <td>6.100467</td>\n",
       "      <td>6.151279</td>\n",
       "      <td>...</td>\n",
       "      <td>7.296280</td>\n",
       "      <td>8.398117</td>\n",
       "      <td>8.520551</td>\n",
       "      <td>8.262182</td>\n",
       "      <td>8.709437</td>\n",
       "      <td>8.075784</td>\n",
       "      <td>7.755317</td>\n",
       "      <td>7.640533</td>\n",
       "      <td>7.552652</td>\n",
       "      <td>7.487444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>2.A.4</td>\n",
       "      <td>Other Process Uses of Carbonates</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>79.831092</td>\n",
       "      <td>80.004196</td>\n",
       "      <td>80.353056</td>\n",
       "      <td>80.537018</td>\n",
       "      <td>...</td>\n",
       "      <td>54.686823</td>\n",
       "      <td>60.498810</td>\n",
       "      <td>61.561995</td>\n",
       "      <td>56.832102</td>\n",
       "      <td>55.661802</td>\n",
       "      <td>56.312477</td>\n",
       "      <td>54.243322</td>\n",
       "      <td>53.547126</td>\n",
       "      <td>52.899752</td>\n",
       "      <td>52.042818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>2.B</td>\n",
       "      <td>Chemical Industry</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>121.914103</td>\n",
       "      <td>120.777157</td>\n",
       "      <td>130.596123</td>\n",
       "      <td>138.419563</td>\n",
       "      <td>...</td>\n",
       "      <td>191.530653</td>\n",
       "      <td>206.524810</td>\n",
       "      <td>212.735650</td>\n",
       "      <td>207.968714</td>\n",
       "      <td>209.909273</td>\n",
       "      <td>209.372942</td>\n",
       "      <td>217.631730</td>\n",
       "      <td>219.316223</td>\n",
       "      <td>227.640213</td>\n",
       "      <td>229.836230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>2.C</td>\n",
       "      <td>Metal Industry</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>168.226839</td>\n",
       "      <td>159.212107</td>\n",
       "      <td>171.170031</td>\n",
       "      <td>180.915257</td>\n",
       "      <td>...</td>\n",
       "      <td>74.735119</td>\n",
       "      <td>86.198840</td>\n",
       "      <td>88.845487</td>\n",
       "      <td>87.322426</td>\n",
       "      <td>85.795342</td>\n",
       "      <td>84.873300</td>\n",
       "      <td>82.447748</td>\n",
       "      <td>81.257518</td>\n",
       "      <td>82.014755</td>\n",
       "      <td>82.247114</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>2.D</td>\n",
       "      <td>Non-Energy Products from Fuels and Solvent Use</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>217.887817</td>\n",
       "      <td>216.958287</td>\n",
       "      <td>217.087427</td>\n",
       "      <td>217.293264</td>\n",
       "      <td>...</td>\n",
       "      <td>128.569987</td>\n",
       "      <td>129.521926</td>\n",
       "      <td>125.700205</td>\n",
       "      <td>124.439956</td>\n",
       "      <td>124.462660</td>\n",
       "      <td>125.046265</td>\n",
       "      <td>124.982794</td>\n",
       "      <td>124.508358</td>\n",
       "      <td>127.110742</td>\n",
       "      <td>127.784554</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>3.C.2</td>\n",
       "      <td>Liming</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>30.197873</td>\n",
       "      <td>30.487865</td>\n",
       "      <td>30.905281</td>\n",
       "      <td>31.407229</td>\n",
       "      <td>...</td>\n",
       "      <td>15.530726</td>\n",
       "      <td>16.439274</td>\n",
       "      <td>15.425627</td>\n",
       "      <td>17.540868</td>\n",
       "      <td>15.752995</td>\n",
       "      <td>16.445840</td>\n",
       "      <td>15.856211</td>\n",
       "      <td>15.156252</td>\n",
       "      <td>15.384944</td>\n",
       "      <td>15.956536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>3.C.3</td>\n",
       "      <td>Urea application</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>3.245495</td>\n",
       "      <td>3.415580</td>\n",
       "      <td>3.711844</td>\n",
       "      <td>5.050244</td>\n",
       "      <td>...</td>\n",
       "      <td>10.260019</td>\n",
       "      <td>11.643962</td>\n",
       "      <td>12.124360</td>\n",
       "      <td>13.023224</td>\n",
       "      <td>13.312837</td>\n",
       "      <td>13.791025</td>\n",
       "      <td>14.473040</td>\n",
       "      <td>15.471824</td>\n",
       "      <td>15.593926</td>\n",
       "      <td>14.694288</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>4.C</td>\n",
       "      <td>Incineration and Open Burning of Waste</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>7.101811</td>\n",
       "      <td>7.255954</td>\n",
       "      <td>7.414454</td>\n",
       "      <td>7.575319</td>\n",
       "      <td>...</td>\n",
       "      <td>10.808026</td>\n",
       "      <td>11.215204</td>\n",
       "      <td>11.074501</td>\n",
       "      <td>10.202975</td>\n",
       "      <td>11.735014</td>\n",
       "      <td>10.183195</td>\n",
       "      <td>10.936423</td>\n",
       "      <td>10.902948</td>\n",
       "      <td>10.632786</td>\n",
       "      <td>10.520028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>Annex_I</td>\n",
       "      <td>5.B</td>\n",
       "      <td>Other</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>...</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "      <td>4.919200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>Int. Aviation</td>\n",
       "      <td>1.A.3.a</td>\n",
       "      <td>Civil Aviation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>168.816953</td>\n",
       "      <td>168.816953</td>\n",
       "      <td>178.760913</td>\n",
       "      <td>186.622555</td>\n",
       "      <td>...</td>\n",
       "      <td>433.421749</td>\n",
       "      <td>457.471950</td>\n",
       "      <td>474.450089</td>\n",
       "      <td>478.193813</td>\n",
       "      <td>486.338265</td>\n",
       "      <td>502.588404</td>\n",
       "      <td>530.376870</td>\n",
       "      <td>553.786633</td>\n",
       "      <td>584.863307</td>\n",
       "      <td>609.402253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>Int. Shipping</td>\n",
       "      <td>1.A.3.d</td>\n",
       "      <td>Water-borne Navigation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>353.856240</td>\n",
       "      <td>353.856240</td>\n",
       "      <td>370.877710</td>\n",
       "      <td>391.732931</td>\n",
       "      <td>...</td>\n",
       "      <td>616.141890</td>\n",
       "      <td>662.889335</td>\n",
       "      <td>663.506321</td>\n",
       "      <td>616.930412</td>\n",
       "      <td>614.897420</td>\n",
       "      <td>636.150763</td>\n",
       "      <td>660.114846</td>\n",
       "      <td>678.676733</td>\n",
       "      <td>697.135045</td>\n",
       "      <td>703.568764</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.1.a</td>\n",
       "      <td>Main Activity Electricity and Heat Production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>481.972015</td>\n",
       "      <td>481.971973</td>\n",
       "      <td>522.113082</td>\n",
       "      <td>573.778259</td>\n",
       "      <td>...</td>\n",
       "      <td>6330.379121</td>\n",
       "      <td>6851.251319</td>\n",
       "      <td>7436.644571</td>\n",
       "      <td>7808.554064</td>\n",
       "      <td>8155.119229</td>\n",
       "      <td>8305.291302</td>\n",
       "      <td>8350.858858</td>\n",
       "      <td>8502.831447</td>\n",
       "      <td>8813.023022</td>\n",
       "      <td>9098.673398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.1.bc</td>\n",
       "      <td>Petroleum Refining - Manufacture of Solid Fuel...</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>110.283691</td>\n",
       "      <td>110.283691</td>\n",
       "      <td>119.690529</td>\n",
       "      <td>137.044896</td>\n",
       "      <td>...</td>\n",
       "      <td>838.271028</td>\n",
       "      <td>919.961733</td>\n",
       "      <td>952.375893</td>\n",
       "      <td>964.366690</td>\n",
       "      <td>993.399335</td>\n",
       "      <td>973.670401</td>\n",
       "      <td>943.527349</td>\n",
       "      <td>888.204006</td>\n",
       "      <td>886.144087</td>\n",
       "      <td>878.262493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>116</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.2</td>\n",
       "      <td>Manufacturing Industries and Construction</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>906.198487</td>\n",
       "      <td>906.198487</td>\n",
       "      <td>949.031416</td>\n",
       "      <td>998.046671</td>\n",
       "      <td>...</td>\n",
       "      <td>4188.614163</td>\n",
       "      <td>4564.528872</td>\n",
       "      <td>4771.264240</td>\n",
       "      <td>4789.410568</td>\n",
       "      <td>4837.997749</td>\n",
       "      <td>4873.564068</td>\n",
       "      <td>4853.064410</td>\n",
       "      <td>4719.917569</td>\n",
       "      <td>4699.469276</td>\n",
       "      <td>4849.533218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>119</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.3.a</td>\n",
       "      <td>Civil Aviation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>8.122051</td>\n",
       "      <td>8.122051</td>\n",
       "      <td>8.630163</td>\n",
       "      <td>10.112890</td>\n",
       "      <td>...</td>\n",
       "      <td>68.594142</td>\n",
       "      <td>79.635974</td>\n",
       "      <td>79.198116</td>\n",
       "      <td>84.227215</td>\n",
       "      <td>91.221945</td>\n",
       "      <td>98.732773</td>\n",
       "      <td>110.371913</td>\n",
       "      <td>119.859817</td>\n",
       "      <td>132.652504</td>\n",
       "      <td>134.203412</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.3.b_noRES</td>\n",
       "      <td>Road Transportation no resuspension</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>280.141830</td>\n",
       "      <td>280.142421</td>\n",
       "      <td>305.634183</td>\n",
       "      <td>336.764357</td>\n",
       "      <td>...</td>\n",
       "      <td>2083.287645</td>\n",
       "      <td>2217.565662</td>\n",
       "      <td>2352.713089</td>\n",
       "      <td>2517.077426</td>\n",
       "      <td>2627.879028</td>\n",
       "      <td>2707.375148</td>\n",
       "      <td>2787.817842</td>\n",
       "      <td>2862.162732</td>\n",
       "      <td>2934.617171</td>\n",
       "      <td>2968.618736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>129</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.3.c</td>\n",
       "      <td>Railways</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>77.796687</td>\n",
       "      <td>77.796687</td>\n",
       "      <td>76.707723</td>\n",
       "      <td>74.696596</td>\n",
       "      <td>...</td>\n",
       "      <td>33.830839</td>\n",
       "      <td>34.582529</td>\n",
       "      <td>34.286817</td>\n",
       "      <td>34.325872</td>\n",
       "      <td>33.802275</td>\n",
       "      <td>32.907673</td>\n",
       "      <td>31.214421</td>\n",
       "      <td>31.017499</td>\n",
       "      <td>31.046222</td>\n",
       "      <td>31.665284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.3.d</td>\n",
       "      <td>Water-borne Navigation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>10.115653</td>\n",
       "      <td>10.115653</td>\n",
       "      <td>10.454572</td>\n",
       "      <td>10.746488</td>\n",
       "      <td>...</td>\n",
       "      <td>75.657565</td>\n",
       "      <td>79.343131</td>\n",
       "      <td>83.150809</td>\n",
       "      <td>92.082735</td>\n",
       "      <td>99.828432</td>\n",
       "      <td>104.772500</td>\n",
       "      <td>94.283541</td>\n",
       "      <td>97.276527</td>\n",
       "      <td>109.123498</td>\n",
       "      <td>111.957314</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.3.e</td>\n",
       "      <td>Other Transportation</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>20.960383</td>\n",
       "      <td>20.960383</td>\n",
       "      <td>23.212701</td>\n",
       "      <td>25.303250</td>\n",
       "      <td>...</td>\n",
       "      <td>26.178816</td>\n",
       "      <td>19.083983</td>\n",
       "      <td>22.723168</td>\n",
       "      <td>22.630016</td>\n",
       "      <td>23.769235</td>\n",
       "      <td>33.792333</td>\n",
       "      <td>37.956614</td>\n",
       "      <td>25.573964</td>\n",
       "      <td>23.294117</td>\n",
       "      <td>23.846574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.4</td>\n",
       "      <td>Other Sectors</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>390.792380</td>\n",
       "      <td>390.799156</td>\n",
       "      <td>411.673107</td>\n",
       "      <td>434.372386</td>\n",
       "      <td>...</td>\n",
       "      <td>1262.638512</td>\n",
       "      <td>1287.145972</td>\n",
       "      <td>1347.520260</td>\n",
       "      <td>1376.403126</td>\n",
       "      <td>1401.535357</td>\n",
       "      <td>1422.586527</td>\n",
       "      <td>1449.760662</td>\n",
       "      <td>1474.621951</td>\n",
       "      <td>1512.691057</td>\n",
       "      <td>1561.795249</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.A.5</td>\n",
       "      <td>Non-Specified</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>129.137998</td>\n",
       "      <td>129.137998</td>\n",
       "      <td>137.105276</td>\n",
       "      <td>143.001223</td>\n",
       "      <td>...</td>\n",
       "      <td>196.662156</td>\n",
       "      <td>189.123397</td>\n",
       "      <td>168.643437</td>\n",
       "      <td>192.375466</td>\n",
       "      <td>193.834117</td>\n",
       "      <td>192.246426</td>\n",
       "      <td>187.286185</td>\n",
       "      <td>175.630912</td>\n",
       "      <td>171.881567</td>\n",
       "      <td>175.947018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.B.1</td>\n",
       "      <td>Solid Fuels</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>36.158074</td>\n",
       "      <td>36.137505</td>\n",
       "      <td>38.751911</td>\n",
       "      <td>41.732591</td>\n",
       "      <td>...</td>\n",
       "      <td>197.596966</td>\n",
       "      <td>181.416237</td>\n",
       "      <td>193.019849</td>\n",
       "      <td>216.019351</td>\n",
       "      <td>229.400934</td>\n",
       "      <td>271.948773</td>\n",
       "      <td>298.748228</td>\n",
       "      <td>307.269490</td>\n",
       "      <td>333.390693</td>\n",
       "      <td>340.755272</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>1.B.2</td>\n",
       "      <td>Oil and Natural Gas</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>260.368171</td>\n",
       "      <td>299.467864</td>\n",
       "      <td>340.348653</td>\n",
       "      <td>408.971353</td>\n",
       "      <td>...</td>\n",
       "      <td>195.590476</td>\n",
       "      <td>192.452535</td>\n",
       "      <td>188.395484</td>\n",
       "      <td>219.951241</td>\n",
       "      <td>219.878046</td>\n",
       "      <td>225.941368</td>\n",
       "      <td>226.104882</td>\n",
       "      <td>232.502869</td>\n",
       "      <td>221.920100</td>\n",
       "      <td>219.109310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>2.A.1</td>\n",
       "      <td>Cement production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>75.223060</td>\n",
       "      <td>83.256095</td>\n",
       "      <td>90.135629</td>\n",
       "      <td>95.786610</td>\n",
       "      <td>...</td>\n",
       "      <td>1096.180805</td>\n",
       "      <td>1184.484004</td>\n",
       "      <td>1269.356520</td>\n",
       "      <td>1326.275858</td>\n",
       "      <td>1156.431352</td>\n",
       "      <td>1191.198661</td>\n",
       "      <td>1164.515613</td>\n",
       "      <td>1192.483386</td>\n",
       "      <td>1216.988889</td>\n",
       "      <td>1230.617356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>160</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>2.A.2</td>\n",
       "      <td>Lime production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>46.273097</td>\n",
       "      <td>47.015480</td>\n",
       "      <td>50.974940</td>\n",
       "      <td>54.870285</td>\n",
       "      <td>...</td>\n",
       "      <td>162.234437</td>\n",
       "      <td>167.418485</td>\n",
       "      <td>183.382219</td>\n",
       "      <td>198.828190</td>\n",
       "      <td>207.340475</td>\n",
       "      <td>207.692133</td>\n",
       "      <td>222.596250</td>\n",
       "      <td>252.635250</td>\n",
       "      <td>253.307250</td>\n",
       "      <td>261.049500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>2.A.3</td>\n",
       "      <td>Glass Production</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.254226</td>\n",
       "      <td>0.260681</td>\n",
       "      <td>0.267101</td>\n",
       "      <td>0.273426</td>\n",
       "      <td>...</td>\n",
       "      <td>1.551723</td>\n",
       "      <td>1.562937</td>\n",
       "      <td>1.577265</td>\n",
       "      <td>1.588149</td>\n",
       "      <td>1.593403</td>\n",
       "      <td>1.599149</td>\n",
       "      <td>1.606591</td>\n",
       "      <td>1.615320</td>\n",
       "      <td>1.624116</td>\n",
       "      <td>1.632998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>2.A.4</td>\n",
       "      <td>Other Process Uses of Carbonates</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>12.598031</td>\n",
       "      <td>12.665230</td>\n",
       "      <td>12.798998</td>\n",
       "      <td>12.879114</td>\n",
       "      <td>...</td>\n",
       "      <td>37.624196</td>\n",
       "      <td>39.870247</td>\n",
       "      <td>42.409933</td>\n",
       "      <td>55.145332</td>\n",
       "      <td>53.800110</td>\n",
       "      <td>58.033683</td>\n",
       "      <td>61.136953</td>\n",
       "      <td>61.640663</td>\n",
       "      <td>65.958367</td>\n",
       "      <td>69.359377</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>2.B</td>\n",
       "      <td>Chemical Industry</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>20.032803</td>\n",
       "      <td>20.655604</td>\n",
       "      <td>23.482605</td>\n",
       "      <td>25.590035</td>\n",
       "      <td>...</td>\n",
       "      <td>361.402422</td>\n",
       "      <td>390.720217</td>\n",
       "      <td>418.398872</td>\n",
       "      <td>433.482287</td>\n",
       "      <td>443.212629</td>\n",
       "      <td>455.948287</td>\n",
       "      <td>461.393466</td>\n",
       "      <td>468.210308</td>\n",
       "      <td>474.369416</td>\n",
       "      <td>477.794156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>2.C</td>\n",
       "      <td>Metal Industry</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>73.011816</td>\n",
       "      <td>68.118190</td>\n",
       "      <td>72.769041</td>\n",
       "      <td>75.923171</td>\n",
       "      <td>...</td>\n",
       "      <td>170.951207</td>\n",
       "      <td>197.616612</td>\n",
       "      <td>215.291040</td>\n",
       "      <td>232.315176</td>\n",
       "      <td>258.501550</td>\n",
       "      <td>285.161552</td>\n",
       "      <td>285.330759</td>\n",
       "      <td>287.343546</td>\n",
       "      <td>300.090507</td>\n",
       "      <td>316.478164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>2.D</td>\n",
       "      <td>Non-Energy Products from Fuels and Solvent Use</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>19.820817</td>\n",
       "      <td>20.268858</td>\n",
       "      <td>20.950655</td>\n",
       "      <td>21.650757</td>\n",
       "      <td>...</td>\n",
       "      <td>67.400468</td>\n",
       "      <td>64.009411</td>\n",
       "      <td>64.876564</td>\n",
       "      <td>66.284667</td>\n",
       "      <td>67.713874</td>\n",
       "      <td>68.787320</td>\n",
       "      <td>68.243710</td>\n",
       "      <td>69.882354</td>\n",
       "      <td>71.911656</td>\n",
       "      <td>73.267610</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>175</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>3.C.2</td>\n",
       "      <td>Liming</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>13.289532</td>\n",
       "      <td>14.130215</td>\n",
       "      <td>17.388798</td>\n",
       "      <td>16.801431</td>\n",
       "      <td>...</td>\n",
       "      <td>41.191744</td>\n",
       "      <td>41.439955</td>\n",
       "      <td>41.667900</td>\n",
       "      <td>42.161204</td>\n",
       "      <td>44.504316</td>\n",
       "      <td>48.443468</td>\n",
       "      <td>43.806236</td>\n",
       "      <td>44.372132</td>\n",
       "      <td>45.025034</td>\n",
       "      <td>45.767186</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>3.C.3</td>\n",
       "      <td>Urea application</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>5.845461</td>\n",
       "      <td>6.482326</td>\n",
       "      <td>7.281820</td>\n",
       "      <td>7.909864</td>\n",
       "      <td>...</td>\n",
       "      <td>64.165142</td>\n",
       "      <td>62.584522</td>\n",
       "      <td>66.792198</td>\n",
       "      <td>66.974833</td>\n",
       "      <td>69.597430</td>\n",
       "      <td>66.410853</td>\n",
       "      <td>67.597757</td>\n",
       "      <td>65.130810</td>\n",
       "      <td>64.352208</td>\n",
       "      <td>63.198340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>4.C</td>\n",
       "      <td>Incineration and Open Burning of Waste</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.545612</td>\n",
       "      <td>0.548953</td>\n",
       "      <td>0.569582</td>\n",
       "      <td>0.591585</td>\n",
       "      <td>...</td>\n",
       "      <td>4.188982</td>\n",
       "      <td>4.515509</td>\n",
       "      <td>4.952577</td>\n",
       "      <td>5.402767</td>\n",
       "      <td>5.805969</td>\n",
       "      <td>5.763262</td>\n",
       "      <td>5.875250</td>\n",
       "      <td>5.986389</td>\n",
       "      <td>6.094731</td>\n",
       "      <td>6.202563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>193</th>\n",
       "      <td>Non-Annex_I</td>\n",
       "      <td>5.B</td>\n",
       "      <td>Other</td>\n",
       "      <td>fossil</td>\n",
       "      <td>CO2 (fossil)</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>40.810400</td>\n",
       "      <td>40.924000</td>\n",
       "      <td>41.037500</td>\n",
       "      <td>41.151000</td>\n",
       "      <td>...</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "      <td>42.570000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>48 rows × 55 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        IPCC_annex ipcc_code_2006_for_standard_report  \\\n",
       "3          Annex_I                            1.A.1.a   \n",
       "8          Annex_I                           1.A.1.bc   \n",
       "13         Annex_I                              1.A.2   \n",
       "16         Annex_I                            1.A.3.a   \n",
       "21         Annex_I                      1.A.3.b_noRES   \n",
       "26         Annex_I                            1.A.3.c   \n",
       "31         Annex_I                            1.A.3.d   \n",
       "36         Annex_I                            1.A.3.e   \n",
       "41         Annex_I                              1.A.4   \n",
       "46         Annex_I                              1.A.5   \n",
       "51         Annex_I                              1.B.1   \n",
       "54         Annex_I                              1.B.2   \n",
       "56         Annex_I                              2.A.1   \n",
       "57         Annex_I                              2.A.2   \n",
       "58         Annex_I                              2.A.3   \n",
       "59         Annex_I                              2.A.4   \n",
       "61         Annex_I                                2.B   \n",
       "64         Annex_I                                2.C   \n",
       "65         Annex_I                                2.D   \n",
       "72         Annex_I                              3.C.2   \n",
       "73         Annex_I                              3.C.3   \n",
       "84         Annex_I                                4.C   \n",
       "90         Annex_I                                5.B   \n",
       "93   Int. Aviation                            1.A.3.a   \n",
       "99   Int. Shipping                            1.A.3.d   \n",
       "106    Non-Annex_I                            1.A.1.a   \n",
       "111    Non-Annex_I                           1.A.1.bc   \n",
       "116    Non-Annex_I                              1.A.2   \n",
       "119    Non-Annex_I                            1.A.3.a   \n",
       "124    Non-Annex_I                      1.A.3.b_noRES   \n",
       "129    Non-Annex_I                            1.A.3.c   \n",
       "134    Non-Annex_I                            1.A.3.d   \n",
       "139    Non-Annex_I                            1.A.3.e   \n",
       "144    Non-Annex_I                              1.A.4   \n",
       "149    Non-Annex_I                              1.A.5   \n",
       "154    Non-Annex_I                              1.B.1   \n",
       "157    Non-Annex_I                              1.B.2   \n",
       "159    Non-Annex_I                              2.A.1   \n",
       "160    Non-Annex_I                              2.A.2   \n",
       "161    Non-Annex_I                              2.A.3   \n",
       "162    Non-Annex_I                              2.A.4   \n",
       "164    Non-Annex_I                                2.B   \n",
       "167    Non-Annex_I                                2.C   \n",
       "168    Non-Annex_I                                2.D   \n",
       "175    Non-Annex_I                              3.C.2   \n",
       "176    Non-Annex_I                              3.C.3   \n",
       "187    Non-Annex_I                                4.C   \n",
       "193    Non-Annex_I                                5.B   \n",
       "\n",
       "               ipcc_code_2006_for_standard_report_name fossil_bio  \\\n",
       "3        Main Activity Electricity and Heat Production     fossil   \n",
       "8    Petroleum Refining - Manufacture of Solid Fuel...     fossil   \n",
       "13           Manufacturing Industries and Construction     fossil   \n",
       "16                                      Civil Aviation     fossil   \n",
       "21                 Road Transportation no resuspension     fossil   \n",
       "26                                            Railways     fossil   \n",
       "31                              Water-borne Navigation     fossil   \n",
       "36                                Other Transportation     fossil   \n",
       "41                                       Other Sectors     fossil   \n",
       "46                                       Non-Specified     fossil   \n",
       "51                                         Solid Fuels     fossil   \n",
       "54                                 Oil and Natural Gas     fossil   \n",
       "56                                   Cement production     fossil   \n",
       "57                                     Lime production     fossil   \n",
       "58                                    Glass Production     fossil   \n",
       "59                    Other Process Uses of Carbonates     fossil   \n",
       "61                                   Chemical Industry     fossil   \n",
       "64                                      Metal Industry     fossil   \n",
       "65      Non-Energy Products from Fuels and Solvent Use     fossil   \n",
       "72                                              Liming     fossil   \n",
       "73                                    Urea application     fossil   \n",
       "84              Incineration and Open Burning of Waste     fossil   \n",
       "90                                               Other     fossil   \n",
       "93                                      Civil Aviation     fossil   \n",
       "99                              Water-borne Navigation     fossil   \n",
       "106      Main Activity Electricity and Heat Production     fossil   \n",
       "111  Petroleum Refining - Manufacture of Solid Fuel...     fossil   \n",
       "116          Manufacturing Industries and Construction     fossil   \n",
       "119                                     Civil Aviation     fossil   \n",
       "124                Road Transportation no resuspension     fossil   \n",
       "129                                           Railways     fossil   \n",
       "134                             Water-borne Navigation     fossil   \n",
       "139                               Other Transportation     fossil   \n",
       "144                                      Other Sectors     fossil   \n",
       "149                                      Non-Specified     fossil   \n",
       "154                                        Solid Fuels     fossil   \n",
       "157                                Oil and Natural Gas     fossil   \n",
       "159                                  Cement production     fossil   \n",
       "160                                    Lime production     fossil   \n",
       "161                                   Glass Production     fossil   \n",
       "162                   Other Process Uses of Carbonates     fossil   \n",
       "164                                  Chemical Industry     fossil   \n",
       "167                                     Metal Industry     fossil   \n",
       "168     Non-Energy Products from Fuels and Solvent Use     fossil   \n",
       "175                                             Liming     fossil   \n",
       "176                                   Urea application     fossil   \n",
       "187             Incineration and Open Burning of Waste     fossil   \n",
       "193                                              Other     fossil   \n",
       "\n",
       "              gas   unit         1970         1971         1972         1973  \\\n",
       "3    CO2 (fossil)  MtCO2  3213.815550  3315.826787  3539.201951  3814.658085   \n",
       "8    CO2 (fossil)  MtCO2   569.719736   570.625564   595.947283   658.840504   \n",
       "13   CO2 (fossil)  MtCO2  2972.892721  2706.383463  2762.210279  2907.952202   \n",
       "16   CO2 (fossil)  MtCO2   158.235781   163.988506   168.030012   172.142863   \n",
       "21   CO2 (fossil)  MtCO2  1412.217678  1489.615814  1599.277709  1704.497199   \n",
       "26   CO2 (fossil)  MtCO2   150.252780   142.100079   145.504659   147.813581   \n",
       "31   CO2 (fossil)  MtCO2    78.824287    83.041373    80.029488    85.261555   \n",
       "36   CO2 (fossil)  MtCO2    76.675159    77.660443    79.308104    76.992672   \n",
       "41   CO2 (fossil)  MtCO2  2202.055872  2206.685635  2285.567351  2305.021305   \n",
       "46   CO2 (fossil)  MtCO2   204.808594   213.331921   225.844137   239.613884   \n",
       "51   CO2 (fossil)  MtCO2   278.961408   253.984905   256.808946   272.801035   \n",
       "54   CO2 (fossil)  MtCO2    44.639152    45.309202    44.960362    49.613265   \n",
       "56   CO2 (fossil)  MtCO2   218.694788   226.591632   246.536747   252.978564   \n",
       "57   CO2 (fossil)  MtCO2    69.836505    70.366975    73.739580    77.143060   \n",
       "58   CO2 (fossil)  MtCO2     5.997217     6.049004     6.100467     6.151279   \n",
       "59   CO2 (fossil)  MtCO2    79.831092    80.004196    80.353056    80.537018   \n",
       "61   CO2 (fossil)  MtCO2   121.914103   120.777157   130.596123   138.419563   \n",
       "64   CO2 (fossil)  MtCO2   168.226839   159.212107   171.170031   180.915257   \n",
       "65   CO2 (fossil)  MtCO2   217.887817   216.958287   217.087427   217.293264   \n",
       "72   CO2 (fossil)  MtCO2    30.197873    30.487865    30.905281    31.407229   \n",
       "73   CO2 (fossil)  MtCO2     3.245495     3.415580     3.711844     5.050244   \n",
       "84   CO2 (fossil)  MtCO2     7.101811     7.255954     7.414454     7.575319   \n",
       "90   CO2 (fossil)  MtCO2     4.919200     4.919200     4.919200     4.919200   \n",
       "93   CO2 (fossil)  MtCO2   168.816953   168.816953   178.760913   186.622555   \n",
       "99   CO2 (fossil)  MtCO2   353.856240   353.856240   370.877710   391.732931   \n",
       "106  CO2 (fossil)  MtCO2   481.972015   481.971973   522.113082   573.778259   \n",
       "111  CO2 (fossil)  MtCO2   110.283691   110.283691   119.690529   137.044896   \n",
       "116  CO2 (fossil)  MtCO2   906.198487   906.198487   949.031416   998.046671   \n",
       "119  CO2 (fossil)  MtCO2     8.122051     8.122051     8.630163    10.112890   \n",
       "124  CO2 (fossil)  MtCO2   280.141830   280.142421   305.634183   336.764357   \n",
       "129  CO2 (fossil)  MtCO2    77.796687    77.796687    76.707723    74.696596   \n",
       "134  CO2 (fossil)  MtCO2    10.115653    10.115653    10.454572    10.746488   \n",
       "139  CO2 (fossil)  MtCO2    20.960383    20.960383    23.212701    25.303250   \n",
       "144  CO2 (fossil)  MtCO2   390.792380   390.799156   411.673107   434.372386   \n",
       "149  CO2 (fossil)  MtCO2   129.137998   129.137998   137.105276   143.001223   \n",
       "154  CO2 (fossil)  MtCO2    36.158074    36.137505    38.751911    41.732591   \n",
       "157  CO2 (fossil)  MtCO2   260.368171   299.467864   340.348653   408.971353   \n",
       "159  CO2 (fossil)  MtCO2    75.223060    83.256095    90.135629    95.786610   \n",
       "160  CO2 (fossil)  MtCO2    46.273097    47.015480    50.974940    54.870285   \n",
       "161  CO2 (fossil)  MtCO2     0.254226     0.260681     0.267101     0.273426   \n",
       "162  CO2 (fossil)  MtCO2    12.598031    12.665230    12.798998    12.879114   \n",
       "164  CO2 (fossil)  MtCO2    20.032803    20.655604    23.482605    25.590035   \n",
       "167  CO2 (fossil)  MtCO2    73.011816    68.118190    72.769041    75.923171   \n",
       "168  CO2 (fossil)  MtCO2    19.820817    20.268858    20.950655    21.650757   \n",
       "175  CO2 (fossil)  MtCO2    13.289532    14.130215    17.388798    16.801431   \n",
       "176  CO2 (fossil)  MtCO2     5.845461     6.482326     7.281820     7.909864   \n",
       "187  CO2 (fossil)  MtCO2     0.545612     0.548953     0.569582     0.591585   \n",
       "193  CO2 (fossil)  MtCO2    40.810400    40.924000    41.037500    41.151000   \n",
       "\n",
       "     ...         2009         2010         2011         2012         2013  \\\n",
       "3    ...  5434.038774  5697.005749  5702.852710  5609.772496  5489.523630   \n",
       "8    ...   681.302764   702.701907   663.974103   668.893197   674.105611   \n",
       "13   ...  1413.887302  1539.465763  1524.550049  1549.817713  1528.040393   \n",
       "16   ...   208.858129   211.842706   216.147437   217.164181   221.185512   \n",
       "21   ...  2931.487375  2987.419348  2914.213790  2856.974915  2902.279772   \n",
       "26   ...    48.905404    51.225435    61.246852    56.406584    57.184112   \n",
       "31   ...    59.801974    56.407708    66.789075    65.497970    62.884001   \n",
       "36   ...   122.692584   139.784968   136.271811   128.427155   134.693367   \n",
       "41   ...  1799.485279  1832.718520  1730.958026  1656.426855  1740.985484   \n",
       "46   ...    22.101318    22.339847    28.709985    30.122639    30.325389   \n",
       "51   ...    93.244887   104.790282   103.309120   109.526988    94.738056   \n",
       "54   ...   113.484430    94.738898   102.790347    77.567246    73.188266   \n",
       "56   ...   197.423490   203.470836   208.228500   208.341352   210.026684   \n",
       "57   ...    55.619768    63.420261    62.790133    62.221542    60.837754   \n",
       "58   ...     7.296280     8.398117     8.520551     8.262182     8.709437   \n",
       "59   ...    54.686823    60.498810    61.561995    56.832102    55.661802   \n",
       "61   ...   191.530653   206.524810   212.735650   207.968714   209.909273   \n",
       "64   ...    74.735119    86.198840    88.845487    87.322426    85.795342   \n",
       "65   ...   128.569987   129.521926   125.700205   124.439956   124.462660   \n",
       "72   ...    15.530726    16.439274    15.425627    17.540868    15.752995   \n",
       "73   ...    10.260019    11.643962    12.124360    13.023224    13.312837   \n",
       "84   ...    10.808026    11.215204    11.074501    10.202975    11.735014   \n",
       "90   ...     4.919200     4.919200     4.919200     4.919200     4.919200   \n",
       "93   ...   433.421749   457.471950   474.450089   478.193813   486.338265   \n",
       "99   ...   616.141890   662.889335   663.506321   616.930412   614.897420   \n",
       "106  ...  6330.379121  6851.251319  7436.644571  7808.554064  8155.119229   \n",
       "111  ...   838.271028   919.961733   952.375893   964.366690   993.399335   \n",
       "116  ...  4188.614163  4564.528872  4771.264240  4789.410568  4837.997749   \n",
       "119  ...    68.594142    79.635974    79.198116    84.227215    91.221945   \n",
       "124  ...  2083.287645  2217.565662  2352.713089  2517.077426  2627.879028   \n",
       "129  ...    33.830839    34.582529    34.286817    34.325872    33.802275   \n",
       "134  ...    75.657565    79.343131    83.150809    92.082735    99.828432   \n",
       "139  ...    26.178816    19.083983    22.723168    22.630016    23.769235   \n",
       "144  ...  1262.638512  1287.145972  1347.520260  1376.403126  1401.535357   \n",
       "149  ...   196.662156   189.123397   168.643437   192.375466   193.834117   \n",
       "154  ...   197.596966   181.416237   193.019849   216.019351   229.400934   \n",
       "157  ...   195.590476   192.452535   188.395484   219.951241   219.878046   \n",
       "159  ...  1096.180805  1184.484004  1269.356520  1326.275858  1156.431352   \n",
       "160  ...   162.234437   167.418485   183.382219   198.828190   207.340475   \n",
       "161  ...     1.551723     1.562937     1.577265     1.588149     1.593403   \n",
       "162  ...    37.624196    39.870247    42.409933    55.145332    53.800110   \n",
       "164  ...   361.402422   390.720217   418.398872   433.482287   443.212629   \n",
       "167  ...   170.951207   197.616612   215.291040   232.315176   258.501550   \n",
       "168  ...    67.400468    64.009411    64.876564    66.284667    67.713874   \n",
       "175  ...    41.191744    41.439955    41.667900    42.161204    44.504316   \n",
       "176  ...    64.165142    62.584522    66.792198    66.974833    69.597430   \n",
       "187  ...     4.188982     4.515509     4.952577     5.402767     5.805969   \n",
       "193  ...    42.570000    42.570000    42.570000    42.570000    42.570000   \n",
       "\n",
       "            2014         2015         2016         2017         2018  \n",
       "3    5299.182358  5077.945208  4929.190635  4813.571128  4806.399160  \n",
       "8     675.014980   687.200447   678.450499   675.199487   676.541685  \n",
       "13   1536.131137  1510.605271  1516.043312  1531.083444  1561.854007  \n",
       "16    224.288113   230.979320   239.397098   250.956904   254.104536  \n",
       "21   2909.505739  2983.026499  3010.393941  3032.812185  3042.232143  \n",
       "26     57.149435    59.446145    57.974854    58.674508    59.333129  \n",
       "31     59.939346    61.688319    61.607988    61.152662    61.139501  \n",
       "36    125.875656   124.428583   131.610487   133.308594   142.276621  \n",
       "41   1674.202368  1667.363322  1669.706138  1698.416411  1743.825979  \n",
       "46     29.962261    29.948848    29.505735    28.014742    27.777712  \n",
       "51     90.175374   120.173820   100.622461   108.655472   107.413885  \n",
       "54     77.043578    79.850279    78.059969    74.873738    86.814592  \n",
       "56    215.595958   210.448760   211.044965   216.719631   217.435656  \n",
       "57     61.043851    58.408170    57.107250    58.403250    59.041500  \n",
       "58      8.075784     7.755317     7.640533     7.552652     7.487444  \n",
       "59     56.312477    54.243322    53.547126    52.899752    52.042818  \n",
       "61    209.372942   217.631730   219.316223   227.640213   229.836230  \n",
       "64     84.873300    82.447748    81.257518    82.014755    82.247114  \n",
       "65    125.046265   124.982794   124.508358   127.110742   127.784554  \n",
       "72     16.445840    15.856211    15.156252    15.384944    15.956536  \n",
       "73     13.791025    14.473040    15.471824    15.593926    14.694288  \n",
       "84     10.183195    10.936423    10.902948    10.632786    10.520028  \n",
       "90      4.919200     4.919200     4.919200     4.919200     4.919200  \n",
       "93    502.588404   530.376870   553.786633   584.863307   609.402253  \n",
       "99    636.150763   660.114846   678.676733   697.135045   703.568764  \n",
       "106  8305.291302  8350.858858  8502.831447  8813.023022  9098.673398  \n",
       "111   973.670401   943.527349   888.204006   886.144087   878.262493  \n",
       "116  4873.564068  4853.064410  4719.917569  4699.469276  4849.533218  \n",
       "119    98.732773   110.371913   119.859817   132.652504   134.203412  \n",
       "124  2707.375148  2787.817842  2862.162732  2934.617171  2968.618736  \n",
       "129    32.907673    31.214421    31.017499    31.046222    31.665284  \n",
       "134   104.772500    94.283541    97.276527   109.123498   111.957314  \n",
       "139    33.792333    37.956614    25.573964    23.294117    23.846574  \n",
       "144  1422.586527  1449.760662  1474.621951  1512.691057  1561.795249  \n",
       "149   192.246426   187.286185   175.630912   171.881567   175.947018  \n",
       "154   271.948773   298.748228   307.269490   333.390693   340.755272  \n",
       "157   225.941368   226.104882   232.502869   221.920100   219.109310  \n",
       "159  1191.198661  1164.515613  1192.483386  1216.988889  1230.617356  \n",
       "160   207.692133   222.596250   252.635250   253.307250   261.049500  \n",
       "161     1.599149     1.606591     1.615320     1.624116     1.632998  \n",
       "162    58.033683    61.136953    61.640663    65.958367    69.359377  \n",
       "164   455.948287   461.393466   468.210308   474.369416   477.794156  \n",
       "167   285.161552   285.330759   287.343546   300.090507   316.478164  \n",
       "168    68.787320    68.243710    69.882354    71.911656    73.267610  \n",
       "175    48.443468    43.806236    44.372132    45.025034    45.767186  \n",
       "176    66.410853    67.597757    65.130810    64.352208    63.198340  \n",
       "187     5.763262     5.875250     5.986389     6.094731     6.202563  \n",
       "193    42.570000    42.570000    42.570000    42.570000    42.570000  \n",
       "\n",
       "[48 rows x 55 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> EDGAR <<<<<<<<\")\n",
    "\n",
    "\"\"\"\n",
    "IMPORTANT: EDGAR website separates fossil CO2 emissions and biomass (organic) emissions. In their main methodology paper they exclude biomass emissions\n",
    "(and other LULUCF emission), so we will do the same here too. To reiterate - biomass (organic) emissions are OMITTED here, but has its own dataset (\"df_co2_org\").\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "#separate co2-organic (biomass) dataset (excluded from main dataset)\n",
    "df_co2_org = pd.read_excel(\"../raw_data/\" + \"EDGAR/v60_CO2_org_short-cycle_C_1970_2018.xls\", 0, header=9)\n",
    "df_co2_org.insert(7,\"gas\",\"CO2 (bio)\")\n",
    "\n",
    "#read MAIN data\n",
    "df_co2_fossil = pd.read_excel(\"../raw_data/\" + \"EDGAR/v60_CO2_excl_short-cycle_org_C_1970_2018.xls\", 0, header=9)\n",
    "df_co2_fossil.insert(7,\"gas\",\"CO2 (fossil)\")\n",
    "df_ch4 = pd.read_excel(\"../raw_data/\" + \"EDGAR/v60_CH4_1970_2018.xls\", 0, header=9)\n",
    "df_ch4.insert(7,\"gas\",\"CH4\")\n",
    "df_n2o = pd.read_excel(\"../raw_data/\" + \"EDGAR/v60_N2O_1970_2018.xls\", 0, header=9)\n",
    "df_n2o.insert(7,\"gas\",\"N2O\")\n",
    "\n",
    "#convert into one dataframe\n",
    "df_edgar = pd.concat([df_ch4, df_co2_fossil, df_n2o]).reset_index().drop(\"index\", axis=1)\n",
    "\n",
    "#clean\n",
    "df_edgar = df_edgar.rename(columns={\"Name\":\"name\"})\n",
    "df_edgar = df_edgar.rename(columns={\"Country_code_A3\":\"code\"})\n",
    "\n",
    "df_edgar.columns = df_edgar.columns.str.replace(\"Y_\",\"\")\n",
    "    #df_edgar.ipcc_code_2006_for_standard_report = df_edgar.ipcc_code_2006_for_standard_report.str.replace(\".\",\"\")\n",
    "    \n",
    "# IMPORTANT: CONVERT TO CO2equivalents!\n",
    "GWP_100_ipcc_ar6 = {\n",
    "        \"CO2 (fossil)\" : 1,\n",
    "        \"CO2 (bio)\" : 1,\n",
    "        \"CH4\" : 27.9,\n",
    "        \"N2O\" : 273,\n",
    "        }\n",
    "for i in range(len(df_edgar)):\n",
    "    df_edgar.loc[i,\"1970\":] = df_edgar.loc[i,\"1970\":] * GWP_100_ipcc_ar6[df_edgar.loc[i,\"gas\"]]\n",
    "\n",
    "# divide by 1000 (converting gigagrams to Mt)    \n",
    "df_edgar.loc[:,\"1970\":] = df_edgar.loc[:,\"1970\":] / 1000\n",
    "\n",
    "#add unit column\n",
    "df_edgar.insert(8,\"unit\",\"MtCO2\")\n",
    "\n",
    "#separate co2-organic (biomass) dataset\n",
    "df_co2_org = pd.read_excel(\"../raw_data/\" + \"EDGAR/v60_CO2_org_short-cycle_C_1970_2018.xls\", 0, header=9)\n",
    "df_co2_org.insert(7,\"gas\",\"CO2 (bio)\")\n",
    "\n",
    "\n",
    "#no need to make codes nor names! \n",
    "# woooo! (except if I want to make codes for country groups such as \"IPCC_annex\" or \"C_group_IM24_sh\")\n",
    "\n",
    "\n",
    "#global! (not inlcluded in main)\n",
    "df_edgar_global = df_edgar.groupby([\"IPCC_annex\",\"ipcc_code_2006_for_standard_report\", \"ipcc_code_2006_for_standard_report_name\",\"fossil_bio\", \"gas\", \"unit\"]).sum().reset_index()\n",
    "df_edgar_global.to_csv(\"../clean_data/global_subsets/EDGAR_global.csv\")\n",
    "\n",
    "print(\"NO DUPLICATES HERE!\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_edgar.to_csv(\"../clean_data/EDGAR.csv\", index=None)\n",
    "df_edgar = pd.read_csv(\"../clean_data/EDGAR.csv\")\n",
    "df_edgar_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57fb31a4-67b7-4f59-b7ed-9e6928fbafbc",
   "metadata": {},
   "source": [
    "# PRIMAP-hist (Version 2.31)\n",
    "INFO:\n",
    "- Potsdam-Institut für Klimafolgenforschung\n",
    "- Gütschow, J.; Jeffery, L.; Gieseke, R.; Gebel, R.; Stevens, D.; Krapp, M.; Rocha, M. (2016): The PRIMAP-hist national historical emissions time series, Earth Syst. Sci. Data, 8, 571-603, https://doi.org/10.5194/essd-8-571-2016\n",
    "- Dataset downloaded from https://zenodo.org/record/5494497 (Guetschow-et-al-2021-PRIMAP-hist_v2.3.1_20-Sep_2021.csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "ed490405-f089-4f9d-9401-76e7427f51be",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> PRIMAP-hist <<<<<<<<\n",
      "MISSING names:\n",
      " ['ANNEXI', 'AOSIS', 'BASIC', 'EARTH', 'EU27BX', 'LDC', 'NONANNEXI', 'UMBRELLA']\n",
      "\n",
      ">>> 0 DUPLICATES:\n",
      "NO DUPLICATES HERE!\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>scenario (PRIMAP-hist)</th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>entity</th>\n",
       "      <th>unit</th>\n",
       "      <th>category (IPCC2006_PRIMAP)</th>\n",
       "      <th>1750</th>\n",
       "      <th>1751</th>\n",
       "      <th>1752</th>\n",
       "      <th>...</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3799</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTCR</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>CH4</td>\n",
       "      <td>Gg CH4 / yr</td>\n",
       "      <td>1</td>\n",
       "      <td>5450.0</td>\n",
       "      <td>5470.0</td>\n",
       "      <td>5480.0</td>\n",
       "      <td>...</td>\n",
       "      <td>110000.0</td>\n",
       "      <td>112000.00</td>\n",
       "      <td>112000.00</td>\n",
       "      <td>109000.00</td>\n",
       "      <td>108000.00</td>\n",
       "      <td>107000.00</td>\n",
       "      <td>107000.00</td>\n",
       "      <td>107000.00</td>\n",
       "      <td>108000.0</td>\n",
       "      <td>111000.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3800</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTCR</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>CH4</td>\n",
       "      <td>Gg CH4 / yr</td>\n",
       "      <td>1.A</td>\n",
       "      <td>5400.0</td>\n",
       "      <td>5410.0</td>\n",
       "      <td>5430.0</td>\n",
       "      <td>...</td>\n",
       "      <td>11000.0</td>\n",
       "      <td>10900.00</td>\n",
       "      <td>10900.00</td>\n",
       "      <td>10900.00</td>\n",
       "      <td>10900.00</td>\n",
       "      <td>11000.00</td>\n",
       "      <td>10900.00</td>\n",
       "      <td>11000.00</td>\n",
       "      <td>11100.0</td>\n",
       "      <td>10900.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3801</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTCR</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>CH4</td>\n",
       "      <td>Gg CH4 / yr</td>\n",
       "      <td>1.B</td>\n",
       "      <td>53.0</td>\n",
       "      <td>53.0</td>\n",
       "      <td>53.1</td>\n",
       "      <td>...</td>\n",
       "      <td>98700.0</td>\n",
       "      <td>101000.00</td>\n",
       "      <td>101000.00</td>\n",
       "      <td>98500.00</td>\n",
       "      <td>96700.00</td>\n",
       "      <td>96500.00</td>\n",
       "      <td>96000.00</td>\n",
       "      <td>96300.00</td>\n",
       "      <td>96900.0</td>\n",
       "      <td>99600.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3802</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTCR</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>CH4</td>\n",
       "      <td>Gg CH4 / yr</td>\n",
       "      <td>1.B.1</td>\n",
       "      <td>53.0</td>\n",
       "      <td>53.0</td>\n",
       "      <td>53.1</td>\n",
       "      <td>...</td>\n",
       "      <td>37800.0</td>\n",
       "      <td>40000.00</td>\n",
       "      <td>39100.00</td>\n",
       "      <td>37300.00</td>\n",
       "      <td>35700.00</td>\n",
       "      <td>35100.00</td>\n",
       "      <td>33600.00</td>\n",
       "      <td>33900.00</td>\n",
       "      <td>33500.0</td>\n",
       "      <td>36200.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3803</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTCR</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>CH4</td>\n",
       "      <td>Gg CH4 / yr</td>\n",
       "      <td>1.B.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>60800.0</td>\n",
       "      <td>61100.00</td>\n",
       "      <td>61600.00</td>\n",
       "      <td>61000.00</td>\n",
       "      <td>60900.00</td>\n",
       "      <td>61300.00</td>\n",
       "      <td>62300.00</td>\n",
       "      <td>62300.00</td>\n",
       "      <td>63300.0</td>\n",
       "      <td>63300.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18029</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTTP</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PFCS (AR4GWP100)</td>\n",
       "      <td>Gg CO2 / yr</td>\n",
       "      <td>M.0.EL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>148000.0</td>\n",
       "      <td>115000.00</td>\n",
       "      <td>112000.00</td>\n",
       "      <td>114000.00</td>\n",
       "      <td>113000.00</td>\n",
       "      <td>113000.00</td>\n",
       "      <td>122000.00</td>\n",
       "      <td>121000.00</td>\n",
       "      <td>120000.0</td>\n",
       "      <td>120000.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18030</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTTP</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PFCS (SARGWP100)</td>\n",
       "      <td>Gg CO2 / yr</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>126000.0</td>\n",
       "      <td>98200.00</td>\n",
       "      <td>95700.00</td>\n",
       "      <td>97300.00</td>\n",
       "      <td>96600.00</td>\n",
       "      <td>96300.00</td>\n",
       "      <td>104000.00</td>\n",
       "      <td>103000.00</td>\n",
       "      <td>102000.0</td>\n",
       "      <td>102000.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18031</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTTP</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PFCS (SARGWP100)</td>\n",
       "      <td>Gg CO2 / yr</td>\n",
       "      <td>M.0.EL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>126000.0</td>\n",
       "      <td>98200.00</td>\n",
       "      <td>95700.00</td>\n",
       "      <td>97300.00</td>\n",
       "      <td>96600.00</td>\n",
       "      <td>96300.00</td>\n",
       "      <td>104000.00</td>\n",
       "      <td>103000.00</td>\n",
       "      <td>102000.0</td>\n",
       "      <td>102000.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18032</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTTP</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>SF6</td>\n",
       "      <td>Gg SF6 / yr</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>5.3</td>\n",
       "      <td>5.35</td>\n",
       "      <td>7.55</td>\n",
       "      <td>7.87</td>\n",
       "      <td>6.43</td>\n",
       "      <td>6.24</td>\n",
       "      <td>6.35</td>\n",
       "      <td>6.68</td>\n",
       "      <td>7.2</td>\n",
       "      <td>7.13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18033</th>\n",
       "      <td>PRIMAP-hist_v2.3.1</td>\n",
       "      <td>HISTTP</td>\n",
       "      <td>EARTH</td>\n",
       "      <td>NaN</td>\n",
       "      <td>SF6</td>\n",
       "      <td>Gg SF6 / yr</td>\n",
       "      <td>M.0.EL</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>5.3</td>\n",
       "      <td>5.35</td>\n",
       "      <td>7.55</td>\n",
       "      <td>7.87</td>\n",
       "      <td>6.43</td>\n",
       "      <td>6.24</td>\n",
       "      <td>6.35</td>\n",
       "      <td>6.68</td>\n",
       "      <td>7.2</td>\n",
       "      <td>7.13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>190 rows × 277 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   source scenario (PRIMAP-hist)   code name  \\\n",
       "3799   PRIMAP-hist_v2.3.1                 HISTCR  EARTH  NaN   \n",
       "3800   PRIMAP-hist_v2.3.1                 HISTCR  EARTH  NaN   \n",
       "3801   PRIMAP-hist_v2.3.1                 HISTCR  EARTH  NaN   \n",
       "3802   PRIMAP-hist_v2.3.1                 HISTCR  EARTH  NaN   \n",
       "3803   PRIMAP-hist_v2.3.1                 HISTCR  EARTH  NaN   \n",
       "...                   ...                    ...    ...  ...   \n",
       "18029  PRIMAP-hist_v2.3.1                 HISTTP  EARTH  NaN   \n",
       "18030  PRIMAP-hist_v2.3.1                 HISTTP  EARTH  NaN   \n",
       "18031  PRIMAP-hist_v2.3.1                 HISTTP  EARTH  NaN   \n",
       "18032  PRIMAP-hist_v2.3.1                 HISTTP  EARTH  NaN   \n",
       "18033  PRIMAP-hist_v2.3.1                 HISTTP  EARTH  NaN   \n",
       "\n",
       "                 entity         unit category (IPCC2006_PRIMAP)    1750  \\\n",
       "3799                CH4  Gg CH4 / yr                          1  5450.0   \n",
       "3800                CH4  Gg CH4 / yr                        1.A  5400.0   \n",
       "3801                CH4  Gg CH4 / yr                        1.B    53.0   \n",
       "3802                CH4  Gg CH4 / yr                      1.B.1    53.0   \n",
       "3803                CH4  Gg CH4 / yr                      1.B.2     0.0   \n",
       "...                 ...          ...                        ...     ...   \n",
       "18029  PFCS (AR4GWP100)  Gg CO2 / yr                     M.0.EL     0.0   \n",
       "18030  PFCS (SARGWP100)  Gg CO2 / yr                          2     0.0   \n",
       "18031  PFCS (SARGWP100)  Gg CO2 / yr                     M.0.EL     0.0   \n",
       "18032               SF6  Gg SF6 / yr                          2     0.0   \n",
       "18033               SF6  Gg SF6 / yr                     M.0.EL     0.0   \n",
       "\n",
       "         1751    1752  ...      2010       2011       2012       2013  \\\n",
       "3799   5470.0  5480.0  ...  110000.0  112000.00  112000.00  109000.00   \n",
       "3800   5410.0  5430.0  ...   11000.0   10900.00   10900.00   10900.00   \n",
       "3801     53.0    53.1  ...   98700.0  101000.00  101000.00   98500.00   \n",
       "3802     53.0    53.1  ...   37800.0   40000.00   39100.00   37300.00   \n",
       "3803      0.0     0.0  ...   60800.0   61100.00   61600.00   61000.00   \n",
       "...       ...     ...  ...       ...        ...        ...        ...   \n",
       "18029     0.0     0.0  ...  148000.0  115000.00  112000.00  114000.00   \n",
       "18030     0.0     0.0  ...  126000.0   98200.00   95700.00   97300.00   \n",
       "18031     0.0     0.0  ...  126000.0   98200.00   95700.00   97300.00   \n",
       "18032     0.0     0.0  ...       5.3       5.35       7.55       7.87   \n",
       "18033     0.0     0.0  ...       5.3       5.35       7.55       7.87   \n",
       "\n",
       "            2014       2015       2016       2017      2018       2019  \n",
       "3799   108000.00  107000.00  107000.00  107000.00  108000.0  111000.00  \n",
       "3800    10900.00   11000.00   10900.00   11000.00   11100.0   10900.00  \n",
       "3801    96700.00   96500.00   96000.00   96300.00   96900.0   99600.00  \n",
       "3802    35700.00   35100.00   33600.00   33900.00   33500.0   36200.00  \n",
       "3803    60900.00   61300.00   62300.00   62300.00   63300.0   63300.00  \n",
       "...          ...        ...        ...        ...       ...        ...  \n",
       "18029  113000.00  113000.00  122000.00  121000.00  120000.0  120000.00  \n",
       "18030   96600.00   96300.00  104000.00  103000.00  102000.0  102000.00  \n",
       "18031   96600.00   96300.00  104000.00  103000.00  102000.0  102000.00  \n",
       "18032       6.43       6.24       6.35       6.68       7.2       7.13  \n",
       "18033       6.43       6.24       6.35       6.68       7.2       7.13  \n",
       "\n",
       "[190 rows x 277 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> PRIMAP-hist <<<<<<<<\")\n",
    "\n",
    "\"\"\"\n",
    "Warning! CH4, N2O, NF3 and SF6 have not been converted to CO2eqs yet!\n",
    "\"\"\"\n",
    "\n",
    "df_primap = pd.read_csv(\"../raw_data/PRIMAP-hist/Guetschow-et-al-2021-PRIMAP-hist_v2.3.1_20-Sep_2021.csv\")\n",
    "\n",
    "#clean\n",
    "df_primap = df_primap.rename(columns={\"area (ISO3)\":\"code\"}) \n",
    "\n",
    "\n",
    "# COUNTRY_DICT NOT NECESSARY HERE, INVERSE (creating names out of ISO codes)\n",
    "df_primap.insert(3, \"name\", np.nan)\n",
    "COUNTRY_NAME_GENERATOR_FROM_ISO3(df_primap, show_missing = True, show_duplicates_nan = True)\n",
    "\n",
    "\n",
    "#global\n",
    "df_primap_global = df_primap[df_primap.code==\"EARTH\"]\n",
    "df_primap_global.to_csv(\"../clean_data/global_subsets/PRIMAP-hist_global.csv\")#[df_primap_global[\"category (IPCC2006_PRIMAP)\"]==\"M.0.EL\"].groupby([\"entity\",\"unit\",\"scenario (PRIMAP-hist)\"]).sum()[\"2019\"]/1000\n",
    "\n",
    "print(\"NO DUPLICATES HERE!\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_primap.to_csv(\"../clean_data/PRIMAP-hist.csv\", index=None)\n",
    "df_primap = pd.read_csv(\"../clean_data/PRIMAP-hist.csv\")\n",
    "df_primap_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "176303ef-8ca8-4ce2-997d-63589723cc74",
   "metadata": {},
   "source": [
    "# Minx et al 2021\n",
    "INFO:\n",
    "- A comprehensive and synthetic dataset for global, regional and national greenhouse gas emissions by sector 1970-2018 with an extension to 2019 \n",
    "- VERSION 5 from Zenodo https://zenodo.org/record/5844489\n",
    "- Using *essd_ghg_data.xlsx* file. Gives two convenient columns for ar4 and ar5 GWPs\n",
    "\n",
    "NOTES:\n",
    "- 2020 non-CO2 emissions seem to be incomplete"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "5692b87a-574f-46d3-b7eb-96a197139a21",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> Minx et al 2021 <<<<<<<<\n",
      "NO DUPLICATES HERE!\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>region_ar6_6</th>\n",
       "      <th>region_ar6_10</th>\n",
       "      <th>region_ar6_22</th>\n",
       "      <th>region_ar6_dev</th>\n",
       "      <th>year</th>\n",
       "      <th>sector_title</th>\n",
       "      <th>subsector_title</th>\n",
       "      <th>gas</th>\n",
       "      <th>gwp100_ar5</th>\n",
       "      <th>gwp100_ar6</th>\n",
       "      <th>value_native</th>\n",
       "      <th>value_ar5</th>\n",
       "      <th>value_ar6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ABW</td>\n",
       "      <td>Aruba</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Caribbean</td>\n",
       "      <td>developing</td>\n",
       "      <td>1970</td>\n",
       "      <td>Buildings</td>\n",
       "      <td>Residential</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5.452099e+02</td>\n",
       "      <td>5.452099e+02</td>\n",
       "      <td>5.452099e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ABW</td>\n",
       "      <td>Aruba</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Caribbean</td>\n",
       "      <td>developing</td>\n",
       "      <td>1970</td>\n",
       "      <td>Buildings</td>\n",
       "      <td>Residential</td>\n",
       "      <td>CH4</td>\n",
       "      <td>28</td>\n",
       "      <td>27.0</td>\n",
       "      <td>1.609412e-01</td>\n",
       "      <td>4.506354e+00</td>\n",
       "      <td>4.345413e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ABW</td>\n",
       "      <td>Aruba</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Caribbean</td>\n",
       "      <td>developing</td>\n",
       "      <td>1970</td>\n",
       "      <td>Buildings</td>\n",
       "      <td>Residential</td>\n",
       "      <td>N2O</td>\n",
       "      <td>265</td>\n",
       "      <td>273.0</td>\n",
       "      <td>4.802031e-03</td>\n",
       "      <td>1.272538e+00</td>\n",
       "      <td>1.310954e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ABW</td>\n",
       "      <td>Aruba</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Caribbean</td>\n",
       "      <td>developing</td>\n",
       "      <td>1970</td>\n",
       "      <td>Energy systems</td>\n",
       "      <td>Electricity &amp; heat</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.091397e+04</td>\n",
       "      <td>3.091397e+04</td>\n",
       "      <td>3.091397e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ABW</td>\n",
       "      <td>Aruba</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Latin America and Caribbean</td>\n",
       "      <td>Caribbean</td>\n",
       "      <td>developing</td>\n",
       "      <td>1970</td>\n",
       "      <td>Energy systems</td>\n",
       "      <td>Electricity &amp; heat</td>\n",
       "      <td>CH4</td>\n",
       "      <td>28</td>\n",
       "      <td>27.0</td>\n",
       "      <td>4.155721e-01</td>\n",
       "      <td>1.163602e+01</td>\n",
       "      <td>1.122045e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568336</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>2020</td>\n",
       "      <td>Industry</td>\n",
       "      <td>Metals</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.720092e+05</td>\n",
       "      <td>7.720092e+05</td>\n",
       "      <td>7.720092e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568337</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>2020</td>\n",
       "      <td>Industry</td>\n",
       "      <td>Other (industry)</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>9.523549e+05</td>\n",
       "      <td>9.523549e+05</td>\n",
       "      <td>9.523549e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568338</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>2020</td>\n",
       "      <td>Transport</td>\n",
       "      <td>Other (transport)</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.099450e+05</td>\n",
       "      <td>3.099450e+05</td>\n",
       "      <td>3.099450e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568339</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>2020</td>\n",
       "      <td>Transport</td>\n",
       "      <td>Rail</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.491236e+05</td>\n",
       "      <td>1.491236e+05</td>\n",
       "      <td>1.491236e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568340</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>2020</td>\n",
       "      <td>Transport</td>\n",
       "      <td>Road</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.236932e+06</td>\n",
       "      <td>2.236932e+06</td>\n",
       "      <td>2.236932e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>568341 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       code      name                 region_ar6_6  \\\n",
       "0       ABW     Aruba  Latin America and Caribbean   \n",
       "1       ABW     Aruba  Latin America and Caribbean   \n",
       "2       ABW     Aruba  Latin America and Caribbean   \n",
       "3       ABW     Aruba  Latin America and Caribbean   \n",
       "4       ABW     Aruba  Latin America and Caribbean   \n",
       "...     ...       ...                          ...   \n",
       "568336  ZWE  Zimbabwe                       Africa   \n",
       "568337  ZWE  Zimbabwe                       Africa   \n",
       "568338  ZWE  Zimbabwe                       Africa   \n",
       "568339  ZWE  Zimbabwe                       Africa   \n",
       "568340  ZWE  Zimbabwe                       Africa   \n",
       "\n",
       "                      region_ar6_10   region_ar6_22 region_ar6_dev  year  \\\n",
       "0       Latin America and Caribbean       Caribbean     developing  1970   \n",
       "1       Latin America and Caribbean       Caribbean     developing  1970   \n",
       "2       Latin America and Caribbean       Caribbean     developing  1970   \n",
       "3       Latin America and Caribbean       Caribbean     developing  1970   \n",
       "4       Latin America and Caribbean       Caribbean     developing  1970   \n",
       "...                             ...             ...            ...   ...   \n",
       "568336                       Africa  Eastern Africa     developing  2020   \n",
       "568337                       Africa  Eastern Africa     developing  2020   \n",
       "568338                       Africa  Eastern Africa     developing  2020   \n",
       "568339                       Africa  Eastern Africa     developing  2020   \n",
       "568340                       Africa  Eastern Africa     developing  2020   \n",
       "\n",
       "          sector_title     subsector_title  gas  gwp100_ar5  gwp100_ar6  \\\n",
       "0            Buildings         Residential  CO2           1         1.0   \n",
       "1            Buildings         Residential  CH4          28        27.0   \n",
       "2            Buildings         Residential  N2O         265       273.0   \n",
       "3       Energy systems  Electricity & heat  CO2           1         1.0   \n",
       "4       Energy systems  Electricity & heat  CH4          28        27.0   \n",
       "...                ...                 ...  ...         ...         ...   \n",
       "568336        Industry              Metals  CO2           1         1.0   \n",
       "568337        Industry    Other (industry)  CO2           1         1.0   \n",
       "568338       Transport   Other (transport)  CO2           1         1.0   \n",
       "568339       Transport               Rail   CO2           1         1.0   \n",
       "568340       Transport                Road  CO2           1         1.0   \n",
       "\n",
       "        value_native     value_ar5     value_ar6  \n",
       "0       5.452099e+02  5.452099e+02  5.452099e+02  \n",
       "1       1.609412e-01  4.506354e+00  4.345413e+00  \n",
       "2       4.802031e-03  1.272538e+00  1.310954e+00  \n",
       "3       3.091397e+04  3.091397e+04  3.091397e+04  \n",
       "4       4.155721e-01  1.163602e+01  1.122045e+01  \n",
       "...              ...           ...           ...  \n",
       "568336  7.720092e+05  7.720092e+05  7.720092e+05  \n",
       "568337  9.523549e+05  9.523549e+05  9.523549e+05  \n",
       "568338  3.099450e+05  3.099450e+05  3.099450e+05  \n",
       "568339  1.491236e+05  1.491236e+05  1.491236e+05  \n",
       "568340  2.236932e+06  2.236932e+06  2.236932e+06  \n",
       "\n",
       "[568341 rows x 15 columns]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> Minx et al 2021 <<<<<<<<\")\n",
    "\n",
    "\"\"\"\n",
    "IPCC categorization cannot be integrated into the main dataset (yet). This is because there are various subsectors (and some sectors such as INdustry and Energy) that overlap.\n",
    "\"\"\"\n",
    "#read file (excel with multiple interesting sheets)\n",
    "# minx_info = pd.read_excel(\"raw_data/Minx/essd_ghg_data.xlsx\",0) #only info\n",
    "# minx_metadata = pd.read_excel(\"raw_data/Minx/essd_ghg_data.xlsx\",1) #not very useful\n",
    "minx_data = pd.read_excel(\"../raw_data/Minx/essd_ghg_data.xlsx\",2) #MAIN DATA \n",
    "# minx_sector_classification = pd.read_excel(\"raw_data/Minx/essd_ghg_data.xlsx\",3) #includes description and IPCC categories!\n",
    "# minx_region_classification = pd.read_excel(\"raw_data/Minx/essd_ghg_data.xlsx\",4) #not useful\n",
    "# minx_100_yr_gwps = pd.read_excel(\"raw_data/Minx/essd_ghg_data.xlsx\",5) #both ar5 and ar6 GWPS for all included gases\n",
    "# minx_CH4_gwps = pd.read_excel(\"raw_data/Minx/essd_ghg_data.xlsx\",6) # CH4 classification and chapters\n",
    "\n",
    "#choose dataset\n",
    "df_minx = minx_data\n",
    "\n",
    "#clean a little\n",
    "df_minx = df_minx.rename(columns={\"country\":\"name\", \"ISO\":\"code\", \"value\":\"value_native\"})\n",
    "\n",
    "# add columns for both ar5 and ar6 gwps\n",
    "df_minx = df_minx.assign(value_ar5=df_minx.value_native*df_minx.gwp100_ar5)\n",
    "df_minx = df_minx.assign(value_ar6=df_minx.value_native*df_minx.gwp100_ar6)\n",
    "\n",
    "#codes and names are amazing\n",
    "print(\"NO DUPLICATES HERE!\")\n",
    "\n",
    "#global\n",
    "df_minx_global = df_minx.groupby([\"year\",\"region_ar6_6\",\"region_ar6_10\",\"region_ar6_22\",\"region_ar6_dev\",\"sector_title\",\"subsector_title\",\"gas\",\"gwp100_ar5\", \"gwp100_ar6\"]).sum()\n",
    "df_minx_global = df_minx_global.reset_index()\n",
    "df_minx_global.to_csv(\"../clean_data/global_subsets/Minx2021_global.csv\")\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_minx.to_csv(\"../clean_data/Minx2021.csv\", index=None)\n",
    "df_minx = pd.read_csv(\"../clean_data/Minx2021.csv\")\n",
    "df_minx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "f801a97e-9c04-4db3-8053-7e5c1f5f99a2",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>region_ar6_6</th>\n",
       "      <th>region_ar6_10</th>\n",
       "      <th>region_ar6_22</th>\n",
       "      <th>region_ar6_dev</th>\n",
       "      <th>sector_title</th>\n",
       "      <th>subsector_title</th>\n",
       "      <th>gas</th>\n",
       "      <th>gwp100_ar5</th>\n",
       "      <th>gwp100_ar6</th>\n",
       "      <th>value_native</th>\n",
       "      <th>value_ar5</th>\n",
       "      <th>value_ar6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1970</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>AFOLU</td>\n",
       "      <td>Biomass burning (CH4, N2O)</td>\n",
       "      <td>CH4</td>\n",
       "      <td>28</td>\n",
       "      <td>27.0</td>\n",
       "      <td>4.444608e+03</td>\n",
       "      <td>1.244490e+05</td>\n",
       "      <td>1.200044e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1970</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>AFOLU</td>\n",
       "      <td>Biomass burning (CH4, N2O)</td>\n",
       "      <td>N2O</td>\n",
       "      <td>265</td>\n",
       "      <td>273.0</td>\n",
       "      <td>1.152306e+02</td>\n",
       "      <td>3.053610e+04</td>\n",
       "      <td>3.145794e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1970</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>AFOLU</td>\n",
       "      <td>Enteric Fermentation (CH4)</td>\n",
       "      <td>CH4</td>\n",
       "      <td>28</td>\n",
       "      <td>27.0</td>\n",
       "      <td>5.980416e+05</td>\n",
       "      <td>1.674517e+07</td>\n",
       "      <td>1.614712e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1970</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>AFOLU</td>\n",
       "      <td>Managed soils and pasture (CO2, N2O)</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.944938e+04</td>\n",
       "      <td>4.944938e+04</td>\n",
       "      <td>4.944938e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1970</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Africa</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>developing</td>\n",
       "      <td>AFOLU</td>\n",
       "      <td>Managed soils and pasture (CO2, N2O)</td>\n",
       "      <td>N2O</td>\n",
       "      <td>265</td>\n",
       "      <td>273.0</td>\n",
       "      <td>2.585428e+04</td>\n",
       "      <td>6.851385e+06</td>\n",
       "      <td>7.058220e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110296</th>\n",
       "      <td>2020</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>ldc</td>\n",
       "      <td>Industry</td>\n",
       "      <td>Cement</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>8.939050e+05</td>\n",
       "      <td>8.939050e+05</td>\n",
       "      <td>8.939050e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110297</th>\n",
       "      <td>2020</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>ldc</td>\n",
       "      <td>Industry</td>\n",
       "      <td>Chemicals</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.910756e+05</td>\n",
       "      <td>3.910756e+05</td>\n",
       "      <td>3.910756e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110298</th>\n",
       "      <td>2020</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>ldc</td>\n",
       "      <td>Industry</td>\n",
       "      <td>Other (industry)</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>9.325430e+05</td>\n",
       "      <td>9.325430e+05</td>\n",
       "      <td>9.325430e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110299</th>\n",
       "      <td>2020</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>ldc</td>\n",
       "      <td>Transport</td>\n",
       "      <td>Other (transport)</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.180720e+05</td>\n",
       "      <td>1.180720e+05</td>\n",
       "      <td>1.180720e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110300</th>\n",
       "      <td>2020</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>Middle East</td>\n",
       "      <td>ldc</td>\n",
       "      <td>Transport</td>\n",
       "      <td>Road</td>\n",
       "      <td>CO2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.415364e+06</td>\n",
       "      <td>2.415364e+06</td>\n",
       "      <td>2.415364e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>110301 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        year region_ar6_6 region_ar6_10   region_ar6_22 region_ar6_dev  \\\n",
       "0       1970       Africa        Africa  Eastern Africa     developing   \n",
       "1       1970       Africa        Africa  Eastern Africa     developing   \n",
       "2       1970       Africa        Africa  Eastern Africa     developing   \n",
       "3       1970       Africa        Africa  Eastern Africa     developing   \n",
       "4       1970       Africa        Africa  Eastern Africa     developing   \n",
       "...      ...          ...           ...             ...            ...   \n",
       "110296  2020  Middle East   Middle East     Middle East            ldc   \n",
       "110297  2020  Middle East   Middle East     Middle East            ldc   \n",
       "110298  2020  Middle East   Middle East     Middle East            ldc   \n",
       "110299  2020  Middle East   Middle East     Middle East            ldc   \n",
       "110300  2020  Middle East   Middle East     Middle East            ldc   \n",
       "\n",
       "       sector_title                       subsector_title  gas  gwp100_ar5  \\\n",
       "0             AFOLU            Biomass burning (CH4, N2O)  CH4          28   \n",
       "1             AFOLU            Biomass burning (CH4, N2O)  N2O         265   \n",
       "2             AFOLU            Enteric Fermentation (CH4)  CH4          28   \n",
       "3             AFOLU  Managed soils and pasture (CO2, N2O)  CO2           1   \n",
       "4             AFOLU  Managed soils and pasture (CO2, N2O)  N2O         265   \n",
       "...             ...                                   ...  ...         ...   \n",
       "110296     Industry                                Cement  CO2           1   \n",
       "110297     Industry                             Chemicals  CO2           1   \n",
       "110298     Industry                      Other (industry)  CO2           1   \n",
       "110299    Transport                     Other (transport)  CO2           1   \n",
       "110300    Transport                                  Road  CO2           1   \n",
       "\n",
       "        gwp100_ar6  value_native     value_ar5     value_ar6  \n",
       "0             27.0  4.444608e+03  1.244490e+05  1.200044e+05  \n",
       "1            273.0  1.152306e+02  3.053610e+04  3.145794e+04  \n",
       "2             27.0  5.980416e+05  1.674517e+07  1.614712e+07  \n",
       "3              1.0  4.944938e+04  4.944938e+04  4.944938e+04  \n",
       "4            273.0  2.585428e+04  6.851385e+06  7.058220e+06  \n",
       "...            ...           ...           ...           ...  \n",
       "110296         1.0  8.939050e+05  8.939050e+05  8.939050e+05  \n",
       "110297         1.0  3.910756e+05  3.910756e+05  3.910756e+05  \n",
       "110298         1.0  9.325430e+05  9.325430e+05  9.325430e+05  \n",
       "110299         1.0  1.180720e+05  1.180720e+05  1.180720e+05  \n",
       "110300         1.0  2.415364e+06  2.415364e+06  2.415364e+06  \n",
       "\n",
       "[110301 rows x 13 columns]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_minx_global"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "946f5e03-3082-4a87-b379-a17eecaca2f9",
   "metadata": {},
   "source": [
    "# UNFCCC Annex I Submissions\n",
    "- from Open Climate Data: https://github.com/openclimatedata\n",
    "- all values in current state are converted to CO2equivalents using AR6 GWPs, to get native units DIVIDE BY GWPs columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "7eec8e22-a742-4a39-8dd6-cbc640c51420",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> UNFCCC Annex I <<<<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 2 Missing codes (NaN) in this df\n",
      " ['European Union (Convention)', 'European Union (KP)']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>Parent Category</th>\n",
       "      <th>Category</th>\n",
       "      <th>Gas</th>\n",
       "      <th>Unit</th>\n",
       "      <th>GWP_AR4_conditional</th>\n",
       "      <th>GWP_AR6_conditional</th>\n",
       "      <th>Base year</th>\n",
       "      <th>1990</th>\n",
       "      <th>...</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "      <th>2019</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AUS</td>\n",
       "      <td>Australia</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A.1 Energy Industries</td>\n",
       "      <td>Aggregate GHGs</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>143.211000</td>\n",
       "      <td>143.211000</td>\n",
       "      <td>...</td>\n",
       "      <td>226.660000</td>\n",
       "      <td>220.712000</td>\n",
       "      <td>222.471000</td>\n",
       "      <td>211.473000</td>\n",
       "      <td>205.297000</td>\n",
       "      <td>212.042000</td>\n",
       "      <td>219.422000</td>\n",
       "      <td>218.428000</td>\n",
       "      <td>214.601000</td>\n",
       "      <td>213.814000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AUS</td>\n",
       "      <td>Australia</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A.1 Energy Industries</td>\n",
       "      <td>CH4</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>25</td>\n",
       "      <td>25</td>\n",
       "      <td>0.152966</td>\n",
       "      <td>0.152966</td>\n",
       "      <td>...</td>\n",
       "      <td>0.582400</td>\n",
       "      <td>0.451877</td>\n",
       "      <td>0.545782</td>\n",
       "      <td>0.515282</td>\n",
       "      <td>0.647907</td>\n",
       "      <td>0.727285</td>\n",
       "      <td>0.916665</td>\n",
       "      <td>0.860737</td>\n",
       "      <td>0.952913</td>\n",
       "      <td>0.887035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AUS</td>\n",
       "      <td>Australia</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A.1 Energy Industries</td>\n",
       "      <td>CO2</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>142.551000</td>\n",
       "      <td>142.551000</td>\n",
       "      <td>...</td>\n",
       "      <td>224.948000</td>\n",
       "      <td>219.013000</td>\n",
       "      <td>220.698000</td>\n",
       "      <td>209.739000</td>\n",
       "      <td>203.485000</td>\n",
       "      <td>210.371000</td>\n",
       "      <td>217.548000</td>\n",
       "      <td>216.606000</td>\n",
       "      <td>212.752000</td>\n",
       "      <td>212.031000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AUS</td>\n",
       "      <td>Australia</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A.1 Energy Industries</td>\n",
       "      <td>N2O</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>298</td>\n",
       "      <td>298</td>\n",
       "      <td>0.506877</td>\n",
       "      <td>0.506877</td>\n",
       "      <td>...</td>\n",
       "      <td>1.128845</td>\n",
       "      <td>1.246874</td>\n",
       "      <td>1.227379</td>\n",
       "      <td>1.219109</td>\n",
       "      <td>1.164060</td>\n",
       "      <td>0.943790</td>\n",
       "      <td>0.958067</td>\n",
       "      <td>0.961715</td>\n",
       "      <td>0.896387</td>\n",
       "      <td>0.896199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AUS</td>\n",
       "      <td>Australia</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A.2 Manufacturing Industries and Construction</td>\n",
       "      <td>Aggregate GHGs</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>36.256200</td>\n",
       "      <td>36.256200</td>\n",
       "      <td>...</td>\n",
       "      <td>39.742000</td>\n",
       "      <td>40.918900</td>\n",
       "      <td>42.912900</td>\n",
       "      <td>46.012100</td>\n",
       "      <td>46.383400</td>\n",
       "      <td>42.487300</td>\n",
       "      <td>40.968500</td>\n",
       "      <td>40.070500</td>\n",
       "      <td>40.950900</td>\n",
       "      <td>40.792600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58316</th>\n",
       "      <td>USA</td>\n",
       "      <td>United States of America</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions without LULUCF</td>\n",
       "      <td>NF3</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>17200</td>\n",
       "      <td>17200</td>\n",
       "      <td>0.047922</td>\n",
       "      <td>0.047922</td>\n",
       "      <td>...</td>\n",
       "      <td>0.557691</td>\n",
       "      <td>0.568919</td>\n",
       "      <td>0.572732</td>\n",
       "      <td>0.498578</td>\n",
       "      <td>0.516564</td>\n",
       "      <td>0.566444</td>\n",
       "      <td>0.569480</td>\n",
       "      <td>0.594546</td>\n",
       "      <td>0.616978</td>\n",
       "      <td>0.605124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58317</th>\n",
       "      <td>USA</td>\n",
       "      <td>United States of America</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions without LULUCF</td>\n",
       "      <td>N2O</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>298</td>\n",
       "      <td>298</td>\n",
       "      <td>452.656040</td>\n",
       "      <td>452.656040</td>\n",
       "      <td>...</td>\n",
       "      <td>454.950640</td>\n",
       "      <td>445.575560</td>\n",
       "      <td>416.764920</td>\n",
       "      <td>463.857860</td>\n",
       "      <td>473.989860</td>\n",
       "      <td>468.244420</td>\n",
       "      <td>450.793540</td>\n",
       "      <td>446.272880</td>\n",
       "      <td>459.212040</td>\n",
       "      <td>457.140940</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58318</th>\n",
       "      <td>USA</td>\n",
       "      <td>United States of America</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions without LULUCF</td>\n",
       "      <td>PFCs</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>24.255700</td>\n",
       "      <td>24.255700</td>\n",
       "      <td>...</td>\n",
       "      <td>4.737930</td>\n",
       "      <td>7.315630</td>\n",
       "      <td>6.404460</td>\n",
       "      <td>6.122770</td>\n",
       "      <td>5.760310</td>\n",
       "      <td>5.204580</td>\n",
       "      <td>4.390520</td>\n",
       "      <td>4.092780</td>\n",
       "      <td>4.699570</td>\n",
       "      <td>4.484390</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58319</th>\n",
       "      <td>USA</td>\n",
       "      <td>United States of America</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions without LULUCF</td>\n",
       "      <td>SF6</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>22800</td>\n",
       "      <td>22800</td>\n",
       "      <td>28.845648</td>\n",
       "      <td>28.845648</td>\n",
       "      <td>...</td>\n",
       "      <td>7.288636</td>\n",
       "      <td>8.207681</td>\n",
       "      <td>6.921738</td>\n",
       "      <td>6.513755</td>\n",
       "      <td>6.557417</td>\n",
       "      <td>5.490035</td>\n",
       "      <td>6.020796</td>\n",
       "      <td>5.873713</td>\n",
       "      <td>5.694847</td>\n",
       "      <td>5.902920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58320</th>\n",
       "      <td>USA</td>\n",
       "      <td>United States of America</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions without LULUCF</td>\n",
       "      <td>Unspecified mix of HFCs and PFCs</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.227404</td>\n",
       "      <td>0.227404</td>\n",
       "      <td>...</td>\n",
       "      <td>9.943280</td>\n",
       "      <td>10.644300</td>\n",
       "      <td>11.333700</td>\n",
       "      <td>12.024100</td>\n",
       "      <td>12.686100</td>\n",
       "      <td>14.077900</td>\n",
       "      <td>15.011000</td>\n",
       "      <td>15.964000</td>\n",
       "      <td>16.358800</td>\n",
       "      <td>16.462600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>58321 rows × 39 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      code                      name Parent Category  \\\n",
       "0      AUS                 Australia       1. Energy   \n",
       "1      AUS                 Australia       1. Energy   \n",
       "2      AUS                 Australia       1. Energy   \n",
       "3      AUS                 Australia       1. Energy   \n",
       "4      AUS                 Australia       1. Energy   \n",
       "...    ...                       ...             ...   \n",
       "58316  USA  United States of America          Totals   \n",
       "58317  USA  United States of America          Totals   \n",
       "58318  USA  United States of America          Totals   \n",
       "58319  USA  United States of America          Totals   \n",
       "58320  USA  United States of America          Totals   \n",
       "\n",
       "                                              Category  \\\n",
       "0                              1.A.1 Energy Industries   \n",
       "1                              1.A.1 Energy Industries   \n",
       "2                              1.A.1 Energy Industries   \n",
       "3                              1.A.1 Energy Industries   \n",
       "4      1.A.2 Manufacturing Industries and Construction   \n",
       "...                                                ...   \n",
       "58316               Total GHG emissions without LULUCF   \n",
       "58317               Total GHG emissions without LULUCF   \n",
       "58318               Total GHG emissions without LULUCF   \n",
       "58319               Total GHG emissions without LULUCF   \n",
       "58320               Total GHG emissions without LULUCF   \n",
       "\n",
       "                                    Gas     Unit  GWP_AR4_conditional  \\\n",
       "0                        Aggregate GHGs  MtCO2eq                    1   \n",
       "1                                   CH4  MtCO2eq                   25   \n",
       "2                                   CO2  MtCO2eq                    1   \n",
       "3                                   N2O  MtCO2eq                  298   \n",
       "4                        Aggregate GHGs  MtCO2eq                    1   \n",
       "...                                 ...      ...                  ...   \n",
       "58316                               NF3  MtCO2eq                17200   \n",
       "58317                               N2O  MtCO2eq                  298   \n",
       "58318                              PFCs  MtCO2eq                    1   \n",
       "58319                               SF6  MtCO2eq                22800   \n",
       "58320  Unspecified mix of HFCs and PFCs  MtCO2eq                    1   \n",
       "\n",
       "       GWP_AR6_conditional   Base year        1990  ...        2010  \\\n",
       "0                        1  143.211000  143.211000  ...  226.660000   \n",
       "1                       25    0.152966    0.152966  ...    0.582400   \n",
       "2                        1  142.551000  142.551000  ...  224.948000   \n",
       "3                      298    0.506877    0.506877  ...    1.128845   \n",
       "4                        1   36.256200   36.256200  ...   39.742000   \n",
       "...                    ...         ...         ...  ...         ...   \n",
       "58316                17200    0.047922    0.047922  ...    0.557691   \n",
       "58317                  298  452.656040  452.656040  ...  454.950640   \n",
       "58318                    1   24.255700   24.255700  ...    4.737930   \n",
       "58319                22800   28.845648   28.845648  ...    7.288636   \n",
       "58320                    1    0.227404    0.227404  ...    9.943280   \n",
       "\n",
       "             2011        2012        2013        2014        2015        2016  \\\n",
       "0      220.712000  222.471000  211.473000  205.297000  212.042000  219.422000   \n",
       "1        0.451877    0.545782    0.515282    0.647907    0.727285    0.916665   \n",
       "2      219.013000  220.698000  209.739000  203.485000  210.371000  217.548000   \n",
       "3        1.246874    1.227379    1.219109    1.164060    0.943790    0.958067   \n",
       "4       40.918900   42.912900   46.012100   46.383400   42.487300   40.968500   \n",
       "...           ...         ...         ...         ...         ...         ...   \n",
       "58316    0.568919    0.572732    0.498578    0.516564    0.566444    0.569480   \n",
       "58317  445.575560  416.764920  463.857860  473.989860  468.244420  450.793540   \n",
       "58318    7.315630    6.404460    6.122770    5.760310    5.204580    4.390520   \n",
       "58319    8.207681    6.921738    6.513755    6.557417    5.490035    6.020796   \n",
       "58320   10.644300   11.333700   12.024100   12.686100   14.077900   15.011000   \n",
       "\n",
       "             2017        2018        2019  \n",
       "0      218.428000  214.601000  213.814000  \n",
       "1        0.860737    0.952913    0.887035  \n",
       "2      216.606000  212.752000  212.031000  \n",
       "3        0.961715    0.896387    0.896199  \n",
       "4       40.070500   40.950900   40.792600  \n",
       "...           ...         ...         ...  \n",
       "58316    0.594546    0.616978    0.605124  \n",
       "58317  446.272880  459.212040  457.140940  \n",
       "58318    4.092780    4.699570    4.484390  \n",
       "58319    5.873713    5.694847    5.902920  \n",
       "58320   15.964000   16.358800   16.462600  \n",
       "\n",
       "[58321 rows x 39 columns]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> UNFCCC Annex I <<<<<<<<\")\n",
    "\n",
    "df_unfccc_ai = pd.read_csv(\"../raw_data/UNFCCC/detailed-data-by-country-annex-one.csv\")\n",
    "\n",
    "#clean\n",
    "df_unfccc_ai = df_unfccc_ai.rename(columns={\"Party\":\"name\"})\n",
    "df_unfccc_ai = df_unfccc_ai.replace({\"CO₂\":\"CO2\",\n",
    "                                       \"N₂O\": \"N2O\",\n",
    "                                       \"CH₄\": \"CH4\",\n",
    "                                       \"SF₆\": \"SF6\",\n",
    "                                       \"NF₃\":\"NF3\"})\n",
    "\n",
    "#add GWPs\n",
    "GWP_100_ipcc_ar6 = {\n",
    "        \"Aggregate GHGs\" : 1,\n",
    "        \"CO2\" : 1,\n",
    "        \"CH4\" : 27.9,\n",
    "        \"N2O\" : 273,\n",
    "        \"SF6\" : 25200,\n",
    "        \"NF3\" : 17400,\n",
    "        \"HFCs\": 1, #unit is already in CO2eq\n",
    "        \"PFCs\": 1, #unit is already in CO2eq\n",
    "        \"Aggregate F-gases\": 1,#unit is already in CO2eq\n",
    "        \"Unspecified mix of HFCs and PFCs\": 1 #unit is already in CO2eq\n",
    "        }\n",
    "GWP_100_ipcc_ar6_df = pd.DataFrame(GWP_100_ipcc_ar6.keys(), GWP_100_ipcc_ar6.values()).reset_index().set_index(0)\n",
    "GWP_100_ipcc_ar6_df.index.name = \"gas\"\n",
    "GWP_100_ipcc_ar4 = {\n",
    "        \"Aggregate GHGs\" : 1,\n",
    "        \"CO2\" : 1,\n",
    "        \"CH4\" : 25,\n",
    "        \"N2O\" : 298,\n",
    "        \"SF6\" : 22800,\n",
    "        \"NF3\" : 17200,\n",
    "        \"HFCs\": 1, #unit is already in CO2eq\n",
    "        \"PFCs\": 1, #unit is already in CO2eq\n",
    "        \"Aggregate F-gases\": 1,#unit is already in CO2eq\n",
    "        \"Unspecified mix of HFCs and PFCs\": 1 #unit is already in CO2eq\n",
    "        }\n",
    "GWP_100_ipcc_ar4_df = pd.DataFrame(GWP_100_ipcc_ar4.keys(), GWP_100_ipcc_ar4.values()).reset_index().set_index(0)\n",
    "GWP_100_ipcc_ar4_df.index.name = \"gas\"\n",
    "df_unfccc_ai.insert(5,\"GWP_AR4_conditional\",df_unfccc_ai.merge(GWP_100_ipcc_ar4_df, left_on=\"Gas\", right_on=\"gas\", how=\"left\")[\"index\"])\n",
    "df_unfccc_ai.insert(6,\"GWP_AR6_conditional\",df_unfccc_ai.merge(GWP_100_ipcc_ar4_df, left_on=\"Gas\", right_on=\"gas\", how=\"left\")[\"index\"])\n",
    "\n",
    "# optional stuff\n",
    "# df_unfccc_ai.insert(1, \"IPCC_cat\", df_unfccc_ai[\"Category\"].str[0])\n",
    "\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_unfccc_ai.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_unfccc_ai, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "\n",
    "#fix units and convert\n",
    "df_unfccc_ai.loc[df_unfccc_ai.Unit==\"kt\",\"Base year\":]                   = df_unfccc_ai.loc[df_unfccc_ai.Unit==\"kt\",\"Base year\":].div(1000).mul(df_unfccc_ai.loc[df_unfccc_ai.Unit==\"kt\",\"Unit\":][\"GWP_AR6_conditional\"], axis=0)\n",
    "df_unfccc_ai.loc[df_unfccc_ai.Unit==\"kt CO₂ equivalent\",\"Base year\":]    = df_unfccc_ai.loc[df_unfccc_ai.Unit==\"kt CO₂ equivalent\",\"Base year\":].div(1000).mul(df_unfccc_ai.loc[df_unfccc_ai.Unit==\"kt CO₂ equivalent\",\"Unit\":][\"GWP_AR6_conditional\"], axis=0)\n",
    "df_unfccc_ai.loc[df_unfccc_ai.Unit==\"t CO₂ equivalent\",\"Base year\":]     = df_unfccc_ai.loc[df_unfccc_ai.Unit==\"t CO₂ equivalent\",\"Base year\":].div(1000000).mul(df_unfccc_ai.loc[df_unfccc_ai.Unit==\"t CO₂ equivalent\",\"Unit\":][\"GWP_AR6_conditional\"], axis=0)\n",
    "df_unfccc_ai.loc[df_unfccc_ai.Unit==\"t\",\"Base year\":]                    = df_unfccc_ai.loc[df_unfccc_ai.Unit==\"t\",\"Base year\":].div(1000000).mul(df_unfccc_ai.loc[df_unfccc_ai.Unit==\"t\",\"Unit\":][\"GWP_AR6_conditional\"], axis=0)\n",
    "df_unfccc_ai.loc[:,\"Unit\"] = \"MtCO2eq\"\n",
    "    \n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_unfccc_ai.to_csv(\"../clean_data/UNFCCC_AI.csv\", index=None)\n",
    "df_unfccc_ai = pd.read_csv(\"../clean_data/UNFCCC_AI.csv\")\n",
    "df_unfccc_ai"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8aa87a1-e8d4-49f4-b53a-e88c597c4cc7",
   "metadata": {},
   "source": [
    "# UNFCCC Non-Annex I\n",
    "- from Open Climate Data: https://github.com/openclimatedata\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb78ef6c-f69c-463a-90fd-24eefc7997a9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "979cc552-17cb-4802-a377-3b514a9da07c",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true,
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> UNFCCC Non-Annex I <<<<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 0 Missing codes (NaN) in this df\n",
      " []\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>Parent Category</th>\n",
       "      <th>Category</th>\n",
       "      <th>Gas</th>\n",
       "      <th>Unit</th>\n",
       "      <th>1990</th>\n",
       "      <th>1991</th>\n",
       "      <th>1992</th>\n",
       "      <th>1993</th>\n",
       "      <th>...</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>2014</th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "      <th>2017</th>\n",
       "      <th>2018</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A Fuel Combustion - Sectoral Approach</td>\n",
       "      <td>Aggregate GHGs</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.301</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A Fuel Combustion - Sectoral Approach</td>\n",
       "      <td>CH4</td>\n",
       "      <td>Mt (substance)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.002</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A Fuel Combustion - Sectoral Approach</td>\n",
       "      <td>CO2</td>\n",
       "      <td>Mt (substance)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9.639</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A Fuel Combustion - Sectoral Approach</td>\n",
       "      <td>N2O</td>\n",
       "      <td>Mt (substance)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.002</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AFG</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1. Energy</td>\n",
       "      <td>1.A.1 Energy Industries</td>\n",
       "      <td>Aggregate GHGs</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.046</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25553</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions excluding LULUCF/LUCF</td>\n",
       "      <td>N2O</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25554</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions including LULUCF/LUCF</td>\n",
       "      <td>Aggregate GHGs</td>\n",
       "      <td>MtCO2eq</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25555</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions including LULUCF/LUCF</td>\n",
       "      <td>CH4</td>\n",
       "      <td>Mt (substance)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25556</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions including LULUCF/LUCF</td>\n",
       "      <td>CO2</td>\n",
       "      <td>Mt (substance)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25557</th>\n",
       "      <td>ZWE</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>Totals</td>\n",
       "      <td>Total GHG emissions including LULUCF/LUCF</td>\n",
       "      <td>N2O</td>\n",
       "      <td>Mt (substance)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>25558 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      code         name Parent Category  \\\n",
       "0      AFG  Afghanistan       1. Energy   \n",
       "1      AFG  Afghanistan       1. Energy   \n",
       "2      AFG  Afghanistan       1. Energy   \n",
       "3      AFG  Afghanistan       1. Energy   \n",
       "4      AFG  Afghanistan       1. Energy   \n",
       "...    ...          ...             ...   \n",
       "25553  ZWE     Zimbabwe          Totals   \n",
       "25554  ZWE     Zimbabwe          Totals   \n",
       "25555  ZWE     Zimbabwe          Totals   \n",
       "25556  ZWE     Zimbabwe          Totals   \n",
       "25557  ZWE     Zimbabwe          Totals   \n",
       "\n",
       "                                        Category             Gas  \\\n",
       "0        1.A Fuel Combustion - Sectoral Approach  Aggregate GHGs   \n",
       "1        1.A Fuel Combustion - Sectoral Approach             CH4   \n",
       "2        1.A Fuel Combustion - Sectoral Approach             CO2   \n",
       "3        1.A Fuel Combustion - Sectoral Approach             N2O   \n",
       "4                        1.A.1 Energy Industries  Aggregate GHGs   \n",
       "...                                          ...             ...   \n",
       "25553  Total GHG emissions excluding LULUCF/LUCF             N2O   \n",
       "25554  Total GHG emissions including LULUCF/LUCF  Aggregate GHGs   \n",
       "25555  Total GHG emissions including LULUCF/LUCF             CH4   \n",
       "25556  Total GHG emissions including LULUCF/LUCF             CO2   \n",
       "25557  Total GHG emissions including LULUCF/LUCF             N2O   \n",
       "\n",
       "                 Unit  1990  1991  1992  1993  ...  2009  2010  2011  2012  \\\n",
       "0             MtCO2eq   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "1      Mt (substance)   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "2      Mt (substance)   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "3      Mt (substance)   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "4             MtCO2eq   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "...               ...   ...   ...   ...   ...  ...   ...   ...   ...   ...   \n",
       "25553         MtCO2eq   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "25554         MtCO2eq   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "25555  Mt (substance)   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "25556  Mt (substance)   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "25557  Mt (substance)   NaN   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   \n",
       "\n",
       "         2013  2014  2015  2016  2017  2018  \n",
       "0      10.301   NaN   NaN   NaN   NaN   NaN  \n",
       "1       0.002   NaN   NaN   NaN   NaN   NaN  \n",
       "2       9.639   NaN   NaN   NaN   NaN   NaN  \n",
       "3       0.002   NaN   NaN   NaN   NaN   NaN  \n",
       "4       5.046   NaN   NaN   NaN   NaN   NaN  \n",
       "...       ...   ...   ...   ...   ...   ...  \n",
       "25553     NaN   NaN   NaN   NaN   NaN   NaN  \n",
       "25554     NaN   NaN   NaN   NaN   NaN   NaN  \n",
       "25555     NaN   NaN   NaN   NaN   NaN   NaN  \n",
       "25556     NaN   NaN   NaN   NaN   NaN   NaN  \n",
       "25557     NaN   NaN   NaN   NaN   NaN   NaN  \n",
       "\n",
       "[25558 rows x 35 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> UNFCCC Non-Annex I <<<<<<<<\")\n",
    "\n",
    "\n",
    "df_unfccc_nai = pd.read_csv(\"../raw_data/UNFCCC//detailed-data-by-country-non-annex-one.csv\")\n",
    "\n",
    "\n",
    "# clean\n",
    "df_unfccc_nai = df_unfccc_nai.rename(columns={\"Party\":\"name\"})\n",
    "df_unfccc_nai = df_unfccc_nai.replace({\"CO₂\":\"CO2\",\n",
    "                   \"N₂O\": \"N2O\",\n",
    "                   \"CH₄\": \"CH4\",\n",
    "                   \"SF₆\": \"SF6\",\n",
    "                   \"NF₃\":\"NF3\"})\n",
    "\n",
    "\n",
    "\n",
    "#   IMPOSSIBLE to convert to common units?\n",
    "# GWP_100_ipcc_ar6 = {\n",
    "#         \"Aggregate GHGs\" : 1,\n",
    "#         \"CO2\" : 1,\n",
    "#         \"CH4\" : 27.9,\n",
    "#         \"N2O\" : 273,\n",
    "#         \"SF6\" : 1, #unit is already in CO2eq\n",
    "#         \"NF3\" : 1, #unit is already in CO2eq\n",
    "#         \"HFCs\": 1, #unit is already in CO2eq\n",
    "#         \"PFCs\": 1, #unit is already in CO2eq\n",
    "#         \"Aggregate F-gases\": 1,#unit is already in CO2eq\n",
    "#         \"Unspecified mix of HFCs and PFCs\": 1 #unit is already in CO2eq\n",
    "#         }\n",
    "# GWP_100_ipcc_ar6_df = pd.DataFrame(GWP_100_ipcc_ar6.keys(), GWP_100_ipcc_ar6.values()).reset_index().set_index(0)\n",
    "# GWP_100_ipcc_ar6_df.index.name = \"gas\"\n",
    "# df_unfccc_nai.insert(5,\"GWP_AR6_conditional\",df_unfccc_nai.merge(GWP_100_ipcc_ar6_df, left_on=\"Gas\", right_on=\"gas\", how=\"left\")[\"index\"])\n",
    "# df_unfccc_nai.loc[:, \"1990\":] = df_unfccc_nai.loc[:, \"1990\":].mul(df_unfccc_nai[\"GWP_AR6_conditional\"], axis=0)\n",
    "# df_unfccc_nai.loc[:,\"Unit\"]= \"MtCO2eq\"\n",
    "\n",
    "\n",
    "\n",
    "#Convert from Gg to Mt\n",
    "df_unfccc_nai.loc[:, \"1990\":] = df_unfccc_nai.loc[:, \"1990\":].div(1000)\n",
    "df_unfccc_nai.loc[:,\"Unit\"]= df_unfccc_nai.loc[:,\"Unit\"].replace({\"Gg CO₂ equivalent\":\"MtCO2eq\",\n",
    "                                                                  \"Gg\": \"Mt (substance)\"})\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_unfccc_nai.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_unfccc_nai, show_missing=True,show_duplicates_nan=True)\n",
    "\n",
    "\n",
    "\n",
    "#------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "df_unfccc_nai.to_csv(\"../clean_data/UNFCCC_NAI.csv\", index=None)\n",
    "df_unfccc_nai = pd.read_csv(\"../clean_data/UNFCCC_NAI.csv\")\n",
    "df_unfccc_nai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37d20445-7b95-40b1-883a-2da9805f6f81",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebef7bb7-6769-4c91-b96d-93fd70f62530",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06b33056-edf3-41a2-9050-8af3266b3072",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "4cfe3f04-fc77-4adb-83da-d62563bbdc5a",
   "metadata": {},
   "source": [
    "# Other Inventories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c168f4d-c6e6-48b3-99f8-acf0ca2ef49e",
   "metadata": {},
   "source": [
    "- Carbon Majors\n",
    "- EDGAR FOOD?\n",
    "- LUCF from GCP\n",
    "- LUCF from MINX\n",
    "- CEDS (which is not mainly for GHG but for pollutants. Still quite nice)\n",
    "- carbon monitor"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa90e81a-a0d1-4cbb-bd68-6ae844a288f6",
   "metadata": {},
   "source": [
    "# Carbon Monitor\n",
    "- Downloaded from https://carbonmonitor.org/\n",
    "- Liu, Z., Ciais, P., Deng, Z. et al. Near-real-time monitoring of global CO2 emissions reveals the effects of the COVID-19 pandemic. Nat Commun 11, 5172 (2020). https://doi.org/10.1038/s41467-020-18922-7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a1756277-e331-4583-bb8f-38e288a37645",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>>>>>> Carbon Monitor <<<<<<\n",
      "PATCH APPLIED\n",
      ">>> 3 Missing codes (NaN) in this df\n",
      " ['EU27 & UK', 'ROW', 'WORLD']\n",
      "\n",
      ">>> 0 DUPLICATES:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>date</th>\n",
       "      <th>sector</th>\n",
       "      <th>unit</th>\n",
       "      <th>value</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BRA</td>\n",
       "      <td>Brazil</td>\n",
       "      <td>2019-01-01</td>\n",
       "      <td>Power</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.096802</td>\n",
       "      <td>1546272000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>CHN</td>\n",
       "      <td>China</td>\n",
       "      <td>2019-01-01</td>\n",
       "      <td>Power</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>14.059700</td>\n",
       "      <td>1546272000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>EU27 &amp; UK</td>\n",
       "      <td>2019-01-01</td>\n",
       "      <td>Power</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>1.872390</td>\n",
       "      <td>1546272000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FRA</td>\n",
       "      <td>France</td>\n",
       "      <td>2019-01-01</td>\n",
       "      <td>Power</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.051081</td>\n",
       "      <td>1546272000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DEU</td>\n",
       "      <td>Germany</td>\n",
       "      <td>2019-01-01</td>\n",
       "      <td>Power</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.316436</td>\n",
       "      <td>1546272000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99619</th>\n",
       "      <td>RUS</td>\n",
       "      <td>Russia</td>\n",
       "      <td>2022-03-31</td>\n",
       "      <td>International Aviation</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.005468</td>\n",
       "      <td>1648656000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99620</th>\n",
       "      <td>ESP</td>\n",
       "      <td>Spain</td>\n",
       "      <td>2022-03-31</td>\n",
       "      <td>International Aviation</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.050990</td>\n",
       "      <td>1648656000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99621</th>\n",
       "      <td>UGA</td>\n",
       "      <td>UK</td>\n",
       "      <td>2022-03-31</td>\n",
       "      <td>International Aviation</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.071560</td>\n",
       "      <td>1648656000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99622</th>\n",
       "      <td>USA</td>\n",
       "      <td>US</td>\n",
       "      <td>2022-03-31</td>\n",
       "      <td>International Aviation</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>0.172706</td>\n",
       "      <td>1648656000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99623</th>\n",
       "      <td>NaN</td>\n",
       "      <td>WORLD</td>\n",
       "      <td>2022-03-31</td>\n",
       "      <td>International Aviation</td>\n",
       "      <td>MtCO2</td>\n",
       "      <td>1.126300</td>\n",
       "      <td>1648656000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>99624 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      code       name        date                  sector   unit      value  \\\n",
       "0      BRA     Brazil  2019-01-01                   Power  MtCO2   0.096802   \n",
       "1      CHN      China  2019-01-01                   Power  MtCO2  14.059700   \n",
       "2      NaN  EU27 & UK  2019-01-01                   Power  MtCO2   1.872390   \n",
       "3      FRA     France  2019-01-01                   Power  MtCO2   0.051081   \n",
       "4      DEU    Germany  2019-01-01                   Power  MtCO2   0.316436   \n",
       "...    ...        ...         ...                     ...    ...        ...   \n",
       "99619  RUS     Russia  2022-03-31  International Aviation  MtCO2   0.005468   \n",
       "99620  ESP      Spain  2022-03-31  International Aviation  MtCO2   0.050990   \n",
       "99621  UGA         UK  2022-03-31  International Aviation  MtCO2   0.071560   \n",
       "99622  USA         US  2022-03-31  International Aviation  MtCO2   0.172706   \n",
       "99623  NaN      WORLD  2022-03-31  International Aviation  MtCO2   1.126300   \n",
       "\n",
       "        timestamp  \n",
       "0      1546272000  \n",
       "1      1546272000  \n",
       "2      1546272000  \n",
       "3      1546272000  \n",
       "4      1546272000  \n",
       "...           ...  \n",
       "99619  1648656000  \n",
       "99620  1648656000  \n",
       "99621  1648656000  \n",
       "99622  1648656000  \n",
       "99623  1648656000  \n",
       "\n",
       "[99624 rows x 7 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print( \">>>>>>> Carbon Monitor <<<<<<\")\n",
    "\n",
    "#download the file\n",
    "df_carbon_mon = pd.read_csv(\"https://datas.carbonmonitor.org/API/downloadFullDataset.php?source=carbon_global\")\n",
    "df_carbon_mon.date = pd.to_datetime(df_carbon_mon.date, format= \"%d/%m/%Y\")\n",
    "\n",
    "#clean\n",
    "df_carbon_mon = df_carbon_mon.rename(columns={\"country\":\"name\"})\n",
    "df_carbon_mon.insert(3,\"unit\",\"MtCO2\")\n",
    "\n",
    "# CODE_GENERATOR_ISO3 \n",
    "df_carbon_mon.insert(0, \"code\", np.nan)\n",
    "CODE_GENERATOR_ISO3(df_carbon_mon, show_missing=True, show_duplicates_nan=True)\n",
    "\n",
    "\n",
    "# global\n",
    "df_carbon_mon_global = df_carbon_mon[df_carbon_mon.name==\"WORLD\"]\n",
    "df_carbon_mon_global.to_csv(\"../clean_data/global_subsets/Carbon-Monitor_global.csv\")\n",
    "\n",
    "# #------------------------------------------------------------------------------------------------------------------------------\n",
    "df_carbon_mon.to_csv(\"../clean_data/Carbon-Monitor.csv\")\n",
    "df_carbon_mon = pd.read_csv(\"../clean_data/Carbon-Monitor.csv\", index_col=0)\n",
    "df_carbon_mon"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf38ed45-a221-49bb-91f9-b936e368d1ad",
   "metadata": {},
   "source": [
    "# Carbon Majors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8307ec9f-22de-4ab7-8724-8df044b42641",
   "metadata": {},
   "source": [
    "# Climate Accountability Institute\n",
    "\n",
    "Principal Investigator: Richard Heede\n",
    "\n",
    "Some links:\n",
    "- https://climateaccountability.org/carbonmajors_dataset2020.html\n",
    "- https://climateaccountability.org/pdf/MRR%209.1%20Apr14R.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6ce6b6ba-74eb-4e28-bcec-a151f002b79a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MtCO2e</th>\n",
       "      <th>% of global</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Entity</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Saudi Aramco - Saudi Arabia</th>\n",
       "      <td>1931</td>\n",
       "      <td>4.84%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Gazprom - Russia</th>\n",
       "      <td>1527</td>\n",
       "      <td>3.83%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>National Iranian Oil Co. - Iran</th>\n",
       "      <td>1266</td>\n",
       "      <td>3.17%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coal India - India</th>\n",
       "      <td>1214</td>\n",
       "      <td>3.04%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PetroChina / China Natl Petroleum</th>\n",
       "      <td>883</td>\n",
       "      <td>2.21%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Rosneft - Russian Federation</th>\n",
       "      <td>846</td>\n",
       "      <td>2.12%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Abu Dhabi - United Arab Emirates</th>\n",
       "      <td>692</td>\n",
       "      <td>1.74%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ExxonMobil - USA</th>\n",
       "      <td>580</td>\n",
       "      <td>1.45%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Iraq National Oil Co. - Iraq</th>\n",
       "      <td>566</td>\n",
       "      <td>1.42%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Royal Dutch Shell - The Netherlands</th>\n",
       "      <td>550</td>\n",
       "      <td>1.38%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BP - UK</th>\n",
       "      <td>549</td>\n",
       "      <td>1.38%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kuwait Petroleum - Kuwait</th>\n",
       "      <td>444</td>\n",
       "      <td>1.11%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Chevron - USA</th>\n",
       "      <td>443</td>\n",
       "      <td>1.11%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Total SA - France</th>\n",
       "      <td>404</td>\n",
       "      <td>1.01%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Peabody Energy - USA</th>\n",
       "      <td>398</td>\n",
       "      <td>1.00%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sonatrach - Algeria</th>\n",
       "      <td>398</td>\n",
       "      <td>1.00%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Petrobras - Brazil</th>\n",
       "      <td>385</td>\n",
       "      <td>0.97%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pemex - Mexico</th>\n",
       "      <td>380</td>\n",
       "      <td>0.95%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Glencore - Switzerland</th>\n",
       "      <td>355</td>\n",
       "      <td>0.89%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lukoil - Russian Federation</th>\n",
       "      <td>341</td>\n",
       "      <td>0.86%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Top Twenty</th>\n",
       "      <td>14153</td>\n",
       "      <td>35.49%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Global (2018)</th>\n",
       "      <td>39878</td>\n",
       "      <td>100.00%</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     MtCO2e % of global\n",
       "Entity                                                 \n",
       "Saudi Aramco - Saudi Arabia            1931       4.84%\n",
       "Gazprom - Russia                       1527       3.83%\n",
       "National Iranian Oil Co. - Iran        1266       3.17%\n",
       "Coal India - India                     1214       3.04%\n",
       "PetroChina / China Natl Petroleum       883       2.21%\n",
       "Rosneft - Russian Federation            846       2.12%\n",
       "Abu Dhabi - United Arab Emirates        692       1.74%\n",
       "ExxonMobil - USA                        580       1.45%\n",
       "Iraq National Oil Co. - Iraq            566       1.42%\n",
       "Royal Dutch Shell - The Netherlands     550       1.38%\n",
       "BP - UK                                 549       1.38%\n",
       "Kuwait Petroleum - Kuwait               444       1.11%\n",
       "Chevron - USA                           443       1.11%\n",
       "Total SA - France                       404       1.01%\n",
       "Peabody Energy - USA                    398       1.00%\n",
       "Sonatrach - Algeria                     398       1.00%\n",
       "Petrobras - Brazil                      385       0.97%\n",
       "Pemex - Mexico                          380       0.95%\n",
       "Glencore - Switzerland                  355       0.89%\n",
       "Lukoil - Russian Federation             341       0.86%\n",
       "Top Twenty                            14153      35.49%\n",
       "Global (2018)                         39878     100.00%"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_carbon_majors = pd.read_csv(\"../clean_data/Carbon-Majors-2018.csv\")\n",
    "df_carbon_majors = df_carbon_majors.set_index(\"Entity\")\n",
    "df_carbon_majors"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "toc-autonumbering": false,
  "toc-showtags": false,
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}