{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.2 RDD Data Processing\n",
    "\n",
    "This notebook demonstrates some of the more advanced uses of Spark RDD API for data processing.\n",
    "\n",
    "We will analyze the `Air Temperature` data set from BOM,  which contains historial maximum daily temperatures from multiple weather stations in New South Wales going back to the 19th century.\n",
    "\n",
    "Let's have look at the data first (`data/nsw_temp.csv` file):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Product code,Bureau of Meteorology station number,Year,Month,Day,Maximum temperature (Degree C),Days of accumulation of maximum temperature,Quality\r\n",
      "IDCJAC0010,061087,1965,01,01,25.6,,Y\r\n",
      "IDCJAC0010,061087,1965,01,02,32.2,1,Y\r\n",
      "IDCJAC0010,061087,1965,01,03,23.1,1,Y\r\n",
      "IDCJAC0010,061087,1965,01,04,25.6,1,Y\r\n"
     ]
    }
   ],
   "source": [
    "%%sh \n",
    "\n",
    "head -n 5 data/nsw_temp.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data appears to be the `csv` format with the header with the column names in the first line.\n",
    "\n",
    "Let's load the data as an RDD:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lines in the RDD: 352737 \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[u'Product code,Bureau of Meteorology station number,Year,Month,Day,Maximum temperature (Degree C),Days of accumulation of maximum temperature,Quality',\n",
       " u'IDCJAC0010,061087,1965,01,01,25.6,,Y',\n",
       " u'IDCJAC0010,061087,1965,01,02,32.2,1,Y',\n",
       " u'IDCJAC0010,061087,1965,01,03,23.1,1,Y',\n",
       " u'IDCJAC0010,061087,1965,01,04,25.6,1,Y']"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# use `textFile` to create an RDD of lines in the text files\n",
    "\n",
    "dataRDD = sc.textFile('data/nsw_temp.csv')\n",
    "\n",
    "# let's see how many lines are there in the RDD\n",
    "\n",
    "print(\"Lines in the RDD: %s \" % dataRDD.count())\n",
    "\n",
    "# let's preview the RDD\n",
    "dataRDD.take(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Important RDD internals\n",
    "\n",
    "In order to be able to process RDDs in the distributed manner Spark splits the RDD's data into multiple chunks called paritition.\n",
    "\n",
    "For a text file the partition may represent a range of lines in the file.  \n",
    "\n",
    "The number of partitions in the RDD will determine the maximum level of parallelism possible for processing of this RDD, that is the RDD with say two partitions can only be processed by two executors in parallel **even if many more are available**. \n",
    "\n",
    "Spark usually make a good guess about how many partition to use but in come cases, when the input data set is small and processing is compute intensive the number may need to be adjusted (to match the number of executors)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "# check the number of paritions in the RDD\n",
    "print(dataRDD.getNumPartitions())\n",
    "\n",
    "# force Spark to split the file to at least 4 partitions\n",
    "dataRDD = sc.textFile('data/nsw_temp.csv', 4)\n",
    "print(dataRDD.getNumPartitions())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RDDs may contain any *serializable* (with `pickle`) objects. \n",
    "\n",
    "For our data set let's represent each measurement as a `namedtuple` `BomRecord`.\n",
    "\n",
    "Let's convert our data set to the `RDD` of `BomRecord`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[BomRecord(product_code=u'IDCJAC0010', station_number=u'061087', year=u'1965', month=u'01', day=u'01', max_temp=u'25.6', days_of_acc=u'', quality=u'Y'),\n",
       " BomRecord(product_code=u'IDCJAC0010', station_number=u'061087', year=u'1965', month=u'01', day=u'02', max_temp=u'32.2', days_of_acc=u'1', quality=u'Y'),\n",
       " BomRecord(product_code=u'IDCJAC0010', station_number=u'061087', year=u'1965', month=u'01', day=u'03', max_temp=u'23.1', days_of_acc=u'1', quality=u'Y')]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from collections import namedtuple\n",
    "\n",
    "# define the namedtuple BomRecord\n",
    "BomRecord = namedtuple('BomRecord', \n",
    "        ['product_code', 'station_number', 'year', 'month', 'day', 'max_temp', 'days_of_acc', 'quality'])\n",
    "\n",
    "# filter out the header line and convert each line to the BomRecord\n",
    "bomRDD = dataRDD \\\n",
    "    .filter(lambda line: line.startswith('IDCJAC')) \\\n",
    "    .map(lambda line: BomRecord(*line.split(',')))\n",
    "\n",
    "bomRDD.take(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks good, but we can improve it by converting each column to its actual `python` type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[BomRecord(product_code='IDCJAC0010', station_number='061087', year=1965, month=1, day=1, max_temp=25.6, days_of_acc=None, quality=True),\n",
       " BomRecord(product_code='IDCJAC0010', station_number='061087', year=1965, month=1, day=2, max_temp=32.2, days_of_acc='1', quality=True),\n",
       " BomRecord(product_code='IDCJAC0010', station_number='061087', year=1965, month=1, day=3, max_temp=23.1, days_of_acc='1', quality=True)]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# returns a function that convert a list of string values to \n",
    "# the values of types defined in `types`\n",
    "def apply_types(*types):\n",
    "    return lambda strlist: ( t(v) if v else None for t,v in zip(types, strlist))\n",
    "\n",
    "bomRDD = dataRDD \\\n",
    "    .filter(lambda line: line.startswith('IDCJAC')) \\\n",
    "    .map(lambda line: BomRecord(*apply_types(str, str, int, int, int, float, str, bool)(line.split(','))))\n",
    "\n",
    "bomRDD.take(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another important consideration in working with RDDs is *caching*, that is keeping the entire (or part) of the RDD in memory, to improve the performance.\n",
    "\n",
    "There are two reasons for this:\n",
    "\n",
    "* accessing data in memory is much faster than acessing data on a (spinning) disk ( ~ 10^5 times faster)\n",
    "* caching can be used to capture the results of compute intensive transformations \n",
    "\n",
    "Let's see how that works in practice:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Serialized 1x Replicated\n",
      "Memory Serialized 1x Replicated\n"
     ]
    }
   ],
   "source": [
    "# print the current storage level\n",
    "print(bomRDD.getStorageLevel())\n",
    "\n",
    "# mark the RDD for caching\n",
    "bomRDD.cache()\n",
    "\n",
    "# print the cached storage level\n",
    "print(bomRDD.getStorageLevel())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "352696"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Perform an action in the RDD\n",
    "\n",
    "bomRDD.count()\n",
    "\n",
    "# Peform the action again and notice if it runs faster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the RDD is cached in memory (Memory Serialized 1x Replicated) the second `count` operation works much faster.\n",
    "\n",
    "### Analysing Data\n",
    "\n",
    "Now that we have created a convenient representation of the data and cached them for performance we are ready to ask a few analytical questions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example: how many stations and years of observations are there in the dataset as well as what are some of the statistics of the maximum temperature measurements?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#stations: 41 \n",
      "#years: 155\n",
      "Max temp stats:\n",
      "(count: 314092, mean: 22.8129516193, stdev: 5.31792943713, max: 46.5, min: 0.0)\n"
     ]
    }
   ],
   "source": [
    "import operator\n",
    "\n",
    "print(\"#stations: %s \" % bomRDD.map(lambda r:r.station_number).distinct().count())\n",
    "print(\"#years: %s\" % bomRDD.map(operator.attrgetter('year')).distinct().count())\n",
    "\n",
    "print(\"Max temp stats:\")\n",
    "print(bomRDD.map(operator.attrgetter('max_temp')).filter(lambda v: v is not None).stats())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check how many observations there are for each year:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1859, 365), (1860, 366), (1861, 365)]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "observationsPerYear = bomRDD.map(lambda r: (r.year, 1)) \\\n",
    "    .reduceByKey(operator.add) \\\n",
    "    .sortByKey() \\\n",
    "    .collect()\n",
    "    \n",
    "observationsPerYear[0:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can use `matplotlib` together with `pandas` to visualise the result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEPCAYAAABMTw/iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt0VPW5//H3w1WCXIJKwHARRCigIojxbkNV1GML1rYU\n9Kci1rNaoV7Oak+hVg3HnuOlp621v6W/ZW+ixWPVtgqWAnpoalG5IyggYJVLggSCAoIIBJ7fH3tP\nmGAmmSSTmT3D57VWFjvf+e49zwyZJ988+7u/29wdERHJXS0yHYCIiDQvJXoRkRynRC8ikuOU6EVE\ncpwSvYhIjlOiFxHJcUklejO7y8zeMbOVZjbdzNqYWb6ZzTWztWY2x8w6xfWfYmbrzWyNmY2Max8W\nHmOdmT3SHC9IRERqqjfRm9nJwHeBYe5+JtAKGAdMBl519wHAPGBK2H8QMAYYCFwFPGZmFh7uceAW\nd+8P9DezK1L8ekRE5CjJlm5aAu3NrBXQDigHRgPTwsenAdeE26OAZ929yt03AOuBIjPrBnRw98Vh\nv6fi9hERkWZSb6J39y3AT4FNBAl+l7u/ChS4e0XYZyvQNdylENgcd4jysK0QKItrLwvbRESkGSVT\nuulMMHrvDZxMMLK/Hjh67QStpSAiEkGtkuhzGfC+u38EYGZ/Bi4AKsyswN0rwrLMtrB/OdAzbv8e\nYVui9s8xM/3SEBFpBHe3o9uSqdFvAs4zs+PCk6qXAquBGcD4sM9NwEvh9gxgbDgzpw/QD1gUlnd2\nmVlReJwb4/apLVjuu+8+3D0rvrIp1myLN5tizbZ4synWbIs3E7EmUu+I3t0XmdkLwHLgYPjvE0AH\n4DkzmwBsJJhpg7uvNrPnwl8GB4Hb/EgEE4EngeOAWe4+u77nFxGRpkmmdIO7TwWmHtX8EUFZp7b+\nDwAP1NK+FDijgTGKiEgTRPrK2OLi4kyHkLRsihWyK95sihWyK95sihWyK94oxWp11XUyxcw8inGJ\niESZmeG1nIxNqnQTFaeccgobN27MdBgSAb1792bDhg2ZDkMkK2TViD78bZWBiCRq9LOQPQ4fhh07\n4KSTMh1JbtqzB44/PthONKKPdI1eRLJXRQWMGwcFBdC3L/TvD5MmwaJFoN/RqfHii/DFL9bfT4le\nRJrF7NlQWQnLlsGuXfD889C9e5D8hw2DrVszHWF2q6qCyZPhk0/q76tELyLNoqwMhg+Hnj2hRQsY\nMgTuvhvWrw9G+LNmZTrC7Pa73wV/Ge3fX39fJfoIufnmm7n33nszHUaDnX766bz22muZDkMiprwc\nCmtZtrBFC7j0Unj99fTHlCs+/RRKSuAnP4EDB+rvr0TfTM4991zee+89PvjgA84+++xMh5Mytf0y\neuedd7jkkksyFJFEVXk59OhR+2MXXqhE3xRPPw3nnAMXXaQRfcZUVVWxadMm+vXrx9KlSyOV6A8d\nOpTpEOQYUVZW+4ge4PTT4cMPgxq+NNwLL8ANN0CbNkr0GfP2228zaNAgAJYsWcLQoUOrH3v33XcZ\nMWIE+fn5nHHGGcycObPGvtu3b2fkyJF07NiRESNGsGnTpurH7rrrLgoKCujUqRNDhgxh9erVABw4\ncIDvfe979O7dm+7du3PbbbexP/zf//vf/07Pnj15+OGH6d69OxMmTGDQoEHMiiuQHjp0iK5du/LW\nW28BMGbMGLp3705+fj7FxcWsWbMGgF/96ldMnz6dhx9+mI4dOzJ69GgA+vTpw7x586pjufPOOyks\nLKRHjx7cddddHDx4sEYsP/vZzygoKKCwsJAnn3yyOo5Zs2YxePBgOnbsWN1Pslei0g1Ay5Zw7rnw\nxhvpjSkX7NgRzFy68kpo2za50k3GV3hLsAKb1yZRe1T87ne/886dO3teXp63a9fO8/PzvVWrVt6x\nY0fPz8/39evXe79+/fzBBx/0gwcP+rx587xDhw6+bt06d3cfP368d+zY0efPn+8HDhzwO+64wy+6\n6CJ3d58zZ44PHz7cd+/e7e7u7777rm/dutXd3e+8804fPXq079y50/fs2eOjRo3yH/7wh+7uXlpa\n6q1atfIpU6b4gQMH/LPPPvP777/fr7/++uq4X375ZR80aFCN17F3714/cOCA33XXXX7WWWdVPzZ+\n/Hi/5557arzuU045xf/3f//X3d3vueceP//8872ystIrKyv9ggsu8HvvvbdGLCUlJV5VVeWzZs3y\nvLw837lzp7u7d+/e3V9//XV3d9+5c6cvX7484Xsd9Z+FY93+/e6tW7tXVSXuU1Li/u//nr6YcsVv\nf+t+7bXB9uHD7mbuhw4F34efi8/n1NoaM/3V2EQfnINu+ldTXXzxxb5ixQrftGmTDx06tLr9H//4\nh3fv3r1G33HjxvnUqVPdPUii48aNq35sz5493qpVKy8rK/N58+b5gAEDfMGCBX748OEax2jfvr2/\n//771d+/8cYb3qdPH3cPkmvbtm39wIED1Y+/99573qFDB9+3b5+7u19//fV+//331/paPv74Yzez\n6l8w9SX6U0891WfPnl392Jw5c2rEkpeX54diP5Xu3rVrV1+4cKG7u/fu3dufeOKJ6ueqixJ9tG3Y\n4N6jR919XnnF/cIL0xNPLrn6avfp049837ate/hRTpjoc6p0k6pU3xgff/wx+fn5dO7cmTfffJPi\n4mIGDBjA2rVr6dKlC48++ihbtmyhZ8+eNfbr3bs35eVH7r8S/3j79u3Jz89ny5YtjBgxgkmTJjFx\n4kQKCgr49re/zZ49e9i+fTuffvopZ599Nl26dKFLly5cddVV7Nixo/o4J510Eq1bt67+/tRTT2XQ\noEHMnDmTffv2MWPGDK677joADh8+zOTJk+nXrx+dO3emT58+mBmVSRZTt2zZQq9evWq8vi1btlR/\nf8IJJ9CixZEfu7y8PPbs2QPAH//4R/7yl7/Qu3dvRowYwYIFC5J6Tomeuso2MeeeC2+9lVyNWQK7\nd8Nrr8HVVx9pa9u2/vcwq9a6ibL8/Hw+/vhj/vCHP1BaWsrjjz/Otddey3e/+11GjBgBwPz589m8\neXON/TZt2sSAAQOqv49/fM+ePXz00UecfPLJAEyaNIlJkyZRWVnJN77xDX7yk59QUlJCXl4eq1at\nonv37rXGFtznpaaxY8fyzDPPcOjQIQYPHkzfvn0BeOaZZ5g5cybz5s2jV69e7Nq1i/z8/NhfWrUe\nK97JJ5/Mxo0bGThwIAAbN26sjr8+Z599Ni+++CKHDh3il7/8JWPGjKlxjkKyR10zbmI6dAiull22\nDM4/Pz1xRdlnnwUnWQ8dCi6Cmj8fFiwI3qMRI+Dkk2HlSrj4YujU6ch+bdrUX6fPqRF9FCxdupRh\nw4YBsHz58uptCKZc5uXl8fDDD1NVVUVpaSkvv/wy48aNq+4za9Ys3njjDQ4cOMA999zD+eefT2Fh\nIUuWLGHRokVUVVXRrl07jjvuOFq0aIGZceutt3LnnXeyfft2AMrLy5k7d26dcY4dO5a5c+fy+OOP\nV4/mAT755BPatm1Lfn4+e/fuZcqUKTWSe0FBAe+//37C444bN44f//jHVFZWUllZyf33388NN9xQ\n7/t28OBBnnnmGXbv3k3Lli3p0KEDLVu2rHc/iaa6ZtzEGzoU3n67+ePJBq+/Dj/4AcybFyT0K66A\nv/4VbrsNtm0L2isr4Z57au6XzIheiT7Fli1bxrBhw/joo49o1aoVneJ+9bZu3ZqZM2cya9YsTjzx\nRCZNmsTTTz/NaaedBgSj5euuu46SkhJOOOEEli9fzu9//3sAdu/eza233kqXLl3o06cPJ554It//\n/vcBeOihh+jXrx/nnXcenTt3ZuTIkaxbt67OOLt168b555/PggUL+OY3v1ndfuONN9KrVy8KCws5\n/fTTueCCC2rsd8stt7Bq1Sq6dOnCtddeWx13zI9+9COGDx/OmWeeyZAhQxg+fDh33313wjji9336\n6afp06cPnTt35oknnuCZZ56p8zVIdCVTugHIzw/KERL8cvzSl2DaNHjiCbj5Zhg4EK65Bn7+86B9\n2jQ477ya+yUzxVKrV0pW0s9CtI0bB1/5CsT9sVirqVODUsV//Ed64oqy//qv4Jfegw82bL+BA+FP\nfwr+1eqVIpI2yZZuOnbUiD4mmfMatUlmRF9vojez/ma23MyWhf/uMrPbzSzfzOaa2Vozm2NmneL2\nmWJm681sjZmNjGsfZmYrzWydmT3S8JckItkg2dKNEv0Ryf5yPFoyF03Vm+jdfZ27D3X3YcDZwF7g\nz8Bk4FV3HwDMA6YAmNkgYAwwELgKeMyOFGIfB25x9/5AfzO7ouEvS0SizB22bEk+0SezzO6xINlf\njkdrjpOxlwH/dPfNwGhgWtg+Dbgm3B4FPOvuVe6+AVgPFJlZN6CDuy8O+z0Vt4+I5IjKSmjfHtq1\nq7+vRvRHNDbRN8f0ym8CsakQBe5eAeDuW4GuYXshED9ZvDxsKwTK4trLwjYRySENSVhK9IGDB4M1\nbLp1a/i+KR3Rm1lrgtH682HT0VMeNAVCRJToG+HDD6Fr12Cxt4ZK5mRsQ66MvQpY6u6xa+ErzKzA\n3SvCssy2sL0ciL/Ov0fYlqi9ViUlJdXbxcXFFBcX07t373qvzJRjQ+/evTMdgiRQVpb87JEOHZTo\nofFlm9LSUtavL2X6dFixInG/hiT6ccD/xH0/AxgPPATcBLwU1z7dzH5OUJrpByxydw9n7BQBi4Eb\ngUcTPVl8oo/ZsGFDA8IVkUzQiL7hGvLLMV5xcTHDhhVz+eVw440wderUWvslVboxszyCE7F/imt+\nCLjczNYClwIPArj7auA5YDUwC7gt7uqnicBvgHXAenef3eBXJiKR1pBE36ED7NnT+MUEc0VjR/SQ\n3PTKpEb07v4pcNJRbR8RJP/a+j8APFBL+1LgjGSeU0SyU0NGpy1bBrNz9u6F449v3riirCmJPiUX\nTImINERDk5bKN42/KhZSdMGUiEhDKNE3XGOvigWtXikiabZ3b7Cuepcuye+jRK/SjYhkkVjCasgs\n6GN9imVDloyojUo3IpJWjRmZHusj+h07ghPSeXmN218jehFJq8bMBz/WFzZrStkGUji9UkQkGRrR\nJ2/hwuA2ivv3Ny3Rp3oJBBGROpWXQ79+DdvnWE30Dz0U3Au2vBy+/vXGH0cjehFJq7IyKC5u2D4d\nO8JHHzVLOJG2ZAn87W9w6qlNO46mV4pIWql0k5yKimDph759m36s5liPXkQkISX6mrZtg699DQ4f\nrtm+dCkMH96waaiJaEQvImlTVQXbtzf85hm5PI/+T38KvpYurdm+eHGQ6FNB0ytFJG22boUTT4TW\nrRu2Xy5Pr3z+eRg8GGbMqNm+ZEnqEr0umBKRtGnsfPBcLd1s2xaM5B95BF566Ui7e2oTvUb0IpI2\njV2BMVcT/YsvwpVXwogRwcnXDz4I2rdsgUOHoGfPuvdPlkb0IpI2jV2BMVcT/fPPwze+Eay5/+Uv\nHxnVx0bzqborqkb0ItLs3noLNmxQ6SbGHVauhEWL4KqrgrbRo4/U6VNZtgFdMCUizWj7drjrLigt\nDUoR27bB0083/Dht2wbTD/fvD7az0U9/CtOmBdvl5cHdsqZMObJQ2WWXwfjxcOaZsGkT/P73qXvu\nZKZXKtGLSIPt3w9Dh8I3vwlr1wYJbcsW6Nq14ccyC6ZYfvJJ9ib6//kfmDwZTj8dTjoJunev+Xhe\nHqxaFfxybNECBg1K3XOnrHRjZp3M7HkzW2Nmq8zsXDPLN7O5ZrbWzOaYWae4/lPMbH3Yf2Rc+zAz\nW2lm68zskca+MBHJrCVLgmT2059C+/ZBsi4sbPjUyphsnmK5bx+sWQNf/WowYj86ycd07x48fvrp\nQbJPlVSejP0FMMvdBwJDgHeBycCr7j4AmAdMATCzQcAYYCBwFfCYWfVph8eBW9y9P9DfzK5o0CsS\nkUiYPx8uuih1x8vmOv3y5TBwYLCmfCakZERvZh2Bi939dwDuXuXuu4DRQFiVYhpwTbg9Cng27LcB\nWA8UmVk3oIO7Lw77PRW3j4hkkX/8Q4k+ZuFCKCrK3POnakTfB6g0s9+Z2TIze8LM8oACd68AcPet\nQKw6Vwhsjtu/PGwrBMri2svCNhHJIocPwxtvwIUXpu6Y2ZzoFy2Cc8/N3PO3ahWcDD96PZ0afZI5\nDjAMmOjuS8zs5wRlGz+q39HfN0lJSUn1dnFxMcUNXftURJrF6tVwwgkNX9OmLtmc6BcuhPvuy8xz\nl5aWUlpaSosWcO+9ifslk+jLgM3uviT8/o8Eib7CzArcvSIsy2wLHy8H4q/56hG2JWqvVXyiF5Ho\nSHV9HrI30W/fHqyl379/Zp4/Ngj+2c/g+9+H//zPqbX2q7d0E5ZnNptZ7KVcCqwCZgDjw7abgNhq\nDjOAsWbWxsz6AP2ARWF5Z5eZFYUnZ2+M20dEskRzJPrY9Mpss2gRnHNOamfRNEZ9c+mTnUd/OzDd\nzFoD7wM3Ay2B58xsArCRYKYN7r7azJ4DVgMHgdvcPVbWmQg8CRxHMItndkNfkIhk1vz5cM89qT1m\nto7oM30iNqa+E7JJJXp3XwGcU8tDlyXo/wDwQC3tS4EzknlOEYmesjLYuzf1pYpu3eDvf0/tMdNh\n0SL4zncyHUX9Uyy11o2IJO3NN+H881O3IFfMV78Kf/kL7NqV2uM2t/feC+bQZ1p9I3olehFJWnNN\nJTzpJLj88mApgWzh3vgVO1NNI3oRSZnmrEl/61vw6183z7FTZdu2I9s7dgRr2LRvn7l4YjSiF5GU\nqKqCZcuCWSbN4bLLgumKy5c3z/Gb6p//rLm8cGNvtNIcNKIXkZRYtSq4K1Lnzs1z/JYtYcKE6I7q\nt26FzZuDRcwgOmUbqH96pRK9iCQlHVMJb7klqNNHcarljh3Bvxs2BP+WlUVrRK/SjYg02cKFzb+m\nS48eQQkndhOPKIkl+vffD/6NUqLXiF5EUiJdi3fdfjv88pd1L9KVCZWVwb+xm3xHLdFrRC8iTfLJ\nJ8FI9ow0XO544YXBrfjmzGn+52qIHTsgPz+aI/r6TsbqVoIiUq8lS2DIkCChNDezYFR/773BCeAD\nB4IVM1euhBNPDG5hGLuL0yWXpG8Jgh07ghlHsUQfpVk3KVkCQUSObZs3w6mnpu/5xo0LbqK9dWsw\nG+fSS4MbkcemX27dGjw+bx7MmpWemCorg+mVL78cfK8RvYjklM8+S++t8tq2Tby++pVXBv9u2RLc\ng9U99Usy1GbHjuCvh0cfDZZqOHw4WIwtClSjF5Em++wzOO64TEdR08knB798/vnP9Dzfjh3Qt29w\nA/S33gpG8+n4BZMMXTAlIk0WxUQPwSyghQvT81w7dgR31urbN7hnblTKNqDplSKSAlFN9EVFwbTP\n5uZeM9G/9lq0Er0umBKRJtu/P5qJPl0j+t27g9ffti306RPcHD1KiV4jehFpss8+C5JJ1Jx9djDt\nsq4klwo7dgRTOyEY0e/dG71ErxG9iDRJVEs3xx8P/foFyb45VVYGZRsIRvQQrUSvk7Ei0mRRTfSQ\nnvJNrD4PwYgeopXoUzKiN7MNZrbCzJab2aKwLd/M5prZWjObY2ad4vpPMbP1ZrbGzEbGtQ8zs5Vm\nts7MHmn8yxKRdIpyoi8qSk+ij5VuevUKLuKKUqJP1Yj+MFDs7kPdPXbB8WTgVXcfAMwDpgCY2SBg\nDDAQuAp4zKx6tunjwC3u3h/ob2ZXNPD1iEgGRDnRX3RRcGNx9+Z7jvgRfZs2wZIQscQfBak6GWu1\n9B0NxBYTnQZcE26PAp519yp33wCsB4rMrBvQwd0Xh/2eittHRCIsyon+C1+AVq2at04fX6MHOOus\n5nuuxkjV9EoHXjGzxWb2rbCtwN0rANx9K9A1bC8ENsftWx62FQJlce1lYZuIRNz+/dGcdQPB1alf\n+QrMnNl8zxFfuomi+kb0ya51c6G7f2hmJwFzzWwtQfKPl9I/nEpKSqq3i4uLKS4uTuXhRaQBojyi\nhyDR3303/OhHzXP8+NJNlJSWllJaWsoHH8C6dYn7JZXo3f3D8N/tZvYiUARUmFmBu1eEZZnY/dHL\ngZ5xu/cI2xK11yo+0YtIZkU90V9yCaxdG6xq2a1b6o9/dOkmKmKD4DfeCBL9xo1Ta+1Xb+nGzPLM\n7Phwuz0wEngbmAGMD7vdBLwUbs8AxppZGzPrA/QDFoXlnV1mVhSenL0xbh8RibCoJ/o2bWDkSPjL\nX5rn+NlQumlqjb4AmG9my4EFwEx3nws8BFwelnEuBR4EcPfVwHPAamAWcJt79fnwicBvgHXAenef\n3ZgXJSLpFfVEDzBqVPPV6aNauompb3qleXPOSWokM/MoxiVyrOrRAxYsiNbc8aPt2AGnnBKsFd8i\nxZeC5uUFNz1p3z61x02Vdevg6qvhvfcMd//c4sm6MlZE6hXlWTcxJ5wQ3NN148bUHvfTT4M5+nl5\nqT1uKmn1ShFpsmwo3QAMGhTcZzaVYmWbqNxkpDZavVJEmiybEv3q1ak9ZtTr86ARvYg0UVVVcH/U\nVllwh+nBg1Of6Csroz3jBjSiF5Emit10JMqli5jmKN28++6RpYmjSiN6EWmSbCnbAAwcCGvWBH+B\npMq8eTBiROqO1xxatw6mlyaiRC8idcqGGTcxnTsHX5s2peZ4hw9DaWn0E70Z/PGPiR9XoheROmXT\niB5Se0J2xQro2hVOPjk1x8sUJXoRqdOxnOjnzYMvfSk1x8okJXoRqVO2JfrBg1N3QjYb6vPJUKIX\nkTplW6JP1Yj+4EGYPx9yYYV0JXoRqVM2Jvo1az5/a8GqKli/Hj78MLnjLF0arJ1z0kkpDzHtsuAS\nCBHJpM8+y55ZNxCsd5OfH6xLH1vczB127gyucO3ZM1igrT5/+1tujOZBiV5E6hG7YCqbvPMO7NlT\ns61z52C++SmnwNtvwxln1H2MN9+EG25othDTSqUbEalTtpVuADp0gO7da361axcs43DLLfCrX9W9\nv3sw6j/vvPTE29yU6EWkTtmY6OsyYQJMnw779iXu88EHweg/yuvvN4QSvYjUKdcSfe/eUFQEL7yQ\nuM+bb8L552fH+j7JUKIXkTpl28nYZPzrv8ITTyR+PJfKNqBELyL1yLURPcCXvwwbNsDy5bU/vmBB\nMKLPFUknejNrYWbLzGxG+H2+mc01s7VmNsfMOsX1nWJm681sjZmNjGsfZmYrzWydmT2S2pciIs0h\nG2fd1Kd1a5g4ER599POP7dsXXHA1bFj642ouDRnR3wHEX282GXjV3QcA84ApAGY2CBgDDASuAh4z\nq650PQ7c4u79gf5mdkUT4xeRZpaLI3qAW2+FF1+Ebdtqti9dGiyj0K5dZuJqDkklejPrAfwL8Ou4\n5tHAtHB7GnBNuD0KeNbdq9x9A7AeKDKzbkAHd18c9nsqbh8RiahcTfQnnABf//rna/W5Vp+H5C+Y\n+jnwfaBTXFuBu1cAuPtWM+sathcCb8b1Kw/bqoCyuPaysF1EIixXEz3AHXfAxRcHa9ocOgSbNwe1\n+xdfzHRkqVVvojezq4EKd3/LzIrr6Op1PNZgJSUl1dvFxcUU58q1yCJZJhdn3cScfjq88kpwX1iz\nYN58377ZU7YpLS2ltLS03n7JjOgvBEaZ2b8A7YAOZvY0sNXMCty9IizLxCpd5UDPuP17hG2J2msV\nn+hFJHNyeUQPMHx4piNovKMHwVOnTq21X701enf/obv3cve+wFhgnrvfAMwExofdbgJeCrdnAGPN\nrI2Z9QH6AYvcfSuwy8yKwpOzN8btIyIRleuJ/ljQlEXNHgSeM7MJwEaCmTa4+2oze45ghs5B4Db3\n6gVDJwJPAscBs9x9dhOeX0TSIBenVx5rzI9etDkCzMyjGJfIsWjECLj33ty401KuMzPc/XMLN+jK\nWBGpk0o32U+JXkTqlMuzbo4VSvQiUieN6LOfEr2I1EmJPvsp0YtInTTrJvsp0YtInTSiz35K9CJS\nJyX67KdELyIJuWvWTS5QoheRhA4ehJYtgy/JXkr0IpKQyja5QYleRBLSjJvcoEQvIglpRJ8blOhF\nJCEl+tygRC8iCWnGTW5QoheRhDSizw1K9CKSkBJ9blCiF5GElOhzgxK9iCSk6ZW5QYleRBLSiD43\n1JvozaytmS00s+Vm9raZ3Re255vZXDNba2ZzzKxT3D5TzGy9ma0xs5Fx7cPMbKWZrTOzR5rnJYlI\nqmjWTW6oN9G7+35ghLsPBc4CrjKzImAy8Kq7DwDmAVMAzGwQMAYYCFwFPGZmsZvVPg7c4u79gf5m\ndkWqX5CIpI5G9LkhqdKNu38abrYFWgEOjAamhe3TgGvC7VHAs+5e5e4bgPVAkZl1Azq4++Kw31Nx\n+4hIBCnR54akEr2ZtTCz5cBW4JUwWRe4ewWAu28FuobdC4HNcbuXh22FQFlce1nYJiIRpUSfG1ol\n08ndDwNDzawj8GczG0wwqq/RLZWBlZSUVG8XFxdTXFycysOLSBI06ybaSktLKS0trbdfUok+xt13\nm1kpcCVQYWYF7l4RlmW2hd3KgZ5xu/UI2xK11yo+0YtIZmhEH21HD4KnTp1aa79kZt2cGJtRY2bt\ngMuBNcAMYHzY7SbgpXB7BjDWzNqYWR+gH7AoLO/sMrOi8OTsjXH7iEgEbd4MBQWZjkKaKpkRfXdg\nmpm1IPjF8Ad3n2VmC4DnzGwCsJFgpg3uvtrMngNWAweB29w9VtaZCDwJHAfMcvfZKX01IpJSixfD\nHXdkOgppKjuSg6PDzDyKcYkcSz75BLp1g507oXXrTEcjyTAz3N2ObteVsSJSq6VLYcgQJflcoEQv\nIrVatAjOOSfTUUgqKNGLSK0WL4aiokxHIamgRC8itdKIPnco0YvI51RUwO7d0K9fpiORVFCiF5HP\nWbwYhg+HFsoQOUH/jSLyOarP55YGLYEgIrnt7LNhxQo4fBj++tdMRyOpokQvItU++AC2bIEuXaCV\nskPO0JWxIgKAe3Bx1L59ukgqW+nKWBGpUyzBK8nnHiV6EQGCtW06dsx0FNIclOhFBAjmzSvR5yYl\nehEBgkQ5SZYkAAAMTElEQVTfoUOmo5DmoEQvIoBKN7lMiV5EAJVucpkSvYgAKt3kMiV6EQFUusll\nSvQiAqh0k8vqTfRm1sPM5pnZKjN728xuD9vzzWyuma01szlm1ilunylmtt7M1pjZyLj2YWa20szW\nmdkjzfOSRKQxVLrJXcmM6KuAf3P3wcD5wEQz+wIwGXjV3QcA84ApAGY2CBgDDASuAh4zs9gluY8D\nt7h7f6C/mV2R0lcjIo2m0k3uqjfRu/tWd38r3N4DrAF6AKOBaWG3acA14fYo4Fl3r3L3DcB6oMjM\nugEd3H1x2O+puH1EJMNUusldDarRm9kpwFnAAqDA3Ssg+GUAdA27FQKb43YrD9sKgbK49rKwTUQi\nQKWb3JX0QqRmdjzwAnCHu+8xs6OXl0zpcpMlJSXV28XFxRQXF6fy8CJyFJVusk9paSmlpaX19ksq\n0ZtZK4Ik/7S7vxQ2V5hZgbtXhGWZbWF7OdAzbvceYVui9lrFJ3oRaX4q3WSfowfBU6dOrbVfsqWb\n3wKr3f0XcW0zgPHh9k3AS3HtY82sjZn1AfoBi8Lyzi4zKwpPzt4Yt4+IZJhKN7mr3hG9mV0IXA+8\nbWbLCUo0PwQeAp4zswnARoKZNrj7ajN7DlgNHARui7uLyETgSeA4YJa7z07tyxGRxlLpJnfpDlMi\nAgRJfvNm6NSp/r4STYnuMKVELyIcPhzcWerAAWjZMtPRSGPpVoIiktDevZCXpySfq5ToRUQnYnOc\nEr2IaGpljlOiFxHNuMlxSvQiotJNjlOiFxGVbnKcEr2IqHST45ToRUSlmxynRC8iKt3kOCV6EVHp\nJscp0YuISjc5ToleRFS6yXFK9CKi0k2OU6IXEZVucpwSvYiodJPjlOhFRKWbHKdELyIq3eQ4JXoR\nUekmxyVzc/DfAF8GKtz9zLAtH/gD0BvYAIxx913hY1OACUAVcIe7zw3bh1HzxuB3pvrFiGTKqlWw\ncSN8+imYBbfli321bBm0RdXhw8EtBPPyMh2JNJd67xlrZhcBe4Cn4hL9Q8AOd3/YzH4A5Lv7ZDMb\nBEwHzgF6AK8Cp7m7m9lCYJK7LzazWcAv3H1OgufUPWMlq3TvDoMHHxkVHzx45OvQoczGlozCQpg+\nPdNRSFM16ebgZtYbmBmX6N8FvujuFWbWDSh19y+Y2WTA3f2hsN9fgRJgIzDP3QeF7WPD/b+T4PmU\n6CVrHDgAxx8P+/bpnquSWam+OXhXd68AcPetQNewvRDYHNevPGwrBMri2svCNpGst2ULFBQoyUt0\n1VujT1LKh98lJSXV28XFxRQXF6f6KURSorwcevTIdBRyLCotLaW0tLTefo1N9BVmVhBXutkWtpcD\nPeP69QjbErUnFJ/oRaKsvDyocYuk29GD4KlTp9baL9nSjYVfMTOA8eH2TcBLce1jzayNmfUB+gGL\nwvLOLjMrMjMDbozbRySrlZUp0Uu0JTO98hmgGDjBzDYB9wEPAs+b2QSCE61jANx9tZk9B6wGDgK3\nxZ1VnUjN6ZWzU/tSRDJDpRuJuqRm3aSbZt1INhk7FkaNguuuy3QkcqxL9awbEQmpdCNRp0Qv0kQq\n3UjUqXQj0gSHD0O7drBzZ/CvSCapdCPSDCorg1UfleQlypToRZpAc+glGyjRizSBEr1kAyV6kSbQ\niVjJBkr0Ik2gqZWSDZToRZpApRvJBkr0Ik2g0o1kAyV6kSZQ6UaygRK9SBOodCPZIFU3Hkm5Xr0y\nHYFI/dwhPz/TUYjULbJLIGzcGL24RI52/PHQpUumoxAJNOnm4OmmtW5ERBpOa92IiByjlOhFRHKc\nEr2ISI5Le6I3syvN7F0zW2dmP0j384uIHGvSmujNrAXwf4ErgMHAODP7QqL+paWlaYqs6bIpVsiu\neLMpVsiueLMpVsiueKMUa7pH9EXAenff6O4HgWeB0Yk6R+mNqk82xQrZFW82xQrZFW82xQrZFW+U\nYk13oi8ENsd9Xxa2iYhIM9HJWBGRHJfWC6bM7DygxN2vDL+fDLi7P3RUP10tJSLSCBm/MtbMWgJr\ngUuBD4FFwDh3X5O2IEREjjFpXdTM3Q+Z2SRgLkHZ6DdK8iIizSuSa92IiEjqpHse/W/MrMLMVsa1\nDTGzN81suZktMrPhcY+daWZvmNk7ZrbCzNqE7cPMbGV40dUjGY73nLC9lZk9Gca1Kjz/ENun2eNN\nEGvs/VthZi+Z2fFxj00xs/VmtsbMRqYz1obGa2aXmdmSsH2xmY1IZ7wNfW/Dx3uZ2Sdm9m/pjLUx\n8Wbyc9bAn4NMf8Z6mNm88LnfNrPbw/Z8M5trZmvNbI6ZdYrbJ6Ofs2runrYv4CLgLGBlXNscYGS4\nfRXwt3C7JbACOD38Pp8jf4EsBM4Jt2cBV0Qg3nHAM+F2O+ADoFe64k0Q6yLgonB7PPAf4fYgYDlB\n6e4U4L2IvLeJ4h0CdAu3BwNlcftE6r2Ne/x54A/Av6Uz1ka8txn9nDUw1kx/xroBZ4XbxxOcb/wC\n8BDw72H7D4AHw+2Mf85iX2kd0bv7fODjo5oPA7HfgJ2B8nB7JLDC3d8J9/3Y3d3MugEd3H1x2O8p\n4JoIxOtAewtOOOcB+4Hd6Yo3Qaynhe0ArwJfC7dHAc+6e5W7bwDWA0UReG9rjdfdV7j71nB7FXCc\nmbWO6HuLmY0G3gdWxbVF8r0lw5+zBsaa6c/YVnd/K9zeA6wBehBc9Dkt7DYt7rkz/jmLicI8+ruA\n/zazTcDDwJSwvT+Amc0O/2z/ftheSHChVUy6L7pKFO8LwKcEs4k2AP/t7jvJbLyrzGxUuD2G4IcS\nPn/hWnnYlun3NlG81czs68AyD66sjtx7G5YZ/h2YCsRPc4vqexvFz1miWCPzGTOzUwj+ElkAFLh7\nBQS/DICuYbfIfM6ikOi/A9zh7r0Ikuhvw/ZWwIUEf65dDHw1vjabQYniPReoIvjzri/wvfCHIZMm\nABPNbDHQHjiQ4XjqU2e8ZjYYeAD41wzEdrREsd4H/NzdP81YZLVLFG8UP2eJYo3EZyz8Zf4CQR7Y\nQ/CXRrzIzXCJwj1jb3L3OwDc/QUz+3XYXga85u4fA5jZLGAYMB3oGbd/D46UTzIZ7zhgtrsfBrab\n2evAcGB+puJ193UEC8hhZqcBV4cPlSeIKVF7WtQRL2bWA/gTcEP4ZzBkMN46Yj0X+JqZPUxQ7z5k\nZp8RxB7F9zZyn7M6Ys34Z8zMWhEk+afd/aWwucLMCty9IizLbAvbI/M5y8SI3qj5J225mX0RwMwu\nJahjQXDS8wwzOy58c78IrAr/NNplZkVmZsCNwEs0n2Tj3QR8KWxvD5wHrElzvDViNbOTwn9bAD8C\n/l/40AxgrJm1MbM+QD9gUabf20Txmlln4GXgB+6+INY/iu+tu1/i7n3dvS/wCPBf7v5YVN9bovE5\nqy/Wx8OHovAZ+y2w2t1/Edc2g+CkMcBNcc8dlc9Z2mfdPANsITiJsgm4GbgAWEJwdvpNYGhc/+uA\nd4CVwANx7WcDbxMk2V9EIV6CPzGfC+N9h5qzLZo93gSx3k4wM+BdgoQT338KwSyANYSziCLw3tYa\nL3A38AmwLHzflwEnRvW9jdvvvnT/HDTyZyFjn7MG/hxk+jN2IXAIeCvu5/BKoAvBSeO1BBeDdo7b\nJ6Ofs9iXLpgSEclxUTgZKyIizUiJXkQkxynRi4jkOCV6EZEcp0QvIpLjlOhFRHKcEr2ISI5Tohdp\nBuFVnSKRoB9GOeaZ2VQzuyPu+x+b2e1m9j0Lbi7zlpndF/f4ny24AcrbZvatuPZPzOy/zWw5weX5\nIpGgRC8SrF9yI0C49shYgqVwT3P3ImAoMNzMLgr73+zu5wDnAHeYWX7Y3h54092HuvsbaX0FInWI\nwuqVIhnl7hvNrNLMhhAsgbsMKAIuN7NlBAtutQdOI1gp8U4zi90ookfYvohgCd0/pTt+kfoo0YsE\nfk2woFY3ghH+ZQQLfP0qvlO4cumXgHPdfb+Z/Q04Lnz4M9fiURJBKt2IBF4kWIlwOMHSvXOACeFy\nuJjZyeHyuZ2Aj8Mk/wVq1uINkQjSiF4EcPeD4ej843BU/kqYyN8MyvZ8AvwfYDbwbTNbRbAs7Zvx\nh0lz2CJJ0TLFIlRPh1wKfN3d/5npeERSSaUbOeaZ2UCCG0C8oiQvuUgjehGRHKcRvYhIjlOiFxHJ\ncUr0IiI5ToleRCTHKdGLiOQ4JXoRkRz3/wEY/LOTVSkdXgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11248e8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "observationsPerYearPD = pd.DataFrame.from_records(observationsPerYear, columns=['year', '#observations'], index='year')\n",
    "plt.close()\n",
    "observationsPerYearPD.plot(kind='line')\n",
    "plt.show()\n",
    "display()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we can find the station with most observations and calculate and plot average yearly temperatures for this station over the entire period of data collection: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Station: 066062 has most observations: 56317\n"
     ]
    }
   ],
   "source": [
    "# find the station with the most observations\n",
    "\n",
    "top_station_id, top_number = bomRDD \\\n",
    "    .filter(lambda r: r.max_temp is not None) \\\n",
    "    .map(lambda r: (r.station_number, 1)) \\\n",
    "    .reduceByKey(operator.add) \\\n",
    "    .sortBy(lambda t:t[1], False) \\\n",
    "    .first()\n",
    "    \n",
    "print(\"Station: %s has most observations: %s\" % (top_station_id, top_number))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(1859, 21.399452054794523), (1860, 20.276986301369863), (1861, 20.864657534246582)]\n"
     ]
    }
   ],
   "source": [
    "# compute the yearly averages for the station with most observations\n",
    "\n",
    "seqOp = (lambda sumAndCount, v: (sumAndCount[0] + v, sumAndCount[1] + 1))\n",
    "combOp = (lambda sumAndCount1, sumAndCount2: (sumAndCount1[0] + sumAndCount2[0], sumAndCount1[1] + sumAndCount2[1]))\n",
    "\n",
    "averageYearlyTemps = bomRDD \\\n",
    "    .filter(lambda r: r.station_number == top_station_id) \\\n",
    "    .filter(lambda r: r.max_temp is not None) \\\n",
    "    .map(lambda r: (r.year, r.max_temp)) \\\n",
    "    .aggregateByKey((0.0, 0), seqOp, combOp) \\\n",
    "    .map(lambda (year,sumAndCount): (year, sumAndCount[0]/ sumAndCount[1])) \\\n",
    "    .sortByKey() \\\n",
    "    .collect()\n",
    "    \n",
    "print(averageYearlyTemps[0:3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEPCAYAAAC5sYRSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl4HNWV9t8jyZIsyfsi7zZewGATMDFmMSQyW4A4LAmQ\nACEJDCErMOTLhEAgmJmQgcw8GTwkTBjCYkIIW8ISEggwoBhIsE284BUb77Ys2bIkW4slWdL5/jh9\nXdXVVd3V3dXd1dL5PY8edVfXcnupt06999xziZmhKIqi9C4Kct0ARVEUJXhU3BVFUXohKu6Koii9\nEBV3RVGUXoiKu6IoSi9ExV1RFKUXklDciWgcEb1FRGuJaDUR3RhZ/q9EtIqIVhDRa0Q0ymP7bbb1\nlgb9BhRFUZRYKFGee0S0RzHzSiKqAPAPABcD2MXMLZF1bgRwHDN/y2X7LQA+ycyNgbdeURRFcSVh\n5M7Mtcy8MvK4BcB6AGONsEcoB9DjsQvycxxFURQlOIqSWZmIJgE4EcCSyPOfAPgKgCYA8zw2YwBv\nEFE3gP9l5odTbayiKIrij4S2zJEVxZKpBvBvzPyS47VbAfRn5gUu241m5j1ENALAGwC+y8zvpttw\nRVEUxRtf4k5ERQBeAfAqMy90eX08gD8z8/EJ9nMXgGZm/rnLa1rkRlEUJUmYmdyW+/XCHwWwzi7s\nRDTV9volEC8+CiIqi0T8IKJyAOcBWBOnkbjrrrvAzHnxl09tzbf25lNb8629+dTWfGtvttsaj4Se\nOxHNBXA1gNVEtALiod8O4HoiOgZAN4DtAL4ZWX80gIeZeT6ASgAvRKLyIgC/ZebXEx1TURRFSY+E\n4s7M7wEodHnpNY/19wCYH3m8FdIBqyiKomSR0KUoVlVV5boJvsmntgL51d58aiuQX+3Np7YC+dXe\nMLXVd7ZMpiEiDktbFEVR8gEiAnt0qCaV554LJk2ahO3bt+e6GUoImThxIrZt25brZiiKb+64A5g0\nCbj++swfK/Tivn379oS9wkrfhMg1YFGU0PL++8B//AdQXg5ceWVmjxU6z11RFKW30tIC/OIXwC23\nAG+/ndljqbgriqJkiZYW4NRTge9/H3jxxcweS8VdURQlS7S2AhUVwLRpwJYtmT2WiruiKEqWaGkR\nv33yZGDz5sweS8Vd8c1f//pXjB8/PtfNUJS8paVFIvfJk4GtW4Eer0LpAaDirviGmTVDRVFSpLsb\n6OwE+veX6H3wYKCmJnPHU3FPk/vuuw9Tp07FwIEDMXPmTLz00kvo7OzEkCFDsG7duiPr1dfXo6ys\nDPX19QCAn/3sZxgzZgzGjRuHRx55BAUFBdiSwIS79tpr8Z3vfAcXXnghBgwYgDPPPBN1dXW45ZZb\nMHToUBx33HFYtWqVZ9tetPXgfPvb38Zll1125Pmtt96Kc8891/PYbW1tuPDCC1FTU4MBAwZg4MCB\nqK2tBTPj3nvvxdSpUzFixAh86UtfQlNTEwBJYy0oKMDjjz+OCRMmYNiwYXjooYfwwQcf4IQTTsDQ\noUNx4403HjnGokWLcMYZZ+DGG2/E4MGDcdxxx+Gtt97y+U0oSrhpbRVRN/HRlCkZtmZyXUXNVt2M\n3fBaHhaef/55rq2tZWbmZ599lisqKri2tpb/6Z/+ie+4444j6/3yl7/kCy64gJmZX331VR49ejSv\nX7+eDx06xF/+8pe5oKCAN2/eHPdYX/va13jEiBG8YsUK7ujo4LPOOouPOuoofvLJJ7mnp4fvuOMO\nnjdvnmfbysvLjzxva2vjY445hhctWsSLFy/mESNGcE1NTdzjV1dX8/jx46OW3X///XzaaadxTU0N\nd3Z28je/+U2+8sormZl527ZtTET8rW99izs6OviNN97g0tJSvvTSS7m+vp53797NI0eO5MWLFzMz\n8+OPP85FRUW8cOFC7urq4meeeYYHDRrEjY2Nru0J+29DUezs3s08apT1/CtfYX700fT2GTkH3DXV\n64Vs/6Uq7kAwf0Fx4okn8ssvv8xvvvkmT5ky5cjyuXPn8pNPPsnMzNdddx3ffvvtR177+OOPfYv7\nDTfccOT5Aw88wMcdd9yR56tXr+YhQ4YkbJth6dKlPHToUJ40aRI/88wzCd+bm7gfe+yx/NZbbx15\nXlNTw/369ePu7m7etm0bFxQU8J49e468PmzYMH722WePPP/CF77ACxcuZGYR97Fjx0btf86cOUc+\nNycq7ko+sXEj89Sp1vMFC5h/9KP09hlP3PPelglK3lPliSeewKxZszBkyBAMGTIEa9euRX19PebN\nm4e2tjYsW7YM27dvx6pVq3DppZcCAGpqaqI6JsePH+97FG5lZeWRx/3794953tJiTW3r1TbDySef\njMmTJ4OZcfnll6f0/rdv345LL70UQ4cOPWIN9evXD3V1dUfWGTlypO82jx07Nmr/EydORE0mjUlF\nyRKmM9WQaVsm78U9l+zYsQM33HADHnzwQTQ2NqKxsREzZsyQq2ZBAa644go89dRT+N3vfof58+ej\nrKwMADB69Gjs2rUraj9Bd1TGa5vhl7/8JTo7OzFmzBjcd999Cffp1sYJEybg1VdfRUNDAxoaGtDY\n2IjW1laMHj06pXbv3r075n2MGTMmpX0pSpgwaZAGFfcQ09raioKCAgwfPhw9PT147LHHsGaNNdHU\nVVddhWeeeQZPPfUUrrrqqiPLr7jiCjz22GPYsGED2tra8JOf/CSwNhnxTtS2jRs34s4778Rvf/tb\nPPHEE/iP//gPfPjhh3H3XVlZif379+PgwYNHln3jG9/A7bffjh07dgAA9u3bh5dffjmmPX7Zu3cv\nHnjgAXR1deG5557Dhg0bcOGFFya1D0UJI2YAk2HKlMwOZFJxT4Njjz0W/+///T+ceuqpGDVqFNau\nXYszzjjjyOtz5sxBeXk59uzZgwsuuODI8vPPPx833XQT5s2bh6OPPhqnnXYaAKCkpCTu8fxE92ad\neG3r7u7GNddcg9tuuw0zZ87E1KlTcc899+Caa67B4cOHPfd9zDHH4Morr8TkyZMxdOhQ1NbW4uab\nb8bFF1+M8847D4MGDcLpp5+OpUuXerY50fNTTjkFmzZtwvDhw3HnnXfi97//PYYMGZLwfStK2HHa\nMiNGAB0dwIEDmTle6Ou5R+oV56BF2WPDhg04/vjj0dHRgYKCvnu9XbRoER555BEsXrzY1/p94beh\n9B4ef1yKhS1aZC074QTgsceAk05KbZ/x6rn3XSXJMS+++CI6OzvR2NiIW2+9FRdddFGfFnZF6e04\nbRkgs767qkmOeOihhzBy5EhMmzYNRUVFePDBBwEAM2fOxMCBA4/8mQFDv/vd77LSrn//938/ckz7\n32c/+9msHF9ReitOWwZIz3c/dCj+62rLKHmL/jaUfOLOO4F+/YAf/9ha9qtfAcuXA//7v8nv76OP\ngOnT1ZZRFEXJKW62zJgxwJ49qe3PNmTFFRV3RVGUAGltBZ57Lna5my0zfDiwf39qx0lb3IloHBG9\nRURriWg1Ed0YWf6vRLSKiFYQ0WtENMpj+/OJaAMRbSSiW1N5E4qiKPnC3/4GXHGFZMfYcQ5iAoBh\nwxKLtBeJLgp+JsjuAvA9Zl5JRBUA/kFEbwD4GTP/GAAign8XgG/ZNySiAgC/AHA2gBoAy4joJWbe\n4PcNTJw4UcvMKq5MnDgx101QlBj27JGp9H7wA2DqVMAMfXGzZYYNy1zknlDcmbkWQG3kcQsRrQcw\n1iHQ5QDcys7PAbCJmbcDABE9DeBiAL7Ffdu2bX5XVRSll3H4sHRC5hO1tSLod90FfPGLgKmo4WbL\nDBkig5i6u4HCwuSOk+iikJTnTkSTAJwIYEnk+U+IaAeAqwD82GWTsQB22p7viixTFEVJyKxZljjm\nC3v2AKNGAZ/5DNDQIBE74G7LFBYCgwYBkSkQkiLtyN0QsWSeB3AzM7cAADPfAeCOiJd+I4AFyTfR\nYsECa/OqqipUVVWlsztFUfKcmhqgsREYm0ch4Z49wMkny6QclZXA3r3AUUe52zKA5bsPG5Z439XV\n1aiurgYAvPde/HV9iTsRFUGE/TfM/JLLKk8B+DNixX03gAm25+Miy1yxi7uiKH0bZqC5GWhry3VL\nkqO2FjBFUSsrgbo6EXc3WwZIzne3B71vvgl89NHdnuv6tWUeBbCOmReaBUQ01fb6JQDWu2y3DMBU\nIppIRMUAvgTgZZf1FEVRoujoALq68k/cjS0DACNHSuQOuNsyQOqdqmnbMkQ0F8DVAFYT0QoADOB2\nANcT0TEAugFsB/DNyPqjATzMzPOZuZuIvgvgdciF5BFmdrsIKIqiRNHcLP/zUdydkTvgbcukmuue\ndiokM78HwK0f9zWP9fcAmG97/hqAYxIdR1EUxY4R90Q1VMJEa6tk+AwaJM9N5N7VBXR2AqWlsduk\nkuve0yN9EfHQEaqKooSSfIzca2vFkjFDc0zk3toqlozbkJ1UbJmmJmDAgPjrqLgrihJKghL3P/wB\neOqp9NvjB7slA0jkbsTdzZIBvMU9XmReXy92TjxU3BVFCSVm3vR0bZl//AP44IP02+MHp7ibVEiv\nTBlARNrNljnrLMBr5ks/qZMq7oqihJKgIvdsplMaW8ZgIvd44h4vcvcaoL9/f+LI3fcgJkVRlGwS\nlLi3tEgnZzbwityN5+6Gl7i3tgK7drlvo7aMoih5S1DZMtmM3J3iPnSo1I5pbEw+cm9t9S69sH9/\nYltGI3dFUUJJczMweHAwkXu2cNoyhYUiwtu2JRZ3ZiubpqdHLmpe4q6Ru6IoeUtzs9ga+eS5OyN3\nQHz3LVu8bZmSEvkzdyqA1d544q4dqoqi5CVG3NO1ZVpavMV94UKJkoPCTdwrK4HNm70jdyDWmjHt\n9fLc/XSoqrgrihJKmpsl6g3ClnHbR08PcMstwMGD6e3f0NUlJX5HjIheXlkpkXsy4t7aKqNc1ZZR\nFKXXkWlbprlZfO6OjvT2b9i7V0S6yNGTOXIksHWrty0DxOa6t7ZKmePu7mi7xuCnQ1XFXVGUUBKU\nuLe0WBNm2DETZAQl7vZqkHaMtZRs5F5eLgLvFr1r5K4oSt5ibJl0PPfubrk4uF0gzPD+zs7U92/H\nzW8H5D0AyYt7WZmIu9N37+kR+2fo0Pjt0VRIRVFCSRCRe2sr0L+/7MOeaggEH7nX1Ul7nZhl8WwZ\nr8h96NDYyP3AAblQJJpbViN3RVFCSRDibnLli4tjRTxocT9wQI7lxE/k7ua5e9kyfiwZQMVdUZSQ\nEoQtY2q6lJXFXiSCFvfmZmDgwNjlJnIPynP305kKqLgripJB2tuBf/mX5LdjFmFONxWyuVnqnmdD\n3A8edBd3kxqZjC3T1uYt7jt3uts/TlTcFUXJGHV1wMMPx1/nnXdi12lrk1GbFRXS4dndndrx7ZG7\nM2Mm3Q7V3/1Oyg0YDh50n0CjtFRy1hNF7l62jLND9Q9/AC64IHH7VNwVRckYra2JbZVly4DFi6OX\nmYibSMSxvT2142fSlrn3Xmm7vc1ukTsAXHcdMG6c976GDLHaA3jbMq2twKuvApddlrh9Ku6KomSM\ntjaJjOMN8W9sjBVeI+6AuzD7JZO2zPbt0YLsZcsAwM9/bs2r6sbgwdIhazDiPmqU2DWmZPHLLwOn\nnx47CtYNFXclNHR0BOd/KuHAWCHxIu+mpsyJe6LIvX//1H5zTU0ixnZBtrc5WQYOlIuDuQiaPPei\nIhFyY/889RRw5ZX+9qniroSG++8H7rsv161QgsSPuDc2xlo3dqHs3z/1jBmzn/Jyd3GvrEzNczcz\nJNnFPV7knoiiInmfpjyxfXIP47s3NIh9dcklPveZWlMUJXgaGoIbLaiEAyOo8cQ5V5F7Y6PYHqlE\n7kbc/doyfjDWzMCB0eI+fjxw441SG/688/zfHSQUdyIaB+AJAJUAegD8LzM/QEQ/A/A5AB0ANgO4\nlplj6qsR0TYAByLbHmbmOf6apvQ1Wlulsp7Se/AbuWfScx8xwtuWmTw5dXEvLQ3OlgHEk29qEjG3\ni/vPfw6sWyfnxmmn+d+fn8i9C8D3mHklEVUA+AcRvQHgdQA/ZOYeIroXwG2RPyc9AKqYudF/s5S+\nSGtrsLW1lcyxYgUwa1bi9Yy4J4rcneIflC3T0gIcdZR7KmRTk+TRu4n7K68ANTXADTe473f7dmDm\nTEvcmYOL3IFocZ84Uf6SJaHnzsy1zLwy8rgFwHoAY5n5TWY2p+L7ALwSfcjPcRTFq8ATAGzYkN22\nKPE57TSx0RJhvs9cRu5utkxXlwjo8OHu4v7OO9Fpjk62bQNOOMES444OSdssKUmtnYAVuQPWIKZ0\nSEp0iWgSgBMBLHG8dB2AVz02YwBvENEyIvp6sg1U+g5eOdFdXcDxx8vIPCX3HD4sYuY2qbMTv5F7\nJj13t1RIE2X37+/ez7NrV/y5V424GzFO15IBJHI3+7NH7qniu0M1Ysk8D+DmSARvlv8I4qU/5bHp\nXGbeQ0QjICK/npnfdVtxwYIFRx5XVVWhqqrKb/OUXkBbW3TVPkNNjQj8kiXiRyq5xQj1/v3AtGnx\n103kuZv0V2dfS3OzVT7XVHVMBXuHal2dtbyxUcTUOXepYefO+HnpRtwffFCep2vJAN62jJ3q6mpU\nV1f72p8vcSeiIoiw/4aZX7It/xqACwGc5bUtM++J/N9HRC8AmAMgobgrfY/WVskIcGIi9iVL/I3M\nUzKLEWw/kXuibJmmJquuyuHDVhnb5mbg6KPlcVlZ8KmQTU0yKtStWiQgvzm33yIgAnz4sHTGGjEO\nQtzttoyXuDuD3rvvvttzf35tmUcBrGPmhWYBEZ0P4F8AXMTMrv3NRFQWifhBROUAzgOwxucxlT5G\na6t7hLZjh2Q8LHGagTZqaoBFizLXNsXCfEdB2DKNjSKyTgHPdCpkU5MVuTvFvadHhvy7zd4ESGfq\npEnRNkpQtow9ci8rS29/CcWdiOYCuBrAWUS0goiWE9EFAB4AUAGxWpYT0YOR9UcT0SuRzSsBvEtE\nKyCdrn9k5tfTa7LSW2lrcxeBnTuBiy8Gli/3TpVcuRJ47LHMtk8RkoncW1slOvayZYy4OzNinNky\nQZcfsIu703Pfu1cicy/Pfds2Effyctn28OFgI/fOTsm+KS5Ob38JbRlmfg+A2w2Kq9sWsWHmRx5v\nhXTAKkpCWlvdsw127JAO1XffBVavdk/Ba2/X0gXZIpnIva1NMlLi2TKDB8eKrzNyb0wxkdqrKmS8\nyH3nTmlzInEnEkE+cCBYz91YMm79T8mgKYpKaIhny0yYAJxyirc1096uo1uzRbKR+7Bh0ZH7pk3W\nd5VJW4bZOxXSHNdL3KdPjy/uJu/ciHsQtoyJ3IPIlAFU3JWQwBzflhk/PrG4a+SeHZL13IcNi/5e\nv/514PWIOWsiaKf1EoQt09lp5Z572TJuHaq7dsUXd+O5A5YgBxW5m7RQFXel19DeLtkJ7e0i9Hb8\nRu4q7tmhtVWi3lQj9+ZmqzaLPXKPZ8ukki1jonazD78dqjt3AlOmyGO3u0FjywCWlZIJWyZdVNyV\nUNDWJidzUVH0CWWsmuHDxXffvj26nofh0KHYE/Hb305v/k3FnbY2udim6rm3tsr3CFgpiZmwZcwA\nJsA9FdKrQ9XcKVZUuEfvW7dGR+5qyyhKHMwP2nkLbk40IsmBPu44KaLkxC1yf/JJfwKkJEdbm3wn\nqUbuLS1yNwZYg4ns33t3t6xvBC4dcY8XuXt57rt2yaxJbuLe0CDtGz5cngdty2jkrvQ6TF6vM4Iz\n4m4YNiy6zKrBrUO1o0Mj90zQ2uo/cnfz3L0idyO+LS3R2SKpFg5zRv/2bBlzUXHz3ONF7ps2AVOn\nWm0L0pYpLZULR0ND+jnugIq7EhJMJ5LzRDZ+u8E516TBGbkzi9iruCdPc3P80sttbTKorLs7/ufL\nLK8PHRobuRtxt0fuZl8HDkQP/Q8ycjf9OV6ee3c3sGePTJDhJu4ffxxdciFIW4ZI2lRTo5G70otI\nZMsYBg92z3l2iruJ4lXck+db3wKeftr7dfNdmbIBXhw6JOJZXm59D2aATkODfF9ukbszCk5V3O0d\nqv36iXiauUi9xL22Vi5GJSWyrXOU6scfS+RuCDLP3exv924Vd6UX4WXLOCN3+5BvO+3tEnV1d8tz\nc8KquCfP/v3Axo3er5u7rETibr7T/v2tyL21VURz7Fi5cJvI3S7gzsg91VRIe4cqEH0Mrw5V47cD\n8h7dbBl75G5+j83NwYi7Ru5Kr8OvLRNP3AHrRFVxT53mZmDLFu/XjWgnEnfznZaWWt+D8dMnThRr\nxkTuTlvGGbmnmwoJWBkznZ3yV14e67nb7xS9bBmvyD1dWwaQ37dG7kqvwh7lxetQTSTu5kRVcU+d\n5mZJ9/Oirc2fuNutNmfkPmGC5IsfPCgC6bRlgvLc3SJ3E7WbAU7JiLszcs+ELRNU5K4TZCuhwER5\n9hIEzP7F3e7pAtkV964uyc/vLTQ3S/EsL5KxZZyRu1k2cSKwZo08LiyMtWXsQmlsGebk6q3YO1QB\n6xitrTK9HhAr7nZbxinuDQ3i2Y8YYS0zv0fnhSRV1JZReh3mpLffgu/fL8JgP0H9Ru7meTbE/ZRT\n5Ha9t9DcLB2LXtFyMraM03M3gjtxIrBqlXyfQPQdmzNyLyqKHdzm9304I/fWVmD9euDYY2WZ03Ov\nr7fE2ynumzdHp0ECVqRdXBzMBX7QoOgc/3RQcVdCgZstU1cHVFZGrxcvFRLITeS+dSuwb1/mj5MK\nV18t1TSToblZbBMvaybZyN3+nZplEyZImeYhQ2R5vA5V5+t+8Yrc162TwXCA3DUQWamf9gjcKe5O\nSwaQdtbWBmPJANbFTvPclV6DvUPVfpKbH7shXiokkH3PvbNT2uM2VVuuqamRlMZ4mS9ODh8WoTvu\nOO9OVb+Ru92WsUfuxpYxdWWA6Ds2N/862YyZPXuA//s/mS3JYMR9/XpL3IHoTlX7BcEp7s7OVEDE\nnTk4cTcXNY3clV6Dmy3jFsHFs2UKC7Mv7iZijzeZcq747W9lVqFk7ipM5Dplire4+43c7baMPXKv\nqLD6Uey2TKLI3e93WV8PnHMOcMMNwNlnR+/DRO7GlgGiffd44u4WuZeWyvZB+O2A9XmouCu9Bjdb\nxmQ12DG1PJyVI9vb5bVs2zKm4zFs4s4s0w7On5+cuJv0wcmTMxO5262akSPdbRm3yN28vnChNSm1\nF1deCVx0EXDbbdHLy8vlwrF5szU/KxBf3O2DmNwid0B+d0HbMpoto/Qa7LZMQ4Msc4vgSkpktKGz\n5nV7u5xg2Y7c6+rkf9jEfcUKEabPfx6orva/nemEnDwZePtt93WS9dxNp2VPT7R4TpwY7THHu2Pr\n3x/461+Bn/4UKCgQW8U2T/QRurqA994DXnwx9rWyMuDDD+WuoX9/a7m9U9XePucgpmyIe5C2jIq7\nEgrcyg80NcWe5IBlzTjFffjw7Iu7idzD5rk/8QTwla9IdBwvrdGJXdzdInczqUr//v5tGZNP3t4e\nXfFwwgQrcnfaMm6R+w9+ADz+uJQHuOoq4IMPgDFjotfbuFFGv7qJY1kZ8M470X474M9zP3BA2u/s\n4Afk96i2jKJ4YITAGcE5bRnA3Xc/dCg3tkxYI/c//hG4/HIR92RtmQEDgKOOkkFGbvZXcbH0bwwZ\nIt+RKfngxC7kJh3SdKgCwLnnWvPhxhvEBMg2l14KfPGLst0//RNw662xx1y9GvjEJ9zbU1YmGTpO\ncffjudtLTzvRyF1R4uCWNnfggDWgxM6QIdEZM8yxtkx7u4hUNiL3ysrwiXt9vXx2TU2pifuAAfJ9\n1NUBo0ZZr9vtsKIiWa+pSaJ4J6Y0MGANZDIdqgDwjW9Y6zov6k6xfOCB6Kj56quBCy+MPeaHH8qk\nLm6Ulcnvw0vce3qkDSYN0S7uu3fLHYEbYfXcNXJXQoG9Q9WvLWPo6hIftrw8OnIfPDg7kfuUKeES\n9+5uK+tlxIjUxB1wt2bM92SIZ83YLwQmcveaiML+vbtF7pMnR2939NFyXOexE0XugLu4d3ZadlNB\nRBX9inuQtszAgfJ70jx3pddghCAVW6a9XSJDu3fa0SERftDiXlMDXH+99XzvXjkZw+S5HzwowlRY\naAmis3StF3ZxP+qoWHF3dmSPGiWWhRv2C4GJ3J0Diwzmjq29XSLo0tL47SwoAE46SXx3O4kid0Am\nv7ZjIndn2/yK+1lnySjlICgokI7bggCUOeEuiGgcEb1FRGuJaDUR3RhZ/jMiWk9EK4no90TkemNC\nROcT0QYi2khELi6ZonjbMn4idyPudu80U5H75s0yMMh40Xv3SlQZpsjdeVFMJnq3j9D0E7mfcYZk\nsbjh5rl7Re6FhXJx3rdPolc/NWRmz44W9wMHxI6yD1yyYwZPOY9vggK3Ea2HDsnFZvfu2M5bw1VX\nSV592PBzfegC8D1mngHgNADfJaLpAF4HMIOZTwSwCcBtzg2JqADALwB8BsAMAFdGtlWUKLzy3JMV\n90zbMo2N0tY9e+R5XV34xN05PiAZcbeXyR01KjbTxhm5n3MO8Oab7vuyr2v33L385P795XN1+87d\nOPnkaHFfs0Ysl8JC9/UrK2UbJ16Re0GBZRfFi9zDSkJxZ+ZaZl4ZedwCYD2Ascz8JjP3RFZ7H4BL\n1xfmANjEzNuZ+TCApwFcHEzTld6CSa/zU34A8G/LDB6cWqnYeJiO3I0bpd379om4h8mWSVfcTeQ+\nbJhEwnackfvcueJzHzwYuy/7uvZsGTdbBpB1k6nTMns2sGyZ9fzDD739dgCYNw949tnY5V7iDli5\n7jU1vVDc7RDRJAAnAljieOk6AK+6bDIWgN2R2xVZpvQR1qwBnnwy/jodHVblP7vnHi9yt2fLZDty\nB4CPPpLHZWWSXx+2yN3+uaUq7sOHx3ZYmpRVQ//+4je7WTP2KN1v5F5b6z9yP+oo2ae5i4rXmWpw\ns3vM78ZN3M0o1XyM3H2nQhJRBYDnAdwcieDN8h8BOMzMT6XbmAULFhx5XFVVhSq3IWhKXvHeezIM\n/stf9l4SYdBNAAAgAElEQVTHLhjGlunuluVuWQjOypCHDrl77sOHZ0bcy8slcjdpkG6TOuQSN8/d\n70AmP5G7U5zPPlusmc99znvdTETuRJbv/rnPSeR++eX+trXj5bkD8ryxUUZNuw1gyjbV1dWo9jnk\n2Je4E1ERRNh/w8wv2ZZ/DcCFAM7y2HQ3ANskaRgXWeaKXdyV3kFDg0Tv8SZacIpAW5s1bZlb1kA8\nW8bYI5mM3GfPtsR95MjwiXuQtkyiyB0Q3/3aa2P3ZV/XT+RuxN1v5A5Y4j5tWvxMmXjEs2UqKqRg\n2MiR3l5+NnEGvXfffbfnun5tmUcBrGPmhWYBEZ0P4F8AXMTMHR7bLQMwlYgmElExgC8BeNnnMZVe\nQEODCMaOHd7rOMX90CFvSwZwF/f+/aNtGVNI7NCh2FGW6dDUBJx6qoh7XV20uAd5nHRwinsyo1T9\n2DJOcT7pJPGkjT1icLto2wcJOTEdqskMCDr5ZOBXvwLOPFPqzgwf7n9bgxF3+wArQ0WFWHBemTJh\nxk8q5FwAVwM4i4hWENFyIroAwAMAKgC8EVn2YGT90UT0CgAwczeA70Iya9YCeJqZ12fovSghxIjD\nmjXe69gjvJISqSne2OjemQr471A1ud7JzuATDxO5b98uPmxlpfQVFBeHZ77WoCL38nIZIGZ/X84O\nVUA+43nzgL/8xVrmHO1ZWioXevsgISepRO5VVVLad+1a4Lvf9b+dnUSe+0cf5Z/fDviwZZj5PQBu\nNyTTXJaBmfcAmG97/hqAY1JtoJLfNDRIx9fq1cBnP+u+jj3CI0rcseY3z72kxLoTKCkJ5v00Noqg\njx0LLFkCHBP5ZZvoPYiRhely4AAwc6b1PNU8dyIrejdlINra3D3zq68G/uu/gK99TZ6b78QIef/+\n4t/HG1ZfVia11pMR98GDgX/7N//ru2F+N4cPx7avokJ+u6efnt4xcoGOUFUySkMD8OlPx4/cnT5s\nonxnM+O8sUG8smXs4h4UZvago4+WCoOmk23AgPD47kHluQOxvrtb5A5Ih+aWLSKEbuuVloq4e3Wm\nAtZFPag6LX5J1KFqKk3mGyruSkYx4m5OejecnXRlZSLuXrZMv34iFqbz1MuWyZS4Dx4s4r57t/jZ\ngIhAWHLdg7JlgNiMGTfPHZDv5PrrgYcekuduF2w/kbvpK8kmifLcW1tV3BUlhoYGGaK+caPc9rqR\nbOQORKdD5ipyB6LFPayR+4AB8rmY2ZCc/OMf8t9MqGG3sJydql6ROyDi/tRT8jk4LwJG3ONF7ma/\n2Y7cE2XLACruihIFs4j72LFSC3vTJvf13AbGJOpYs/vuXnnuQYt7e7vk35eVWV57PtgyRN7R++7d\n0kF84IAVtdtTVt0idy9xHz8e+NSngHvvlQuCmy0TL3I3syPlInKP16EKqLgrShSHDln1OWbO9Pbd\nvSJ3L1sGiBZ3kwqZaVvGRO1E4Y7c3QqueYn7O+/I/82bYy0ZIDZy97JlDPfcIyUBzjsv9jvdvz+x\nLQOEz3MHemkqpKKkSkODTIkGyOASL9/dKe5+UuKc4u60ZdrbMyfugGSP3HST1caweO7M3uLuNkrV\niPvHH7uLu98OVcOMGZISuX498N//bS03fSR+bJkwee4VFfKX7QtOEKi4KxnDLu7xInc3WyaR526v\nL5OtDlW7uBcUAAsXWhZGWCL3lhb5LPr1i14eL3KfN0/E3Z4GafDboepk0iTghBOs58ZyCastE28Q\nUz5aMoCKu5JB7OJ+zDHenrubLdPeHt+WsUeUXnnupaWZE3cnYfHcnX67wW2i7IYGmSf18su9I/dk\nOlTjYSbf8BO5BzWrkV/iRe4TJwKnnZbd9gRFrxH3lSvlVlAJD3ZxHzjQW/zcbBkgfgQ3fLgVUWYr\nWyaeuIfFlvGa4GTy5NiL63vvSSmF6dMtcXeKW6qRuxM/kXtZmbxelOWZneN1qJ5wAvDYY9ltT1D0\nGnF/7LHEpWWV7GIX93i2hZstAyQv7pm2ZZqa4ot7mCP3E0+UAMjO4sVSk2Xq1OA8dy9M5J7IlsmF\nt23vUA1iYuqw0GvE/cCB2MJFSm7xK+5utgwQ35YZMcI7cmeW/8XF4bVl/vY377xzJ889JymYfvAS\n9098Qvo87Pt55x1JXRw7VrarrfWXLZOKuJvvNJEtk22/HZDfTXOz/G6Ki7N//EzRa8S9qUnFPWzY\nxb2kRIpQuQ1k8hJ3v5G7M8+9s1M6FE0aZpDi7nXBSTZy/8Y3gLfeSrweM3DNNd79FYC8P5P14iXu\ngwZJTv7HH8vz1lbJXpozRz6nyZMlsneK+6BBIuidnfL9dXWlVqfHT+Q+dmziyTYyQUmJXMAqKvzN\n3Zov9Bpx18g9fNjFncia1caJW/mBoiJL5N1w2jL2PHdjyQDpi/tf/2pN5ebXc3/9deDPf46/39pa\nKZKViIMH5f1s3eq9zvPPywUA8J6aEBD/2FgzixcDn/yk9RlPnQosXx4r7kTyHTY0WN9TKgLoJ3I/\n+mjgmWeS33e6lJTI+4vXtnyk14i7Ru7hwy7ugHd06xa5Dx4cX0SGD7dS+5y2jMlxN/tKVdx7eoBv\nftOql+LXlnn2WSDeZDmHD8uFae3axG2orZX/W7ZYy7q6ou2VV1+VEsT79sWvg2/33V95JbpKp/Hd\n3TJVjO+eamcq4C9yzxXFxe5pkPlO3or7//xPtGd54ID8uLu6ctcmJRq/4u7WoZrIex06VMS2uzu2\nQzWoyP0vf5EJOT74QJ4nitzNe1u71urYdcNclFIV9+9/3ypz290tdwrTp0s7vWwZwBJ3ZuBPfwLm\nz7demzpVlnuJe329XEBGjEjcZjf8ZMvkCvNbUXEPCf/2b9E/+AMHxGf1O1ekknlSjdzLyuJ3pgLy\nXQ8cKGJmxL2wUPzj1tZgxP3++6VOysaNsg8/4s4soh2vs7SuTgb5rF+fePam2lq5aNl/68uXy+xD\nnZ3A0qUyNP6ii/yL+9q1cld03HHWa9MiszO4ibvpVP3d74AvfCF+e70Is4CGuW3pkLfi3t4u4gFY\nQ66PPlqtmTCRji3jJ2vC+O5G3AEr88E8T1Xc164FVq0CvvpV4Nhj5XEiW6a5Gdi5U/7Hi9xra2VQ\n18CB8acfNOvOnm157ubiMWIE8MILYslccIFMN5dI3MePl3Y9+qhYMnbba+pU+e8mcMOGSTueflom\n5UgFIvlONHLPHr1C3E2kNn68inuY8CPuzLG2zMyZ0ZaBF27iXlwsnZDpRu4PPih+e0mJNQmzn8jd\nWC3xxL2uTjJXZsxI3KlaVyezAG3ZIp+VuTP98Y+ljUbcZ8+Wjt94HapE0qn60EOxn++4cfLZeUXu\nzzwjdxvTXOdf80dpaTgF1KQ/hrFt6ZCX4s4sJ7TJvzXRyujRKu5hwaQk2iM1N3Hv7BQrxV4LZcYM\n4JZbEh/DdKo6I/cgxH31aqm5Aohw/u1v0lYvASgrk4vUmjUilPFsmdpaYNQosUUS+e61teKnFxXJ\n733tWtnukkskPXLTJmDuXBkm39mZeJq6E0+U/1VV0csLC+W1UaNitxk2TLJrvvzl+G1NxMKF7vvP\nNea3Esa7inTIS3E/fNiqFQ5YQ67DKu4tLRKB9SUaGyVqt9/6u4m705JJBjOQ6dAhq8PO2DLpivve\nvVY539mzgTffjJ/BU1goF5hly4CTToqO3NvagJ//3Hpuj9z9iHtlpcxDu2WLiPeMGXIx/OY3gc98\nRh4TiTWze3f8/opTTwUuvNC6GNpZskTy3Z0MGybv74tfjN/WRHzlK9kvLeCHggJpl0buIcBERSZy\nD7u4L1oE/PCHuW5FdnFaMoC7uKc64hGQyH3vXrnYm1vroGwZu7jPnCn79LJkDAMGiEB+8pPR4r5j\nB3DHHZJaCViC7ceWMVH+5Mniu5vIHZB9Pv64te7s2fI/nrhfcUXyueTTp0vfQ6qZMvlASYmKeygw\n4m4i97DbMnv3hrNdmcSvuKcTuQ8fLpFqSYkVUTttmbKy5MW9s1OifyPm/fqJZZFI3CsqRMhnzYoW\n9/Z2acPu3fK8rs6yZdati58xYxd3e+QOWCNwDUbc49kyRLJdMpx+OvDII8ltk2+ouIcEc+Lkiy1T\nX9/3bJlkIvd0xH3XrmiLwY/n/vrrwE9+4r3f+nrZt10EZ89OnJ5ZUSFpiaNGRXvu5rEpIWAi98GD\nJWNm5073/fX0SFtGjrTE3R65Ozn5ZLl7cLNclPgUF/dBcSeicUT0FhGtJaLVRHRTZPllRLSGiLqJ\n6KQ4228jolVEtIKIlgbRaKctE/bIvb7eGozSV9i/33/kno4t4xR3P7bMK68AP/2pNdmHE7slYzjj\nDOkojceAARJV2+vKA7HibiJ3IH6n6v79Iv7FxeK5L1kigu/VKTlmjOTkK8nTVyP3LgDfY+YZAE4D\n8B0img5gNYBLAfw1wfY9AKqYeRYzz0mrtRGctoyJ3EeNkhMn0cCQbFNfL1kdfiv79QayYcuMGCFR\nrzNyT9ShunKlBAK//rX7ft3E/YtfBB5+OH57KioSi3tnp1x87JOYeAmyifABidxXr5b9xyvLEMZs\nlHygT4o7M9cy88rI4xYA6wGMZeaPmHkTgERlhMjPcZKhvV1ObGeHqhkkYS9RGgbq60XYw9auTLJv\nX3Y6VPftc4/cvQYxMQMffgj88pfy51auwk3c/XjVgwfLXLElJbG2TGGhiLjZt9nXxIkyrN8N47cD\nwIQJso2XJaOkR58UdztENAnAiQCWJLEZA3iDiJYR0deTOZ4X7e1yC+rsUAXkZAibNbNvn1x8+pLv\nvnKlCJ2dTHSoAtGdik7Pvbg4utDWtm3SjvPPlxKzL78cu183cffDAw9ILnhpaWzkbqYZtEfjgH9x\nLy6WQXqmM1UJlt4o7r6zTomoAsDzAG6ORPB+mcvMe4hoBETk1zPzu24rLliw4MjjqqoqVDlHWkRo\nb5cTe906OYnsU4sZ390pLLmCWSL3k0+WkzUs7cokzFLzxJ6mBwQv7oMGSX6y05bZs8cSdyIreq+o\nkIuOGcjz3e+K1fL5z0fvN1VxNxcbN1tmxgy5kNTURFsnEyd6lyCwe/MAcMopUn9dCZ5f/So/zs3q\n6mpUxys5asOXuBNREUTYf8PMLyXTGGbeE/m/j4heADAHQEJxj4ep323qTNsj99Gjw9V5afzfiRMz\nH7n39Mhnk6rN4aSrK7VBJ6Z0rNP/DdqWIRJBjdehCniL+8yZ7pkqe/emN8zeTdyHDpULxtKlqUXu\nQG5qnfcVPvnJXLfAH86g9+677/Zc168t8yiAdcy80ON1V9+diMoiET+IqBzAeQDW+DymJ6akqxF3\nt8g9LJi0ulGjMn/R+fOfZSb7oPj852Ojbz8sWeIeYQYduQOx4u60ZYBo390u7oMGyW/HSaqRu70N\nHR1Wx74pjzBtmsyYZBfskSMlAGhri92PU9wVJRn8pELOBXA1gLMi6YzLieh8IrqEiHYCOBXAK0T0\namT90UT0SmTzSgDvEtEKAO8D+CMzv55uo83JYiYRyAdxr6zMfOReVyeRYVDZQjt3ysjagweT227J\nErEQnASd5w4kL+6rVmVe3AsL5c9MKWgXd2fkXlAgXrqbNeP05xUlGRLedDPzewAKPV5+0WX9PQDm\nRx5vhXTABoo5WbxsmSXJdPdmGHvkvibte5b4NDXJ8XbtEsFIl4MHRZB++lOpa+6XpUvd7yC8Ivd0\nhHT48OiiY8XFIuRu4t7YKL8XUz9lwAA5fne3iLEhXXEHrOi9uDha3NvbY6PxCRPEmpk+PXq5Ru5K\nOuTlCFW7uDsj95EjrZlukmX3bnc7oacHWLHCmiMzGbIZuTc1yf8VK4LZ38GDkgHy8MPRk0XEo6ND\nLmJuHqYpBWBqrACZsWXs/wFL3FetkgmYTRpiQUH03KeAVVY3CHE36ZDm93r00fLcGY17+e7ODlVF\nSYa8Fvdhw+REbGuz0phGjEh9NqYtW6Sq37Zt1rJ775WUuVNPBZ57Lvl92sU90557U5OMaFy+PJj9\nNTdLCt9XvyrFz/ywcqVEqG6CbWqh2P3ldG2ZESNiO1SBaHEfPx64+WapzHii4z7Sac20tkpHbbrl\nX+3pkPbIHYgVbDdxP3xYvs9hw9Jrh9J3yWtxHzpUhHjgQCsaSydyN5G1yTRqbQXuuQd4+23guutS\nqy5ot2UyHbkfOAB86lPBRO4dHWJXlJYCZ54pFz0/LF0aP13Pac2kU34AkGOZglmAJep2wX/ySeDO\nO6Xw1yWXRG/vFPcgonbTDqe4T54s2Ud+xH3FCpkco9DLEFWUBOSluNuzZbZsia6CN3y4+KqpDPWv\nrZWIzYj7m2+KeEyfLsfr7Ex+n/X1El2OGJF6u/zS1AScdVYwkXtzs3jSpk74smX+Omqrq907Uw1u\n4p5OlDx/vlx4DW62TEkJ8LnPyd3HOedEb59NcS8uFsvKWV3STdwXLgS+8Y3026H0XUIt7s8+KzWr\nndhtmS1boqv1FRXJCWtGryZDbS1w8cUSqTNLgSkzHZkzd9kv+/bJBaeoSE7qVO8q/NDUJOVmm5vT\nP05zs9wRAWJL9esXbVe58dRTYsvEm0TZKe7p2jJO3GyZeGRK3EtLYz13QGwuJ86BTLt3y/R511+f\nfjuUvkuoxf3jj90r5tltmR07YutXp+q719YCn/60ROhbtgB/+pNEfICIRiribmwZIPOdqk1NcgGZ\nNSt9a+bgQUvc7dG7F6tWia/9wgvxS+MGbcs4cYvc45HNyN2LsWMlfdfUufnFL4BrrklcYlhR4hFq\ncW9ocBdDe+Te1RV7EqTqu9fWSiplVRXwX/8l4mZmhU/HljHinumBTE1NIlZBizsg9pRT3NvaJANk\nzBiZx/O//1uyUeJRXp7ZyD0fxb24WI65e7dc7B5+GLjppvTboPRtQi/ubhG4PXIHYiP3kSNTi9zN\n3JZVVbEzxKdqy2Qzcj9wQC50J52Uvu9+8KB47oaTT5bOUjtbt8r/Dz4Qy+bKKxPvN9ORe9htGS+M\n7/6jHwHnnQdMmZJ+G5S+TajFvbExfuQetLibQSNVVXJHYBf3VGyZ7m5romggs5F7d7eI5sCBku63\nalV6+7N77oBkpCxfHt0hvHWriNCYMdYFLBFBd6g6ycfIHRBxf+AB4MUXpRSxoqRLqMW9oUGEwFl3\no71dTp7ychFdN1smWXHv6bEi96OPlghq7lzr9VRsGZN3bopvZTJyN5F2QYEIxa5d6ZUhcNoyQ4fK\nxWn9emvZtm2SrpcM+dChGsRE0KmI+4svAk8/nXiuVkXxQ+jFHYgV6o4OOVmIRHTcOlST9dwbG0V4\nzGTLP/lJ9LD2VGwZuyUDZDZyt5dgMHZKKiNqDU5xB2J9923bZPq3ZLCLu7lY2j/ndMnXyP3ii6VI\n26mnpn9sRQHyQNxHj46Ndu0ny7BhwUTuiep4BCHumRylahd3QKySdAqomTx3O86MmXQj96AtGcB9\nEFM8Bg2yyjYA1t1bujg9d/uEIm6ceipw9dXpH1dRDKEX9+nTY4XaLu5ukXsq4p7opC4uTt6WcYr7\n4MHJV1j0i72+DiAXxZqa1PfnFrmfcIJMUWdIV9yDtmSA9GyZw4flN5eLyF1Rgia04n7okHjGkybF\nj9xvuCF2RGRYI3dnXZUgcYvcgxb344+XSZqNlx9E5B5kpgyQni1TWyu/nSCG/Ku4K7kmhXl2skND\ng0TllZXxI/cvfzl221Q890yIuxmdajBVETOBU9zTrWvvZssMHy6R9o4d8t20tSXf+ZjpyN0+d6of\n7OK+Z49cFIMg2VRIRQma0EbujY2SNeCWYZLoZBkyRMQpGRslkbj7sWX2749+Xl8fXdXPPmlE0GQj\ncges6N1E7eQ6B5c3mfbci4ulg7bA5y/bLu41NcGJu302ps5O/xcbRQmK0Iq7idzdLBaTCulFQYFE\nmfX1/o8XROQ+c2b0MRsacifu6UbufsU9WbJhy/i1ZIDoCTsyIe6myF2yF0FFSZfQi7tb5G5SIeOR\nrO+eaEqzROLOLMJut4PM3Ychk567s0M1W5F7slRUiJgCmbFlSkuTE/fCQmlDc3NmxF0tGSVXhF7c\nR45M3pYBkvfdE816k2iEamenjGptbLSWOcW9pETWyUTZ32x47oDUjglz5D5qFPDyy8ltY6yZIMXd\neO4q7kquCL24OztUe3okZS2Rh5lK5J7IlonnuRvBiifuRJmzZrLluR97rFTr3Lgx+QFMQOY9dyLg\n9NOT28Yu7qNHB9MOjdyVXBN6cR82TITLlEP162EmI+5dXXK8eJkfiWwZN3E3JXjtZEvc0xmlyuwd\nuZeWSsT+9tvpR+6ZsGVSIRORu4q7kmtCK+6m4FZhofw3HZV+T5ZkxH3fPrmIxMtvTmTLeEXuztGz\nmfLdneJOlPpApkOHrKwTNz7xCYm60xH3rq7M2DKpYMQ96FRIFXcll4RW3E3kDkR3qibKlDEk47kn\n6kwFkrdlurpExJ3WRqZy3U25XzupliBwlvt1cvzx8j78VoK006+fzEa0fHm4Ivd9++QzTOU9uVFS\nop67klsSijsRjSOit4hoLRGtJqKbIssvI6I1RNRNRCfF2f58ItpARBuJ6Fa/DWtosCwNexSeicjd\nr7gnE7mbipDOfOtM2jLOMgypRu5efrvh+OOlimGq6X1nnw383/9lxnNPhUGDgA0bpM/Fb358ItSW\nUXKNnxGqXQC+x8wriagCwD+I6HUAqwFcCuAhrw2JqADALwCcDaAGwDIieomZNyQ6qFfk7icNEkhO\n3JcvF8GKh7FlmN1FzSnuzs5UQybEvacnNhUSSC9yjyfu552XXoR7zjkyAfRRR4XHllm/PjhLBlBx\nV3JPwjiFmWuZeWXkcQuA9QDGMvNHzLwJQLz4bQ6ATcy8nZkPA3gawMV+GmYX90xH7osXA5/6VPx1\niopE1L3SGFtaRLj9iHvQnntLi4hkkeNSnWrk7tWZaujfP7rWfbJ8+tPA++/LiN6wRO5Bi7umQiq5\nJqmbUCKaBOBEAEt8bjIWwE7b812RZQmJ57n7OVn8eu5dXcDf/w6ccUbideNZMy0twPjx0baMm7gH\n6bn//veSkujsTDVkKnJPl4ED5U7p7bfDI+5btmjkrvQufBcOi1gyzwO4ORLBB86CBQsAiM3Q0lKF\ngQOrAEgUviFi5Pg9WQYMkHz4trb4t/4rVoh/bC8T4IWxZtwEyYi7uQhl2pZhBr7/fWDePOCWW9zF\nPVOeexCcfbZE72GxZXp6gstxB1TclcxQXV2N6upqX+v6EnciKoII+2+Y+aUk2rIbwATb83GRZa4Y\nca+vBx580Orcsg9k8pstQyQXhX37RLy98GPJGOJlzBhxNxehTIv7unVy4XrhBeDyy2P9diD1yN05\nf2omOOcc4J57whO5A8HbMiruStBUVVWhqqrqyPO7777bc12/tsyjANYx80KP171892UAphLRRCIq\nBvAlAAkHh9stGSC6BEEyJ4sf3/2dd5ITd7+2jFuOOyCRahCe+yuviKifeSbw61/7j9wfeQS48065\nY/GaYzVRKmQQnHaaXOjCErkDwdsy6rkrucRPKuRcAFcDOIuIVhDR8kh64yVEtBPAqQBeIaJXI+uP\nJqJXAICZuwF8F8DrANYCeJqZ17sfycIp7vbI3W+2DJDYd+/pEXE/80x/+4s3kKmlRVLpOjvlz2/k\nvnlzahNZ/+lPwGc/C1x7rUys7CbuAwdKG377W2ubBQtEcC65RCJnQ2cn8MAD8jgbtkxJCbBokZQz\nyDWZEneN3JVcktCWYeb3AHiN3XzRZf09AObbnr8G4JhkGmXPcQesCJw52Mh93Tq5iPg9qRPZMgMG\nSLsbG+VvypTY9ZziXlUlovuJT/hrAyCfz8qV4rcXFLjPIwuINfXqq2KB1NUB990H/OEPkukye7Z0\nyBo+/hi46SYR/YMHg/Wfvbj88swfww8q7kpvJFQzMXV1STqfM3IvLZW/AweCFffFi/1H7UBiW6ai\nwhJ3r2wZp7g3NYlFkoy4v/aaCLv5HK691jvSnjFD1j/rLOCuu6wUxnHjgF27rPXM4z/+MTuee5gY\nOlQuzPbfXLoYW+bQoWD3qyh+CZW4d3S4iztgpUMmK+7OcsF2Vq8GZs3y375EtowR94YGb1umrMzy\n5bu7ZbsVK4CvftV/O/70J2D+fOv5T38af7ToCScAu3dHf25jx8aK+/DhwEsvSRsz7bmHiYED5c4l\nyAk1iorkrqqlJdg7AkXxS6hqy5g5J03RMDtGqJMR90Se+6ZNwLRp/tuXyJaxR+5+PHczqnXlSv9t\nAIAlS6LvOAoLEw+bd35mY8ZI2QUzKGvXLuDqqyXnf9euvhW5A/L7CpqSErnbVFtGyQWhFHen5w5Y\nnap+UyGBxLbMxo3A0Uf7b18ytowfcW9ulih55crkOlXr6xPXwklEcbG0z3w+u3ZJ5+YZZwAffND3\nxD0TGCtRxV3JBaEUd7cRl6lE7vHE/dAheW3CBPfX3fBryxhxd+vktIv7wYNy/IoKmdnID4cPS8Et\nt7z2ZBk3TuwaQMR93DjgoovkeV+yZTJFSYn8llXclVwQKnE3wulWBMtE7smkQppBTG5s3iz1yJ31\nWOJhj9ybmgD7QDG7uO/fL8KdKM/d1HCZNcu/NWPuaoKoXmj33Y24f+5z8lwj9/RRW0bJJaESdxO5\nu4l7qp67SaF0kqzfDkR77u+9B9x+u/WaXdy3b5fHbpN/OCP3gQOBE0+UTlU/1Nf7K5XgB3vGzM6d\n8nzsWOCZZ7KTCtnbUXFXckkoxf3gQe/IPRlx799fJocwU839+MdWJJ+quJvIvaXFsny6ukT0S0tF\n3LdudffbTZvs4j5ggIi738h9//7gJpQYO1ZsmZYWeV+mE/uKK4Kra96XKS1VW0bJHaE6he2Ru9MW\nSCVyN9sZO+feeyWNEBBxT6YzFYj23FtarDTL1laJ1IlE1Lds8SfuJp981qzkIvegxN1E7rt3y+Mg\nU9SzdBQAAA/LSURBVAEVCQaam1XcldwQWnF3i9yNuPvNlgEs333NGumMfOMNWZ6uLdPSIn9tbZYl\nA4io79rlLe52z93YMpMmyWMzT2w89u8PzpYxkbvx25VgKSkRS1DFXckFoRJ3M9ORl+eerC1j3+6D\nD2So/5tvSk2ZIGwZQPbtFHfm5GyZggIZaLRqVeI2ZCJyV3HPDOZ3quKu5IJQibupoldQEBudDxpk\nFeRK5mQxnaoffABcdplEyn//u+wnWUFz2jKAu7jb/ztxs2UAmTR648bEbQiyQ9Vky5jOVCVYzG9Y\nxV3JBaETd7eoHbDqs+/cmXrkPns2cO65wP/8DzB5cvKdhk5bBhCryE3c3dIgAfdsGQCYOlXSMxMR\nZIfqgAHS4bxmjYp7JlBxV3JJ3og7IEJdU5O8uO/YAXz0kRTnOvdc4Lnnku9MBWJtmaKiWHGvqJDl\n8SL3tjaxbuxzlU6Z4k/cg4zcARH1999Xcc8EKu5KLgmVuHd0xK8lXlmZfAfVyJHisx9zjAjrvHlS\nTyVZvx2ItWUmToy1ZUzGjJe49+sndwyHD0e/1ylTpHhVIoKM3AGxZrZvV3HPBOq5K7kkVOLuJ3IH\nkvfct2wRSwYQu2TOnNTE3WnLTJ4cG7kD8cUdsKwZp7hv2ZK4xkwmInf7fyU4TOReXJzbdih9k1CV\n/E0k7qZYVrKpkIAl7gDwxBOpFd5y2jIzZ8ZG7oB/cbfbMgMHynyie/ZIxcZDh+SzGDUqettMRO7F\nxcHuUxFKSiQQ0fEDSi7o9ZG7m7hPnZpaYSynLTNlinvk/rOfSXVFL0yuu9OCsneqPvywzIxkp6vL\nu2ZNqpiSAzoiNXjMJDOKkgtCdUr7jdyTOWGGD5dStjNnpt8+N1vGLXL/1Keinztxi9yB6E7VxYul\nI9hOQ4MIu1vNmlSZNEn+lOAxkbui5IJQ2TIdHcFH7v36yVypQeC0ZeyRezKz7bh57oDVqcoMvPuu\ntN1O0JYMIPOrfvKTwe5TEVTclVwSysg9XrZMYWFyZXqDxC1bpqlJ/uJF6k7KyiRq7+iQxwZjy3z8\nsdgkdXXWTElA8J2pgHye6rdnBrVllFwSOnF3qwhpqKzM7clibJmeHvHMzaTK27YlJ+79+4twDxgQ\n3dlmbJl335VSCUOHRs8Bm4nIXckcGrkruSR04p7Ic3/nney2yY6xZQ4dkpO2sFCsos2bkxf3vXtj\n71CMLfPOO9Ihawp7GTIRuSuZQ8VdySUJxZ2IxhHRW0S0lohWE9FNkeVDiOh1IvqIiP5CRK6STETb\niGgVEa0goqXxjpVI3AEpj5srjC1j70CtrJT0xVQid6e4jxghGTF//rNMgO0m7hq55w8q7kou8RO5\ndwH4HjPPAHAagO8Q0XQAPwTwJjMfA+AtALd5bN8DoIqZZzHznHgHStShmmtM5O4UdyB5z93YMnaI\nJHrv6ABmzIieKQkIttyvknnUc1dySUJxZ+ZaZl4ZedwCYD2AcQAuBrAostoiAJd47IL8HAfwF7nn\nEuO528XdZPAEEbkD0qk6d650qGrknt/MnGnNSaso2SapvBMimgTgRADvA6hk5jpALgBENNJjMwbw\nBhF1A/hfZn7Ya/9hF3cvWwZITdwnTox9be5cK4Nm3Dipi2PQDtX8Yvp0+VOUXOBb3ImoAsDzAG5m\n5hYiclZB8aqKMpeZ9xDRCIjIr2fmd91W3LlzARobgQceAM47rwpVVVV+m5cV3GyZVCP3vXvdB1b9\n8z9bj7VDVVEUO9XV1aiurva1ri9xJ6IiiLD/hplfiiyuI6JKZq4jolEA9rpty8x7Iv/3EdELAOYA\ncBX30tIFAIB77glnPQ43W8ZE7vZ89UQYz90rn9+gtoyiKHaqqqKD3rvvvttzXb+pkI8CWMfMC23L\nXgbwtcjjrwJ4ybkREZVFIn4QUTmA8wCs8TrI3r1iyYRR2AFvW6asLLmSAP37iwWVSNxNh6qpFKkd\nqoqi+MVPKuRcAFcDOCuSzriciM4HcB+Ac4noIwBnA7g3sv5oInolsnklgHeJaAXEp/8jM7/udazO\nzvD67YC7LTNmTPKC27+//E9UvGzAALloHDhg1aKJV21SURTFkNCWYeb3AHjFpee4rL8HwPzI462Q\nDljfhFnc+/WTcgAHDkSLu5+Jre0YcU8UuQOWNbN5s3S25qr0gqIo+UWoRqgC4RZ3IoneGxpi67cn\ng/Hn/ZQdNtbMCy8Al16a3HEURem7hErcCwv9RbO5pLhYvO9ksmOcJBu5b98O/PGPwMUXp35MRVH6\nFqES99LScEfugHvknizJiPu4ccAzzwATJrjnxSuKorgRKnEvKelb4u7Hlhk7FnjrLeASr/G/iqIo\nLoRK3PMhcg/CljGeu19bBlBxVxQlOVTckyTbtsy0aTKE/fjjUz+eoih9DxX3JCkpCa5D1Y8tc+yx\nwJo14R3YpShKOAmduOdDtkxXV/q2TL9+cqHwQ5ATYiuK0jcIlbjnS4cqkJ64DxkC3H+/RuOKomSO\nUIn7178OnHRSrlsRnyDEvaAA+Pa3g2mPoiiKG6EazH7ttbluQWKKi+V/eXlu26EoihKPUEXu+UBJ\nifjlRuQVRVHCiIp7kpSUpGfJKIqiZAMV9yQpLlZxVxQl/Ki4J4lG7oqi5AMq7kmi4q4oSj6g4p4k\nassoipIPqLgniUbuiqLkAyruSaLirihKPqDiniRqyyiKkg+EaoRqPjBunNaEURQl/BAz57oNAAAi\n4rC0RVEUJR8gIjCza7iZ0JYhonFE9BYRrSWi1UR0U2T5ECJ6nYg+IqK/EJFrPUciOp+INhDRRiK6\nNb23oiiKovjBj+feBeB7zDwDwGkAvkNE0wH8EMCbzHwMgLcA3ObckIgKAPwCwGcAzABwZWRbT6qr\nq5N6A7kkn9oK5Fd786mtQH61N5/aCuRXe8PU1oTizsy1zLwy8rgFwHoA4wBcDGBRZLVFANxm+ZwD\nYBMzb2fmwwCejmznSZg+nETkU1uB/GpvPrUVyK/25lNbgfxqb5jamlS2DBFNAnAigPcBVDJzHSAX\nAAAjXTYZC2Cn7fmuyDJFURQlg/gWdyKqAPA8gJsjEbyz91N7QxVFUUKCr2wZIioC8AqAV5l5YWTZ\negBVzFxHRKMAvM3Mxzq2OxXAAmY+P/L8hwCYme9zOYZeHBRFUZLEK1vGb577owDWGWGP8DKArwG4\nD8BXAbzkst0yAFOJaCKAPQC+BODKZBqoKIqiJE/CyJ2I5gJYDGA1xHphALcDWArgWQDjAWwHcAUz\nNxHRaAAPM/P8yPbnA1gIsYAeYeZ7M/ReFEVRlAihGcSkKIqiBEfGa8sQ0SNEVEdEH9qWnUBEfyei\nFUS0lIhm2177BBH9jYjWENEqIiqOLD+JiD6MDIa6P8ftPTmyvIiIHo+0a22kT8Fsk/H2erTVfH6r\niOilSEe4ee02ItpEROuJ6LxstjXZ9hLROUT0QWT5MiKal832JvvZRl6fQETNRPS9bLY1lfbm8jxL\n8neQ63Ms6UGcuT7PjsDMGf0DcAYkffJD27K/ADgv8vgCSGcsABQCWAVgZuT5EFh3F0sAnBx5/GcA\nnwlBe68E8FTkcX8AWwFMyFZ7Pdq6FMAZkcdfA/CvkcfHAVgB6WeZBODjkHy2Xu09AcCoyOMZAHbZ\ntgnVZ2t7/TkAz0AG/WWtrSl8tjk9z5Jsa67PsVEATow8rgDwEYDpkL7GH0SW3wrg3sjjnJ9n5i/j\nkTszvwug0bG4B4C50g0GsDvy+DwAq5h5TWTbRmZmkmycAcy8LLLeE3AfNJXt9jKAciIqBFAGoAPA\nwWy116Ot0yLLAeBNAF+IPL4IwNPM3MXM2wBsAjAnBJ+ta3uZeRXL+Akw81oApUTUL6SfLYjoYgBb\nAKy1LQvlZ4scn2dJtjXX51iygzhzfp4ZclXy9xYA/0lEOwD8DFbpgqMBgIhei9yS/0tk+VjIAChD\ntgdDebX3eQBtkEygbQD+k5mbkNv2riWiiyKPr4D8EIHYAWW7I8ty/dl6tfcIRHQZgOUso5xD99lG\nLIQfALgbgD3rK6yfbRjPM6+2huYcI3+DOENznuVK3L8FGQw1ASKcj0aWFwGYC7kVOxPApXavNYd4\ntfcUSO2dUQAmA/h+5AeQS66D1P9ZBqAcQGeO25OIuO0lohkA/h3ADTlomxOvtt4F4L+YuS1nLXPH\nq71hPM+82hqKc4zycBBnruq5f5WZbwYAZn6eiH4dWb4LwGJmbgQAIvozgJMA/BaScmkYB8sayWV7\nrwTwGjP3ANhHRO8BmA3g3Vy1l5k3Qgq1gYimAfhs5KXdHm3yWp4V4rQXRDQOwB8AXBO5xQVy2N44\nbT0FwBeI6GcQ/7qbiNohbQ/jZxu68yxOW3N+jpEM4nwewG+Y2YznqSOiSrYGce6NLA/NeZatyJ0Q\nfbu6m4g+DQBEdDbElwKk4/J4IiqNfKCfBrA2cttzgIjmEBEB+ArcB01lu707AJwVWV4O4FQA67Pc\n3qi2EtGIyP8CAHcA+FXkpZcBfImIionoKABTASzN9Wfr1V4iGgwZFX0rM79v1g/jZ8vMn2Lmycw8\nGcD9AH7KzA+G9bNFOM6zRG39n8hLYTjH4g3iBKIHcYblPMtKtsxTAGogHSE7AFwL4HQAH0B6lf8O\nYJZt/asArAHwIYB/ty3/JGQg1SYAC8PQXsjt47OR9q5BdJZExtvr0dabID36GyAiY1//Nkjv/XpE\nsn9C8Nm6thfAjwA0A1ge+dyXAxge1s/Wtt1d2f4dpPhbyNl5luTvINfn2FwA3QBW2n6H5wMYCun4\n/QjA6wAG27bJ6Xlm/nQQk6IoSi9EJ8hWFEXphai4K4qi9EJU3BVFUXohKu6Koii9EBV3RVGUXoiK\nu6IoSi9ExV1RFKUXouKuKAERGV2pKKFAf4xKn4SI7iaim23Pf0JENxHR90kmZFlJRHfZXn+BZNKQ\n1UR0vW15MxH9JxGtgAyNV5RQoOKu9FUehdT3QKTWx5cgZWWnMfMcALMAzCaiMyLrX8vMJwM4GcDN\nRDQksrwcwN+ZeRYz/y2r70BR4pCrqpCKklOYeTsR1RPRCZBysssBzAFwLhEthxS1KgcwDVKB8J+J\nyEyuMC6yfCmkHO0fst1+RUmEirvSl/k1pGjVKEgkfw6kiNbD9pUiFUHPAnAKM3cQ0dsASiMvt7MW\naFJCiNoySl/mRUiFv9mQMrh/AXBdpLQsiGhMpBTtIACNEWGfjmhvnaAoIUQjd6XPwsyHI1F4YyT6\nfiMi3n8XGx7NAL4M4DUA3ySitZASr3+37ybLzVYUX2jJX6XPEkld/AeAy5h5c67boyhBoraM0ich\nomMhkya8ocKu9EY0clcURemFaOSuKIrSC1FxVxRF6YWouCuKovRCVNwVRVF6ISruiqIovRAVd0VR\nlF7I/wcwJq7OwVYunAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1125cf6d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualise the results\n",
    "\n",
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "averageYearlyTempsPD = pd.DataFrame.from_records(averageYearlyTemps, columns=['year', 'avg_max_temp'], index='year')\n",
    "plt.close()\n",
    "averageYearlyTempsPD.plot()\n",
    "plt.show()\n",
    "display()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*How you interpret the chart above is up to you ;)*\n",
    "\n",
    "You can now play around modifying pieces of the code.\n",
    "\n",
    "When you are done and you are running off the local machine remember to *close the notebook* with `File/Close and Halt`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "PySpark",
   "language": "python",
   "name": "pyspark"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}