{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EnergyPlus Output Data Analysis Example\n",
    "    \n",
    "Created by Clayton Miller (miller.clayton@arch.ethz.ch)\n",
    "\n",
    "The goal of this notebook is to give a user a glimpse at the loading and manipulation of a .csv output of EnergyPlus\n",
    "\n",
    "Execute the cells in this notebook one at a time and try to understand what each code snippet is doing. There will be text notation before many of the cells attempting to explain what is going on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we load some libraries that we will use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import datetime\n",
    "from datetime import timedelta\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following is a Python `fucntion` that I created ot read a .csv file, do a conversion and change the timestamp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loadsimdata(file,pointname,ConvFactor):\n",
    "    df = pd.read_csv(file)\n",
    "    df['desiredpoint'] = df[pointname]*ConvFactor\n",
    "    df.index = eplustimestamp(df)\n",
    "    pointdf = df['desiredpoint']\n",
    "    return pointdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open an EnergyPlus Output CSV file\n",
    "\n",
    "First let's open the .csv output file from an EnergyPlus run and visualize it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData = pd.read_csv('EnergyplusSimulationData.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can select a certain column and see what's inside -- there is quite a few columns in this output file. We can see using the `.info` function from pandas that there are 256 columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData['AIRNODE_ZONENODE_U1_S:System Node Setpoint Temp[C](Hourly) ']#.plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData['Date/Time'].tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We need to convert 24:00:00 to 00:00:00 for it to play nice with Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Function to convert timestamps\n",
    "def eplustimestamp(simdata):\n",
    "    timestampdict={}\n",
    "    for i,row in simdata.T.iteritems():\n",
    "        timestamp = str(2013) + row['Date/Time']\n",
    "        try:\n",
    "            timestampdict[i] = datetime.datetime.strptime(timestamp,'%Y %m/%d  %H:%M:%S')\n",
    "        except ValueError:\n",
    "            tempts = timestamp.replace(' 24', ' 23')\n",
    "            timestampdict[i] = datetime.datetime.strptime(tempts,'%Y %m/%d  %H:%M:%S')\n",
    "            timestampdict[i] += timedelta(hours=1)\n",
    "    timestampseries = pd.Series(timestampdict)\n",
    "    return timestampseries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData.index = eplustimestamp(SimulationData)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData['AIRNODE_ZONENODE_U1_S:System Node Setpoint Temp[C](Hourly) ']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ColumnsList = pd.Series(SimulationData.columns)\n",
    "ColumnsList.head(100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Built-in String Query functions to find desired columns\n",
    "\n",
    "We can use the string query functions in Pandas to find columns to select for visualization, etc without manually inputting them.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTempPointList = list(ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))])\n",
    "ZoneTempPointList"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "BasementZoneTemp = list(ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))&(ColumnsList.str.contains(\"U1\"))])\n",
    "GroundFloorZoneTemp = list(ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))&(ColumnsList.str.contains(\"00\"))])\n",
    "Floor1ZoneTemp = list(ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))&(ColumnsList.str.contains(\"01\"))])\n",
    "Floor2ZoneTemp = list(ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))&(ColumnsList.str.contains(\"02\"))])\n",
    "Floor3ZoneTemp = list(ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))&(ColumnsList.str.contains(\"03\"))])\n",
    "Floor4ZoneTemp = list(ColumnsList[(ColumnsList.str.endswith(\"Zone Mean Air Temperature [C](Hourly)\"))&(ColumnsList.str.contains(\"04\"))])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTemp = SimulationData[ZoneTempPointList]#.drop(['EMS:All Zones Total Heating Energy {J}(Hourly)'],axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTemp.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization\n",
    "\n",
    "Now that we have the data in a Pandas Dataframe, let's do some analysis -- visualization, statistics, etc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTemp.plot(figsize=(25,15))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTemp[BasementZoneTemp].plot(figsize=(25,10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTemp[GroundFloorZoneTemp].truncate(before='2013-03-10',after='2013-03-14').plot(figsize=(25,10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTemp[Floor1ZoneTemp].plot(figsize=(25,10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZoneTemp[Floor2ZoneTemp].plot(figsize=(25,10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Zooming in using the `.truncate()` pandas method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SimulationData['Environment:Outdoor Dry Bulb [C](Hourly)'].truncate(before='2013-03-10',after='2013-03-14').plot(figsize=(25,10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's take a deeper look at the Floor 2 Zone Temperations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Floor2Temps = ZoneTemp[Floor2ZoneTemp]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Floor2Temps.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Floor2Temps.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Heatmaps\n",
    "\n",
    "Heatmaps are a great way to visualize performance data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from matplotlib.backends.backend_pdf import PdfPages\n",
    "import os\n",
    "import matplotlib.dates as mdates\n",
    "import datetime as dt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Floor1Energy = list(ColumnsList[(ColumnsList.str.endswith(\"Total Heating Energy {J}(Hourly)\"))&(ColumnsList.str.contains(\"01\"))])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Floor1Energy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_hourly = SimulationData.resample('H').mean()\n",
    "\n",
    "#Add the Date and time for pivoting\n",
    "df_hourly['Date'] = df_hourly.index.map(lambda t: t.date())\n",
    "df_hourly['Time'] = df_hourly.index.map(lambda t: t.time())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "numberofplots = len(Floor1Energy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pointcounter = 1\n",
    "fig = plt.figure(figsize=(40, 4 * numberofplots))\n",
    "\n",
    "\n",
    "for energypoint in Floor1Energy:\n",
    "    \n",
    "    print \"Loading data from \"+energypoint\n",
    "    \n",
    "    #Pivot\n",
    "    df_pivot = pd.pivot_table(df_hourly, values=energypoint, index='Time', columns='Date')\n",
    "\n",
    "    # Get the data\n",
    "    x = mdates.drange(df_pivot.columns[0], df_pivot.columns[-1] + datetime.timedelta(days=1), dt.timedelta(days=1))\n",
    "    y = np.linspace(1, 24, 24)\n",
    "\n",
    "    # Plot\n",
    "    ax = fig.add_subplot(numberofplots, 1, pointcounter)\n",
    "\n",
    "    data = np.ma.masked_invalid(np.array(df_pivot))\n",
    "\n",
    "    qmesh = ax.pcolormesh(x, y, data)\n",
    "    cbar = fig.colorbar(qmesh, ax=ax)\n",
    "    cbar.ax.tick_params(labelsize= 24)\n",
    "    ax.axis('tight')\n",
    "\n",
    "    try:\n",
    "        plt.title(energypoint, fontsize=26)\n",
    "\n",
    "    except IndexError:\n",
    "        continue\n",
    "\n",
    "    # Set up as dates\n",
    "    ax.xaxis_date()\n",
    "    fig.autofmt_xdate()\n",
    "    fig.subplots_adjust(hspace=.5)\n",
    "    \n",
    "    pointcounter += 1\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}