{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Add time series to `System`"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Originally Contributed by**: Clayton Barrows"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Introduction"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "An example of how to parse add time series data to a `System` using [PowerSystems.jl](github.com/NREL-SIIP/PowerSystems.jl)\n",
    "\n",
    "For example, a `System` created by [parsing a MATPOWER file](https://nbviewer.jupyter.org/github/NREL-SIIP/SIIPExamples.jl/blob/master/notebook/2_PowerSystems_examples/parse_matpower.ipynb)\n",
    "doesn't contain any time series data. So a user may want to add time series to the `System`\n",
    "### Dependencies\n",
    "Let's use the 5-bus dataset we parsed in the MATPOWER example"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ Info: extending matpower format with data: areas 1x3\n",
      "[ Info: extending matpower format with data: gen_name 7x4\n",
      "[ Info: extending matpower format by appending matrix \"gen_name\" in to \"gen\"\n",
      "[ Info: reversing the orientation of branch 6 (4, 3) to be consistent with other parallel branches\n",
      "[ Info: the voltage setpoint on generator 4 does not match the value at bus 4\n",
      "[ Info: the voltage setpoint on generator 1 does not match the value at bus 1\n",
      "[ Info: the voltage setpoint on generator 5 does not match the value at bus 10\n",
      "[ Info: the voltage setpoint on generator 2 does not match the value at bus 1\n",
      "[ Info: the voltage setpoint on generator 3 does not match the value at bus 3\n",
      "[ Info: removing 1 cost terms from generator 4: [4000.0, 0.0]\n",
      "[ Info: removing 1 cost terms from generator 1: [1400.0, 0.0]\n",
      "[ Info: removing 1 cost terms from generator 5: [1000.0, 0.0]\n",
      "[ Info: removing 1 cost terms from generator 2: [1500.0, 0.0]\n",
      "[ Info: removing 3 cost terms from generator 6: Float64[]\n",
      "[ Info: removing 3 cost terms from generator 7: Float64[]\n",
      "[ Info: removing 1 cost terms from generator 3: [3000.0, 0.0]\n",
      "┌ Info: Constructing System from Power Models\n",
      "│   data[\"name\"] = \"nesta_case5_pjm\"\n",
      "└   data[\"source_type\"] = \"matpower\"\n",
      "[ Info: Reading bus data\n",
      "[ Info: Reading generator data\n",
      "[ Info: Reading branch data\n",
      "┌ Warning: Rate 1200.0 MW for bus2-bus3-i_4 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus1-bus2-i_1 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus1-bus4-i_2 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus4-bus5-i_7 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 1000.0 MW for bus1-bus5-i_3 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "[ Info: Reading branch data\n",
      "[ Info: Reading DC Line data\n",
      "[ Info: Reading storage data\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "System\n======\nSystem Units Base: SYSTEM_BASE\nBase Power: 100.0\nBase Frequency: 60.0\n\nComponents\n==========\nNum components: 30\n\n\u001b[1m9×3 DataFrame\u001b[0m\n\u001b[1m Row \u001b[0m│\u001b[1m ConcreteType             \u001b[0m\u001b[1m SuperTypes                        \u001b[0m\u001b[1m Count \u001b[0m\n\u001b[1m     \u001b[0m│\u001b[90m String                   \u001b[0m\u001b[90m String                            \u001b[0m\u001b[90m Int64 \u001b[0m\n─────┼────────────────────────────────────────────────────────────────────\n   1 │ Arc                       Topology <: Component <: Infrast…      6\n   2 │ Area                      AggregationTopology <: Topology …      1\n   3 │ Bus                       Topology <: Component <: Infrast…      5\n   4 │ Line                      ACBranch <: Branch <: Device <: …      5\n   5 │ LoadZone                  AggregationTopology <: Topology …      1\n   6 │ PhaseShiftingTransformer  ACBranch <: Branch <: Device <: …      2\n   7 │ PowerLoad                 StaticLoad <: ElectricLoad <: St…      3\n   8 │ RenewableDispatch         RenewableGen <: Generator <: Sta…      2\n   9 │ ThermalStandard           ThermalGen <: Generator <: Stati…      5\n\nTimeSeriesContainer\n===================\nComponents with time series data: 0\nTotal StaticTimeSeries: 0\nTotal Forecasts: 0\n",
      "text/html": [
       "<h1>System</h1>\n",
       "<p><b>Base Power</b>: 100.0</p>\n",
       "<h2>Components</h2>\n",
       "<p><b>Num components</b>: 30</p>\n",
       "<table class=\"data-frame\"><thead><tr><th></th><th>ConcreteType</th><th>SuperTypes</th><th>Count</th></tr><tr><th></th><th>String</th><th>String</th><th>Int64</th></tr></thead><tbody><p>9 rows × 3 columns</p><tr><th>1</th><td>Arc</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>6</td></tr><tr><th>2</th><td>Area</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>3</th><td>Bus</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>4</th><td>Line</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>5</th><td>LoadZone</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>6</th><td>PhaseShiftingTransformer</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>7</th><td>PowerLoad</td><td>StaticLoad &lt;: ElectricLoad &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>3</td></tr><tr><th>8</th><td>RenewableDispatch</td><td>RenewableGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>9</th><td>ThermalStandard</td><td>ThermalGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr></tbody></table>\n",
       "\n",
       "<h2>TimeSeriesContainer</h2>\n",
       "<p><b>Components with time series data</b>: 0</p>\n",
       "<p><b>Total StaticTimeSeries</b>: 0</p>\n",
       "<p><b>Total Forecasts</b>: 0</p>\n",
       "<p><b>Resolution</b>: 0 seconds</p>\n"
      ]
     },
     "metadata": {},
     "execution_count": 1
    }
   ],
   "cell_type": "code",
   "source": [
    "using SIIPExamples\n",
    "using PowerSystems\n",
    "using JSON3\n",
    "\n",
    "pkgpath = dirname(dirname(pathof(SIIPExamples)))\n",
    "include(joinpath(pkgpath, \"test\", \"2_PowerSystems_examples\", \"02_parse_matpower.jl\"))"
   ],
   "metadata": {},
   "execution_count": 1
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Define pointers to time series files\n",
    "For example, if we want to add a bunch of time series files, say one for each load and\n",
    "one for each renewable generator, we need to define pointers to each .csv file containing\n",
    "the time series in the following format (PowerSystems.jl also supports a CSV format for this file)"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "                       \"component_name\": \"SolarBusC\",\n",
      "                 \"normalization_factor\": 1,\n",
      "                                 \"name\": \"max_active_power\",\n",
      "     \"scaling_factor_multiplier_module\": \"PowerSystems\",\n",
      "                            \"data_file\": \"./gen/Renewable/PV/da_solar5.csv\",\n",
      "                           \"resolution\": 3600,\n",
      "                               \"module\": \"PowerSystems\",\n",
      "                             \"category\": \"Generator\",\n",
      "            \"scaling_factor_multiplier\": \"get_max_active_power\",\n",
      "                           \"simulation\": \"DAY_AHEAD\",\n",
      "                                 \"type\": \"SingleTimeSeries\"\n",
      "  },\n",
      "  {\n",
      "                       \"component_name\": \"WindBusA\",\n",
      "                 \"normalization_factor\": 1,\n",
      "                                 \"name\": \"max_active_power\",\n",
      "     \"scaling_factor_multiplier_module\": \"PowerSystems\",\n",
      "                            \"data_file\": \"./gen/Renewable/WIND/da_wind5.csv\",\n",
      "                           \"resolution\": 3600,\n",
      "                               \"module\": \"PowerSystems\",\n",
      "                             \"category\": \"Generator\",\n",
      "            \"scaling_factor_multiplier\": \"get_max_active_power\",\n",
      "                           \"simulation\": \"DAY_AHEAD\",\n",
      "                                 \"type\": \"SingleTimeSeries\"\n",
      "  },\n",
      "  {\n",
      "                       \"component_name\": \"bus2\",\n",
      "                 \"normalization_factor\": 1,\n",
      "                                 \"name\": \"max_active_power\",\n",
      "     \"scaling_factor_multiplier_module\": \"PowerSystems\",\n",
      "                            \"data_file\": \"./load/da_load5.csv\",\n",
      "                           \"resolution\": 3600,\n",
      "                               \"module\": \"PowerSystems\",\n",
      "                             \"category\": \"ElectricLoad\",\n",
      "            \"scaling_factor_multiplier\": \"get_max_active_power\",\n",
      "                           \"simulation\": \"DAY_AHEAD\",\n",
      "                                 \"type\": \"SingleTimeSeries\"\n",
      "  },\n",
      "  {\n",
      "                       \"component_name\": \"bus3\",\n",
      "                 \"normalization_factor\": 1,\n",
      "                                 \"name\": \"max_active_power\",\n",
      "     \"scaling_factor_multiplier_module\": \"PowerSystems\",\n",
      "                            \"data_file\": \"./load/da_load5.csv\",\n",
      "                           \"resolution\": 3600,\n",
      "                               \"module\": \"PowerSystems\",\n",
      "                             \"category\": \"ElectricLoad\",\n",
      "            \"scaling_factor_multiplier\": \"get_max_active_power\",\n",
      "                           \"simulation\": \"DAY_AHEAD\",\n",
      "                                 \"type\": \"SingleTimeSeries\"\n",
      "  },\n",
      "  {\n",
      "                       \"component_name\": \"bus4\",\n",
      "                 \"normalization_factor\": 1,\n",
      "                                 \"name\": \"max_active_power\",\n",
      "     \"scaling_factor_multiplier_module\": \"PowerSystems\",\n",
      "                            \"data_file\": \"./load/da_load5.csv\",\n",
      "                           \"resolution\": 3600,\n",
      "                               \"module\": \"PowerSystems\",\n",
      "                             \"category\": \"ElectricLoad\",\n",
      "            \"scaling_factor_multiplier\": \"get_max_active_power\",\n",
      "                           \"simulation\": \"DAY_AHEAD\",\n",
      "                                 \"type\": \"SingleTimeSeries\"\n",
      "  }\n",
      "]"
     ]
    }
   ],
   "cell_type": "code",
   "source": [
    "FORECASTS_DIR = joinpath(base_dir, \"forecasts\", \"5bus_ts\")\n",
    "fname = joinpath(FORECASTS_DIR, \"timeseries_pointers_da.json\")\n",
    "open(fname, \"r\") do f\n",
    "    JSON3.@pretty JSON3.read(f)\n",
    "end"
   ],
   "metadata": {},
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Read and assign time series to `System` using these parameters."
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "System\n======\nSystem Units Base: SYSTEM_BASE\nBase Power: 100.0\nBase Frequency: 60.0\n\nComponents\n==========\nNum components: 30\n\n\u001b[1m9×3 DataFrame\u001b[0m\n\u001b[1m Row \u001b[0m│\u001b[1m ConcreteType             \u001b[0m\u001b[1m SuperTypes                        \u001b[0m\u001b[1m Count \u001b[0m\n\u001b[1m     \u001b[0m│\u001b[90m String                   \u001b[0m\u001b[90m String                            \u001b[0m\u001b[90m Int64 \u001b[0m\n─────┼────────────────────────────────────────────────────────────────────\n   1 │ Arc                       Topology <: Component <: Infrast…      6\n   2 │ Area                      AggregationTopology <: Topology …      1\n   3 │ Bus                       Topology <: Component <: Infrast…      5\n   4 │ Line                      ACBranch <: Branch <: Device <: …      5\n   5 │ LoadZone                  AggregationTopology <: Topology …      1\n   6 │ PhaseShiftingTransformer  ACBranch <: Branch <: Device <: …      2\n   7 │ PowerLoad                 StaticLoad <: ElectricLoad <: St…      3\n   8 │ RenewableDispatch         RenewableGen <: Generator <: Sta…      2\n   9 │ ThermalStandard           ThermalGen <: Generator <: Stati…      5\n\nTimeSeriesContainer\n===================\nComponents with time series data: 5\nTotal StaticTimeSeries: 5\nTotal Forecasts: 0\nResolution: 60 minutes\n",
      "text/html": [
       "<h1>System</h1>\n",
       "<p><b>Base Power</b>: 100.0</p>\n",
       "<h2>Components</h2>\n",
       "<p><b>Num components</b>: 30</p>\n",
       "<table class=\"data-frame\"><thead><tr><th></th><th>ConcreteType</th><th>SuperTypes</th><th>Count</th></tr><tr><th></th><th>String</th><th>String</th><th>Int64</th></tr></thead><tbody><p>9 rows × 3 columns</p><tr><th>1</th><td>Arc</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>6</td></tr><tr><th>2</th><td>Area</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>3</th><td>Bus</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>4</th><td>Line</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>5</th><td>LoadZone</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>6</th><td>PhaseShiftingTransformer</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>7</th><td>PowerLoad</td><td>StaticLoad &lt;: ElectricLoad &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>3</td></tr><tr><th>8</th><td>RenewableDispatch</td><td>RenewableGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>9</th><td>ThermalStandard</td><td>ThermalGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr></tbody></table>\n",
       "\n",
       "<h2>TimeSeriesContainer</h2>\n",
       "<p><b>Components with time series data</b>: 5</p>\n",
       "<p><b>Total StaticTimeSeries</b>: 5</p>\n",
       "<p><b>Total Forecasts</b>: 0</p>\n",
       "<p><b>Resolution</b>: 60 minutes</p>\n"
      ]
     },
     "metadata": {},
     "execution_count": 3
    }
   ],
   "cell_type": "code",
   "source": [
    "add_time_series!(sys, fname)\n",
    "sys"
   ],
   "metadata": {},
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "source": [
    "---\n",
    "\n",
    "*This notebook was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl).*"
   ],
   "metadata": {}
  }
 ],
 "nbformat_minor": 3,
 "metadata": {
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.5.4"
  },
  "kernelspec": {
   "name": "julia-1.5",
   "display_name": "Julia 1.5.4",
   "language": "julia"
  }
 },
 "nbformat": 4
}