{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Sequential Simulations with [PowerSimulations.jl](https://github.com/NREL-SIIP/PowerSimulations.jl)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Originally Contributed by**: Clayton Barrows"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Introduction"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "PowerSimulations.jl supports simulations that consist of sequential optimization problems\n",
    "where results from previous problems inform subsequent problems in a variety of ways. This\n",
    "example demonstrates some of these capabilities to represent electricity market clearing.\n",
    "This example is intended to be an extension of the\n",
    "[OperationsProblem example.](https://nbviewer.jupyter.org/github/NREL-SIIP/SIIPExamples.jl/blob/master/notebook/3_PowerSimulations_examples/01_operations_problems.ipynb)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Dependencies"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "using SIIPExamples"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Modeling Packages"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "using PowerSystems\n",
    "using PowerSimulations\n",
    "const PSI = PowerSimulations\n",
    "using PowerSystemCaseBuilder\n",
    "using Dates"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Optimization packages"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "using HiGHS #solver"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Optimizer\n",
    "It's most convenient to define an optimizer instance upfront and pass it into the\n",
    "`DecisionModel` constructor. For this example, we can use the free HiGHS solver with a\n",
    "relatively relaxed MIP gap (`ratioGap`) setting to improve speed."
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "solver = optimizer_with_attributes(HiGHS.Optimizer, \"mip_rel_gap\" => 0.5)"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Hourly day-ahead system\n",
    "First, we'll create a `System` with hourly data to represent day-ahead forecasted wind,\n",
    "solar, and load profiles:"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "sys_DA = build_system(PSITestSystems, \"modified_RTS_GMLC_DA_sys\")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 5-Minute system\n",
    "The RTS data also includes 5-minute resolution time series data. So, we can create another\n",
    "`System` to represent 15 minute ahead forecasted data for a \"real-time\" market:"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "sys_RT = build_system(PSITestSystems, \"modified_RTS_GMLC_RT_sys\")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "## `ProblemTemplate`s define stages\n",
    "Sequential simulations in PowerSimulations are created by defining `OperationsProblems`\n",
    "that represent stages, and how information flows between executions of a stage and\n",
    "between different stages.\n",
    "\n",
    "Let's start by defining a two stage simulation that might look like a typical day-Ahead\n",
    "and real-time electricity market clearing process."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Day-ahead unit commitment stage\n",
    "First, we can define a unit commitment template for the day ahead problem. We can use the\n",
    "included UC template, but in this example, we'll replace the `ThermalBasicUnitCommitment`\n",
    "with the slightly more complex `ThermalStandardUnitCommitment` for the thermal generators."
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "template_uc = template_unit_commitment()\n",
    "set_device_model!(template_uc, ThermalStandard, ThermalStandardUnitCommitment)"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Define the reference model for the real-time economic dispatch\n",
    "In addition to the manual specification process demonstrated in the OperationsProblem\n",
    "example, PSI also provides pre-specified templates for some standard problems:"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "template_ed = template_economic_dispatch(\n",
    "    network = NetworkModel(StandardPTDFModel, PTDF = PTDF(sys_DA), use_slacks = true),#NetworkModel(CopperPlatePowerModel, use_slacks = true),\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Define the `SimulationModels`\n",
    "`DecisionModel`s define the problems that are executed in the simulation.\n",
    "The actual problem will change as the stage gets updated to represent\n",
    "different time periods, but the formulations applied to the components is constant within\n",
    "a stage. In this case, we want to define two stages with the `ProblemTemplate`s\n",
    "and the `System`s that we've already created."
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "models = SimulationModels(\n",
    "    decision_models = [\n",
    "        DecisionModel(template_uc, sys_DA, optimizer = solver, name = \"UC\"),\n",
    "        DecisionModel(template_ed, sys_RT, optimizer = solver, name = \"ED\"),\n",
    "    ],\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### `SimulationSequence`\n",
    "Similar to an `ProblemTemplate`, the `SimulationSequence` provides a template of\n",
    "how to execute a sequential set of operations problems."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "print_struct(SimulationSequence)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let's review some of the `SimulationSequence` arguments."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Chronologies\n",
    "In PowerSimulations, chronologies define where information is flowing. There are two types\n",
    "of chronologies.\n",
    " - inter-stage chronologies: Define how information flows between stages. e.g. day-ahead\n",
    "solutions are used to inform economic dispatch problems\n",
    " - intra-stage chronologies: Define how information flows between multiple executions of a\n",
    "single stage. e.g. the dispatch setpoints of the first period of an economic dispatch problem\n",
    "are constrained by the ramping limits from setpoints in the final period of the previous problem."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### `FeedForward`\n",
    "The definition of exactly what information is passed using the defined chronologies is\n",
    "accomplished with `FeedForward`. Specifically, `FeedForward` is used\n",
    "to define what to do with information being passed with an inter-stage chronology. Let's\n",
    "define a `FeedForward` that affects the semi-continuous range constraints of thermal generators\n",
    "in the economic dispatch problems based on the value of the unit-commitment variables."
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "feedforward = Dict(\n",
    "    \"ED\" => [\n",
    "        SemiContinuousFeedforward(\n",
    "            component_type = ThermalStandard,\n",
    "            source = OnVariable,\n",
    "            affected_values = [ActivePowerVariable],\n",
    "        ),\n",
    "    ],\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Sequencing\n",
    "The stage problem length, look-ahead, and other details surrounding the temporal Sequencing\n",
    "of stages are controlled using the structure of the time series data in the `System`s.\n",
    "So, to define a typical day-ahead - real-time sequence:\n",
    " - Day ahead problems should represent 48 hours, advancing 24 hours after each execution (24-hour look-ahead)\n",
    " - Real time problems should represent 1 hour (12 5-minute periods), advancing 15 min after each execution (15 min look-ahead)\n",
    "We can adjust the time series data to reflect this structure in each `System`:\n",
    "- `transform_single_time_series!(sys_DA, 48, Hour(1))`\n",
    "- `transform_single_time_series!(sys_RT, 12, Minute(15))`\n",
    "\n",
    "Now we can put it all together to define a `SimulationSequence`"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "DA_RT_sequence = SimulationSequence(\n",
    "    models = models,\n",
    "    ini_cond_chronology = InterProblemChronology(),\n",
    "    feedforwards = feedforward,\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "## `Simulation`\n",
    "Now, we can build and execute a simulation using the `SimulationSequence` and `Stage`s\n",
    "that we've defined."
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "sim = Simulation(\n",
    "    name = \"rts-test\",\n",
    "    steps = 2,\n",
    "    models = models,\n",
    "    sequence = DA_RT_sequence,\n",
    "    simulation_folder = mktempdir(\".\", cleanup = true),\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Build simulation"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "build!(sim)"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Execute simulation\n",
    "the following command returns the status of the simulation (0: is proper execution) and\n",
    "stores the results in a set of HDF5 files on disk."
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "execute!(sim, enable_progress_bar = false)"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Results\n",
    "To access the results, we need to load the simulation result metadata and then make\n",
    "requests to the specific data of interest. This allows you to efficiently access the\n",
    "results of interest without overloading resources."
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "results = SimulationResults(sim);\n",
    "uc_results = get_problem_results(results, \"UC\"); # UC stage result metadata\n",
    "ed_results = get_problem_results(results, \"ED\"); # ED stage result metadata"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "We can see that the results `uc_results` contain a number of variables:"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "list_variable_names(uc_results)"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "and a number of parameters (this pattern also works for aux_variables, expressions, and duals)"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "list_parameter_names(uc_results)"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we can read the specific results of interest for a specific problem, time window (optional),\n",
    "and set of variables, duals, or parameters (optional)"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "read_variables(\n",
    "    uc_results,\n",
    "    [\n",
    "        \"ActivePowerVariable__RenewableDispatch\",\n",
    "        \"ActivePowerVariable__HydroDispatch\",\n",
    "        \"StopVariable__ThermalStandard\",\n",
    "    ],\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "Or if we want the result of just one variable, parameter, or dual (must be defined in the\n",
    "problem definition), we can use:"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "read_parameter(\n",
    "    ed_results,\n",
    "    \"ActivePowerTimeSeriesParameter__RenewableFix\",\n",
    "    initial_time = DateTime(\"2020-01-01T06:00:00\"),\n",
    "    count = 5,\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "* note that this returns the results of each execution step in a separate dataframe *\n",
    "If you want the realized results (without lookahead periods), you can call `read_realized_*`:"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "read_realized_variables(\n",
    "    uc_results,\n",
    "    [\"ActivePowerVariable__ThermalStandard\", \"ActivePowerVariable__RenewableDispatch\"],\n",
    ")"
   ],
   "metadata": {},
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Plotting\n",
    "Take a look at the [plotting examples.](https://nbviewer.jupyter.org/github/NREL-SIIP/SIIPExamples.jl/blob/master/notebook/3_PowerSimulations_examples/04_bar_stack_plots.ipynb)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "---\n",
    "\n",
    "*This notebook was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl).*"
   ],
   "metadata": {}
  }
 ],
 "nbformat_minor": 3,
 "metadata": {
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.7.2"
  },
  "kernelspec": {
   "name": "julia-1.7",
   "display_name": "Julia 1.7.2",
   "language": "julia"
  }
 },
 "nbformat": 4
}