{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Serializing PowerSystem Data"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Originally Contributed by**: Clayton Barrows"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Introduction"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "PowerSystems.jl supports serializing/deserializing data with JSON. This notebook\n",
    "provides an example of how to write and read a `System` to/from disk."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Dependencies\n",
    "Let's use a dataset from the [tabular data parsing example](https://nbviewer.jupyter.org/github/NREL-SIIP/SIIPExamples.jl/blob/master/notebook/2_PowerSystems_examples/parse_matpower.ipynb)"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ Info: extending matpower format with data: areas 1x3\n",
      "[ Info: extending matpower format with data: gen_name 7x4\n",
      "[ Info: extending matpower format by appending matrix \"gen_name\" in to \"gen\"\n",
      "[ Info: reversing the orientation of branch 6 (4, 3) to be consistent with other parallel branches\n",
      "[ Info: the voltage setpoint on generator 4 does not match the value at bus 4\n",
      "[ Info: the voltage setpoint on generator 1 does not match the value at bus 1\n",
      "[ Info: the voltage setpoint on generator 5 does not match the value at bus 10\n",
      "[ Info: the voltage setpoint on generator 2 does not match the value at bus 1\n",
      "[ Info: the voltage setpoint on generator 3 does not match the value at bus 3\n",
      "[ Info: removing 1 cost terms from generator 4: [4000.0, 0.0]\n",
      "[ Info: removing 1 cost terms from generator 1: [1400.0, 0.0]\n",
      "[ Info: removing 1 cost terms from generator 5: [1000.0, 0.0]\n",
      "[ Info: removing 1 cost terms from generator 2: [1500.0, 0.0]\n",
      "[ Info: removing 3 cost terms from generator 6: Float64[]\n",
      "[ Info: removing 3 cost terms from generator 7: Float64[]\n",
      "[ Info: removing 1 cost terms from generator 3: [3000.0, 0.0]\n",
      "┌ Info: Constructing System from Power Models\n",
      "│   data[\"name\"] = \"nesta_case5_pjm\"\n",
      "└   data[\"source_type\"] = \"matpower\"\n",
      "[ Info: Reading bus data\n",
      "[ Info: Reading generator data\n",
      "[ Info: Reading branch data\n",
      "┌ Warning: Rate 1200.0 MW for bus2-bus3-i_4 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus1-bus2-i_1 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus1-bus4-i_2 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus4-bus5-i_7 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 1000.0 MW for bus1-bus5-i_3 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "[ Info: Reading branch data\n",
      "[ Info: Reading DC Line data\n",
      "[ Info: Reading storage data\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "System\n======\nSystem Units Base: SYSTEM_BASE\nBase Power: 100.0\nBase Frequency: 60.0\n\nComponents\n==========\nNum components: 30\n\n\u001b[1m9×3 DataFrame\u001b[0m\n\u001b[1m Row \u001b[0m│\u001b[1m ConcreteType             \u001b[0m\u001b[1m SuperTypes                        \u001b[0m\u001b[1m Count \u001b[0m\n\u001b[1m     \u001b[0m│\u001b[90m String                   \u001b[0m\u001b[90m String                            \u001b[0m\u001b[90m Int64 \u001b[0m\n─────┼────────────────────────────────────────────────────────────────────\n   1 │ Arc                       Topology <: Component <: Infrast…      6\n   2 │ Area                      AggregationTopology <: Topology …      1\n   3 │ Bus                       Topology <: Component <: Infrast…      5\n   4 │ Line                      ACBranch <: Branch <: Device <: …      5\n   5 │ LoadZone                  AggregationTopology <: Topology …      1\n   6 │ PhaseShiftingTransformer  ACBranch <: Branch <: Device <: …      2\n   7 │ PowerLoad                 StaticLoad <: ElectricLoad <: St…      3\n   8 │ RenewableDispatch         RenewableGen <: Generator <: Sta…      2\n   9 │ ThermalStandard           ThermalGen <: Generator <: Stati…      5\n\nTimeSeriesContainer\n===================\nComponents with time series data: 0\nTotal StaticTimeSeries: 0\nTotal Forecasts: 0\n",
      "text/html": [
       "<h1>System</h1>\n",
       "<p><b>Base Power</b>: 100.0</p>\n",
       "<h2>Components</h2>\n",
       "<p><b>Num components</b>: 30</p>\n",
       "<table class=\"data-frame\"><thead><tr><th></th><th>ConcreteType</th><th>SuperTypes</th><th>Count</th></tr><tr><th></th><th>String</th><th>String</th><th>Int64</th></tr></thead><tbody><p>9 rows × 3 columns</p><tr><th>1</th><td>Arc</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>6</td></tr><tr><th>2</th><td>Area</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>3</th><td>Bus</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>4</th><td>Line</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>5</th><td>LoadZone</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>6</th><td>PhaseShiftingTransformer</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>7</th><td>PowerLoad</td><td>StaticLoad &lt;: ElectricLoad &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>3</td></tr><tr><th>8</th><td>RenewableDispatch</td><td>RenewableGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>9</th><td>ThermalStandard</td><td>ThermalGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr></tbody></table>\n",
       "\n",
       "<h2>TimeSeriesContainer</h2>\n",
       "<p><b>Components with time series data</b>: 0</p>\n",
       "<p><b>Total StaticTimeSeries</b>: 0</p>\n",
       "<p><b>Total Forecasts</b>: 0</p>\n",
       "<p><b>Resolution</b>: 0 seconds</p>\n"
      ]
     },
     "metadata": {},
     "execution_count": 1
    }
   ],
   "cell_type": "code",
   "source": [
    "using SIIPExamples\n",
    "\n",
    "pkgpath = dirname(dirname(pathof(SIIPExamples)))\n",
    "include(joinpath(pkgpath, \"test\", \"2_PowerSystems_examples\", \"02_parse_matpower.jl\"))"
   ],
   "metadata": {},
   "execution_count": 1
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Write data to a temporary directory"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Serializing to /var/folders/27/2jr8c7gn4j72fvrg4qt81zrw8w_711/T/jl_9wbDpp/system.json\n",
      "[ Info: Serialized time series data to /var/folders/27/2jr8c7gn4j72fvrg4qt81zrw8w_711/T/jl_9wbDpp/system_time_series_storage.h5.\n",
      "[ Info: Serialized System to /var/folders/27/2jr8c7gn4j72fvrg4qt81zrw8w_711/T/jl_9wbDpp/system.json\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "0.023214340209960938"
     },
     "metadata": {},
     "execution_count": 2
    }
   ],
   "cell_type": "code",
   "source": [
    "folder = mktempdir()\n",
    "path = joinpath(folder, \"system.json\")\n",
    "println(\"Serializing to $path\")\n",
    "to_json(sys, path)\n",
    "\n",
    "filesize(path) / (1024 * 1024) #MiB"
   ],
   "metadata": {},
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Read the JSON file and create a new `System`"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ Info: Loaded time series from storage file existing=system_time_series_storage.h5 new=/var/folders/27/2jr8c7gn4j72fvrg4qt81zrw8w_711/T/jl_BTRFoK\n",
      "┌ Warning: Rate 1000.0 MW for bus1-bus5-i_3 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus1-bus4-i_2 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus1-bus2-i_1 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 200.0 MW for bus4-bus5-i_7 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n",
      "┌ Warning: Rate 1200.0 MW for bus2-bus3-i_4 is larger than the max expected in the range of (min = 134.0, max = 145.0).\n",
      "└ @ PowerSystems ~/.julia/packages/PowerSystems/N2l8o/src/utils/IO/branchdata_checks.jl:148\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "System\n======\nSystem Units Base: SYSTEM_BASE\nBase Power: 100.0\nBase Frequency: 60.0\n\nComponents\n==========\nNum components: 30\n\n\u001b[1m9×3 DataFrame\u001b[0m\n\u001b[1m Row \u001b[0m│\u001b[1m ConcreteType             \u001b[0m\u001b[1m SuperTypes                        \u001b[0m\u001b[1m Count \u001b[0m\n\u001b[1m     \u001b[0m│\u001b[90m String                   \u001b[0m\u001b[90m String                            \u001b[0m\u001b[90m Int64 \u001b[0m\n─────┼────────────────────────────────────────────────────────────────────\n   1 │ Arc                       Topology <: Component <: Infrast…      6\n   2 │ Area                      AggregationTopology <: Topology …      1\n   3 │ Bus                       Topology <: Component <: Infrast…      5\n   4 │ Line                      ACBranch <: Branch <: Device <: …      5\n   5 │ LoadZone                  AggregationTopology <: Topology …      1\n   6 │ PhaseShiftingTransformer  ACBranch <: Branch <: Device <: …      2\n   7 │ PowerLoad                 StaticLoad <: ElectricLoad <: St…      3\n   8 │ RenewableDispatch         RenewableGen <: Generator <: Sta…      2\n   9 │ ThermalStandard           ThermalGen <: Generator <: Stati…      5\n\nTimeSeriesContainer\n===================\nComponents with time series data: 0\nTotal StaticTimeSeries: 0\nTotal Forecasts: 0\n",
      "text/html": [
       "<h1>System</h1>\n",
       "<p><b>Base Power</b>: 100.0</p>\n",
       "<h2>Components</h2>\n",
       "<p><b>Num components</b>: 30</p>\n",
       "<table class=\"data-frame\"><thead><tr><th></th><th>ConcreteType</th><th>SuperTypes</th><th>Count</th></tr><tr><th></th><th>String</th><th>String</th><th>Int64</th></tr></thead><tbody><p>9 rows × 3 columns</p><tr><th>1</th><td>Arc</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>6</td></tr><tr><th>2</th><td>Area</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>3</th><td>Bus</td><td>Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>4</th><td>Line</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr><tr><th>5</th><td>LoadZone</td><td>AggregationTopology &lt;: Topology &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>1</td></tr><tr><th>6</th><td>PhaseShiftingTransformer</td><td>ACBranch &lt;: Branch &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>7</th><td>PowerLoad</td><td>StaticLoad &lt;: ElectricLoad &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>3</td></tr><tr><th>8</th><td>RenewableDispatch</td><td>RenewableGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>2</td></tr><tr><th>9</th><td>ThermalStandard</td><td>ThermalGen &lt;: Generator &lt;: StaticInjection &lt;: Device &lt;: Component &lt;: InfrastructureSystemsComponent &lt;: InfrastructureSystemsType &lt;: Any</td><td>5</td></tr></tbody></table>\n",
       "\n",
       "<h2>TimeSeriesContainer</h2>\n",
       "<p><b>Components with time series data</b>: 0</p>\n",
       "<p><b>Total StaticTimeSeries</b>: 0</p>\n",
       "<p><b>Total Forecasts</b>: 0</p>\n",
       "<p><b>Resolution</b>: 0 seconds</p>\n"
      ]
     },
     "metadata": {},
     "execution_count": 3
    }
   ],
   "cell_type": "code",
   "source": [
    "sys2 = System(path)"
   ],
   "metadata": {},
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "source": [
    "---\n",
    "\n",
    "*This notebook was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl).*"
   ],
   "metadata": {}
  }
 ],
 "nbformat_minor": 3,
 "metadata": {
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.5.4"
  },
  "kernelspec": {
   "name": "julia-1.5",
   "display_name": "Julia 1.5.4",
   "language": "julia"
  }
 },
 "nbformat": 4
}