{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Transmission Network Expansion Planning (TNEP) with PowerModels.jl\n",
    "This tutorial describes how to run the TNEP feature of PowerModels.jl together with pandapower.\n",
    "For more details on PowerModels TNEP see:\n",
    "\n",
    "https://lanl-ansi.github.io/PowerModels.jl/stable/specifications/#Transmission-Network-Expansion-Planning-(TNEP)-1 \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Installation\n",
    "Apart from the Julia, PowerModels, Ipopt and JuMP Installation itself (see the opf_powermodels.ipynb), you need to install\n",
    "some more libraries.\n",
    "  \n",
    "The TNEP problem is a mixed-integer non-linear problem, which is especially easy to solve (not). To be able to solve \n",
    "these kind of problems, you need a suitable solver. Either you use commercial ones (like Knitro) or the open-source\n",
    "Juniper solver (which is partly developed by Carleton Coffrin from PowerModels itself):\n",
    "\n",
    "* Juniper: https://github.com/lanl-ansi/Juniper.jl\n",
    "\n",
    "Additionally CBC is needed:\n",
    "\n",
    "* CBC: https://github.com/coin-or/Cbc\n",
    "* CBC Julia interface: https://github.com/JuliaOpt/Cbc.jl\n",
    "\n",
    "Note that Juniper is a heuristic based solver. Another non-heuristic option would be to use Alpine.jl:\n",
    "* Alpine: https://github.com/lanl-ansi/Alpine.jl\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prepare the Input Data\n",
    "To put it simple, the goal of the optimization is to find a set of new lines from a pre-defined set of possible \n",
    "new lines so that not voltage or line loading violations are violated.   \n",
    "\n",
    "In order to start the optimization, we have to define certain things:\n",
    "1. The \"common\" pandapower grid data with line loading and voltage limits\n",
    "2. The set of available new lines to choose from \n",
    "\n",
    "## 1. Create the grid\n",
    "In this example we use the CIGRE medium voltage grid from pandapower.networks and define the limits for all lines /\n",
    "buses as:\n",
    "* max line loading limit: 60%\n",
    "* min voltage magnitude: 0.95 p.u.\n",
    "* max voltage magnitude: 1.05 p.u.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandapower.networks as nw\n",
    "from pandapower.converter.pandamodels.to_pm import init_ne_line\n",
    "\n",
    "def cigre_grid():\n",
    "    net = nw.create_cigre_network_mv()\n",
    "\n",
    "    net[\"bus\"].loc[:, \"min_vm_pu\"] = 0.95\n",
    "    net[\"bus\"].loc[:, \"max_vm_pu\"] = 1.05\n",
    "\n",
    "    net[\"line\"].loc[:, \"max_loading_percent\"] = 60.\n",
    "    return net\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Define the new line measures to choose from\n",
    "Since we want to solve a line loading problem, we define \"parallel\" lines to all existing lines to choose from. To \n",
    "define this, two steps are necessary:\n",
    "1. Create new lines in the existing \"line\" DataFrame and set them out of service\n",
    "2. Create the \"ne_line\" DataFrame which specifies which lines are the possible ones to be built. This DataFrame is \n",
    "similar to the line DataFrame, except that is has an additional column \"construction_cost\". These define the costs\n",
    "for the lines to be built.\n",
    "\n",
    "Note that it is important to set the lines \"out of service\" in the line DataFrame. Otherwise, they are already \"built\".\n",
    "In the \"ne_line\" DataFrame the lines are set \"in service\". The init_ne_line() function takes care of this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "def define_possible_new_lines(net):\n",
    "    # Here the possible new lines are a copy of all the lines which are already in the grid \n",
    "    max_idx = max(net[\"line\"].index)\n",
    "    net[\"line\"] = pd.concat([net[\"line\"]] * 2, ignore_index=True) # duplicate\n",
    "    # they must be set out of service in the line DataFrame (otherwise they are already \"built\")\n",
    "    net[\"line\"].loc[max_idx + 1:, \"in_service\"] = False\n",
    "    # get the index of the new lines\n",
    "    new_lines = net[\"line\"].loc[max_idx + 1:].index\n",
    "\n",
    "    # creates the new line DataFrame net[\"ne_line\"] which defines the measures to choose from. The costs are defined\n",
    "    # exemplary as 1. for every line. \n",
    "    init_ne_line(net, new_lines, construction_costs=np.ones(len(new_lines)))\n",
    "\n",
    "    return net\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run the optimization\n",
    "Now we run the optimization and print the results. First we init the grid with the new lines and check if some\n",
    "limits are violated (otherwise there is not much to optimize). Then we run \"runpm_tnep(net)\" and print the newly\n",
    "built lines and assert the line loading limits with a power flow calculation.\n",
    "\n",
    "The newly built lines can be found in the DataFrame net[\"res_ne_line\"], which has one column \"built\". A newly\n",
    "built line is marked as True, otherwise False.\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import pandapower as pp\n",
    "\n",
    "def pm_tnep_cigre():\n",
    "    # get the grid\n",
    "    net = cigre_grid()\n",
    "    # add the possible new lines\n",
    "    define_possible_new_lines(net)\n",
    "    # check if max line loading percent is violated (should be)\n",
    "    pp.runpp(net)\n",
    "    print(\"Max line loading prior to optimization:\")\n",
    "    print(net.res_line.loading_percent.max())\n",
    "    assert np.any(net[\"res_line\"].loc[:, \"loading_percent\"] > net[\"line\"].loc[:, \"max_loading_percent\"])\n",
    "\n",
    "    # run power models tnep optimization\n",
    "    pp.runpm_tnep(net)\n",
    "    # print the information about the newly built lines\n",
    "    print(\"These lines are to be built:\")\n",
    "    print(net[\"res_ne_line\"])\n",
    "    \n",
    "    # set lines to be built in service\n",
    "    lines_to_built = net[\"res_ne_line\"].loc[net[\"res_ne_line\"].loc[:, \"built\"], \"built\"].index\n",
    "    net[\"line\"].loc[lines_to_built, \"in_service\"] = True\n",
    "    \n",
    "    # run a power flow calculation again and check if max_loading percent is still violated\n",
    "    pp.runpp(net)\n",
    "    \n",
    "    # check max line loading results\n",
    "    assert not np.any(net[\"res_line\"].loc[:, \"loading_percent\"] > net[\"line\"].loc[:, \"max_loading_percent\"])\n",
    "    \n",
    "    print(\"Max line loading after the optimization:\")\n",
    "    print(net.res_line.loading_percent.max())\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notes\n",
    "Juniper is based on a heuristic, it does not necessarly find the global optimum. For this use another solver\n",
    "\n",
    "In the PowerModels OPF formulation, generator limits are taken into account. This means you have to specify limits \n",
    "for all gens, ext_grids and controllable sgens / loads. Optionally costs for these can be defined. The CIGRE MV grid\n",
    "has pre-defined limits set for the ext_grid. In other cases you might get an error. Here is a code snippet:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def define_ext_grid_limits(net):\n",
    "    # define limits\n",
    "    net[\"ext_grid\"].loc[:, \"min_p_mw\"] = -9999.\n",
    "    net[\"ext_grid\"].loc[:, \"max_p_mw\"] = 9999.\n",
    "    net[\"ext_grid\"].loc[:, \"min_q_mvar\"] = -9999.\n",
    "    net[\"ext_grid\"].loc[:, \"max_q_mvar\"] = 9999.\n",
    "    # define costs\n",
    "    for i in net.ext_grid.index:\n",
    "        pp.create_poly_cost(net, i, 'ext_grid', cp1_eur_per_mw=1) "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}