{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# This Notebook will dive more into the Action class" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is recommended to have a look at the [0_basic_functionalities](0_basic_functionalities.ipynb) and [1_Observation_Agents](1_Observation_Agents.ipynb) notebooks before getting into this one." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Objectives**\n", "\n", "This notebook will cover the basic of how use at best the *Action* class to perform actions (modify) on the powergrid efficiently. Indeed there are multiple concepts behind this class that are not very clear.\n", "\n", "This notebook will be focused on the manipulation of Action from an expert system point of view to demonstrate how a desired action is fundamentally taken in the Grid2Op environment. For a more automatic way to handle action (for example using machine learning, the notebook [3_TrainingAnAgent](3_TrainingAnAgent.ipynb) will give a more detailed example." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys\n", "import grid2op" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
run previous cell, wait for 2 seconds
\n", "" ], "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = None\n", "try:\n", " from jyquickhelper import add_notebook_menu\n", " res = add_notebook_menu()\n", "except ModuleNotFoundError:\n", " print(\"Impossible to automatically add a menu / table of content to this notebook.\\nYou can download \\\"jyquickhelper\\\" package with: \\n\\\"pip install jyquickhelper\\\"\")\n", "res" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## I) A few comments on actions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is recommended to have built locally the documentation, and to refer to the [action](https://grid2op.readthedocs.io/en/latest/action.html) pages of the documentation or to the [Action.py](grid2op/Action/Action.py) file for a more detailed view on these two classes below." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### I.A) \"change\" vs \"set\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To manipulate a powergrid, we decided to introduce two distinct (yet close) concepts that will affect the objects differently:\n", "\n", "- **change** will:\n", " - connect a powerline if it was disconnected, or reconnect if it was connected. Similar to a \"toggle\" operation.\n", " - assign the object to the second bus if it was connected to bus 1 or assign it to bus 1 if it was connected to bus 2\n", "- **set** will:\n", " - connect a powerline (regardless of its previous status) or disconnect it (regardless of its previous status)\n", " - assign the object (component in the power network) to a specific bus (regardless of its previous bus and status -- for powerline)\n", " \n", "This is another change compared to the previous pypownet implementation, were only the `change` concept was implemented. Having these two things helps understand what is really going on on the powergrid and allows to represent better the intention of the Agent, especially in debugging phase.\n", "\n", "Of course, it is perfectly possible to use only the `change` capability and thus being closer to the original implementation.\n", "\n", "Let's give a \"concrete\" example to highlight the difference between these two methods. Suppose we have a substation with 5 elements:\n", "- the origin of powerline $l_1$\n", "- the extremity of powerline $l_2$\n", "- the extremity of powerline $l_3$\n", "- a load $c_1$\n", "- a generator $g_1$\n", "\n", "Let's also assume the original configuration (before the action is applied, *i.e.,* the configuration of the observation at time *t*) is:\n", "\n", "| Object Name | Original Bus | Original Status|\n", "|------------------|--------------|----------------|\n", "| $l_1$ (origin) | 1 | connected |\n", "| $l_2$ (extremity)| 2 | connected |\n", "| $l_3$ (extremity)| NA | disconnected |\n", "| $c_1$ | 1 | NA |\n", "| $g_1 $ | 2 | NA |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's say:\n", "* action $a_1$ consists in \"**change** the status of origin of powerline $l_1$\". After applying this action, the status of powerline 1 is: \"disconnected\".\n", "* action $a_2$ consists in \"**change** the status of origin of powerline $l_3$\". After applying this action, the status of powerline 2 is: \"connected\" ***\\****.\n", "* action $a_3$ \"**set** the bus of $c_1$ to bus 1\". It is equivalent to doing absolutely nothing.\n", "* action $a_4$ \"**set** the bus of $g_1$ to bus 1\". It will change the bus of $g_1$ and assign it to bus 1.\n", "\n", "\\* **NB** Another breaking change compared to the pypownet implementation is the introduction of \"ambiguous\" action. When a powerline is disconneted, it is not connected to any bus (by definition). So if you reconnect it without specifying on which \"bus\" it's \"ambiguous\". This action will not be implemented and the episode will terminate. More on this concept later." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The previous actions are **equivalent** to:\n", "* $a_1 \\to$ **set** status of $l_1$ to \"disconnected\"\n", "* $a_2 \\to$ **set** status of $l_3$ to \"connected\"\n", "* $a_3 \\to$ do nothing\n", "* $a_4 \\to$ **change** bus of $g_1$ " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### I.B) \"ambiguous\" action" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Some actions are \"ambiguous\" it means that they cannot be properly and / or univocally interpretaded. **These actions will raise an \"AmbiguousAction\" exception if attempted to be used on the powergrid. This will immediately terminate the episode.**\n", "\n", "For a detailed list of ambiguous actions, the documentation is the **only** official sources. Are presented here only some example. The documentation is available at [\\_check\\_for\\_ambiguity](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.Action._check_for_ambiguity).\n", "\n", "An action can be ambiguous in the following context:\n", "\n", " - It affects the \"*injections*\" in an incorrect way:\n", "\n", " - it tries to modify a load (setting active or reactive values) that doesn't exist on the powergrid\n", " - it sets the values of a generator that doesn't exist (setting its voltage setpoint or active production)\n", "\n", " - It affects the \"*powerlines*\" in an incorrect manner:\n", "\n", " - it tries to change the status or to assign to a specific bus a powerline that doesn't exist\n", " - somes lines are reconnected bus the action doesn't specify on which bus.\n", " - the same action tries to **changed** and **set** the status of a powerline\n", "\n", " - It has an ambiguous behaviour concerning the topology of some substations\n", "\n", " - the same action tries to affect with **changed** and **set** the bus of a given object. \n", " - the bus is trying to be modified (**set** or **changed**) on a object not present on the powergrid" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## II) Action Helper / action space" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**IMPORTANT NOTICE** Each *Agent* has its own 'action helper' attribute that can be called by `self.action_space`. This is the only recommended way to create a valid *Action*. Using its constructor is strongly NOT recommended, as it requires a deep knowledge of all the elements in the powergrid, as well as their names, their type, the order in which they are used in the backend etc. For performance reasons, no sanity check are performed to make sure the would be created action this way is compatible with the backend.\n", "\n", "In the next cell, we retrieve the action space used by the Agent." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "2 main classes are usefull when dealing with *Action* in Grid2Op. The *Action* class is the most basic one. The *ActionHelper* is a tool that helps create and manipulate some actions. \n", "\n", "As in most of our notebooks, we start by creating an Environment. We will use the `case14_fromfile` provided as an example. We will then extract the complete action space (*action_space*) that is possible to perform on the power grid as a dictionary. When a specific action such as *change* or *set* is needed to be performed then we can apply this change to the *action_space* dictionary by accessing the relevant key in it as discussed below." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/tezirg/Code/Grid2Op.BDonnot/getting_started/grid2op/MakeEnv/Make.py:223: UserWarning: You are using a development environment. This environment is not intended for training agents.\n", " warnings.warn(_MAKE_DEV_ENV_WARN)\n" ] } ], "source": [ "# import the usefull class\n", "env = grid2op.make(test=True)\n", "action_space = env.action_space" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As opposed to the previous plateform, [pypownet](https://github.com/MarvinLer/pypownet), Grid2Op doesn't restrict anything on the type of Action. Generally speaking, an Action can modify production, loads, topology etc. By default though, an Action that an Agent can performed is a [TopologyAction](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.TopologyAction), a specific type restricting it's usage to:\n", "\n", "- change the status of the powerline (reconnect / disconnect) them\n", "- change the way the objects (end of a powerline, generator or load) are interconnected at substations.\n", "\n", "We will focus on this class in this notebook." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then best way to get an action is to give a `dictionnary` to the \"action space\" of the player. For example, to get the \"do nothing\" action, you can just pass the empty dictionnary." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "do_nothing = action_space({})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### II.A) line status modification" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### a) modify multiple powerlines" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want to change (or set) the status of most of the powerlines, you can create a vector having the same size as the number of powerlines of the grid, and pass it to the dictionnary with relevant keys \"set_line_status\", \"change_line_status\" and its values \"set_status\", \"change_status\" respectively.\n", "\n", "The following code will:\n", "- change the status of powerlines with id 0,1,2\n", "- set the status \"connected\" for powerline with id 3,4\n", "- set the status \"disconnected\" for powerlines with id 5 and 6" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "change_status = action_space.get_change_line_status_vect()\n", "change_status[0] = True\n", "change_status[1] = True\n", "change_status[2] = True\n", "\n", "set_status = action_space.get_set_line_status_vect()\n", "set_status[3] = 1\n", "set_status[4] = 1\n", "set_status[5] = -1\n", "set_status[6] = -1\n", "\n", "this_first_act = action_space({\"set_line_status\": set_status, \"change_line_status\": change_status})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**NB** even if it can handles different type, for performance reason it's better to follow the type of data mentionned bellow. Note also that the data will be interpreted as:\n", "* for **change** having a boolean vector\n", " * `True` meaning \"*change*\"\n", " * `False` meaning \"*don't change*\"\n", "* for **set** it's an integer vector with: \n", " * `0` meaning \"*do nothing*\"\n", " * `1` meaning \"*connect it*\"\n", " * `-1` meaning \"*disconnect it*\"\n", " \n", "For convenience reason, an Action object can be easily inspected by using the \"`print`\" method. It will summarize on which object it has an impact:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - force reconnection of 2 powerlines ([3 4])\n", "\t - force disconnection of 2 powerlines ([5 6])\n", "\t - switch status of 3 powerlines ([0 1 2])\n", "\t - NOT switch anything in the topology\n", "\t - NOT force any particular bus configuration\n" ] } ], "source": [ "print(this_first_act)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(True,\n", " Grid2OpException AmbiguousAction InvalidLineStatus InvalidLineStatus(\"You ask to reconnect powerline 3 yet didn't tell on which bus.\",))" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "this_first_act.is_ambiguous()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**NB** this action is ambiguous so it cannot be implemented on the powergrid. Indeed, powerlines 3 and 4 are reconnected, but we don't specify on which bus! Implementing this action on a grid will be equivalent to doing nothing." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### b) modify multiple powerlines at the same time" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It's not always convenient to manipulate all the status of all the powerlines, or change it. Fore mor convenience, it's possible to modify only a few of them. The syntax is the following." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - force reconnection of 2 powerlines ([3 4])\n", "\t - force disconnection of 2 powerlines ([5 6])\n", "\t - switch status of 3 powerlines ([0 1 2])\n", "\t - NOT switch anything in the topology\n", "\t - NOT force any particular bus configuration\n" ] } ], "source": [ "the_same_act = action_space({\"set_line_status\": [(3,1), (4,1), (5,-1), (6,-1)],\n", " \"change_line_status\": [0,1,2]\n", " })\n", "print(the_same_act)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can check that the two actions are indeed equal:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "the_same_act == this_first_act" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### II.B) substations reconfiguration / topology changes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One of the interesting aspect of Grid2Op is to be able to modify the topology of the powergrid. In other words it allows to reconfigure the way the objects (generator, load, end of powerlines) are interconnected at their substation.\n", "\n", "Comparable to the status change, topological change can be interpreted in two disctincts manners, as described above. The most interesting interactions \n", "\n", "In this section we study how to modify the topology of the powergrid.\n", "\n", "The underlying way to represent the topology is through a integer vector, having the same dimension as the number of objects of the grid, and for each objects it says one which bus it's connected. Manipulating this vector can be done, but is absolutely not handy. We present here the way to use the helper to change this topology more easily.\n", "\n", "To **set** the bus to which a load is connected, it is recommended to do:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - NOT force any line status\n", "\t - NOT switch any line status\n", "\t - NOT switch anything in the topology\n", "\t - Set the bus of the following element:\n", "\t \t - assign bus 2 to load 0 [on substation 1]\n" ] } ], "source": [ "set_bus_load_0 = action_space({\"set_bus\": {\"loads_id\": [(0,2)]}})\n", "print(set_bus_load_0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To change the bus to which a generator is connected, it is recommended to do:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To **change** the bus a similar interface can be used:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - NOT force any line status\n", "\t - NOT switch any line status\n", "\t - Change the bus of the following element:\n", "\t \t - switch bus of load 0 [on substation 1]\n", "\t - NOT force any particular bus configuration\n" ] } ], "source": [ "change_bus_load_0 = action_space({\"change_bus\": {\"loads_id\": [0]}})\n", "print(change_bus_load_0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The API is really similar for generator:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - NOT force any line status\n", "\t - NOT switch any line status\n", "\t - NOT switch anything in the topology\n", "\t - Set the bus of the following element:\n", "\t \t - assign bus 2 to generator 4 [on substation 0]\n", "\t \t - assign bus 2 to generator 3 [on substation 7]\n" ] } ], "source": [ "change_bus_gen_0_and_1 = action_space({\"change_bus\": {\"generators_id\": [0,1]}})\n", "set_bus_gen_3_and_4 = action_space({\"set_bus\": {\"generators_id\": [(3,2), (4,2)]}})\n", "print(set_bus_gen_3_and_4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It follows the same mechanism for each ends of the powerlines:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - NOT force any line status\n", "\t - NOT switch any line status\n", "\t - NOT switch anything in the topology\n", "\t - Set the bus of the following element:\n", "\t \t - assign bus 2 to line (extremity) 18 [on substation 7]\n" ] } ], "source": [ "change_bus_lines_or_0 = action_space({\"change_bus\": {\"lines_or_id\": [0]}})\n", "set_bus_lines_or_4 = action_space({\"set_bus\": {\"lines_or_id\": [(3,2)]}})\n", "change_bus_lines_ex_15 = action_space({\"change_bus\": {\"lines_ex_id\": [15]}})\n", "set_bus_lines_ex_18 = action_space({\"set_bus\": {\"lines_ex_id\": [(18,2)]}})\n", "print(set_bus_lines_ex_18)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When reconnecting a powerline, if the bus to which this powerline is reconnected are not specified, the action is ambiguous and thus will not be implemented. It is, in that case, recommended to use the `reconnect_powerline` method as followed:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - force reconnection of 1 powerlines ([1])\n", "\t - NOT switch any line status\n", "\t - NOT switch anything in the topology\n", "\t - Set the bus of the following element:\n", "\t \t - assign bus 1 to line (origin) 1 [on substation 0]\n", "\t \t - assign bus 1 to line (extremity) 1 [on substation 4]\n" ] } ], "source": [ "reconnecting_line_1 = action_space.reconnect_powerline(line_id=1,bus_or=1,bus_ex=1)\n", "print(reconnecting_line_1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### II.C) substations reconfiguration / topology changes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For convenience, it might be better sometimes to use the name of some object to change their bus in case the ID's are not known. Grid2Op allows to do that, only for changing or setting a bus. This method takes longer than the methods showed above. If they are used at all, it's recommended NOT to use them for training an Agent. Their main goal aims at debugging and / or understanding the behaviour of an Agent. \n", "\n", "These methods are:\n", "* [`action_space.change_bus`](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.ActionSpace.change_bus) ($\\leftarrow$ this is a link)\n", "* [`action_space.set_bus`](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.ActionSpace.set_bus) ($\\leftarrow$ this is a link)\n", "\n", "Please refer to the official documentation for a complete detail of their behaviour. To sum up, we can use them this way:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - NOT force any line status\n", "\t - NOT switch any line status\n", "\t - NOT switch anything in the topology\n", "\t - Set the bus of the following element:\n", "\t \t - assign bus 2 to generator 0 [on substation 1]\n" ] } ], "source": [ "my_act = action_space.set_bus(\"gen_1_0\", # mandatory name of the element\n", " new_bus=2, # mandatory the new bus to connect it too\n", " type_element=\"gen\", # optional the type of the element, one of \"line\", \"gen\" or \"load\"\n", " previous_action=None # optional: if you want to combine multiple action, you can do it with this\n", " )\n", "print(my_act)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This action will:\n", "\t - NOT change anything to the injections\n", "\t - NOT perform any redispatching action\n", "\t - NOT force any line status\n", "\t - NOT switch any line status\n", "\t - NOT switch anything in the topology\n", "\t - Set the bus of the following element:\n", "\t \t - assign bus 2 to line (origin) 3 [on substation 1]\n", "\t \t - assign bus 2 to generator 0 [on substation 1]\n" ] } ], "source": [ "action_space.set_bus(\"1_3_3\", # mandatory name of the element\n", " extremity=\"or\", # mandatory, which extrmity to change\n", " new_bus=2, # mandatory the new bus to connect it too\n", " type_element=\"line\", # optional the type of the element, one of \"line\", \"gen\" or \"load\"\n", " previous_action=my_act # optional: if you want to combine multiple action, you can do it with this\n", " )\n", "print(my_act)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## III) Manipulating Action" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is absolutely **NOT** recommended to use *Actions* outside of the action space." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 2 }