{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Basic Agent Sudy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is recommended to have a look at the [0_basic_functionalities](0_basic_functionalities.ipynb), [1_Observation_Agents](1_Observation_Agents.ipynb), [2_Action_GridManipulation](3_TrainingAnAgent.ipynb) and [3_TrainingAnAgent](3_TrainingAnAgent.ipynb) notebooks before getting into this one."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Objectives**\n",
"\n",
"In this notebook we will expose how to study an Agent. For this notebook to be interested, we first use a dummy agent, and then we look at how to study his behaviour from the file saved.\n",
"\n",
"This notebook will also show you how to use the Graphical User Interface built for analyzing grid2Op agents called \"Grid2Viz\".\n",
"\n",
"It is more than recommended to know how to define an Agent and use a Runner before doing this tutotial!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Evaluate the performance of a simple Agen"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import sys\n",
"import grid2op\n",
"import copy\n",
"import numpy as np\n",
"import shutil\n",
"import seaborn as sns\n",
"import plotly.graph_objects as go\n",
"\n",
"from tqdm.notebook import tqdm\n",
"from grid2op.Agent import PowerLineSwitch\n",
"from grid2op.Reward import L2RPNReward\n",
"from grid2op.Runner import Runner\n",
"from grid2op.Chronics import GridStateFromFileWithForecasts, Multifolder\n",
"path_agents = \"study_agent_getting_started\"\n",
"max_iter = 30"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the next cell we evaluate the agent \"PowerLineSwitch\" and save the results of this evaluation in \"study_agent_getting_started\""
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/donnotben/Documents/Grid2Op_dev/getting_started/grid2op/MakeEnv.py:667: UserWarning:\n",
"\n",
"Your are using only 2 chronics for this environment. More can be download by running, from a command line:\n",
"python -m grid2op.download --name \"case14_realistic\" --path_save PATH\\WHERE\\YOU\\WANT\\TO\\DOWNLOAD\\DATA\n",
"\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "f5756e4b0c984816b6b7bbf95b3a745a",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"HBox(children=(FloatProgress(value=0.0, description='episode', max=2.0, style=ProgressStyle(description_width=…"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "85cb0edd011d4e54b2f760ed2383fdc5",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"HBox(children=(FloatProgress(value=0.0, description='episode', max=30.0, style=ProgressStyle(description_width…"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "7e75ecd01dc547e08753a7f12803b1b2",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"HBox(children=(FloatProgress(value=0.0, description='episode', max=30.0, style=ProgressStyle(description_width…"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"The results for the evaluated agent are:\n",
"\tFor chronics with id 000\n",
"\t\t - cumulative reward: 1094.640995\n",
"\t\t - number of time steps completed: 2 / 30\n",
"\tFor chronics with id 001\n",
"\t\t - cumulative reward: 4374.827712\n",
"\t\t - number of time steps completed: 5 / 30\n"
]
}
],
"source": [
"scoring_function = L2RPNReward\n",
"env = grid2op.make()\n",
"# env.chronics_handler.set_max_iter(max_iter)\n",
"shutil.rmtree(os.path.abspath(path_agents), ignore_errors=True)\n",
"if not os.path.exists(path_agents):\n",
" os.mkdir(path_agents)\n",
"\n",
"# make a runner for this agent\n",
"path_agent = os.path.join(path_agents, \"PowerLineSwitch\")\n",
"shutil.rmtree(os.path.abspath(path_agent), ignore_errors=True)\n",
"\n",
"runner = Runner(**env.get_params_for_runner(),\n",
" agentClass=PowerLineSwitch\n",
" )\n",
"res = runner.run(path_save=path_agent, nb_episode=2, \n",
" max_iter=max_iter,\n",
" pbar=tqdm)\n",
"print(\"The results for the evaluated agent are:\")\n",
"for _, chron_id, cum_reward, nb_time_step, max_ts in res:\n",
" msg_tmp = \"\\tFor chronics with id {}\\n\".format(chron_id)\n",
" msg_tmp += \"\\t\\t - cumulative reward: {:.6f}\\n\".format(cum_reward)\n",
" msg_tmp += \"\\t\\t - number of time steps completed: {:.0f} / {:.0f}\".format(nb_time_step, max_ts)\n",
" print(msg_tmp)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Looking at the results, understand the behaviour of the Agent"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The content of the folder is the following:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"000 dict_action_space.json\t dict_observation_space.json\r\n",
"001 dict_env_modification_space.json\r\n"
]
}
],
"source": [
"!ls $path_agent"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As we notice, there are 2 json that represents the action space and the observation space, used to read the data store. These objects can be loaded this way:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"from grid2op.Utils import ActionSpace, ObservationSpace\n",
"\n",
"action_space = ActionSpace.from_dict(os.path.join(path_agent, \"dict_action_space.json\"))\n",
"env_modif_space = ActionSpace.from_dict(os.path.join(path_agent, \"dict_env_modification_space.json\"))\n",
"observation_space = ObservationSpace.from_dict(os.path.join(path_agent, \"dict_observation_space.json\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can load the data corresponding to episode 1 for example, we can load the actions and the observations and de-serialize them properly into proper objects:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"path_episode_1 = os.path.join(path_agent, \"001\")\n",
"actions_npy = np.load(os.path.join(path_episode_1, \"actions.npy\"))\n",
"li_actions = []\n",
"for i in range(actions_npy.shape[0]):\n",
" try:\n",
" tmp = action_space.from_vect(actions_npy[i,:])\n",
" li_actions.append(tmp)\n",
" except:\n",
" break\n",
" \n",
"observations_npy = np.load(os.path.join(path_episode_1, \"observations.npy\"))\n",
"li_observations = []\n",
"for i in range(observations_npy.shape[0]):\n",
" try:\n",
" tmp = observation_space.from_vect(observations_npy[i,:])\n",
" li_observations.append(tmp)\n",
" except:\n",
" break\n",
" \n",
"env_modifications = np.load(os.path.join(path_episode_1, \"env_modifications.npy\"))\n",
"li_env_modifs = []\n",
"for i in range(env_modifications.shape[0]):\n",
" try:\n",
" tmp = env_modif_space.from_vect(env_modifications[i,:])\n",
" li_env_modifs.append(tmp)\n",
" except:\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Inspect the actions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And now we can start to study the given agent, for example, let's inspect its actions and wonder how many powerlines it has disconnected (for example, this is probably not the best thing to do here...)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"line_disc = 0\n",
"line_reco = 0\n",
"for act in li_actions:\n",
" dict_ = act.as_dict() # representation of an action as a dictionnary, see the documentation for more information\n",
" if \"set_line_status\" in dict_:\n",
" line_reco += dict_[\"set_line_status\"][\"nb_connected\"]\n",
" line_disc += dict_[\"set_line_status\"][\"nb_disconnected\"]\n",
"line_disc"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also wonder how many times this Agent acted on the powerline with id 1, and inspect how many times it has change its status:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"line_disconnected = 0\n",
"for act in li_actions:\n",
" dict_ = act.effect_on(line_id=1) # which effect has this action action on the substation with id 1\n",
" # other objects are: load_id, gen_id, line_id or substation_id\n",
" if dict_['set_line_status'] == 1 :\n",
" line_disconnected += 1\n",
"line_disconnected"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Inspect the modification of the environment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For example, we might want to inspect the number of hazards and maintenance of a total scenario, to see how difficult it was."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"nb_hazards = 0\n",
"nb_maintenance = 0\n",
"for act in li_env_modifs:\n",
" dict_ = act.as_dict() # representation of an action as a dictionnary, see the documentation for more information\n",
" if \"nb_hazards\" in dict_:\n",
" nb_hazards += 1\n",
" if \"nb_maintenance\" in dict_:\n",
" nb_maintenance += 1\n",
"nb_maintenance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Inspect the observations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For example, let's look at the value consumed by load 1. For this cell to work, it requires plotly for displaying the results."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
" \n",
" "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"plotlyServerURL": "https://plot.ly"
},
"data": [
{
"type": "scatter",
"x": [
0,
1,
2,
3,
4,
5
],
"y": [
87.4,
85.8,
87.1,
87.8,
86.9,
86.9
]
}
],
"layout": {
"template": {
"data": {
"bar": [
{
"error_x": {
"color": "#2a3f5f"
},
"error_y": {
"color": "#2a3f5f"
},
"marker": {
"line": {
"color": "#E5ECF6",
"width": 0.5
}
},
"type": "bar"
}
],
"barpolar": [
{
"marker": {
"line": {
"color": "#E5ECF6",
"width": 0.5
}
},
"type": "barpolar"
}
],
"carpet": [
{
"aaxis": {
"endlinecolor": "#2a3f5f",
"gridcolor": "white",
"linecolor": "white",
"minorgridcolor": "white",
"startlinecolor": "#2a3f5f"
},
"baxis": {
"endlinecolor": "#2a3f5f",
"gridcolor": "white",
"linecolor": "white",
"minorgridcolor": "white",
"startlinecolor": "#2a3f5f"
},
"type": "carpet"
}
],
"choropleth": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "choropleth"
}
],
"contour": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "contour"
}
],
"contourcarpet": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "contourcarpet"
}
],
"heatmap": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "heatmap"
}
],
"heatmapgl": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "heatmapgl"
}
],
"histogram": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "histogram"
}
],
"histogram2d": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "histogram2d"
}
],
"histogram2dcontour": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "histogram2dcontour"
}
],
"mesh3d": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "mesh3d"
}
],
"parcoords": [
{
"line": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "parcoords"
}
],
"pie": [
{
"automargin": true,
"type": "pie"
}
],
"scatter": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatter"
}
],
"scatter3d": [
{
"line": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatter3d"
}
],
"scattercarpet": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattercarpet"
}
],
"scattergeo": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattergeo"
}
],
"scattergl": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattergl"
}
],
"scattermapbox": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattermapbox"
}
],
"scatterpolar": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterpolar"
}
],
"scatterpolargl": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterpolargl"
}
],
"scatterternary": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterternary"
}
],
"surface": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"type": "surface"
}
],
"table": [
{
"cells": {
"fill": {
"color": "#EBF0F8"
},
"line": {
"color": "white"
}
},
"header": {
"fill": {
"color": "#C8D4E3"
},
"line": {
"color": "white"
}
},
"type": "table"
}
]
},
"layout": {
"annotationdefaults": {
"arrowcolor": "#2a3f5f",
"arrowhead": 0,
"arrowwidth": 1
},
"coloraxis": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"colorscale": {
"diverging": [
[
0,
"#8e0152"
],
[
0.1,
"#c51b7d"
],
[
0.2,
"#de77ae"
],
[
0.3,
"#f1b6da"
],
[
0.4,
"#fde0ef"
],
[
0.5,
"#f7f7f7"
],
[
0.6,
"#e6f5d0"
],
[
0.7,
"#b8e186"
],
[
0.8,
"#7fbc41"
],
[
0.9,
"#4d9221"
],
[
1,
"#276419"
]
],
"sequential": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
],
"sequentialminus": [
[
0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1,
"#f0f921"
]
]
},
"colorway": [
"#636efa",
"#EF553B",
"#00cc96",
"#ab63fa",
"#FFA15A",
"#19d3f3",
"#FF6692",
"#B6E880",
"#FF97FF",
"#FECB52"
],
"font": {
"color": "#2a3f5f"
},
"geo": {
"bgcolor": "white",
"lakecolor": "white",
"landcolor": "#E5ECF6",
"showlakes": true,
"showland": true,
"subunitcolor": "white"
},
"hoverlabel": {
"align": "left"
},
"hovermode": "closest",
"mapbox": {
"style": "light"
},
"paper_bgcolor": "white",
"plot_bgcolor": "#E5ECF6",
"polar": {
"angularaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"bgcolor": "#E5ECF6",
"radialaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
}
},
"scene": {
"xaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
},
"yaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
},
"zaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
}
},
"shapedefaults": {
"line": {
"color": "#2a3f5f"
}
},
"ternary": {
"aaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"baxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"bgcolor": "#E5ECF6",
"caxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
}
},
"title": {
"x": 0.05
},
"xaxis": {
"automargin": true,
"gridcolor": "white",
"linecolor": "white",
"ticks": "",
"title": {
"standoff": 15
},
"zerolinecolor": "white",
"zerolinewidth": 2
},
"yaxis": {
"automargin": true,
"gridcolor": "white",
"linecolor": "white",
"ticks": "",
"title": {
"standoff": 15
},
"zerolinecolor": "white",
"zerolinewidth": 2
}
}
},
"title": {
"text": "Consumption of load 1"
},
"xaxis": {
"title": {
"text": "Time step"
}
},
"yaxis": {
"title": {
"text": "Load (MW)"
}
}
}
},
"text/html": [
"\n",
" \n",
" \n",
"
\n",
" \n",
"
\n",
" \n",
" \n",
"
\n",
" \n",
"
\n",
" \n",
" \n",
"
\n",
" \n",
"
\n",
" \n",
" \n",
"
\n",
" \n",
"