{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Repairing Code Automatically\n",
    "\n",
    "So far, we have discussed how to track failures and how to locate defects in code. Let us now discuss how to _repair_ defects – that is, to correct the code such that the failure no longer occurs. We will discuss how to _repair code automatically_ – by systematically searching through possible fixes and evolving the most promising candidates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:29.256586Z",
     "iopub.status.busy": "2023-11-12T12:47:29.256416Z",
     "iopub.status.idle": "2023-11-12T12:47:29.294650Z",
     "shell.execute_reply": "2023-11-12T12:47:29.294356Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"640\"\n",
       "            height=\"360\"\n",
       "            src=\"https://www.youtube-nocookie.com/embed/UJTf7cW0idI\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x103a95cc0>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bookutils import YouTubeVideo\n",
    "YouTubeVideo(\"UJTf7cW0idI\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Prerequisites**\n",
    "\n",
    "* Re-read the [introduction to debugging](Intro_Debugging.ipynb), notably on how to properly fix code.\n",
    "* We make use of automatic fault localization, as discussed in the [chapter on statistical debugging](StatisticalDebugger.ipynb).\n",
    "* We make extensive use of code transformations, as discussed in the [chapter on tracing executions](Tracer.ipynb).\n",
    "* We make use of [delta debugging](DeltaDebugger.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:29.315182Z",
     "iopub.status.busy": "2023-11-12T12:47:29.314987Z",
     "iopub.status.idle": "2023-11-12T12:47:29.318221Z",
     "shell.execute_reply": "2023-11-12T12:47:29.317883Z"
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import bookutils.setup"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "## Synopsis\n",
    "<!-- Automatically generated. Do not edit. -->\n",
    "\n",
    "To [use the code provided in this chapter](Importing.ipynb), write\n",
    "\n",
    "```python\n",
    ">>> from debuggingbook.Repairer import <identifier>\n",
    "```\n",
    "\n",
    "and then make use of the following features.\n",
    "\n",
    "\n",
    "This chapter provides tools and techniques for automated repair of program code. The `Repairer` class takes a `RankingDebugger` debugger as input (such as `OchiaiDebugger` from the [chapter on statistical debugging](StatisticalDebugger.ipynb). A typical setup looks like this:\n",
    "\n",
    "```python\n",
    "from debuggingbook.StatisticalDebugger import OchiaiDebugger\n",
    "\n",
    "debugger = OchiaiDebugger()\n",
    "for inputs in TESTCASES:\n",
    "    with debugger:\n",
    "        test_foo(inputs)\n",
    "...\n",
    "\n",
    "repairer = Repairer(debugger)\n",
    "```\n",
    "Here, `test_foo()` is a function that raises an exception if the tested function `foo()` fails. If `foo()` passes, `test_foo()` should not raise an exception.\n",
    "\n",
    "The `repair()` method of a `Repairer` searches for a repair of the code covered in the debugger (except for methods whose name starts or ends in `test`, such that `foo()`, not `test_foo()` is repaired). `repair()` returns the best fix candidate as a pair `(tree, fitness)` where `tree` is a [Python abstract syntax tree](http://docs.python.org/3/library/ast) (AST) of the fix candidate, and `fitness` is the fitness of the candidate (a value between 0 and 1). A `fitness` of 1.0 means that the candidate passed all tests. A typical usage looks like this:\n",
    "\n",
    "```python\n",
    "tree, fitness = repairer.repair()\n",
    "print(ast.unparse(tree), fitness)\n",
    "```\n",
    "\n",
    "Here is a complete example for the `middle()` program. This is the original source code of `middle()`:\n",
    "\n",
    "```python\n",
    "def middle(x, y, z):  # type: ignore\n",
    "    if y < z:\n",
    "        if x < y:\n",
    "            return y\n",
    "        elif x < z:\n",
    "            return y\n",
    "    else:\n",
    "        if x > y:\n",
    "            return y\n",
    "        elif x > z:\n",
    "            return x\n",
    "    return z\n",
    "```\n",
    "We set up a function `middle_test()` that tests it. The `middle_debugger`  collects testcases and outcomes:\n",
    "\n",
    "```python\n",
    ">>> middle_debugger = OchiaiDebugger()\n",
    ">>> for x, y, z in MIDDLE_PASSING_TESTCASES + MIDDLE_FAILING_TESTCASES:\n",
    ">>>     with middle_debugger:\n",
    ">>>         middle_test(x, y, z)\n",
    "```\n",
    "The repairer is instantiated with the debugger used (`middle_debugger`):\n",
    "\n",
    "```python\n",
    ">>> middle_repairer = Repairer(middle_debugger)\n",
    "```\n",
    "The `repair()` method of the repairer attempts to repair the function invoked by the test (`middle()`).\n",
    "\n",
    "```python\n",
    ">>> tree, fitness = middle_repairer.repair()\n",
    "```\n",
    "The returned AST `tree` can be output via `ast.unparse()`:\n",
    "\n",
    "```python\n",
    ">>> print(ast.unparse(tree))\n",
    "def middle(x, y, z):\n",
    "    if y < z:\n",
    "        if x < y:\n",
    "            return y\n",
    "        elif x < z:\n",
    "            return x\n",
    "    elif x > y:\n",
    "        return y\n",
    "    elif x > z:\n",
    "        return x\n",
    "    return z\n",
    "\n",
    "```\n",
    "The `fitness` value shows how well the repaired program fits the tests. A fitness value of 1.0 shows that the repaired program satisfies all tests.\n",
    "\n",
    "```python\n",
    ">>> fitness\n",
    "1.0\n",
    "```\n",
    "Hence, the above program indeed is a perfect repair in the sense that all previously failing tests now pass – our repair was successful.\n",
    "\n",
    "Here are the classes defined in this chapter. A `Repairer` repairs a program, using a `StatementMutator` and a `CrossoverOperator` to evolve a population of candidates.\n",
    "\n",
    "![](PICS/Repairer-synopsis-1.svg)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Automatic Code Repairs\n",
    "\n",
    "So far, we have discussed how to locate defects in code, how to track failures back to the defects that caused them, and how to systematically determine failure conditions. Let us now address the last step in debugging – namely, how to _automatically fix code_.\n",
    "\n",
    "Already in the [introduction to debugging](Intro_Debugging.ipynb), we have discussed how to fix code manually. Notably, we have established that a _diagnosis_ (which induces a fix) should show _causality_ (i.e., how the defect causes the failure) and _incorrectness_ (how the defect is wrong). Is it possible to obtain such a diagnosis automatically?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In this chapter, we introduce a technique of _automatic code repair_ – that is, for a given failure, automatically determine a fix that makes the failure go away. To do so, we randomly (but systematically) _mutate_ the program code – that is, insert, change, and delete fragments – until we find a change that actually causes the failing test to pass."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "If this sounds like an audacious idea, that is because it is. But not only is _automated program repair_ one of the hottest topics of software research in the last decade, it is also being increasingly deployed in industry. At Facebook, for instance, every failing test report comes with an automatically generated _repair suggestion_ – a suggestion that already has been validated to work. Programmers can apply the suggestion as is or use it as basis for their own fixes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### The middle() Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let us introduce our ongoing example. In the [chapter on statistical debugging](StatisticalDebugger.ipynb), we have introduced the `middle()` function – a function that returns the \"middle\" of three numbers `x`, `y`, and `z`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:29.320106Z",
     "iopub.status.busy": "2023-11-12T12:47:29.320010Z",
     "iopub.status.idle": "2023-11-12T12:47:29.948163Z",
     "shell.execute_reply": "2023-11-12T12:47:29.947833Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from StatisticalDebugger import middle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:29.950050Z",
     "iopub.status.busy": "2023-11-12T12:47:29.949890Z",
     "iopub.status.idle": "2023-11-12T12:47:29.951543Z",
     "shell.execute_reply": "2023-11-12T12:47:29.951302Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# ignore\n",
    "from bookutils import print_content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:29.952992Z",
     "iopub.status.busy": "2023-11-12T12:47:29.952890Z",
     "iopub.status.idle": "2023-11-12T12:47:29.954467Z",
     "shell.execute_reply": "2023-11-12T12:47:29.954229Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# ignore\n",
    "import inspect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:29.955963Z",
     "iopub.status.busy": "2023-11-12T12:47:29.955844Z",
     "iopub.status.idle": "2023-11-12T12:47:30.017834Z",
     "shell.execute_reply": "2023-11-12T12:47:30.017554Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "708  \u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):  \u001b[37m# type: ignore\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "709      \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "710          \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "711              \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "712          \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "713              \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "714      \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "715          \u001b[34mif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "716              \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "717          \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "718              \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "719      \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "# ignore\n",
    "_, first_lineno = inspect.getsourcelines(middle)\n",
    "middle_source = inspect.getsource(middle)\n",
    "print_content(middle_source, '.py', start_line_number=first_lineno)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In most cases, `middle()` just runs fine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.019424Z",
     "iopub.status.busy": "2023-11-12T12:47:30.019311Z",
     "iopub.status.idle": "2023-11-12T12:47:30.021463Z",
     "shell.execute_reply": "2023-11-12T12:47:30.021195Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle(4, 5, 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In some other cases, though, it does not work correctly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.023051Z",
     "iopub.status.busy": "2023-11-12T12:47:30.022940Z",
     "iopub.status.idle": "2023-11-12T12:47:30.024989Z",
     "shell.execute_reply": "2023-11-12T12:47:30.024714Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle(2, 1, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Validated Repairs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Now, if we only want a repair that fixes this one given failure, this would be very easy. All we have to do is to replace the entire body by a single statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.026636Z",
     "iopub.status.busy": "2023-11-12T12:47:30.026524Z",
     "iopub.status.idle": "2023-11-12T12:47:30.028153Z",
     "shell.execute_reply": "2023-11-12T12:47:30.027903Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def middle_sort_of_fixed(x, y, z):  # type: ignore\n",
    "    return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "You will concur that the failure no longer occurs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.029650Z",
     "iopub.status.busy": "2023-11-12T12:47:30.029549Z",
     "iopub.status.idle": "2023-11-12T12:47:30.031551Z",
     "shell.execute_reply": "2023-11-12T12:47:30.031289Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_sort_of_fixed(2, 1, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "But this, of course, is not the aim of automatic fixes, nor of fixes in general: We want our fixes not only to make the given failure go away, but we also want the resulting code to be _correct_ (which, of course, is a lot harder)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Automatic repair techniques therefore assume the existence of a _test suite_ that can check whether an implementation satisfies its requirements. Better yet, one can use the test suite to gradually check _how close_ one is to perfection: A piece of code that satisfies 99% of all tests is better than one that satisfies ~33% of all tests, as `middle_sort_of_fixed()` would do (assuming the test suite evenly checks the input space)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Genetic Optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The common approach for automatic repair follows the principle of _genetic optimization_. Roughly spoken, genetic optimization is a _metaheuristic_ inspired by the process of _natural selection_. The idea is to _evolve_ a selection of _candidate solutions_ towards a maximum _fitness_:\n",
    "\n",
    "1. Have a selection of _candidates_.\n",
    "2. Determine the _fitness_ of each candidate.\n",
    "3. Retain those candidates with the _highest fitness_.\n",
    "4. Create new candidates from the retained candidates, by applying genetic operations:\n",
    "    * _Mutation_ mutates some aspect of a candidate.\n",
    "    * _CrossoverOperator_ creates new candidates combining features of two candidates.\n",
    "5. Repeat until an optimal solution is found."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Applied for automated program repair, this means the following steps:\n",
    "\n",
    "1. Have a _test suite_ with both failing and passing tests that helps to assert correctness of possible solutions.\n",
    "2. With the test suite, use [fault localization](StatisticalDebugger.ipynb) to determine potential code locations to be fixed.\n",
    "3. Systematically _mutate_ the code (by adding, changing, or deleting code) and _cross_ code to create possible fix candidates.\n",
    "4. Identify the _fittest_ fix candidates – that is, those that satisfy the most tests.\n",
    "5. _Evolve_ the fittest candidates until a perfect fix is found, or until time resources are depleted."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let us illustrate these steps in the following sections."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## A Test Suite"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In automated repair, the larger and the more thorough the test suite, the higher the quality of the resulting fix (if any). Hence, if we want to repair `middle()` automatically, we need a good test suite – with good inputs, but also with good checks. Note that running the test suite commonly takes the most time of automated repair, so a large test suite also comes with extra cost."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let us first focus on achieving high-quality repairs. Hence, we will use the extensive test suites introduced in the [chapter on statistical debugging](StatisticalDebugger.ipynb):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.033396Z",
     "iopub.status.busy": "2023-11-12T12:47:30.033272Z",
     "iopub.status.idle": "2023-11-12T12:47:30.034893Z",
     "shell.execute_reply": "2023-11-12T12:47:30.034655Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from StatisticalDebugger import MIDDLE_PASSING_TESTCASES, MIDDLE_FAILING_TESTCASES"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The `middle_test()` function fails whenever `middle()` returns an incorrect result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.036434Z",
     "iopub.status.busy": "2023-11-12T12:47:30.036331Z",
     "iopub.status.idle": "2023-11-12T12:47:30.038158Z",
     "shell.execute_reply": "2023-11-12T12:47:30.037909Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def middle_test(x: int, y: int, z: int) -> None:\n",
    "    m = middle(x, y, z)\n",
    "    assert m == sorted([x, y, z])[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.039809Z",
     "iopub.status.busy": "2023-11-12T12:47:30.039685Z",
     "iopub.status.idle": "2023-11-12T12:47:30.041322Z",
     "shell.execute_reply": "2023-11-12T12:47:30.041051Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from ExpectError import ExpectError"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.042769Z",
     "iopub.status.busy": "2023-11-12T12:47:30.042679Z",
     "iopub.status.idle": "2023-11-12T12:47:30.044539Z",
     "shell.execute_reply": "2023-11-12T12:47:30.044292Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/var/folders/n2/xd9445p97rb3xh7m1dfx8_4h0006ts/T/ipykernel_33724/3661663124.py\", line 2, in <cell line: 1>\n",
      "    middle_test(2, 1, 3)\n",
      "  File \"/var/folders/n2/xd9445p97rb3xh7m1dfx8_4h0006ts/T/ipykernel_33724/40742806.py\", line 3, in middle_test\n",
      "    assert m == sorted([x, y, z])[1]\n",
      "AssertionError (expected)\n"
     ]
    }
   ],
   "source": [
    "with ExpectError():\n",
    "    middle_test(2, 1, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Locating the Defect"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Our next step is to find potential defect locations – that is, those locations in the code our mutations should focus upon. Since we already do have two test suites, we can make use of [statistical debugging](StatisticalDebugger.ipynb) to identify likely faulty locations.  Our `OchiaiDebugger` ranks individual code lines by how frequently they are executed in failing runs (and not in passing runs)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.046287Z",
     "iopub.status.busy": "2023-11-12T12:47:30.046167Z",
     "iopub.status.idle": "2023-11-12T12:47:30.047820Z",
     "shell.execute_reply": "2023-11-12T12:47:30.047548Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from StatisticalDebugger import OchiaiDebugger, RankingDebugger"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.049377Z",
     "iopub.status.busy": "2023-11-12T12:47:30.049263Z",
     "iopub.status.idle": "2023-11-12T12:47:30.056166Z",
     "shell.execute_reply": "2023-11-12T12:47:30.055832Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "middle_debugger = OchiaiDebugger()\n",
    "\n",
    "for x, y, z in MIDDLE_PASSING_TESTCASES + MIDDLE_FAILING_TESTCASES:\n",
    "    with middle_debugger:\n",
    "        middle_test(x, y, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We see that the upper half of the `middle()` code is definitely more suspicious:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.057802Z",
     "iopub.status.busy": "2023-11-12T12:47:30.057704Z",
     "iopub.status.idle": "2023-11-12T12:47:30.086073Z",
     "shell.execute_reply": "2023-11-12T12:47:30.085776Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 1:  70%\">   1 def middle_test(x: int, y: int, z: int) -&gt; None:</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 2:  70%\">   2     m = middle(x, y, z)</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 3:  70%\">   3     assert m == sorted([x, y, z])[1]</pre>\n",
       "\n",
       "<p/><pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 708:  70%\"> 708 def middle(x, y, z):  # type: ignore</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 709:  70%\"> 709     if y &lt; z:</pre>\n",
       "<pre style=\"background-color:hsl(15.155513200675452, 100.0%, 80%)\"\n",
       "                    title=\"Line 710:  87%\"> 710         if x &lt; y:</pre>\n",
       "<pre style=\"background-color:hsl(120.0, 8.0%, 80%)\"\n",
       "                    title=\"Line 711:   0%\"> 711             return y</pre>\n",
       "<pre style=\"background-color:hsl(11.799643839908121, 100.0%, 80%)\"\n",
       "                    title=\"Line 712:  90%\"> 712         elif x &lt; z:</pre>\n",
       "<pre style=\"background-color:hsl(3.9916213145203594, 100.0%, 80%)\"\n",
       "                    title=\"Line 713:  96%\"> 713             return y</pre>\n",
       "<pre title=\"Line 714: not executed\"> 714     else:\n",
       "</pre>\n",
       "<pre style=\"background-color:hsl(120.0, 69.0%, 80%)\"\n",
       "                    title=\"Line 715:   0%\"> 715         if x &gt; y:</pre>\n",
       "<pre style=\"background-color:hsl(120.0, 19.0%, 80%)\"\n",
       "                    title=\"Line 716:   0%\"> 716             return y</pre>\n",
       "<pre style=\"background-color:hsl(120.0, 50.0%, 80%)\"\n",
       "                    title=\"Line 717:   0%\"> 717         elif x &gt; z:</pre>\n",
       "<pre style=\"background-color:hsl(120.0, 19.0%, 80%)\"\n",
       "                    title=\"Line 718:   0%\"> 718             return x</pre>\n",
       "<pre style=\"background-color:hsl(120.0, 47.0%, 80%)\"\n",
       "                    title=\"Line 719:   0%\"> 719     return z</pre>\n"
      ],
      "text/markdown": [
       "| `middle_test` | `x=9, y=0, z=5` | `x=0, y=1, z=9` | `x=9, y=0, z=9` | `x=2, y=9, z=0` | `x=4, y=7, z=2` | `x=1, y=6, z=1` | `x=7, y=5, z=4` | `x=6, y=6, z=2` | `x=7, y=6, z=4` | `x=0, y=7, z=7` | `x=3, y=9, z=7` | `x=2, y=3, z=8` | `x=2, y=6, z=3` | `x=2, y=8, z=4` | `x=8, y=7, z=5` | `x=0, y=0, z=6` | `x=5, y=2, z=5` | `x=9, y=6, z=9` | `x=8, y=5, z=5` | `x=1, y=7, z=0` | `x=0, y=4, z=4` | `x=8, y=7, z=5` | `x=7, y=1, z=5` | `x=3, y=5, z=3` | `x=6, y=3, z=2` | `x=3, y=9, z=8` | `x=1, y=3, z=8` | `x=5, y=9, z=1` | `x=5, y=5, z=7` | `x=9, y=0, z=3` | `x=5, y=6, z=3` | `x=7, y=9, z=8` | `x=2, y=2, z=3` | `x=1, y=8, z=1` | `x=3, y=3, z=3` | `x=7, y=3, z=6` | `x=3, y=7, z=6` | `x=2, y=3, z=1` | `x=8, y=0, z=6` | `x=5, y=4, z=0` | `x=5, y=9, z=2` | `x=6, y=7, z=1` | `x=1, y=1, z=7` | `x=1, y=6, z=8` | `x=0, y=9, z=1` | `x=5, y=9, z=7` | `x=1, y=5, z=6` | `x=1, y=6, z=1` | `x=7, y=8, z=4` | `x=2, y=8, z=4` | `x=3, y=4, z=3` | `x=8, y=4, z=5` | `x=0, y=5, z=8` | `x=9, y=7, z=0` | `x=1, y=5, z=4` | `x=9, y=8, z=9` | `x=9, y=2, z=4` | `x=3, y=2, z=0` | `x=8, y=5, z=2` | `x=4, y=9, z=7` | `x=4, y=8, z=0` | `x=5, y=9, z=1` | `x=7, y=6, z=4` | `x=8, y=9, z=2` | `x=9, y=1, z=4` | `x=9, y=7, z=2` | `x=1, y=9, z=5` | `x=1, y=9, z=1` | `x=7, y=8, z=0` | `x=4, y=8, z=6` | `x=9, y=2, z=1` | `x=9, y=4, z=5` | `x=0, y=4, z=2` | `x=6, y=8, z=4` | `x=0, y=0, z=1` | `x=6, y=4, z=3` | `x=8, y=0, z=3` | `x=9, y=5, z=1` | `x=7, y=6, z=6` | `x=4, y=6, z=0` | `x=1, y=1, z=0` | `x=8, y=4, z=1` | `x=1, y=6, z=2` | `x=4, y=5, z=1` | `x=1, y=1, z=6` | `x=1, y=2, z=1` | `x=2, y=4, z=2` | `x=7, y=3, z=3` | `x=2, y=7, z=4` | `x=0, y=8, z=0` | `x=4, y=6, z=4` | `x=1, y=5, z=2` | `x=0, y=8, z=2` | `x=3, y=5, z=6` | `x=7, y=2, z=4` | `x=5, y=4, z=5` | `x=2, y=3, z=4` | `x=2, y=2, z=4` | `x=3, y=6, z=2` | `x=9, y=4, z=0` | `x=2, y=1, z=5` | `x=5, y=2, z=9` | `x=2, y=1, z=3` | `x=3, y=0, z=7` | `x=4, y=1, z=6` | `x=2, y=0, z=9` | `x=4, y=3, z=7` | `x=5, y=3, z=8` | `x=6, y=0, z=9` | `x=2, y=0, z=4` | `x=4, y=0, z=5` | `x=6, y=5, z=9` | `x=5, y=3, z=9` | `x=7, y=0, z=9` | `x=5, y=3, z=9` | `x=2, y=1, z=6` | `x=3, y=2, z=4` | `x=2, y=1, z=5` | `x=3, y=2, z=6` | `x=6, y=4, z=7` | `x=5, y=2, z=6` | `x=4, y=2, z=5` | `x=6, y=2, z=9` | `x=4, y=2, z=8` | `x=2, y=0, z=3` | `x=2, y=1, z=5` | `x=2, y=1, z=9` | `x=7, y=6, z=9` | `x=6, y=0, z=9` | `x=7, y=6, z=9` | `x=8, y=6, z=9` | `x=7, y=6, z=9` | `x=1, y=0, z=2` | `x=1, y=0, z=3` | `x=3, y=2, z=7` | `x=6, y=5, z=9` | `x=1, y=0, z=3` | `x=6, y=4, z=7` | `x=8, y=7, z=9` | `x=2, y=0, z=4` | `x=5, y=1, z=7` | `x=6, y=4, z=9` | `x=8, y=0, z=9` | `x=2, y=1, z=4` | `x=4, y=1, z=8` | `x=3, y=0, z=4` | `x=3, y=1, z=5` | `x=7, y=5, z=8` | `x=6, y=2, z=9` | `x=2, y=0, z=4` | `x=7, y=6, z=9` | `x=2, y=1, z=6` | `x=3, y=0, z=8` | `x=5, y=3, z=8` | `x=5, y=0, z=9` | `x=4, y=0, z=9` | `x=7, y=5, z=8` | `x=4, y=2, z=7` | `x=3, y=1, z=6` | `x=2, y=0, z=4` | `x=4, y=0, z=8` | `x=2, y=1, z=8` | `x=5, y=3, z=9` | `x=4, y=3, z=5` | `x=6, y=4, z=9` | `x=7, y=6, z=8` | `x=3, y=1, z=9` | `x=4, y=2, z=7` | `x=1, y=0, z=6` | `x=1, y=0, z=5` | `x=7, y=0, z=9` | `x=4, y=0, z=7` | `x=3, y=2, z=9` | `x=4, y=1, z=6` | `x=5, y=2, z=9` | `x=8, y=0, z=9` | `x=2, y=1, z=9` | `x=4, y=3, z=6` | `x=6, y=0, z=9` | `x=3, y=1, z=4` | `x=7, y=6, z=8` | `x=6, y=1, z=7` | `x=6, y=2, z=7` | `x=3, y=0, z=9` | `x=5, y=4, z=9` | `x=8, y=6, z=9` | `x=5, y=3, z=6` | `x=2, y=1, z=7` | `x=3, y=0, z=5` | `x=6, y=1, z=8` | `x=2, y=0, z=5` | `x=2, y=0, z=9` | `x=6, y=5, z=8` | `x=2, y=0, z=4` | `x=5, y=0, z=9` | `x=1, y=0, z=9` | `x=3, y=1, z=6` | `x=3, y=1, z=5` | `x=5, y=1, z=9` | `x=6, y=2, z=7` | \n",
       "| ------------- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">   middle:708</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">   middle:709</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(15.155513200675452, 100.0%, 80%)\" title=\" 87%\">   middle:710</samp> |    X |    X |    X |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    X |    X |    X |    - |    - |    - |    - |    X |    - |    - |    - |    X |    - |    X |    X |    - |    - |    X |    - |    - |    X |    - |    - |    X |    - |    - |    - |    X |    X |    - |    - |    X |    - |    - |    - |    - |    X |    X |    - |    - |    X |    X |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    X |    - |    - |    X |    - |    X |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    X |    X |    X |    X |    X |    - |    - |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(120.0, 8.0%, 80%)\" title=\"  0%\">   middle:711</samp> |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    X |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - | \n",
       "| <samp style=\"background-color: hsl(11.799643839908121, 100.0%, 80%)\" title=\" 90%\">   middle:712</samp> |    X |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    X |    X |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    X |    X |    - |    - |    X |    - |    - |    X |    - |    - |    X |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    X |    X |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    X |    - |    - |    X |    - |    X |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    X |    - |    X |    - |    - |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(3.9916213145203594, 100.0%, 80%)\" title=\" 96%\">   middle:713</samp> |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(120.0, 69.0%, 80%)\" title=\"  0%\">   middle:715</samp> |    - |    - |    - |    X |    X |    X |    X |    X |    X |    X |    X |    - |    X |    X |    X |    - |    - |    - |    X |    X |    X |    X |    - |    X |    X |    X |    - |    X |    - |    - |    X |    X |    - |    X |    X |    - |    X |    X |    - |    X |    X |    X |    - |    - |    X |    X |    - |    X |    X |    X |    X |    - |    - |    X |    X |    - |    - |    X |    X |    X |    X |    X |    X |    X |    - |    X |    X |    X |    X |    X |    X |    - |    X |    X |    - |    X |    - |    X |    X |    X |    X |    X |    X |    X |    - |    X |    X |    X |    X |    X |    X |    X |    X |    - |    - |    - |    - |    - |    X |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - | \n",
       "| <samp style=\"background-color: hsl(120.0, 19.0%, 80%)\" title=\"  0%\">   middle:716</samp> |    - |    - |    - |    - |    - |    - |    X |    - |    X |    - |    - |    - |    - |    - |    X |    - |    - |    - |    X |    - |    - |    X |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    X |    X |    - |    - |    - |    X |    - |    - |    X |    - |    - |    - |    - |    X |    - |    - |    - |    - |    X |    - |    X |    X |    - |    - |    X |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - | \n",
       "| <samp style=\"background-color: hsl(120.0, 50.0%, 80%)\" title=\"  0%\">   middle:717</samp> |    - |    - |    - |    X |    X |    X |    - |    X |    - |    X |    X |    - |    X |    X |    - |    - |    - |    - |    - |    X |    X |    - |    - |    X |    - |    X |    - |    X |    - |    - |    X |    X |    - |    X |    X |    - |    X |    X |    - |    - |    X |    X |    - |    - |    X |    X |    - |    X |    X |    X |    X |    - |    - |    - |    X |    - |    - |    - |    - |    X |    X |    X |    - |    X |    - |    - |    X |    X |    X |    X |    - |    - |    X |    X |    - |    - |    - |    - |    - |    X |    X |    - |    X |    X |    - |    X |    X |    - |    X |    X |    X |    X |    X |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - | \n",
       "| <samp style=\"background-color: hsl(120.0, 19.0%, 80%)\" title=\"  0%\">   middle:718</samp> |    - |    - |    - |    X |    X |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    X |    - |    - |    - |    - |    - |    - |    X |    - |    - |    X |    X |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    X |    - |    X |    - |    - |    - |    - |    X |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    X |    X |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - | \n",
       "| <samp style=\"background-color: hsl(120.0, 47.0%, 80%)\" title=\"  0%\">   middle:719</samp> |    X |    - |    X |    - |    - |    X |    - |    - |    - |    X |    X |    - |    X |    X |    - |    - |    X |    X |    - |    - |    X |    - |    X |    X |    - |    X |    - |    - |    - |    X |    - |    X |    - |    X |    X |    X |    X |    - |    X |    - |    - |    - |    - |    - |    X |    X |    - |    X |    - |    X |    X |    X |    - |    - |    X |    X |    X |    - |    - |    X |    - |    - |    - |    - |    X |    - |    X |    X |    - |    X |    - |    X |    X |    - |    - |    - |    X |    - |    - |    - |    - |    - |    X |    - |    - |    X |    X |    - |    X |    X |    X |    X |    X |    - |    X |    X |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">middle_test:1</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">middle_test:2</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">middle_test:3</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n"
      ],
      "text/plain": [
       "[('middle', 713), ('middle', 712), ('middle', 710), ('middle_test', 2), ('middle', 708), ('middle_test', 3), ('middle_test', 1), ('middle', 709), ('middle', 716), ('middle', 719), ('middle', 711), ('middle', 717), ('middle', 715), ('middle', 718)]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_debugger"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The most suspicious line is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.087738Z",
     "iopub.status.busy": "2023-11-12T12:47:30.087628Z",
     "iopub.status.idle": "2023-11-12T12:47:30.121675Z",
     "shell.execute_reply": "2023-11-12T12:47:30.121390Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "713            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "# ignore\n",
    "location = middle_debugger.rank()[0]\n",
    "(func_name, lineno) = location\n",
    "lines, first_lineno = inspect.getsourcelines(middle)\n",
    "print(lineno, end=\"\")\n",
    "print_content(lines[lineno - first_lineno], '.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "with a suspiciousness of:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.123363Z",
     "iopub.status.busy": "2023-11-12T12:47:30.123265Z",
     "iopub.status.idle": "2023-11-12T12:47:30.125655Z",
     "shell.execute_reply": "2023-11-12T12:47:30.125393Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9667364890456637"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# ignore\n",
    "middle_debugger.suspiciousness(location)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Random Code Mutations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Our third step in automatic code repair is to _randomly mutate the code_. Specifically, we want to randomly _delete_, _insert_, and _replace_ statements in the program to be repaired. However, simply synthesizing code _from scratch_ is unlikely to yield anything meaningful – the number of combinations is simply far too high. Already for a three-character identifier name, we have more than 200,000 combinations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.127311Z",
     "iopub.status.busy": "2023-11-12T12:47:30.127190Z",
     "iopub.status.idle": "2023-11-12T12:47:30.128787Z",
     "shell.execute_reply": "2023-11-12T12:47:30.128518Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.130245Z",
     "iopub.status.busy": "2023-11-12T12:47:30.130156Z",
     "iopub.status.idle": "2023-11-12T12:47:30.132326Z",
     "shell.execute_reply": "2023-11-12T12:47:30.131948Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string.ascii_letters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.133875Z",
     "iopub.status.busy": "2023-11-12T12:47:30.133766Z",
     "iopub.status.idle": "2023-11-12T12:47:30.135985Z",
     "shell.execute_reply": "2023-11-12T12:47:30.135709Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "210357"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(string.ascii_letters + '_') * \\\n",
    "  len(string.ascii_letters + '_' + string.digits) * \\\n",
    "  len(string.ascii_letters + '_' + string.digits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Hence, we do _not_ synthesize code from scratch, but instead _reuse_ elements from the program to be fixed, hypothesizing that \"a program that contains an error in one area likely implements the correct behavior elsewhere\" \\cite{LeGoues2012}. This insight has been dubbed the *plastic surgery hypothesis*: content of new code can often be assembled out of fragments of code that already exist in the code base \\citeBarr2014}."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "For our \"plastic surgery\", we do not operate on a _textual_ representation of the program, but rather on a _structural_ representation, which by construction allows us to avoid lexical and syntactical errors in the first place.\n",
    "\n",
    "This structural representation is the _abstract syntax tree_ (AST), which we already have seen in various chapters, such as the [chapter on delta debugging](DeltaDebugger.ipynb), the [chapter on tracing](Tracer.ipynb), and excessively in the [chapter on slicing](Slicer.ipynb). The [official Python `ast` reference](http://docs.python.org/3/library/ast) is complete, but a bit brief; the documentation [\"Green Tree Snakes - the missing Python AST docs\"](https://greentreesnakes.readthedocs.io/en/latest/) provides an excellent introduction.\n",
    "\n",
    "Recapitulating, an AST is a tree representation of the program, showing a hierarchical structure of the program's elements. Here is the AST for our `middle()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.137611Z",
     "iopub.status.busy": "2023-11-12T12:47:30.137501Z",
     "iopub.status.idle": "2023-11-12T12:47:30.139102Z",
     "shell.execute_reply": "2023-11-12T12:47:30.138814Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import ast\n",
    "import inspect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.140615Z",
     "iopub.status.busy": "2023-11-12T12:47:30.140510Z",
     "iopub.status.idle": "2023-11-12T12:47:30.142072Z",
     "shell.execute_reply": "2023-11-12T12:47:30.141832Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from bookutils import print_content, show_ast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.143707Z",
     "iopub.status.busy": "2023-11-12T12:47:30.143588Z",
     "iopub.status.idle": "2023-11-12T12:47:30.145272Z",
     "shell.execute_reply": "2023-11-12T12:47:30.145031Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def middle_tree() -> ast.AST:\n",
    "    return ast.parse(inspect.getsource(middle))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.146783Z",
     "iopub.status.busy": "2023-11-12T12:47:30.146673Z",
     "iopub.status.idle": "2023-11-12T12:47:30.566795Z",
     "shell.execute_reply": "2023-11-12T12:47:30.566373Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"1862pt\" height=\"476pt\" viewBox=\"0.00 0.00 1862.00 476.00\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 472)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-472 1858,-472 1858,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<text text-anchor=\"start\" x=\"197.62\" y=\"-445.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">FunctionDef</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<text text-anchor=\"middle\" x=\"67\" y=\"-372.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;middle&quot;</text>\n",
       "</g>\n",
       "<!-- 0&#45;&#45;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0--1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M247,-431C247,-431 174.24,-413.42 117,-396 114.02,-395.09 110.95,-394.13 107.86,-393.13\"/>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<text text-anchor=\"start\" x=\"133.88\" y=\"-373.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">arguments</text>\n",
       "</g>\n",
       "<!-- 0&#45;&#45;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>0--2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M247,-431C247,-431 217.68,-410.94 195.47,-395.74\"/>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>9</title>\n",
       "<text text-anchor=\"start\" x=\"464.75\" y=\"-373.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">If</text>\n",
       "</g>\n",
       "<!-- 0&#45;&#45;9 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>0--9</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M247,-431C247,-431 386.32,-398.94 446.19,-385.17\"/>\n",
       "</g>\n",
       "<!-- 70 -->\n",
       "<g id=\"node71\" class=\"node\">\n",
       "<title>70</title>\n",
       "<text text-anchor=\"start\" x=\"1129.25\" y=\"-373.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Return</text>\n",
       "</g>\n",
       "<!-- 0&#45;&#45;70 -->\n",
       "<g id=\"edge70\" class=\"edge\">\n",
       "<title>0--70</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M247,-431C247,-431 959.36,-390.16 1121.45,-380.87\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<text text-anchor=\"start\" x=\"14.62\" y=\"-301.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">arg</text>\n",
       "</g>\n",
       "<!-- 2&#45;&#45;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>2--3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M158,-359C158,-359 104.5,-341.4 63,-324 59.94,-322.72 56.76,-321.32 53.62,-319.89\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<text text-anchor=\"start\" x=\"86.62\" y=\"-301.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">arg</text>\n",
       "</g>\n",
       "<!-- 2&#45;&#45;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>2--5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M158,-359C158,-359 135.24,-338.94 117.99,-323.74\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>7</title>\n",
       "<text text-anchor=\"start\" x=\"158.62\" y=\"-301.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">arg</text>\n",
       "</g>\n",
       "<!-- 2&#45;&#45;7 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>2--7</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M158,-359C158,-359 163.01,-338.94 166.82,-323.74\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<text text-anchor=\"middle\" x=\"27\" y=\"-228.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;x&quot;</text>\n",
       "</g>\n",
       "<!-- 3&#45;&#45;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>3--4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M27,-288.05C27,-277.05 27,-262.76 27,-251.79\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>6</title>\n",
       "<text text-anchor=\"middle\" x=\"99\" y=\"-228.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;y&quot;</text>\n",
       "</g>\n",
       "<!-- 5&#45;&#45;6 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>5--6</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M99,-288.05C99,-277.05 99,-262.76 99,-251.79\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>8</title>\n",
       "<text text-anchor=\"middle\" x=\"171\" y=\"-228.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;z&quot;</text>\n",
       "</g>\n",
       "<!-- 7&#45;&#45;8 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>7--8</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M171,-288.05C171,-277.05 171,-262.76 171,-251.79\"/>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>10</title>\n",
       "<text text-anchor=\"start\" x=\"322.12\" y=\"-301.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Compare</text>\n",
       "</g>\n",
       "<!-- 9&#45;&#45;10 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>9--10</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M490,-359C490,-359 428.7,-336.07 387.52,-320.66\"/>\n",
       "</g>\n",
       "<!-- 18 -->\n",
       "<g id=\"node19\" class=\"node\">\n",
       "<title>18</title>\n",
       "<text text-anchor=\"start\" x=\"594.75\" y=\"-301.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">If</text>\n",
       "</g>\n",
       "<!-- 9&#45;&#45;18 -->\n",
       "<g id=\"edge18\" class=\"edge\">\n",
       "<title>9--18</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M490,-359C490,-359 542.8,-334.7 576.22,-319.32\"/>\n",
       "</g>\n",
       "<!-- 44 -->\n",
       "<g id=\"node45\" class=\"node\">\n",
       "<title>44</title>\n",
       "<text text-anchor=\"start\" x=\"1314.75\" y=\"-301.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">If</text>\n",
       "</g>\n",
       "<!-- 9&#45;&#45;44 -->\n",
       "<g id=\"edge44\" class=\"edge\">\n",
       "<title>9--44</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M490,-359C490,-359 1153.65,-317.57 1296.23,-308.67\"/>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>11</title>\n",
       "<text text-anchor=\"start\" x=\"226.5\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 10&#45;&#45;11 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>10--11</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M345,-287C345,-287 300.02,-264.07 269.8,-248.66\"/>\n",
       "</g>\n",
       "<!-- 14 -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>14</title>\n",
       "<text text-anchor=\"middle\" x=\"315\" y=\"-228.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Lt</text>\n",
       "</g>\n",
       "<!-- 10&#45;&#45;14 -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>10--14</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M345,-287C345,-287 333.43,-266.94 324.66,-251.74\"/>\n",
       "</g>\n",
       "<!-- 15 -->\n",
       "<g id=\"node16\" class=\"node\">\n",
       "<title>15</title>\n",
       "<text text-anchor=\"start\" x=\"370.5\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 10&#45;&#45;15 -->\n",
       "<g id=\"edge15\" class=\"edge\">\n",
       "<title>10--15</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M345,-287C345,-287 361.2,-266.94 373.48,-251.74\"/>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>12</title>\n",
       "<text text-anchor=\"middle\" x=\"171\" y=\"-156.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;y&quot;</text>\n",
       "</g>\n",
       "<!-- 11&#45;&#45;12 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>11--12</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M236,-215C236,-215 210.93,-194.94 191.92,-179.74\"/>\n",
       "</g>\n",
       "<!-- 13 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>13</title>\n",
       "<text text-anchor=\"middle\" x=\"243\" y=\"-156.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 11&#45;&#45;13 -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>11--13</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M236,-215C236,-215 238.7,-194.94 240.75,-179.74\"/>\n",
       "</g>\n",
       "<!-- 16 -->\n",
       "<g id=\"node17\" class=\"node\">\n",
       "<title>16</title>\n",
       "<text text-anchor=\"middle\" x=\"315\" y=\"-156.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;z&quot;</text>\n",
       "</g>\n",
       "<!-- 15&#45;&#45;16 -->\n",
       "<g id=\"edge16\" class=\"edge\">\n",
       "<title>15--16</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M380,-215C380,-215 354.93,-194.94 335.92,-179.74\"/>\n",
       "</g>\n",
       "<!-- 17 -->\n",
       "<g id=\"node18\" class=\"node\">\n",
       "<title>17</title>\n",
       "<text text-anchor=\"middle\" x=\"387\" y=\"-156.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 15&#45;&#45;17 -->\n",
       "<g id=\"edge17\" class=\"edge\">\n",
       "<title>15--17</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M380,-215C380,-215 382.7,-194.94 384.75,-179.74\"/>\n",
       "</g>\n",
       "<!-- 19 -->\n",
       "<g id=\"node20\" class=\"node\">\n",
       "<title>19</title>\n",
       "<text text-anchor=\"start\" x=\"516.12\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Compare</text>\n",
       "</g>\n",
       "<!-- 18&#45;&#45;19 -->\n",
       "<g id=\"edge19\" class=\"edge\">\n",
       "<title>18--19</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M613,-287C613,-287 586.77,-266.94 566.89,-251.74\"/>\n",
       "</g>\n",
       "<!-- 27 -->\n",
       "<g id=\"node28\" class=\"node\">\n",
       "<title>27</title>\n",
       "<text text-anchor=\"start\" x=\"636.25\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Return</text>\n",
       "</g>\n",
       "<!-- 18&#45;&#45;27 -->\n",
       "<g id=\"edge27\" class=\"edge\">\n",
       "<title>18--27</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M613,-287C613,-287 631.52,-266.94 645.55,-251.74\"/>\n",
       "</g>\n",
       "<!-- 31 -->\n",
       "<g id=\"node32\" class=\"node\">\n",
       "<title>31</title>\n",
       "<text text-anchor=\"start\" x=\"882.75\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">If</text>\n",
       "</g>\n",
       "<!-- 18&#45;&#45;31 -->\n",
       "<g id=\"edge31\" class=\"edge\">\n",
       "<title>18--31</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M613,-287C613,-287 794.32,-253.08 864.09,-240.03\"/>\n",
       "</g>\n",
       "<!-- 20 -->\n",
       "<g id=\"node21\" class=\"node\">\n",
       "<title>20</title>\n",
       "<text text-anchor=\"start\" x=\"442.5\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 19&#45;&#45;20 -->\n",
       "<g id=\"edge20\" class=\"edge\">\n",
       "<title>19--20</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M543,-215C543,-215 510.19,-194.69 485.58,-179.45\"/>\n",
       "</g>\n",
       "<!-- 23 -->\n",
       "<g id=\"node24\" class=\"node\">\n",
       "<title>23</title>\n",
       "<text text-anchor=\"middle\" x=\"531\" y=\"-156.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Lt</text>\n",
       "</g>\n",
       "<!-- 19&#45;&#45;23 -->\n",
       "<g id=\"edge23\" class=\"edge\">\n",
       "<title>19--23</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M543,-215C543,-215 538.37,-194.94 534.86,-179.74\"/>\n",
       "</g>\n",
       "<!-- 24 -->\n",
       "<g id=\"node25\" class=\"node\">\n",
       "<title>24</title>\n",
       "<text text-anchor=\"start\" x=\"586.5\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 19&#45;&#45;24 -->\n",
       "<g id=\"edge24\" class=\"edge\">\n",
       "<title>19--24</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M543,-215C543,-215 566.15,-194.94 583.69,-179.74\"/>\n",
       "</g>\n",
       "<!-- 21 -->\n",
       "<g id=\"node22\" class=\"node\">\n",
       "<title>21</title>\n",
       "<text text-anchor=\"middle\" x=\"387\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;x&quot;</text>\n",
       "</g>\n",
       "<!-- 20&#45;&#45;21 -->\n",
       "<g id=\"edge21\" class=\"edge\">\n",
       "<title>20--21</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M452,-143C452,-143 426.93,-122.94 407.92,-107.74\"/>\n",
       "</g>\n",
       "<!-- 22 -->\n",
       "<g id=\"node23\" class=\"node\">\n",
       "<title>22</title>\n",
       "<text text-anchor=\"middle\" x=\"459\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 20&#45;&#45;22 -->\n",
       "<g id=\"edge22\" class=\"edge\">\n",
       "<title>20--22</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M452,-143C452,-143 454.7,-122.94 456.75,-107.74\"/>\n",
       "</g>\n",
       "<!-- 25 -->\n",
       "<g id=\"node26\" class=\"node\">\n",
       "<title>25</title>\n",
       "<text text-anchor=\"middle\" x=\"531\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;y&quot;</text>\n",
       "</g>\n",
       "<!-- 24&#45;&#45;25 -->\n",
       "<g id=\"edge25\" class=\"edge\">\n",
       "<title>24--25</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M596,-143C596,-143 570.93,-122.94 551.92,-107.74\"/>\n",
       "</g>\n",
       "<!-- 26 -->\n",
       "<g id=\"node27\" class=\"node\">\n",
       "<title>26</title>\n",
       "<text text-anchor=\"middle\" x=\"603\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 24&#45;&#45;26 -->\n",
       "<g id=\"edge26\" class=\"edge\">\n",
       "<title>24--26</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M596,-143C596,-143 598.7,-122.94 600.75,-107.74\"/>\n",
       "</g>\n",
       "<!-- 28 -->\n",
       "<g id=\"node29\" class=\"node\">\n",
       "<title>28</title>\n",
       "<text text-anchor=\"start\" x=\"658.5\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 27&#45;&#45;28 -->\n",
       "<g id=\"edge28\" class=\"edge\">\n",
       "<title>27--28</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M664.39,-216.05C666.59,-205.05 669.45,-190.76 671.64,-179.79\"/>\n",
       "</g>\n",
       "<!-- 29 -->\n",
       "<g id=\"node30\" class=\"node\">\n",
       "<title>29</title>\n",
       "<text text-anchor=\"middle\" x=\"675\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;y&quot;</text>\n",
       "</g>\n",
       "<!-- 28&#45;&#45;29 -->\n",
       "<g id=\"edge29\" class=\"edge\">\n",
       "<title>28--29</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M682,-143C682,-143 679.3,-122.94 677.25,-107.74\"/>\n",
       "</g>\n",
       "<!-- 30 -->\n",
       "<g id=\"node31\" class=\"node\">\n",
       "<title>30</title>\n",
       "<text text-anchor=\"middle\" x=\"747\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 28&#45;&#45;30 -->\n",
       "<g id=\"edge30\" class=\"edge\">\n",
       "<title>28--30</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M682,-143C682,-143 707.07,-122.94 726.08,-107.74\"/>\n",
       "</g>\n",
       "<!-- 32 -->\n",
       "<g id=\"node33\" class=\"node\">\n",
       "<title>32</title>\n",
       "<text text-anchor=\"start\" x=\"862.12\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Compare</text>\n",
       "</g>\n",
       "<!-- 31&#45;&#45;32 -->\n",
       "<g id=\"edge32\" class=\"edge\">\n",
       "<title>31--32</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M901,-215C901,-215 897.14,-194.94 894.22,-179.74\"/>\n",
       "</g>\n",
       "<!-- 40 -->\n",
       "<g id=\"node41\" class=\"node\">\n",
       "<title>40</title>\n",
       "<text text-anchor=\"start\" x=\"982.25\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Return</text>\n",
       "</g>\n",
       "<!-- 31&#45;&#45;40 -->\n",
       "<g id=\"edge40\" class=\"edge\">\n",
       "<title>31--40</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M901,-215C901,-215 943.19,-194.3 974.33,-179.03\"/>\n",
       "</g>\n",
       "<!-- 33 -->\n",
       "<g id=\"node34\" class=\"node\">\n",
       "<title>33</title>\n",
       "<text text-anchor=\"start\" x=\"802.5\" y=\"-85.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 32&#45;&#45;33 -->\n",
       "<g id=\"edge33\" class=\"edge\">\n",
       "<title>32--33</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M891,-143C891,-143 863.23,-122.94 842.18,-107.74\"/>\n",
       "</g>\n",
       "<!-- 36 -->\n",
       "<g id=\"node37\" class=\"node\">\n",
       "<title>36</title>\n",
       "<text text-anchor=\"middle\" x=\"891\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Lt</text>\n",
       "</g>\n",
       "<!-- 32&#45;&#45;36 -->\n",
       "<g id=\"edge36\" class=\"edge\">\n",
       "<title>32--36</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M891,-143C891,-143 891,-122.94 891,-107.74\"/>\n",
       "</g>\n",
       "<!-- 37 -->\n",
       "<g id=\"node38\" class=\"node\">\n",
       "<title>37</title>\n",
       "<text text-anchor=\"start\" x=\"946.5\" y=\"-85.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 32&#45;&#45;37 -->\n",
       "<g id=\"edge37\" class=\"edge\">\n",
       "<title>32--37</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M891,-143C891,-143 918.77,-122.94 939.82,-107.74\"/>\n",
       "</g>\n",
       "<!-- 34 -->\n",
       "<g id=\"node35\" class=\"node\">\n",
       "<title>34</title>\n",
       "<text text-anchor=\"middle\" x=\"747\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;x&quot;</text>\n",
       "</g>\n",
       "<!-- 33&#45;&#45;34 -->\n",
       "<g id=\"edge34\" class=\"edge\">\n",
       "<title>33--34</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M812,-71C812,-71 786.93,-50.94 767.92,-35.74\"/>\n",
       "</g>\n",
       "<!-- 35 -->\n",
       "<g id=\"node36\" class=\"node\">\n",
       "<title>35</title>\n",
       "<text text-anchor=\"middle\" x=\"819\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 33&#45;&#45;35 -->\n",
       "<g id=\"edge35\" class=\"edge\">\n",
       "<title>33--35</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M812,-71C812,-71 814.7,-50.94 816.75,-35.74\"/>\n",
       "</g>\n",
       "<!-- 38 -->\n",
       "<g id=\"node39\" class=\"node\">\n",
       "<title>38</title>\n",
       "<text text-anchor=\"middle\" x=\"891\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;z&quot;</text>\n",
       "</g>\n",
       "<!-- 37&#45;&#45;38 -->\n",
       "<g id=\"edge38\" class=\"edge\">\n",
       "<title>37--38</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M956,-71C956,-71 930.93,-50.94 911.92,-35.74\"/>\n",
       "</g>\n",
       "<!-- 39 -->\n",
       "<g id=\"node40\" class=\"node\">\n",
       "<title>39</title>\n",
       "<text text-anchor=\"middle\" x=\"963\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 37&#45;&#45;39 -->\n",
       "<g id=\"edge39\" class=\"edge\">\n",
       "<title>37--39</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M956,-71C956,-71 958.7,-50.94 960.75,-35.74\"/>\n",
       "</g>\n",
       "<!-- 41 -->\n",
       "<g id=\"node42\" class=\"node\">\n",
       "<title>41</title>\n",
       "<text text-anchor=\"start\" x=\"1018.5\" y=\"-85.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 40&#45;&#45;41 -->\n",
       "<g id=\"edge41\" class=\"edge\">\n",
       "<title>40--41</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1013.78,-144.05C1018.18,-133.05 1023.89,-118.76 1028.28,-107.79\"/>\n",
       "</g>\n",
       "<!-- 42 -->\n",
       "<g id=\"node43\" class=\"node\">\n",
       "<title>42</title>\n",
       "<text text-anchor=\"middle\" x=\"1035\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;y&quot;</text>\n",
       "</g>\n",
       "<!-- 41&#45;&#45;42 -->\n",
       "<g id=\"edge42\" class=\"edge\">\n",
       "<title>41--42</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1042,-71C1042,-71 1039.3,-50.94 1037.25,-35.74\"/>\n",
       "</g>\n",
       "<!-- 43 -->\n",
       "<g id=\"node44\" class=\"node\">\n",
       "<title>43</title>\n",
       "<text text-anchor=\"middle\" x=\"1107\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 41&#45;&#45;43 -->\n",
       "<g id=\"edge43\" class=\"edge\">\n",
       "<title>41--43</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1042,-71C1042,-71 1067.07,-50.94 1086.08,-35.74\"/>\n",
       "</g>\n",
       "<!-- 45 -->\n",
       "<g id=\"node46\" class=\"node\">\n",
       "<title>45</title>\n",
       "<text text-anchor=\"start\" x=\"1236.12\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Compare</text>\n",
       "</g>\n",
       "<!-- 44&#45;&#45;45 -->\n",
       "<g id=\"edge45\" class=\"edge\">\n",
       "<title>44--45</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1332,-287C1332,-287 1306.15,-266.94 1286.57,-251.74\"/>\n",
       "</g>\n",
       "<!-- 53 -->\n",
       "<g id=\"node54\" class=\"node\">\n",
       "<title>53</title>\n",
       "<text text-anchor=\"start\" x=\"1356.25\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Return</text>\n",
       "</g>\n",
       "<!-- 44&#45;&#45;53 -->\n",
       "<g id=\"edge53\" class=\"edge\">\n",
       "<title>44--53</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1332,-287C1332,-287 1350.9,-266.94 1365.23,-251.74\"/>\n",
       "</g>\n",
       "<!-- 57 -->\n",
       "<g id=\"node58\" class=\"node\">\n",
       "<title>57</title>\n",
       "<text text-anchor=\"start\" x=\"1533.75\" y=\"-229.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">If</text>\n",
       "</g>\n",
       "<!-- 44&#45;&#45;57 -->\n",
       "<g id=\"edge57\" class=\"edge\">\n",
       "<title>44--57</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1332,-287C1332,-287 1458.9,-255.58 1515.4,-241.59\"/>\n",
       "</g>\n",
       "<!-- 46 -->\n",
       "<g id=\"node47\" class=\"node\">\n",
       "<title>46</title>\n",
       "<text text-anchor=\"start\" x=\"1162.5\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 45&#45;&#45;46 -->\n",
       "<g id=\"edge46\" class=\"edge\">\n",
       "<title>45--46</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1263,-215C1263,-215 1230.19,-194.69 1205.58,-179.45\"/>\n",
       "</g>\n",
       "<!-- 49 -->\n",
       "<g id=\"node50\" class=\"node\">\n",
       "<title>49</title>\n",
       "<text text-anchor=\"middle\" x=\"1251\" y=\"-156.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Gt</text>\n",
       "</g>\n",
       "<!-- 45&#45;&#45;49 -->\n",
       "<g id=\"edge49\" class=\"edge\">\n",
       "<title>45--49</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1263,-215C1263,-215 1258.37,-194.94 1254.86,-179.74\"/>\n",
       "</g>\n",
       "<!-- 50 -->\n",
       "<g id=\"node51\" class=\"node\">\n",
       "<title>50</title>\n",
       "<text text-anchor=\"start\" x=\"1306.5\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 45&#45;&#45;50 -->\n",
       "<g id=\"edge50\" class=\"edge\">\n",
       "<title>45--50</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1263,-215C1263,-215 1286.15,-194.94 1303.69,-179.74\"/>\n",
       "</g>\n",
       "<!-- 47 -->\n",
       "<g id=\"node48\" class=\"node\">\n",
       "<title>47</title>\n",
       "<text text-anchor=\"middle\" x=\"1107\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;x&quot;</text>\n",
       "</g>\n",
       "<!-- 46&#45;&#45;47 -->\n",
       "<g id=\"edge47\" class=\"edge\">\n",
       "<title>46--47</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1172,-143C1172,-143 1146.93,-122.94 1127.92,-107.74\"/>\n",
       "</g>\n",
       "<!-- 48 -->\n",
       "<g id=\"node49\" class=\"node\">\n",
       "<title>48</title>\n",
       "<text text-anchor=\"middle\" x=\"1179\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 46&#45;&#45;48 -->\n",
       "<g id=\"edge48\" class=\"edge\">\n",
       "<title>46--48</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1172,-143C1172,-143 1174.7,-122.94 1176.75,-107.74\"/>\n",
       "</g>\n",
       "<!-- 51 -->\n",
       "<g id=\"node52\" class=\"node\">\n",
       "<title>51</title>\n",
       "<text text-anchor=\"middle\" x=\"1251\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;y&quot;</text>\n",
       "</g>\n",
       "<!-- 50&#45;&#45;51 -->\n",
       "<g id=\"edge51\" class=\"edge\">\n",
       "<title>50--51</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1316,-143C1316,-143 1290.93,-122.94 1271.92,-107.74\"/>\n",
       "</g>\n",
       "<!-- 52 -->\n",
       "<g id=\"node53\" class=\"node\">\n",
       "<title>52</title>\n",
       "<text text-anchor=\"middle\" x=\"1323\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 50&#45;&#45;52 -->\n",
       "<g id=\"edge52\" class=\"edge\">\n",
       "<title>50--52</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1316,-143C1316,-143 1318.7,-122.94 1320.75,-107.74\"/>\n",
       "</g>\n",
       "<!-- 54 -->\n",
       "<g id=\"node55\" class=\"node\">\n",
       "<title>54</title>\n",
       "<text text-anchor=\"start\" x=\"1378.5\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 53&#45;&#45;54 -->\n",
       "<g id=\"edge54\" class=\"edge\">\n",
       "<title>53--54</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1384.39,-216.05C1386.59,-205.05 1389.45,-190.76 1391.64,-179.79\"/>\n",
       "</g>\n",
       "<!-- 55 -->\n",
       "<g id=\"node56\" class=\"node\">\n",
       "<title>55</title>\n",
       "<text text-anchor=\"middle\" x=\"1395\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;y&quot;</text>\n",
       "</g>\n",
       "<!-- 54&#45;&#45;55 -->\n",
       "<g id=\"edge55\" class=\"edge\">\n",
       "<title>54--55</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1402,-143C1402,-143 1399.3,-122.94 1397.25,-107.74\"/>\n",
       "</g>\n",
       "<!-- 56 -->\n",
       "<g id=\"node57\" class=\"node\">\n",
       "<title>56</title>\n",
       "<text text-anchor=\"middle\" x=\"1467\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 54&#45;&#45;56 -->\n",
       "<g id=\"edge56\" class=\"edge\">\n",
       "<title>54--56</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1402,-143C1402,-143 1427.07,-122.94 1446.08,-107.74\"/>\n",
       "</g>\n",
       "<!-- 58 -->\n",
       "<g id=\"node59\" class=\"node\">\n",
       "<title>58</title>\n",
       "<text text-anchor=\"start\" x=\"1548.12\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Compare</text>\n",
       "</g>\n",
       "<!-- 57&#45;&#45;58 -->\n",
       "<g id=\"edge58\" class=\"edge\">\n",
       "<title>57--58</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1561,-215C1561,-215 1567.17,-194.94 1571.85,-179.74\"/>\n",
       "</g>\n",
       "<!-- 66 -->\n",
       "<g id=\"node67\" class=\"node\">\n",
       "<title>66</title>\n",
       "<text text-anchor=\"start\" x=\"1685.25\" y=\"-157.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Return</text>\n",
       "</g>\n",
       "<!-- 57&#45;&#45;66 -->\n",
       "<g id=\"edge66\" class=\"edge\">\n",
       "<title>57--66</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1561,-215C1561,-215 1633.44,-189.72 1677.3,-174.41\"/>\n",
       "</g>\n",
       "<!-- 59 -->\n",
       "<g id=\"node60\" class=\"node\">\n",
       "<title>59</title>\n",
       "<text text-anchor=\"start\" x=\"1522.5\" y=\"-85.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 58&#45;&#45;59 -->\n",
       "<g id=\"edge59\" class=\"edge\">\n",
       "<title>58--59</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1583,-143C1583,-143 1566.03,-122.94 1553.16,-107.74\"/>\n",
       "</g>\n",
       "<!-- 62 -->\n",
       "<g id=\"node63\" class=\"node\">\n",
       "<title>62</title>\n",
       "<text text-anchor=\"middle\" x=\"1611\" y=\"-84.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Gt</text>\n",
       "</g>\n",
       "<!-- 58&#45;&#45;62 -->\n",
       "<g id=\"edge62\" class=\"edge\">\n",
       "<title>58--62</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1583,-143C1583,-143 1593.8,-122.94 1601.99,-107.74\"/>\n",
       "</g>\n",
       "<!-- 63 -->\n",
       "<g id=\"node64\" class=\"node\">\n",
       "<title>63</title>\n",
       "<text text-anchor=\"start\" x=\"1666.5\" y=\"-85.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 58&#45;&#45;63 -->\n",
       "<g id=\"edge63\" class=\"edge\">\n",
       "<title>58--63</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1583,-143C1583,-143 1626.58,-120.34 1656.2,-104.94\"/>\n",
       "</g>\n",
       "<!-- 60 -->\n",
       "<g id=\"node61\" class=\"node\">\n",
       "<title>60</title>\n",
       "<text text-anchor=\"middle\" x=\"1467\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;x&quot;</text>\n",
       "</g>\n",
       "<!-- 59&#45;&#45;60 -->\n",
       "<g id=\"edge60\" class=\"edge\">\n",
       "<title>59--60</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1532,-71C1532,-71 1506.93,-50.94 1487.92,-35.74\"/>\n",
       "</g>\n",
       "<!-- 61 -->\n",
       "<g id=\"node62\" class=\"node\">\n",
       "<title>61</title>\n",
       "<text text-anchor=\"middle\" x=\"1539\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 59&#45;&#45;61 -->\n",
       "<g id=\"edge61\" class=\"edge\">\n",
       "<title>59--61</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1532,-71C1532,-71 1534.7,-50.94 1536.75,-35.74\"/>\n",
       "</g>\n",
       "<!-- 64 -->\n",
       "<g id=\"node65\" class=\"node\">\n",
       "<title>64</title>\n",
       "<text text-anchor=\"middle\" x=\"1611\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;z&quot;</text>\n",
       "</g>\n",
       "<!-- 63&#45;&#45;64 -->\n",
       "<g id=\"edge64\" class=\"edge\">\n",
       "<title>63--64</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1676,-71C1676,-71 1650.93,-50.94 1631.92,-35.74\"/>\n",
       "</g>\n",
       "<!-- 65 -->\n",
       "<g id=\"node66\" class=\"node\">\n",
       "<title>65</title>\n",
       "<text text-anchor=\"middle\" x=\"1683\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 63&#45;&#45;65 -->\n",
       "<g id=\"edge65\" class=\"edge\">\n",
       "<title>63--65</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1676,-71C1676,-71 1678.7,-50.94 1680.75,-35.74\"/>\n",
       "</g>\n",
       "<!-- 67 -->\n",
       "<g id=\"node68\" class=\"node\">\n",
       "<title>67</title>\n",
       "<text text-anchor=\"start\" x=\"1738.5\" y=\"-85.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 66&#45;&#45;67 -->\n",
       "<g id=\"edge67\" class=\"edge\">\n",
       "<title>66--67</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1720.89,-144.05C1727.97,-133.05 1737.15,-118.76 1744.21,-107.79\"/>\n",
       "</g>\n",
       "<!-- 68 -->\n",
       "<g id=\"node69\" class=\"node\">\n",
       "<title>68</title>\n",
       "<text text-anchor=\"middle\" x=\"1755\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;x&quot;</text>\n",
       "</g>\n",
       "<!-- 67&#45;&#45;68 -->\n",
       "<g id=\"edge68\" class=\"edge\">\n",
       "<title>67--68</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1762,-71C1762,-71 1759.3,-50.94 1757.25,-35.74\"/>\n",
       "</g>\n",
       "<!-- 69 -->\n",
       "<g id=\"node70\" class=\"node\">\n",
       "<title>69</title>\n",
       "<text text-anchor=\"middle\" x=\"1827\" y=\"-12.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 67&#45;&#45;69 -->\n",
       "<g id=\"edge69\" class=\"edge\">\n",
       "<title>67--69</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1762,-71C1762,-71 1787.07,-50.94 1806.08,-35.74\"/>\n",
       "</g>\n",
       "<!-- 71 -->\n",
       "<g id=\"node72\" class=\"node\">\n",
       "<title>71</title>\n",
       "<text text-anchor=\"start\" x=\"1597.5\" y=\"-301.95\" font-family=\"Courier,monospace\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#004080\">Name</text>\n",
       "</g>\n",
       "<!-- 70&#45;&#45;71 -->\n",
       "<g id=\"edge71\" class=\"edge\">\n",
       "<title>70--71</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1186.71,-372.02C1273.33,-358.84 1507.44,-323.22 1587.06,-311.1\"/>\n",
       "</g>\n",
       "<!-- 72 -->\n",
       "<g id=\"node73\" class=\"node\">\n",
       "<title>72</title>\n",
       "<text text-anchor=\"middle\" x=\"1614\" y=\"-228.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">&quot;z&quot;</text>\n",
       "</g>\n",
       "<!-- 71&#45;&#45;72 -->\n",
       "<g id=\"edge72\" class=\"edge\">\n",
       "<title>71--72</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1621,-287C1621,-287 1618.3,-266.94 1616.25,-251.74\"/>\n",
       "</g>\n",
       "<!-- 73 -->\n",
       "<g id=\"node74\" class=\"node\">\n",
       "<title>73</title>\n",
       "<text text-anchor=\"middle\" x=\"1686\" y=\"-228.95\" font-family=\"Courier,monospace\" font-size=\"14.00\" fill=\"#008040\">Load</text>\n",
       "</g>\n",
       "<!-- 71&#45;&#45;73 -->\n",
       "<g id=\"edge73\" class=\"edge\">\n",
       "<title>71--73</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1621,-287C1621,-287 1646.07,-266.94 1665.08,-251.74\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_ast(middle_tree())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    " You see that it consists of one function definition (`FunctionDef`) with three `arguments` and two statements – one `If` and one `Return`. Each `If` subtree has three branches – one for the condition (`test`), one for the body to be executed if the condition is true (`body`), and one for the `else` case (`orelse`). The `body` and `orelse` branches again are lists of statements."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "An AST can also be shown as text, which is more compact, yet reveals more information. `ast.dump()` gives not only the class names of elements, but also how they are constructed – actually, the whole expression can be used to construct an AST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.568651Z",
     "iopub.status.busy": "2023-11-12T12:47:30.568529Z",
     "iopub.status.idle": "2023-11-12T12:47:30.570905Z",
     "shell.execute_reply": "2023-11-12T12:47:30.570648Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Module(body=[FunctionDef(name='middle', args=arguments(posonlyargs=[], args=[arg(arg='x'), arg(arg='y'), arg(arg='z')], kwonlyargs=[], kw_defaults=[], defaults=[]), body=[If(test=Compare(left=Name(id='y', ctx=Load()), ops=[Lt()], comparators=[Name(id='z', ctx=Load())]), body=[If(test=Compare(left=Name(id='x', ctx=Load()), ops=[Lt()], comparators=[Name(id='y', ctx=Load())]), body=[Return(value=Name(id='y', ctx=Load()))], orelse=[If(test=Compare(left=Name(id='x', ctx=Load()), ops=[Lt()], comparators=[Name(id='z', ctx=Load())]), body=[Return(value=Name(id='y', ctx=Load()))], orelse=[])])], orelse=[If(test=Compare(left=Name(id='x', ctx=Load()), ops=[Gt()], comparators=[Name(id='y', ctx=Load())]), body=[Return(value=Name(id='y', ctx=Load()))], orelse=[If(test=Compare(left=Name(id='x', ctx=Load()), ops=[Gt()], comparators=[Name(id='z', ctx=Load())]), body=[Return(value=Name(id='x', ctx=Load()))], orelse=[])])]), Return(value=Name(id='z', ctx=Load()))], decorator_list=[])], type_ignores=[])\n"
     ]
    }
   ],
   "source": [
    "print(ast.dump(middle_tree()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This is the path to the first `return` statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.572373Z",
     "iopub.status.busy": "2023-11-12T12:47:30.572262Z",
     "iopub.status.idle": "2023-11-12T12:47:30.574629Z",
     "shell.execute_reply": "2023-11-12T12:47:30.574362Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Return(value=Name(id='y', ctx=Load()))\""
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ast.dump(middle_tree().body[0].body[0].body[0].body[0])  # type: ignore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Picking Statements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For our mutation operators, we want to use statements from the program itself. Hence, we need a means to find those very statements. The `StatementVisitor` class iterates through an AST, adding all statements it finds in function definitions to its `statements` list. To do so, it subclasses the Python `ast` `NodeVisitor` class, described in the [official Python `ast` reference](http://docs.python.org/3/library/ast)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.576180Z",
     "iopub.status.busy": "2023-11-12T12:47:30.576065Z",
     "iopub.status.idle": "2023-11-12T12:47:30.577736Z",
     "shell.execute_reply": "2023-11-12T12:47:30.577457Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from ast import NodeVisitor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.579297Z",
     "iopub.status.busy": "2023-11-12T12:47:30.579169Z",
     "iopub.status.idle": "2023-11-12T12:47:30.580923Z",
     "shell.execute_reply": "2023-11-12T12:47:30.580673Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# ignore\n",
    "from typing import Any, Callable, Optional, Type, Tuple\n",
    "from typing import Dict, Union, Set, List, cast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.582516Z",
     "iopub.status.busy": "2023-11-12T12:47:30.582398Z",
     "iopub.status.idle": "2023-11-12T12:47:30.587038Z",
     "shell.execute_reply": "2023-11-12T12:47:30.586717Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementVisitor(NodeVisitor):\n",
    "    \"\"\"Visit all statements within function defs in an AST\"\"\"\n",
    "\n",
    "    def __init__(self) -> None:\n",
    "        self.statements: List[Tuple[ast.AST, str]] = []\n",
    "        self.func_name = \"\"\n",
    "        self.statements_seen: Set[Tuple[ast.AST, str]] = set()\n",
    "        super().__init__()\n",
    "\n",
    "    def add_statements(self, node: ast.AST, attr: str) -> None:\n",
    "        elems: List[ast.AST] = getattr(node, attr, [])\n",
    "        if not isinstance(elems, list):\n",
    "            elems = [elems]  # type: ignore\n",
    "\n",
    "        for elem in elems:\n",
    "            stmt = (elem, self.func_name)\n",
    "            if stmt in self.statements_seen:\n",
    "                continue\n",
    "\n",
    "            self.statements.append(stmt)\n",
    "            self.statements_seen.add(stmt)\n",
    "\n",
    "    def visit_node(self, node: ast.AST) -> None:\n",
    "        # Any node other than the ones listed below\n",
    "        self.add_statements(node, 'body')\n",
    "        self.add_statements(node, 'orelse')\n",
    "\n",
    "    def visit_Module(self, node: ast.Module) -> None:\n",
    "        # Module children are defs, classes and globals - don't add\n",
    "        super().generic_visit(node)\n",
    "\n",
    "    def visit_ClassDef(self, node: ast.ClassDef) -> None:\n",
    "        # Class children are defs and globals - don't add\n",
    "        super().generic_visit(node)\n",
    "\n",
    "    def generic_visit(self, node: ast.AST) -> None:\n",
    "        self.visit_node(node)\n",
    "        super().generic_visit(node)\n",
    "\n",
    "    def visit_FunctionDef(self,\n",
    "                          node: Union[ast.FunctionDef, ast.AsyncFunctionDef]) -> None:\n",
    "        if not self.func_name:\n",
    "            self.func_name = node.name\n",
    "\n",
    "        self.visit_node(node)\n",
    "        super().generic_visit(node)\n",
    "        self.func_name = \"\"\n",
    "\n",
    "    def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> None:\n",
    "        return self.visit_FunctionDef(node)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The function `all_statements()` returns all statements in the given AST `tree`. If an `ast` class `tp` is given, it only returns instances of that class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.588823Z",
     "iopub.status.busy": "2023-11-12T12:47:30.588697Z",
     "iopub.status.idle": "2023-11-12T12:47:30.591207Z",
     "shell.execute_reply": "2023-11-12T12:47:30.590883Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def all_statements_and_functions(tree: ast.AST, \n",
    "                                 tp: Optional[Type] = None) -> \\\n",
    "                                 List[Tuple[ast.AST, str]]:\n",
    "    \"\"\"\n",
    "    Return a list of pairs (`statement`, `function`) for all statements in `tree`.\n",
    "    If `tp` is given, return only statements of that class.\n",
    "    \"\"\"\n",
    "\n",
    "    visitor = StatementVisitor()\n",
    "    visitor.visit(tree)\n",
    "    statements = visitor.statements\n",
    "    if tp is not None:\n",
    "        statements = [s for s in statements if isinstance(s[0], tp)]\n",
    "\n",
    "    return statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.592801Z",
     "iopub.status.busy": "2023-11-12T12:47:30.592682Z",
     "iopub.status.idle": "2023-11-12T12:47:30.594732Z",
     "shell.execute_reply": "2023-11-12T12:47:30.594422Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def all_statements(tree: ast.AST, tp: Optional[Type] = None) -> List[ast.AST]:\n",
    "    \"\"\"\n",
    "    Return a list of all statements in `tree`.\n",
    "    If `tp` is given, return only statements of that class.\n",
    "    \"\"\"\n",
    "\n",
    "    return [stmt for stmt, func_name in all_statements_and_functions(tree, tp)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here are all the `return` statements in `middle()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.596426Z",
     "iopub.status.busy": "2023-11-12T12:47:30.596297Z",
     "iopub.status.idle": "2023-11-12T12:47:30.598902Z",
     "shell.execute_reply": "2023-11-12T12:47:30.598542Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<ast.Return at 0x14eb0a9b0>,\n",
       " <ast.Return at 0x14eb09c30>,\n",
       " <ast.Return at 0x14eb0b280>,\n",
       " <ast.Return at 0x14eb09bd0>,\n",
       " <ast.Return at 0x14eb0ab00>]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_statements(middle_tree(), ast.Return)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.600615Z",
     "iopub.status.busy": "2023-11-12T12:47:30.600470Z",
     "iopub.status.idle": "2023-11-12T12:47:30.603238Z",
     "shell.execute_reply": "2023-11-12T12:47:30.602959Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(<ast.If at 0x14eb0ba30>, 'middle'),\n",
       " (<ast.If at 0x14eb0baf0>, 'middle'),\n",
       " (<ast.If at 0x14eb0a770>, 'middle'),\n",
       " (<ast.If at 0x14eb0a5f0>, 'middle'),\n",
       " (<ast.If at 0x14eb0b6d0>, 'middle')]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_statements_and_functions(middle_tree(), ast.If)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can randomly pick an element:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.604839Z",
     "iopub.status.busy": "2023-11-12T12:47:30.604719Z",
     "iopub.status.idle": "2023-11-12T12:47:30.606392Z",
     "shell.execute_reply": "2023-11-12T12:47:30.606132Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.607989Z",
     "iopub.status.busy": "2023-11-12T12:47:30.607876Z",
     "iopub.status.idle": "2023-11-12T12:47:30.610564Z",
     "shell.execute_reply": "2023-11-12T12:47:30.610312Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'return y'"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_node = random.choice(all_statements(middle_tree()))\n",
    "ast.unparse(random_node)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Mutating Statements\n",
    "\n",
    "The main part in mutation, however, is to actually mutate the code of the program under test. To this end, we introduce a `StatementMutator` class – a subclass of `NodeTransformer`, described in the [official Python `ast` reference](http://docs.python.org/3/library/ast)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The constructor provides various keyword arguments to configure the mutator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.612144Z",
     "iopub.status.busy": "2023-11-12T12:47:30.612006Z",
     "iopub.status.idle": "2023-11-12T12:47:30.613660Z",
     "shell.execute_reply": "2023-11-12T12:47:30.613424Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from ast import NodeTransformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.615172Z",
     "iopub.status.busy": "2023-11-12T12:47:30.615038Z",
     "iopub.status.idle": "2023-11-12T12:47:30.616756Z",
     "shell.execute_reply": "2023-11-12T12:47:30.616494Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.618256Z",
     "iopub.status.busy": "2023-11-12T12:47:30.618142Z",
     "iopub.status.idle": "2023-11-12T12:47:30.621399Z",
     "shell.execute_reply": "2023-11-12T12:47:30.621124Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(NodeTransformer):\n",
    "    \"\"\"Mutate statements in an AST for automated repair.\"\"\"\n",
    "\n",
    "    def __init__(self,\n",
    "                 suspiciousness_func:\n",
    "                     Optional[Callable[[Tuple[Callable, int]], float]] = None,\n",
    "                 source: Optional[List[ast.AST]] = None,\n",
    "                 log: Union[bool, int] = False) -> None:\n",
    "        \"\"\"\n",
    "        Constructor.\n",
    "        `suspiciousness_func` is a function that takes a location\n",
    "        (function, line_number) and returns a suspiciousness value\n",
    "        between 0 and 1.0. If not given, all locations get the same \n",
    "        suspiciousness of 1.0.\n",
    "        `source` is a list of statements to choose from.\n",
    "        \"\"\"\n",
    "\n",
    "        super().__init__()\n",
    "        self.log = log\n",
    "\n",
    "        if suspiciousness_func is None:\n",
    "            def suspiciousness_func(location: Tuple[Callable, int]) -> float:\n",
    "                return 1.0\n",
    "        assert suspiciousness_func is not None\n",
    "\n",
    "        self.suspiciousness_func: Callable = suspiciousness_func\n",
    "\n",
    "        if source is None:\n",
    "            source = []\n",
    "        self.source = source\n",
    "\n",
    "        if self.log > 1:\n",
    "            for i, node in enumerate(self.source):\n",
    "                print(f\"Source for repairs #{i}:\")\n",
    "                print_content(ast.unparse(node), '.py')\n",
    "                print()\n",
    "                print()\n",
    "\n",
    "        self.mutations = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Choosing Suspicious Statements to Mutate\n",
    "\n",
    "We start with deciding which AST nodes to mutate. The method `node_suspiciousness()` returns the suspiciousness for a given node, by invoking the suspiciousness function `suspiciousness_func` given during initialization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.622796Z",
     "iopub.status.busy": "2023-11-12T12:47:30.622709Z",
     "iopub.status.idle": "2023-11-12T12:47:30.624202Z",
     "shell.execute_reply": "2023-11-12T12:47:30.623944Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import warnings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.625483Z",
     "iopub.status.busy": "2023-11-12T12:47:30.625401Z",
     "iopub.status.idle": "2023-11-12T12:47:30.627736Z",
     "shell.execute_reply": "2023-11-12T12:47:30.627406Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def node_suspiciousness(self, stmt: ast.AST, func_name: str) -> float:\n",
    "        if not hasattr(stmt, 'lineno'):\n",
    "            warnings.warn(f\"{self.format_node(stmt)}: Expected line number\")\n",
    "            return 0.0\n",
    "\n",
    "        suspiciousness = self.suspiciousness_func((func_name, stmt.lineno))\n",
    "        if suspiciousness is None:  # not executed\n",
    "            return 0.0\n",
    "\n",
    "        return suspiciousness\n",
    "\n",
    "    def format_node(self, node: ast.AST) -> str:  # type: ignore\n",
    "        ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The method `node_to_be_mutated()` picks a node (statement) to be mutated. It determines the suspiciousness of all statements, and invokes `random.choices()`, using the suspiciousness as weight. Unsuspicious statements (with zero weight) will not be chosen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.629235Z",
     "iopub.status.busy": "2023-11-12T12:47:30.629139Z",
     "iopub.status.idle": "2023-11-12T12:47:30.632249Z",
     "shell.execute_reply": "2023-11-12T12:47:30.631848Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def node_to_be_mutated(self, tree: ast.AST) -> ast.AST:\n",
    "        statements = all_statements_and_functions(tree)\n",
    "        assert len(statements) > 0, \"No statements\"\n",
    "\n",
    "        weights = [self.node_suspiciousness(stmt, func_name) \n",
    "                   for stmt, func_name in statements]\n",
    "        stmts = [stmt for stmt, func_name in statements]\n",
    "\n",
    "        if self.log > 1:\n",
    "            print(\"Weights:\")\n",
    "            for i, stmt in enumerate(statements):\n",
    "                node, func_name = stmt\n",
    "                print(f\"{weights[i]:.2} {self.format_node(node)}\")\n",
    "\n",
    "        if sum(weights) == 0.0:\n",
    "            # No suspicious line\n",
    "            return random.choice(stmts)\n",
    "        else:\n",
    "            return random.choices(stmts, weights=weights)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Choosing a Mutation Method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The method `visit()` is invoked on all nodes. For nodes marked with a `mutate_me` attribute, it randomly chooses a mutation method (`choose_op()`) and then invokes it on the node.\n",
    "\n",
    "According to the rules of `NodeTransformer`, the mutation method can return\n",
    "\n",
    "* a new node or a list of nodes, replacing the current node;\n",
    "* `None`, deleting it; or\n",
    "* the node itself, keeping things as they are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.634072Z",
     "iopub.status.busy": "2023-11-12T12:47:30.633931Z",
     "iopub.status.idle": "2023-11-12T12:47:30.635625Z",
     "shell.execute_reply": "2023-11-12T12:47:30.635362Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.637126Z",
     "iopub.status.busy": "2023-11-12T12:47:30.636992Z",
     "iopub.status.idle": "2023-11-12T12:47:30.638721Z",
     "shell.execute_reply": "2023-11-12T12:47:30.638477Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "RE_SPACE = re.compile(r'[ \\t\\n]+')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.640246Z",
     "iopub.status.busy": "2023-11-12T12:47:30.640141Z",
     "iopub.status.idle": "2023-11-12T12:47:30.642575Z",
     "shell.execute_reply": "2023-11-12T12:47:30.642315Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def choose_op(self) -> Callable:\n",
    "        return random.choice([self.insert, self.swap, self.delete])\n",
    "\n",
    "    def visit(self, node: ast.AST) -> ast.AST:\n",
    "        super().visit(node)  # Visits (and transforms?) children\n",
    "\n",
    "        if not node.mutate_me:  # type: ignore\n",
    "            return node\n",
    "\n",
    "        op = self.choose_op()\n",
    "        new_node = op(node)\n",
    "        self.mutations += 1\n",
    "\n",
    "        if self.log:\n",
    "            print(f\"{node.lineno:4}:{op.__name__ + ':':7} \"\n",
    "                  f\"{self.format_node(node)} \"\n",
    "                  f\"becomes {self.format_node(new_node)}\")\n",
    "\n",
    "        return new_node"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Swapping Statements\n",
    "\n",
    "Our first mutator is `swap()`, which replaces the current node `NODE` by a random node found in `source` (using a newly defined `choose_statement()`).\n",
    "\n",
    "As a rule of thumb, we try to avoid inserting entire subtrees with all attached statements; and try to respect only the first line of a node. If the new node has the form \n",
    "\n",
    "```python\n",
    "if P:\n",
    "    BODY\n",
    "```\n",
    "\n",
    "we thus only insert \n",
    "\n",
    "```python\n",
    "if P: \n",
    "    pass\n",
    "```\n",
    "\n",
    "since the statements in `BODY` have a later chance to get inserted. The same holds for all constructs that have a `BODY`, i.e. `while`, `for`, `try`, `with`, and more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.644056Z",
     "iopub.status.busy": "2023-11-12T12:47:30.643943Z",
     "iopub.status.idle": "2023-11-12T12:47:30.645704Z",
     "shell.execute_reply": "2023-11-12T12:47:30.645486Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def choose_statement(self) -> ast.AST:\n",
    "        return copy.deepcopy(random.choice(self.source))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.647089Z",
     "iopub.status.busy": "2023-11-12T12:47:30.646982Z",
     "iopub.status.idle": "2023-11-12T12:47:30.649511Z",
     "shell.execute_reply": "2023-11-12T12:47:30.649235Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def swap(self, node: ast.AST) -> ast.AST:\n",
    "        \"\"\"Replace `node` with a random node from `source`\"\"\"\n",
    "        new_node = self.choose_statement()\n",
    "\n",
    "        if isinstance(new_node, ast.stmt):\n",
    "            # The source `if P: X` is added as `if P: pass`\n",
    "            if hasattr(new_node, 'body'):\n",
    "                new_node.body = [ast.Pass()]  # type: ignore\n",
    "            if hasattr(new_node, 'orelse'):\n",
    "                new_node.orelse = []  # type: ignore\n",
    "            if hasattr(new_node, 'finalbody'):\n",
    "                new_node.finalbody = []  # type: ignore\n",
    "\n",
    "        # ast.copy_location(new_node, node)\n",
    "        return new_node"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Inserting Statements\n",
    "\n",
    "Our next mutator is `insert()`, which randomly chooses some node from `source` and inserts it after the current node `NODE`. (If `NODE` is a `return` statement, then we insert the new node _before_ `NODE`.)\n",
    "\n",
    "If the statement to be inserted has the form\n",
    "\n",
    "```python\n",
    "if P:\n",
    "    BODY\n",
    "```\n",
    "\n",
    "we only insert the \"header\" of the `if`, resulting in\n",
    "\n",
    "```python\n",
    "if P: \n",
    "    NODE\n",
    "```\n",
    "\n",
    "Again, this applies to all constructs that have a `BODY`, i.e., `while`, `for`, `try`, `with`, and more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.651067Z",
     "iopub.status.busy": "2023-11-12T12:47:30.650958Z",
     "iopub.status.idle": "2023-11-12T12:47:30.653755Z",
     "shell.execute_reply": "2023-11-12T12:47:30.653501Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def insert(self, node: ast.AST) -> Union[ast.AST, List[ast.AST]]:\n",
    "        \"\"\"Insert a random node from `source` after `node`\"\"\"\n",
    "        new_node = self.choose_statement()\n",
    "\n",
    "        if isinstance(new_node, ast.stmt) and hasattr(new_node, 'body'):\n",
    "            # Inserting `if P: X` as `if P:`\n",
    "            new_node.body = [node]  # type: ignore\n",
    "            if hasattr(new_node, 'orelse'):\n",
    "                new_node.orelse = []  # type: ignore\n",
    "            if hasattr(new_node, 'finalbody'):\n",
    "                new_node.finalbody = []  # type: ignore\n",
    "            # ast.copy_location(new_node, node)\n",
    "            return new_node\n",
    "\n",
    "        # Only insert before `return`, not after it\n",
    "        if isinstance(node, ast.Return):\n",
    "            if isinstance(new_node, ast.Return):\n",
    "                return new_node\n",
    "            else:\n",
    "                return [new_node, node]\n",
    "\n",
    "        return [node, new_node]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Deleting Statements\n",
    "\n",
    "Our last mutator is `delete()`, which deletes the current node `NODE`. The standard case is to replace `NODE` by a `pass` statement.\n",
    "\n",
    "If the statement to be deleted has the form\n",
    "\n",
    "```python\n",
    "if P:\n",
    "    BODY\n",
    "```\n",
    "\n",
    "we only delete the \"header\" of the `if`, resulting in\n",
    "\n",
    "```python\n",
    "BODY\n",
    "```\n",
    "\n",
    "Again, this applies to all constructs that have a `BODY`, i.e., `while`, `for`, `try`, `with`, and more. If the statement to be deleted has multiple branches, a random branch is chosen (e.g., the `else` branch of an `if` statement)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.655451Z",
     "iopub.status.busy": "2023-11-12T12:47:30.655338Z",
     "iopub.status.idle": "2023-11-12T12:47:30.657848Z",
     "shell.execute_reply": "2023-11-12T12:47:30.657555Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def delete(self, node: ast.AST) -> None:\n",
    "        \"\"\"Delete `node`.\"\"\"\n",
    "\n",
    "        branches = [attr for attr in ['body', 'orelse', 'finalbody']\n",
    "                    if hasattr(node, attr) and getattr(node, attr)]\n",
    "        if branches:\n",
    "            # Replace `if P: S` by `S`\n",
    "            branch = random.choice(branches)\n",
    "            new_node = getattr(node, branch)\n",
    "            return new_node\n",
    "\n",
    "        if isinstance(node, ast.stmt):\n",
    "            # Avoid empty bodies; make this a `pass` statement\n",
    "            new_node = ast.Pass()\n",
    "            ast.copy_location(new_node, node)\n",
    "            return new_node\n",
    "\n",
    "        return None  # Just delete"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.659310Z",
     "iopub.status.busy": "2023-11-12T12:47:30.659205Z",
     "iopub.status.idle": "2023-11-12T12:47:30.660801Z",
     "shell.execute_reply": "2023-11-12T12:47:30.660564Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from bookutils import quiz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.662150Z",
     "iopub.status.busy": "2023-11-12T12:47:30.662068Z",
     "iopub.status.idle": "2023-11-12T12:47:30.666886Z",
     "shell.execute_reply": "2023-11-12T12:47:30.666637Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    \n",
       "    <script>\n",
       "    var bad_answers = new Map();\n",
       "\n",
       "    function answer(quiz_id) {\n",
       "        ans = 0;\n",
       "        for (i = 1;; i++) {\n",
       "            checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "            if (!checkbox)\n",
       "                break;\n",
       "            if (checkbox.checked)\n",
       "                ans |= (1 << i);\n",
       "        }\n",
       "        return ans;\n",
       "    }\n",
       "    function check_selection(quiz_id, correct_answer, multiple_choice, hint) {\n",
       "        given_answer = answer(quiz_id);\n",
       "        if (given_answer == correct_answer)\n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Correct!\";\n",
       "            document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "\n",
       "            for (i = 1;; i++) {\n",
       "                checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "                if (!checkbox)\n",
       "                    break;\n",
       "    \n",
       "                if (checkbox.checked) {\n",
       "                    label.style.fontWeight = \"bold\";\n",
       "                }\n",
       "                else {\n",
       "                    label.style.textDecoration = \"line-through\";\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "        else \n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Try again\";\n",
       "            \n",
       "            if (!bad_answers.has(quiz_id)) {\n",
       "                bad_answers.set(quiz_id, 1);\n",
       "            }\n",
       "            else {\n",
       "                bad_answers.set(quiz_id, bad_answers.get(quiz_id) + 1);\n",
       "            }\n",
       "\n",
       "            if (bad_answers.get(quiz_id) >= 2 && hint.length > 0) {\n",
       "                document.getElementById(quiz_id + \"-hint\").innerHTML = \n",
       "                    \"&nbsp;&nbsp;(Hint: <code>\" + hint + \"</code>)\";\n",
       "            }\n",
       "\n",
       "            if (!multiple_choice) {\n",
       "                for (i = 1;; i++) {\n",
       "                    checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                    label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "\n",
       "                    if (!checkbox)\n",
       "                        break;\n",
       "                    if (checkbox.checked) {\n",
       "                        label.style.textDecoration = \"line-through\";\n",
       "                    }\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "    function clear_selection(quiz_id) {\n",
       "        document.getElementById(quiz_id + \"-submit\").value = \"Submit\";\n",
       "        document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "    }\n",
       "    </script>\n",
       "    \n",
       "    <div class=\"quiz\">\n",
       "    <h3 class=\"quiz_title\">Quiz</h3>\n",
       "    <p>\n",
       "    <div class=\"quiz_question\">Why are statements replaced by <code>pass</code> rather than deleted?</div>\n",
       "    </p>\n",
       "    <p>\n",
       "    <div class=\"quiz_options\" title=\"Check all that apply.\">\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a47a7780-8159-11ee-8795-6298cf1a5790\" id=\"a47a7780-8159-11ee-8795-6298cf1a5790-1\" onclick=\"clear_selection('a47a7780-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a47a7780-8159-11ee-8795-6298cf1a5790-1-label\" for=\"a47a7780-8159-11ee-8795-6298cf1a5790-1\">Because <code>if P: pass</code> is valid Python, while <code>if P:</code> is not</label><br>\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a47a7780-8159-11ee-8795-6298cf1a5790\" id=\"a47a7780-8159-11ee-8795-6298cf1a5790-2\" onclick=\"clear_selection('a47a7780-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a47a7780-8159-11ee-8795-6298cf1a5790-2-label\" for=\"a47a7780-8159-11ee-8795-6298cf1a5790-2\">Because in Python, bodies for <code>if</code>, <code>while</code>, etc. cannot be empty</label><br>\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a47a7780-8159-11ee-8795-6298cf1a5790\" id=\"a47a7780-8159-11ee-8795-6298cf1a5790-3\" onclick=\"clear_selection('a47a7780-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a47a7780-8159-11ee-8795-6298cf1a5790-3-label\" for=\"a47a7780-8159-11ee-8795-6298cf1a5790-3\">Because a <code>pass</code> node makes a target for future mutations</label><br>\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a47a7780-8159-11ee-8795-6298cf1a5790\" id=\"a47a7780-8159-11ee-8795-6298cf1a5790-4\" onclick=\"clear_selection('a47a7780-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a47a7780-8159-11ee-8795-6298cf1a5790-4-label\" for=\"a47a7780-8159-11ee-8795-6298cf1a5790-4\">Because it causes the tests to pass</label><br>\n",
       "    \n",
       "    </div>\n",
       "    </p>\n",
       "    <input id=\"a47a7780-8159-11ee-8795-6298cf1a5790-submit\" type=\"submit\" value=\"Submit\" onclick=\"check_selection('a47a7780-8159-11ee-8795-6298cf1a5790', 14, 1, '[3 ^ n for n in range(3)]')\">\n",
       "    <span class=\"quiz_hint\" id=\"a47a7780-8159-11ee-8795-6298cf1a5790-hint\"></span>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quiz(\"Why are statements replaced by `pass` rather than deleted?\",\n",
    "     [\n",
    "         \"Because `if P: pass` is valid Python, while `if P:` is not\",\n",
    "         \"Because in Python, bodies for `if`, `while`, etc. cannot be empty\",\n",
    "         \"Because a `pass` node makes a target for future mutations\",\n",
    "         \"Because it causes the tests to pass\"\n",
    "     ], '[3 ^ n for n in range(3)]')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Indeed, Python's `compile()` will fail if any of the bodies is an empty list. Also, it leaves us a statement that can be evolved further."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Helpers\n",
    "\n",
    "For logging purposes, we introduce a helper function `format_node()` that returns a short string representation of the node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.668448Z",
     "iopub.status.busy": "2023-11-12T12:47:30.668339Z",
     "iopub.status.idle": "2023-11-12T12:47:30.671073Z",
     "shell.execute_reply": "2023-11-12T12:47:30.670768Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    NODE_MAX_LENGTH = 20\n",
    "\n",
    "    def format_node(self, node: ast.AST) -> str:\n",
    "        \"\"\"Return a string representation for `node`.\"\"\"\n",
    "        if node is None:\n",
    "            return \"None\"\n",
    "\n",
    "        if isinstance(node, list):\n",
    "            return \"; \".join(self.format_node(elem) for elem in node)\n",
    "\n",
    "        s = RE_SPACE.sub(' ', ast.unparse(node)).strip()\n",
    "        if len(s) > self.NODE_MAX_LENGTH - len(\"...\"):\n",
    "            s = s[:self.NODE_MAX_LENGTH] + \"...\"\n",
    "        return repr(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### All Together\n",
    "\n",
    "Let us now create the main entry point, which is `mutate()`. It picks the node to be mutated and marks it with a `mutate_me` attribute. By calling `visit()`, it then sets off the `NodeTransformer` transformation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.672517Z",
     "iopub.status.busy": "2023-11-12T12:47:30.672399Z",
     "iopub.status.idle": "2023-11-12T12:47:30.674887Z",
     "shell.execute_reply": "2023-11-12T12:47:30.674652Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class StatementMutator(StatementMutator):\n",
    "    def mutate(self, tree: ast.AST) -> ast.AST:\n",
    "        \"\"\"Mutate the given AST `tree` in place. Return mutated tree.\"\"\"\n",
    "\n",
    "        assert isinstance(tree, ast.AST)\n",
    "\n",
    "        tree = copy.deepcopy(tree)\n",
    "\n",
    "        if not self.source:\n",
    "            self.source = all_statements(tree)\n",
    "\n",
    "        for node in ast.walk(tree):\n",
    "            node.mutate_me = False  # type: ignore\n",
    "\n",
    "        node = self.node_to_be_mutated(tree)\n",
    "        node.mutate_me = True  # type: ignore\n",
    "\n",
    "        self.mutations = 0\n",
    "\n",
    "        tree = self.visit(tree)\n",
    "\n",
    "        if self.mutations == 0:\n",
    "            warnings.warn(\"No mutations found\")\n",
    "\n",
    "        ast.fix_missing_locations(tree)\n",
    "        return tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here are a number of transformations applied by `StatementMutator`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.676287Z",
     "iopub.status.busy": "2023-11-12T12:47:30.676183Z",
     "iopub.status.idle": "2023-11-12T12:47:30.686147Z",
     "shell.execute_reply": "2023-11-12T12:47:30.685862Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   9:insert: 'return y' becomes 'return y'\n",
      "   8:insert: 'if x > y: return y e...' becomes 'if x < y: if x > y: ...'\n",
      "  12:insert: 'return z' becomes 'if y < z: return z...'\n",
      "   3:swap:   'if x < y: return y e...' becomes 'return x'\n",
      "   3:swap:   'if x < y: return y e...' becomes 'return z'\n",
      "   3:swap:   'if x < y: return y e...' becomes 'return x'\n",
      "  11:swap:   'return x' becomes 'return y'\n",
      "  10:insert: 'if x > z: return x...' becomes 'if x > z: return x...'; 'return z'\n",
      "  12:delete: 'return z' becomes 'pass'\n",
      "   8:swap:   'if x > y: return y e...' becomes 'if y < z: pass'\n"
     ]
    }
   ],
   "source": [
    "mutator = StatementMutator(log=True)\n",
    "for i in range(10):\n",
    "    new_tree = mutator.mutate(middle_tree())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This is the effect of the last mutator applied on `middle`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.688011Z",
     "iopub.status.busy": "2023-11-12T12:47:30.687897Z",
     "iopub.status.idle": "2023-11-12T12:47:30.720983Z",
     "shell.execute_reply": "2023-11-12T12:47:30.720627Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mpass\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(new_tree), '.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Fitness\n",
    "\n",
    "Now that we can apply random mutations to code, let us find out how good these mutations are. Given our test suites for `middle`, we can check for a given code candidate how many of the previously passing test cases it passes, and how many of the failing test cases it passes. The more tests pass, the higher the _fitness_ of the candidate."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Not all passing tests have the same value, though. We want to prevent _regressions_ – that is, having a fix that breaks a previously passing test. The values of `WEIGHT_PASSING` and `WEIGHT_FAILING` set the relative weight (or importance) of passing vs. failing tests; we see that keeping passing tests passing is far more important than fixing failing tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.722934Z",
     "iopub.status.busy": "2023-11-12T12:47:30.722793Z",
     "iopub.status.idle": "2023-11-12T12:47:30.724717Z",
     "shell.execute_reply": "2023-11-12T12:47:30.724351Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "WEIGHT_PASSING = 0.99\n",
    "WEIGHT_FAILING = 0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.726373Z",
     "iopub.status.busy": "2023-11-12T12:47:30.726238Z",
     "iopub.status.idle": "2023-11-12T12:47:30.729507Z",
     "shell.execute_reply": "2023-11-12T12:47:30.729093Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def middle_fitness(tree: ast.AST) -> float:\n",
    "    \"\"\"Compute fitness of a `middle()` candidate given in `tree`\"\"\"\n",
    "    original_middle = middle\n",
    "\n",
    "    try:\n",
    "        code = compile(cast(ast.Module, tree), '<fitness>', 'exec')\n",
    "    except ValueError:\n",
    "        return 0  # Compilation error\n",
    "\n",
    "    exec(code, globals())\n",
    "\n",
    "    passing_passed = 0\n",
    "    failing_passed = 0\n",
    "\n",
    "    # Test how many of the passing runs pass\n",
    "    for x, y, z in MIDDLE_PASSING_TESTCASES:\n",
    "        try:\n",
    "            middle_test(x, y, z)\n",
    "            passing_passed += 1\n",
    "        except AssertionError:\n",
    "            pass\n",
    "\n",
    "    passing_ratio = passing_passed / len(MIDDLE_PASSING_TESTCASES)\n",
    "\n",
    "    # Test how many of the failing runs pass\n",
    "    for x, y, z in MIDDLE_FAILING_TESTCASES:\n",
    "        try:\n",
    "            middle_test(x, y, z)\n",
    "            failing_passed += 1\n",
    "        except AssertionError:\n",
    "            pass\n",
    "\n",
    "    failing_ratio = failing_passed / len(MIDDLE_FAILING_TESTCASES)\n",
    "\n",
    "    fitness = (WEIGHT_PASSING * passing_ratio +\n",
    "               WEIGHT_FAILING * failing_ratio)\n",
    "\n",
    "    globals()['middle'] = original_middle\n",
    "    return fitness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Our faulty `middle()` program has a fitness of `WEIGHT_PASSING` (99%), because it passes all the passing tests (but none of the failing ones)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.731191Z",
     "iopub.status.busy": "2023-11-12T12:47:30.731064Z",
     "iopub.status.idle": "2023-11-12T12:47:30.733925Z",
     "shell.execute_reply": "2023-11-12T12:47:30.733640Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.99"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_fitness(middle_tree())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Our \"sort of fixed\" version of `middle()` gets a much lower fitness:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.735431Z",
     "iopub.status.busy": "2023-11-12T12:47:30.735340Z",
     "iopub.status.idle": "2023-11-12T12:47:30.737659Z",
     "shell.execute_reply": "2023-11-12T12:47:30.737384Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4258"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_fitness(ast.parse(\"def middle(x, y, z): return x\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In the [chapter on statistical debugging](StatisticalDebugger), we also defined a fixed version of `middle()`. This gets a fitness of 1.0, passing all tests. (We won't use this fixed version for automated repairs.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.739314Z",
     "iopub.status.busy": "2023-11-12T12:47:30.739180Z",
     "iopub.status.idle": "2023-11-12T12:47:30.741012Z",
     "shell.execute_reply": "2023-11-12T12:47:30.740652Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from StatisticalDebugger import middle_fixed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.742904Z",
     "iopub.status.busy": "2023-11-12T12:47:30.742748Z",
     "iopub.status.idle": "2023-11-12T12:47:30.744835Z",
     "shell.execute_reply": "2023-11-12T12:47:30.744547Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "middle_fixed_source = \\\n",
    "    inspect.getsource(middle_fixed).replace('middle_fixed', 'middle').strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.746538Z",
     "iopub.status.busy": "2023-11-12T12:47:30.746406Z",
     "iopub.status.idle": "2023-11-12T12:47:30.748952Z",
     "shell.execute_reply": "2023-11-12T12:47:30.748559Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_fitness(ast.parse(middle_fixed_source))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Population\n",
    "\n",
    "We now set up a _population_ of fix candidates to evolve over time.  A higher population size will yield more candidates to check, but also need more time to test; a lower population size will yield fewer candidates, but allow for more evolution steps.  We choose a population size of 40 (from \\cite{LeGoues2012})."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.750710Z",
     "iopub.status.busy": "2023-11-12T12:47:30.750572Z",
     "iopub.status.idle": "2023-11-12T12:47:30.752289Z",
     "shell.execute_reply": "2023-11-12T12:47:30.751974Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "POPULATION_SIZE = 40\n",
    "middle_mutator = StatementMutator()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.753911Z",
     "iopub.status.busy": "2023-11-12T12:47:30.753785Z",
     "iopub.status.idle": "2023-11-12T12:47:30.781938Z",
     "shell.execute_reply": "2023-11-12T12:47:30.781634Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "MIDDLE_POPULATION = [middle_tree()] + \\\n",
    "    [middle_mutator.mutate(middle_tree()) for i in range(POPULATION_SIZE - 1)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We sort the fix candidates according to their fitness. This actually runs all tests on all candidates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.783522Z",
     "iopub.status.busy": "2023-11-12T12:47:30.783432Z",
     "iopub.status.idle": "2023-11-12T12:47:30.789323Z",
     "shell.execute_reply": "2023-11-12T12:47:30.789051Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "MIDDLE_POPULATION.sort(key=middle_fitness, reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The candidate with the highest fitness is still our original (faulty) `middle()` code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.790847Z",
     "iopub.status.busy": "2023-11-12T12:47:30.790747Z",
     "iopub.status.idle": "2023-11-12T12:47:30.792784Z",
     "shell.execute_reply": "2023-11-12T12:47:30.792537Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "def middle(x, y, z):\n",
      "    if y < z:\n",
      "        if x < y:\n",
      "            return y\n",
      "        elif x < z:\n",
      "            return y\n",
      "    elif x > y:\n",
      "        return y\n",
      "    elif x > z:\n",
      "        return x\n",
      "    return z 0.99\n"
     ]
    }
   ],
   "source": [
    "print(ast.unparse(MIDDLE_POPULATION[0]),\n",
    "      middle_fitness(MIDDLE_POPULATION[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "At the other end of the spectrum, the candidate with the lowest fitness has some vital functionality removed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.794172Z",
     "iopub.status.busy": "2023-11-12T12:47:30.794087Z",
     "iopub.status.idle": "2023-11-12T12:47:30.796188Z",
     "shell.execute_reply": "2023-11-12T12:47:30.795915Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "def middle(x, y, z):\n",
      "    if y < z:\n",
      "        if x < y:\n",
      "            return y\n",
      "        elif x < z:\n",
      "            return y\n",
      "    else:\n",
      "        return y\n",
      "    return z 0.5445\n"
     ]
    }
   ],
   "source": [
    "print(ast.unparse(MIDDLE_POPULATION[-1]),\n",
    "      middle_fitness(MIDDLE_POPULATION[-1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Evolution\n",
    "\n",
    "To evolve our population of candidates, we fill up the population with mutations created from the population, using a `StatementMutator` as described above to create these mutations. Then we reduce the population to its original size, keeping the fittest candidates.\n",
    "<!-- TODO: shouldn't there be some kind of randomness to also keep sometimes candidates with lesser fitness? -->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.797817Z",
     "iopub.status.busy": "2023-11-12T12:47:30.797705Z",
     "iopub.status.idle": "2023-11-12T12:47:30.800078Z",
     "shell.execute_reply": "2023-11-12T12:47:30.799828Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def evolve_middle() -> None:\n",
    "    global MIDDLE_POPULATION\n",
    "\n",
    "    source = all_statements(middle_tree())\n",
    "    mutator = StatementMutator(source=source)\n",
    "\n",
    "    n = len(MIDDLE_POPULATION)\n",
    "\n",
    "    offspring: List[ast.AST] = []\n",
    "    while len(offspring) < n:\n",
    "        parent = random.choice(MIDDLE_POPULATION)\n",
    "        offspring.append(mutator.mutate(parent))\n",
    "\n",
    "    MIDDLE_POPULATION += offspring\n",
    "    MIDDLE_POPULATION.sort(key=middle_fitness, reverse=True)\n",
    "    MIDDLE_POPULATION = MIDDLE_POPULATION[:n]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This is what happens when evolving our population for the first time; the original source is still our best candidate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.801648Z",
     "iopub.status.busy": "2023-11-12T12:47:30.801519Z",
     "iopub.status.idle": "2023-11-12T12:47:30.830719Z",
     "shell.execute_reply": "2023-11-12T12:47:30.830432Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "evolve_middle()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.832249Z",
     "iopub.status.busy": "2023-11-12T12:47:30.832157Z",
     "iopub.status.idle": "2023-11-12T12:47:30.834364Z",
     "shell.execute_reply": "2023-11-12T12:47:30.834087Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "def middle(x, y, z):\n",
      "    if y < z:\n",
      "        if x < y:\n",
      "            return y\n",
      "        elif x < z:\n",
      "            return y\n",
      "    elif x > y:\n",
      "        return y\n",
      "    elif x > z:\n",
      "        return x\n",
      "    return z 0.99\n"
     ]
    }
   ],
   "source": [
    "tree = MIDDLE_POPULATION[0]\n",
    "print(ast.unparse(tree), middle_fitness(tree))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.836382Z",
     "iopub.status.busy": "2023-11-12T12:47:30.836266Z",
     "iopub.status.idle": "2023-11-12T12:47:30.838040Z",
     "shell.execute_reply": "2023-11-12T12:47:30.837790Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert middle_fitness(tree) < 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "However, nothing keeps us from evolving for a few generations more..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.839350Z",
     "iopub.status.busy": "2023-11-12T12:47:30.839260Z",
     "iopub.status.idle": "2023-11-12T12:47:30.900212Z",
     "shell.execute_reply": "2023-11-12T12:47:30.899851Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration  1: fitness = 1.0  "
     ]
    }
   ],
   "source": [
    "for i in range(50):\n",
    "    evolve_middle()\n",
    "    best_middle_tree = MIDDLE_POPULATION[0]\n",
    "    fitness = middle_fitness(best_middle_tree)\n",
    "    print(f\"\\rIteration {i:2}: fitness = {fitness}  \", end=\"\")\n",
    "    if fitness >= 1.0:\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.901979Z",
     "iopub.status.busy": "2023-11-12T12:47:30.901837Z",
     "iopub.status.idle": "2023-11-12T12:47:30.903700Z",
     "shell.execute_reply": "2023-11-12T12:47:30.903454Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert middle_fitness(best_middle_tree) >= 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Success! We find a candidate that actually passes all tests, including the failing ones. Here is the candidate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.905236Z",
     "iopub.status.busy": "2023-11-12T12:47:30.905120Z",
     "iopub.status.idle": "2023-11-12T12:47:30.940506Z",
     "shell.execute_reply": "2023-11-12T12:47:30.940173Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 1  \u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      " 2      \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      " 3          \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      " 4              \u001b[34mif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      " 5                  \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      " 6          \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      " 7              \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      " 8      \u001b[34melif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      " 9          \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "10      \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "11          \u001b[34mif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "12              \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "13          \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m\n",
      "14      \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(best_middle_tree), '.py', start_line_number=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "... and yes, it passes all tests:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.942222Z",
     "iopub.status.busy": "2023-11-12T12:47:30.942099Z",
     "iopub.status.idle": "2023-11-12T12:47:30.944370Z",
     "shell.execute_reply": "2023-11-12T12:47:30.944099Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "original_middle = middle\n",
    "code = compile(cast(ast.Module, best_middle_tree), '<string>', 'exec')\n",
    "exec(code, globals())\n",
    "\n",
    "for x, y, z in MIDDLE_PASSING_TESTCASES + MIDDLE_FAILING_TESTCASES:\n",
    "    middle_test(x, y, z)\n",
    "\n",
    "middle = original_middle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "As the code is already validated by hundreds of test cases, it is very valuable for the programmer. Even if the programmer decides not to use the code as is, the location gives very strong hints on which code to examine and where to apply a fix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "However, a closer look at our fix candidate shows that there is some amount of redundancy – that is, superfluous statements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.945849Z",
     "iopub.status.busy": "2023-11-12T12:47:30.945738Z",
     "iopub.status.idle": "2023-11-12T12:47:30.950381Z",
     "shell.execute_reply": "2023-11-12T12:47:30.950048Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    \n",
       "    <script>\n",
       "    var bad_answers = new Map();\n",
       "\n",
       "    function answer(quiz_id) {\n",
       "        ans = 0;\n",
       "        for (i = 1;; i++) {\n",
       "            checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "            if (!checkbox)\n",
       "                break;\n",
       "            if (checkbox.checked)\n",
       "                ans |= (1 << i);\n",
       "        }\n",
       "        return ans;\n",
       "    }\n",
       "    function check_selection(quiz_id, correct_answer, multiple_choice, hint) {\n",
       "        given_answer = answer(quiz_id);\n",
       "        if (given_answer == correct_answer)\n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Correct!\";\n",
       "            document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "\n",
       "            for (i = 1;; i++) {\n",
       "                checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "                if (!checkbox)\n",
       "                    break;\n",
       "    \n",
       "                if (checkbox.checked) {\n",
       "                    label.style.fontWeight = \"bold\";\n",
       "                }\n",
       "                else {\n",
       "                    label.style.textDecoration = \"line-through\";\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "        else \n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Try again\";\n",
       "            \n",
       "            if (!bad_answers.has(quiz_id)) {\n",
       "                bad_answers.set(quiz_id, 1);\n",
       "            }\n",
       "            else {\n",
       "                bad_answers.set(quiz_id, bad_answers.get(quiz_id) + 1);\n",
       "            }\n",
       "\n",
       "            if (bad_answers.get(quiz_id) >= 2 && hint.length > 0) {\n",
       "                document.getElementById(quiz_id + \"-hint\").innerHTML = \n",
       "                    \"&nbsp;&nbsp;(Hint: <code>\" + hint + \"</code>)\";\n",
       "            }\n",
       "\n",
       "            if (!multiple_choice) {\n",
       "                for (i = 1;; i++) {\n",
       "                    checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                    label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "\n",
       "                    if (!checkbox)\n",
       "                        break;\n",
       "                    if (checkbox.checked) {\n",
       "                        label.style.textDecoration = \"line-through\";\n",
       "                    }\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "    function clear_selection(quiz_id) {\n",
       "        document.getElementById(quiz_id + \"-submit\").value = \"Submit\";\n",
       "        document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "    }\n",
       "    </script>\n",
       "    \n",
       "    <div class=\"quiz\">\n",
       "    <h3 class=\"quiz_title\">Quiz</h3>\n",
       "    <p>\n",
       "    <div class=\"quiz_question\">Some of the lines in our fix candidate are redundant. Which are these?</div>\n",
       "    </p>\n",
       "    <p>\n",
       "    <div class=\"quiz_options\" title=\"Check all that apply.\">\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a4a5c124-8159-11ee-8795-6298cf1a5790\" id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-1\" onclick=\"clear_selection('a4a5c124-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-1-label\" for=\"a4a5c124-8159-11ee-8795-6298cf1a5790-1\">Line 3: <code>if x &lt; y:</code></label><br>\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a4a5c124-8159-11ee-8795-6298cf1a5790\" id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-2\" onclick=\"clear_selection('a4a5c124-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-2-label\" for=\"a4a5c124-8159-11ee-8795-6298cf1a5790-2\">Line 4: <code>if x &lt; z:</code></label><br>\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a4a5c124-8159-11ee-8795-6298cf1a5790\" id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-3\" onclick=\"clear_selection('a4a5c124-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-3-label\" for=\"a4a5c124-8159-11ee-8795-6298cf1a5790-3\">Line 5: <code>return y</code></label><br>\n",
       "    \n",
       "        <input type=\"checkbox\" name=\"a4a5c124-8159-11ee-8795-6298cf1a5790\" id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-4\" onclick=\"clear_selection('a4a5c124-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-4-label\" for=\"a4a5c124-8159-11ee-8795-6298cf1a5790-4\">Line 13: <code>return z</code></label><br>\n",
       "    \n",
       "    </div>\n",
       "    </p>\n",
       "    <input id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-submit\" type=\"submit\" value=\"Submit\" onclick=\"check_selection('a4a5c124-8159-11ee-8795-6298cf1a5790', 20, 1, '[eval(chr(100 - x)) for x in [48, 50]]')\">\n",
       "    <span class=\"quiz_hint\" id=\"a4a5c124-8159-11ee-8795-6298cf1a5790-hint\"></span>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quiz(\"Some of the lines in our fix candidate are redundant. \"\n",
    "     \"Which are these?\",\n",
    "    [\n",
    "        \"Line 3: `if x < y:`\",\n",
    "        \"Line 4: `if x < z:`\",\n",
    "        \"Line 5: `return y`\",\n",
    "        \"Line 13: `return z`\"\n",
    "    ], '[eval(chr(100 - x)) for x in [48, 50]]')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Simplifying"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "As demonstrated in the chapter on [reducing failure-inducing inputs](DeltaDebugger.ipynb), we can use delta debugging on code to get rid of these superfluous statements."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The trick for simplification is to have the test function (`test_middle_lines()`) declare a fitness of 1.0 as a \"failure\". Delta debugging will then simplify the input as long as the \"failure\" (and hence the maximum fitness obtained) persists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:30.952090Z",
     "iopub.status.busy": "2023-11-12T12:47:30.951973Z",
     "iopub.status.idle": "2023-11-12T12:47:31.020134Z",
     "shell.execute_reply": "2023-11-12T12:47:31.019795Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from DeltaDebugger import DeltaDebugger"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.021900Z",
     "iopub.status.busy": "2023-11-12T12:47:31.021795Z",
     "iopub.status.idle": "2023-11-12T12:47:31.023893Z",
     "shell.execute_reply": "2023-11-12T12:47:31.023584Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "middle_lines = ast.unparse(best_middle_tree).strip().split('\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.025561Z",
     "iopub.status.busy": "2023-11-12T12:47:31.025412Z",
     "iopub.status.idle": "2023-11-12T12:47:31.027303Z",
     "shell.execute_reply": "2023-11-12T12:47:31.027041Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def test_middle_lines(lines: List[str]) -> None:\n",
    "    source = \"\\n\".join(lines)\n",
    "    tree = ast.parse(source)\n",
    "    assert middle_fitness(tree) < 1.0  # \"Fail\" only while fitness is 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.028777Z",
     "iopub.status.busy": "2023-11-12T12:47:31.028673Z",
     "iopub.status.idle": "2023-11-12T12:47:31.030566Z",
     "shell.execute_reply": "2023-11-12T12:47:31.030299Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "with DeltaDebugger() as dd:\n",
    "    test_middle_lines(middle_lines)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.032091Z",
     "iopub.status.busy": "2023-11-12T12:47:31.031969Z",
     "iopub.status.idle": "2023-11-12T12:47:31.037709Z",
     "shell.execute_reply": "2023-11-12T12:47:31.037442Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "reduced_lines = dd.min_args()['lines']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.039216Z",
     "iopub.status.busy": "2023-11-12T12:47:31.039118Z",
     "iopub.status.idle": "2023-11-12T12:47:31.040991Z",
     "shell.execute_reply": "2023-11-12T12:47:31.040738Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "reduced_source = \"\\n\".join(reduced_lines)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.042529Z",
     "iopub.status.busy": "2023-11-12T12:47:31.042442Z",
     "iopub.status.idle": "2023-11-12T12:47:31.075000Z",
     "shell.execute_reply": "2023-11-12T12:47:31.074671Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "repaired_source = ast.unparse(ast.parse(reduced_source))  # normalize\n",
    "print_content(repaired_source, '.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.076503Z",
     "iopub.status.busy": "2023-11-12T12:47:31.076408Z",
     "iopub.status.idle": "2023-11-12T12:47:31.078340Z",
     "shell.execute_reply": "2023-11-12T12:47:31.077998Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert len(reduced_lines) < len(middle_lines)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Success! Delta Debugging has eliminated the superfluous statements. We can present the difference to the original as a patch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.079989Z",
     "iopub.status.busy": "2023-11-12T12:47:31.079869Z",
     "iopub.status.idle": "2023-11-12T12:47:31.081729Z",
     "shell.execute_reply": "2023-11-12T12:47:31.081460Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "original_source = ast.unparse(ast.parse(middle_source))  # normalize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.083368Z",
     "iopub.status.busy": "2023-11-12T12:47:31.083241Z",
     "iopub.status.idle": "2023-11-12T12:47:31.105964Z",
     "shell.execute_reply": "2023-11-12T12:47:31.105657Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from ChangeDebugger import diff, print_patch  # minor dependency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.107760Z",
     "iopub.status.busy": "2023-11-12T12:47:31.107622Z",
     "iopub.status.idle": "2023-11-12T12:47:31.140010Z",
     "shell.execute_reply": "2023-11-12T12:47:31.139570Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@@ -\u001b[34m87\u001b[39;49;00m,\u001b[34m37\u001b[39;49;00m +\u001b[34m87\u001b[39;49;00m,\u001b[34m37\u001b[39;49;00m @@\u001b[37m\u001b[39;49;00m\n",
      "  x < z:\u001b[37m\u001b[39;49;00m\n",
      "\u001b[37m\u001b[39;49;00m\n",
      "-            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "\u001b[37m\u001b[39;49;00m\n",
      "+            \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "\u001b[37m\u001b[39;49;00m\n",
      "     \u001b[34melif\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n"
     ]
    }
   ],
   "source": [
    "for patch in diff(original_source, repaired_source):\n",
    "    print_patch(patch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can present this patch to the programmer, who will then immediately know what to fix in the `middle()` code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Crossover\n",
    "\n",
    "So far, we have only applied one kind of genetic operators – mutation. There is a second one, though, also inspired by natural selection. \n",
    "\n",
    "The *crossover* operation mutates two strands of genes, as illustrated in the following picture. We have two parents (red and blue), each as a sequence of genes. To create \"crossed\" children, we pick a _crossover point_ and exchange the strands at this very point:\n",
    "\n",
    "![](https://upload.wikimedia.org/wikipedia/commons/thumb/5/56/OnePointCrossover.svg/500px-OnePointCrossover.svg.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We implement a `CrossoverOperator` class that implements such an operation on two randomly chosen statement lists of two programs. It is used as\n",
    "\n",
    "```python\n",
    "crossover = CrossoverOperator()\n",
    "crossover.crossover(tree_p1, tree_p2)\n",
    "```\n",
    "\n",
    "where `tree_p1` and `tree_p2` are two ASTs that are changed in place."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Excursion: Implementing Crossover"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Crossing Statement Lists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Applied on programs, a crossover mutation takes two parents and \"crosses\" a list of statements. As an example, if our \"parents\" `p1()` and `p2()` are defined as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.142134Z",
     "iopub.status.busy": "2023-11-12T12:47:31.141936Z",
     "iopub.status.idle": "2023-11-12T12:47:31.143874Z",
     "shell.execute_reply": "2023-11-12T12:47:31.143597Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def p1():  # type: ignore\n",
    "    a = 1\n",
    "    b = 2\n",
    "    c = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.145620Z",
     "iopub.status.busy": "2023-11-12T12:47:31.145491Z",
     "iopub.status.idle": "2023-11-12T12:47:31.147309Z",
     "shell.execute_reply": "2023-11-12T12:47:31.147016Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def p2():  # type: ignore\n",
    "    x = 1\n",
    "    y = 2\n",
    "    z = 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Then a crossover operation would produce one child with a body\n",
    "\n",
    "```python\n",
    "a = 1\n",
    "y = 2\n",
    "z = 3\n",
    "```\n",
    "\n",
    "and another child with a body\n",
    "\n",
    "```python\n",
    "x = 1\n",
    "b = 2\n",
    "c = 3\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can easily implement this in a `CrossoverOperator` class in a method `cross_bodies()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.149033Z",
     "iopub.status.busy": "2023-11-12T12:47:31.148908Z",
     "iopub.status.idle": "2023-11-12T12:47:31.151867Z",
     "shell.execute_reply": "2023-11-12T12:47:31.151550Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class CrossoverOperator:\n",
    "    \"\"\"A class for performing statement crossover of Python programs\"\"\"\n",
    "\n",
    "    def __init__(self, log: Union[bool, int] = False):\n",
    "        \"\"\"Constructor. If `log` is set, turn on logging.\"\"\"\n",
    "        self.log = log\n",
    "\n",
    "    def cross_bodies(self, body_1: List[ast.AST], body_2: List[ast.AST]) -> \\\n",
    "        Tuple[List[ast.AST], List[ast.AST]]:\n",
    "        \"\"\"Crossover the statement lists `body_1` x `body_2`. Return new lists.\"\"\"\n",
    "\n",
    "        assert isinstance(body_1, list)\n",
    "        assert isinstance(body_2, list)\n",
    "\n",
    "        crossover_point_1 = len(body_1) // 2\n",
    "        crossover_point_2 = len(body_2) // 2\n",
    "        return (body_1[:crossover_point_1] + body_2[crossover_point_2:],\n",
    "                body_2[:crossover_point_2] + body_1[crossover_point_1:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here's the `CrossoverOperatorMutator` applied on `p1` and `p2`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.153658Z",
     "iopub.status.busy": "2023-11-12T12:47:31.153438Z",
     "iopub.status.idle": "2023-11-12T12:47:31.155893Z",
     "shell.execute_reply": "2023-11-12T12:47:31.155596Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "tree_p1: ast.Module = ast.parse(inspect.getsource(p1))\n",
    "tree_p2: ast.Module = ast.parse(inspect.getsource(p2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.157433Z",
     "iopub.status.busy": "2023-11-12T12:47:31.157323Z",
     "iopub.status.idle": "2023-11-12T12:47:31.159750Z",
     "shell.execute_reply": "2023-11-12T12:47:31.159492Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<ast.Assign at 0x14f007f70>,\n",
       " <ast.Assign at 0x14f0057b0>,\n",
       " <ast.Assign at 0x14f005510>]"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "body_p1 = tree_p1.body[0].body  # type: ignore\n",
    "body_p2 = tree_p2.body[0].body  # type: ignore\n",
    "body_p1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.180481Z",
     "iopub.status.busy": "2023-11-12T12:47:31.180315Z",
     "iopub.status.idle": "2023-11-12T12:47:31.182408Z",
     "shell.execute_reply": "2023-11-12T12:47:31.182066Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "crosser = CrossoverOperator()\n",
    "tree_p1.body[0].body, tree_p2.body[0].body = crosser.cross_bodies(body_p1, body_p2)  # type: ignore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.184011Z",
     "iopub.status.busy": "2023-11-12T12:47:31.183892Z",
     "iopub.status.idle": "2023-11-12T12:47:31.214014Z",
     "shell.execute_reply": "2023-11-12T12:47:31.213676Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mp1\u001b[39;49;00m():\u001b[37m\u001b[39;49;00m\n",
      "    a = \u001b[34m1\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    y = \u001b[34m2\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    z = \u001b[34m3\u001b[39;49;00m\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(tree_p1), '.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.215855Z",
     "iopub.status.busy": "2023-11-12T12:47:31.215724Z",
     "iopub.status.idle": "2023-11-12T12:47:31.246791Z",
     "shell.execute_reply": "2023-11-12T12:47:31.246487Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mp2\u001b[39;49;00m():\u001b[37m\u001b[39;49;00m\n",
      "    x = \u001b[34m1\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    b = \u001b[34m2\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    c = \u001b[34m3\u001b[39;49;00m\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(tree_p2), '.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Applying Crossover on Programs\n",
    "\n",
    "Applying the crossover operation on arbitrary programs is a bit more complex, though. We first have to _find_ lists of statements that we actually can cross over. The `can_cross()` method returns True if we have a list of statements that we can cross. Python modules and classes are excluded, because changing the ordering of definitions will not have much impact on the program functionality, other than introducing errors due to  dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.248436Z",
     "iopub.status.busy": "2023-11-12T12:47:31.248281Z",
     "iopub.status.idle": "2023-11-12T12:47:31.250845Z",
     "shell.execute_reply": "2023-11-12T12:47:31.250564Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class CrossoverOperator(CrossoverOperator):\n",
    "    # In modules and class defs, the ordering of elements does not matter (much)\n",
    "    SKIP_LIST = {ast.Module, ast.ClassDef}\n",
    "\n",
    "    def can_cross(self, tree: ast.AST, body_attr: str = 'body') -> bool:\n",
    "        if any(isinstance(tree, cls) for cls in self.SKIP_LIST):\n",
    "            return False\n",
    "\n",
    "        body = getattr(tree, body_attr, [])\n",
    "        return body is not None and len(body) >= 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here comes our method `crossover_attr()` which searches for crossover possibilities. It takes two ASTs `t1` and `t2` and an attribute (typically `'body'`) and retrieves the attribute lists $l_1$ (from `t1.<attr>`) and $l_2$ (from `t2.<attr>`).\n",
    "\n",
    "If $l_1$ and $l_2$ can be crossed, it crosses them, and is done. Otherwise\n",
    "\n",
    "* If there is a pair of elements $e_1 \\in l_1$ and $e_2 \\in l_2$ that has the same name – say, functions of the same name –, it applies itself to $e_1$ and $e_2$.\n",
    "* Otherwise, it creates random pairs of elements $e_1 \\in l_1$ and $e_2 \\in l_2$ and applies itself on these very pairs.\n",
    "\n",
    "`crossover_attr()` changes `t1` and `t2` in place and returns True if a crossover was found; it returns False otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.252406Z",
     "iopub.status.busy": "2023-11-12T12:47:31.252279Z",
     "iopub.status.idle": "2023-11-12T12:47:31.256456Z",
     "shell.execute_reply": "2023-11-12T12:47:31.256175Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class CrossoverOperator(CrossoverOperator):\n",
    "    def crossover_attr(self, t1: ast.AST, t2: ast.AST, body_attr: str) -> bool:\n",
    "        \"\"\"\n",
    "        Crossover the bodies `body_attr` of two trees `t1` and `t2`.\n",
    "        Return True if successful.\n",
    "        \"\"\"\n",
    "        assert isinstance(t1, ast.AST)\n",
    "        assert isinstance(t2, ast.AST)\n",
    "        assert isinstance(body_attr, str)\n",
    "\n",
    "        if not getattr(t1, body_attr, None) or not getattr(t2, body_attr, None):\n",
    "            return False\n",
    "\n",
    "        if self.crossover_branches(t1, t2):\n",
    "            return True\n",
    "\n",
    "        if self.log > 1:\n",
    "            print(f\"Checking {t1}.{body_attr} x {t2}.{body_attr}\")\n",
    "\n",
    "        body_1 = getattr(t1, body_attr)\n",
    "        body_2 = getattr(t2, body_attr)\n",
    "\n",
    "        # If both trees have the attribute, we can cross their bodies\n",
    "        if self.can_cross(t1, body_attr) and self.can_cross(t2, body_attr):\n",
    "            if self.log:\n",
    "                print(f\"Crossing {t1}.{body_attr} x {t2}.{body_attr}\")\n",
    "\n",
    "            new_body_1, new_body_2 = self.cross_bodies(body_1, body_2)\n",
    "            setattr(t1, body_attr, new_body_1)\n",
    "            setattr(t2, body_attr, new_body_2)\n",
    "            return True\n",
    "\n",
    "        # Strategy 1: Find matches in class/function of same name\n",
    "        for child_1 in body_1:\n",
    "            if hasattr(child_1, 'name'):\n",
    "                for child_2 in body_2:\n",
    "                    if (hasattr(child_2, 'name') and\n",
    "                           child_1.name == child_2.name):\n",
    "                        if self.crossover_attr(child_1, child_2, body_attr):\n",
    "                            return True\n",
    "\n",
    "        # Strategy 2: Find matches anywhere\n",
    "        for child_1 in random.sample(body_1, len(body_1)):\n",
    "            for child_2 in random.sample(body_2, len(body_2)):\n",
    "                if self.crossover_attr(child_1, child_2, body_attr):\n",
    "                    return True\n",
    "\n",
    "        return False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We have a special case for `if` nodes, where we can cross their body and `else` branches. (In Python, `for` and `while` also have `else` branches, but swapping these with loop bodies is likely to create havoc.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.258022Z",
     "iopub.status.busy": "2023-11-12T12:47:31.257900Z",
     "iopub.status.idle": "2023-11-12T12:47:31.260793Z",
     "shell.execute_reply": "2023-11-12T12:47:31.260513Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class CrossoverOperator(CrossoverOperator):\n",
    "    def crossover_branches(self, t1: ast.AST, t2: ast.AST) -> bool:\n",
    "        \"\"\"Special case:\n",
    "        `t1` = `if P: S1 else: S2` x `t2` = `if P': S1' else: S2'`\n",
    "        becomes\n",
    "        `t1` = `if P: S2' else: S1'` and `t2` = `if P': S2 else: S1`\n",
    "        Returns True if successful.\n",
    "        \"\"\"\n",
    "        assert isinstance(t1, ast.AST)\n",
    "        assert isinstance(t2, ast.AST)\n",
    "\n",
    "        if (hasattr(t1, 'body') and hasattr(t1, 'orelse') and\n",
    "            hasattr(t2, 'body') and hasattr(t2, 'orelse')):\n",
    "\n",
    "            t1 = cast(ast.If, t1)  # keep mypy happy\n",
    "            t2 = cast(ast.If, t2)\n",
    "\n",
    "            if self.log:\n",
    "                print(f\"Crossing branches {t1} x {t2}\")\n",
    "\n",
    "            t1.body, t1.orelse, t2.body, t2.orelse = \\\n",
    "                t2.orelse, t2.body, t1.orelse, t1.body\n",
    "            return True\n",
    "\n",
    "        return False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The method `crossover()` is the main entry point. It checks for the special `if` case as described above; if not, it searches for possible crossover points. It raises `CrossoverError` if not successful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.262338Z",
     "iopub.status.busy": "2023-11-12T12:47:31.262209Z",
     "iopub.status.idle": "2023-11-12T12:47:31.264652Z",
     "shell.execute_reply": "2023-11-12T12:47:31.264383Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class CrossoverOperator(CrossoverOperator):\n",
    "    def crossover(self, t1: ast.AST, t2: ast.AST) -> Tuple[ast.AST, ast.AST]:\n",
    "        \"\"\"Do a crossover of ASTs `t1` and `t2`.\n",
    "        Raises `CrossoverError` if no crossover is found.\"\"\"\n",
    "        assert isinstance(t1, ast.AST)\n",
    "        assert isinstance(t2, ast.AST)\n",
    "\n",
    "        for body_attr in ['body', 'orelse', 'finalbody']:\n",
    "            if self.crossover_attr(t1, t2, body_attr):\n",
    "                return t1, t2\n",
    "\n",
    "        raise CrossoverError(\"No crossover found\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.266071Z",
     "iopub.status.busy": "2023-11-12T12:47:31.265956Z",
     "iopub.status.idle": "2023-11-12T12:47:31.267646Z",
     "shell.execute_reply": "2023-11-12T12:47:31.267371Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class CrossoverError(ValueError):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### End of Excursion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Crossover in Action"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let us put our `CrossoverOperator` in action. Here is a test case for crossover, involving more deeply nested structures:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.269396Z",
     "iopub.status.busy": "2023-11-12T12:47:31.269250Z",
     "iopub.status.idle": "2023-11-12T12:47:31.271277Z",
     "shell.execute_reply": "2023-11-12T12:47:31.270969Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def p1():  # type: ignore\n",
    "    if True:\n",
    "        print(1)\n",
    "        print(2)\n",
    "        print(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.273430Z",
     "iopub.status.busy": "2023-11-12T12:47:31.273080Z",
     "iopub.status.idle": "2023-11-12T12:47:31.275464Z",
     "shell.execute_reply": "2023-11-12T12:47:31.275080Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def p2():  # type: ignore\n",
    "    if True:\n",
    "        print(a)\n",
    "        print(b)\n",
    "    else:\n",
    "        print(c)\n",
    "        print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We invoke the `crossover()` method with two ASTs from `p1` and `p2`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.277585Z",
     "iopub.status.busy": "2023-11-12T12:47:31.277407Z",
     "iopub.status.idle": "2023-11-12T12:47:31.280659Z",
     "shell.execute_reply": "2023-11-12T12:47:31.280325Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "crossover = CrossoverOperator()\n",
    "tree_p1 = ast.parse(inspect.getsource(p1))\n",
    "tree_p2 = ast.parse(inspect.getsource(p2))\n",
    "crossover.crossover(tree_p1, tree_p2);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here is the crossed offspring, mixing statement lists of `p1` and `p2`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.282343Z",
     "iopub.status.busy": "2023-11-12T12:47:31.282240Z",
     "iopub.status.idle": "2023-11-12T12:47:31.313994Z",
     "shell.execute_reply": "2023-11-12T12:47:31.313636Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mp1\u001b[39;49;00m():\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m \u001b[34mTrue\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(c)\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(d)\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(a)\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(b)\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(tree_p1), '.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.315718Z",
     "iopub.status.busy": "2023-11-12T12:47:31.315592Z",
     "iopub.status.idle": "2023-11-12T12:47:31.347949Z",
     "shell.execute_reply": "2023-11-12T12:47:31.347636Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mp2\u001b[39;49;00m():\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m \u001b[34mTrue\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(\u001b[34m1\u001b[39;49;00m)\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(\u001b[34m2\u001b[39;49;00m)\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(\u001b[34m3\u001b[39;49;00m)\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(tree_p2), '.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here is our special case for `if` nodes in action, crossing our `middle()` tree with `p2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.349911Z",
     "iopub.status.busy": "2023-11-12T12:47:31.349774Z",
     "iopub.status.idle": "2023-11-12T12:47:31.352466Z",
     "shell.execute_reply": "2023-11-12T12:47:31.352098Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "middle_t1, middle_t2 = crossover.crossover(middle_tree(),\n",
    "                                          ast.parse(inspect.getsource(p2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We see how the resulting offspring encompasses elements of both sources:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.354511Z",
     "iopub.status.busy": "2023-11-12T12:47:31.354338Z",
     "iopub.status.idle": "2023-11-12T12:47:31.387459Z",
     "shell.execute_reply": "2023-11-12T12:47:31.387138Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(c)\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(d)\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(a)\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[36mprint\u001b[39;49;00m(b)\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(middle_t1), '.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.389406Z",
     "iopub.status.busy": "2023-11-12T12:47:31.389253Z",
     "iopub.status.idle": "2023-11-12T12:47:31.422186Z",
     "shell.execute_reply": "2023-11-12T12:47:31.421841Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mp2\u001b[39;49;00m():\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m \u001b[34mTrue\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(middle_t2), '.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## A Repairer Class\n",
    "\n",
    "So far, we have applied all our techniques on the `middle()` program only. Let us now create a `Repairer` class that applies automatic program repair on arbitrary Python programs. The idea is that you can apply it on some statistical debugger, for which you have gathered passing and failing test cases, and then invoke its `repair()` method to find a \"best\" fix candidate:\n",
    "\n",
    "```python\n",
    "debugger = OchiaiDebugger()\n",
    "with debugger:\n",
    "    <passing test>\n",
    "with debugger:\n",
    "    <failing test>\n",
    "...\n",
    "repairer = Repairer(debugger)\n",
    "repairer.repair()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Excursion: Implementing Repairer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The main argument to the `Repairer` constructor is the `debugger` to get information from. On top of that, it also allows customizing the classes used for mutation, crossover, and reduction.\n",
    "Setting `targets` allows defining a set of functions to repair; setting `sources` allows setting a set of sources to take repairs from.\n",
    "The constructor then sets up the environment for running tests and repairing, as described below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.424210Z",
     "iopub.status.busy": "2023-11-12T12:47:31.424067Z",
     "iopub.status.idle": "2023-11-12T12:47:31.426060Z",
     "shell.execute_reply": "2023-11-12T12:47:31.425594Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from StackInspector import StackInspector  # minor dependency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.427901Z",
     "iopub.status.busy": "2023-11-12T12:47:31.427771Z",
     "iopub.status.idle": "2023-11-12T12:47:31.432580Z",
     "shell.execute_reply": "2023-11-12T12:47:31.432218Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(StackInspector):\n",
    "    \"\"\"A class for automatic repair of Python programs\"\"\"\n",
    "\n",
    "    def __init__(self, debugger: RankingDebugger, *,\n",
    "                 targets: Optional[List[Any]] = None,\n",
    "                 sources: Optional[List[Any]] = None,\n",
    "                 log: Union[bool, int] = False,\n",
    "                 mutator_class: Type = StatementMutator,\n",
    "                 crossover_class: Type = CrossoverOperator,\n",
    "                 reducer_class: Type = DeltaDebugger,\n",
    "                 globals: Optional[Dict[str, Any]] = None):\n",
    "        \"\"\"Constructor.\n",
    "`debugger`: a `RankingDebugger` to take tests and coverage from.\n",
    "`targets`: a list of functions/modules to be repaired.\n",
    "    (default: the covered functions in `debugger`, except tests)\n",
    "`sources`: a list of functions/modules to take repairs from.\n",
    "    (default: same as `targets`)\n",
    "`globals`: if given, a `globals()` dict for executing targets\n",
    "    (default: `globals()` of caller)\"\"\"\n",
    "\n",
    "        assert isinstance(debugger, RankingDebugger)\n",
    "        self.debugger = debugger\n",
    "        self.log = log\n",
    "\n",
    "        if targets is None:\n",
    "            targets = self.default_functions()\n",
    "        if not targets:\n",
    "            raise ValueError(\"No targets to repair\")\n",
    "\n",
    "        if sources is None:\n",
    "            sources = self.default_functions()\n",
    "        if not sources:\n",
    "            raise ValueError(\"No sources to take repairs from\")\n",
    "\n",
    "        if self.debugger.function() is None:\n",
    "            raise ValueError(\"Multiple entry points observed\")\n",
    "\n",
    "        self.target_tree: ast.AST = self.parse(targets)\n",
    "        self.source_tree: ast.AST = self.parse(sources)\n",
    "\n",
    "        self.log_tree(\"Target code to be repaired:\", self.target_tree)\n",
    "        if ast.dump(self.target_tree) != ast.dump(self.source_tree):\n",
    "            self.log_tree(\"Source code to take repairs from:\", \n",
    "                          self.source_tree)\n",
    "\n",
    "        self.fitness_cache: Dict[str, float] = {}\n",
    "\n",
    "        self.mutator: StatementMutator = \\\n",
    "            mutator_class(\n",
    "                source=all_statements(self.source_tree),\n",
    "                suspiciousness_func=self.debugger.suspiciousness,\n",
    "                log=(self.log >= 3))\n",
    "        self.crossover: CrossoverOperator = crossover_class(log=(self.log >= 3))\n",
    "        self.reducer: DeltaDebugger = reducer_class(log=(self.log >= 3))\n",
    "\n",
    "        if globals is None:\n",
    "            globals = self.caller_globals()  # see below\n",
    "\n",
    "        self.globals = globals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "When we access or execute functions, we do so in the  caller's environment, not ours. The `caller_globals()` method from `StackInspector` acts as replacement for `globals()`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Helper Functions\n",
    "\n",
    "The constructor uses a number of helper functions to create its environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.434527Z",
     "iopub.status.busy": "2023-11-12T12:47:31.434393Z",
     "iopub.status.idle": "2023-11-12T12:47:31.436793Z",
     "shell.execute_reply": "2023-11-12T12:47:31.436406Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def getsource(self, item: Union[str, Any]) -> str:\n",
    "        \"\"\"Get the source for `item`. Can also be a string.\"\"\"\n",
    "\n",
    "        if isinstance(item, str):\n",
    "            item = self.globals[item]\n",
    "        return inspect.getsource(item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.438645Z",
     "iopub.status.busy": "2023-11-12T12:47:31.438514Z",
     "iopub.status.idle": "2023-11-12T12:47:31.441107Z",
     "shell.execute_reply": "2023-11-12T12:47:31.440768Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def default_functions(self) -> List[Callable]:\n",
    "        \"\"\"Return the set of functions to be repaired.\n",
    "        Functions whose names start or end in `test` are excluded.\"\"\"\n",
    "        def is_test(name: str) -> bool:\n",
    "            return name.startswith('test') or name.endswith('test')\n",
    "\n",
    "        return [func for func in self.debugger.covered_functions()\n",
    "                if not is_test(func.__name__)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.442752Z",
     "iopub.status.busy": "2023-11-12T12:47:31.442626Z",
     "iopub.status.idle": "2023-11-12T12:47:31.444799Z",
     "shell.execute_reply": "2023-11-12T12:47:31.444468Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def log_tree(self, description: str, tree: Any) -> None:\n",
    "        \"\"\"Print out `tree` as source code prefixed by `description`.\"\"\"\n",
    "        if self.log:\n",
    "            print(description)\n",
    "            print_content(ast.unparse(tree), '.py')\n",
    "            print()\n",
    "            print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.446722Z",
     "iopub.status.busy": "2023-11-12T12:47:31.446524Z",
     "iopub.status.idle": "2023-11-12T12:47:31.449619Z",
     "shell.execute_reply": "2023-11-12T12:47:31.449243Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def parse(self, items: List[Any]) -> ast.AST:\n",
    "        \"\"\"Read in a list of items into a single tree\"\"\"\n",
    "        tree = ast.parse(\"\")\n",
    "        for item in items:\n",
    "            if isinstance(item, str):\n",
    "                item = self.globals[item]\n",
    "\n",
    "            item_lines, item_first_lineno = inspect.getsourcelines(item)\n",
    "\n",
    "            try:\n",
    "                item_tree = ast.parse(\"\".join(item_lines))\n",
    "            except IndentationError:\n",
    "                # inner function or likewise\n",
    "                warnings.warn(f\"Can't parse {item.__name__}\")\n",
    "                continue\n",
    "\n",
    "            ast.increment_lineno(item_tree, item_first_lineno - 1)\n",
    "            tree.body += item_tree.body\n",
    "\n",
    "        return tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Running Tests\n",
    "\n",
    "Now that we have set the environment for `Repairer`, we can implement one step of automatic repair after the other. The method `run_test_set()` runs the given `test_set` (`DifferenceDebugger.PASS` or `DifferenceDebugger.FAIL`), returning the number of passed tests. If `validate` is set, it checks whether the outcomes are as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.451429Z",
     "iopub.status.busy": "2023-11-12T12:47:31.451326Z",
     "iopub.status.idle": "2023-11-12T12:47:31.455101Z",
     "shell.execute_reply": "2023-11-12T12:47:31.454773Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def run_test_set(self, test_set: str, validate: bool = False) -> int:\n",
    "        \"\"\"\n",
    "        Run given `test_set`\n",
    "        (`DifferenceDebugger.PASS` or `DifferenceDebugger.FAIL`).\n",
    "        If `validate` is set, check expectations.\n",
    "        Return number of passed tests.\n",
    "        \"\"\"\n",
    "        passed = 0\n",
    "        collectors = self.debugger.collectors[test_set]\n",
    "        function = self.debugger.function()\n",
    "        assert function is not None\n",
    "        # FIXME: function may have been redefined\n",
    "\n",
    "        for c in collectors:\n",
    "            if self.log >= 4:\n",
    "                print(f\"Testing {c.id()}...\", end=\"\")\n",
    "\n",
    "            try:\n",
    "                function(**c.args())\n",
    "            except Exception as err:\n",
    "                if self.log >= 4:\n",
    "                    print(f\"failed ({err.__class__.__name__})\")\n",
    "\n",
    "                if validate and test_set == self.debugger.PASS:\n",
    "                    raise err.__class__(\n",
    "                        f\"{c.id()} should have passed, but failed\")\n",
    "                continue\n",
    "\n",
    "            passed += 1\n",
    "            if self.log >= 4:\n",
    "                print(\"passed\")\n",
    "\n",
    "            if validate and test_set == self.debugger.FAIL:\n",
    "                raise FailureNotReproducedError(\n",
    "                    f\"{c.id()} should have failed, but passed\")\n",
    "\n",
    "        return passed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.456796Z",
     "iopub.status.busy": "2023-11-12T12:47:31.456665Z",
     "iopub.status.idle": "2023-11-12T12:47:31.458556Z",
     "shell.execute_reply": "2023-11-12T12:47:31.458187Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class FailureNotReproducedError(ValueError):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here is how we use `run_tests_set()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.460454Z",
     "iopub.status.busy": "2023-11-12T12:47:31.460331Z",
     "iopub.status.idle": "2023-11-12T12:47:31.464007Z",
     "shell.execute_reply": "2023-11-12T12:47:31.463598Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "repairer = Repairer(middle_debugger)\n",
    "assert repairer.run_test_set(middle_debugger.PASS) == \\\n",
    "    len(MIDDLE_PASSING_TESTCASES)\n",
    "assert repairer.run_test_set(middle_debugger.FAIL) == 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The method `run_tests()` runs passing and failing tests, weighing the passed test cases to obtain the overall fitness."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.465943Z",
     "iopub.status.busy": "2023-11-12T12:47:31.465830Z",
     "iopub.status.idle": "2023-11-12T12:47:31.468838Z",
     "shell.execute_reply": "2023-11-12T12:47:31.468510Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def weight(self, test_set: str) -> float:\n",
    "        \"\"\"\n",
    "        Return the weight of `test_set`\n",
    "        (`DifferenceDebugger.PASS` or `DifferenceDebugger.FAIL`).\n",
    "        \"\"\"\n",
    "        return {\n",
    "            self.debugger.PASS: WEIGHT_PASSING,\n",
    "            self.debugger.FAIL: WEIGHT_FAILING\n",
    "        }[test_set]\n",
    "\n",
    "    def run_tests(self, validate: bool = False) -> float:\n",
    "        \"\"\"Run passing and failing tests, returning weighted fitness.\"\"\"\n",
    "        fitness = 0.0\n",
    "\n",
    "        for test_set in [self.debugger.PASS, self.debugger.FAIL]:\n",
    "            passed = self.run_test_set(test_set, validate=validate)\n",
    "            ratio = passed / len(self.debugger.collectors[test_set])\n",
    "            fitness += self.weight(test_set) * ratio\n",
    "\n",
    "        return fitness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The method `validate()` ensures the observed tests can be adequately reproduced."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.470838Z",
     "iopub.status.busy": "2023-11-12T12:47:31.470571Z",
     "iopub.status.idle": "2023-11-12T12:47:31.472843Z",
     "shell.execute_reply": "2023-11-12T12:47:31.472514Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def validate(self) -> None:\n",
    "        fitness = self.run_tests(validate=True)\n",
    "        assert fitness == self.weight(self.debugger.PASS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.474527Z",
     "iopub.status.busy": "2023-11-12T12:47:31.474391Z",
     "iopub.status.idle": "2023-11-12T12:47:31.477651Z",
     "shell.execute_reply": "2023-11-12T12:47:31.477272Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "repairer = Repairer(middle_debugger)\n",
    "repairer.validate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### (Re)defining Functions\n",
    "\n",
    "Our `run_tests()` methods above do not yet redefine the function to be repaired. This is done by the `fitness()` function, which compiles and defines the given repair candidate `tree` before testing it.  It caches and returns the fitness."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.479415Z",
     "iopub.status.busy": "2023-11-12T12:47:31.479292Z",
     "iopub.status.idle": "2023-11-12T12:47:31.510332Z",
     "shell.execute_reply": "2023-11-12T12:47:31.509851Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def fitness(self, tree: ast.AST) -> float:\n",
    "        \"\"\"Test `tree`, returning its fitness\"\"\"\n",
    "        key = cast(str, ast.dump(tree))\n",
    "        if key in self.fitness_cache:\n",
    "            return self.fitness_cache[key]\n",
    "\n",
    "        # Save defs\n",
    "        original_defs: Dict[str, Any] = {}\n",
    "        for name in self.toplevel_defs(tree):\n",
    "            if name in self.globals:\n",
    "                original_defs[name] = self.globals[name]\n",
    "            else:\n",
    "                warnings.warn(f\"Couldn't find definition of {repr(name)}\")\n",
    "\n",
    "        assert original_defs, f\"Couldn't find any definition\"\n",
    "\n",
    "        if self.log >= 3:\n",
    "            print(\"Repair candidate:\")\n",
    "            print_content(ast.unparse(tree), '.py')\n",
    "            print()\n",
    "\n",
    "        # Create new definition\n",
    "        try:\n",
    "            code = compile(cast(ast.Module, tree), '<Repairer>', 'exec')\n",
    "        except ValueError:  # Compilation error\n",
    "            code = None\n",
    "\n",
    "        if code is None:\n",
    "            if self.log >= 3:\n",
    "                print(f\"Fitness = 0.0 (compilation error)\")\n",
    "\n",
    "            fitness = 0.0\n",
    "            return fitness\n",
    "\n",
    "        # Execute new code, defining new functions in `self.globals`\n",
    "        exec(code, self.globals)\n",
    "\n",
    "        # Set new definitions in the namespace (`__globals__`)\n",
    "        # of the function we will be calling.\n",
    "        function = self.debugger.function()\n",
    "        assert function is not None\n",
    "        assert hasattr(function, '__globals__')\n",
    "\n",
    "        for name in original_defs:\n",
    "            function.__globals__[name] = self.globals[name]  # type: ignore\n",
    "\n",
    "        fitness = self.run_tests(validate=False)\n",
    "\n",
    "        # Restore definitions\n",
    "        for name in original_defs:\n",
    "            function.__globals__[name] = original_defs[name]  # type: ignore\n",
    "            self.globals[name] = original_defs[name]\n",
    "\n",
    "        if self.log >= 3:\n",
    "            print(f\"Fitness = {fitness}\")\n",
    "\n",
    "        self.fitness_cache[key] = fitness\n",
    "        return fitness"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The helper function `toplevel_defs()` helps to save and restore the environment before and after redefining the function under repair."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.512529Z",
     "iopub.status.busy": "2023-11-12T12:47:31.512383Z",
     "iopub.status.idle": "2023-11-12T12:47:31.514807Z",
     "shell.execute_reply": "2023-11-12T12:47:31.514449Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def toplevel_defs(self, tree: ast.AST) -> List[str]:\n",
    "        \"\"\"Return a list of names of defined functions and classes in `tree`\"\"\"\n",
    "        visitor = DefinitionVisitor()\n",
    "        visitor.visit(tree)\n",
    "        assert hasattr(visitor, 'definitions')\n",
    "        return visitor.definitions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.516671Z",
     "iopub.status.busy": "2023-11-12T12:47:31.516525Z",
     "iopub.status.idle": "2023-11-12T12:47:31.519567Z",
     "shell.execute_reply": "2023-11-12T12:47:31.519160Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class DefinitionVisitor(NodeVisitor):\n",
    "    def __init__(self) -> None:\n",
    "        self.definitions: List[str] = []\n",
    "\n",
    "    def add_definition(self, node: Union[ast.ClassDef, \n",
    "                                         ast.FunctionDef, \n",
    "                                         ast.AsyncFunctionDef]) -> None:\n",
    "        self.definitions.append(node.name)\n",
    "\n",
    "    def visit_FunctionDef(self, node: ast.FunctionDef) -> None:\n",
    "        self.add_definition(node)\n",
    "\n",
    "    def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> None:\n",
    "        self.add_definition(node)\n",
    "\n",
    "    def visit_ClassDef(self, node: ast.ClassDef) -> None:\n",
    "        self.add_definition(node)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here's an example for `fitness()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.521649Z",
     "iopub.status.busy": "2023-11-12T12:47:31.521472Z",
     "iopub.status.idle": "2023-11-12T12:47:31.556373Z",
     "shell.execute_reply": "2023-11-12T12:47:31.556059Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target code to be repaired:\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "repairer = Repairer(middle_debugger, log=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.558382Z",
     "iopub.status.busy": "2023-11-12T12:47:31.558082Z",
     "iopub.status.idle": "2023-11-12T12:47:31.561591Z",
     "shell.execute_reply": "2023-11-12T12:47:31.561270Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.99"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "good_fitness = repairer.fitness(middle_tree())\n",
    "good_fitness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.563217Z",
     "iopub.status.busy": "2023-11-12T12:47:31.563092Z",
     "iopub.status.idle": "2023-11-12T12:47:31.565079Z",
     "shell.execute_reply": "2023-11-12T12:47:31.564613Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert good_fitness >= 0.99, \"fitness() failed\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.567002Z",
     "iopub.status.busy": "2023-11-12T12:47:31.566848Z",
     "iopub.status.idle": "2023-11-12T12:47:31.569872Z",
     "shell.execute_reply": "2023-11-12T12:47:31.569503Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4258"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bad_middle_tree = ast.parse(\"def middle(x, y, z): return x\")\n",
    "bad_fitness = repairer.fitness(bad_middle_tree)\n",
    "bad_fitness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.571538Z",
     "iopub.status.busy": "2023-11-12T12:47:31.571387Z",
     "iopub.status.idle": "2023-11-12T12:47:31.573240Z",
     "shell.execute_reply": "2023-11-12T12:47:31.572855Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert bad_fitness < 0.5, \"fitness() failed\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Repairing\n",
    "\n",
    "Now for the actual `repair()` method, which creates a `population` and then evolves it until the fitness is 1.0 or the given number of iterations is spent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.574922Z",
     "iopub.status.busy": "2023-11-12T12:47:31.574796Z",
     "iopub.status.idle": "2023-11-12T12:47:31.576549Z",
     "shell.execute_reply": "2023-11-12T12:47:31.576243Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import traceback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.578297Z",
     "iopub.status.busy": "2023-11-12T12:47:31.578142Z",
     "iopub.status.idle": "2023-11-12T12:47:31.582742Z",
     "shell.execute_reply": "2023-11-12T12:47:31.582403Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def initial_population(self, size: int) -> List[ast.AST]:\n",
    "        \"\"\"Return an initial population of size `size`\"\"\"\n",
    "        return [self.target_tree] + \\\n",
    "            [self.mutator.mutate(copy.deepcopy(self.target_tree))\n",
    "                for i in range(size - 1)]\n",
    "\n",
    "    def repair(self, population_size: int = POPULATION_SIZE, iterations: int = 100) -> \\\n",
    "        Tuple[ast.AST, float]:\n",
    "        \"\"\"\n",
    "        Repair the function we collected test runs from.\n",
    "        Use a population size of `population_size` and\n",
    "        at most `iterations` iterations.\n",
    "        Returns a pair (`ast`, `fitness`) where \n",
    "        `ast` is the AST of the repaired function, and\n",
    "        `fitness` is its fitness (between 0 and 1.0)\n",
    "        \"\"\"\n",
    "        self.validate()\n",
    "\n",
    "        population = self.initial_population(population_size)\n",
    "\n",
    "        last_key = ast.dump(self.target_tree)\n",
    "\n",
    "        for iteration in range(iterations):\n",
    "            population = self.evolve(population)\n",
    "\n",
    "            best_tree = population[0]\n",
    "            fitness = self.fitness(best_tree)\n",
    "\n",
    "            if self.log:\n",
    "                print(f\"Evolving population: \"\n",
    "                      f\"iteration{iteration:4}/{iterations} \"\n",
    "                      f\"fitness = {fitness:.5}   \\r\", end=\"\")\n",
    "\n",
    "            if self.log >= 2:\n",
    "                best_key = ast.dump(best_tree)\n",
    "                if best_key != last_key:\n",
    "                    print()\n",
    "                    print()\n",
    "                    self.log_tree(f\"New best code (fitness = {fitness}):\",\n",
    "                                  best_tree)\n",
    "                    last_key = best_key\n",
    "\n",
    "            if fitness >= 1.0:\n",
    "                break\n",
    "\n",
    "        if self.log:\n",
    "            print()\n",
    "\n",
    "        if self.log and self.log < 2:\n",
    "            self.log_tree(f\"Best code (fitness = {fitness}):\", best_tree)\n",
    "\n",
    "        best_tree = self.reduce(best_tree)\n",
    "        fitness = self.fitness(best_tree)\n",
    "\n",
    "        self.log_tree(f\"Reduced code (fitness = {fitness}):\", best_tree)\n",
    "\n",
    "        return best_tree, fitness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Evolving\n",
    "\n",
    "The evolution of our population takes place in the `evolve()` method. In contrast to the `evolve_middle()` function, above, we use crossover to create the offspring, which we still mutate afterwards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.584487Z",
     "iopub.status.busy": "2023-11-12T12:47:31.584369Z",
     "iopub.status.idle": "2023-11-12T12:47:31.587711Z",
     "shell.execute_reply": "2023-11-12T12:47:31.587385Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def evolve(self, population: List[ast.AST]) -> List[ast.AST]:\n",
    "        \"\"\"Evolve the candidate population by mutating and crossover.\"\"\"\n",
    "        n = len(population)\n",
    "\n",
    "        # Create offspring as crossover of parents\n",
    "        offspring: List[ast.AST] = []\n",
    "        while len(offspring) < n:\n",
    "            parent_1 = copy.deepcopy(random.choice(population))\n",
    "            parent_2 = copy.deepcopy(random.choice(population))\n",
    "            try:\n",
    "                self.crossover.crossover(parent_1, parent_2)\n",
    "            except CrossoverError:\n",
    "                pass  # Just keep parents\n",
    "            offspring += [parent_1, parent_2]\n",
    "\n",
    "        # Mutate offspring\n",
    "        offspring = [self.mutator.mutate(tree) for tree in offspring]\n",
    "\n",
    "        # Add it to population\n",
    "        population += offspring\n",
    "\n",
    "        # Keep the fitter part of the population\n",
    "        population.sort(key=self.fitness_key, reverse=True)\n",
    "        population = population[:n]\n",
    "\n",
    "        return population"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "A second difference is that we not only sort by fitness, but also by tree size – with equal fitness, a smaller tree thus will be favored. This helps keeping fixes and patches small."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.589627Z",
     "iopub.status.busy": "2023-11-12T12:47:31.589511Z",
     "iopub.status.idle": "2023-11-12T12:47:31.591893Z",
     "shell.execute_reply": "2023-11-12T12:47:31.591589Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def fitness_key(self, tree: ast.AST) -> Tuple[float, int]:\n",
    "        \"\"\"Key to be used for sorting the population\"\"\"\n",
    "        tree_size = len([node for node in ast.walk(tree)])\n",
    "        return (self.fitness(tree), -tree_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Simplifying\n",
    "\n",
    "The last step in repairing is simplifying the code. As demonstrated in the chapter on [reducing failure-inducing inputs](DeltaDebugger.ipynb), we can use delta debugging on code to get rid of superfluous statements. To this end, we convert the tree to lines, run delta debugging on them, and then convert it back to a tree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.593724Z",
     "iopub.status.busy": "2023-11-12T12:47:31.593588Z",
     "iopub.status.idle": "2023-11-12T12:47:31.596090Z",
     "shell.execute_reply": "2023-11-12T12:47:31.595773Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def reduce(self, tree: ast.AST) -> ast.AST:\n",
    "        \"\"\"Simplify `tree` using delta debugging.\"\"\"\n",
    "\n",
    "        original_fitness = self.fitness(tree)\n",
    "        source_lines = ast.unparse(tree).split('\\n')\n",
    "\n",
    "        with self.reducer:\n",
    "            self.test_reduce(source_lines, original_fitness)\n",
    "\n",
    "        reduced_lines = self.reducer.min_args()['source_lines']\n",
    "        reduced_source = \"\\n\".join(reduced_lines)\n",
    "\n",
    "        return ast.parse(reduced_source)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "As dicussed above, we simplify the code by having the test function (`test_reduce()`) declare reaching the maximum fitness obtained so far as a \"failure\". Delta debugging will then simplify the input as long as the \"failure\" (and hence the maximum fitness obtained) persists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.597692Z",
     "iopub.status.busy": "2023-11-12T12:47:31.597587Z",
     "iopub.status.idle": "2023-11-12T12:47:31.600537Z",
     "shell.execute_reply": "2023-11-12T12:47:31.600081Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Repairer(Repairer):\n",
    "    def test_reduce(self, source_lines: List[str], original_fitness: float) -> None:\n",
    "        \"\"\"Test function for delta debugging.\"\"\"\n",
    "\n",
    "        try:\n",
    "            source = \"\\n\".join(source_lines)\n",
    "            tree = ast.parse(source)\n",
    "            fitness = self.fitness(tree)\n",
    "            assert fitness < original_fitness\n",
    "\n",
    "        except AssertionError:\n",
    "            raise\n",
    "        except SyntaxError:\n",
    "            raise\n",
    "        except IndentationError:\n",
    "            raise\n",
    "        except Exception:\n",
    "            # traceback.print_exc()  # Uncomment to see internal errors\n",
    "            raise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### End of Excursion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Repairer in Action\n",
    "\n",
    "Let us go and apply `Repairer` in practice. We initialize it with `middle_debugger`, which has (still) collected the passing and failing runs for `middle_test()`. We also set `log` for some diagnostics along the way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.602271Z",
     "iopub.status.busy": "2023-11-12T12:47:31.602149Z",
     "iopub.status.idle": "2023-11-12T12:47:31.637191Z",
     "shell.execute_reply": "2023-11-12T12:47:31.636804Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target code to be repaired:\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "repairer = Repairer(middle_debugger, log=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We now invoke `repair()` to evolve our population. After a few iterations, we find a tree with perfect fitness."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:31.639051Z",
     "iopub.status.busy": "2023-11-12T12:47:31.638919Z",
     "iopub.status.idle": "2023-11-12T12:47:32.011547Z",
     "shell.execute_reply": "2023-11-12T12:47:32.011186Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evolving population: iteration   0/100 fitness = 1.0   \r\n",
      "Best code (fitness = 1.0):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m\n",
      "\n",
      "Reduced code (fitness = 1.0):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "best_tree, fitness = repairer.repair()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.013382Z",
     "iopub.status.busy": "2023-11-12T12:47:32.013252Z",
     "iopub.status.idle": "2023-11-12T12:47:32.058090Z",
     "shell.execute_reply": "2023-11-12T12:47:32.056010Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(best_tree), '.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.059869Z",
     "iopub.status.busy": "2023-11-12T12:47:32.059687Z",
     "iopub.status.idle": "2023-11-12T12:47:32.064150Z",
     "shell.execute_reply": "2023-11-12T12:47:32.063220Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fitness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.065892Z",
     "iopub.status.busy": "2023-11-12T12:47:32.065779Z",
     "iopub.status.idle": "2023-11-12T12:47:32.067936Z",
     "shell.execute_reply": "2023-11-12T12:47:32.067429Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert fitness >= 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Again, we have a perfect solution. Here, we did not even need to simplify the code in the last iteration, as our `fitness_key()` function favors smaller implementations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Removing HTML Markup\n",
    "\n",
    "Let us apply `Repairer` on our other ongoing example, namely `remove_html_markup()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.070620Z",
     "iopub.status.busy": "2023-11-12T12:47:32.070373Z",
     "iopub.status.idle": "2023-11-12T12:47:32.073543Z",
     "shell.execute_reply": "2023-11-12T12:47:32.072717Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def remove_html_markup(s):  # type: ignore\n",
    "    tag = False\n",
    "    quote = False\n",
    "    out = \"\"\n",
    "\n",
    "    for c in s:\n",
    "        if c == '<' and not quote:\n",
    "            tag = True\n",
    "        elif c == '>' and not quote:\n",
    "            tag = False\n",
    "        elif c == '\"' or c == \"'\" and tag:\n",
    "            quote = not quote\n",
    "        elif not tag:\n",
    "            out = out + c\n",
    "\n",
    "    return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.075260Z",
     "iopub.status.busy": "2023-11-12T12:47:32.075120Z",
     "iopub.status.idle": "2023-11-12T12:47:32.077644Z",
     "shell.execute_reply": "2023-11-12T12:47:32.076807Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def remove_html_markup_tree() -> ast.AST:\n",
    "    return ast.parse(inspect.getsource(remove_html_markup))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To run `Repairer` on `remove_html_markup()`, we need a test and a test suite. `remove_html_markup_test()` raises an exception if applying `remove_html_markup()` on the given `html` string does not yield the `plain` string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.079491Z",
     "iopub.status.busy": "2023-11-12T12:47:32.079362Z",
     "iopub.status.idle": "2023-11-12T12:47:32.081301Z",
     "shell.execute_reply": "2023-11-12T12:47:32.080989Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def remove_html_markup_test(html: str, plain: str) -> None:\n",
    "    outcome = remove_html_markup(html)\n",
    "    assert outcome == plain, \\\n",
    "        f\"Got {repr(outcome)}, expected {repr(plain)}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Now for the test suite. We use a simple fuzzing scheme to create dozens of passing and failing test cases in `REMOVE_HTML_PASSING_TESTCASES` and `REMOVE_HTML_FAILING_TESTCASES`, respectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Excursion: Creating HTML Test Cases"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.084070Z",
     "iopub.status.busy": "2023-11-12T12:47:32.083686Z",
     "iopub.status.idle": "2023-11-12T12:47:32.087717Z",
     "shell.execute_reply": "2023-11-12T12:47:32.087169Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def random_string(length: int = 5, start: int = ord(' '), end: int = ord('~')) -> str:\n",
    "    return \"\".join(chr(random.randrange(start, end + 1)) for i in range(length))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.089354Z",
     "iopub.status.busy": "2023-11-12T12:47:32.089227Z",
     "iopub.status.idle": "2023-11-12T12:47:32.091549Z",
     "shell.execute_reply": "2023-11-12T12:47:32.091160Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'@YeYg'"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_string()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.093772Z",
     "iopub.status.busy": "2023-11-12T12:47:32.093451Z",
     "iopub.status.idle": "2023-11-12T12:47:32.095739Z",
     "shell.execute_reply": "2023-11-12T12:47:32.095334Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def random_id(length: int = 2) -> str:\n",
    "    return random_string(start=ord('a'), end=ord('z'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.098366Z",
     "iopub.status.busy": "2023-11-12T12:47:32.098127Z",
     "iopub.status.idle": "2023-11-12T12:47:32.102215Z",
     "shell.execute_reply": "2023-11-12T12:47:32.100736Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'eaaem'"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_id()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.103812Z",
     "iopub.status.busy": "2023-11-12T12:47:32.103687Z",
     "iopub.status.idle": "2023-11-12T12:47:32.106267Z",
     "shell.execute_reply": "2023-11-12T12:47:32.105681Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def random_plain() -> str:\n",
    "    return random_string().replace('<', '').replace('>', '')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.112177Z",
     "iopub.status.busy": "2023-11-12T12:47:32.111743Z",
     "iopub.status.idle": "2023-11-12T12:47:32.115399Z",
     "shell.execute_reply": "2023-11-12T12:47:32.114848Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def random_string_noquotes() -> str:\n",
    "    return random_string().replace('\"', '').replace(\"'\", '')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.120556Z",
     "iopub.status.busy": "2023-11-12T12:47:32.120427Z",
     "iopub.status.idle": "2023-11-12T12:47:32.123554Z",
     "shell.execute_reply": "2023-11-12T12:47:32.122698Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def random_html(depth: int = 0) -> Tuple[str, str]:\n",
    "    prefix = random_plain()\n",
    "    tag = random_id()\n",
    "\n",
    "    if depth > 0:\n",
    "        html, plain = random_html(depth - 1)\n",
    "    else:\n",
    "        html = plain = random_plain()\n",
    "\n",
    "    attr = random_id()\n",
    "    value = '\"' + random_string_noquotes() + '\"'\n",
    "    postfix = random_plain()\n",
    "\n",
    "    return f'{prefix}<{tag} {attr}={value}>{html}</{tag}>{postfix}', \\\n",
    "        prefix + plain + postfix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.128653Z",
     "iopub.status.busy": "2023-11-12T12:47:32.128531Z",
     "iopub.status.idle": "2023-11-12T12:47:32.130597Z",
     "shell.execute_reply": "2023-11-12T12:47:32.130345Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('L1JA|<uidfm lmvie=\"QNSD:\">BvE\"8</uidfm>v@rNS', 'L1JA|BvE\"8v@rNS')"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_html()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.132224Z",
     "iopub.status.busy": "2023-11-12T12:47:32.132028Z",
     "iopub.status.idle": "2023-11-12T12:47:32.134095Z",
     "shell.execute_reply": "2023-11-12T12:47:32.133844Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def remove_html_testcase(expected: bool = True) -> Tuple[str, str]:\n",
    "    while True:\n",
    "        html, plain = random_html()\n",
    "        outcome = (remove_html_markup(html) == plain)\n",
    "        if outcome == expected:\n",
    "            return html, plain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.135543Z",
     "iopub.status.busy": "2023-11-12T12:47:32.135380Z",
     "iopub.status.idle": "2023-11-12T12:47:32.150920Z",
     "shell.execute_reply": "2023-11-12T12:47:32.150660Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "REMOVE_HTML_TESTS = 100\n",
    "REMOVE_HTML_PASSING_TESTCASES = \\\n",
    "    [remove_html_testcase(True) for i in range(REMOVE_HTML_TESTS)]\n",
    "REMOVE_HTML_FAILING_TESTCASES = \\\n",
    "    [remove_html_testcase(False) for i in range(REMOVE_HTML_TESTS)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### End of Excursion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here is a passing test case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.152456Z",
     "iopub.status.busy": "2023-11-12T12:47:32.152344Z",
     "iopub.status.idle": "2023-11-12T12:47:32.154512Z",
     "shell.execute_reply": "2023-11-12T12:47:32.154249Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('Sg$VT<fqlui ppzww=\"!EyHN\">J9Ji </fqlui>.)!$', 'Sg$VTJ9Ji .)!$')"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "REMOVE_HTML_PASSING_TESTCASES[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.155977Z",
     "iopub.status.busy": "2023-11-12T12:47:32.155838Z",
     "iopub.status.idle": "2023-11-12T12:47:32.157788Z",
     "shell.execute_reply": "2023-11-12T12:47:32.157519Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "html, plain = REMOVE_HTML_PASSING_TESTCASES[0]\n",
    "remove_html_markup_test(html, plain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here is a failing test case (containing a double quote in the plain text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.159398Z",
     "iopub.status.busy": "2023-11-12T12:47:32.159127Z",
     "iopub.status.idle": "2023-11-12T12:47:32.161329Z",
     "shell.execute_reply": "2023-11-12T12:47:32.161065Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('3AGe<qcguk yewyq=\"wA^<S\">7\"!%H</qcguk>6azh_', '3AGe7\"!%H6azh_')"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "REMOVE_HTML_FAILING_TESTCASES[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.162868Z",
     "iopub.status.busy": "2023-11-12T12:47:32.162763Z",
     "iopub.status.idle": "2023-11-12T12:47:32.164715Z",
     "shell.execute_reply": "2023-11-12T12:47:32.164469Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/var/folders/n2/xd9445p97rb3xh7m1dfx8_4h0006ts/T/ipykernel_33724/2578453007.py\", line 3, in <cell line: 1>\n",
      "    remove_html_markup_test(html, plain)\n",
      "  File \"/var/folders/n2/xd9445p97rb3xh7m1dfx8_4h0006ts/T/ipykernel_33724/700130947.py\", line 3, in remove_html_markup_test\n",
      "    assert outcome == plain, \\\n",
      "AssertionError: Got '3AGe7!%H</qcguk>6azh_', expected '3AGe7\"!%H6azh_' (expected)\n"
     ]
    }
   ],
   "source": [
    "with ExpectError():\n",
    "    html, plain = REMOVE_HTML_FAILING_TESTCASES[0]\n",
    "    remove_html_markup_test(html, plain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We run our tests, collecting the outcomes in `html_debugger`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.166102Z",
     "iopub.status.busy": "2023-11-12T12:47:32.165964Z",
     "iopub.status.idle": "2023-11-12T12:47:32.167541Z",
     "shell.execute_reply": "2023-11-12T12:47:32.167296Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "html_debugger = OchiaiDebugger()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.168917Z",
     "iopub.status.busy": "2023-11-12T12:47:32.168810Z",
     "iopub.status.idle": "2023-11-12T12:47:32.241995Z",
     "shell.execute_reply": "2023-11-12T12:47:32.241716Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "for html, plain in (REMOVE_HTML_PASSING_TESTCASES + \n",
    "                    REMOVE_HTML_FAILING_TESTCASES):\n",
    "    with html_debugger:\n",
    "        remove_html_markup_test(html, plain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The suspiciousness distribution will not be of much help here – pretty much all lines in `remove_html_markup()` have the same suspiciousness."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.243587Z",
     "iopub.status.busy": "2023-11-12T12:47:32.243490Z",
     "iopub.status.idle": "2023-11-12T12:47:32.305937Z",
     "shell.execute_reply": "2023-11-12T12:47:32.305618Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 1:  70%\">   1 def remove_html_markup_test(html: str, plain: str) -&gt; None:</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 2:  70%\">   2     outcome = remove_html_markup(html)</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 3:  70%\">   3     assert outcome == plain, \\</pre>\n",
       "<pre style=\"background-color:hsl(0.0, 100.0%, 80%)\"\n",
       "                    title=\"Line 4: 100%\">   4         f&quot;Got {repr(outcome)}, expected {repr(plain)}&quot;</pre>\n",
       "\n",
       "<p/><pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 1:  70%\">   1 def remove_html_markup(s):  # type: ignore</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 2:  70%\">   2     tag = False</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 3:  70%\">   3     quote = False</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 4:  70%\">   4     out = &quot;&quot;</pre>\n",
       "<pre title=\"Line 5: not executed\">   5 &nbsp;</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 6:  70%\">   6     for c in s:</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 7:  70%\">   7         if c == &#x27;&lt;&#x27; and not quote:</pre>\n",
       "<pre style=\"background-color:hsl(53.3989532967738, 100.0%, 80%)\"\n",
       "                    title=\"Line 8:  55%\">   8             tag = True</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 9:  70%\">   9         elif c == &#x27;&gt;&#x27; and not quote:</pre>\n",
       "<pre style=\"background-color:hsl(51.966394858339115, 100.0%, 80%)\"\n",
       "                    title=\"Line 10:  56%\">  10             tag = False</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 11:  70%\">  11         elif c == &#x27;&quot;&#x27; or c == &quot;&#x27;&quot; and tag:</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 12:  70%\">  12             quote = not quote</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 13:  70%\">  13         elif not tag:</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 14:  70%\">  14             out = out + c</pre>\n",
       "<pre title=\"Line 15: not executed\">  15 &nbsp;</pre>\n",
       "<pre style=\"background-color:hsl(35.14718625761431, 100.0%, 80%)\"\n",
       "                    title=\"Line 16:  70%\">  16     return out</pre>\n"
      ],
      "text/markdown": [
       "| `remove_html_markup_test` | `html='Sg$VT<fqlui ppzww=\"!EyHN\">J9Ji </fqlui>.)!$', plain='Sg$VTJ9Ji .)!$'` | `html='c[7dZ<kjbmr ojjtm=\"#=N)E\">C||=_</kjbmr>:IPwT', plain='c[7dZC||=_:IPwT'` | `html='?mkp5<bitdr vdkxj=\"sGTy8\">wc4VG</bitdr>p~(8;', plain='?mkp5wc4VGp~(8;'` | `html='kJ(^X<uogwr tfflc=\"j]TU&\">o1GvK</uogwr>-r=rx', plain='kJ(^Xo1GvK-r=rx'` | `html='\\'36)=<uzwbs jmmav=\"7b%,F\">z(s)H</uzwbs>$2!K5', plain=\"'36)=z(s)H$2!K5\"` | `html='N2_IR<woyih dnlcv=\"LGfRr\">bB*-</woyih>AfI!B', plain='N2_IRbB*-AfI!B'` | `html='pVJ.N<cngab wyijq=\"=j@MR\">sf|[</cngab>x;}5G', plain='pVJ.Nsf|[x;}5G'` | `html='GbMf1<jqfil qbyrm=\"z} a \">U9W7K</jqfil>0 RRn', plain='GbMf1U9W7K0 RRn'` | `html='+JMW-<giqyv xzhti=\"]uGa)\">[[kT{</giqyv>[(\\\\Y', plain='+JMW-[[kT{[(\\\\Y'` | `html='6dxT <gpmly eysfk=\"T|{gX\">f}b6?</gpmly>1N-2G', plain='6dxT f}b6?1N-2G'` | `html=';Uxz2<eqafg duhwy=\"|w<^^\">r}LpZ</eqafg>9DU;k', plain=';Uxz2r}LpZ9DU;k'` | `html='us;pj<eexqv vnduq=\"N506g\">g9H?V</eexqv>StQ{&', plain='us;pjg9H?VStQ{&'` | `html='UA4<pwjld aqpan=\"i$91a\">|^_na</pwjld>W]LK`', plain='UA4|^_naW]LK`'` | `html='ewF*0<ckqkj hdvzg=\"RzAB2\">|h0pI</ckqkj>4 J%L', plain='ewF*0|h0pI4 J%L'` | `html='RO9Ng<xxxki svzzi=\"pm, m\">1O58A</xxxki>[Dwc}', plain='RO9Ng1O58A[Dwc}'` | `html=';,EyY<uumwm bwxgm=\"Vh`t\">UYhzv</uumwm>1$yFX', plain=';,EyYUYhzv1$yFX'` | `html='jq^xl<sqxsf qutte=\"/w3UK\">?)hnU</sqxsf>pEV:U', plain='jq^xl?)hnUpEV:U'` | `html='GU^M5<jcedu jhamv=\"&P\\\\!$\">k/!E?</jcedu>+03,e', plain='GU^M5k/!E?+03,e'` | `html='O!nl4<fesqz bsrop=\";zA}<\">=eXxN</fesqz>fNF7', plain='O!nl4=eXxNfNF7'` | `html='S0iSL<drjlz zkpqb=\")uR=\">5$Djr</drjlz>0xmUJ', plain='S0iSL5$Djr0xmUJ'` | `html='`J,6~<sbpak xfqcu=\"?Y&75\">st_/1</sbpak>?9vT', plain='`J,6~st_/1?9vT'` | `html='7b~7?<pfikt tecvj=\"Mlj.E\">]z@M\\'</pfikt>czAPi', plain=\"7b~7?]z@M'czAPi\"` | `html='FdTLX<lcytb ivmcn=\"IJyW+\">-`/`^</lcytb>.Nq$@', plain='FdTLX-`/`^.Nq$@'` | `html='Q|tMF<wiwsx skvou=\"5(\\\\Ff\">I)5&C</wiwsx>??r`(', plain='Q|tMFI)5&C??r`('` | `html='G6/=\\'<ifmez nhedo=\"ro1-v\">JEC</ifmez>)zgh)', plain=\"G6/='JEC)zgh)\"` | `html='=-(.8<rafdr edphw=\"S\\\\0Ng\">)Qr+3</rafdr>G;pNr', plain='=-(.8)Qr+3G;pNr'` | `html='[$QM<thdrz uiyct=\"JB|R\">}kc?/</thdrz>zec2j', plain='[$QM}kc?/zec2j'` | `html='U|LON<fyhca lubfb=\"7\\\\$Z\">/dC# </fyhca>b\\\\7Y', plain='U|LON/dC# b\\\\7Y'` | `html='G]{`6<kbtfr xvzyz=\";9,3\">|Q)Q</kbtfr>x.KHF', plain='G]{`6|Q)Qx.KHF'` | `html='9_/q^<yazuv kghwy=\"sh0?[\">Cxk+w</yazuv>lgC[{', plain='9_/q^Cxk+wlgC[{'` | `html='y3r&H<vtyxy liuxn=\"z7sA#\">a/u2Z</vtyxy> B%kV', plain='y3r&Ha/u2Z B%kV'` | `html='J3VCv<atnzq wyczb=\"enwa\">mS3mS</atnzq>otN0', plain='J3VCvmS3mSotN0'` | `html='\\\\m{Z6<uhzyk laoyr=\"du$zc\">lgztl</uhzyk>L{TF', plain='\\\\m{Z6lgztlL{TF'` | `html='55x4i<xzgcn lscjf=\"t2><=\">He*R:</xzgcn>;)Xb', plain='55x4iHe*R:;)Xb'` | `html='3*sMU<quoap jashm=\"6K_b1\">.hx@b</quoap>d-#p ', plain='3*sMU.hx@bd-#p '` | `html='bFJH7<wepdv ubwpe=\"#A*i`\">,kWy1</wepdv>V*7a4', plain='bFJH7,kWy1V*7a4'` | `html='\\'I,!k<zrfju usqiz=\"Ir3>b\">KEK(b</zrfju>WLCLG', plain=\"'I,!kKEK(bWLCLG\"` | `html='0J3e<sycax enwwk=\"s_6M\\\\\">TXP11</sycax>)rn|%', plain='0J3eTXP11)rn|%'` | `html='RfVB!<jilbe yybau=\".*-zZ\">Ld9Db</jilbe>.8D/', plain='RfVB!Ld9Db.8D/'` | `html='im/RZ<yyifm jgmez=\"t]S&p\">] u8(</yyifm>h3JF9', plain='im/RZ] u8(h3JF9'` | `html='G{jv0<hkvzn qugwv=\"Lfghu\">r32Ti</hkvzn>oa0$F', plain='G{jv0r32Tioa0$F'` | `html='Im#f.<ooxxe jiowc=\"qo2U(\">)QY/f</ooxxe>F]m`6', plain='Im#f.)QY/fF]m`6'` | `html='v/{0o<zugde xzdcd=\")82Vr\">by|@,</zugde>!\\'6#', plain=\"v/{0oby|@,!'6#\"` | `html='h{-VT<qrpzt tlvrn=\"~tyfl\"> 1wQa</qrpzt>vu|bz', plain='h{-VT 1wQavu|bz'` | `html='9t(c<dmwua rmrpg=\"Izs%\">1!&RQ</dmwua>ZH#la', plain='9t(c1!&RQZH#la'` | `html='$p.)7<adxlo suixv=\"qx!9!\">;];`N</adxlo>5*Nu/', plain='$p.)7;];`N5*Nu/'` | `html='ML=)a<mvzqb jufqz=\".y2J(\">a_l\\'E</mvzqb>P9{|-', plain=\"ML=)aa_l'EP9{|-\"` | `html='LGTG}<wcknz erjya=\"5Im;M\">A]j`?</wcknz>5{,^X', plain='LGTG}A]j`?5{,^X'` | `html='m,iUT<lmxug cotio=\"oAH0S\">\\'jOzM</lmxug>iT2=H', plain=\"m,iUT'jOzMiT2=H\"` | `html='9H%6!<vlece syoyv=\"JJ$}=\">|zx6m</vlece>~7GP;', plain='9H%6!|zx6m~7GP;'` | `html='MFr*8<rdofc kizhu=\"dz= X\">C; F-</rdofc>ZQSGd', plain='MFr*8C; F-ZQSGd'` | `html='F.8)G<vpjpe fahhu=\"fOL+g\">mn.V}</vpjpe>-uk /', plain='F.8)Gmn.V}-uk /'` | `html='Kxp=_<lcfud ovkdu=\"hW$[y\">i!_G^</lcfud>+SDm.', plain='Kxp=_i!_G^+SDm.'` | `html='k]w\\'`<izliw nrmsr=\"e,Dc>\">DIho`</izliw>&fD(N', plain=\"k]w'`DIho`&fD(N\"` | `html='h~k)7<wukfd gmowo=\",\\\\pFE\">wIh!b</wukfd>\\\\D:|d', plain='h~k)7wIh!b\\\\D:|d'` | `html='(jj@9<dvztv bfxni=\"xAG}S\">HRScI</dvztv>g}9=m', plain='(jj@9HRScIg}9=m'` | `html='rrCyc<dzcti oieth=\"1/8@c\">#q_CM</dzcti>#)`j,', plain='rrCyc#q_CM#)`j,'` | `html='r_l!Y<zownz evjyz=\"l5LhD\">bL/CV</zownz>vkSQ', plain='r_l!YbL/CVvkSQ'` | `html='^*G6q<dheex dpqdi=\"@tU;E\">Uv|cA</dheex>h8jqE', plain='^*G6qUv|cAh8jqE'` | `html='Ta;mM<fqomu utvjj=\"hUlGK\">&#}qy</fqomu>;Jr])', plain='Ta;mM&#}qy;Jr])'` | `html='R^C-p<vgmzw nkpsg=\"+|E6B\">LRR:6</vgmzw>#sS1C', plain='R^C-pLRR:6#sS1C'` | `html='(6`[&<uqxgg ydbjq=\"!WL\\\\?\">|@@W}</uqxgg>$j8-s', plain='(6`[&|@@W}$j8-s'` | `html='_YB!;<ougfv taefy=\"hT%Ca\">SF:sT</ougfv>U^GK8', plain='_YB!;SF:sTU^GK8'` | `html='#ancJ<eqlda iamvx=\"ZL=g&\">$}b\\'A</eqlda>b4NB]', plain=\"#ancJ$}b'Ab4NB]\"` | `html='S^foP<xxald sonjk=\"kV^j=\">5 T-=</xxald>sszJb', plain='S^foP5 T-=sszJb'` | `html='HJ6Jy<zzwwx oqezq=\"WWJxx\">85ZAn</zzwwx>%h&d#', plain='HJ6Jy85ZAn%h&d#'` | `html='7.:z^<ydzju mgofm=\"u;X~~\">B|&0S</ydzju>~o2j', plain='7.:z^B|&0S~o2j'` | `html='(0:$I<zmxfn gyixx=\"b.mua\">UFMdG</zmxfn>um?jU', plain='(0:$IUFMdGum?jU'` | `html='PdnU~<fektq slnhn=\"9[&5\">&/;7X</fektq>\\'r,Tv', plain=\"PdnU~&/;7X'r,Tv\"` | `html='P`,Jz<zrcql qyjwg=\"3Pr(p\">3&9)K</zrcql>{WLp5', plain='P`,Jz3&9)K{WLp5'` | `html='/8BmY<djqil qgctb=\" QDuP\">b@7Q9</djqil>.dpu]', plain='/8BmYb@7Q9.dpu]'` | `html='.n:f;<lktiw eabzv=\"9??+E\">gYJsw</lktiw>j:VO&', plain='.n:f;gYJswj:VO&'` | `html='TPUiP<kxant nugjy=\"Xag_S\">;h^D0</kxant>YR[j', plain='TPUiP;h^D0YR[j'` | `html='@u04r<ykufs kipki=\"Gj&%B\">SycYF</ykufs>n9[v8', plain='@u04rSycYFn9[v8'` | `html='k1mAe<wyoxb ajbhj=\"8JIK7\">XJ/vM</wyoxb>O4\\'\\\\#', plain=\"k1mAeXJ/vMO4'\\\\#\"` | `html='[JT7\\\\<cdwqx necac=\"E&[T`\">(;)B2</cdwqx>0l0K', plain='[JT7\\\\(;)B20l0K'` | `html='Q-E\\\\@<qgopk vpxme=\"B:Y@|\">4chbS</qgopk>O.%h', plain='Q-E\\\\@4chbSO.%h'` | `html='`8]zY<zloti egnyg=\"%X`!Q\">OvIV#</zloti>^SYN', plain='`8]zYOvIV#^SYN'` | `html='7T \\'Z<jiyqc vnaqy=\"}J<-l\">!yMBu</jiyqc>Fem~o', plain=\"7T 'Z!yMBuFem~o\"` | `html='eTqeE<wxjiy hztby=\"8&vsp\">7oMP8</wxjiy>CCw#o', plain='eTqeE7oMP8CCw#o'` | `html='=;FK6<damej qdikk=\"+4%fJ\">@KaJr</damej>e7@B;', plain='=;FK6@KaJre7@B;'` | `html='AbZXc<xrerc hgamj=\"};1b4\">KI6x</xrerc>C0y1z', plain='AbZXcKI6xC0y1z'` | `html='de G]<fedxb trdwh=\".633?\">F[.FZ</fedxb>C0aWH', plain='de G]F[.FZC0aWH'` | `html='(8wa`<wuooo lyavf=\".>~aB\">LuxO!</wuooo>px+|_', plain='(8wa`LuxO!px+|_'` | `html='5Ub;j<lqecd uhwdh=\"Bs>x\">*+^Y</lqecd>ENJ{5', plain='5Ub;j*+^YENJ{5'` | `html='kvoFC<ggvds mmwkl=\"@-)zp\">x+`OB</ggvds>t~ZR=', plain='kvoFCx+`OBt~ZR='` | `html='CTIUH<xwihb wyexf=\"TRB[<\">szx.</xwihb>C]zno', plain='CTIUHszx.C]zno'` | `html='+v)#7<yqreg aviqz=\"N<V.\">]OqTr</yqreg>J~7y ', plain='+v)#7]OqTrJ~7y '` | `html='5:]W!<wwfbc zeuyu=\"`c}%C\">\\\\b;!&</wwfbc>:j$?.', plain='5:]W!\\\\b;!&:j$?.'` | `html='My$$H<wawvn swetg=\"Zt,j\\\\\">O#=h</wawvn>F`NvE', plain='My$$HO#=hF`NvE'` | `html='K(]1X<tfmqe iqdyy=\"wL`>2\">j7l9 </tfmqe>c[fJ2', plain='K(]1Xj7l9 c[fJ2'` | `html='JX7aS<zdxma ggegy=\"+>NS.\">]ea$</zdxma>xl[/0', plain='JX7aS]ea$xl[/0'` | `html='*TH?M<ttoum fmebh=\"[({Ms\">l\\\\4_</ttoum>:w3MB', plain='*TH?Ml\\\\4_:w3MB'` | `html='cd ~)<gxqgl sgfbe=\"9]h|7\">tPG{p</gxqgl>$X1cz', plain='cd ~)tPG{p$X1cz'` | `html='w^1X*<mdkfg tyesd=\"7dz-u\">BD)Q:</mdkfg>Vj9~n', plain='w^1X*BD)Q:Vj9~n'` | `html='c+k&<wpbyj zwklz=\"<;7A7\">}xR4Z</wpbyj>28})F', plain='c+k&}xR4Z28})F'` | `html='6t0qe<ktswq oqjym=\"uHV?*\">J;L`]</ktswq>A\\'8y', plain=\"6t0qeJ;L`]A'8y\"` | `html='r7+,]<xttwg oidet=\"k*C*l\">*N*E0</xttwg>yvzxU', plain='r7+,]*N*E0yvzxU'` | `html='#6(RD<uanco lwzif=\"lH>p\">.0puq</uanco>ykQ^/', plain='#6(RD.0puqykQ^/'` | `html='N&p:<xdcfb fvaec=\"Z?Qt6\">i+-..</xdcfb>ZaN*%', plain='N&p:i+-..ZaN*%'` | `html='3AGe<qcguk yewyq=\"wA^<S\">7\"!%H</qcguk>6azh_', plain='3AGe7\"!%H6azh_'` | `html='1qCi2<bhidt hobxo=\"NP+M)\">H\"\\\\m~</bhidt>#yi\"(', plain='1qCi2H\"\\\\m~#yi\"('` | `html='{]#_j<jphtd jheqj=\"!1@*d\">2Ha6a</jphtd>#]o|\"', plain='{]#_j2Ha6a#]o|\"'` | `html='::%hZ<yclyt zpexk=\"ZK|.=\">.8Nb4</yclyt>\"z5t7', plain='::%hZ.8Nb4\"z5t7'` | `html='A#ZLz<pjfpe xeldg=\"_~Umr\">hZ9}d</pjfpe>\"&E22', plain='A#ZLzhZ9}d\"&E22'` | `html='mzd%4<ndtob edpfx=\"!FV1m\">@e+-a</ndtob>bZGP\"', plain='mzd%4@e+-abZGP\"'` | `html='T4*dS<hgxqg tihiv=\";eh0\">/RO{\"</hgxqg>@G#v', plain='T4*dS/RO{\"@G#v'` | `html='nGbOR<fipdc sjnqk=\"M@Jw\">FBFO[</fipdc>@D\\\\\"`', plain='nGbORFBFO[@D\\\\\"`'` | `html='9?c3;<ccpms mtbrh=\"!EO^1\">fq\"Mv</ccpms>s&fr7', plain='9?c3;fq\"Mvs&fr7'` | `html='\"Y^0?<tqfen yuzeb=\"b}g[u\">-1XPj</tqfen>y04ex', plain='\"Y^0?-1XPjy04ex'` | `html='fhj\"x<zxkoe eykdh=\"]c{F+\">GrOt3</zxkoe>0=9!r', plain='fhj\"xGrOt30=9!r'` | `html='0m@U|<rkhcu hbsyk=\"T*/sT\">9D_7r</rkhcu>u5O\"}', plain='0m@U|9D_7ru5O\"}'` | `html='dKO\"j<mcvki gxqgc=\"JF^P1\">De6KM</mcvki>hmAFC', plain='dKO\"jDe6KMhmAFC'` | `html='QI8%`<qypvu xtkso=\"bFLjI\">%},-\"</qypvu>\\\\%MC0', plain='QI8%`%},-\"\\\\%MC0'` | `html='v|~X\"<ybebk mizjl=\"pWzw\">kmjI|</ybebk>jGETx', plain='v|~X\"kmjI|jGETx'` | `html='QcuA\"<uukns yyfmt=\"M-:Z%\">dE8j3</uukns>T1-xZ', plain='QcuA\"dE8j3T1-xZ'` | `html='D.}Jm<wcgnl ytqxl=\"t?:V*\">@\"u{I</wcgnl>_12C ', plain='D.}Jm@\"u{I_12C '` | `html='Z\"Wls<qvqik stiup=\"u/@BX\">,T(NX</qvqik>`hlE\\\\', plain='Z\"Wls,T(NX`hlE\\\\'` | `html='&R&\\\\f<itwyt kvpvc=\"tyqIo\">=kg*P</itwyt>*\"!3-', plain='&R&\\\\f=kg*P*\"!3-'` | `html='5J;dD<moqgp oulkg=\"-^i,L\">#8coX</moqgp>oE \";', plain='5J;dD#8coXoE \";'` | `html='pS:d\\'<osobk tvydx=\"zi70d\">arbC.</osobk>Xk3\"u', plain='pS:d\\'arbC.Xk3\"u'` | `html='`lSot<nntrj rireu=\"g>[\\\\q\">|b{kG</nntrj>\"\"]yi', plain='`lSot|b{kG\"\"]yi'` | `html='ry`::<icbim nmffu=\"}`3b|\">@k*xL</icbim>@v\"H6', plain='ry`::@k*xL@v\"H6'` | `html='W76\\\\ <olraq rsvga=\"6B9 %\">=mdG7</olraq>}&\"Jo', plain='W76\\\\ =mdG7}&\"Jo'` | `html='`Hofd<vhjkf fyqyq=\"$Qay&\">]v\"gf</vhjkf>t?,9*', plain='`Hofd]v\"gft?,9*'` | `html='lQs3V<teqpt zlngs=\"QqNuX\">\"OfSM</teqpt>OboL&', plain='lQs3V\"OfSMOboL&'` | `html='\"Pjqq<sxbpi ccyko=\"H2}_y\">LC712</sxbpi>k@_?)', plain='\"PjqqLC712k@_?)'` | `html='h9tH*<dggpb nvpwb=\"o3>B\">.NwH$</dggpb>`Kz8\"', plain='h9tH*.NwH$`Kz8\"'` | `html='QL\" Z<nkebi fjsar=\"k<fJ-\">w,-,,</nkebi>u8d-7', plain='QL\" Zw,-,,u8d-7'` | `html=';bHXr<xndoo dvtkl=\"SrESZ\">tM2__</xndoo>Y8\"Ec', plain=';bHXrtM2__Y8\"Ec'` | `html=';3\\\\Zv<yxnnh cnybv=\"[s8iZ\">B\"T\\'Z</yxnnh>LfSg)', plain=';3\\\\ZvB\"T\\'ZLfSg)'` | `html='G-V,8<mwqvg avzmt=\"+%2c \">n\"jh?</mwqvg>57I@Z', plain='G-V,8n\"jh?57I@Z'` | `html='V#,UP<pruja rqxub=\"n|ykj\">\"7W,/</pruja>|F;5\\'', plain='V#,UP\"7W,/|F;5\\''` | `html='ALm(K<viosr qqotc=\"hw;US\">\"OMfG</viosr>~V#{y', plain='ALm(K\"OMfG~V#{y'` | `html='6\"{q%<ktgcr xgpfp=\"dCSOx\">2b@hG</ktgcr>.FL9D', plain='6\"{q%2b@hG.FL9D'` | `html='QnX\"v<eagdt uewtq=\",3gbe\">XfOYu</eagdt>MBLs5', plain='QnX\"vXfOYuMBLs5'` | `html='p)*6o<jzxlr twojg=\"@tn4M\">\"J&h</jzxlr>EfU =', plain='p)*6o\"J&hEfU ='` | `html=', /\"&<eppqq hbctt=\"$h$~w\">5S-pd</eppqq>G9G5', plain=', /\"&5S-pdG9G5'` | `html='1OCA^<lslup rwxly=\"gqb$+\">h#c\"K</lslup>*QN\\\\', plain='1OCA^h#c\"K*QN\\\\'` | `html='R6\",q<vxwbu feqiy=\";yZg7\">,\\'qx9</vxwbu>-_s+a', plain='R6\",q,\\'qx9-_s+a'` | `html='q7=h\"<mazcs hwcol=\"3vYp\">b(Bn~</mazcs>5d,:r', plain='q7=h\"b(Bn~5d,:r'` | `html='686D<fqtlp belqc=\"jv4R6\">5X,m\"</fqtlp>Oh`kb', plain='686D5X,m\"Oh`kb'` | `html='\"urp3<qzdud chzlf=\">5e\\\\L\">ew^7F</qzdud>?kHll', plain='\"urp3ew^7F?kHll'` | `html='1FH?P<wkuio mirol=\"pd9-8\">7+f(\"</wkuio>/919j', plain='1FH?P7+f(\"/919j'` | `html=';-E|0<vxuxu jwtqh=\"GCPbF\">5e\"@$</vxuxu>e-\\'@h', plain=';-E|05e\"@$e-\\'@h'` | `html='4y;9^<yqtzq balnq=\"{^Nld\">:z T\"</yqtzq>Z;+=q', plain='4y;9^:z T\"Z;+=q'` | `html='V,!t<xmyrn hjpns=\"17A/c\">=D\"N7</xmyrn>iEjb7', plain='V,!t=D\"N7iEjb7'` | `html='tJ\"yO<lmyni orwqm=\"faFCd\">O.N-</lmyni>~38:P', plain='tJ\"yOO.N-~38:P'` | `html='5_\"@q<bzshz qlmim=\"]~xS\">w&gx(</bzshz>j4.$2', plain='5_\"@qw&gx(j4.$2'` | `html='tpJxZ<ulnsk uatqc=\"D72C!\">pom\"9</ulnsk>&8tFr', plain='tpJxZpom\"9&8tFr'` | `html='\"+rJK<hmzxp lufht=\"^l][v\">Wg`Lr</hmzxp>9f-0', plain='\"+rJKWg`Lr9f-0'` | `html='1*vJ&<vosvy pbuxt=\"i-jo8\">\"5sBm</vosvy>xf_g]', plain='1*vJ&\"5sBmxf_g]'` | `html='jjcmn<xeuza neyus=\"uQMYI\">vq0-\"</xeuza>dC\\'|;', plain='jjcmnvq0-\"dC\\'|;'` | `html='H8Lf1<twpvu iqekq=\"5oiq5\">&(DeI</twpvu>OB0\"/', plain='H8Lf1&(DeIOB0\"/'` | `html=',l4K<basmc bmymn=\"==@`*\">woG%\"</basmc>w_9\\\\=', plain=',l4KwoG%\"w_9\\\\='` | `html='W5c]h<ajxfb jkqnu=\"/dn##\">\"I{h9</ajxfb>cmCFS', plain='W5c]h\"I{h9cmCFS'` | `html='3#(2 <wjcrf jywhc=\"i_D>(\">.EoiN</wjcrf>vUc#\"', plain='3#(2 .EoiNvUc#\"'` | `html='F\"C1V<xmsfz tqhqv=\"Z`o6j\">Ptkkn</xmsfz>d^m4,', plain='F\"C1VPtkknd^m4,'` | `html='3bwoO<ypwuj hnenh=\"}>%%`\">F1BQ\"</ypwuj>7=9Vd', plain='3bwoOF1BQ\"7=9Vd'` | `html='D1F-+<eipxm fzkng=\"=Ch2}\">\\\\cVX`</eipxm>7t:\"D', plain='D1F-+\\\\cVX`7t:\"D'` | `html='.[?=!<yttbl qdwlo=\";TSYW\">4i\"-{</yttbl>{11np', plain='.[?=!4i\"-{{11np'` | `html=')cXlL<qqxlm hicza=\".E.|%\">Q9\"\\'\\'</qqxlm>LRev1', plain=')cXlLQ9\"\\'\\'LRev1'` | `html='@|8Ft<knbnp btbbc=\"&l8Vp\">\\'D\"z~</knbnp>Ol%#4', plain='@|8Ft\\'D\"z~Ol%#4'` | `html='PR\"Q9<rdxmm oxyqx=\"A`8.c\">vxT`6</rdxmm>1{pVe', plain='PR\"Q9vxT`61{pVe'` | `html='\"j5?F<bwlrc dyfms=\"LF3%7\">Dp8R7</bwlrc>qQz^?', plain='\"j5?FDp8R7qQz^?'` | `html=']L{V^<mbhdt yuwiu=\"^;x.J\">MkEVO</mbhdt>]\"G@Y', plain=']L{V^MkEVO]\"G@Y'` | `html='DN@\\'B<perzp mutsb=\"#_?*n\">31xWp</perzp>:F^.\"', plain='DN@\\'B31xWp:F^.\"'` | `html='xKRw9<hdqei bkodg=\"s]thu\">qTs\"l</hdqei>1vg+g', plain='xKRw9qTs\"l1vg+g'` | `html='y+BtE<aszxk eernd=\"pt~5\">|By/e</aszxk>U\\\\C\"W', plain='y+BtE|By/eU\\\\C\"W'` | `html='/j{m\"<saxwi xpkwf=\"xr9`6\">R@,+</saxwi>.195E', plain='/j{m\"R@,+.195E'` | `html='cv\"V9<dzglv oweny=\";V@*.\">\\\\,*BD</dzglv>5a^&(', plain='cv\"V9\\\\,*BD5a^&('` | `html='\"sy;6<goomo piyjv=\"!2*,R\">4o]T~</goomo>/tI~ ', plain='\"sy;64o]T~/tI~ '` | `html='1_ 5U<rasqu cxeuv=\"I*90/\">jM]`F</rasqu>\"ZsqZ', plain='1_ 5UjM]`F\"ZsqZ'` | `html='%\\'NeY<abels mfwvr=\"#w{L]\">\"yxnc</abels>{pv|?', plain='%\\'NeY\"yxnc{pv|?'` | `html='n{vAX<hiwvo nklxr=\"L@G$2\">PMFNv</hiwvo>k\"ut@', plain='n{vAXPMFNvk\"ut@'` | `html='Nm]M_<ziucq tdvuk=\"!dJ[I\">Pm__|</ziucq>$\"71', plain='Nm]M_Pm__|$\"71'` | `html='[Tst<bkoty ckgnx=\"[m34;\">\"*=.@</bkoty>b%0Ox', plain='[Tst\"*=.@b%0Ox'` | `html='P+\"~V<iqgkh mvirm=\"?BD\\\\m\">LteI3</iqgkh>*Wp\\\\[', plain='P+\"~VLteI3*Wp\\\\['` | `html='g^1:y<cxzbf nclyo=\"5~Ww/\">U6-);</cxzbf>I*\"{a', plain='g^1:yU6-);I*\"{a'` | `html='\\'\"x~~<ykxfm bovas=\"41p=E\">[Vk@v</ykxfm>1[\"2a', plain='\\'\"x~~[Vk@v1[\"2a'` | `html=']Y(+H<izkdq hzxfb=\"@9YTQ\">AJ\"@G</izkdq>maR 8', plain=']Y(+HAJ\"@GmaR 8'` | `html='FsWD,<mgxok kvhut=\"Z&97m\">{&jCh</mgxok>qm\"49', plain='FsWD,{&jChqm\"49'` | `html=',?k%C<olhrw cxrdq=\"973A2\">Li\"=i</olhrw>/NHm/', plain=',?k%CLi\"=i/NHm/'` | `html='.fE[<rvxtl gnlyd=\"IBpJI\">/:5*f</rvxtl>qHJ\"(', plain='.fE[/:5*fqHJ\"('` | `html='K*Xht<rwboe mnrsw=\"2_?20\">im\"9^</rwboe>K+p2c', plain='K*Xhtim\"9^K+p2c'` | `html='i\"[c6<cvfea jdfsh=\">J?3Z\">^zs=x</cvfea>vo1Oz', plain='i\"[c6^zs=xvo1Oz'` | `html='\\\\iBTt<rljvk kaebh=\"mll!p\">#S\")8</rljvk>{%uOf', plain='\\\\iBTt#S\")8{%uOf'` | `html='84BNw<tzavb supxl=\"la#4O\">qkPO\"</tzavb>Y@Nbj', plain='84BNwqkPO\"Y@Nbj'` | `html='x3nG&<uahfg oevxe=\"N{IJ\">Ti\"wv</uahfg>]:Zhx', plain='x3nG&Ti\"wv]:Zhx'` | `html='ap!48<edecz krshx=\"pjG6i\">,?[}U</edecz>\"1OT3', plain='ap!48,?[}U\"1OT3'` | `html='+@s\"o<eokjc ublrk=\" #zm~\">lO_:}</eokjc>q/f}3', plain='+@s\"olO_:}q/f}3'` | `html='o+=[y<rmczx fvpap=\"b^q;;\">n=S&\"</rmczx>@PO~M', plain='o+=[yn=S&\"@PO~M'` | `html='C$ZQ<nryuk cjaal=\"9niYA\">Gw!Xb</nryuk>]G)\"~', plain='C$ZQGw!Xb]G)\"~'` | `html='V\\\\\"S<vgpqp hczle=\"g7U>R\">[Q~_</vgpqp>!W6?i', plain='V\\\\\"S[Q~_!W6?i'` | `html='|^J5o<zqugo xyqzn=\"ef=+q\">{Y\"#f</zqugo> giGB', plain='|^J5o{Y\"#f giGB'` | `html='Y7Wl[<bbnay ubreh=\"{9p}\">]\"|b|</bbnay>_0IW;', plain='Y7Wl[]\"|b|_0IW;'` | `html='XU\\'Wo<mtieb qrxzi=\"Yt)Pq\">\"\\\\n2[</mtieb>RwGQh', plain='XU\\'Wo\"\\\\n2[RwGQh'` | `html='\"a\"U{<cwfpc pugkm=\"`:yWp\">e\"J{{</cwfpc>LDMuI', plain='\"a\"U{e\"J{{LDMuI'` | `html='|vT#y<cdruy cmhmd=\"~j=8\">tc2*.</cdruy>\"u.Q', plain='|vT#ytc2*.\"u.Q'` | `html='O8#k:<kgloc flioj=\"HMB^\">jL(u\"</kgloc>y~;uf', plain='O8#k:jL(u\"y~;uf'` | \n",
       "| ------------------------- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">     remove_html_markup:1</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">     remove_html_markup:2</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">     remove_html_markup:3</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">     remove_html_markup:4</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">     remove_html_markup:6</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">     remove_html_markup:7</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(53.3989532967738, 100.0%, 80%)\" title=\" 55%\">     remove_html_markup:8</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    - |    - |    X |    - |    X |    - |    - |    X |    - |    X |    X |    X |    X |    X |    X |    X |    X |    - |    X |    X |    X |    X |    X |    X |    X |    - |    - |    X |    - |    X |    - |    - |    X |    - |    X |    X |    X |    X |    - |    - |    X |    - |    X |    X |    X |    X |    X |    X |    - |    X |    X |    X |    X |    X |    - |    - |    X |    X |    X |    X |    - |    - |    - |    X |    X |    X |    X |    X |    - |    X |    - |    X |    X |    X |    X |    X |    - |    X |    X |    X |    X |    - |    X |    X |    - |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">     remove_html_markup:9</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(51.966394858339115, 100.0%, 80%)\" title=\" 56%\">    remove_html_markup:10</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    - |    - |    X |    - |    X |    - |    - |    X |    - |    X |    X |    X |    X |    X |    X |    X |    X |    - |    X |    - |    X |    X |    X |    X |    X |    - |    - |    X |    - |    X |    - |    - |    X |    X |    X |    X |    X |    X |    - |    - |    X |    - |    X |    X |    X |    X |    X |    X |    - |    X |    X |    X |    X |    X |    - |    - |    X |    X |    X |    X |    - |    - |    - |    X |    X |    X |    X |    X |    - |    X |    - |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    - |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">    remove_html_markup:11</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">    remove_html_markup:12</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">    remove_html_markup:13</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">    remove_html_markup:14</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">    remove_html_markup:16</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">remove_html_markup_test:1</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">remove_html_markup_test:2</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(35.14718625761431, 100.0%, 80%)\" title=\" 70%\">remove_html_markup_test:3</samp> |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n",
       "| <samp style=\"background-color: hsl(0.0, 100.0%, 80%)\" title=\"100%\">remove_html_markup_test:4</samp> |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    - |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X |    X | \n"
      ],
      "text/plain": [
       "[('remove_html_markup_test', 4), ('remove_html_markup', 2), ('remove_html_markup', 11), ('remove_html_markup', 14), ('remove_html_markup_test', 1), ('remove_html_markup', 3), ('remove_html_markup', 6), ('remove_html_markup', 12), ('remove_html_markup', 9), ('remove_html_markup_test', 2), ('remove_html_markup', 1), ('remove_html_markup', 7), ('remove_html_markup', 4), ('remove_html_markup', 13), ('remove_html_markup', 16), ('remove_html_markup_test', 3), ('remove_html_markup', 10), ('remove_html_markup', 8)]"
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "html_debugger"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let us create our repairer and run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.307573Z",
     "iopub.status.busy": "2023-11-12T12:47:32.307459Z",
     "iopub.status.idle": "2023-11-12T12:47:32.340992Z",
     "shell.execute_reply": "2023-11-12T12:47:32.340712Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target code to be repaired:\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag):\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "html_repairer = Repairer(html_debugger, log=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:32.342384Z",
     "iopub.status.busy": "2023-11-12T12:47:32.342296Z",
     "iopub.status.idle": "2023-11-12T12:47:38.676382Z",
     "shell.execute_reply": "2023-11-12T12:47:38.676074Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evolving population: iteration  19/20 fitness = 0.99   \r\n",
      "Best code (fitness = 0.99):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag):\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n",
      "Reduced code (fitness = 0.99):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag):\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "best_tree, fitness = html_repairer.repair(iterations=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.677952Z",
     "iopub.status.busy": "2023-11-12T12:47:38.677849Z",
     "iopub.status.idle": "2023-11-12T12:47:38.679620Z",
     "shell.execute_reply": "2023-11-12T12:47:38.679322Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert fitness < 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We see that the \"best\" code is still our original code, with no changes. And we can set `iterations` to 50, 100, 200... – our `Repairer` won't be able to repair it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.681108Z",
     "iopub.status.busy": "2023-11-12T12:47:38.680991Z",
     "iopub.status.idle": "2023-11-12T12:47:38.685792Z",
     "shell.execute_reply": "2023-11-12T12:47:38.685498Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    \n",
       "    <script>\n",
       "    var bad_answers = new Map();\n",
       "\n",
       "    function answer(quiz_id) {\n",
       "        ans = 0;\n",
       "        for (i = 1;; i++) {\n",
       "            checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "            if (!checkbox)\n",
       "                break;\n",
       "            if (checkbox.checked)\n",
       "                ans |= (1 << i);\n",
       "        }\n",
       "        return ans;\n",
       "    }\n",
       "    function check_selection(quiz_id, correct_answer, multiple_choice, hint) {\n",
       "        given_answer = answer(quiz_id);\n",
       "        if (given_answer == correct_answer)\n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Correct!\";\n",
       "            document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "\n",
       "            for (i = 1;; i++) {\n",
       "                checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "                if (!checkbox)\n",
       "                    break;\n",
       "    \n",
       "                if (checkbox.checked) {\n",
       "                    label.style.fontWeight = \"bold\";\n",
       "                }\n",
       "                else {\n",
       "                    label.style.textDecoration = \"line-through\";\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "        else \n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Try again\";\n",
       "            \n",
       "            if (!bad_answers.has(quiz_id)) {\n",
       "                bad_answers.set(quiz_id, 1);\n",
       "            }\n",
       "            else {\n",
       "                bad_answers.set(quiz_id, bad_answers.get(quiz_id) + 1);\n",
       "            }\n",
       "\n",
       "            if (bad_answers.get(quiz_id) >= 2 && hint.length > 0) {\n",
       "                document.getElementById(quiz_id + \"-hint\").innerHTML = \n",
       "                    \"&nbsp;&nbsp;(Hint: <code>\" + hint + \"</code>)\";\n",
       "            }\n",
       "\n",
       "            if (!multiple_choice) {\n",
       "                for (i = 1;; i++) {\n",
       "                    checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                    label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "\n",
       "                    if (!checkbox)\n",
       "                        break;\n",
       "                    if (checkbox.checked) {\n",
       "                        label.style.textDecoration = \"line-through\";\n",
       "                    }\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "    function clear_selection(quiz_id) {\n",
       "        document.getElementById(quiz_id + \"-submit\").value = \"Submit\";\n",
       "        document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "    }\n",
       "    </script>\n",
       "    \n",
       "    <div class=\"quiz\">\n",
       "    <h3 class=\"quiz_title\">Quiz</h3>\n",
       "    <p>\n",
       "    <div class=\"quiz_question\">Why couldn't <code>Repairer()</code> repair <code>remove_html_markup()</code>?</div>\n",
       "    </p>\n",
       "    <p>\n",
       "    <div class=\"quiz_options\" title=\"Pick a choice.\">\n",
       "    \n",
       "        <input type=\"radio\" name=\"a94210fc-8159-11ee-8795-6298cf1a5790\" id=\"a94210fc-8159-11ee-8795-6298cf1a5790-1\" onclick=\"clear_selection('a94210fc-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a94210fc-8159-11ee-8795-6298cf1a5790-1-label\" for=\"a94210fc-8159-11ee-8795-6298cf1a5790-1\">The population is too small!</label><br>\n",
       "    \n",
       "        <input type=\"radio\" name=\"a94210fc-8159-11ee-8795-6298cf1a5790\" id=\"a94210fc-8159-11ee-8795-6298cf1a5790-2\" onclick=\"clear_selection('a94210fc-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a94210fc-8159-11ee-8795-6298cf1a5790-2-label\" for=\"a94210fc-8159-11ee-8795-6298cf1a5790-2\">The suspiciousness is too evenly distributed!</label><br>\n",
       "    \n",
       "        <input type=\"radio\" name=\"a94210fc-8159-11ee-8795-6298cf1a5790\" id=\"a94210fc-8159-11ee-8795-6298cf1a5790-3\" onclick=\"clear_selection('a94210fc-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a94210fc-8159-11ee-8795-6298cf1a5790-3-label\" for=\"a94210fc-8159-11ee-8795-6298cf1a5790-3\">We need more test cases!</label><br>\n",
       "    \n",
       "        <input type=\"radio\" name=\"a94210fc-8159-11ee-8795-6298cf1a5790\" id=\"a94210fc-8159-11ee-8795-6298cf1a5790-4\" onclick=\"clear_selection('a94210fc-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a94210fc-8159-11ee-8795-6298cf1a5790-4-label\" for=\"a94210fc-8159-11ee-8795-6298cf1a5790-4\">We need more iterations!</label><br>\n",
       "    \n",
       "        <input type=\"radio\" name=\"a94210fc-8159-11ee-8795-6298cf1a5790\" id=\"a94210fc-8159-11ee-8795-6298cf1a5790-5\" onclick=\"clear_selection('a94210fc-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a94210fc-8159-11ee-8795-6298cf1a5790-5-label\" for=\"a94210fc-8159-11ee-8795-6298cf1a5790-5\">There is no statement in the source with a correct condition!</label><br>\n",
       "    \n",
       "        <input type=\"radio\" name=\"a94210fc-8159-11ee-8795-6298cf1a5790\" id=\"a94210fc-8159-11ee-8795-6298cf1a5790-6\" onclick=\"clear_selection('a94210fc-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"a94210fc-8159-11ee-8795-6298cf1a5790-6-label\" for=\"a94210fc-8159-11ee-8795-6298cf1a5790-6\">The population is too big!</label><br>\n",
       "    \n",
       "    </div>\n",
       "    </p>\n",
       "    <input id=\"a94210fc-8159-11ee-8795-6298cf1a5790-submit\" type=\"submit\" value=\"Submit\" onclick=\"check_selection('a94210fc-8159-11ee-8795-6298cf1a5790', 32, 0, '5242880 &gt;&gt; 20')\">\n",
       "    <span class=\"quiz_hint\" id=\"a94210fc-8159-11ee-8795-6298cf1a5790-hint\"></span>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quiz(\"Why couldn't `Repairer()` repair `remove_html_markup()`?\",\n",
    "     [\n",
    "         \"The population is too small!\",\n",
    "         \"The suspiciousness is too evenly distributed!\",\n",
    "         \"We need more test cases!\",\n",
    "         \"We need more iterations!\",\n",
    "         \"There is no statement in the source with a correct condition!\",\n",
    "         \"The population is too big!\",\n",
    "     ], '5242880 >> 20')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "You can explore all the hypotheses above by changing the appropriate parameters, but you won't be able to change the outcome. The problem is that, unlike `middle()`, there is no statement (or combination thereof) in `remove_html_markup()` that could be used to make the failure go away. For this, we need to mutate another aspect of the code, which we will explore in the next section."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Mutating Conditions\n",
    "\n",
    "The `Repairer` class is very configurable. The individual steps in automated repair can all be replaced by providing own classes in the keyword arguments of its `__init__()` constructor:\n",
    "\n",
    "* To change fault localization, pass a different `debugger` that is a subclass of `RankingDebugger`.\n",
    "* To change the mutation operator, set `mutator_class` to a subclass of `StatementMutator`.\n",
    "* To change the crossover operator, set `crossover_class` to a subclass of `CrossoverOperator`.\n",
    "* To change the reduction algorithm, set `reducer_class` to a subclass of `Reducer`.\n",
    "\n",
    "In this section, we will explore how to extend the mutation operator such that it can mutate _conditions_ for control constructs such as `if`, `while`, or `for`. To this end, we introduce a new class `ConditionMutator` subclassing `StatementMutator`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Collecting Conditions\n",
    "\n",
    "Let us start with a few simple supporting functions. The function `all_conditions()` retrieves all control conditions from an AST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.687576Z",
     "iopub.status.busy": "2023-11-12T12:47:38.687438Z",
     "iopub.status.idle": "2023-11-12T12:47:38.690354Z",
     "shell.execute_reply": "2023-11-12T12:47:38.690026Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def all_conditions(trees: Union[ast.AST, List[ast.AST]],\n",
    "                   tp: Optional[Type] = None) -> List[ast.expr]:\n",
    "    \"\"\"\n",
    "    Return all conditions from the AST (or AST list) `trees`.\n",
    "    If `tp` is given, return only elements of that type.\n",
    "    \"\"\"\n",
    "\n",
    "    if not isinstance(trees, list):\n",
    "        assert isinstance(trees, ast.AST)\n",
    "        trees = [trees]\n",
    "\n",
    "    visitor = ConditionVisitor()\n",
    "    for tree in trees:\n",
    "        visitor.visit(tree)\n",
    "    conditions = visitor.conditions\n",
    "    if tp is not None:\n",
    "        conditions = [c for c in conditions if isinstance(c, tp)]\n",
    "\n",
    "    return conditions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "`all_conditions()` uses a `ConditionVisitor` class to walk the tree and collect the conditions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.692062Z",
     "iopub.status.busy": "2023-11-12T12:47:38.691930Z",
     "iopub.status.idle": "2023-11-12T12:47:38.695663Z",
     "shell.execute_reply": "2023-11-12T12:47:38.695267Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class ConditionVisitor(NodeVisitor):\n",
    "    def __init__(self) -> None:\n",
    "        self.conditions: List[ast.expr] = []\n",
    "        self.conditions_seen: Set[str] = set()\n",
    "        super().__init__()\n",
    "\n",
    "    def add_conditions(self, node: ast.AST, attr: str) -> None:\n",
    "        elems = getattr(node, attr, [])\n",
    "        if not isinstance(elems, list):\n",
    "            elems = [elems]\n",
    "\n",
    "        elems = cast(List[ast.expr], elems)\n",
    "\n",
    "        for elem in elems:\n",
    "            elem_str = ast.unparse(elem)\n",
    "            if elem_str not in self.conditions_seen:\n",
    "                self.conditions.append(elem)\n",
    "                self.conditions_seen.add(elem_str)\n",
    "\n",
    "    def visit_BoolOp(self, node: ast.BoolOp) -> ast.AST:\n",
    "        self.add_conditions(node, 'values')\n",
    "        return super().generic_visit(node)\n",
    "\n",
    "    def visit_UnaryOp(self, node: ast.UnaryOp) -> ast.AST:\n",
    "        if isinstance(node.op, ast.Not):\n",
    "            self.add_conditions(node, 'operand')\n",
    "        return super().generic_visit(node)\n",
    "\n",
    "    def generic_visit(self, node: ast.AST) -> ast.AST:\n",
    "        if hasattr(node, 'test'):\n",
    "            self.add_conditions(node, 'test')\n",
    "        return super().generic_visit(node)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here are all the conditions in `remove_html_markup()`. This is some material to construct new conditions from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.697357Z",
     "iopub.status.busy": "2023-11-12T12:47:38.697248Z",
     "iopub.status.idle": "2023-11-12T12:47:38.700896Z",
     "shell.execute_reply": "2023-11-12T12:47:38.700616Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"c == '<' and (not quote)\",\n",
       " \"c == '<'\",\n",
       " 'not quote',\n",
       " 'quote',\n",
       " \"c == '>' and (not quote)\",\n",
       " \"c == '>'\",\n",
       " 'c == \\'\"\\' or (c == \"\\'\" and tag)',\n",
       " 'c == \\'\"\\'',\n",
       " 'c == \"\\'\" and tag',\n",
       " 'c == \"\\'\"',\n",
       " 'tag',\n",
       " 'not tag']"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[ast.unparse(cond).strip()\n",
    "    for cond in all_conditions(remove_html_markup_tree())]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Mutating Conditions\n",
    "\n",
    "Here comes our `ConditionMutator` class. We subclass from `StatementMutator` and set an attribute `self.conditions` containing all the conditions in the source. The method `choose_condition()` randomly picks a condition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.702617Z",
     "iopub.status.busy": "2023-11-12T12:47:38.702455Z",
     "iopub.status.idle": "2023-11-12T12:47:38.705148Z",
     "shell.execute_reply": "2023-11-12T12:47:38.704795Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class ConditionMutator(StatementMutator):\n",
    "    \"\"\"Mutate conditions in an AST\"\"\"\n",
    "\n",
    "    def __init__(self, *args: Any, **kwargs: Any) -> None:\n",
    "        \"\"\"Constructor. Arguments are as with `StatementMutator` constructor.\"\"\"\n",
    "        super().__init__(*args, **kwargs)\n",
    "        self.conditions = all_conditions(self.source)\n",
    "        if self.log:\n",
    "            print(\"Found conditions\",\n",
    "                  [ast.unparse(cond).strip() \n",
    "                   for cond in self.conditions])\n",
    "\n",
    "    def choose_condition(self) -> ast.expr:\n",
    "        \"\"\"Return a random condition from source.\"\"\"\n",
    "        return copy.deepcopy(random.choice(self.conditions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The actual mutation takes place in the `swap()` method. If the node to be replaced has a `test` attribute (i.e. a controlling predicate), then we pick a random condition `cond` from the source and randomly chose from:\n",
    "\n",
    "* **set**: We change `test` to `cond`.\n",
    "* **not**: We invert `test`.\n",
    "* **and**: We replace `test` by `cond and test`.\n",
    "* **or**: We replace `test` by `cond or test`.\n",
    "\n",
    "Over time, this might lead to operators propagating across the population."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.706809Z",
     "iopub.status.busy": "2023-11-12T12:47:38.706686Z",
     "iopub.status.idle": "2023-11-12T12:47:38.710072Z",
     "shell.execute_reply": "2023-11-12T12:47:38.709746Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class ConditionMutator(ConditionMutator):\n",
    "    def choose_bool_op(self) -> str:\n",
    "        return random.choice(['set', 'not', 'and', 'or'])\n",
    "\n",
    "    def swap(self, node: ast.AST) -> ast.AST:\n",
    "        \"\"\"Replace `node` condition by a condition from `source`\"\"\"\n",
    "        if not hasattr(node, 'test'):\n",
    "            return super().swap(node)\n",
    "\n",
    "        node = cast(ast.If, node)\n",
    "\n",
    "        cond = self.choose_condition()\n",
    "        new_test = None\n",
    "\n",
    "        choice = self.choose_bool_op()\n",
    "\n",
    "        if choice == 'set':\n",
    "            new_test = cond\n",
    "        elif choice == 'not':\n",
    "            new_test = ast.UnaryOp(op=ast.Not(), operand=node.test)\n",
    "        elif choice == 'and':\n",
    "            new_test = ast.BoolOp(op=ast.And(), values=[cond, node.test])\n",
    "        elif choice == 'or':\n",
    "            new_test = ast.BoolOp(op=ast.Or(), values=[cond, node.test])\n",
    "        else:\n",
    "            raise ValueError(\"Unknown boolean operand\")\n",
    "\n",
    "        if new_test:\n",
    "            # ast.copy_location(new_test, node)\n",
    "            node.test = new_test\n",
    "\n",
    "        return node"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can use the mutator just like `StatementMutator`, except that some of the mutations will also include new conditions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.711688Z",
     "iopub.status.busy": "2023-11-12T12:47:38.711555Z",
     "iopub.status.idle": "2023-11-12T12:47:38.714860Z",
     "shell.execute_reply": "2023-11-12T12:47:38.714478Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found conditions [\"c == '<' and (not quote)\", \"c == '<'\", 'not quote', 'quote', \"c == '>' and (not quote)\", \"c == '>'\", 'c == \\'\"\\' or (c == \"\\'\" and tag)', 'c == \\'\"\\'', 'c == \"\\'\" and tag', 'c == \"\\'\"', 'tag', 'not tag']\n"
     ]
    }
   ],
   "source": [
    "mutator = ConditionMutator(source=all_statements(remove_html_markup_tree()),\n",
    "                           log=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.716990Z",
     "iopub.status.busy": "2023-11-12T12:47:38.716850Z",
     "iopub.status.idle": "2023-11-12T12:47:38.731193Z",
     "shell.execute_reply": "2023-11-12T12:47:38.730806Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   2:insert: 'tag = False' becomes 'for c in s: tag = Fa...'\n",
      "  10:insert: 'tag = False' becomes 'tag = False'; 'out = out + c'\n",
      "   8:insert: 'tag = True' becomes 'if c == \\'\"\\' or (c ==...'\n",
      "  12:insert: 'quote = not quote' becomes 'quote = not quote'; 'tag = True'\n",
      "  10:delete: 'tag = False' becomes 'pass'\n",
      "  12:insert: 'quote = not quote' becomes \"if c == '>' and (not...\"\n",
      "   3:insert: 'quote = False' becomes 'quote = False'; \"out = ''\"\n",
      "  14:swap:   'out = out + c' becomes 'quote = False'\n",
      "  12:insert: 'quote = not quote' becomes 'for c in s: quote = ...'\n",
      "   3:delete: 'quote = False' becomes 'pass'\n"
     ]
    }
   ],
   "source": [
    "for i in range(10):\n",
    "    new_tree = mutator.mutate(remove_html_markup_tree())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let us put our new mutator to action, again in a `Repairer()`. To activate it, all we need to do is to pass it as `mutator_class` keyword argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.733088Z",
     "iopub.status.busy": "2023-11-12T12:47:38.732982Z",
     "iopub.status.idle": "2023-11-12T12:47:38.767398Z",
     "shell.execute_reply": "2023-11-12T12:47:38.767109Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target code to be repaired:\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag):\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "condition_repairer = Repairer(html_debugger,\n",
    "                              mutator_class=ConditionMutator,\n",
    "                              log=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We might need more iterations for this one. Let us see..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:47:38.769176Z",
     "iopub.status.busy": "2023-11-12T12:47:38.769052Z",
     "iopub.status.idle": "2023-11-12T12:48:30.722144Z",
     "shell.execute_reply": "2023-11-12T12:48:30.721685Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evolving population: iteration 136/200 fitness = 0.99   \r\n",
      "\n",
      "New best code (fitness = 0.99):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n",
      "Evolving population: iteration 162/200 fitness = 0.99   \r\n",
      "\n",
      "New best code (fitness = 0.99):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n",
      "Evolving population: iteration 167/200 fitness = 1.0   \r\n",
      "\n",
      "New best code (fitness = 1.0):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m tag \u001b[35mand\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "                out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n",
      "\n",
      "Reduced code (fitness = 1.0):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m tag \u001b[35mand\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "best_tree, fitness = condition_repairer.repair(iterations=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.724048Z",
     "iopub.status.busy": "2023-11-12T12:48:30.723916Z",
     "iopub.status.idle": "2023-11-12T12:48:30.725902Z",
     "shell.execute_reply": "2023-11-12T12:48:30.725619Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "repaired_source = ast.unparse(best_tree)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.727534Z",
     "iopub.status.busy": "2023-11-12T12:48:30.727424Z",
     "iopub.status.idle": "2023-11-12T12:48:30.757747Z",
     "shell.execute_reply": "2023-11-12T12:48:30.757449Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m tag \u001b[35mand\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(repaired_source, '.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.759302Z",
     "iopub.status.busy": "2023-11-12T12:48:30.759210Z",
     "iopub.status.idle": "2023-11-12T12:48:30.761038Z",
     "shell.execute_reply": "2023-11-12T12:48:30.760777Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert fitness >= 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Success again! We have automatically repaired `remove_html_markup()` – the resulting code passes all tests, including those that were previously failing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Again, we can present the fix as a patch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.762668Z",
     "iopub.status.busy": "2023-11-12T12:48:30.762559Z",
     "iopub.status.idle": "2023-11-12T12:48:30.764849Z",
     "shell.execute_reply": "2023-11-12T12:48:30.764600Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "original_source = ast.unparse(remove_html_markup_tree())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.766425Z",
     "iopub.status.busy": "2023-11-12T12:48:30.766311Z",
     "iopub.status.idle": "2023-11-12T12:48:30.796564Z",
     "shell.execute_reply": "2023-11-12T12:48:30.796294Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@@ -\u001b[34m210\u001b[39;49;00m,\u001b[34m53\u001b[39;49;00m +\u001b[34m210\u001b[39;49;00m,\u001b[34m39\u001b[39;49;00m @@\u001b[37m\u001b[39;49;00m\n",
      " lse\u001b[37m\u001b[39;49;00m\n",
      "\u001b[37m\u001b[39;49;00m\n",
      "-        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag):\u001b[37m\u001b[39;49;00m\n",
      "\u001b[37m\u001b[39;49;00m\n",
      "+        \u001b[34melif\u001b[39;49;00m tag \u001b[35mand\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n"
     ]
    }
   ],
   "source": [
    "for patch in diff(original_source, repaired_source):\n",
    "    print_patch(patch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "However, looking at the patch, one may come up with doubts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.798226Z",
     "iopub.status.busy": "2023-11-12T12:48:30.798141Z",
     "iopub.status.idle": "2023-11-12T12:48:30.802137Z",
     "shell.execute_reply": "2023-11-12T12:48:30.801854Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    \n",
       "    <script>\n",
       "    var bad_answers = new Map();\n",
       "\n",
       "    function answer(quiz_id) {\n",
       "        ans = 0;\n",
       "        for (i = 1;; i++) {\n",
       "            checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "            if (!checkbox)\n",
       "                break;\n",
       "            if (checkbox.checked)\n",
       "                ans |= (1 << i);\n",
       "        }\n",
       "        return ans;\n",
       "    }\n",
       "    function check_selection(quiz_id, correct_answer, multiple_choice, hint) {\n",
       "        given_answer = answer(quiz_id);\n",
       "        if (given_answer == correct_answer)\n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Correct!\";\n",
       "            document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "\n",
       "            for (i = 1;; i++) {\n",
       "                checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "                if (!checkbox)\n",
       "                    break;\n",
       "    \n",
       "                if (checkbox.checked) {\n",
       "                    label.style.fontWeight = \"bold\";\n",
       "                }\n",
       "                else {\n",
       "                    label.style.textDecoration = \"line-through\";\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "        else \n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Try again\";\n",
       "            \n",
       "            if (!bad_answers.has(quiz_id)) {\n",
       "                bad_answers.set(quiz_id, 1);\n",
       "            }\n",
       "            else {\n",
       "                bad_answers.set(quiz_id, bad_answers.get(quiz_id) + 1);\n",
       "            }\n",
       "\n",
       "            if (bad_answers.get(quiz_id) >= 2 && hint.length > 0) {\n",
       "                document.getElementById(quiz_id + \"-hint\").innerHTML = \n",
       "                    \"&nbsp;&nbsp;(Hint: <code>\" + hint + \"</code>)\";\n",
       "            }\n",
       "\n",
       "            if (!multiple_choice) {\n",
       "                for (i = 1;; i++) {\n",
       "                    checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                    label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "\n",
       "                    if (!checkbox)\n",
       "                        break;\n",
       "                    if (checkbox.checked) {\n",
       "                        label.style.textDecoration = \"line-through\";\n",
       "                    }\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "    function clear_selection(quiz_id) {\n",
       "        document.getElementById(quiz_id + \"-submit\").value = \"Submit\";\n",
       "        document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "    }\n",
       "    </script>\n",
       "    \n",
       "    <div class=\"quiz\">\n",
       "    <h3 class=\"quiz_title\">Quiz</h3>\n",
       "    <p>\n",
       "    <div class=\"quiz_question\">Is this actually the best solution?</div>\n",
       "    </p>\n",
       "    <p>\n",
       "    <div class=\"quiz_options\" title=\"Pick a choice.\">\n",
       "    \n",
       "        <input type=\"radio\" name=\"c8527e1e-8159-11ee-8795-6298cf1a5790\" id=\"c8527e1e-8159-11ee-8795-6298cf1a5790-1\" onclick=\"clear_selection('c8527e1e-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"c8527e1e-8159-11ee-8795-6298cf1a5790-1-label\" for=\"c8527e1e-8159-11ee-8795-6298cf1a5790-1\">Yes, sure, of course. Why?</label><br>\n",
       "    \n",
       "        <input type=\"radio\" name=\"c8527e1e-8159-11ee-8795-6298cf1a5790\" id=\"c8527e1e-8159-11ee-8795-6298cf1a5790-2\" onclick=\"clear_selection('c8527e1e-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"c8527e1e-8159-11ee-8795-6298cf1a5790-2-label\" for=\"c8527e1e-8159-11ee-8795-6298cf1a5790-2\">Err - what happened to single quotes?</label><br>\n",
       "    \n",
       "    </div>\n",
       "    </p>\n",
       "    <input id=\"c8527e1e-8159-11ee-8795-6298cf1a5790-submit\" type=\"submit\" value=\"Submit\" onclick=\"check_selection('c8527e1e-8159-11ee-8795-6298cf1a5790', 4, 0, '')\">\n",
       "    <span class=\"quiz_hint\" id=\"c8527e1e-8159-11ee-8795-6298cf1a5790-hint\"></span>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quiz(\"Is this actually the best solution?\",\n",
    "    [\n",
    "        \"Yes, sure, of course. Why?\",\n",
    "        \"Err - what happened to single quotes?\"\n",
    "    ], 1 << 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Indeed – our solution does not seem to handle single quotes anymore. Why is that so?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.803602Z",
     "iopub.status.busy": "2023-11-12T12:48:30.803516Z",
     "iopub.status.idle": "2023-11-12T12:48:30.806640Z",
     "shell.execute_reply": "2023-11-12T12:48:30.806367Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    \n",
       "    <script>\n",
       "    var bad_answers = new Map();\n",
       "\n",
       "    function answer(quiz_id) {\n",
       "        ans = 0;\n",
       "        for (i = 1;; i++) {\n",
       "            checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "            if (!checkbox)\n",
       "                break;\n",
       "            if (checkbox.checked)\n",
       "                ans |= (1 << i);\n",
       "        }\n",
       "        return ans;\n",
       "    }\n",
       "    function check_selection(quiz_id, correct_answer, multiple_choice, hint) {\n",
       "        given_answer = answer(quiz_id);\n",
       "        if (given_answer == correct_answer)\n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Correct!\";\n",
       "            document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "\n",
       "            for (i = 1;; i++) {\n",
       "                checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "                if (!checkbox)\n",
       "                    break;\n",
       "    \n",
       "                if (checkbox.checked) {\n",
       "                    label.style.fontWeight = \"bold\";\n",
       "                }\n",
       "                else {\n",
       "                    label.style.textDecoration = \"line-through\";\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "        else \n",
       "        {\n",
       "            document.getElementById(quiz_id + \"-submit\").value = \"Try again\";\n",
       "            \n",
       "            if (!bad_answers.has(quiz_id)) {\n",
       "                bad_answers.set(quiz_id, 1);\n",
       "            }\n",
       "            else {\n",
       "                bad_answers.set(quiz_id, bad_answers.get(quiz_id) + 1);\n",
       "            }\n",
       "\n",
       "            if (bad_answers.get(quiz_id) >= 2 && hint.length > 0) {\n",
       "                document.getElementById(quiz_id + \"-hint\").innerHTML = \n",
       "                    \"&nbsp;&nbsp;(Hint: <code>\" + hint + \"</code>)\";\n",
       "            }\n",
       "\n",
       "            if (!multiple_choice) {\n",
       "                for (i = 1;; i++) {\n",
       "                    checkbox = document.getElementById(quiz_id + \"-\" + i.toString());\n",
       "                    label = document.getElementById(quiz_id + \"-\" + i.toString() + \"-label\")\n",
       "\n",
       "                    if (!checkbox)\n",
       "                        break;\n",
       "                    if (checkbox.checked) {\n",
       "                        label.style.textDecoration = \"line-through\";\n",
       "                    }\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "    function clear_selection(quiz_id) {\n",
       "        document.getElementById(quiz_id + \"-submit\").value = \"Submit\";\n",
       "        document.getElementById(quiz_id + \"-hint\").innerHTML = \"\";\n",
       "    }\n",
       "    </script>\n",
       "    \n",
       "    <div class=\"quiz\">\n",
       "    <h3 class=\"quiz_title\">Quiz</h3>\n",
       "    <p>\n",
       "    <div class=\"quiz_question\">Why aren't single quotes handled in the solution?</div>\n",
       "    </p>\n",
       "    <p>\n",
       "    <div class=\"quiz_options\" title=\"Pick a choice.\">\n",
       "    \n",
       "        <input type=\"radio\" name=\"c8535028-8159-11ee-8795-6298cf1a5790\" id=\"c8535028-8159-11ee-8795-6298cf1a5790-1\" onclick=\"clear_selection('c8535028-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"c8535028-8159-11ee-8795-6298cf1a5790-1-label\" for=\"c8535028-8159-11ee-8795-6298cf1a5790-1\">Because they're not important. I mean, y'know, who uses 'em anyway?</label><br>\n",
       "    \n",
       "        <input type=\"radio\" name=\"c8535028-8159-11ee-8795-6298cf1a5790\" id=\"c8535028-8159-11ee-8795-6298cf1a5790-2\" onclick=\"clear_selection('c8535028-8159-11ee-8795-6298cf1a5790')\">\n",
       "        <label id=\"c8535028-8159-11ee-8795-6298cf1a5790-2-label\" for=\"c8535028-8159-11ee-8795-6298cf1a5790-2\">Because they are not part of our tests? Let me look up how they are constructed...</label><br>\n",
       "    \n",
       "    </div>\n",
       "    </p>\n",
       "    <input id=\"c8535028-8159-11ee-8795-6298cf1a5790-submit\" type=\"submit\" value=\"Submit\" onclick=\"check_selection('c8535028-8159-11ee-8795-6298cf1a5790', 4, 0, '')\">\n",
       "    <span class=\"quiz_hint\" id=\"c8535028-8159-11ee-8795-6298cf1a5790-hint\"></span>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 180,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quiz(\"Why aren't single quotes handled in the solution?\",\n",
    "    [\n",
    "        \"Because they're not important. \"\n",
    "            \"I mean, y'know, who uses 'em anyway?\",\n",
    "        \"Because they are not part of our tests? \"\n",
    "            \"Let me look up how they are constructed...\"\n",
    "    ], 1 << 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Correct! Our test cases do not include single quotes – at least not in the interior of HTML tags – and thus, automatic repair did not care to preserve their handling."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "How can we fix this? An easy way is to include an appropriate test case in our set – a test case that passes with the original `remove_html_markup()`, yet fails with the \"repaired\" `remove_html_markup()` as shown above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.808282Z",
     "iopub.status.busy": "2023-11-12T12:48:30.808176Z",
     "iopub.status.idle": "2023-11-12T12:48:30.810013Z",
     "shell.execute_reply": "2023-11-12T12:48:30.809756Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "with html_debugger:\n",
    "    remove_html_markup_test(\"<foo quote='>abc'>me</foo>\", \"me\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let us repeat the repair with the extended test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:30.811595Z",
     "iopub.status.busy": "2023-11-12T12:48:30.811493Z",
     "iopub.status.idle": "2023-11-12T12:48:31.922673Z",
     "shell.execute_reply": "2023-11-12T12:48:31.922379Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evolving population: iteration   2/200 fitness = 1.0   \r\n",
      "\n",
      "New best code (fitness = 1.0):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m tag \u001b[35mand\u001b[39;49;00m (c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag)):\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "        tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n",
      "\n",
      "Reduced code (fitness = 1.0):\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m tag \u001b[35mand\u001b[39;49;00m (c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag)):\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "best_tree, fitness = condition_repairer.repair(iterations=200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here is the final tree:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:31.924431Z",
     "iopub.status.busy": "2023-11-12T12:48:31.924316Z",
     "iopub.status.idle": "2023-11-12T12:48:31.955725Z",
     "shell.execute_reply": "2023-11-12T12:48:31.955458Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mremove_html_markup\u001b[39;49;00m(s):\u001b[37m\u001b[39;49;00m\n",
      "    tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    quote = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    out = \u001b[33m'\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mfor\u001b[39;49;00m c \u001b[35min\u001b[39;49;00m s:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m<\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mTrue\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m c == \u001b[33m'\u001b[39;49;00m\u001b[33m>\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m (\u001b[35mnot\u001b[39;49;00m quote):\u001b[37m\u001b[39;49;00m\n",
      "            tag = \u001b[34mFalse\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m tag \u001b[35mand\u001b[39;49;00m (c == \u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m \u001b[35mor\u001b[39;49;00m (c == \u001b[33m\"\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[35mand\u001b[39;49;00m tag)):\u001b[37m\u001b[39;49;00m\n",
      "            quote = \u001b[35mnot\u001b[39;49;00m quote\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "            out = out + c\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m \u001b[35mnot\u001b[39;49;00m tag:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mreturn\u001b[39;49;00m out\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "print_content(ast.unparse(best_tree), '.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "And here is its fitness:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:31.957497Z",
     "iopub.status.busy": "2023-11-12T12:48:31.957367Z",
     "iopub.status.idle": "2023-11-12T12:48:31.959396Z",
     "shell.execute_reply": "2023-11-12T12:48:31.959112Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fitness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:31.960834Z",
     "iopub.status.busy": "2023-11-12T12:48:31.960728Z",
     "iopub.status.idle": "2023-11-12T12:48:31.962259Z",
     "shell.execute_reply": "2023-11-12T12:48:31.962019Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert fitness >= 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The revised candidate now passes _all_ tests (including the tricky quote test we added last). Its condition now properly checks for `tag` _and_ both quotes. (The `tag` inside the parentheses is still redundant, but so be it.) From this example, we can learn a few lessons about the possibilities and risks of automated repair:\n",
    "\n",
    "* First, automatic repair is highly dependent on the quality of the checking tests. The risk is that the repair may overspecialize towards the test.\n",
    "* Second, when based on \"plastic surgery\", automated repair is highly dependent on the sources that program fragments are chosen from. If there is a hint of a solution somewhere in the code, there is a chance that automated repair will catch it up.\n",
    "* Third, automatic repair is a deeply heuristic approach. Its behavior will vary widely with any change to the parameters (and the underlying random number generators).\n",
    "* Fourth, automatic repair can take a long time. The examples we have in this chapter take less than a minute to compute, and neither Python nor our implementation is exactly fast. But as the search space grows, automated repair will take much longer.\n",
    "\n",
    "On the other hand, even an incomplete automated repair candidate can be much better than nothing at all – it may provide all the essential ingredients (such as the location or the involved variables) for a successful fix. When users of automated repair techniques are aware of its limitations and its assumptions, there is lots of potential in automated repair. Enjoy!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Limitations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The `Repairer` class is tested on our example programs, but not much more. Things that do not work include\n",
    "\n",
    "* Functions with inner functions are not repaired."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Synopsis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This chapter provides tools and techniques for automated repair of program code. The `Repairer` class takes a `RankingDebugger` debugger as input (such as `OchiaiDebugger` from the [chapter on statistical debugging](StatisticalDebugger.ipynb). A typical setup looks like this:\n",
    "\n",
    "```python\n",
    "from debuggingbook.StatisticalDebugger import OchiaiDebugger\n",
    "\n",
    "debugger = OchiaiDebugger()\n",
    "for inputs in TESTCASES:\n",
    "    with debugger:\n",
    "        test_foo(inputs)\n",
    "...\n",
    "\n",
    "repairer = Repairer(debugger)\n",
    "```\n",
    "Here, `test_foo()` is a function that raises an exception if the tested function `foo()` fails. If `foo()` passes, `test_foo()` should not raise an exception."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The `repair()` method of a `Repairer` searches for a repair of the code covered in the debugger (except for methods whose name starts or ends in `test`, such that `foo()`, not `test_foo()` is repaired). `repair()` returns the best fix candidate as a pair `(tree, fitness)` where `tree` is a [Python abstract syntax tree](http://docs.python.org/3/library/ast) (AST) of the fix candidate, and `fitness` is the fitness of the candidate (a value between 0 and 1). A `fitness` of 1.0 means that the candidate passed all tests. A typical usage looks like this:\n",
    "\n",
    "```python\n",
    "tree, fitness = repairer.repair()\n",
    "print(ast.unparse(tree), fitness)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here is a complete example for the `middle()` program. This is the original source code of `middle()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:31.963847Z",
     "iopub.status.busy": "2023-11-12T12:48:31.963746Z",
     "iopub.status.idle": "2023-11-12T12:48:31.993778Z",
     "shell.execute_reply": "2023-11-12T12:48:31.993498Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x, y, z):  \u001b[37m# type: ignore\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34mif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "        \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "            \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "    \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m"
     ]
    }
   ],
   "source": [
    "# ignore\n",
    "print_content(middle_source, '.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We set up a function `middle_test()` that tests it. The `middle_debugger`  collects testcases and outcomes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:31.995304Z",
     "iopub.status.busy": "2023-11-12T12:48:31.995218Z",
     "iopub.status.idle": "2023-11-12T12:48:31.996889Z",
     "shell.execute_reply": "2023-11-12T12:48:31.996646Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "middle_debugger = OchiaiDebugger()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:31.998361Z",
     "iopub.status.busy": "2023-11-12T12:48:31.998270Z",
     "iopub.status.idle": "2023-11-12T12:48:32.004804Z",
     "shell.execute_reply": "2023-11-12T12:48:32.004551Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "for x, y, z in MIDDLE_PASSING_TESTCASES + MIDDLE_FAILING_TESTCASES:\n",
    "    with middle_debugger:\n",
    "        middle_test(x, y, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The repairer is instantiated with the debugger used (`middle_debugger`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:32.006366Z",
     "iopub.status.busy": "2023-11-12T12:48:32.006271Z",
     "iopub.status.idle": "2023-11-12T12:48:32.009105Z",
     "shell.execute_reply": "2023-11-12T12:48:32.008833Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "middle_repairer = Repairer(middle_debugger)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The `repair()` method of the repairer attempts to repair the function invoked by the test (`middle()`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:32.010664Z",
     "iopub.status.busy": "2023-11-12T12:48:32.010575Z",
     "iopub.status.idle": "2023-11-12T12:48:32.470744Z",
     "shell.execute_reply": "2023-11-12T12:48:32.470442Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "tree, fitness = middle_repairer.repair()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The returned AST `tree` can be output via `ast.unparse()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:32.472568Z",
     "iopub.status.busy": "2023-11-12T12:48:32.472467Z",
     "iopub.status.idle": "2023-11-12T12:48:32.474371Z",
     "shell.execute_reply": "2023-11-12T12:48:32.474082Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "def middle(x, y, z):\n",
      "    if y < z:\n",
      "        if x < y:\n",
      "            return y\n",
      "        elif x < z:\n",
      "            return x\n",
      "    elif x > y:\n",
      "        return y\n",
      "    elif x > z:\n",
      "        return x\n",
      "    return z\n"
     ]
    }
   ],
   "source": [
    "print(ast.unparse(tree))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The `fitness` value shows how well the repaired program fits the tests. A fitness value of 1.0 shows that the repaired program satisfies all tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:32.475915Z",
     "iopub.status.busy": "2023-11-12T12:48:32.475807Z",
     "iopub.status.idle": "2023-11-12T12:48:32.477985Z",
     "shell.execute_reply": "2023-11-12T12:48:32.477735Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fitness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:32.479518Z",
     "iopub.status.busy": "2023-11-12T12:48:32.479416Z",
     "iopub.status.idle": "2023-11-12T12:48:32.480944Z",
     "shell.execute_reply": "2023-11-12T12:48:32.480708Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# docassert\n",
    "assert fitness >= 1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Hence, the above program indeed is a perfect repair in the sense that all previously failing tests now pass – our repair was successful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here are the classes defined in this chapter. A `Repairer` repairs a program, using a `StatementMutator` and a `CrossoverOperator` to evolve a population of candidates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:32.482502Z",
     "iopub.status.busy": "2023-11-12T12:48:32.482415Z",
     "iopub.status.idle": "2023-11-12T12:48:32.483941Z",
     "shell.execute_reply": "2023-11-12T12:48:32.483690Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# ignore\n",
    "from ClassDiagram import display_class_hierarchy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:32.485336Z",
     "iopub.status.busy": "2023-11-12T12:48:32.485254Z",
     "iopub.status.idle": "2023-11-12T12:48:33.024491Z",
     "shell.execute_reply": "2023-11-12T12:48:33.024072Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"619pt\" height=\"593pt\"\n",
       " viewBox=\"0.00 0.00 618.62 592.75\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 588.75)\">\n",
       "<g id=\"a_graph0\"><a xlink:title=\"Repairer class hierarchy\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-588.75 614.62,-588.75 614.62,4 -4,4\"/>\n",
       "</a>\n",
       "</g>\n",
       "<!-- Repairer -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>Repairer</title>\n",
       "<g id=\"a_node1\"><a xlink:href=\"#\" xlink:title=\"class Repairer:&#10;A class for automatic repair of Python programs\">\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"15,-0.5 15,-225 151,-225 151,-0.5 15,-0.5\"/>\n",
       "<text text-anchor=\"start\" x=\"56.38\" y=\"-208.7\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#6a0dad\">Repairer</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"15,-199.75 151,-199.75\"/>\n",
       "<g id=\"a_node1_0\"><a xlink:href=\"#\" xlink:title=\"Repairer\">\n",
       "<g id=\"a_node1_1\"><a xlink:href=\"#\" xlink:title=\"__init__(self, debugger: StatisticalDebugger.RankingDebugger, *, targets: Optional[List[Any]] = None, sources: Optional[List[Any]] = None, log: Union[bool, int] = False, mutator_class: Type = &lt;class &#39;StatementMutator&#39;&gt;, crossover_class: Type = &lt;class &#39;CrossoverOperator&#39;&gt;, reducer_class: Type = &lt;class &#39;DeltaDebugger.DeltaDebugger&#39;&gt;, globals: Optional[Dict[str, Any]] = None):&#10;Constructor.&#10;`debugger`: a `RankingDebugger` to take tests and coverage from.&#10;`targets`: a list of functions/modules to be repaired.&#10;(default: the covered functions in `debugger`, except tests)&#10;`sources`: a list of functions/modules to take repairs from.&#10;(default: same as `targets`)&#10;`globals`: if given, a `globals()` dict for executing targets&#10;(default: `globals()` of caller)\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-187.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-size=\"10.00\">__init__()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_2\"><a xlink:href=\"#\" xlink:title=\"repair(self, population_size: int = 40, iterations: int = 100) &#45;&gt; Tuple[ast.AST, float]:&#10;Repair the function we collected test runs from.&#10;Use a population size of `population_size` and&#10;at most `iterations` iterations.&#10;Returns a pair (`ast`, `fitness`) where&#10;`ast` is the AST of the repaired function, and&#10;`fitness` is its fitness (between 0 and 1.0)\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-176\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-size=\"10.00\">repair()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_3\"><a xlink:href=\"#\" xlink:title=\"default_functions(self) &#45;&gt; List[Callable]:&#10;Return the set of functions to be repaired.&#10;Functions whose names start or end in `test` are excluded.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-163.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">default_functions()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_4\"><a xlink:href=\"#\" xlink:title=\"evolve(self, population: List[ast.AST]) &#45;&gt; List[ast.AST]:&#10;Evolve the candidate population by mutating and crossover.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-152.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">evolve()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_5\"><a xlink:href=\"#\" xlink:title=\"fitness(self, tree: ast.AST) &#45;&gt; float:&#10;Test `tree`, returning its fitness\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-141.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">fitness()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_6\"><a xlink:href=\"#\" xlink:title=\"fitness_key(self, tree: ast.AST) &#45;&gt; Tuple[float, int]:&#10;Key to be used for sorting the population\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-130\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">fitness_key()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_7\"><a xlink:href=\"#\" xlink:title=\"getsource(self, item: Union[str, Any]) &#45;&gt; str:&#10;Get the source for `item`. Can also be a string.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-118.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">getsource()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_8\"><a xlink:href=\"#\" xlink:title=\"initial_population(self, size: int) &#45;&gt; List[ast.AST]:&#10;Return an initial population of size `size`\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-107.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">initial_population()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_9\"><a xlink:href=\"#\" xlink:title=\"log_tree(self, description: str, tree: Any) &#45;&gt; None:&#10;Print out `tree` as source code prefixed by `description`.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-96.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">log_tree()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_10\"><a xlink:href=\"#\" xlink:title=\"parse(self, items: List[Any]) &#45;&gt; ast.AST:&#10;Read in a list of items into a single tree\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-85\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">parse()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_11\"><a xlink:href=\"#\" xlink:title=\"reduce(self, tree: ast.AST) &#45;&gt; ast.AST:&#10;Simplify `tree` using delta debugging.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-73.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">reduce()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_12\"><a xlink:href=\"#\" xlink:title=\"run_test_set(self, test_set: str, validate: bool = False) &#45;&gt; int:&#10;Run given `test_set`&#10;(`DifferenceDebugger.PASS` or `DifferenceDebugger.FAIL`).&#10;If `validate` is set, check expectations.&#10;Return number of passed tests.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-62.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">run_test_set()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_13\"><a xlink:href=\"#\" xlink:title=\"run_tests(self, validate: bool = False) &#45;&gt; float:&#10;Run passing and failing tests, returning weighted fitness.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-51.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">run_tests()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_14\"><a xlink:href=\"#\" xlink:title=\"test_reduce(self, source_lines: List[str], original_fitness: float) &#45;&gt; None:&#10;Test function for delta debugging.\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-40\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">test_reduce()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_15\"><a xlink:href=\"#\" xlink:title=\"toplevel_defs(self, tree: ast.AST) &#45;&gt; List[str]:&#10;Return a list of names of defined functions and classes in `tree`\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-28.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">toplevel_defs()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_16\"><a xlink:href=\"#\" xlink:title=\"validate(self) &#45;&gt; None\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-17.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">validate()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node1_17\"><a xlink:href=\"#\" xlink:title=\"weight(self, test_set: str) &#45;&gt; float:&#10;Return the weight of `test_set`&#10;(`DifferenceDebugger.PASS` or `DifferenceDebugger.FAIL`).\">\n",
       "<text text-anchor=\"start\" x=\"23\" y=\"-6.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">weight()</text>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</g>\n",
       "<!-- StackInspector -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>StackInspector</title>\n",
       "<g id=\"a_node2\"><a xlink:href=\"StackInspector.ipynb\" xlink:title=\"class StackInspector:&#10;Provide functions to inspect the stack\">\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-327.88 0,-372.38 166,-372.38 166,-327.88 0,-327.88\"/>\n",
       "<text text-anchor=\"start\" x=\"36.5\" y=\"-356.07\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#6a0dad\">StackInspector</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"0,-347.12 166,-347.12\"/>\n",
       "<g id=\"a_node2_18\"><a xlink:href=\"#\" xlink:title=\"StackInspector\">\n",
       "<g id=\"a_node2_19\"><a xlink:href=\"StackInspector.ipynb\" xlink:title=\"_generated_function_cache = {(&#39;middle_test&#39;, 1): &lt;function middle_test at 0x14eade8c0&gt;, (&#39;test_reduce&#39;, 2): &lt;function test_reduce at 0x14ebbe7a0&gt;, (&#39;remove_html_markup_test&#39;, 1): &lt;function remove_html_markup_test at 0x14ec5a050&gt;}\">\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-333.62\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">_generated_function_cache</text>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</g>\n",
       "<!-- Repairer&#45;&gt;StackInspector -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>Repairer&#45;&gt;StackInspector</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M83,-225.23C83,-258.57 83,-292.58 83,-316.51\"/>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"79.5,-316.21 83,-326.21 86.5,-316.21 79.5,-316.21\"/>\n",
       "</g>\n",
       "<!-- ConditionMutator -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>ConditionMutator</title>\n",
       "<g id=\"a_node3\"><a xlink:href=\"#\" xlink:title=\"class ConditionMutator:&#10;Mutate conditions in an AST\">\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"191.5,-73.62 191.5,-151.88 318.5,-151.88 318.5,-73.62 191.5,-73.62\"/>\n",
       "<text text-anchor=\"start\" x=\"199.5\" y=\"-135.57\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#6a0dad\">ConditionMutator</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"191.5,-126.62 318.5,-126.62\"/>\n",
       "<g id=\"a_node3_20\"><a xlink:href=\"#\" xlink:title=\"ConditionMutator\">\n",
       "<g id=\"a_node3_21\"><a xlink:href=\"#\" xlink:title=\"__init__(self, *args: Any, **kwargs: Any) &#45;&gt; None:&#10;Constructor. Arguments are as with `StatementMutator` constructor.\">\n",
       "<text text-anchor=\"start\" x=\"201\" y=\"-114.12\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-style=\"italic\" font-size=\"10.00\">__init__()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node3_22\"><a xlink:href=\"#\" xlink:title=\"choose_bool_op(self) &#45;&gt; str\">\n",
       "<text text-anchor=\"start\" x=\"201\" y=\"-101.88\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">choose_bool_op()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node3_23\"><a xlink:href=\"#\" xlink:title=\"choose_condition(self) &#45;&gt; ast.expr:&#10;Return a random condition from source.\">\n",
       "<text text-anchor=\"start\" x=\"201\" y=\"-90.62\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">choose_condition()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node3_24\"><a xlink:href=\"#\" xlink:title=\"swap(self, node: ast.AST) &#45;&gt; ast.AST:&#10;Replace `node` condition by a condition from `source`\">\n",
       "<text text-anchor=\"start\" x=\"201\" y=\"-80.38\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-style=\"italic\" font-size=\"10.00\">swap()</text>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</g>\n",
       "<!-- StatementMutator -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>StatementMutator</title>\n",
       "<g id=\"a_node4\"><a xlink:href=\"#\" xlink:title=\"class StatementMutator:&#10;Mutate statements in an AST for automated repair.\">\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"184,-262 184,-438.25 326,-438.25 326,-262 184,-262\"/>\n",
       "<text text-anchor=\"start\" x=\"197.62\" y=\"-421.95\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#6a0dad\">StatementMutator</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"184,-413 326,-413\"/>\n",
       "<g id=\"a_node4_25\"><a xlink:href=\"#\" xlink:title=\"StatementMutator\">\n",
       "<g id=\"a_node4_26\"><a xlink:href=\"#\" xlink:title=\"NODE_MAX_LENGTH = 20\">\n",
       "<text text-anchor=\"start\" x=\"210\" y=\"-399.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">NODE_MAX_LENGTH</text>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"184,-393.75 326,-393.75\"/>\n",
       "<g id=\"a_node4_27\"><a xlink:href=\"#\" xlink:title=\"StatementMutator\">\n",
       "<g id=\"a_node4_28\"><a xlink:href=\"#\" xlink:title=\"__init__(self, suspiciousness_func: Optional[Callable[[Tuple[Callable, int]], float]] = None, source: Optional[List[ast.AST]] = None, log: Union[bool, int] = False) &#45;&gt; None:&#10;Constructor.&#10;`suspiciousness_func` is a function that takes a location&#10;(function, line_number) and returns a suspiciousness value&#10;between 0 and 1.0. If not given, all locations get the same&#10;suspiciousness of 1.0.&#10;`source` is a list of statements to choose from.\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-381.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-style=\"italic\" font-size=\"10.00\">__init__()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_29\"><a xlink:href=\"#\" xlink:title=\"mutate(self, tree: ast.AST) &#45;&gt; ast.AST:&#10;Mutate the given AST `tree` in place. Return mutated tree.\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-370\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-size=\"10.00\">mutate()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_30\"><a xlink:href=\"#\" xlink:title=\"choose_op(self) &#45;&gt; Callable\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-357.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">choose_op()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_31\"><a xlink:href=\"#\" xlink:title=\"choose_statement(self) &#45;&gt; ast.AST\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-346.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">choose_statement()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_32\"><a xlink:href=\"#\" xlink:title=\"delete(self, node: ast.AST) &#45;&gt; None:&#10;Delete `node`.\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-335.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">delete()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_33\"><a xlink:href=\"#\" xlink:title=\"format_node(self, node: ast.AST) &#45;&gt; str:&#10;Return a string representation for `node`.\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-324\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">format_node()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_34\"><a xlink:href=\"#\" xlink:title=\"insert(self, node: ast.AST) &#45;&gt; Union[ast.AST, List[ast.AST]]:&#10;Insert a random node from `source` after `node`\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-312.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">insert()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_35\"><a xlink:href=\"#\" xlink:title=\"node_suspiciousness(self, stmt: ast.AST, func_name: str) &#45;&gt; float\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-301.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">node_suspiciousness()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_36\"><a xlink:href=\"#\" xlink:title=\"node_to_be_mutated(self, tree: ast.AST) &#45;&gt; ast.AST\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-290.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">node_to_be_mutated()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_37\"><a xlink:href=\"#\" xlink:title=\"swap(self, node: ast.AST) &#45;&gt; ast.AST:&#10;Replace `node` with a random node from `source`\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-280\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-style=\"italic\" font-size=\"10.00\">swap()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node4_38\"><a xlink:href=\"#\" xlink:title=\"visit(self, node: ast.AST) &#45;&gt; ast.AST:&#10;Visit a node.\">\n",
       "<text text-anchor=\"start\" x=\"192\" y=\"-268.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-style=\"italic\" font-size=\"10.00\">visit()</text>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</g>\n",
       "<!-- ConditionMutator&#45;&gt;StatementMutator -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>ConditionMutator&#45;&gt;StatementMutator</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M255,-152.35C255,-179.07 255,-215.9 255,-250.48\"/>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"251.5,-250.1 255,-260.1 258.5,-250.1 251.5,-250.1\"/>\n",
       "</g>\n",
       "<!-- NodeTransformer -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>NodeTransformer</title>\n",
       "<g id=\"a_node5\"><a xlink:href=\"ast.ipynb\" xlink:title=\"class NodeTransformer:&#10;A :class:`NodeVisitor` subclass that walks the abstract syntax tree and&#10;allows modification of nodes.&#10;&#10;The `NodeTransformer` will walk the AST and use the return value of the&#10;visitor methods to replace or remove the old node. &#160;If the return value of&#10;the visitor method is ``None``, the node will be removed from its location,&#10;otherwise it is replaced with the return value. &#160;The return value may be the&#10;original node in which case no replacement takes place.&#10;&#10;Here is an example transformer that rewrites all occurrences of name lookups&#10;(``foo``) to ``data[&#39;foo&#39;]``::&#10;&#10;class RewriteName(NodeTransformer):&#10;&#10;def visit_Name(self, node):&#10;return Subscript(&#10;value=Name(id=&#39;data&#39;, ctx=Load()),&#10;slice=Constant(value=node.id),&#10;ctx=node.ctx&#10;)&#10;&#10;Keep in mind that if the node you&#39;re operating on has child nodes you must&#10;either transform the child nodes yourself or call the :meth:`generic_visit`&#10;method for the node first.&#10;&#10;For nodes that were part of a collection of statements (that applies to all&#10;statement nodes), the visitor may also return a list of nodes rather than&#10;just a single node.&#10;&#10;Usually you use the transformer like this::&#10;&#10;node = YourTransformer().visit(node)\">\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"194.88,-475.25 194.88,-511.25 315.12,-511.25 315.12,-475.25 194.88,-475.25\"/>\n",
       "<text text-anchor=\"start\" x=\"202.88\" y=\"-489.95\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-style=\"italic\" font-size=\"14.00\" fill=\"#6a0dad\">NodeTransformer</text>\n",
       "</a>\n",
       "</g>\n",
       "</g>\n",
       "<!-- StatementMutator&#45;&gt;NodeTransformer -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>StatementMutator&#45;&gt;NodeTransformer</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M255,-438.49C255,-447.42 255,-455.96 255,-463.5\"/>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"251.5,-463.28 255,-473.28 258.5,-463.28 251.5,-463.28\"/>\n",
       "</g>\n",
       "<!-- NodeVisitor -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>NodeVisitor</title>\n",
       "<g id=\"a_node6\"><a xlink:href=\"ast.ipynb\" xlink:title=\"class NodeVisitor:&#10;A node visitor base class that walks the abstract syntax tree and calls a&#10;visitor function for every node found. &#160;This function may return a value&#10;which is forwarded by the `visit` method.&#10;&#10;This class is meant to be subclassed, with the subclass adding visitor&#10;methods.&#10;&#10;Per default the visitor functions for the nodes are ``&#39;visit_&#39;`` +&#10;class name of the node. &#160;So a `TryFinally` node visit function would&#10;be `visit_TryFinally`. &#160;This behavior can be changed by overriding&#10;the `visit` method. &#160;If no visitor function exists for a node&#10;(return value `None`) the `generic_visit` visitor is used instead.&#10;&#10;Don&#39;t use the `NodeVisitor` if you want to apply changes to nodes during&#10;traversing. &#160;For this a special visitor exists (`NodeTransformer`) that&#10;allows modifications.\">\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"212.88,-548.25 212.88,-584.25 297.12,-584.25 297.12,-548.25 212.88,-548.25\"/>\n",
       "<text text-anchor=\"start\" x=\"220.88\" y=\"-562.95\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-style=\"italic\" font-size=\"14.00\" fill=\"#6a0dad\">NodeVisitor</text>\n",
       "</a>\n",
       "</g>\n",
       "</g>\n",
       "<!-- NodeTransformer&#45;&gt;NodeVisitor -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>NodeTransformer&#45;&gt;NodeVisitor</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M255,-511.44C255,-519.02 255,-528.15 255,-536.71\"/>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"251.5,-536.71 255,-546.71 258.5,-536.71 251.5,-536.71\"/>\n",
       "</g>\n",
       "<!-- CrossoverOperator -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>CrossoverOperator</title>\n",
       "<g id=\"a_node7\"><a xlink:href=\"#\" xlink:title=\"class CrossoverOperator:&#10;A class for performing statement crossover of Python programs\">\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"337,-52.75 337,-172.75 473,-172.75 473,-52.75 337,-52.75\"/>\n",
       "<text text-anchor=\"start\" x=\"345.38\" y=\"-156.45\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-size=\"14.00\" fill=\"#6a0dad\">CrossoverOperator</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"337,-147.5 473,-147.5\"/>\n",
       "<g id=\"a_node7_39\"><a xlink:href=\"#\" xlink:title=\"CrossoverOperator\">\n",
       "<g id=\"a_node7_40\"><a xlink:href=\"#\" xlink:title=\"SKIP_LIST = {&lt;class &#39;ast.ClassDef&#39;&gt;, &lt;class &#39;ast.Module&#39;&gt;}\">\n",
       "<text text-anchor=\"start\" x=\"378\" y=\"-134\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">SKIP_LIST</text>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"337,-128.25 473,-128.25\"/>\n",
       "<g id=\"a_node7_41\"><a xlink:href=\"#\" xlink:title=\"CrossoverOperator\">\n",
       "<g id=\"a_node7_42\"><a xlink:href=\"#\" xlink:title=\"__init__(self, log: Union[bool, int] = False):&#10;Constructor. If `log` is set, turn on logging.\">\n",
       "<text text-anchor=\"start\" x=\"345\" y=\"-115.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-size=\"10.00\">__init__()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node7_43\"><a xlink:href=\"#\" xlink:title=\"crossover(self, t1: ast.AST, t2: ast.AST) &#45;&gt; Tuple[ast.AST, ast.AST]:&#10;Do a crossover of ASTs `t1` and `t2`.&#10;Raises `CrossoverError` if no crossover is found.\">\n",
       "<text text-anchor=\"start\" x=\"345\" y=\"-104.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-size=\"10.00\">crossover()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node7_44\"><a xlink:href=\"#\" xlink:title=\"can_cross(self, tree: ast.AST, body_attr: str = &#39;body&#39;) &#45;&gt; bool\">\n",
       "<text text-anchor=\"start\" x=\"345\" y=\"-92.25\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">can_cross()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node7_45\"><a xlink:href=\"#\" xlink:title=\"cross_bodies(self, body_1: List[ast.AST], body_2: List[ast.AST]) &#45;&gt; Tuple[List[ast.AST], List[ast.AST]]:&#10;Crossover the statement lists `body_1` x `body_2`. Return new lists.\">\n",
       "<text text-anchor=\"start\" x=\"345\" y=\"-81\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">cross_bodies()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node7_46\"><a xlink:href=\"#\" xlink:title=\"crossover_attr(self, t1: ast.AST, t2: ast.AST, body_attr: str) &#45;&gt; bool:&#10;Crossover the bodies `body_attr` of two trees `t1` and `t2`.&#10;Return True if successful.\">\n",
       "<text text-anchor=\"start\" x=\"345\" y=\"-69.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">crossover_attr()</text>\n",
       "</a>\n",
       "</g>\n",
       "<g id=\"a_node7_47\"><a xlink:href=\"#\" xlink:title=\"crossover_branches(self, t1: ast.AST, t2: ast.AST) &#45;&gt; bool:&#10;Special case:&#10;`t1` = `if P: S1 else: S2` x `t2` = `if P&#39;: S1&#39; else: S2&#39;`&#10;becomes&#10;`t1` = `if P: S2&#39; else: S1&#39;` and `t2` = `if P&#39;: S2 else: S1`&#10;Returns True if successful.\">\n",
       "<text text-anchor=\"start\" x=\"345\" y=\"-58.5\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"10.00\">crossover_branches()</text>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</a>\n",
       "</g>\n",
       "</g>\n",
       "<!-- Legend -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>Legend</title>\n",
       "<text text-anchor=\"start\" x=\"491.38\" y=\"-128.75\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-weight=\"bold\" font-size=\"10.00\" fill=\"#6a0dad\">Legend</text>\n",
       "<text text-anchor=\"start\" x=\"491.38\" y=\"-118.75\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-size=\"10.00\">• </text>\n",
       "<text text-anchor=\"start\" x=\"497.38\" y=\"-118.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-weight=\"bold\" font-size=\"8.00\">public_method()</text>\n",
       "<text text-anchor=\"start\" x=\"491.38\" y=\"-108.75\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-size=\"10.00\">• </text>\n",
       "<text text-anchor=\"start\" x=\"497.38\" y=\"-108.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-size=\"8.00\">private_method()</text>\n",
       "<text text-anchor=\"start\" x=\"491.38\" y=\"-98.75\" font-family=\"Raleway, Helvetica, Arial, sans-serif\" font-size=\"10.00\">• </text>\n",
       "<text text-anchor=\"start\" x=\"497.38\" y=\"-98.75\" font-family=\"'Fira Mono', 'Source Code Pro', 'Courier', monospace\" font-style=\"italic\" font-size=\"8.00\">overloaded_method()</text>\n",
       "<text text-anchor=\"start\" x=\"491.38\" y=\"-89.7\" font-family=\"Helvetica,sans-Serif\" font-size=\"9.00\">Hover over names to see doc</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x14ec44f10>"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# ignore\n",
    "display_class_hierarchy([Repairer, ConditionMutator, CrossoverOperator],\n",
    "                        abstract_classes=[\n",
    "                            NodeVisitor,\n",
    "                            NodeTransformer\n",
    "                        ],\n",
    "                        public_methods=[\n",
    "                            Repairer.__init__,\n",
    "                            Repairer.repair,\n",
    "                            StatementMutator.__init__,\n",
    "                            StatementMutator.mutate,\n",
    "                            ConditionMutator.__init__,\n",
    "                            CrossoverOperator.__init__,\n",
    "                            CrossoverOperator.crossover,\n",
    "                        ],\n",
    "                        project='debuggingbook')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Lessons Learned\n",
    "\n",
    "* Automated repair based on genetic optimization uses five ingredients:\n",
    "    1. A _test suite_ to determine passing and failing tests\n",
    "    2. _Defect localization_ (typically obtained from [statistical debugging](StatisticalDebugger.ipynb) with the test suite) to determine potential locations to be fixed\n",
    "    3. _Random code mutations_ and _crossover operations_ to create and evolve a population of inputs\n",
    "    4. A _fitness function_ and a _selection strategy_ to determine the part of the population that should be evolved further\n",
    "    5. A _reducer_ such as [delta debugging](DeltaDebugger.ipynb) to simplify the final candidate with the highest fitness.\n",
    "* The result of automated repair is a _fix candidate_ with the highest fitness for the given tests.\n",
    "* A _fix candidate_ is not guaranteed to be correct or optimal, but gives important hints on how to fix the program.\n",
    "* All the above ingredients offer plenty of settings and alternatives to experiment with."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Background\n",
    "\n",
    "The seminal work in automated repair is [GenProg](https://squareslab.github.io/genprog-code/) \\cite{LeGoues2012}, which heavily inspired our `Repairer` implementation. Major differences between GenProg and `Repairer` include:\n",
    "\n",
    "* GenProg includes its own defect localization (which is also dynamically updated), whereas `Repairer` builds on earlier statistical debugging.\n",
    "* GenProg can apply multiple mutations on programs (or none at all), whereas `Repairer` applies exactly one mutation.\n",
    "* The `StatementMutator` used by `Repairer` includes various special cases for program structures (`if`, `for`, `while`...), whereas GenProg operates on statements only.\n",
    "* GenProg has been tested on large production programs.\n",
    "\n",
    "While GenProg is _the_ seminal work in the area (and arguably the most important software engineering research contribution of the 2010s), there have been a number of important extensions of automated repair. These include:\n",
    "\n",
    "* *AutoFix* \\cite{Pei2014} leverages _program contracts_ (pre- and postconditions) to generate tests and assertions automatically. Not only do such [assertions](Assertions.ipynb) help in fault localization, they also allow for much better validation of fix candidates.\n",
    "* *SemFix* \\cite{Nguyen2013} and its successor *[Angelix](http://angelix.io)* \\cite{Mechtaev2016}\n",
    "introduce automated program repair based on _symbolic analysis_ rather than genetic optimization. This allows leveraging program semantics, which GenProg does not consider.\n",
    "\n",
    "To learn more about automated program repair, see [program-repair.org](http://program-repair.org), the community page dedicated to research in program repair."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exercises"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exercise 1: Automated Repair Parameters\n",
    "\n",
    "Automated Repair is influenced by numerous design choices – the size of the population, the number of iterations, the genetic optimization strategy, and more. How do changes to these design choices affect its effectiveness? \n",
    "\n",
    "* Consider the constants defined in this chapter (such as `POPULATION_SIZE` or `WEIGHT_PASSING` vs. `WEIGHT_FAILING`). How do changes affect the effectiveness of automated repair?\n",
    "* As an effectiveness metric, consider the number of iterations it takes to produce a fix candidate.\n",
    "* Since genetic optimization is a random algorithm, you need to determine effectiveness averages over a large number of runs (say, 100)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    },
    "solution": "hidden",
    "solution2": "hidden",
    "solution2_first": true,
    "solution_first": true
   },
   "source": [
    "### Exercise 2: Elitism\n",
    "\n",
    "[_Elitism_](https://en.wikipedia.org/wiki/Genetic_algorithm#Elitism) (also known as _elitist selection_) is a variant of genetic selection in which a small fraction of the fittest candidates of the last population are included unchanged in the offspring.\n",
    "\n",
    "* Implement elitist selection by subclassing the `evolve()` method. Experiment with various fractions (5%, 10%, 25%) of \"elites\" and see how this improves results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    },
    "solution": "hidden",
    "solution2": "hidden",
    "solution2_first": true,
    "solution_first": true
   },
   "source": [
    "### Exercise 3: Evolving Values\n",
    "\n",
    "Following the steps of `ConditionMutator`, implement a `ValueMutator` class that replaces one constant value by another one found in the source (say, `0` by `1` or `True` by `False`).\n",
    "\n",
    "For validation, consider the following failure in the `square_root()` function from the [chapter on assertions](Assertions.ipynb):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:33.026743Z",
     "iopub.status.busy": "2023-11-12T12:48:33.026597Z",
     "iopub.status.idle": "2023-11-12T12:48:33.028564Z",
     "shell.execute_reply": "2023-11-12T12:48:33.028265Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from Assertions import square_root  # minor dependency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:33.030122Z",
     "iopub.status.busy": "2023-11-12T12:48:33.029995Z",
     "iopub.status.idle": "2023-11-12T12:48:33.031897Z",
     "shell.execute_reply": "2023-11-12T12:48:33.031633Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/var/folders/n2/xd9445p97rb3xh7m1dfx8_4h0006ts/T/ipykernel_33724/1107282428.py\", line 2, in <cell line: 1>\n",
      "    square_root_of_zero = square_root(0)\n",
      "  File \"/Users/zeller/Projects/debuggingbook/notebooks/Assertions.ipynb\", line 61, in square_root\n",
      "    guess = (approx + x / approx) / 2\n",
      "ZeroDivisionError: float division by zero (expected)\n"
     ]
    }
   ],
   "source": [
    "with ExpectError():\n",
    "    square_root_of_zero = square_root(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "solution2": "hidden",
    "solution2_first": true
   },
   "source": [
    "Can your `ValueMutator` automatically fix this failure?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    },
    "solution2": "hidden"
   },
   "source": [
    "**Solution.** Your solution will be effective if it also includes named constants such as `None`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:33.033496Z",
     "iopub.status.busy": "2023-11-12T12:48:33.033360Z",
     "iopub.status.idle": "2023-11-12T12:48:33.034918Z",
     "shell.execute_reply": "2023-11-12T12:48:33.034685Z"
    },
    "slideshow": {
     "slide_type": "skip"
    },
    "solution2": "hidden"
   },
   "outputs": [],
   "source": [
    "import math"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:33.036333Z",
     "iopub.status.busy": "2023-11-12T12:48:33.036249Z",
     "iopub.status.idle": "2023-11-12T12:48:33.038457Z",
     "shell.execute_reply": "2023-11-12T12:48:33.038151Z"
    },
    "slideshow": {
     "slide_type": "skip"
    },
    "solution2": "hidden"
   },
   "outputs": [],
   "source": [
    "def square_root_fixed(x):  # type: ignore\n",
    "    assert x >= 0  # precondition\n",
    "\n",
    "    approx = 0  # <-- FIX: Change `None` to 0\n",
    "    guess = x / 2\n",
    "    while approx != guess:\n",
    "        approx = guess\n",
    "        guess = (approx + x / approx) / 2\n",
    "\n",
    "    assert math.isclose(approx * approx, x)\n",
    "    return approx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-12T12:48:33.039960Z",
     "iopub.status.busy": "2023-11-12T12:48:33.039873Z",
     "iopub.status.idle": "2023-11-12T12:48:33.042257Z",
     "shell.execute_reply": "2023-11-12T12:48:33.041980Z"
    },
    "slideshow": {
     "slide_type": "skip"
    },
    "solution2": "hidden"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 200,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "square_root_fixed(0)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    },
    "solution": "hidden",
    "solution2": "hidden",
    "solution2_first": true,
    "solution_first": true
   },
   "source": [
    "### Exercise 4: Evolving Variable Names\n",
    "\n",
    "Following the steps of `ConditionMutator`, implement a `IdentifierMutator` class that replaces one identifier by another one found in the source (say, `y` by `x`). Does it help to fix the `middle()` error?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    },
    "solution": "hidden",
    "solution2": "hidden",
    "solution2_first": true,
    "solution_first": true
   },
   "source": [
    "### Exercise 5: Parallel Repair\n",
    "\n",
    "Automatic Repair is a technique that is embarrassingly parallel – all tests for one candidate can all be run in parallel, and all tests for _all_ candidates can also be run in parallel. Set up an infrastructure for running concurrent tests using Pythons [asyncio](https://docs.python.org/3/library/asyncio.html) library."
   ]
  }
 ],
 "metadata": {
  "ipub": {
   "bibliography": "fuzzingbook.bib",
   "toc": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "toc-autonumbering": false,
  "vscode": {
   "interpreter": {
    "hash": "4185989cf89c47c310c2629adcadd634093b57a2c49dffb5ae8d0d14fa302f2b"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}