{ "cells": [ { "cell_type": "markdown", "metadata": { "button": false, "new_sheet": false, "run_control": { "read_only": false } }, "source": [ "# Tracking Bugs\n", "\n", "So far, we have assumed that failures would be discovered and fixed by a single programmer during development. But what if the user who discovers a bug is different from the developer who eventually fixes it? In this case, users have to _report_ bugs, and one needs to ensure that reported bugs are systematically _tracked_. This is the job of dedicated _bug tracking systems_, which we will discuss (and demo) in this chapter." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from bookutils import YouTubeVideo\n", "YouTubeVideo(\"bJzHYzvxHm8\")" ] }, { "cell_type": "markdown", "metadata": { "button": false, "new_sheet": false, "run_control": { "read_only": false } }, "source": [ "**Prerequisites**\n", "\n", "* You should have read the [Introduction to Debugging](Intro_Debugging.ipynb)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "button": false, "new_sheet": false, "run_control": { "read_only": false }, "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "import bookutils.setup" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import Intro_Debugging" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "if 'CI' in os.environ:\n", " # Can't run this in our continuous environment,\n", " # since it can't run a headless Web browser\n", " sys.exit(0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "#\n", "# WARNING: Unlike the other chapters in this book, \n", "# this chapter should NOT BE RUN AS A NOTEBOOK:\n", "#\n", "# * It will delete ALL data from an existing \n", "# local _Redmine_ installation.\n", "# * It will create new users and databases in an existing\n", "# local _MySQL_ installation.\n", "# \n", "# The only reason to run this notebook is to create the book chapter,\n", "# which is the task of Andreas Zeller (and possibly some translators).\n", "# If you are not Andreas, you should exactly know what you are doing.\n", "\n", "assert os.getenv('USER') == 'zeller'" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "## Synopsis\n", "\n", "\n", "To [use the code provided in this chapter](Importing.ipynb), write\n", "\n", "```python\n", ">>> from debuggingbook.Tracking import \n", "```\n", "\n", "and then make use of the following features.\n", "\n", "\n", "This chapter provides no functionality that could be used by third-party code.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reporting Issues\n", "\n", "So far, we have always assumed an environment in which software failures would be discovered by the very developers responsible for the code – that is, failures discovered (and fixed) during development. However, failures can also be discovered by third parties, such as\n", "\n", "* _Testers_ whose job it is to test the code of developers\n", "* Other _developers_ using the code\n", "* _Users_ running the code as it is in production\n", "\n", "In all these cases, developers need to be _informed_ about the fact that the program failed; if they won't know that a bug exists, it will be hard to fix it. This means that we have to set up mechanisms for _reporting bugs_ – manual ones and/or automated ones." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### What Goes in a Bug Report?\n", "\n", "Let us start with the information a developer requires to _fix_ a bug. In a 2008 study \\cite{Bettenburg2008}, Bettenburg et al. asked 872 developers from the Apache, Eclipse, and Mozilla projects to complete a survey on the most important information they need. From top to bottom, these were as follows:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Steps to Reproduce (83%)\n", "\n", "This is a list of steps by which the failure would be reproduced. For instance:\n", "> 1. I started the program using `$ python Debugger.py my_code.py`.\n", "> 2. Then, at the `(debugger)` prompt, I entered `run` and pressed the ENTER key.\n", "\n", "The easier it will be for the developer to reproduce the bug, the higher the chances it will be effectively fixed. [Reducing the steps to those relevant to reproduce the bug](DeltaDebugger.ipynb) can be helpful. But at the same time, the main problem experienced by developers as it comes to bug reports is _incomplete information_, and this especially applies to the steps to reproduce." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Stack Traces (57%)\n", "\n", "These give hints on which parts of the code were active at the moment the failure occurred.\n", "> I got this stack trace:\n", "```python\n", "Traceback (most recent call last):\n", " File \"Debugger.py\", line 2, in \n", " handle_command(\"run\")\n", " File \"Debugger.py\", line 3, in handle_command\n", " scope = s.index(\" in \")\n", "ValueError: substring not found (expected)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Even though stack traces are useful, they are seldom reported by regular users, as they are difficult to obtain (or to find if included in log files). Automated crash reports (see below), however, frequently include them." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Test Cases (51%)\n", "\n", "Test cases that reproduce the bug are also seen as important:\n", "> I can reproduce the bug using the following code:\n", "```python\n", "import Debugger\n", "\n", "Debugger.handle_command(\"run\")\n", "```\n", "\n", "Non-developers hardly ever report test cases." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Observed Behavior (33%)\n", "\n", "What the bug reporter observed as a failure.\n", "> The program crashed with a `ValueError`.\n", "\n", "In many cases, this mimics the stack trace or the steps to reproduce the bug." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Screenshots (26%)\n", "\n", "Screenshots can further illustrate the failure.\n", "> Here is a screenshot of the `Debugger` failing in Jupyter.\n", "\n", "Screenshots are helpful for certain bugs, such as GUI errors." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Expected Behavior (22%)\n", "\n", "What the bug reporter expected instead.\n", "> I expected the program not to crash." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Configuration Information (< 12%)\n", "\n", "Perhaps surprisingly, the information that was seen as _least_ relevant for developers was:\n", "\n", "* Version (12%)\n", "* Build information (8%)\n", "* Product (5%)\n", "* Operating system (4%)\n", "* Component (3%)\n", "* Hardware (0%)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The relative low importance of these fields may be surprising as entering them is usually mandated in bug report forms. However, in \\cite{Bettenburg2008}, developers stated that \n", "> [Operating System] fields are rarely needed as most [of] our bugs are usually found on all platforms.\n", "\n", "This not meant to be read as these fields being totally irrelevant, as, of course, there can be bugs that occur only on specific platforms. Also, if a bug is reported for an older version, but is known to be fixed in a more current version, a simple resolution would be to ask the user to upgrade to the fixed version." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Reporting Crashes Automatically" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If a program crashes, it can be a good idea to have it automatically report the failure to developers. A common practice is to have a crashing program show a _bug report dialog_ allowing the user to report the crash to the vendor. The user is typically asked to provide additional details on how the failure came to be, and the crash report would be sent directly to the vendor's database:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![](https://upload.wikimedia.org/wikipedia/commons/b/b0/Screenshot-Bug_Buddy.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The automatic report typically includes a _stack trace_ and _configuration information_. These two do not reveal too many sensitive details about the user, yet already can help a lot in fixing a bug. In the interest of transparency, the user should be able to inspect all information sent to the vendor." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Besides stack traces and configuration information, such _crash reporters_ could, of course, collect much more - say the data the program operated on, logs, recorded steps to reproduce, or automatically recorded screenshots. However, all of these will likely include sensitive information; and despite their potential usefulness, it is typically better to not collect them in the first place." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Effective Issue Reporting\n", "\n", "When writing an issue report, it is important to look at it from _the developer's perspective_. Developers not only require information on how to reproduce the bug; they also want to work _effectively_ and _efficiently_. The following aspects will all help developers with that:\n", "\n", "* __Have a concise summary__. A good summary should _quickly_ and _uniquely_ identify a bug. Make it easy to understand such that the reader can know if the bug has been reported and/or fixed.\n", "* __Be clear and concise__. Provide necessary information (as shown above) and avoid any extras. Use meaningful sentences and simple words. Structure the report into enumerations and bullet lists.\n", "* __Do not assume context__. Make no assumption that the developer knows all about your bug. Find out whether similar issues have been reported before and reference them.\n", "* __Avoid commanding tones__. Developers enjoy autonomy in their work, and coming across as too authoritative hurts morale.\n", "* __Avoid sarcasm__. Same as above. If you think you can get a volunteer to fix a bug in an open source program for you with sarcasm or commands – good luck with that.\n", "* __Do not assume mistakes__. Do not assume some developer (or anyone) has made a mistake. In many cases, issues can be resolved on your side.\n", "* __One issue per report__. If you have multiple issues, split them in multiple reports; this makes it easier to process them." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## An Issue Tracker\n", "\n", "At the developer's end, all issues reported need to be _tracked_ – that is, they have to be _registered_, they have to be _checked_, and of course, they have to be _addressed_. This process takes place via dedicated database systems, so-called _bug tracking systems_. \n", "\n", "The purposes of an issue tracking system include\n", "\n", "* to collect and store all issue reports;\n", "* to check the status of issues at all times; and\n", "* to organize the debugging and development process.\n", "\n", "Let us illustrate how these steps work, using the popular `Redmine` issue tracking system." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Excursion: Setting up Redmine" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To install Redmine, we followed the instructions at https://gist.github.com/johnjohndoe/2763243.\n", "These final steps initialize the database:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import subprocess" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def with_ruby(cmd: str, inp: str = '', timeout: int = 30, show_stdout: bool = False) -> None:\n", " print(f\"$ {cmd}\")\n", " shell = subprocess.Popen(['/bin/sh', '-c',\n", " f'''rvm_redmine=$HOME/.rvm/gems/ruby-2.7.2@redmine; \\\n", "rvm_global=$HOME/.rvm/gems/ruby-2.7.2@global; \\\n", "export GEM_PATH=$rvm_redmine:$rvm_global; \\\n", "export PATH=$rvm_redmine/bin:$rvm_global/bin:$HOME/.rvm/rubies/ruby-2.7.2/bin:$HOME/.rvm/bin:$PATH; \\\n", "cd $HOME/lib/redmine && {cmd}'''],\n", " stdin=subprocess.PIPE,\n", " stdout=subprocess.PIPE,\n", " stderr=subprocess.PIPE,\n", " universal_newlines=True)\n", " try:\n", " stdout_data, stderr_data = shell.communicate(inp, timeout=timeout)\n", " except subprocess.TimeoutExpired:\n", " shell.kill()\n", "# stdout_data, stderr_data = shell.communicate(inp)\n", "# if show_stdout:\n", "# print(stdout_data, end=\"\")\n", "# print(stderr_data, file=sys.stderr, end=\"\")\n", " raise\n", "\n", " print(stderr_data, file=sys.stderr, end=\"\")\n", " if show_stdout:\n", " print(stdout_data, end=\"\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def with_mysql(cmd: str, timeout: int = 2, show_stdout: bool = False) -> None:\n", " print(f\"sql>{cmd}\")\n", " sql = subprocess.Popen([\"mysql\", \"-u\", \"root\",\n", " \"--default-character-set=utf8mb4\"],\n", " stdin=subprocess.PIPE,\n", " stdout=subprocess.PIPE,\n", " stderr=subprocess.PIPE,\n", " universal_newlines=True)\n", " try:\n", " stdout_data, stderr_data = sql.communicate(cmd + ';',\n", " timeout=timeout)\n", " except subprocess.TimeoutExpired:\n", " sql.kill()\n", "# stdout_data, stderr_data = sql.communicate(inp)\n", "# if show_stdout:\n", "# print(stdout_data, end=\"\")\n", "# print(stderr_data, file=sys.stderr, end=\"\")\n", " raise\n", "\n", " print(stderr_data, file=sys.stderr, end=\"\")\n", " if show_stdout:\n", " print(stdout_data, end=\"\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_ruby(\"bundle config set without development test\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_ruby(\"bundle install\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_ruby(\"pkill sql; sleep 5\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "try:\n", " with_ruby(\"mysql.server start\", show_stdout=True)\n", "except subprocess.TimeoutExpired:\n", " pass # Can actually start without producing output" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_mysql(\"drop database redmine\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_mysql(\"drop user 'redmine'@'localhost'\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_mysql(\"create database redmine character set utf8\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_mysql(\"create user 'redmine'@'localhost' identified by 'my_password'\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_mysql(\"grant all privileges on redmine.* to 'redmine'@'localhost'\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_ruby(\"bundle exec rake generate_secret_token\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_ruby(\"RAILS_ENV=production bundle exec rake db:migrate\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_ruby(\"RAILS_ENV=production bundle exec rake redmine:load_default_data\", '\\n')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End of Excursion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Excursion: Starting Redmine" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import time" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from multiprocess import Process # type: ignore" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "from typing import Tuple, Any" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def run_redmine(port: int) -> None:\n", " with_ruby(f'exec rails s -e production -p {port} > redmine.log 2>&1',\n", " timeout=3600)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def start_redmine(port: int = 3000) -> Tuple[Process, str]:\n", " process = Process(target=run_redmine, args=(port,))\n", " process.start()\n", " time.sleep(5)\n", "\n", " url = f\"http://localhost:{port}\"\n", " return process, url" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "redmine_process, redmine_url = start_redmine()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End of Excursion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Excursion: Remote Control with Selenium" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To produce this book, we use Selenium to interact with user interfaces and obtain screenshots.\n", "\n", "[Selenium](https://www.seleniumhq.org) is a framework for testing Web applications by _automating interaction in the browser_. Selenium provides an API that allows one to launch a Web browser, query the state of the user interface, and interact with individual user interface elements. The Selenium API is available in a number of languages; we use the [Selenium API for Python](https://selenium-python.readthedocs.io/index.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A Selenium *web driver* is the interface between a program and a browser controlled by the program." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from selenium import webdriver" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from selenium.webdriver.common.keys import Keys" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following code starts a Firefox browser in the background, which we then control through the web driver." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "BROWSER = 'firefox'" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with_ruby(\"pkill Firefox.app firefox-bin\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note:** If you don't have Firefox installed, you can also set `BROWSER` to `'chrome'` to use Google Chrome instead." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# BROWSER = 'chrome'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When running this outside of Jupyter notebooks, the browser is _headless_, meaning that it does not show on the screen." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from bookutils import rich_output" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# HEADLESS = not rich_output()\n", "HEADLESS = True" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from selenium.webdriver.remote.webdriver import WebDriver" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from selenium.webdriver.common.by import By" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def start_webdriver(browser: str = BROWSER, headless: bool = HEADLESS,\n", " zoom: float = 4.0) -> WebDriver:\n", " if browser == 'firefox':\n", " options = webdriver.FirefoxOptions()\n", " if browser == 'chrome':\n", " options = webdriver.ChromeOptions() # type: ignore\n", "\n", " if headless and browser == 'chrome':\n", " options.add_argument('headless')\n", " else:\n", " options.headless = headless # type: ignore\n", "\n", " # Start the browser, and obtain a _web driver_ object such that we can interact with it.\n", " if browser == 'firefox':\n", " # For firefox, set a higher resolution for our screenshots\n", " options.set_preference('layout.css.devPixelsPerPx', repr(zoom))\n", " redmine_gui = webdriver.Firefox(options=options) # type: ignore\n", "\n", " # We set the window size such that it fits\n", " redmine_gui.set_window_size(500, 600) # was 1024, 600\n", "\n", " elif browser == 'chrome':\n", " redmine_gui = webdriver.Chrome(options=options) # type: ignore\n", " redmine_gui.set_window_size(1024, 510 if headless else 640)\n", "\n", " return redmine_gui" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "redmine_gui = start_webdriver(browser=BROWSER, headless=HEADLESS)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now interact with the browser programmatically. First, we have it navigate to the URL of our Web server:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "redmine_gui.get(redmine_url)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To see what the \"headless\" browser displays, we can obtain a screenshot. We see that it actually displays the home page." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.display import display, Image" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Image(redmine_gui.get_screenshot_as_png())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End of Excursion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Excursion: Screenshots with Drop Shadows" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By default, our screenshots are flat. We add a drop shadow to make them look nicer.\n", "With help from https://graphicdesign.stackexchange.com/questions/117272/how-to-add-drop-shadow-to-a-picture-via-cli" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import tempfile" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def drop_shadow(contents: bytes) -> bytes:\n", " with tempfile.NamedTemporaryFile() as tmp:\n", " tmp.write(contents)\n", " convert = subprocess.Popen(\n", " ['convert', tmp.name,\n", " '(', '+clone', '-background', 'black', '-shadow', '50x10+15+15', ')',\n", " '+swap', '-background', 'none', '-layers', 'merge', '+repage', '-'],\n", " stdout=subprocess.PIPE, stderr=subprocess.PIPE)\n", " stdout_data, stderr_data = convert.communicate()\n", "\n", " if stderr_data:\n", " print(stderr_data.decode(\"utf-8\"), file=sys.stderr, end=\"\")\n", "\n", " return stdout_data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def screenshot(driver: WebDriver, width: int = 500) -> Any:\n", " return Image(drop_shadow(redmine_gui.get_screenshot_as_png()), width=width)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End of Excursion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Excursion: First Registration at Redmine" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "redmine_gui.get(redmine_url + '/login')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "redmine_gui.find_element(By.ID, \"username\").send_keys(\"admin\")\n", "redmine_gui.find_element(By.ID, \"password\").send_keys(\"admin\")\n", "redmine_gui.find_element(By.NAME, \"login\").click()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "time.sleep(2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "if redmine_gui.current_url.endswith('my/password'):\n", " redmine_gui.get(redmine_url + '/my/password')\n", " redmine_gui.find_element(By.ID, \"password\").send_keys(\"admin\")\n", " redmine_gui.find_element(By.ID, \"new_password\").send_keys(\"admin001\")\n", " redmine_gui.find_element(By.ID, \"new_password_confirmation\").send_keys(\"admin001\")\n", " display(screenshot(redmine_gui))\n", " redmine_gui.find_element(By.NAME, \"commit\").click()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "redmine_gui.get(redmine_url + '/logout')\n", "redmine_gui.find_element(By.NAME, \"commit\").click()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End of Excursion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is what the _Redmine_ tracker starts with:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/login')\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After we login, we see our account:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.ID, \"username\").send_keys(\"admin\")\n", "redmine_gui.find_element(By.ID, \"password\").send_keys(\"admin001\")\n", "redmine_gui.find_element(By.NAME, \"login\").click()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Excursion: Creating a Project" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us check out the projects by clicking on \"Projects\"." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/projects')\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now enter a project name and a description." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/projects/new')\n", "screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/projects/new')\n", "redmine_gui.find_element(By.ID, 'project_name').send_keys(\"The Debugging Book\")\n", "redmine_gui.find_element(By.ID, 'project_description').send_keys(\"A Book on Automated Debugging\")\n", "redmine_gui.find_element(By.ID, 'project_identifier').clear()\n", "redmine_gui.find_element(By.ID, 'project_identifier').send_keys(\"debuggingbook\")\n", "redmine_gui.find_element(By.ID, 'project_homepage').send_keys(\"https://www.debuggingbook.org/\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.NAME, 'commit').click()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End of Excursion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We start with a list of projects." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/projects')\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us choose the (one) \"debuggingbook\" project." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/projects/debuggingbook')\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reporting an Issue" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The most basic task of a bug tracker is to _report_ bugs. However, a bug tracker is a bit more general than just bugs – it can also track feature requests, support requests, and more. These are all summarized under the term \"issue\"." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us take the role of a bug reporter – pardon, an _issue_ reporter – and report an issue. We can do this right from the _Redmine_ menu." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/issues/new')\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's give our bug a name:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "issue_title = \"Does not render correctly on Nokia Communicator\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "issue_description = \\\n", "\"\"\"The Debugging Book does not render correctly on the Nokia Communicator 9000.\n", "\n", "Steps to reproduce:\n", "1. On the Nokia, go to \"https://debuggingbook.org/\"\n", "2. From the menu on top, select the chapter \"Tracking Origins\".\n", "3. Scroll down to a place where a graph is supposed to be shown.\n", "4. Instead of the graph, only a blank space is displayed.\n", "\n", "How to fix:\n", "* The graphs seem to come as SVG elements, but the Nokia Communicator does not support SVG rendering. Render them as JPEGs instead.\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/issues/new')\n", "\n", "redmine_gui.find_element(By.ID, 'issue_subject').send_keys(issue_title)\n", "redmine_gui.find_element(By.ID, 'issue_description').send_keys(issue_description)\n", "screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.ID, 'issue_assigned_to_id').click()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.execute_script(\"window.scrollTo(0, document.body.scrollHeight);\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Clicking on \"Create\" creates the new issue report." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.NAME, 'commit').click()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our bug has been assigned an _issue number_ (#1 in that case). This issue number (also known as bug number, or problem report number) will be used to identify the specific issue from this point on in all communication between developers. Developers will know which issues they are working on; and in any change to the software, they will refer to the issue the change relates to." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Over time, issue databases can grow massively, especially for popular products with several users. As an example, consider the [Mozilla _Bugzilla_ issue tracker](https://bugzilla.mozilla.org/), collecting issue reports for the Firefox browser and other Mozilla products." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "from bookutils import quiz" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "quiz(\"How many issues have been reported over time in Mozilla Bugzilla?\",\n", " [\n", " \"More than ten thousand\",\n", " \"More than a hundred thousand\",\n", " \"More than a million\",\n", " \"More than ten million\"\n", " ], '370370367 // 123456789')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Yes, it is that many! Look at the bug IDs in this recent screenshot from the Mozilla Bugzilla database:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(\"https://bugzilla.mozilla.org/buglist.cgi?quicksearch=firefox\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us enter a few more reports into our issue tracker." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Excursion: Adding Some More Issue Reports" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def new_issue(issue_title: str, issue_description: str) -> Any:\n", " redmine_gui.get(redmine_url + '/issues/new')\n", "\n", " redmine_gui.find_element(By.ID, 'issue_subject').send_keys(issue_title)\n", " redmine_gui.find_element(By.ID, 'issue_description').send_keys(issue_description)\n", " redmine_gui.find_element(By.NAME, 'commit').click()\n", " return screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "new_issue(\"Missing a Chapter on Parallel Debugging\",\n", " \"\"\"I am missing a chapter on (automatic) debugging of parallel and distributed systems,\n", "including how to detect and repair data races, log message passing, and more.\n", "In my experience, almost all programs are parallel today, so you are missing\n", "an important subject.\n", "\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "new_issue(\"Missing a PDF version\",\n", " \"\"\"Your 'book' does not provide a printed version. I think that printed books\n", "\n", "* offer a more enjoyable experience for the reader\n", "* allow me to annotate pages with my own remarks\n", "* allow me to set dog-ear bookmatks\n", "* allow me to show off what I'm currently reading (do you have a cover, too?)\n", "\n", "Please provide a printed version - or, at least, produce a PDF version\n", "of the debugging book, and make it available for download, such that I can print it myself.\n", "\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "new_issue(\"No PDF version\",\n", " \"\"\"Can I have a printed version of your book? Please!\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "new_issue(\"Does not work with Python 2.7 or earlier\",\n", " \"\"\"I was deeply disappointed that your hew book requires Python 3.9 or later.\n", "There are still several Python 2.x users out here (I, for one, cannot stand having to\n", "type parentheses for every `print` statement), and I would love to run your code on\n", "my Python 2.7 programs.\n", "\n", "Would it be possible to backport the book's code such that it would run on Python 3.x\n", "as well as Python 2.x? I would suggest that you add simple checks around your code\n", "such as the following:\n", "\n", "```\n", "import sys\n", "\n", "if sys.version_info.major >= 3:\n", " print(\"The result is\", x)\n", "else: \n", " print \"The result is\", x\n", "```\n", "\n", "As an alternative, rewrite the book in Python 2 and have it automatically translate to\n", "Python 3. This way, you could address all Python lovers, not just Python 3 ones.\n", "\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "new_issue(\"Support for C++\",\n", " \"\"\"I had lots of fun with your 'debugging book'. Yet, I was somewhat disappointed\n", "to see that all code examples are in and for Python programs only. Is there a chance\n", "to get them to work on a real programming language such as C or C++? This would also\n", "open the way to discuss several new debugging techniques for bugs that occur in these\n", "languages only. A chapter on C++ move semantics, and how to fix them, for instance,\n", "would be highly appreciated.\n", "\"\"\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End of Excursion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Managing Issues\n", "\n", "Let us now switch sides and take the view of a _developer_ whose job it is to actually _handle_ all these issues. When our developers log in, the first thing they see is that there are a number of new issues that are all \"open\" – that is, in need to be addressed. (For a typical developer, this may well be the first task of the day.)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/projects/debuggingbook\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Clicking on \"View all issues\" shows us all issues reported so far." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + '/projects/debuggingbook/issues')\n", "redmine_gui.execute_script(\"window.scrollTo(0, document.body.scrollHeight);\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us do some bug _triaging_ here. Bug report #2 is not a bug – it is a feature request. We invoke its \"actions\" pop-up menu and mark it as \"Feature\"." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/issues/\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.XPATH, \"//tr[@id='issue-2']//a[@title='Actions']\").click()\n", "time.sleep(0.25)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "tracker_item = redmine_gui.find_element(By.XPATH,\n", " \"//div[@id='context-menu']//a[text()='Tracker']\")\n", "actions = webdriver.ActionChains(redmine_gui)\n", "actions.move_to_element(tracker_item)\n", "actions.perform()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.XPATH, \"//div[@id='context-menu']//a[text()='Feature']\").click()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The same applies to bugs #3 and #4 (missing PDF) and #6 (no support for C++). We mark them as such as well." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "def mark_tracker(issue: int, tracker: str) -> None:\n", " redmine_gui.get(redmine_url + \"/issues/\")\n", " redmine_gui.find_element(By.XPATH, \n", " f\"//tr[@id='issue-{str(issue)}']//a[@title='Actions']\").click()\n", " time.sleep(0.25)\n", "\n", " tracker_item = redmine_gui.find_element(By.XPATH,\n", " \"//div[@id='context-menu']//a[text()='Tracker']\")\n", " actions = webdriver.ActionChains(redmine_gui)\n", " actions.move_to_element(tracker_item)\n", " actions.perform()\n", " time.sleep(0.25)\n", "\n", " redmine_gui.find_element(By.XPATH,\n", " f\"//div[@id='context-menu']//a[text()='{tracker}']\").click()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "mark_tracker(3, \"Feature\")\n", "mark_tracker(4, \"Feature\")\n", "mark_tracker(6, \"Feature\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/issues/\")\n", "redmine_gui.execute_script(\"window.scrollTo(0, document.body.scrollHeight);\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The fact that we marked these issues as \"feature requests\" does not mean that they will not be worked on – on the contrary, a feature requested by management can have an even higher priority than a bug. Right now, though, we will give our first priority to the bugs listed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Assigning Priorities" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After we have decided the priority for individual bugs, we can make this decision explicit by assigning a _priority_ to each bug. This allows our co-developers to see which things are the most pressing to work on." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us assume we have an important customer for whom fixing issue #1 is important. Through the context menu, we can assign issue #1 a priority of \"Urgent\". " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/issues/\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.XPATH, \"//tr[@id='issue-1']//a[@title='Actions']\").click()\n", "time.sleep(0.25)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "priority_item = redmine_gui.find_element(By.XPATH, \"//div[@id='context-menu']//a[text()='Priority']\")\n", "actions = webdriver.ActionChains(redmine_gui)\n", "actions.move_to_element(priority_item)\n", "actions.perform()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.XPATH, \"//div[@id='context-menu']//a[text()='Urgent']\").click()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that issue #1 is now listed as urgent." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/issues/\")\n", "redmine_gui.execute_script(\"window.scrollTo(0, document.body.scrollHeight);\")\n", "screenshot(redmine_gui)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "On top of priority, some issue trackers also allow assigning a _severity_ to an issue, describing the impact of the bug:\n", "\n", "* __Blocker__ (also known as __Showstopper__): Blocks development and/or testing, for instance by breaking build processes.\n", "* __Critical__: Application crash. Loss of data.\n", "* __Major__: Loss of function.\n", "* __Minor__: Incomplete function.\n", "* __Trivial__: Minor issues in user interfaces and documentation.\n", "* __Enhancement__: Request of a new feature or an improvement in an existing one.\n", "\n", "The severity would be an important factor in determining the priority of an issue." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Assigning Issues" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So far, all the listed bugs are \"unassigned\", which means that there is no developer who is currently working on them. Let us assign the \"urgent\" issue #1 to ourselves, such that we have something to do." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/issues/\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.XPATH, \"//tr[@id='issue-1']//a[@title='Actions']\").click()\n", "time.sleep(0.25)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "assignee_item = redmine_gui.find_element(By.XPATH,\n", " \"//div[@id='context-menu']//a[text()='Assignee']\")\n", "actions = webdriver.ActionChains(redmine_gui)\n", "actions.move_to_element(assignee_item)\n", "actions.perform()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By choosing the Actions menu, clicking on `Assignee`, and then `<< me >>`, we can assign the issue to ourselves." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.XPATH, \"//div[@id='context-menu']//a[text()='<< me >>']\").click()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Of course, we could also go and assign the issue to other developers – depending on their competence and current workload. Very clearly, the ability to assign issues to individuals is a feature for managers." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Resolving Issues\n", "\n", "Let us switch perspectives again, and now take a _developer's role_ – that is, we now work on the issues assigned to us. We get these by clicking on \"Issues assigned to me\":" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/projects/debuggingbook/issues?query_id=1\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By clicking on the issue number, we can see all details about the issue." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/issues/1\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can inspect all features of the issue, including its history." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.execute_script(\"window.scrollTo(0, document.body.scrollHeight);\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us assume that we do have everything in hand we need to fix the bug. (Including a Nokia Communicator device for testing, that is.) This will take some time, during which we won't need our bug database." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![](https://upload.wikimedia.org/wikipedia/commons/2/2f/Prague_Astronomical_Clock_animated.gif)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us now assume that we actually have fixed the bug. This means that we have to go back to our bug database such that we can mark the bug as _resolved_. For this, we open the issue again and click on \"Edit\", and then change the status from \"New\" to \"Resolved\"." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.get(redmine_url + \"/issues/1/edit\")\n", "redmine_gui.find_element(By.ID, \"issue_status_id\").click()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.XPATH, \"//option[text()='Resolved']\").click()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "At the bottom, we can add more notes." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.execute_script(\"window.scrollTo(0, document.body.scrollHeight);\")\n", "issue_notes = redmine_gui.find_element(By.ID, \"issue_notes\")\n", "issue_notes.send_keys(\"Will only work for Nokia Communicator Rev B and later; \"\n", " \"Rev A is still unsupported\")\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After clicking on \"Submit\", we are done. Time for the next bug to fix!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "redmine_gui.find_element(By.NAME, \"commit\").click()\n", "screenshot(redmine_gui)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Life Cycle of an Issue" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have successfully reported an issue (as some third party), assigned it (as a manager), and resolved it (as a developer). Great! However, the steps we have been going through are just one way an issue might be handled." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Resolutions\n", "\n", "First, an issue can have multiple _resolutions_ besides being fixed. Typical resolutions include the following:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### FIXED\n", "\n", "A fix for this issue is made and committed. This is the resolution we have seen for the \"Nokia Communicator\" bug #1, above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### INVALID\n", "\n", "The problem described is not an issue. This is what happens to entries in the issue database that actually do not describe an issue, or that do not provide sufficient information to address it." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "#### WONTFIX\n", "\n", "The issue described is indeed an issue, but will never be fixed. This is a decision of which features (and fixes!) are part of the product, and which ones are not. Our listing of hypothetical issues, above, has an entry complaining that the book \"does not work with Python 2.7 or earlier\". This will not be fixed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### DUPLICATE\n", "\n", "The issue is a duplicate of an existing issue. In our listing of hypothetic issues, we have two issues reporting problems with PDF exports. By making one of these a duplicate of the other. we can ensure that both will be fixed (and checked) at the same time.\n", "\n", "If your product has several users, your issue database will have several duplicates, as multiple users will stumble across the same bugs. Some developers consider such duplicates as a burden, as identifying duplicates takes time. However, the study by Bettenburg et al.~\\cite{Bettenburg2008} points out that such duplicates have their value, too. One developer is quoted as\n", "\n", "> Duplicates are not really problems. They often add useful information. That this information were filed under a new report is not ideal thought." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### WORKSFORME\n", "\n", "The developer could not reproduce the issue, and the code provided no hints on why such a behavior might occur. Such issues are often reopened when more information becomes available." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### An Issue Life Cycle" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With this in mind, we can sketch the individual states an issue report goes through: After submission (NEW), the report is assigned to a developer (ASSIGNED), and then resolved (RESOLVED) with one of the resolutions listed above." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "ipub": { "ignore": true } }, "outputs": [], "source": [ "# ignore\n", "from Intro_Debugging import graph # minor dependency" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "ipub": { "ignore": true } }, "outputs": [], "source": [ "# ignore\n", "from IPython.display import display" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "life_cycle = graph()\n", "life_cycle.attr(rankdir='TB')\n", "\n", "life_cycle.node('New', label=\"<NEW>\", penwidth='2.0')\n", "life_cycle.node('Assigned', label=\"<ASSIGNED>\")\n", "\n", "with life_cycle.subgraph() as res:\n", " res.attr(rank='same')\n", " res.node('Resolved', label=\"<RESOLVED>\", penwidth='2.0')\n", " res.node('Resolution',\n", " shape='plain',\n", " fillcolor='white',\n", " label=\"\"\"<Resolution: One of
\n", "• FIXED
\n", "• INVALID
\n", "• DUPLICATE
\n", "• WONTFIX
\n", "• WORKSFORME
\n", ">\"\"\")\n", " res.node('Reopened', label=\"<REOPENED>\", style='invis')\n", "\n", "life_cycle.edge('New', 'Assigned', label=r\"Assigned\\lto developer\")\n", "life_cycle.edge('Assigned', 'Resolved', label=\"Developer has fixed bug\")\n", "\n", "life_cycle.edge('Resolution', 'Resolved', arrowhead='none', style='dashed')\n", "\n", "life_cycle" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Unfortunately, software development is more complicated than that. There are many things that can happen while an issue is being worked on.\n", "\n", "* When an issue report comes in, it may be resolved as \"invalid\" or \"duplicate\" before even being assigned to a developer.\n", "* The developer assigned to an issue might change.\n", "* Fixes suggested by developers might be subject to _quality assurance_, to ensure that they\n", " * fix the error under all circumstances\n", " * do not introduce new errors\n", "* Issues may be _reopened_ if the fix is inadequate or new information becomes available." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With this in mind, the states an issue report can go through become a bit more complex. Note how issues start in a state of \"UNCONFIRMED\" as long as nobody has assessed them, and how issues are marked as \"CLOSED\" (the final state) only after the resolution has been checked by quality assurance." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "\n", "life_cycle.node('Unconfirmed', label=\"<UNCONFIRMED>\", penwidth='2.0')\n", "# life_cycle.node('Verified', label=\"<VERIFIED>\")\n", "life_cycle.node('Closed', label=\"<CLOSED>\", penwidth='2.0')\n", "life_cycle.node('Reopened', label=\"<REOPENED>\", style='filled')\n", "life_cycle.node('New', label=\"<NEW>\", penwidth='1.0')\n", "\n", "life_cycle.edge('Unconfirmed', 'New', label=\"Confirmed as \\\"new\\\"\")\n", "life_cycle.edge('Unconfirmed', 'Closed', label=r\"Resolved\\las \\\"invalid\\\"\\lor \\\"duplicate\\\"\")\n", "life_cycle.edge('Assigned', 'New', label=\"Unassigned\")\n", "life_cycle.edge('Resolved', 'Closed', label=r\"Quality Assurance\\lconfirms fix\")\n", "life_cycle.edge('Resolved', 'Reopened', label=r\"Quality Assurance\\lnot satisfied\")\n", "life_cycle.edge('Reopened', 'Assigned', label=r\"Assigned\\lto developer\")\n", "# life_cycle.edge('Verified', 'Closed', label=\"Bug is closed\")\n", "life_cycle.edge('Closed', 'Reopened', label=r\"Bug is\\lreopened\")\n", "\n", "life_cycle" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Such states (and the implied transitions between them) can be found in any issue tracking system. The actual names may vary (ASSIGNED is also known as IN_PROGRESS; instead of or besides CLOSED, one can also have VERIFIED), but their meanings are always the same. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Whatever the states and resolutions are called (and they can be configured, anyway), the important thing is that they be used consistently for your project and in your organization. This is how at any point, any team member can see\n", "\n", "* what the most pressing tasks are;\n", "* what he or she should be working on; and\n", "* what the overall state of the project is." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The latter point is particularly important, since an issue tracker need not only be used for tracking bugs, but also for tracking and assigning _features_ – notably features that have been decided to be part of the product. Hence, if you want to have some new feature implemented, you can mark it as a \"NEW\" issue, break it into subfeatures, also all marked as \"NEW\", and then assign the associated subtasks to individual developers. The feature would then be ready when all subtasks are completed as \"FIXED\"." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This way, you can _organize an entire software project through the issue tracker._ Start with the first issue \"The product is missing\", and then keep on defining and assigning subtasks; when all are resolved, the product will be ready. Since some issue trackers (including _Redmine_) also allow developers to report how far they got with resolving an issue (say, \"10%\", \"50%\", or \"90%\"), you can even estimate how long resolving the open issues will take." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Over time, an issue database not only becomes an important tool for managing bug fixing and organizing software development, it also holds a trove of data about where, what, and why specific issues were addressed. In the [next chapter](ChangeCounter.ipynb), we will explore how to mine and evaluate such data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "\n", "# We're done, so we shut down Redmine and associated processes\n", "redmine_process.terminate()\n", "redmine_gui.close()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ignore\n", "os.system(\"pkill ruby\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Synopsis" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This chapter provides no functionality that could be used by third-party code." ] }, { "cell_type": "markdown", "metadata": { "button": false, "new_sheet": true, "run_control": { "read_only": false } }, "source": [ "## Lessons Learned\n", "\n", "* In an organization, fixing bugs and issues is best organized through an _issue tracker_.\n", "* An _issue tracker_ allows issue reports\n", " * to be _collected_;\n", " * to assign them with a _state_ (from NEW to CLOSED) in its _life cycle_;\n", " * to be _assigned_ to developers; and\n", " * to assign them with a _resolution_ (including FIXED, DUPLICATE, and WORKSFORME).\n", "* Issue trackers can organize the entire software development, including new products and features." ] }, { "cell_type": "markdown", "metadata": { "button": false, "new_sheet": false, "run_control": { "read_only": false } }, "source": [ "## Next Steps\n", "\n", "* The [chapter on mining version histories](ChangeCounter.ipynb) describes how to _mine_ and _analyze_ the data from bug and version databases." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Background\n", "\n", "The survey by Bettenburg et al. \\cite{Bettenburg2008} is the seminal work on how developers make use of bug reports. The study by Bertram et al. \\cite{Bertram2010} highlights how issue trackers are not just databases, but \"a focal point for communication and coordination for many stakeholders within and beyond the software team.\"\n", "\n", "The report by \\cite{Glerum2009} provides details on how _Windows Error Reporting_ automates the processing of error reports coming from an installed base of a billion machines, providing impressive numbers on bugs in the wild.\n", "\n", "Research on bug databases has very much focused on how to make predictions such as automatically identifying duplicates \\cite{Wang2008}, possible bug locations \\cite{Kim2013}, or automatically assigning issues to developers \\cite{Anvik2006}. Herzig et al. \\cite{Herzig2013}, however point out that much of the information in bug database is not classified properly enough, for instance to reliably distinguish between bugs and features.\n", "\n", "Mozilla's [Bugzilla bug database](https://bugzilla.mozilla.org/) is one of the largest publicly available issue databases. It is worth browsing for real-world bug reports, and also contains important articles on how to write and use bug reports." ] } ], "metadata": { "ipub": { "bibliography": "fuzzingbook.bib", "toc": true }, "kernelspec": { "display_name": "venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.10" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": true, "title_cell": "", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": true }, "toc-autonumbering": false, "vscode": { "interpreter": { "hash": "0af4f07dd039d1b4e562c7a7d0340393b1c66f50605ac6af30beb81aa23b7ef5" } } }, "nbformat": 4, "nbformat_minor": 4 }