{ "cells": [ { "cell_type": "markdown", "id": "0a5a1f61", "metadata": {}, "source": [ "\n", "\n", "
\n", " \n", " \"QuantEcon\"\n", " \n", "
" ] }, { "cell_type": "markdown", "id": "68bfa82e", "metadata": {}, "source": [ "# Debugging and Handling Errors\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "id": "58c2b012", "metadata": {}, "source": [ "## Contents\n", "\n", "- [Debugging and Handling Errors](#Debugging-and-Handling-Errors) \n", " - [Overview](#Overview) \n", " - [Debugging](#Debugging) \n", " - [Handling Errors](#Handling-Errors) \n", " - [Exercises](#Exercises) " ] }, { "cell_type": "markdown", "id": "c1dc913d", "metadata": {}, "source": [ "> “Debugging is twice as hard as writing the code in the first place.\n", "> Therefore, if you write the code as cleverly as possible, you are, by definition,\n", "> not smart enough to debug it.” – Brian Kernighan" ] }, { "cell_type": "markdown", "id": "ecb6c2f8", "metadata": {}, "source": [ "## Overview\n", "\n", "Are you one of those programmers who fills their code with `print` statements when trying to debug their programs?\n", "\n", "Hey, we all used to do that.\n", "\n", "(OK, sometimes we still do that…)\n", "\n", "But once you start writing larger programs you’ll need a better system.\n", "\n", "You may also want to handle potential errors in your code as they occur.\n", "\n", "In this lecture, we will discuss how to debug our programs and improve error handling." ] }, { "cell_type": "markdown", "id": "2d269600", "metadata": {}, "source": [ "## Debugging\n", "\n", "\n", "\n", "Debugging tools for Python vary across platforms, IDEs and editors.\n", "\n", "For example, a [visual debugger](https://jupyterlab.readthedocs.io/en/stable/user/debugger.html) is available in JupyterLab.\n", "\n", "Here we’ll focus on Jupyter Notebook and leave you to explore other settings.\n", "\n", "We’ll need the following imports" ] }, { "cell_type": "code", "execution_count": null, "id": "1e323094", "metadata": { "hide-output": false }, "outputs": [], "source": [ "%matplotlib inline\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "plt.rcParams['figure.figsize'] = (10,6)" ] }, { "cell_type": "markdown", "id": "d65dbf18", "metadata": {}, "source": [ "\n", "" ] }, { "cell_type": "markdown", "id": "a999107c", "metadata": {}, "source": [ "### The `debug` Magic\n", "\n", "Let’s consider a simple (and rather contrived) example" ] }, { "cell_type": "code", "execution_count": null, "id": "1527dede", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def plot_log():\n", " fig, ax = plt.subplots(2, 1)\n", " x = np.linspace(1, 2, 10)\n", " ax.plot(x, np.log(x))\n", " plt.show()\n", "\n", "plot_log() # Call the function, generate plot" ] }, { "cell_type": "markdown", "id": "df9bc537", "metadata": {}, "source": [ "This code is intended to plot the `log` function over the interval $ [1, 2] $.\n", "\n", "But there’s an error here: `plt.subplots(2, 1)` should be just `plt.subplots()`.\n", "\n", "(The call `plt.subplots(2, 1)` returns a NumPy array containing two axes objects, suitable for having two subplots on the same figure)\n", "\n", "The traceback shows that the error occurs at the method call `ax.plot(x, np.log(x))`.\n", "\n", "The error occurs because we have mistakenly made `ax` a NumPy array, and a NumPy array has no `plot` method.\n", "\n", "But let’s pretend that we don’t understand this for the moment.\n", "\n", "We might suspect there’s something wrong with `ax` but when we try to investigate this object, we get the following exception:" ] }, { "cell_type": "code", "execution_count": null, "id": "1d2c9974", "metadata": { "hide-output": false }, "outputs": [], "source": [ "ax" ] }, { "cell_type": "markdown", "id": "fbec0c45", "metadata": {}, "source": [ "The problem is that `ax` was defined inside `plot_log()`, and the name is\n", "lost once that function terminates.\n", "\n", "Let’s try doing it a different way.\n", "\n", "We run the first cell block again, generating the same error" ] }, { "cell_type": "code", "execution_count": null, "id": "8f7e8402", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def plot_log():\n", " fig, ax = plt.subplots(2, 1)\n", " x = np.linspace(1, 2, 10)\n", " ax.plot(x, np.log(x))\n", " plt.show()\n", "\n", "plot_log() # Call the function, generate plot" ] }, { "cell_type": "markdown", "id": "05589137", "metadata": {}, "source": [ "But this time we type in the following cell block" ] }, { "cell_type": "markdown", "id": "34407bd4", "metadata": { "hide-output": false }, "source": [ "```ipython\n", "%debug\n", "```\n" ] }, { "cell_type": "markdown", "id": "4cf64ff8", "metadata": {}, "source": [ "You should be dropped into a new prompt that looks something like this" ] }, { "cell_type": "markdown", "id": "740d02a2", "metadata": { "hide-output": false }, "source": [ "```ipython\n", "ipdb>\n", "```\n" ] }, { "cell_type": "markdown", "id": "c17636a4", "metadata": {}, "source": [ "(You might see pdb> instead)\n", "\n", "Now we can investigate the value of our variables at this point in the program, step forward through the code, etc.\n", "\n", "For example, here we simply type the name `ax` to see what’s happening with\n", "this object:" ] }, { "cell_type": "markdown", "id": "5d78b1d6", "metadata": { "hide-output": false }, "source": [ "```ipython\n", "ipdb> ax\n", "array([,\n", " ], dtype=object)\n", "```\n" ] }, { "cell_type": "markdown", "id": "e4bc9f94", "metadata": {}, "source": [ "It’s now very clear that `ax` is an array, which clarifies the source of the\n", "problem.\n", "\n", "To find out what else you can do from inside `ipdb` (or `pdb`), use the\n", "online help" ] }, { "cell_type": "markdown", "id": "c82e66ca", "metadata": { "hide-output": false }, "source": [ "```ipython\n", "ipdb> h\n", "\n", "Documented commands (type help ):\n", "========================================\n", "EOF bt cont enable jump pdef r tbreak w\n", "a c continue exit l pdoc restart u whatis\n", "alias cl d h list pinfo return unalias where\n", "args clear debug help n pp run unt\n", "b commands disable ignore next q s until\n", "break condition down j p quit step up\n", "\n", "Miscellaneous help topics:\n", "==========================\n", "exec pdb\n", "\n", "Undocumented commands:\n", "======================\n", "retval rv\n", "\n", "ipdb> h c\n", "c(ont(inue))\n", "Continue execution, only stop when a breakpoint is encountered.\n", "```\n" ] }, { "cell_type": "markdown", "id": "df8a67b2", "metadata": {}, "source": [ "### Setting a Break Point\n", "\n", "The preceding approach is handy but sometimes insufficient.\n", "\n", "Consider the following modified version of our function above" ] }, { "cell_type": "code", "execution_count": null, "id": "9c83d887", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def plot_log():\n", " fig, ax = plt.subplots()\n", " x = np.logspace(1, 2, 10)\n", " ax.plot(x, np.log(x))\n", " plt.show()\n", "\n", "plot_log()" ] }, { "cell_type": "markdown", "id": "eb45a297", "metadata": {}, "source": [ "Here the original problem is fixed, but we’ve accidentally written\n", "`np.logspace(1, 2, 10)` instead of `np.linspace(1, 2, 10)`.\n", "\n", "Now there won’t be any exception, but the plot won’t look right.\n", "\n", "To investigate, it would be helpful if we could inspect variables like `x` during execution of the function.\n", "\n", "To this end, we add a “break point” by inserting `breakpoint()` inside the function code block" ] }, { "cell_type": "markdown", "id": "2794cae1", "metadata": { "hide-output": false }, "source": [ "```python3\n", "def plot_log():\n", " breakpoint()\n", " fig, ax = plt.subplots()\n", " x = np.logspace(1, 2, 10)\n", " ax.plot(x, np.log(x))\n", " plt.show()\n", "\n", "plot_log()\n", "```\n" ] }, { "cell_type": "markdown", "id": "2d900dc5", "metadata": {}, "source": [ "Now let’s run the script, and investigate via the debugger" ] }, { "cell_type": "markdown", "id": "7fe506f0", "metadata": { "hide-output": false }, "source": [ "```ipython\n", "> (6)plot_log()\n", "-> fig, ax = plt.subplots()\n", "(Pdb) n\n", "> (7)plot_log()\n", "-> x = np.logspace(1, 2, 10)\n", "(Pdb) n\n", "> (8)plot_log()\n", "-> ax.plot(x, np.log(x))\n", "(Pdb) x\n", "array([ 10. , 12.91549665, 16.68100537, 21.5443469 ,\n", " 27.82559402, 35.93813664, 46.41588834, 59.94842503,\n", " 77.42636827, 100. ])\n", "```\n" ] }, { "cell_type": "markdown", "id": "ec819fd3", "metadata": {}, "source": [ "We used `n` twice to step forward through the code (one line at a time).\n", "\n", "Then we printed the value of `x` to see what was happening with that variable.\n", "\n", "To exit from the debugger, use `q`." ] }, { "cell_type": "markdown", "id": "50a84f25", "metadata": {}, "source": [ "### Other Useful Magics\n", "\n", "In this lecture, we used the `%debug` IPython magic.\n", "\n", "There are many other useful magics:\n", "\n", "- `%precision 4` sets printed precision for floats to 4 decimal places \n", "- `%whos` gives a list of variables and their values \n", "- `%quickref` gives a list of magics \n", "\n", "\n", "The full list of magics is [here](http://ipython.readthedocs.org/en/stable/interactive/magics.html)." ] }, { "cell_type": "markdown", "id": "1a991327", "metadata": {}, "source": [ "## Handling Errors\n", "\n", "\n", "\n", "Sometimes it’s possible to anticipate bugs and errors as we’re writing code.\n", "\n", "For example, the unbiased sample variance of sample $ y_1, \\ldots, y_n $\n", "is defined as\n", "\n", "$$\n", "s^2 := \\frac{1}{n-1} \\sum_{i=1}^n (y_i - \\bar y)^2\n", "\\qquad \\bar y = \\text{ sample mean}\n", "$$\n", "\n", "This can be calculated in NumPy using `np.var`.\n", "\n", "But if you were writing a function to handle such a calculation, you might\n", "anticipate a divide-by-zero error when the sample size is one.\n", "\n", "One possible action is to do nothing — the program will just crash, and spit out an error message.\n", "\n", "But sometimes it’s worth writing your code in a way that anticipates and deals with runtime errors that you think might arise.\n", "\n", "Why?\n", "\n", "- Because the debugging information provided by the interpreter is often less useful than what can be provided by a well written error message. \n", "- Because errors that cause execution to stop interrupt workflows. \n", "- Because it reduces confidence in your code on the part of your users (if you are writing for others). \n", "\n", "\n", "In this section, we’ll discuss different types of errors in Python and techniques to handle potential errors in our programs." ] }, { "cell_type": "markdown", "id": "bd15c009", "metadata": {}, "source": [ "### Errors in Python\n", "\n", "We have seen `AttributeError` and `NameError` in [our previous examples](#debug-magic).\n", "\n", "In Python, there are two types of errors – syntax errors and exceptions.\n", "\n", "\n", "\n", "Here’s an example of a common error type" ] }, { "cell_type": "code", "execution_count": null, "id": "611c0cf7", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def f:" ] }, { "cell_type": "markdown", "id": "ecf1328c", "metadata": {}, "source": [ "Since illegal syntax cannot be executed, a syntax error terminates execution of the program.\n", "\n", "Here’s a different kind of error, unrelated to syntax" ] }, { "cell_type": "code", "execution_count": null, "id": "4cb85327", "metadata": { "hide-output": false }, "outputs": [], "source": [ "1 / 0" ] }, { "cell_type": "markdown", "id": "b39cdc57", "metadata": {}, "source": [ "Here’s another" ] }, { "cell_type": "code", "execution_count": null, "id": "b500da87", "metadata": { "hide-output": false }, "outputs": [], "source": [ "x1 = y1" ] }, { "cell_type": "markdown", "id": "3fb44e23", "metadata": {}, "source": [ "And another" ] }, { "cell_type": "code", "execution_count": null, "id": "6d56888a", "metadata": { "hide-output": false }, "outputs": [], "source": [ "'foo' + 6" ] }, { "cell_type": "markdown", "id": "106d4e65", "metadata": {}, "source": [ "And another" ] }, { "cell_type": "code", "execution_count": null, "id": "76d5f388", "metadata": { "hide-output": false }, "outputs": [], "source": [ "X = []\n", "x = X[0]" ] }, { "cell_type": "markdown", "id": "552eca41", "metadata": {}, "source": [ "On each occasion, the interpreter informs us of the error type\n", "\n", "- `NameError`, `TypeError`, `IndexError`, `ZeroDivisionError`, etc. \n", "\n", "\n", "In Python, these errors are called *exceptions*." ] }, { "cell_type": "markdown", "id": "6590437b", "metadata": {}, "source": [ "### Assertions\n", "\n", "\n", "\n", "Sometimes errors can be avoided by checking whether your program runs as expected.\n", "\n", "A relatively easy way to handle checks is with the `assert` keyword.\n", "\n", "For example, pretend for a moment that the `np.var` function doesn’t\n", "exist and we need to write our own" ] }, { "cell_type": "code", "execution_count": null, "id": "4f05de9c", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def var(y):\n", " n = len(y)\n", " assert n > 1, 'Sample size must be greater than one.'\n", " return np.sum((y - y.mean())**2) / float(n-1)" ] }, { "cell_type": "markdown", "id": "cc6cd842", "metadata": {}, "source": [ "If we run this with an array of length one, the program will terminate and\n", "print our error message" ] }, { "cell_type": "code", "execution_count": null, "id": "eea7738c", "metadata": { "hide-output": false }, "outputs": [], "source": [ "var([1])" ] }, { "cell_type": "markdown", "id": "d5ad69a1", "metadata": {}, "source": [ "The advantage is that we can\n", "\n", "- fail early, as soon as we know there will be a problem \n", "- supply specific information on why a program is failing " ] }, { "cell_type": "markdown", "id": "c3ed60e4", "metadata": {}, "source": [ "### Handling Errors During Runtime\n", "\n", "\n", "\n", "The approach used above is a bit limited, because it always leads to\n", "termination.\n", "\n", "Sometimes we can handle errors more gracefully, by treating special cases.\n", "\n", "Let’s look at how this is done." ] }, { "cell_type": "markdown", "id": "05ccb83f", "metadata": {}, "source": [ "#### Catching Exceptions\n", "\n", "We can catch and deal with exceptions using `try` – `except` blocks.\n", "\n", "Here’s a simple example" ] }, { "cell_type": "code", "execution_count": null, "id": "5f55b3e2", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def f(x):\n", " try:\n", " return 1.0 / x\n", " except ZeroDivisionError:\n", " print('Error: division by zero. Returned None')\n", " return None" ] }, { "cell_type": "markdown", "id": "1cb17c3b", "metadata": {}, "source": [ "When we call `f` we get the following output" ] }, { "cell_type": "code", "execution_count": null, "id": "e05c81db", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f(2)" ] }, { "cell_type": "code", "execution_count": null, "id": "103e1c26", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f(0)" ] }, { "cell_type": "code", "execution_count": null, "id": "8795133a", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f(0.0)" ] }, { "cell_type": "markdown", "id": "d038af10", "metadata": {}, "source": [ "The error is caught and execution of the program is not terminated.\n", "\n", "Note that other error types are not caught.\n", "\n", "If we are worried the user might pass in a string, we can catch that error too" ] }, { "cell_type": "code", "execution_count": null, "id": "f0b5e122", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def f(x):\n", " try:\n", " return 1.0 / x\n", " except ZeroDivisionError:\n", " print('Error: Division by zero. Returned None')\n", " except TypeError:\n", " print(f'Error: x cannot be of type {type(x)}. Returned None')\n", " return None" ] }, { "cell_type": "markdown", "id": "4146e5fa", "metadata": {}, "source": [ "Here’s what happens" ] }, { "cell_type": "code", "execution_count": null, "id": "873d4c32", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f(2)" ] }, { "cell_type": "code", "execution_count": null, "id": "ec079290", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f(0)" ] }, { "cell_type": "code", "execution_count": null, "id": "ec9d2879", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f('foo')" ] }, { "cell_type": "markdown", "id": "d480bee5", "metadata": {}, "source": [ "If we feel lazy we can catch these errors together" ] }, { "cell_type": "code", "execution_count": null, "id": "41d0559f", "metadata": { "hide-output": false }, "outputs": [], "source": [ "def f(x):\n", " try:\n", " return 1.0 / x\n", " except:\n", " print(f'Error. An issue has occurred with x = {x} of type: {type(x)}')\n", " return None" ] }, { "cell_type": "markdown", "id": "44072eaa", "metadata": {}, "source": [ "Here’s what happens" ] }, { "cell_type": "code", "execution_count": null, "id": "eef0690a", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f(2)" ] }, { "cell_type": "code", "execution_count": null, "id": "a9ae27ee", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f(0)" ] }, { "cell_type": "code", "execution_count": null, "id": "aeebf715", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f('foo')" ] }, { "cell_type": "markdown", "id": "3d1c932f", "metadata": {}, "source": [ "In general it’s better to be specific." ] }, { "cell_type": "markdown", "id": "6f9f4405", "metadata": {}, "source": [ "## Exercises" ] }, { "cell_type": "markdown", "id": "c7cbc05f", "metadata": {}, "source": [ "## Exercise 21.1\n", "\n", "Suppose we have a text file `numbers.txt` containing the following lines" ] }, { "cell_type": "markdown", "id": "fe3c97bc", "metadata": { "hide-output": false }, "source": [ "```text\n", "prices\n", "3\n", "8\n", "\n", "7\n", "21\n", "```\n" ] }, { "cell_type": "markdown", "id": "d7301471", "metadata": {}, "source": [ "Using `try` – `except`, write a program to read in the contents of the file and sum the numbers, ignoring lines without numbers.\n", "\n", "You can use the `open()` function we learnt [before](https://python-programming.quantecon.org/python_advanced_features.html#iterators) to open `numbers.txt`." ] }, { "cell_type": "markdown", "id": "f4b48f64", "metadata": {}, "source": [ "## Solution to[ Exercise 21.1](https://python-programming.quantecon.org/#debug_ex1)\n", "\n", "Let’s save the data first" ] }, { "cell_type": "code", "execution_count": null, "id": "ae4e257a", "metadata": { "hide-output": false }, "outputs": [], "source": [ "%%file numbers.txt\n", "prices\n", "3\n", "8\n", "\n", "7\n", "21" ] }, { "cell_type": "code", "execution_count": null, "id": "cdbe7e70", "metadata": { "hide-output": false }, "outputs": [], "source": [ "f = open('numbers.txt')\n", "\n", "total = 0.0\n", "for line in f:\n", " try:\n", " total += float(line)\n", " except ValueError:\n", " pass\n", "\n", "f.close()\n", "\n", "print(total)" ] } ], "metadata": { "date": 1710455931.8385432, "filename": "debugging.md", "kernelspec": { "display_name": "Python", "language": "python3", "name": "python3" }, "title": "Debugging and Handling Errors" }, "nbformat": 4, "nbformat_minor": 5 }