{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Unit Testing\n",
    "\n",
    "How do you know that your function or class behaves correctly? One very natural way to check for correct behavior is to try a few *test cases*, and compare the result to your expectation. You've been informally doing this throughout this course. In these lecture notes, we'll discuss a *formal* and *automated* way to approach testing your code. This is called *unit testing*, and refers to the practice of checking that each function and class (i.e. each *unit* of your program) behaves as you expect. Almost all production code is extensively unit-tested, and often the \"actual\" code is dwarfed by the amount of code used to implement unit tests. \n",
    "\n",
    "For a review of the informal approach to unit testing, let's look at the following function. This function conducts a *reverse lookup* in a dictionary: that is, it finds all the keys corresponding to a user-specified value. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reverse_lookup(D, val):\n",
    "    \"\"\"\n",
    "    Finds all keys in a dictionary D with specified value val. \n",
    "    \"\"\"\n",
    "    if type(D) != dict:\n",
    "        raise TypeError(\"First argument must be a dictionary.\")\n",
    "    \n",
    "    return [key for key in D.keys() if D[key] == val]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can try a few examples to check that the behavior is as we would expect: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "First argument must be a dictionary.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-2-74937265f2aa>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mD\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"Riker\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"Picard\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"Troi\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mreverse_lookup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mD\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"Picard\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-1-8d7aa0acc961>\u001b[0m in \u001b[0;36mreverse_lookup\u001b[0;34m(D, val)\u001b[0m\n\u001b[1;32m      4\u001b[0m     \"\"\"\n\u001b[1;32m      5\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mD\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mdict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"First argument must be a dictionary.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mkey\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mkey\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mD\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeys\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mD\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mval\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: First argument must be a dictionary."
     ]
    }
   ],
   "source": [
    "D = [\"Riker\", \"Picard\", \"Troi\"]\n",
    "reverse_lookup(D, \"Picard\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D = {\n",
    "    \"Riker\" : \"TNG\",\n",
    "    \"Picard\" : \"TNG\", \n",
    "    \"Sisko\" : \"DS9\"\n",
    "}\n",
    "len(reverse_lookup(D, \"TNG\")) == 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(reverse_lookup(D, \"TOS\")) == 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This works fine for simple functions, but more complex functions or classes might have tens or even hundreds of tests. We wouldn't want to re-run all those tests by hand when we change the code (protip: code is **never** finalized). \n",
    "\n",
    "Let's see how to automate these test cases using the `unittest` module.  To organize and automate our testing, we are going to write a custom *class*. Each method of the class corresponds to one or more of these test cases. This class should inherit from the special class `unittest.TestCase`. When the nature of the tests is sufficiently obvious, it is not necessary to supply docstrings. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import unittest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `unittest` module\n",
    "\n",
    "To organize and automate our testing, we are going to write a custom *class*. Each method of the class corresponds to one or more of these test cases. This class should inherit from the special class `unittest.TestCase`. When the nature of the tests is sufficiently obvious, it is not necessary to supply docstrings. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TestReverseLookup(unittest.TestCase):\n",
    "    \n",
    "    def test_type_error(self):\n",
    "        D = [\"Riker\", \"Picard\", \"Sisko\"]\n",
    "        with self.assertRaises(TypeError):\n",
    "            reverse_lookup(D, \"Picard\")\n",
    "            \n",
    "    def test_standard_lookup(self):\n",
    "        D = {\n",
    "            \"Riker\" : \"TNG\", \n",
    "            \"Picard\" : \"TNG\", \n",
    "            \"Sisko\"  : \"DS9\"\n",
    "        }\n",
    "        result = len(reverse_lookup(D, \"TNG\"))\n",
    "        self.assertEqual(result, 2)\n",
    "    \n",
    "    def test_no_matches(self):\n",
    "        D = {\n",
    "            \"Riker\" : \"TNG\", \n",
    "            \"Picard\" : \"TNG\", \n",
    "            \"Sisko\"  : \"DS9\"\n",
    "        }\n",
    "        \n",
    "        result = len(reverse_lookup(D, \"DS9\"))\n",
    "        self.assertEqual(result, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the tests pass, nothing happens. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "tester = TestReverseLookup()\n",
    "tester.test_type_error()\n",
    "tester.test_standard_lookup()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the other hand, when a test fails, an `assertionError` is raised. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "AssertionError",
     "evalue": "1 != 0",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAssertionError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-f255e0a40e98>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtester\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtest_no_matches\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0;31m# ---\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-6-10d94f8896f5>\u001b[0m in \u001b[0;36mtest_no_matches\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     23\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     24\u001b[0m         \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreverse_lookup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mD\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"DS9\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 25\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0massertEqual\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/unittest/case.py\u001b[0m in \u001b[0;36massertEqual\u001b[0;34m(self, first, second, msg)\u001b[0m\n\u001b[1;32m    910\u001b[0m         \"\"\"\n\u001b[1;32m    911\u001b[0m         \u001b[0massertion_func\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getAssertEqualityFunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfirst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msecond\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 912\u001b[0;31m         \u001b[0massertion_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfirst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msecond\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    913\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    914\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0massertNotEqual\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfirst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msecond\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/unittest/case.py\u001b[0m in \u001b[0;36m_baseAssertEqual\u001b[0;34m(self, first, second, msg)\u001b[0m\n\u001b[1;32m    903\u001b[0m             \u001b[0mstandardMsg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'%s != %s'\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0m_common_shorten_repr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfirst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msecond\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    904\u001b[0m             \u001b[0mmsg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_formatMessage\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstandardMsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 905\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfailureException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    906\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    907\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0massertEqual\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfirst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msecond\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAssertionError\u001b[0m: 1 != 0"
     ]
    }
   ],
   "source": [
    "tester.test_no_matches()\n",
    "# ---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tests in Modules\n",
    "\n",
    "While this test setup works well, it's cumbersome -- you need to call and run the tests by hand each time. The standard and much more convenient approach is to embed your tests into the module in which you define your classes and functions. An example of this approach is shown in the accompanying file `unit_test_example.py`. [Download this file here](https://philchodrow.github.io/PIC16A/content/IO_and_modules/modules/unit_test_example.py). The key trick is in the following two lines: \n",
    "\n",
    "```python\n",
    "if __name__ == \"__main__\":\n",
    "    unittest.main()\n",
    "```\n",
    "\n",
    "The `unittest.main()` method will find all classes that inherit from `unittest.TestCase`, construct an instance of each class, and then run each method of each class exactly once, with custom exception handling to ensure that all tests run even if some of them produce `AssertionErrors`. It will then give a summary of the number of failures and the time it took to run the tests. The first line ensures that the unit tests are performed only when running the module as a script, and not when importing the module. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import unit_test_example # tests not run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In PIC16A, you are not generally required to include unit tests with your code. However, you are responsible for understanding how to construct them, and may be asked to do so in assignments and exams. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}