{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "f415941a3a1516f0b3e2b93249aff57d",
     "grade": false,
     "grade_id": "cell-1015fd7f8a1783c7",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# Part 1: Forward Mode Automatic Differentiation\n",
    "\n",
    "Forward mode AD can simply be implemented by defining a class to represent [dual numbers](https://en.wikipedia.org/wiki/Dual_number) which hold the value and its derivative. The following skeleton defines a dual number and implements multiplication. \n",
    "\n",
    "__Tasks:__\n",
    "\n",
    "- Addition (`__add__`) is incomplete - can you finish it? \n",
    "- Can you also implement division (`__truediv__`), subtraction (`__sub__`) and power (`__pow__`)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "33fa490908871a743dabb3060ebe112e",
     "grade": false,
     "grade_id": "cell-a645b47036f56c2e",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "class DualNumber:\n",
    "    def __init__(self, value, dvalue):\n",
    "        self.value = value\n",
    "        self.dvalue = dvalue\n",
    "\n",
    "    def __str__(self):\n",
    "        return str(self.value) + \" + \" + str(self.dvalue) + \"ε\"\n",
    "\n",
    "    def __mul__(self, other):\n",
    "        return DualNumber(self.value * other.value,\n",
    "            self.dvalue * other.value + other.dvalue * self.value)\n",
    "    \n",
    "    def __add__(self, other):\n",
    "        #TODO: finish me\n",
    "        # YOUR CODE HERE\n",
    "        raise NotImplementedError()\n",
    "    \n",
    "    # TODO: add missing methods\n",
    "    # YOUR CODE HERE\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "e9afb52c0524256aa416c8e0ceec60de",
     "grade": true,
     "grade_id": "cell-7923d5c39d574cd7",
     "locked": true,
     "points": 3,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Tests\n",
    "\n",
    "DualNumber(1,0) + DualNumber(1,0) / DualNumber(1,0) - DualNumber(1,0)**DualNumber(1,0)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "be03930a421d27414e745ac5839fddd3",
     "grade": false,
     "grade_id": "cell-2c8ce137dbdd2f9d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Implementing math functions\n",
    "\n",
    "We also need to implement some core math functions. Here's the sine function for a dual number:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "1fc46030449c0860972520bddf2a498d",
     "grade": false,
     "grade_id": "cell-30886f6180e2ca7c",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def sin(x):\n",
    "    return DualNumber(math.sin(x.value), math.cos(x.value)*x.dvalue)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "54b68135467f7b83dfa9749df021c932",
     "grade": false,
     "grade_id": "cell-9b585bf1afe72cca",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "__Task:__ can you implement the _cosine_ (`cos`), _tangent_ (`tan`), and _exponential_ (`exp`) functions in the code block below?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "2bea3862089cbfe7a78ceaa915a87a3b",
     "grade": false,
     "grade_id": "cell-61b9462cb781a8e1",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# TODO: implement additional math functions on dual numbers\n",
    "\n",
    "def cos(x):\n",
    "    # YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "\n",
    "def tan(x):\n",
    "    # YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "\n",
    "def exp(x):\n",
    "    # YOUR CODE HERE\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "29c49048fd70b07ab3819fcb5e4972ec",
     "grade": true,
     "grade_id": "cell-9055ec23e6e9ffb8",
     "locked": true,
     "points": 3,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Tests\n",
    "assert cos(DualNumber(0,0)).value == 1\n",
    "assert tan(DualNumber(0,0)).value == 0\n",
    "assert exp(DualNumber(0,0)).value == 1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "5f834befbd408096b4d11ec9cd22159c",
     "grade": false,
     "grade_id": "cell-0429b8810d0cf3a3",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Time to try it out\n",
    "\n",
    "We're now in a position to try our implementation.\n",
    "\n",
    "__Task:__ \n",
    "\n",
    "- Try running the following code to compute the value of the function $z=x\\cdot y+sin(x)$ given $x=0.5$ and $y=4.2$, together with the derivative $\\partial z/\\partial x$ at that point. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "ff1399f6f7101774bd4d3e9c49ed6458",
     "grade": false,
     "grade_id": "cell-12f7c8fa42784946",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "7719838a896c466bcc58ecb273f89175",
     "grade": false,
     "grade_id": "cell-204b8d399904b523",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "__Task__: Differentiate the above function with respect to $x$ and write the symbolic derivatives in the following box. Verify the result computed above is correct by plugging-in the values into your symbolic gradient expression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "07ca9c4df262fa25d98cea6f0e36d979",
     "grade": true,
     "grade_id": "cell-1ae4d0a0801a142a",
     "locked": false,
     "points": 2,
     "schema_version": 1,
     "solution": true
    }
   },
   "source": [
    "YOUR ANSWER HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Task:__ Now use the code block below to compute the derivative $\\partial z/\\partial y$ of the above expression (at the same point $x=0.5, y=4.2$ as above) and store the derivative in the variable `dzdy` (just the derivative, not the Dual Number). Verify by hand that the result is correct."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "cd3fbc66891ec630d8d3029df8ad0766",
     "grade": false,
     "grade_id": "cell-eab763bf1b218dc8",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "\n",
    "print('dz/dy:', dzdy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "0e76a35421fc942d840771242e268dad",
     "grade": true,
     "grade_id": "cell-a2f9f0ca4b1ed763",
     "locked": true,
     "points": 2,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "#Tests\n",
    "assert dzdy\n",
    "assert type(dzdy) == float\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "827a8ca599d07c42685e571f2491cf48",
     "grade": false,
     "grade_id": "cell-8faa6c905dd61e82",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "__Task:__ Finally, use the code block below to experiment and test the other math functions and methods you created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "7c7f4bfbb93248c24b6fac692d6d3109",
     "grade": true,
     "grade_id": "cell-b71d4bb64928ee38",
     "locked": false,
     "points": 0,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}