{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numba 0.41.0 Release Demo\n",
    "=======================\n",
    "\n",
    "This notebook contains a demonstration of new features present in the 0.41.0 release of Numba. Whilst release notes are produced as part of the [`CHANGE_LOG`](https://github.com/numba/numba/blob/8acc937e3199f9cd9bac1eba5be5d8d2b5508815/CHANGE_LOG), there's nothing like seeing code in action!\n",
    "\n",
    "Included are demonstrations of:\n",
    "\n",
    "* Initial support for Python 3 Unicode strings\n",
    "* Diagnostics showing the optimizations performed by ParallelAccelerator\n",
    "* Newly supported NumPy functions\n",
    "* Literal values support (for developers of Numba/Numba extensions)\n",
    "* Tracebacks from exceptions\n",
    "\n",
    "First, import the necessary from Numba and NumPy..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numba import njit, config, __version__\n",
    "from numba.extending import overload\n",
    "import numpy as np\n",
    "assert tuple(int(x) for x in __version__.split('.')[:2]) >= (0, 41)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unicode strings\n",
    "--------------\n",
    "Initial support for Unicode strings has been implemented for Python versions >= 3.4. Support for fundamental string operations has been added as well as support for strings as arguments and return value. The next release of Numba will contain performance updates and additional features to further enhance string support."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if config.PYVERSION > (3, 4): # Only supported in Python >= 3.4\n",
    "    \n",
    "    @njit\n",
    "    def strings_demo(str1, str2, str3):\n",
    "        # strings, ---^  ---^   ---^\n",
    "        # as arguments are now supported!\n",
    "        \n",
    "        # defining strings in compiled code also works\n",
    "        def1 = 'numba is '\n",
    "        \n",
    "        # as do unicode strings\n",
    "        def2 = '🐍⚡'\n",
    "        \n",
    "        # also string concatenation \n",
    "        print(str1 + str2)\n",
    "        \n",
    "        # comparison operations\n",
    "        print(str1 == str2)\n",
    "        print(str1 < str2)\n",
    "        print(str1 <= str2)\n",
    "        print(str1 > str2)\n",
    "        print(str1 >= str2)\n",
    "        \n",
    "        # {starts,ends}with\n",
    "        print(str1.startswith(str3))\n",
    "        print(str2.endswith(str3))\n",
    "        \n",
    "        # len()\n",
    "        print(len(str1), len(def2), len(str3))\n",
    "        \n",
    "        # str.find()\n",
    "        print(str2.find(str3))\n",
    "        \n",
    "        # in\n",
    "        print(str3 in str2)\n",
    "        \n",
    "        # slicing\n",
    "        print(str2[1:], str1[:1])\n",
    "        \n",
    "        # and finally, strings can also be returned\n",
    "        return '\\nnum' + str1[1::-1] + def1[5:] + def2\n",
    "    \n",
    "    \n",
    "    # run the demo\n",
    "    print(strings_demo('abc', 'zba', 'a'))\n",
    "    \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ParallelAccelerator Optimization Diagnostics\n",
    "-----\n",
    "The `ParallelAccelerator` technology is used when the `parallel=True` kwarg is supplied to `@jit`. This technology is what transforms the decorated function into one that can run on multiple CPUs. Whilst `parallel=True` has been implemented for some time, the optimizations taking place have not been exposed in a manner that is easy to understand. Numba 0.41.0 contains the first cut of a new diagnostics tool that aims to help demystify what ParallelAccelerator does internally as it transforms the function to run in parallel!\n",
    "\n",
    "Documentation for this feature is available [here](http://numba.pydata.org/numba-doc/dev/user/parallel.html#diagnostics), including an explanation of how to interpret the parallel diagnostics output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from numba import prange # import parallel range\n",
    "\n",
    "# decorate a function with `parallel=True` as usual\n",
    "@njit(parallel=True)\n",
    "def test(x):\n",
    "    n = x.shape[0]\n",
    "    a = np.sin(x)                      # parallel array expression\n",
    "    b = np.cos(a * a)                  # parallel array expression\n",
    "    acc = 0                            \n",
    "    for i in prange(n - 2):            # user defined parallel loop\n",
    "        for j in prange(n - 1):        # user defined parallel loop\n",
    "            acc += b[i] + b[j + 1]     # parallel reduction\n",
    "    return acc\n",
    "\n",
    "# run the function\n",
    "test(np.arange(10))\n",
    "\n",
    "# access the diagnostic output via the new `parallel_diagnostics` method on the dispatcher\n",
    "test.parallel_diagnostics(level=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Newly supported NumPy functions\n",
    "------\n",
    "This release contains a number of newly supported NumPy functions:\n",
    "* Triangular matrix creation/manipulation: `tri`, `tril`, `triu`\n",
    "* Partioning and element wise difference computation: `partition`, `ediff1d`\n",
    "* Covariance: `cov`\n",
    "* NaN based reductions: `nancumsum`, `nancumprod`\n",
    "* Conjugation: `conj`, `conjugate`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@njit\n",
    "def numpy_new():\n",
    " \n",
    "    # create some simple array data for use in np.tril and np.triu\n",
    "    a = np.arange(12.).reshape(3, 4)\n",
    "    print('Input array:')\n",
    "    print(a)\n",
    "    \n",
    "    # try out np.tri, np.triu, np.tril\n",
    "    print('np.tri(3)')\n",
    "    print(np.tri(3))\n",
    "    print('np.tril(a)')\n",
    "    print(np.tril(a))\n",
    "    print('np.triu(a, k=1)')\n",
    "    print(np.triu(a, k=1))\n",
    "    \n",
    "    # copy and shuffle the simple array data for use with np.partition, np.ediff1d and np.cov\n",
    "    a_unordered = a.copy()\n",
    "    np.random.seed(0)\n",
    "    np.random.shuffle(a_unordered.ravel())\n",
    "    print('\\nInput array:')\n",
    "    print(a_unordered)\n",
    "    \n",
    "    # try out np.partition, np.ediff1d and np.cov\n",
    "    print('np.partition(a_unordered, 0)')\n",
    "    print(np.partition(a_unordered, 0))\n",
    "    print('np.ediff1d(a_unordered)')\n",
    "    print(np.ediff1d(a_unordered))  \n",
    "    print('np.cov(a_unordered)')\n",
    "    print(np.cov(a_unordered))\n",
    "    \n",
    "    # create some data with NaN present to try out np.nancumsum and np.nancumprod\n",
    "    a_w_nan = a.copy()\n",
    "    a_w_nan.ravel()[::2] = np.nan\n",
    "    print('\\nInput array:')\n",
    "    print(a_w_nan)\n",
    "\n",
    "    # try out np.nancumsum and np.nancumprod\n",
    "    print('np.nancumsum(a_w_nan)')\n",
    "    print(np.nancumsum(a_w_nan))\n",
    "    print('np.nancumprod(a_w_nan)')\n",
    "    print(np.nancumprod(a_w_nan))\n",
    "    \n",
    "    # finally, create some data in the complex domain to try out np.conj and np.conjugate\n",
    "    a_cmplx = a.copy() + a_unordered.copy() * 1j\n",
    "    print('\\nInput array:')\n",
    "    print(a_cmplx)\n",
    "    \n",
    "    # try out np.conj and np.conjugate\n",
    "    print('np.conj(a_cmplx)')\n",
    "    print(np.conj(a_cmplx))\n",
    "    print('np.conjugate(a_cmplx)')\n",
    "    print(np.conjugate(a_cmplx))\n",
    "    \n",
    "    \n",
    "numpy_new()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Literal value support\n",
    "------------------------------\n",
    "Numba 0.41.0 has a significant change made to the typing system that aims to clean up the use of constants. This change takes the form of support for type specific literal values in the type inference mechanism. During typing two passes are now made, the first with anything which is a constant and can expressed as a literal set as such (integers, strings, slices and `make_function` are implemented as literals at present), the second with the standard types used for the constants. This, for example, permits value based dispatch as demonstrated below, but also opens up a lot of future possibilities surrounding typing which were inaccessible prior to this change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numba import generated_jit\n",
    "\n",
    "@generated_jit\n",
    "def myoverload(arg):\n",
    "    literal_val = getattr(arg, 'literal_value', None)\n",
    "    if literal_val is not None:\n",
    "        if literal_val == 100:\n",
    "            def impl_1(arg):\n",
    "                return 'dispatched: impl_1(literal, value 100)'\n",
    "            return impl_1\n",
    "        else:\n",
    "            def impl_2(arg):\n",
    "                return 'dispatched: impl_2(literal, value not 100)'\n",
    "            return impl_2\n",
    "    else:\n",
    "        def impl_3(arg):\n",
    "                return 'dispatched: impl_3(non-literal type)'\n",
    "        return impl_3\n",
    "\n",
    "@njit\n",
    "def example(x):\n",
    "    print(myoverload(100))         # literal value 100, dispatches impl_1\n",
    "    print(myoverload(99))          # literal value 99, dispatches impl_2\n",
    "    a = 50 + 25 + 2 * 10 + 15 // 3 # `a` is const expr value 100\n",
    "    print(myoverload(a))           # `a` has literal value 100, dispatches impl_1\n",
    "    b = 50 * x                     # `b` non-literal, it's an intp type\n",
    "    print(myoverload(b))           # `b` non-literal intp, has no value, dispatches impl_3\n",
    "\n",
    "example(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exceptions with tracebacks relating to Python source.\n",
    "-----------------------------------------------------\n",
    "\n",
    "Finally (and left to last as an exception is raised!), tracebacks from exceptions raised in jitted code now contain a synthesized stack frame containing the location where the exception was raised. The stack frame is based on the Python source from which is was compiled, it looks like a CPython traceback, but is coming from compiled code! This makes it easier to use exceptions in `nopython` mode as it is now possible to find out the location from which they were raised. Try commenting/uncommenting the `@njit` decorator and rerunning!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@njit\n",
    "def raise_exception(x):\n",
    "    if x == 0:\n",
    "        raise Exception('raised x==0. Also, exception arguments are correctly handled', 123, 4j)\n",
    "\n",
    "raise_exception(0)   "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}