{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# The IPython shell"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- IPython is an *enhanced* Python shell for **interactive** computing\n",
    "- Provides facilities for distributed and parallel computing \n",
    "- Allows interactive evaluation of results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18.84955592153876"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from math import pi \n",
    "r = 3.\n",
    "2 * pi * r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "> This flexible style matches well **the spirit of computing in a scientific context**, in\n",
    "> which **determining what computations must be performed next often requires significant**\n",
    "> **work**. An interactive environment lets scientists look at data, test new ideas, combine\n",
    "> algorithmic approaches, and evaluate their outcomes directly.\n",
    "> \n",
    "> [Pérez and Granger, *IPython: A System for Interactive Scientific Computing*](http://dx.doi.org/10.1109/MCSE.2007.53)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "IPython is an enhanced Python shell for interactive distributed and parallel computing. When users are testing new ideas and algorithms in computer vision, evaluating the\n",
    "results directly, in an interactive way, is more convenient than the traditional *compile-then-execute* cycle. A live computing state is convenient because previous intermediate results (images, feature vectors, parameters) are kept available for exploration. [Pérez and Granger](http://dx.doi.org/10.1109/MCSE.2007.53) argue that, in a research context, determining which computations must be performed next often requires significant work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Access to *all session state*\n",
    "- `TAB` completion\n",
    "- System shell integration\n",
    "- Dynamic introspection and help\n",
    "  - Direct manipulation of objects in memory\n",
    "- *Magic commands* (secondary control system)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Session state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "37.69911184307752"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "2 * _1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18.84955592153876"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "In an interactive computing system, users should have access to all session state. In\n",
    "IPython, this access is not just provided by the dynamically attributed variables, but also by numbered output prompts. Previous computations can be retrieved using and underscore \"`_`\" and the number of the output."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### System shell integration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "Other useful capabilities in IPython are `TAB` completion and system shell integration. \n",
    "When the user types the `TAB` key, IPython tries to complete the current prompt with keywords or the names of methods, variables and files in the current directory. This is particularly useful when exploring unfamiliar or large APIs as in OpenCV or in the SciPy library. Regarding system shell integration, IPython cannot only call any system command, but also can capture the shell’s output in Python variables and call system commands with values computed from variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Call a system's command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ls: não é possível acessar '../data/*.jpg': Arquivo ou diretório não encontrado\r\n"
     ]
    }
   ],
   "source": [
    "ls ../data/*.jpg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Store the command output in a Python variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['data/BSD-118035.jpg',\n",
       " 'data/girl.jpg',\n",
       " 'data/skin-training.jpg',\n",
       " 'data/thiago.jpg']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jpgs_list = !ls data/*.jpg\n",
    "jpgs_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Call a system's command passing arguments stored in Python variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20K\tdata/BSD-118035.jpg\n",
      "-rw-r--r-- 1 thiago thiago 18556 Fev 10 14:36 data/BSD-118035.jpg\n"
     ]
    }
   ],
   "source": [
    "fname = jpgs_list[0]\n",
    "!du -h $fname\n",
    "!ls -l $fname"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### *Magic functions*\n",
    "\n",
    "- IPython presents 98 **magic functions**\n",
    "- They are a secondary command system, providing extra capabilities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%magic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Some examples\n",
    "\n",
    "- `%load` and `%save` - load/save code into/from IPython\n",
    "- `%run`- run a Python file inside IPython as a program\n",
    "- `%who` - list all interactive variables\n",
    "- `%timeit` - time execution of a Python statement or expression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `%who`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fname\t jpgs_list\t pi\t r\t \n"
     ]
    }
   ],
   "source": [
    "%who"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fname\t \n"
     ]
    }
   ],
   "source": [
    "%who str"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pi\t r\t \n"
     ]
    }
   ],
   "source": [
    "%who float"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `%timeit`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The slowest run took 106.32 times longer than the fastest. This could mean that an intermediate result is being cached.\n",
      "10000000 loops, best of 3: 47.1 ns per loop\n"
     ]
    }
   ],
   "source": [
    "from random import random\n",
    "%timeit random()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Import the linear algebra module in SciPy\n",
    "from scipy import linalg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Create a random $100 \\times 100$ matrix $A$ and time the Singular Value Decompostion (SVD) $A = U\\Sigma V^\\intercal$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from numpy.random import randn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100 loops, best of 3: 8.47 ms per loop\n"
     ]
    }
   ],
   "source": [
    "%%timeit A = randn(10000).reshape(100,100)\n",
    "linalg.svd(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Timing 2D features detection (SIFT vs. SURF)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import cv2\n",
    "lenna = cv2.imread('data/lenna.tiff', cv2.IMREAD_GRAYSCALE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 loops, best of 3: 150 ms per loop\n"
     ]
    }
   ],
   "source": [
    "sift = cv2.xfeatures2d.SIFT_create()\n",
    "%timeit sift.detectAndCompute(lenna, None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 loops, best of 3: 122 ms per loop\n"
     ]
    }
   ],
   "source": [
    "surf = cv2.xfeatures2d.SURF_create()\n",
    "%timeit surf.detectAndCompute(lenna, None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The IPython notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Web-based version of the IPython shell\n",
    "- Chunks of text and code are organized in **cells**\n",
    "- Cells can be inserted, deleted, rearranged and executed as needed\n",
    "- Notebooks can be **converted** to Python programs, PDF documents and slides\n",
    "  - These slides and the tutorial's handouts are notebooks convertions!\n",
    "- IPython notebooks are **executable documents**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "The IPython *Notebook* is a web-based version of the IPython shell presenting extended functionality. In the notebook, the user can organize formatted text and code blocks in a\n",
    "flexible way. Text and code are organized in cells that can be inserted, deleted, rearranged\n",
    "and executed as needed. IPython notebooks can handle plots, mathematical formulas and code output, everything organized in a single executable document. Notebooks are being used for research notes, and on the production of [articles](http://nbviewer.ipython.org) and [books](http://nbviewer.ipython.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter1_Introduction/Chapter1_Introduction.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Markdown cells\n",
    "\n",
    "In a notebook, a **Markdown cell** is able the render rich-formatted text using the [Markdown markup convention](http://daringfireball.net/projects/markdown). Mathematical notation is defined using LaTeX syntax and rendered by MathJax. \n",
    "\n",
    "    Let $\\mu_I$ and $\\sigma_II$ be the mean and the standard deviation of a grayscale \n",
    "    image $I$. The whitening operation is defined by:\n",
    "    \n",
    "    $W_I[i,j] = \\frac{I[i,j] - \\mu_I}{\\sigma_I}$.\n",
    " \n",
    "    \n",
    "> Let $\\mu_I$ and $\\sigma_II$ be the mean and the standard deviation of a grayscale \n",
    "> image $I$. The whitening operation is defined by:\n",
    ">    \n",
    "> $W_I[i,j] = \\frac{I[i,j] - \\mu_I}{\\sigma_I}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Code cells\n",
    "\n",
    "- Code cells keep (short) Python scripts\n",
    "- The browser sends the code to the server running an *IPython Kernel*\n",
    "- Output is sent back to the browser for exhibition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, World! I'm a string in an IPython notebook code cell.\n"
     ]
    }
   ],
   "source": [
    "hello = \"Hello, World! I'm a string in an IPython notebook code cell.\"\n",
    "print hello"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Hello,',\n",
       " 'World!',\n",
       " \"I'm\",\n",
       " 'a',\n",
       " 'string',\n",
       " 'in',\n",
       " 'an',\n",
       " 'IPython',\n",
       " 'notebook',\n",
       " 'code',\n",
       " 'cell.']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hello.split()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "*Code cells* contain Python code that is sent to the IPython interpreter running on the server, executed, and the resulting output sent back to the browser for exhibition. That means that IPython can be running in a powerful machine like a server while the user is able to perform her work from a leaner system as a laptop or a tablet. If the result of a code block is a plot, it can be exhibited inside the notebook in the browser (*inline plotting*)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Notebook files (`.ipynb`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- IPython notebooks are stored as *JSON files*\n",
    "- Can be converted to:\n",
    "    - a Python script\n",
    "    - a reStructuredText document\n",
    "    - a HTML document\n",
    "    - a LaTeX document\n",
    "    - a Markdown document\n",
    "    - slides\n",
    "  \n",
    "- For details, check:\n",
    "\n",
    "`$ jupyter nbconvert --help`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "IPython notebooks are stored in JSON files that keep the cells’ content. These files uses the extension `.ipynb` and can be exported as Python scripts, HTML documents or even printed. But what makes notebook files suitable to reproducible research is the fact they are executable documents, not only able to store textual and mathematical descriptions but also replicate the computations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Starting IPython\n",
    "\n",
    "The IPython default console (**terminal**) can be started using:\n",
    "    \n",
    "    $ ipython \n",
    "    Python 2.7.6 (default, Mar 22 2014, 22:59:56) \n",
    "    Type \"copyright\", \"credits\" or \"license\" for more information.\n",
    "    \n",
    "    IPython 2.1.0 -- An enhanced Interactive Python.\n",
    "    ?         -> Introduction and overview of IPython's features.\n",
    "    %quickref -> Quick reference.\n",
    "    help      -> Python's own help system.\n",
    "    object?   -> Details about 'object', use 'object??' for extra details.\n",
    "    \n",
    "    In [1]: \n",
    "    \n",
    "Other alternative is **qtconsole**, a console running on a graphical window:\n",
    "\n",
    "    $ ipython qtconsole"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "To run *notebooks*, use **jupyter**:\n",
    "  \n",
    "    $ jupyter notebook\n",
    "    [NotebookApp] Using existing profile dir: u'/home/thiago/.ipython/profile_default'\n",
    "    [NotebookApp] Using MathJax from CDN: http://cdn.mathjax.org/mathjax/latest/MathJax.js\n",
    "    [NotebookApp] Serving notebooks from local directory: /home/thiago\n",
    "    [NotebookApp] 0 active kernels \n",
    "    [NotebookApp] The IPython Notebook is running at: http://localhost:8888/\n",
    "    [NotebookApp] Use Control-C to stop this server and shut down all \n",
    "    kernels (twice to skip confirmation)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The URL to access the served notebooks is `http://localhost:8888`\n",
    "- System is serving the notebooks in the directory `/home/thiago`\n",
    "- User can employ `Ctrl-C` to stop the server and shutdown kernels\n",
    "- Jupyter supports IPython, R and Julia kernels. More languages coming!"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}