{
 "metadata": {
  "name": "",
  "signature": "sha256:1ec29059b8a8e46dac2cf75a06e6734323a300a8c62f7a80d16637a80e5d98ca"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Hands-on: Python Fundamentals -- Modules, Packages, and Library(ies)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "**Objectives:**\n",
      "\n",
      "Upon completion of this lesson, you should be able to\n",
      "\n",
      "1. State the purpose of modules and packages in Python\n",
      "\n",
      "2. Discuss the concepts of *namespace* and variable *scope*\n",
      "\n",
      "3. Import modules and libraries and use imported functions"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Modules"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "* A **Module** is a python file that contains a collection of related definitions.\n",
      "\n",
      "   So congratulations -- you already wrote a module or two without knowing it!\n",
      "\n",
      "\n",
      "* Python has *hundreds* of standard modules\n",
      "\n",
      "* These are organized into what is known as the Python Standard Library (\n",
      "http://docs.python.org/library/)\n",
      "\n",
      "* You can also create and use your own modules"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Purpose of modules"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To bring common functionality together under the common namespace.\n",
      "\n",
      "* It is very similar to how `functions` were used to encapsulate commonly reused code, or how `classes` encapsulate commonly reused functions and data structures.  \n",
      "\n",
      "* `modules` in turn encapsulate everything (functions, classes, etc) under one common \"roof\".\n",
      "\n",
      "* Unlike instances of `classes`, modules exist in a single copy: if I import the same module twice in the same program, it will remain the same exact module."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Using \"import ...\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "* To use functionality from a module, you first have to import the entire module or parts of it into your namespace\n",
      "\n",
      "\n",
      "* To import the entire module, use \n",
      "\n",
      "  ```python\n",
      "  import module_name```\n",
      "\n",
      "  statement, e.g."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import sys"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "you can also import multiple modules in a single line, although such practice is generally discouraged by PEP8:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os, sys"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "and name them differently if desired:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os as builtin_os, sys\n",
      "print builtin_os"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "and it will be exactly the same `os` module:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print os is builtin_os"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "* Upon such imports definitions available in those modules become available as \"attributes\" of those modules"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print os.system"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This, what looks a minor feature, provides a great opportunity to organize namespaces to avoid collisions among definitions (compare to Matlab)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- - -\n",
      "**Excercise #1**\n",
      "\n",
      "Remember functions lecture?  We have used ```global``` keyword to tell Python that variable is from the \"global\" namespace.  Let me reveal you two built-in functions:  **locals()** and **globals()**.\n",
      "\n",
      "*Question*: what structure type do they return?\n",
      "\n",
      "*Question*: how they are different in the current session? (recall useful data type `set`)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- - -\n",
      "**Excercise #2**\n",
      "\n",
      "Import few built-in modules (e.g. os, sys, glob), and inspect via completion or using command ```dir()``` which definitions they provide. \n",
      "\n",
      "Appreciate the fact that your \"locals\" or \"globals\" didn't become bloated."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Using \"from ... import ...\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "* To import specific definitions (e.g. functions, variables, etc) from the module into your local namespace, use\n",
      "\n",
      "   ```python\n",
      "   from module_name import name1, name2\n",
      "   ```\n",
      "   \n",
      "   which will make those available in your ```namespace``` directly:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from os import system, chmod\n",
      "print system"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "but you must be using such imports with care since then it might \"overload\" your previously defined or imported definitions in case of names collision:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# there is a builtin sum function, which you can access also unambigously from __builtin__ module\n",
      "print __builtin__.sum\n",
      "# but numpy also provides its own sum\n",
      "from numpy import sum\n",
      "print sum"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "you can overcome such a problem by importing with **as** again:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from os.path import join as pathjoin\n",
      "print pathjoin"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "while choosing the name carefully so you guaranteed to not overlap with some other name.\n",
      "As you can see from above example, those modules could be nested, so ```path``` is a \"sub-module\" of ```os``` module:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print os\n",
      "print os.path\n",
      "print os.path.join"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Avoid using \"import *\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Even more discouraged generally practice is to import **everything** from a module, e.g.\n",
      "\n",
      "```python\n",
      "from os import *\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It is discouraged because it\n",
      "\n",
      "* Restricts the variable names you can use: `os.name` might override `name`, or vise-versa.\n",
      "\n",
      "* Creates possible name clashes between modules.\n",
      "\n",
      "* Makes the code harder to read and understand: where do symbols come from?\n",
      "\n",
      "* Makes it impossible to guess the functionality by the context and the name (hint: `os.name` is the name of the OS), and to profit usefully from tab completion.\n",
      "\n",
      "* Makes the code impossible to statically check for undefined symbols."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So, even if your favorite PyMVPA author/tutorial tells you to just do\n",
      "\n",
      "```python\n",
      "from mvpa2.suite import *\n",
      "```\n",
      "\n",
      "better be cautious and do\n",
      "\n",
      "```python\n",
      "import mvpa2.suite as mv\n",
      "```\n",
      "\n",
      "to access its functionality via ```mv.``` prefix, e.g. ```mv.Dataset```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Whenever internally modules are organized hierarchically, and could be accessed correspondingly separating levels with \".\", above ```mvpa2.suite``` example shows that some modules provide an interface to access all (selected) the functionality without traversing hierarchy."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Packages"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[Formally](https://docs.python.org/2/tutorial/modules.html#packages) packages are collections of modules brought into the same namespace.  Usually it is done by creating a directory containing ```__init__.py``` and containing all the modules belonging to that package.  You can access ```__package__``` attribute of a module to see which package it originally belonged.\n",
      "\n",
      "And each package is actually presents as its own ```__init__.py`` module:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import email\n",
      "print email\n",
      "print email.__package__\n",
      "print email.parser\n",
      "print email.parser.__package__"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And packages can be nested within"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print email.mime\n",
      "print email.mime.__package__\n",
      "print email.mime.base.__package__"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "!tree /usr/lib/python2.7/email/"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- - -\n",
      "\n",
      "*Practically* you can consider simply consider every package to to be a module ;-)\n",
      "\n",
      "- - -"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Libraries"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "* Libraries are simply larger collections of modules/packages which do not share the common top namespace"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Standard library"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`Standard library` is a collection of modules which comes with stock Python.  It is quite reach in terms of functionality, and that is why Python is said to have \"batteries included\" philosophy."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- - -\n",
      "**Excercise**\n",
      "\n",
      "Visit and explore the [Brief Tour of the Standard Library](https://docs.python.org/2/tutorial/stdlib.html) describing available modules, some of which we have already used.\n",
      "\n",
      "Import and try to use functionality of the **re** and **glob** modules."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}