{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Py4Eng](img/logo.png)\n",
    "\n",
    "# Modules: import and install\n",
    "## Yoav Ram"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modules\n",
    "\n",
    "Luckily, we don't have to create all the code alone. Many people have written functions that perform various tasks.  \n",
    "These functions are grouped into packages, called _modules_, which are usually designed to address a specific set of tasks.  \n",
    "These functions are not always 'ready to use' by our code, but rather have to be _imported_ into it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python version\n",
    "\n",
    "We start with a simple example. Suppose we want to check the version of Python we are using. For this we would need to load the `sys` module using the `import` statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import sys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get a `version_info` object that has details on the specific source code used to compile Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)\n",
      "We are using Python 3.7\n"
     ]
    }
   ],
   "source": [
    "v = sys.version_info\n",
    "print(v)\n",
    "print(\"We are using Python \", v.major, \".\", v.minor, sep='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get more details on the specific _build_ we are using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 15:43:19) \n",
      "[Clang 4.0.1 (tags/RELEASE_401/final)]\n"
     ]
    }
   ],
   "source": [
    "print(sys.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`sys` also has other details on the system we are running on, look it up at the [docs](https://docs.python.org/3.5/library/sys.html). There is also a `platform` module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Platform Darwin\n",
      "Version Darwin Kernel Version 18.5.0: Mon Mar 11 20:40:32 PDT 2019; root:xnu-4903.251.3~3/RELEASE_X86_64\n",
      "Architecture ('64bit', '')\n",
      "Processor i386\n"
     ]
    }
   ],
   "source": [
    "import platform\n",
    "print('Platform', platform.system())\n",
    "print('Version', platform.version())\n",
    "print('Architecture', platform.architecture())\n",
    "print('Processor', platform.processor())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Math module\n",
    "\n",
    "Suppose we want to do some trigonometry, we can (in principal) write our own sine, cosine, etc. But it would be much easier to use the built-in _math_ module.\n",
    "\n",
    "First, we import the module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use values and functions within the math module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.283185307179586\n",
      "14.451326206513047\n",
      "0.49999999999999994\n",
      "0.5000000000000001\n"
     ]
    }
   ],
   "source": [
    "two_pi = 2 * math.pi\n",
    "print(two_pi)\n",
    "radius = 2.3\n",
    "perimeter = two_pi * radius\n",
    "print(perimeter)\n",
    "print(math.sin(math.pi / 6))\n",
    "print(math.cos(math.pi / 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see what's inside the module, we can call the `dir` and `help` functions on it, or just [search the web](https://duckduckgo.com/?q=python+math+module)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'pi', 'pow', 'radians', 'remainder', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']\n"
     ]
    }
   ],
   "source": [
    "print(dir(math))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on module math:\n",
      "\n",
      "NAME\n",
      "    math\n",
      "\n",
      "MODULE REFERENCE\n",
      "    https://docs.python.org/3.7/library/math\n",
      "    \n",
      "    The following documentation is automatically generated from the Python\n",
      "    source files.  It may be incomplete, incorrect or include features that\n",
      "    are considered implementation detail and may vary between Python\n",
      "    implementations.  When in doubt, consult the module reference at the\n",
      "    location listed above.\n",
      "\n",
      "DESCRIPTION\n",
      "    This module is always available.  It provides access to the\n",
      "    mathematical functions defined by the C standard.\n",
      "\n",
      "FUNCTIONS\n",
      "    acos(x, /)\n",
      "        Return the arc cosine (measured in radians) of x.\n",
      "    \n",
      "    acosh(x, /)\n",
      "        Return the inverse hyperbolic cosine of x.\n",
      "    \n",
      "    asin(x, /)\n",
      "        Return the arc sine (measured in radians) of x.\n",
      "    \n",
      "    asinh(x, /)\n",
      "        Return the inverse hyperbolic sine of x.\n",
      "    \n",
      "    atan(x, /)\n",
      "        Return the arc tangent (measured in radians) of x.\n",
      "    \n",
      "    atan2(y, x, /)\n",
      "        Return the arc tangent (measured in radians) of y/x.\n",
      "        \n",
      "        Unlike atan(y/x), the signs of both x and y are considered.\n",
      "    \n",
      "    atanh(x, /)\n",
      "        Return the inverse hyperbolic tangent of x.\n",
      "    \n",
      "    ceil(x, /)\n",
      "        Return the ceiling of x as an Integral.\n",
      "        \n",
      "        This is the smallest integer >= x.\n",
      "    \n",
      "    copysign(x, y, /)\n",
      "        Return a float with the magnitude (absolute value) of x but the sign of y.\n",
      "        \n",
      "        On platforms that support signed zeros, copysign(1.0, -0.0)\n",
      "        returns -1.0.\n",
      "    \n",
      "    cos(x, /)\n",
      "        Return the cosine of x (measured in radians).\n",
      "    \n",
      "    cosh(x, /)\n",
      "        Return the hyperbolic cosine of x.\n",
      "    \n",
      "    degrees(x, /)\n",
      "        Convert angle x from radians to degrees.\n",
      "    \n",
      "    erf(x, /)\n",
      "        Error function at x.\n",
      "    \n",
      "    erfc(x, /)\n",
      "        Complementary error function at x.\n",
      "    \n",
      "    exp(x, /)\n",
      "        Return e raised to the power of x.\n",
      "    \n",
      "    expm1(x, /)\n",
      "        Return exp(x)-1.\n",
      "        \n",
      "        This function avoids the loss of precision involved in the direct evaluation of exp(x)-1 for small x.\n",
      "    \n",
      "    fabs(x, /)\n",
      "        Return the absolute value of the float x.\n",
      "    \n",
      "    factorial(x, /)\n",
      "        Find x!.\n",
      "        \n",
      "        Raise a ValueError if x is negative or non-integral.\n",
      "    \n",
      "    floor(x, /)\n",
      "        Return the floor of x as an Integral.\n",
      "        \n",
      "        This is the largest integer <= x.\n",
      "    \n",
      "    fmod(x, y, /)\n",
      "        Return fmod(x, y), according to platform C.\n",
      "        \n",
      "        x % y may differ.\n",
      "    \n",
      "    frexp(x, /)\n",
      "        Return the mantissa and exponent of x, as pair (m, e).\n",
      "        \n",
      "        m is a float and e is an int, such that x = m * 2.**e.\n",
      "        If x is 0, m and e are both 0.  Else 0.5 <= abs(m) < 1.0.\n",
      "    \n",
      "    fsum(seq, /)\n",
      "        Return an accurate floating point sum of values in the iterable seq.\n",
      "        \n",
      "        Assumes IEEE-754 floating point arithmetic.\n",
      "    \n",
      "    gamma(x, /)\n",
      "        Gamma function at x.\n",
      "    \n",
      "    gcd(x, y, /)\n",
      "        greatest common divisor of x and y\n",
      "    \n",
      "    hypot(x, y, /)\n",
      "        Return the Euclidean distance, sqrt(x*x + y*y).\n",
      "    \n",
      "    isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0)\n",
      "        Determine whether two floating point numbers are close in value.\n",
      "        \n",
      "          rel_tol\n",
      "            maximum difference for being considered \"close\", relative to the\n",
      "            magnitude of the input values\n",
      "          abs_tol\n",
      "            maximum difference for being considered \"close\", regardless of the\n",
      "            magnitude of the input values\n",
      "        \n",
      "        Return True if a is close in value to b, and False otherwise.\n",
      "        \n",
      "        For the values to be considered close, the difference between them\n",
      "        must be smaller than at least one of the tolerances.\n",
      "        \n",
      "        -inf, inf and NaN behave similarly to the IEEE 754 Standard.  That\n",
      "        is, NaN is not close to anything, even itself.  inf and -inf are\n",
      "        only close to themselves.\n",
      "    \n",
      "    isfinite(x, /)\n",
      "        Return True if x is neither an infinity nor a NaN, and False otherwise.\n",
      "    \n",
      "    isinf(x, /)\n",
      "        Return True if x is a positive or negative infinity, and False otherwise.\n",
      "    \n",
      "    isnan(x, /)\n",
      "        Return True if x is a NaN (not a number), and False otherwise.\n",
      "    \n",
      "    ldexp(x, i, /)\n",
      "        Return x * (2**i).\n",
      "        \n",
      "        This is essentially the inverse of frexp().\n",
      "    \n",
      "    lgamma(x, /)\n",
      "        Natural logarithm of absolute value of Gamma function at x.\n",
      "    \n",
      "    log(...)\n",
      "        log(x, [base=math.e])\n",
      "        Return the logarithm of x to the given base.\n",
      "        \n",
      "        If the base not specified, returns the natural logarithm (base e) of x.\n",
      "    \n",
      "    log10(x, /)\n",
      "        Return the base 10 logarithm of x.\n",
      "    \n",
      "    log1p(x, /)\n",
      "        Return the natural logarithm of 1+x (base e).\n",
      "        \n",
      "        The result is computed in a way which is accurate for x near zero.\n",
      "    \n",
      "    log2(x, /)\n",
      "        Return the base 2 logarithm of x.\n",
      "    \n",
      "    modf(x, /)\n",
      "        Return the fractional and integer parts of x.\n",
      "        \n",
      "        Both results carry the sign of x and are floats.\n",
      "    \n",
      "    pow(x, y, /)\n",
      "        Return x**y (x to the power of y).\n",
      "    \n",
      "    radians(x, /)\n",
      "        Convert angle x from degrees to radians.\n",
      "    \n",
      "    remainder(x, y, /)\n",
      "        Difference between x and the closest integer multiple of y.\n",
      "        \n",
      "        Return x - n*y where n*y is the closest integer multiple of y.\n",
      "        In the case where x is exactly halfway between two multiples of\n",
      "        y, the nearest even value of n is used. The result is always exact.\n",
      "    \n",
      "    sin(x, /)\n",
      "        Return the sine of x (measured in radians).\n",
      "    \n",
      "    sinh(x, /)\n",
      "        Return the hyperbolic sine of x.\n",
      "    \n",
      "    sqrt(x, /)\n",
      "        Return the square root of x.\n",
      "    \n",
      "    tan(x, /)\n",
      "        Return the tangent of x (measured in radians).\n",
      "    \n",
      "    tanh(x, /)\n",
      "        Return the hyperbolic tangent of x.\n",
      "    \n",
      "    trunc(x, /)\n",
      "        Truncates the Real x to the nearest Integral toward 0.\n",
      "        \n",
      "        Uses the __trunc__ magic method.\n",
      "\n",
      "DATA\n",
      "    e = 2.718281828459045\n",
      "    inf = inf\n",
      "    nan = nan\n",
      "    pi = 3.141592653589793\n",
      "    tau = 6.283185307179586\n",
      "\n",
      "FILE\n",
      "    /Users/yoavram/miniconda3/envs/Py4Eng/lib/python3.7/lib-dynload/math.cpython-37m-darwin.so\n",
      "\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(help(math))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you only need one or two functions from a module, you can import them, instead of the whole module. This way, we don't have to call the module name every time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from math import pi, sin, cos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.283185307179586\n",
      "0.49999999999999994\n",
      "0.5000000000000001\n"
     ]
    }
   ],
   "source": [
    "# now we can use values and functions within the math module\n",
    "two_pi = 2 * pi\n",
    "print(two_pi)\n",
    "print(sin(pi / 6))\n",
    "print(cos(pi / 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can import using an alias with the `as` command. This is especially useful if you have two module with the same API and you want to change between them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also view the module documentation to see what functions are available and how to use them. Each python module has a documentation page, for example: https://docs.python.org/3/library/math.html\n",
    "\n",
    "Two more useful links:\n",
    "- [PyPI](https://pypi.python.org/pypi): a repository of (almost) all Python modules  \n",
    "- [List of scientific python modules](https://wiki.python.org/moin/NumericAndScientific)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing packages\n",
    "\n",
    "Many Python modules are included within the core distribution, and all you have to do is `import` them. \n",
    "However, many more modules are contained in 3rd party packages that must be downloaded and installed first.  \n",
    "\n",
    "Python pacakges can be installed by downloading the package source code and then running a special script that comes with the pacakge, `python setup.py install`. But this is not the recommended way to install packages.\n",
    "\n",
    "To manage installations and dependencies, most people use a pacakge manager. The most common one is [pip](https://pip.pypa.io/) (which replaced `easy_install` some years ago), which is the package manager recommended by the Python organization. _pip_ installs packages that were uploaded (e.g. _deployed_) to [PyPI](http://pypi.python.org/), the Python Package Index, which is the main repository for Python packages.\n",
    "\n",
    "However, scientific pacakges (such as Numpy and Scipy) sometimes have non-Python dependencies, mostly C and Fortran code, and these dependecies can be hard to compile and link - especially on standard Windows machines. Therefore, [Continuum](http://continuum.io/) started the [conda](http://conda.pydata.org/) project. *conda* is an alternative to *pip*, able to install Python packages, but it can also do more, as it can install non-Python software (even *R*). So *conda* can install Python packages with non-Python dependencies much more smoothly.\n",
    "\n",
    "If we installed Python using the Anaconda distribution (also by *Continuum*) then we already have *pip* and *conda*.\n",
    "Let's start by updating them (remember that prefixing commands with `!` in the notebook runs them as system commands, so we don't have to open a console window):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching package metadata ...........\n",
      "Solving package specifications: ..........\n",
      "\n",
      "# All requested packages already installed.\n",
      "# packages in environment at /Users/yoavram/miniconda3:\n",
      "#\n",
      "conda                     4.2.13                   py35_0    conda-forge\n",
      "pip                       9.0.1                    py35_0    conda-forge\n"
     ]
    }
   ],
   "source": [
    "!conda update conda pip -y -q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install with *conda*\n",
    "\n",
    "We will prefer to use *conda* as it has more *power*.\n",
    "Let's install the *seaborn* package - we'll use it at a later occasion.\n",
    "The `-q` option tells conda to supress progress bars, `-y` tells it to go ahead without prompting us for approval.\n",
    "If you are running *conda* from a console, you can skip these options and see the extra output, but when running from inside a notebook it won't work without `-y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching package metadata ...........\n",
      "Solving package specifications: ..........\n",
      "\n",
      "Package plan for installation in environment /Users/yoavram/miniconda3:\n",
      "\n",
      "The following NEW packages will be INSTALLED:\n",
      "\n",
      "    cycler:          0.10.0-py35_0                      conda-forge\n",
      "    matplotlib:      2.0.0-np112py35_2                  conda-forge\n",
      "    numpy:           1.12.0-py35_blas_openblas_200      conda-forge [blas_openblas]\n",
      "    pandas:          0.19.2-np112py35_1                 conda-forge\n",
      "    patsy:           0.4.1-py35_0                       conda-forge\n",
      "    pyparsing:       2.1.10-py35_0                      conda-forge\n",
      "    python-dateutil: 2.6.0-py35_0                       conda-forge\n",
      "    pytz:            2016.10-py35_0                     conda-forge\n",
      "    scipy:           0.18.1-np112py35_blas_openblas_201 conda-forge [blas_openblas]\n",
      "    seaborn:         0.7.1-py35_0                       conda-forge\n",
      "    statsmodels:     0.8.0-np112py35_0                  conda-forge\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!conda install seaborn -y -q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*conda* installs packages from the [Anaconda.org]() repository, which is different from *PyPI* and has less packages. So some pacakges *conda* just wouldn't find and then we will try *pip*. \n",
    "\n",
    "### Installing with *pip*\n",
    "Let's install another package we will need later on, *watchdog*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: watchdog in /Users/yoavram/miniconda3/lib/python3.5/site-packages\n",
      "Requirement already satisfied: argh>=0.24.1 in /Users/yoavram/miniconda3/lib/python3.5/site-packages (from watchdog)\n",
      "Requirement already satisfied: pathtools>=0.1.1 in /Users/yoavram/miniconda3/lib/python3.5/site-packages (from watchdog)\n",
      "Requirement already satisfied: PyYAML>=3.10 in /Users/yoavram/miniconda3/lib/python3.5/site-packages (from watchdog)\n"
     ]
    }
   ],
   "source": [
    "!pip install watchdog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download pre-compiled installers\n",
    "\n",
    "If nothing else works, you can try looking for your package in [Gohlke's website](http://www.lfd.uci.edu/~gohlke/pythonlibs/). It contains many downloadable installers(`exe` files you just click through to easily install a package) and `wheel` files which you install by calling `pip install filename.wheel`. Make sure to choose the download that fits your python version and operating system.  \n",
    "\n",
    "### Installing from source code\n",
    "\n",
    "Not all modules are available through this website. If you don't find your module here, you might have to install from source. Details [here](https://packaging.python.org/installing/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "- [Detailed guide on installing Python modules with pip](https://packaging.python.org/installing/)\n",
    "- [Packaging and Deployment with Conda](https://speakerdeck.com/teoliphant/packaging-and-deployment-with-conda) by Travis E. Oliphant"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Colophon\n",
    "This notebook was written by [Yoav Ram](http://python.yoavram.com) and is part of the [_Python for Engineers_](https://github.com/yoavram/Py4Eng) course.\n",
    "\n",
    "The notebook was written using [Python](http://python.org/) 3.7.\n",
    "Dependencies listed in [environment.yml](../environment.yml), full versions in [environment_full.yml](../environment_full.yml).\n",
    "\n",
    "This work is licensed under a CC BY-NC-SA 4.0 International License.\n",
    "\n",
    "![Python logo](https://www.python.org/static/community_logos/python-logo.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:Py4Eng]",
   "language": "python",
   "name": "conda-env-Py4Eng-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}