{ "metadata": { "name": "" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook was put together by [Jake Vanderplas](http://www.vanderplas.com) for UW's [Astro 599/AMath 500](http://www.astro.washington.edu/users/vanderplas/Astr599_2014/) course. Source and license info is on [GitHub](https://github.com/jakevdp/2013_fall_ASTR599_2014/)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Extending Python with Compiled Code\n", "\n", "In a previous session, we talked about ways to get performance out of Python using vectorization strategies within NumPy. Today we're going to talk about approaches which involve **interfacing Python to external compiled code**, or **converting Python code to compilable code**. There are several different approaches. We'll give examples of solving a problem by several means:\n", "\n", "- [Ctypes](http://docs.python.org/2/library/ctypes.html): the C-linking utilities included in the Python standard library\n", "- [F2Py](http://cens.ioc.ee/projects/f2py2e/usersguide/): the Fortran to Python utility which is bundled with NumPy\n", "- [Cython](http://cython.org/): a module that allows conversion of Python and Python-like code to C\n", "- [Numba](http://numba.pydata.org/): a Python to LLVM bytecode converter\n", "\n", "There is no way to go into depth on all of these, but I mainly just want to show some examples so that you know the types of problems they solve. At the above links, you'll find more in-depth documentation and tutorials." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are other options we won't talk about here, because they're not as useful for our purposes:\n", "\n", "- [Python C API](http://docs.python.org/2/c-api/): Python is implemented in C, and you can actually write C code which will compile to a Python module! This is extremely low-level, and I would only recommend it if you enjoy pain (see Dan Foreman-Mackie's [blog post](http://dan.iel.fm/posts/python-c-extensions/) for a concise-as-possible introduction)\n", "- [SWIG](http://www.swig.org/): the Simplified Wrapper Interface Generator, can generate wrappers from a variety of low-level languages to a variety of high-level languages. It's used heavily by LSST and other projects.\n", "- [Weave](http://docs.scipy.org/doc/scipy/reference/tutorial/weave.html): included in SciPy, weave is a method of putting C snippets within a Python program. It's largely been superseded by Cython in practice.\n", "- [PyPy](http://pypy.org/): a JIT compiler for Python written in Python. PyPy doesn't support the C backend which most scientific tools rely on, so it's not extremely useful for scientific computing.\n", "\n", "The optimal approach depends on the desired use-case. We'll go through a few examples, from lowest to highest level:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Why Extend Python?\n", "\n", "There are a few reasons you might wish to use the tools outlined below:\n", "\n", "1. Sometimes vectorization of algorithms can lead to excessive memory usage, because it relies on temporary arrays to hold intermediate results. Directly implementing the routine in compiled code can lead to more efficient computation.\n", "\n", "2. Sometimes problems cannot be implemented as vectorized NumPy operations. An example is in tree-based algorithms. This is why I used cython when I wrote the k-neighbors and kernel density estimation code within ``sklearn.neighbors``.\n", "\n", "3. Sometimes there are legacy code-bases that you don't want to have to re-implement: you'd much rather just write some sort of bindings to call the old code from within Python.\n", "\n", "We'll do a **brief demonstration** of some of these tools. These tools do have a bit of a learning curve, so I'll provide references for where you might look to go deeper. The purpose of this lecture is to get a taste of some of the better options for addressing these problems." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Ctypes\n", "\n", "Ctypes is part of the standard library, and offers tools to **call routines from compiled libraries with Python**. These can either be routines in your system libraries, or in shared libraries you compile yourself. We'll see examples of both.\n", "\n", "One weakness of ctypes is that it can be **very platform-specific**. For example, shared-library extensions are different from platform to platform (e.g. ``*.so`` on Linux, ``*.dll`` on Windows, ``*.dylib`` on OSX). Other platform-by-platform issues like 32/64 bit can also make things difficult." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Interfacing to System Libraries" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In Linux & Mac, you'll usually find the standard libraries here (remember that ``!`` lets us execute shell commands within IPython):" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!ls /usr/lib/" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[34marc\u001b[m\u001b[m \u001b[31mlibmecabra.dylib\u001b[m\u001b[m\r\n", "\u001b[31march_tool\u001b[m\u001b[m \u001b[31mlibmenu.5.4.dylib\u001b[m\u001b[m\r\n", "bundle1.o \u001b[35mlibmenu.dylib\u001b[m\u001b[m\r\n", "\u001b[34mc++\u001b[m\u001b[m \u001b[35mlibmx.A.dylib\u001b[m\u001b[m\r\n", "charset.alias \u001b[35mlibmx.dylib\u001b[m\u001b[m\r\n", "\u001b[34mclang\u001b[m\u001b[m \u001b[31mlibncurses.5.4.dylib\u001b[m\u001b[m\r\n", "\u001b[35mcron\u001b[m\u001b[m \u001b[35mlibncurses.5.dylib\u001b[m\u001b[m\r\n", "crt1.10.5.o \u001b[35mlibncurses.dylib\u001b[m\u001b[m\r\n", "crt1.10.6.o \u001b[31mlibneon.27.dylib\u001b[m\u001b[m\r\n", "crt1.o \u001b[35mlibneon.dylib\u001b[m\u001b[m\r\n", "\u001b[34mdtrace\u001b[m\u001b[m \u001b[31mlibnetsnmp.15.1.2.dylib\u001b[m\u001b[m\r\n", "\u001b[31mdyld\u001b[m\u001b[m \u001b[35mlibnetsnmp.15.dylib\u001b[m\u001b[m\r\n", "dylib1.10.5.o \u001b[31mlibnetsnmp.25.dylib\u001b[m\u001b[m\r\n", "dylib1.o \u001b[31mlibnetsnmp.5.2.1.dylib\u001b[m\u001b[m\r\n", "\u001b[34mfreeradius\u001b[m\u001b[m \u001b[35mlibnetsnmp.5.dylib\u001b[m\u001b[m\r\n", "gcrt1.o \u001b[35mlibnetsnmp.dylib\u001b[m\u001b[m\r\n", "\u001b[34mgroff\u001b[m\u001b[m \u001b[31mlibnetsnmpagent.25.dylib\u001b[m\u001b[m\r\n", "lazydylib1.o \u001b[35mlibnetsnmpagent.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibACSClient.dylib\u001b[m\u001b[m \u001b[31mlibnetsnmphelpers.25.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibBSDPClient.A.dylib\u001b[m\u001b[m \u001b[35mlibnetsnmphelpers.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibBSDPClient.dylib\u001b[m\u001b[m \u001b[31mlibnetsnmpmibs.25.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibCRFSuite.dylib\u001b[m\u001b[m \u001b[35mlibnetsnmpmibs.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibCRFSuite0.12.dylib\u001b[m\u001b[m \u001b[31mlibnetsnmptrapd.25.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibCoreStorage.dylib\u001b[m\u001b[m \u001b[35mlibnetsnmptrapd.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibDHCPServer.A.dylib\u001b[m\u001b[m \u001b[31mlibobjc.A.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibDHCPServer.dylib\u001b[m\u001b[m \u001b[35mlibobjc.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibDiagnosticMessagesClient.dylib\u001b[m\u001b[m \u001b[35mlibodbc.a\u001b[m\u001b[m\r\n", "\u001b[35mlibIOKit.A.dylib\u001b[m\u001b[m \u001b[31mlibodfde.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibIOKit.dylib\u001b[m\u001b[m \u001b[31mlibpam.1.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibLTO.dylib\u001b[m\u001b[m \u001b[31mlibpam.2.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibMatch.1.dylib\u001b[m\u001b[m \u001b[35mlibpam.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibMatch.dylib\u001b[m\u001b[m \u001b[31mlibpanel.5.4.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibOpenScriptingUtil.dylib\u001b[m\u001b[m \u001b[35mlibpanel.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibScreenReader.dylib\u001b[m\u001b[m \u001b[31mlibpcap.A.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibSystem.B.dylib\u001b[m\u001b[m \u001b[35mlibpcap.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibSystem.B_debug.dylib\u001b[m\u001b[m \u001b[31mlibpcre.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibSystem.dylib\u001b[m\u001b[m \u001b[35mlibpcre.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibSystem_debug.dylib\u001b[m\u001b[m \u001b[31mlibpcreposix.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibUniversalAccess.dylib\u001b[m\u001b[m \u001b[35mlibpcreposix.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibXplugin.1.dylib\u001b[m\u001b[m \u001b[31mlibpgtypes.3.2.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibXplugin.dylib\u001b[m\u001b[m \u001b[35mlibpgtypes.3.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibalias.A.dylib\u001b[m\u001b[m \u001b[35mlibpgtypes.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibalias.dylib\u001b[m\u001b[m \u001b[35mlibpoll.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibapr-1.0.4.5.dylib\u001b[m\u001b[m \u001b[31mlibpq.5.4.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibapr-1.0.dylib\u001b[m\u001b[m \u001b[35mlibpq.5.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibapr-1.dylib\u001b[m\u001b[m \u001b[35mlibpq.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibaprutil-1.0.3.12.dylib\u001b[m\u001b[m \u001b[35mlibproc.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibaprutil-1.0.dylib\u001b[m\u001b[m libprofile_rt.a\r\n", "\u001b[35mlibaprutil-1.dylib\u001b[m\u001b[m \u001b[31mlibprofile_rt.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibarchive.2.dylib\u001b[m\u001b[m \u001b[35mlibpthread.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibarchive.dylib\u001b[m\u001b[m \u001b[35mlibpython.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibatlas.dylib\u001b[m\u001b[m \u001b[35mlibpython2.5.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibauditd.0.dylib\u001b[m\u001b[m \u001b[35mlibpython2.6.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibauditd.dylib\u001b[m\u001b[m \u001b[35mlibpython2.7.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibauto.dylib\u001b[m\u001b[m \u001b[31mlibquit.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibblas.dylib\u001b[m\u001b[m \u001b[35mlibreadline.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibbsm.0.dylib\u001b[m\u001b[m \u001b[31mlibresolv.9.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibbsm.dylib\u001b[m\u001b[m \u001b[35mlibresolv.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibbz2.1.0.5.dylib\u001b[m\u001b[m \u001b[35mlibrpcsvc.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibbz2.1.0.dylib\u001b[m\u001b[m \u001b[35mlibruby.1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibbz2.dylib\u001b[m\u001b[m \u001b[35mlibruby.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibc++.1.dylib\u001b[m\u001b[m \u001b[31mlibsandbox.1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibc++.dylib\u001b[m\u001b[m \u001b[35mlibsandbox.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibc++abi.dylib\u001b[m\u001b[m \u001b[35mlibsasl2.2.0.1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibc.dylib\u001b[m\u001b[m \u001b[35mlibsasl2.2.0.15.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcblas.dylib\u001b[m\u001b[m \u001b[35mlibsasl2.2.0.21.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcharset.1.0.0.dylib\u001b[m\u001b[m \u001b[35mlibsasl2.2.0.22.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcharset.1.dylib\u001b[m\u001b[m \u001b[31mlibsasl2.2.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcharset.dylib\u001b[m\u001b[m \u001b[35mlibsasl2.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibclang.dylib\u001b[m\u001b[m \u001b[35mlibsqlite3.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibclapack.dylib\u001b[m\u001b[m \u001b[31mlibsqlite3.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcom_err.dylib\u001b[m\u001b[m \u001b[31mlibssl.0.9.7.dylib\u001b[m\u001b[m\r\n", "libcpp_kext.a \u001b[31mlibssl.0.9.8.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcrypto.0.9.7.dylib\u001b[m\u001b[m \u001b[35mlibssl.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcrypto.0.9.8.dylib\u001b[m\u001b[m \u001b[31mlibstdc++.6.0.9.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcrypto.dylib\u001b[m\u001b[m \u001b[35mlibstdc++.6.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcsfde.dylib\u001b[m\u001b[m \u001b[35mlibstdc++.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcups.2.dylib\u001b[m\u001b[m \u001b[31mlibsvn_client-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcups.dylib\u001b[m\u001b[m \u001b[35mlibsvn_client-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcupscgi.1.dylib\u001b[m\u001b[m \u001b[35mlibsvn_client-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcupscgi.dylib\u001b[m\u001b[m \u001b[31mlibsvn_delta-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcupsimage.2.dylib\u001b[m\u001b[m \u001b[35mlibsvn_delta-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcupsimage.dylib\u001b[m\u001b[m \u001b[35mlibsvn_delta-1.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcupsmime.1.dylib\u001b[m\u001b[m \u001b[31mlibsvn_diff-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcupsmime.dylib\u001b[m\u001b[m \u001b[35mlibsvn_diff-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcupsppdc.1.dylib\u001b[m\u001b[m \u001b[35mlibsvn_diff-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcupsppdc.dylib\u001b[m\u001b[m \u001b[31mlibsvn_fs-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcurl.3.dylib\u001b[m\u001b[m \u001b[35mlibsvn_fs-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibcurl.4.dylib\u001b[m\u001b[m \u001b[35mlibsvn_fs-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcurl.dylib\u001b[m\u001b[m \u001b[31mlibsvn_fs_fs-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibcurses.dylib\u001b[m\u001b[m \u001b[35mlibsvn_fs_fs-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibdbm.dylib\u001b[m\u001b[m \u001b[35mlibsvn_fs_fs-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibdes425.dylib\u001b[m\u001b[m \u001b[31mlibsvn_fs_util-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibdl.dylib\u001b[m\u001b[m \u001b[35mlibsvn_fs_util-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibdtrace.dylib\u001b[m\u001b[m \u001b[35mlibsvn_fs_util-1.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibecpg.6.3.dylib\u001b[m\u001b[m \u001b[31mlibsvn_ra-1.0.0.0.dylib\u001b[m\u001b[m\r", "\r\n", "\u001b[35mlibecpg.6.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibecpg.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra-1.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibecpg_compat.3.3.dylib\u001b[m\u001b[m \u001b[31mlibsvn_ra_local-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibecpg_compat.3.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra_local-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibecpg_compat.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra_local-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibedit.2.dylib\u001b[m\u001b[m \u001b[31mlibsvn_ra_neon-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibedit.3.0.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra_neon-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibedit.3.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra_neon-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibedit.dylib\u001b[m\u001b[m \u001b[31mlibsvn_ra_svn-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibexpat.1.5.2.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra_svn-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibexpat.1.dylib\u001b[m\u001b[m \u001b[35mlibsvn_ra_svn-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibexpat.dylib\u001b[m\u001b[m \u001b[31mlibsvn_repos-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibexslt.0.dylib\u001b[m\u001b[m \u001b[35mlibsvn_repos-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibexslt.dylib\u001b[m\u001b[m \u001b[35mlibsvn_repos-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibf77lapack.dylib\u001b[m\u001b[m \u001b[31mlibsvn_subr-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibffi.dylib\u001b[m\u001b[m \u001b[35mlibsvn_subr-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibform.5.4.dylib\u001b[m\u001b[m \u001b[35mlibsvn_subr-1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibform.dylib\u001b[m\u001b[m \u001b[31mlibsvn_wc-1.0.0.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibgcc_s.1.dylib\u001b[m\u001b[m \u001b[35mlibsvn_wc-1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibgcc_s.10.4.dylib\u001b[m\u001b[m \u001b[35mlibsvn_wc-1.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibgcc_s.10.5.dylib\u001b[m\u001b[m \u001b[31mlibsysmon.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibgermantok.dylib\u001b[m\u001b[m \u001b[35mlibtcl.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibgmalloc.B.dylib\u001b[m\u001b[m \u001b[35mlibtcl8.5.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibgmalloc.dylib\u001b[m\u001b[m \u001b[35mlibtclstub8.5.a\u001b[m\u001b[m\r\n", "\u001b[35mlibgssapi_krb5.dylib\u001b[m\u001b[m \u001b[35mlibtermcap.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibhunspell-1.2.0.0.0.dylib\u001b[m\u001b[m \u001b[31mlibtidy.A.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibhunspell-1.2.0.dylib\u001b[m\u001b[m \u001b[35mlibtidy.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibhunspell-1.2.dylib\u001b[m\u001b[m \u001b[35mlibtk.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibiconv.2.4.0.dylib\u001b[m\u001b[m \u001b[35mlibtk8.5.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibiconv.2.dylib\u001b[m\u001b[m \u001b[35mlibtkstub8.5.a\u001b[m\u001b[m\r\n", "\u001b[35mlibiconv.dylib\u001b[m\u001b[m \u001b[35mlibutil.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibicucore.A.dylib\u001b[m\u001b[m \u001b[31mlibutil1.0.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibicucore.dylib\u001b[m\u001b[m \u001b[31mlibxar.1.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibinfo.dylib\u001b[m\u001b[m \u001b[35mlibxar.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibiodbc.2.1.18.dylib\u001b[m\u001b[m \u001b[31mlibxml2.2.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibiodbc.2.dylib\u001b[m\u001b[m \u001b[35mlibxml2.dylib\u001b[m\u001b[m\r\n", "libiodbc.a \u001b[31mlibxsanmgrcommon.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibiodbc.dylib\u001b[m\u001b[m \u001b[31mlibxslt.1.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibiodbcinst.2.1.18.dylib\u001b[m\u001b[m \u001b[35mlibxslt.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibiodbcinst.2.dylib\u001b[m\u001b[m liby.a\r\n", "libiodbcinst.a \u001b[35mlibz.1.1.3.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibiodbcinst.dylib\u001b[m\u001b[m \u001b[31mlibz.1.2.5.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibipconfig.dylib\u001b[m\u001b[m \u001b[35mlibz.1.dylib\u001b[m\u001b[m\r\n", "\u001b[31mlibipsec.A.dylib\u001b[m\u001b[m \u001b[35mlibz.dylib\u001b[m\u001b[m\r\n", "\u001b[35mlibipsec.dylib\u001b[m\u001b[m \u001b[34mpam\u001b[m\u001b[m\r\n", "\u001b[35mlibk5crypto.dylib\u001b[m\u001b[m \u001b[34mphp\u001b[m\u001b[m\r\n", "libkmod.a \u001b[34mpkgconfig\u001b[m\u001b[m\r\n", "libkmodc++.a \u001b[34mpostgresql\u001b[m\u001b[m\r\n", "\u001b[35mlibkrb4.dylib\u001b[m\u001b[m \u001b[35mpython2.5\u001b[m\u001b[m\r\n", "\u001b[35mlibkrb5.dylib\u001b[m\u001b[m \u001b[35mpython2.6\u001b[m\u001b[m\r\n", "\u001b[35mlibkrb524.dylib\u001b[m\u001b[m \u001b[35mpython2.7\u001b[m\u001b[m\r\n", "\u001b[35mlibkrb5support.dylib\u001b[m\u001b[m \u001b[34mrpcsvc\u001b[m\u001b[m\r\n", "libl.a \u001b[35mruby\u001b[m\u001b[m\r\n", "\u001b[31mliblangid.dylib\u001b[m\u001b[m \u001b[34msa\u001b[m\u001b[m\r\n", "\u001b[35mliblapack.dylib\u001b[m\u001b[m \u001b[34msasl2\u001b[m\u001b[m\r\n", "\u001b[35mliblber.dylib\u001b[m\u001b[m \u001b[35msqlite3\u001b[m\u001b[m\r\n", "\u001b[35mlibldap.dylib\u001b[m\u001b[m \u001b[34msystem\u001b[m\u001b[m\r\n", "\u001b[35mlibldap_r.dylib\u001b[m\u001b[m \u001b[35mtclConfig.sh\u001b[m\u001b[m\r\n", "\u001b[35mlibm.dylib\u001b[m\u001b[m \u001b[35mtkConfig.sh\u001b[m\u001b[m\r\n", "\u001b[31mlibmecab.1.0.0.dylib\u001b[m\u001b[m \u001b[34mzsh\u001b[m\u001b[m\r\n", "\u001b[35mlibmecab.dylib\u001b[m\u001b[m\r\n" ] } ], "prompt_number": 1 }, { "cell_type": "markdown", "metadata": {}, "source": [ "CTypes lets you link to any of these libraries and call the functions directly:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from ctypes import CDLL\n", "\n", "libc_name = 'libc.dylib' # OSX\n", "# libc_name = 'libc.so.6' # Linux\n", "# libc_name = 'libc.dll' # Windows\n", "\n", "libc = CDLL(libc_name)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 2 }, { "cell_type": "code", "collapsed": false, "input": [ "libc.time" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 3, "text": [ "<_FuncPtr object at 0x1021a1ae0>" ] } ], "prompt_number": 3 }, { "cell_type": "code", "collapsed": false, "input": [ "print \"Seconds since January 1, 1970:\"\n", "print libc.time()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Seconds since January 1, 1970:\n", "1384797127\n" ] } ], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Wrapping your own functions\n", "You can do more than simply wrapping the functionality of system libraries: you can also create your own C functions and wrap them with CTypes (this will require having a C compiler installed):" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%%file my_sum.c\n", "#include \n", "\n", "// sum all the values in the array x\n", "// x is a pointer to a memory block \n", "// of length n\n", "int sum(int *x, int n)\n", "{\n", " int i, counter;\n", " counter = 0;\n", " for(i=0; i\n", "\n", "// Define a simple array struct, containing\n", "// a pointer and a length\n", "struct Array{\n", " int *x;\n", " int n;\n", "};\n", "\n", "// Sum the values in the array struct\n", "int sum(struct Array a)\n", "{\n", " int counter, i;\n", " counter = 0;\n", " for(i=0; i /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fibmodule.c\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "Reading fortran codes...\r\n", "\tReading file 'fib.f' (format:fix,strict)\r\n", "Post-processing...\r\n", "\tBlock: fib\r\n", "\t\t\tBlock: fib\r\n", "Post-processing (stage 2)...\r\n", "Building modules...\r\n", "\tBuilding module \"fib\"...\r\n", "\t\tConstructing wrapper function \"fib\"...\r\n", "\t\t fib(a,[n])\r\n", "\tWrote C/API module \"fib\" to file \"/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fibmodule.c\"\r\n", "\u001b[39m adding '/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.c' to sources.\u001b[0m\r\n", "\u001b[39m adding '/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7' to include_dirs.\u001b[0m\r\n", "\u001b[39mcopying /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.c -> /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n", "\u001b[39mcopying /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.h -> /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n", "\u001b[39mbuild_src: building npy-pkg config files\u001b[0m\r\n", "\u001b[39mrunning build_ext\u001b[0m\r\n", "\u001b[39mcustomize UnixCCompiler\u001b[0m\r\n", "\u001b[39mcustomize UnixCCompiler using build_ext\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39mcustomize Gnu95FCompiler\u001b[0m\r\n", "\u001b[32mFound executable /usr/local/bin/gfortran\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39mcustomize Gnu95FCompiler\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39mcustomize Gnu95FCompiler using build_ext\u001b[0m\r\n", "\u001b[39mbuilding 'fib' extension\u001b[0m\r\n", "\u001b[39mcompiling C sources\u001b[0m\r\n", "\u001b[39mC compiler: /usr/bin/clang -fno-strict-aliasing -I/Users/jakevdp/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes\r\n", "\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders/_q\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n", "\u001b[39mcompile options: '-I/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7 -I/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/jakevdp/anaconda/include/python2.7 -c'\u001b[0m\r\n", "\u001b[39mclang: /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.c\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.c:2:\r\n", "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.h:13:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:15:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1728:\r\n", "/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\" [-W#warnings]\r\n", "#warning \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\"\r\n", " ^\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.c:338:30: warning: equality comparison with extraneous parentheses [-Wparentheses-equality]\r\n", " if ((fp->defs[i].func==NULL)) {\r\n", " ~~~~~~~~~~~~~~~~^~~~~~\r\n", "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.c:338:30: note: remove extraneous parentheses around the comparison to silence this warning\r\n", " if ((fp->defs[i].func==NULL)) {\r\n", " ~ ^ ~\r\n", "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.c:338:30: note: use '=' to turn this equality comparison into an assignment\r\n", " if ((fp->defs[i].func==NULL)) {\r\n", " ^~\r\n", " =\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "2 warnings generated.\r\n", "\u001b[39mclang: /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fibmodule.c\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fibmodule.c:18:\r\n", "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.h:13:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:15:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1728:\r\n", "/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\" [-W#warnings]\r\n", "#warning \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\"\r\n", " ^\r\n", "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fibmodule.c:111:12: warning: unused function 'f2py_size' [-Wunused-function]\r\n", "static int f2py_size(PyArrayObject* var, ...)\r\n", " ^\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "2 warnings generated.\r\n", "\u001b[39mcompiling Fortran sources\u001b[0m\r\n", "\u001b[39mFortran f77 compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-second-underscore -m64 -fPIC -O3 -funroll-loops\r\n", "Fortran f90 compiler: /usr/local/bin/gfortran -Wall -fno-second-underscore -m64 -fPIC -O3 -funroll-loops\r\n", "Fortran fix compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-second-underscore -Wall -fno-second-underscore -m64 -fPIC -O3 -funroll-loops\u001b[0m\r\n", "\u001b[39mcompile options: '-I/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7 -I/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/jakevdp/anaconda/include/python2.7 -c'\u001b[0m\r\n", "\u001b[39mgfortran:f77: fib.f\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39m/usr/local/bin/gfortran -Wall -m64 -Wall -undefined dynamic_lookup -bundle /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fibmodule.o /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/src.macosx-10.5-x86_64-2.7/fortranobject.o /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI/fib.o -L/usr/local/lib/gcc/x86_64-apple-darwin12.4.0/4.8.1 -lgfortran -o ./fib.so\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39mrunning scons\u001b[0m\r\n", "Removing build directory /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpe_sWoI\r\n" ] } ], "prompt_number": 19 }, { "cell_type": "code", "collapsed": false, "input": [ "import numpy as np\n", "import fib\n", "a = np.zeros(10, dtype='d')\n", "fib.fib(a) \n", "print a " ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "[ 0. 1. 1. 2. 3. 5. 8. 13. 21. 34.]\n" ] } ], "prompt_number": 20 }, { "cell_type": "code", "collapsed": false, "input": [ "a = np.zeros(10000, dtype='d')\n", "%timeit fib.fib(a)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "10000 loops, best of 3: 26.8 \u00b5s per loop\n" ] } ], "prompt_number": 21 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Fortran version is about 300x faster!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Using an interface file\n", "But this is a bit awkward that we have to create the array that will hold the results... It would be nice if this could happen automatically. We can make this happen through the use of *interface files*, which have a ``.pyf`` extension.\n", "\n", "An interface template can be generated automatically with f2py. We'll tell it that we want a module called ``fib2``:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!rm -f _fib2.pyf\n", "!f2py fib.f -m fib2 -h _fib2.pyf" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Reading fortran codes...\r\n", "\tReading file 'fib.f' (format:fix,strict)\r\n", "Post-processing...\r\n", "\tBlock: fib2\r\n", "\t\t\tBlock: fib\r\n", "Post-processing (stage 2)...\r\n", "Saving signatures to file \"./_fib2.pyf\"\r\n" ] } ], "prompt_number": 22 }, { "cell_type": "code", "collapsed": false, "input": [ "!cat _fib2.pyf" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "! -*- f90 -*-\r\n", "! Note: the context of this file is case sensitive.\r\n", "\r\n", "python module fib2 ! in \r\n", " interface ! in :fib2\r\n", " subroutine fib(a,n) ! in :fib2:fib.f\r\n", " double precision dimension(n) :: a\r\n", " integer, optional,check(len(a)>=n),depend(a) :: n=len(a)\r\n", " end subroutine fib\r\n", " end interface \r\n", "end python module fib2\r\n", "\r\n", "! This file was auto-generated with f2py (version:2).\r\n", "! See http://cens.ioc.ee/projects/f2py2e/\r\n" ] } ], "prompt_number": 23 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Take a look at the default settings. We have our subroutine called ``fib(a, n)``, which lists the following:\n", "```\n", "double precision dimension(n) :: a\n", "```\n", "This lists a variable which is an input into the routine: an array of length ``n``\n", "```\n", "integer, optional,check(len(a)>=n),depend(a) :: n=len(a)\n", "```\n", "This tells us what the routine expects ``n`` to be. It is optionally specified, with a default value which is the length of ``a``. Also, there is a check in here that makes sure ``a`` is long enough. Let's check this out:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from fib import fib\n", "a = np.zeros(10)\n", "fib(a, 10) # specify the optional keyword\n", "print a" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "[ 0. 1. 1. 2. 3. 5. 8. 13. 21. 34.]\n" ] } ], "prompt_number": 24 }, { "cell_type": "code", "collapsed": false, "input": [ "a = np.zeros(10)\n", "fib(a, 5) # specify only part of the array\n", "print a" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "[ 0. 1. 1. 2. 3. 0. 0. 0. 0. 0.]\n" ] } ], "prompt_number": 25 }, { "cell_type": "code", "collapsed": false, "input": [ "a = np.zeros(10)\n", "fib(a, 20) # specify a length which is too long" ], "language": "python", "metadata": {}, "outputs": [ { "ename": "error", "evalue": "(len(a)>=n) failed for 1st keyword n: fib:n=20", "output_type": "pyerr", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31merror\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0ma\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mfib\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m20\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# specify a length which is too long\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31merror\u001b[0m: (len(a)>=n) failed for 1st keyword n: fib:n=20" ] } ], "prompt_number": 26 }, { "cell_type": "markdown", "metadata": {}, "source": [ "What we want to do is to modify this interface so that the length ``n`` is an input, and the array ``a`` is an (automatically constructed) output. We'll do this below; for more details on the options available in F2Py interface files, see the [F2Py documentation](http://cens.ioc.ee/projects/f2py2e/usersguide/)." ] }, { "cell_type": "code", "collapsed": false, "input": [ "%%file fib2.pyf\n", "! -*- f90 -*-\n", "! Note: the context of this file is case sensitive.\n", "\n", "python module fib2 ! in \n", " interface ! in :fib2\n", " subroutine fib(a,n) ! in :fib2:fib.f\n", " double precision dimension(n), intent(out), depend(n) :: a\n", " integer intent(in) :: n\n", " end subroutine fib\n", " end interface \n", "end python module fib2" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Overwriting fib2.pyf\n" ] } ], "prompt_number": 27 }, { "cell_type": "markdown", "metadata": {}, "source": [ "We've specified the intent and the dependencies of each variable using the ``f2py`` specifications.\n", "\n", "Here we compile fib2 using the interface file we've created:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!f2py -c -m fib2 fib2.pyf fib.f" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39mrunning build\u001b[0m\r\n", "\u001b[39mrunning config_cc\u001b[0m\r\n", "\u001b[39munifing config_cc, config, build_clib, build_ext, build commands --compiler options\u001b[0m\r\n", "\u001b[39mrunning config_fc\u001b[0m\r\n", "\u001b[39munifing config_fc, config, build_clib, build_ext, build commands --fcompiler options\u001b[0m\r\n", "\u001b[39mrunning build_src\u001b[0m\r\n", "\u001b[39mbuild_src\u001b[0m\r\n", "\u001b[39mbuilding extension \"fib2\" sources\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n", "\u001b[39mf2py options: []\u001b[0m\r\n", "\u001b[39mf2py: fib2.pyf\u001b[0m\r\n", "Reading fortran codes...\r\n", "\tReading file 'fib2.pyf' (format:free)\r\n", "Post-processing...\r\n", "\tBlock: fib2\r\n", "\t\t\tBlock: fib\r\n", "Post-processing (stage 2)...\r\n", "Building modules...\r\n", "\tBuilding module \"fib2\"...\r\n", "\t\tConstructing wrapper function \"fib\"...\r\n", "\t\t a = fib(n)\r\n", "\tWrote C/API module \"fib2\" to file \"/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fib2module.c\"\r\n", "\u001b[39m adding '/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.c' to sources.\u001b[0m\r\n", "\u001b[39m adding '/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7' to include_dirs.\u001b[0m\r\n", "\u001b[39mcopying /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.c -> /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n", "\u001b[39mcopying /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.h -> /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n", "\u001b[39mbuild_src: building npy-pkg config files\u001b[0m\r\n", "\u001b[39mrunning build_ext\u001b[0m\r\n", "\u001b[39mcustomize UnixCCompiler\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39mcustomize UnixCCompiler using build_ext\u001b[0m\r\n", "\u001b[39mcustomize Gnu95FCompiler\u001b[0m\r\n", "\u001b[32mFound executable /usr/local/bin/gfortran\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39mcustomize Gnu95FCompiler\u001b[0m\r\n", "\u001b[39mcustomize Gnu95FCompiler using build_ext\u001b[0m\r\n", "\u001b[39mbuilding 'fib2' extension\u001b[0m\r\n", "\u001b[39mcompiling C sources\u001b[0m\r\n", "\u001b[39mC compiler: /usr/bin/clang -fno-strict-aliasing -I/Users/jakevdp/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes\r\n", "\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders/_q\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7\u001b[0m\r\n", "\u001b[39mcreating /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7\u001b[0m\r\n", "\u001b[39mcompile options: '-I/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7 -I/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/jakevdp/anaconda/include/python2.7 -c'\u001b[0m\r\n", "\u001b[39mclang: /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fib2module.c\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fib2module.c:18:\r\n", "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.h:13:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:15:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1728:\r\n", "/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\" [-W#warnings]\r\n", "#warning \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\"\r\n", " ^\r\n", "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fib2module.c:104:12: warning: unused function 'f2py_size' [-Wunused-function]\r\n", "static int f2py_size(PyArrayObject* var, ...)\r\n", " ^\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "2 warnings generated.\r\n", "\u001b[39mclang: /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.c\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.c:2:\r\n", "In file included from /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.h:13:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:15:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:\r\n", "In file included from /Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1728:\r\n", "/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\" [-W#warnings]\r\n", "#warning \"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION\"\r\n", " ^\r\n", "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.c:338:30: warning: equality comparison with extraneous parentheses [-Wparentheses-equality]\r\n", " if ((fp->defs[i].func==NULL)) {\r\n", " ~~~~~~~~~~~~~~~~^~~~~~\r\n", "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.c:338:30: note: remove extraneous parentheses around the comparison to silence this warning\r\n", " if ((fp->defs[i].func==NULL)) {\r\n", " ~ ^ ~\r\n", "/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.c:338:30: note: use '=' to turn this equality comparison into an assignment\r\n", " if ((fp->defs[i].func==NULL)) {\r\n", " ^~\r\n", " =\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "2 warnings generated.\r\n", "\u001b[39mcompiling Fortran sources\u001b[0m\r\n", "\u001b[39mFortran f77 compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-second-underscore -m64 -fPIC -O3 -funroll-loops\r\n", "Fortran f90 compiler: /usr/local/bin/gfortran -Wall -fno-second-underscore -m64 -fPIC -O3 -funroll-loops\r\n", "Fortran fix compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-second-underscore -Wall -fno-second-underscore -m64 -fPIC -O3 -funroll-loops\u001b[0m\r\n", "\u001b[39mcompile options: '-I/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7 -I/Users/jakevdp/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/jakevdp/anaconda/include/python2.7 -c'\u001b[0m\r\n", "\u001b[39mgfortran:f77: fib.f\u001b[0m\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\u001b[39m/usr/local/bin/gfortran -Wall -m64 -Wall -undefined dynamic_lookup -bundle /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fib2module.o /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/src.macosx-10.5-x86_64-2.7/fortranobject.o /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7/fib.o -L/usr/local/lib/gcc/x86_64-apple-darwin12.4.0/4.8.1 -lgfortran -o ./fib2.so\u001b[0m\r\n", "\u001b[39mrunning scons\u001b[0m\r\n", "Removing build directory /var/folders/_q/9lrsb4wj1b5gy71vm32kqjy80000gn/T/tmpegZnM7\r\n" ] } ], "prompt_number": 28 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can import and call the function like this:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import fib2\n", "print fib2.fib(10)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "[ 0. 1. 1. 2. 3. 5. 8. 13. 21. 34.]\n" ] } ], "prompt_number": 29 }, { "cell_type": "code", "collapsed": false, "input": [ "fib2.fib?" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 30 }, { "cell_type": "code", "collapsed": false, "input": [ "%timeit fib2.fib(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "10000 loops, best of 3: 29.3 \u00b5s per loop\n" ] } ], "prompt_number": 31 }, { "cell_type": "markdown", "metadata": {}, "source": [ "From our simple FORTRAN function, along with an interface file and a compilation with f2py, we now have a Python module which is callable in a very intuitive way.\n", "\n", "If you look into the source code of SciPy, you'll see that this is how much of its functionality is implemented, through wrapping pieces of the [NETLIB](http://netlib.org/) repository." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Learning More\n", "- [F2Py user's guide](http://cens.ioc.ee/projects/f2py2e/usersguide/)\n", "- [SciPy F2Py cookbook](http://wiki.scipy.org/Cookbook/F2Py)\n", "- [numpy docs: using Python as glue](docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html\u200e)\n", "- [Sage F2Py documentation](http://www.sagemath.org/doc/numerical_sage/f2py.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cython\n", "\n", "CTypes and F2Py provide the ability to wrap Fortran, C, and C++ code so that it can be imported into Python. Cython enables this as well, though we will not focus on that part of it here. The biggest part of Cython is that it lets you **convert Python code** and Python-like code into compiled C code, which can run many times faster than the original code.\n", "\n", "Let's see a quick example. Here's a Python function which computes the N^th fibonacci number:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def nth_fib(n):\n", " a, b = 0, 1\n", " for i in range(n):\n", " b, a = a + b, b\n", " return a" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 32 }, { "cell_type": "code", "collapsed": false, "input": [ "[nth_fib(i) for i in range(10)]" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 33, "text": [ "[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]" ] } ], "prompt_number": 33 }, { "cell_type": "code", "collapsed": false, "input": [ "%timeit nth_fib(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "100 loops, best of 3: 3.25 ms per loop\n" ] } ], "prompt_number": 34 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we'll take the **exact same** code, and compile it with Cython. In general, this will be done by saving the code to file, and running ``cython`` on the command line. You can read about that in the [documentation](http://cython.org). Here we'll use IPython's Cython magic to streamline the process:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%load_ext cythonmagic" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 35 }, { "cell_type": "code", "collapsed": false, "input": [ "%%cython\n", "def nth_fib2(n):\n", " a, b = 0, 1\n", " for i in range(n):\n", " b, a = a + b, b\n", " return a" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 36 }, { "cell_type": "code", "collapsed": false, "input": [ "[nth_fib2(i) for i in range(10)]" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 37, "text": [ "[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]" ] } ], "prompt_number": 37 }, { "cell_type": "code", "collapsed": false, "input": [ "%timeit nth_fib2(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "100 loops, best of 3: 2.81 ms per loop\n" ] } ], "prompt_number": 38 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Just compiling the code in Cython gave us a ~10% speedup. But we can do better by adding **type annotations**.\n", "\n", "See, the main reason Python is slow is because it has to do dynamic type checking each time it evaluates an expression. If we can tell Cython what the types are from the beginning, this step can be skipped, and we have large time savings. We do this through a ``cdef`` command. We also do the temporary assignment explicitly to remove the Python tuple assignment:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%%cython\n", "def nth_fib3(int n):\n", " cdef int a = 0\n", " cdef int b = 1\n", " cdef int tmp\n", " for i in range(n):\n", " tmp = b\n", " b = a + b\n", " a = tmp\n", " return a" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 41 }, { "cell_type": "code", "collapsed": false, "input": [ "[nth_fib3(i) for i in range(10)]" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 42, "text": [ "[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]" ] } ], "prompt_number": 42 }, { "cell_type": "code", "collapsed": false, "input": [ "print \"Python only:\"\n", "%timeit nth_fib(10000)\n", "print \"Bare Cython:\"\n", "%timeit nth_fib(10000)\n", "print \"Typed Cython:\"\n", "%timeit nth_fib3(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Python only:\n", "100 loops, best of 3: 3.13 ms per loop" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n", "Bare Cython:\n", "100 loops, best of 3: 3.29 ms per loop" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n", "Typed Cython:\n", "100000 loops, best of 3: 5.99 \u00b5s per loop" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n" ] } ], "prompt_number": 43 }, { "cell_type": "markdown", "metadata": {}, "source": [ "By adding some type information to our script, we sped up the execution by several orders of magnitude! This shows how easy Cython is for simple problems." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One very useful trick to be aware of is ``cython -a``. This will produce an annotated html document showing which lines of the program are causing problems. You can run it like this:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%%file nth_fib.pyx\n", "# Slow version:\n", "def nth_fib2(n):\n", " a, b = 0, 1\n", " for i in range(n):\n", " b, a = a + b, b\n", " return a\n", "\n", "# Fast Version:\n", "def nth_fib3(int n):\n", " cdef int a = 0\n", " cdef int b = 1\n", " cdef int tmp\n", " for i in range(n):\n", " tmp = b\n", " b = a + b\n", " a = tmp\n", " return a" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Overwriting nth_fib.pyx\n" ] } ], "prompt_number": 44 }, { "cell_type": "code", "collapsed": false, "input": [ "!cython -a nth_fib.pyx" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 45 }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we open the resulting HTML file, we'll see a highlighted version of our code. The darkness of the highlight shows how many lines of C code were generated by the line of Python. More yellow lines generally means slower code. This can be very helpful when your code is not running as quickly as you'd expected. To see the results, open [nth_fib.html](files/nth_fib.html) after running the above code." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Using Cython with NumPy\n", "\n", "Cython provides a really nice interface to numpy arrays via the [Typed Memoryview](http://docs.cython.org/src/userguide/memoryviews.html) syntax. Let's implement the same fib function as above, but using Cython.\n", "\n", "First we'll simply compile our Python function again:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%%cython\n", "import numpy as np\n", "\n", "def fib_cy(N):\n", " x = np.zeros(N, dtype=float)\n", " for i in range(N):\n", " if i == 0:\n", " x[i] = 0\n", " elif i == 1:\n", " x[i] = 1\n", " else:\n", " x[i] = x[i - 1] + x[i - 2]\n", " return x" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 46 }, { "cell_type": "code", "collapsed": false, "input": [ "print np.allclose(fib_py(10000), fib_cy(10000))\n", "%timeit fib_py(10000)\n", "%timeit fib_cy(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "True\n", "100 loops, best of 3: 8.73 ms per loop" ] }, { "output_type": "stream", "stream": "stderr", "text": [ "-c:1: RuntimeWarning: overflow encountered in double_scalars\n", "-c:257: RuntimeWarning: overflow encountered in double_scalars\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n", "100 loops, best of 3: 7.58 ms per loop" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n" ] } ], "prompt_number": 47 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again, a very small improvement. Let's add some type information and see how we do:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%%cython\n", "import numpy as np\n", "from numpy cimport float_t\n", "\n", "def fib_cy2(int N):\n", " cdef int i\n", " cdef float_t[::1] x = np.zeros(N, dtype=float)\n", " for i in range(N):\n", " if i == 0:\n", " x[i] = 0\n", " elif i == 1:\n", " x[i] = 1\n", " else:\n", " x[i] = x[i - 1] + x[i - 2]\n", " return x" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 48 }, { "cell_type": "code", "collapsed": false, "input": [ "%timeit fib_cy(10000)\n", "%timeit fib_cy2(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "100 loops, best of 3: 7.82 ms per loop\n", "10000 loops, best of 3: 37.2 \u00b5s per loop" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n" ] } ], "prompt_number": 49 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Wow! Simply adding some type information for the Cython compiler made our code orders of magnitude faster! This is because the array can now be treated as a **contiguous memory block** rather than a **Python object**. This makes each of the indexing operations much more efficient, because they no longer have to go through the Python interface layer." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Learning More\n", "There's a whole lot more to know about Cython, especially the details of working with numpy arrays. Here are some more resources:\n", "\n", "- [Cython userguide](http://docs.cython.org/src/userguide/): lots of general information\n", "- [Cython docs: typed memoryviews](http://docs.cython.org/src/userguide/memoryviews.html): useful for working with numpy arrays\n", "- [Cython and NumPy](http://docs.cython.org/src/tutorial/numpy.html): some of this is currently a bit out-dated, but that should be addressed in the coming months.\n", "- [Scipy Conference 2013 Cython tutorial](http://conference.scipy.org/scipy2013/tutorial_detail.php?id=105) Very good video tutorial by one of Cython's main contributors." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Numba\n", "\n", "You might wonder whether some of that type information could be inferred by the compiler, or whether the compilation phase is really necessary. This is where [Numba](http://numba.pydata.org/) comes in. Numba is an LLVM-based Just In Time (JIT) compiler for Python. It allows simple annotations of Python code which can lead to huge speedups for certain operations.\n", "\n", "Numba comes from Continuum Analytics, a company founded by the creator of NumPy and SciPy, and the same people who are behind the Anaconda Python distribution. Numba is still relatively young, and can be a little finnicky for some problems, but the approach is really clean.\n", "\n", "You can install numba from scratch, but because of the dependencies on LLVM and other difficult-to-install tools, I'd highly recommend just going with [Anaconda](https://store.continuum.io/cshop/anaconda/).\n", "\n", "Let's go back to our Python ``fib()`` function:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def fib(N):\n", " x = np.zeros(N, dtype=np.float64)\n", " for i in range(N):\n", " if i == 0:\n", " x[i] = 0\n", " elif i == 1:\n", " x[i] = 1\n", " else:\n", " x[i] = x[i - 1] + x[i - 2]\n", " return x" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 50 }, { "cell_type": "markdown", "metadata": {}, "source": [ "To speed this up with Numba, we simply use the ``jit`` decorator:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from numba import autojit\n", "fib_nb = autojit(fib)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 51 }, { "cell_type": "code", "collapsed": false, "input": [ "fib_nb(10)" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 52, "text": [ "array([ 0., 1., 1., 2., 3., 5., 8., 13., 21., 34.])" ] } ], "prompt_number": 52 }, { "cell_type": "code", "collapsed": false, "input": [ "%timeit fib(10000)\n", "%timeit fib_nb(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "100 loops, best of 3: 8.51 ms per loop\n", "10000 loops, best of 3: 33.1 \u00b5s per loop" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n" ] }, { "output_type": "stream", "stream": "stderr", "text": [ "-c:9: RuntimeWarning: overflow encountered in double_scalars\n" ] } ], "prompt_number": 53 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Usually, numba is used as a decorator, which essentially does the same as we saw above. You'd write a decorated function this way:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "@autojit\n", "def fib_nb(N):\n", " x = np.zeros(N, dtype=np.float64)\n", " for i in range(N):\n", " if i == 0:\n", " x[i] = 0\n", " elif i == 1:\n", " x[i] = 1\n", " else:\n", " x[i] = x[i - 1] + x[i - 2]\n", " return x" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 54 }, { "cell_type": "code", "collapsed": false, "input": [ "%timeit fib_nb(10000)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "10000 loops, best of 3: 34.2 \u00b5s per loop\n" ] } ], "prompt_number": 55 }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can see that simply by passing our ``fib`` function through the ``autojit`` decorator, we've attained execution which is a few percent faster than what we got with Cython - and there's no fancy compilation stage required! The funcion is compiled *on demand* through LLVM, which makes numba very convenient to use.\n", "\n", "Now for the bad news: Numba is still young, and there are a lot of corner cases where the compiler will just break without any useful error message. For more complicated problems and functions, the output code is not always faster than Python+NumPy, and can sometimes even be slower!\n", "\n", "My recommendation right now is to keep Numba in mind, to try it in your problems, but not necessarily depend on it -- yet." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### More Information on Numba\n", "There's still not a lot of information out there, but here are a few resources:\n", "- [Numba documentation](http://numba.pydata.org/doc.html)\n", "- [Accelerating Python with Numba](http://continuum.io/blog/numba_performance)\n", "- [Numba vs. Cython](http://jakevdp.github.io/blog/2013/06/15/numba-vs-cython-take-2/)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Homework: Accelerating the Mandelbrot Set\n", "\n", "One of the more interesting and beautiful calculations you can do with Python is the [Mandelbrot Set](http://en.wikipedia.org/wiki/Mandelbrot_set), a straightforwardly-generated set of points whose boundary is a fractal. The computation of whether a point is in the mandelbrot set can be done as follows:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%matplotlib inline\n", "import numpy as np\n", "import matplotlib.pyplot as plt" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 56 }, { "cell_type": "code", "collapsed": false, "input": [ "def mandel(x, y, max_iter):\n", " \"\"\"\n", " Given z = x + iy and max_iter, determine whether the candidate\n", " is in the mandelbrot set for the given number of iterations\n", " \"\"\"\n", " c = complex(x, y)\n", " z = 0.0j\n", " for i in range(max_iter):\n", " z = z*z + c\n", " if (z.real*z.real + z.imag*z.imag) >= 4:\n", " return i\n", "\n", " return max_iter\n", "\n", "def create_fractal(Nx, Ny, xmin, xmax, ymin, ymax, max_iter):\n", " \"\"\"Create and return a fractal image\"\"\"\n", " image = np.zeros((Ny, Nx), dtype=float)\n", " dx = (xmax - xmin) * 1. / Nx\n", " dy = (ymax - ymin) * 1. / Ny\n", " \n", " for x in range(Nx):\n", " rpart = xmin + x * dx\n", " for y in range(Ny):\n", " ipart = ymin + y * dy\n", " color = mandel(rpart, ipart, max_iter)\n", " image[y, x] = color\n", " return image" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 57 }, { "cell_type": "code", "collapsed": false, "input": [ "image = create_fractal(300, 200, -2, 1, -1, 1, 20)\n", "plt.imshow(image, cmap=plt.cm.jet)" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 58, "text": [ "" ] }, { "metadata": {}, "output_type": "display_data", "png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD9CAYAAAChtfywAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXl8Y3d97/0+52jfLFuW9/EytmfLzGSSSUKGEEjI1gQI\n5ELCEtqUBprS9pZbSku5ty1tL7fAfR5K4UJbCjy9YQsJZCGQkH2fmWQmM8lkdm/j8b5JtiVrl855\n/vhJI9njRZYlW57o/Xrp5UXS2c/nfH/f33eRNE3TKFGiRIkS6xZ5rTegRIkSJUqsjJKQlyhRosQ6\npyTkJUqUKLHOKQl5iRIlSqxzSkJeokSJEuuckpCXKFGixDon70L+xBNPsGXLFtrb2/n617+e78WX\nKFGiRIk5SPmMI08kEmzevJlnnnmG+vp6Lr/8cu677z62bt2ar1WUKFGiRIk55NUiP3DgAG1tbTQ3\nN6PX6/nYxz7Gr371q3yuokSJEiVKzCGvQj44OMiGDRvO/d3Q0MDg4GA+V1GiRIkSJeagy+fCJEnK\ny2dKlChRosT5LOQJz6uQ19fX09/ff+7v/v5+Ghoa5vnke4Br8rnqJdgIlJPn3Z2DEWgHHMDXgC8V\ncF1rzVdZs/0z68GkB1kCvZL/5c/8Pdj+PvvP1wJuwLCMdViARsCW8b86xCWqB5wq1MWRrXFsphnq\nKwYA2MpJvvjmN2n95SFO7IPhd1ah+8puemlGQqOeQXZEj+G5bYj6d0SorISO3mr2fvQKTu16Dz/y\n/D6BL34btv0DBJPrjQCjwBiQAKaBM0ChKjAlVAhEIRgt0Aq+CvwJ0AMECrSOteKWBd/Jq7Jddtll\ndHZ20tvbS11dHffffz/33XdfPleRIzYKK+KpO7OsgOsogVkPVgPoCiDguVALVLI8EQdxKdrm/M+M\nEHEAgwYWFVWTCUYtaEhIaIQwc6JlE9ZP+PD5Rpn42SQXmw+w7Qsn0BLw+pd8HJ30UXltGYO3bKTT\nVsWx6Haeb3oPneObCHQ5ISyBmrFeI2BK/q4ATqAZ6Afiy9yvbFBksCQPWMHE3EF6J3wFWkdxkVd1\n0+l0fOc73+Gmm24ikUhw9913r3HEioy42/RLfTBHypIvE+Aq0DpKAMUn4jUs3xJfCDdCULOgq2wj\nhrIQ7o+p7Brupvr7HpwBD4cS0BiBiZZyhj+4k+H2i+iknS7a6Iq00TfTDJ4sjp2MuJQTwAhQCK3V\nrYaYOxFPLCcwA3gLtJ7iIO9m6s0338zNN9+8xKea873aBZAQQl6Im9+BGA9XzPPeuwqwvmJilfZP\nQrhRJEnc+LpVyF8zXLP4+ymhqyG3u8dEeuAmIS6f6uyX5ceGh0oa3lHB5s+MoPUEOP5diF0PO/6u\nDP2WFp43XsppNtNHIwPxBmZ8DphM3gPbrxHa5gFiyYWak9s0nbGiKoQOjiPcL/lGJ4sHs6ZBKLb0\n57Mm89qsSL6mEb6iyTyup7gopL9hEZpXYR2pcWIhJldtQAPCqTkfVxdgncXEKuyfJIFRBw5zYU7h\nQiwl5ArCi5bLNpkQ1nd18m8ZqGd27JhJA7M695tYCaCQOPd3TDYQrTOhXR6gexR+9x8kDm7ZSIeu\njQEa8FJBBCPBqAXvlAv8yZVsT+5fgLSQ20n7xzOpSf4slJgrMthNoGoQTQhRXzHzXZtliIOc4EJ1\ntVygKfoK4uS1kf9dNAMtLCziJVZMSsSdqyziSyEjTn+ulDNbxE3zfMaVAHfivH/XMYQdPwoJdMSo\nmPJiO+Vh5jRUqjDgl+jT6hmkgRAmJilnOmX6W1RozNHqrUFY54XyTsoSlFvAoIjzXjDsQCtiPuvC\n4wIVchsigiTf6IHNCLdKiYIgkRbxYkJCXFabyO3hopC+2yTEA6Gd2XegMvtvCQ1FjqMjjkKcDfRR\nziQ12ijVByY4+6/wsxfg0DDc+x6ITSYADQUVHXFkzrfsF1rXolQjvIiFnJ44J+YFXAcWYDtiYqOY\nLISVc4EKeSGQgF2Ada035MLGpC8+EQfx7F6JbVBP2lVhQwwW57IhBlXpUBGjPkK7u5N2xKuVHq5i\nH5fHXsc9MgED6a9KaGykGxMRahminU5ceBbeniaWF2RViXApFZJyCxgLZfqn0AOXkZ9Z6uJhjXzk\nhaSC/PvgjcAOCje+LIHNKCJTijFhzIWYEsl105oRbhUJMW1TP8+yWmJQngAJyizTuB1jmAjTLnXQ\nTifX8Synbz+F8xN+3BfH0I1o52K9HeXwqc9Dn1VjD/sZoIEgFqTMYHCTBhdF4KRRTGJKc7bBgXi4\ndC2wD1JyH2SgO8fjkA0Ok1hXXidA5yIh7udTiIiW9c8FKOQK+RVcK+IKzzI+rMTysRvBbBD+0mLD\njbCkc72kmhHinXJLyBnLkoG2KMhQXTOMyRwCQCfHKVOm2dXxBn/yuf9DxbVhyuVJzr4exGBQefPn\ncPw1CI6KxcT80PFjjc3XDKCbluh/4AyW/2LG+cHW9HZICDHfFIUePUSltL/em9yWpYxUGSH4WxAP\nkS7gfHf+ypAl8VCXpAKGJoK4n9sQ2U9zZ3nXHxeYkJcjZmbyhZ3zU/BK5JViF/EqcnuGS4hLp5y0\niJchXBRmDWrjKHKCSxpfR5YThIwW4rKCAx8OfNiYod49SPD3bOw604V0QEOZgteeg1AYJqbSq4rE\n4fUu2PmDEHt7oGl7EGNDP356GTLUEbMbmPBXJv38KjTFYEAvNnK5DyiF9O3Qgsi5yXdEiyKLV8Gx\nInxM/az30MQLTMgt5E90yxBjYGeellfiPGzrQMRzcdfrEOkLLtKzUM7kMt0q1CTQuSLUOIa50fIk\nu4aOsbfiSjrM7UhoVODBhQdDeZThj9QRe/NN9NNQ/RocH0ln16dQgQkVnnsCJr1gl6D+oW62RDvp\n2dMKVlDkBKomMeF3o5WpoMZhRAGHLIR4CiHqdcBQlvvpJJ04FMrhOC2GQRGutoK6WEAYaylf1/pN\nGrqAhNxB/kQ8lexTCjHMO7IkblAQCSHF6hOvZv7wwKXQIx4A1Rn/KyP5YFChOoG+MkyldYJN1g4u\n5yDu//syG+U+LB98BzPbmpDRqMBLOZOU6yZJbAN9DYu6PjTg4Cjc2A5xG6CCRQtRgQdJ0dDZEkxq\nTlRNRtMkHDY/9dYRRgwNdIfa0kLuBobJvtaKK/nZECI2PV8uZ70iksA0IFxoMU/N+q7fpKELSMhd\n5Md6trN4sk+JnFFkIeK2Ip1vSE1GNpCbT9yAcJ3Uzvm/C3CrOBomcVRNYtXP0G7vZDOnqWWY4Rfj\nVO07TEN9hNC2zfixYdNmqPP0En9pGGWTBqdgNHi+NT6Xlhao/jQcvcZFqFxPJR4UVMqYYkyqZqvj\nJEEsbKKDPfb9PK7dSvfERpiUhRDLiEt/ChaLXpxFZfLnJEIL81WrSq+Ia0XThP+ooKSShlTWo8/8\nAhHyzIpDK8HC8uOySmRFyhIvVhGXEQO6FpYflGtEXH6pgVwmZsCqUb5hgj3le9muHENniNLc140y\nMIgPPzs2aKgOUCOjaOEASCCPBZl6bIyXvgDX3gPDhyCylIoDDIGqQtXoGLXRUcarq7AQQEJju3ac\nHWPHGQ1WUek5yYZaDy873w0tCZiRxbyfgigW+hbLr7OSimoZRuhhNtu7FDpZRLKMr0Z0iR1xAXSx\n3qJZLhAh38DKLWgDIlC4NLGZd+RkrZRiFHEZcReYEMk+yyE5X3iuCuJ87zcBm2Ps2fAyf979bXYM\nv8lwmY3yB4Z4+GdwPAj7AyKg5P0dE2w6NQF6OPMIPP03oJNg5DtwfwxmsnB3zMTA8SzYfuahZncX\ng/c4mXbVoItFqT3bxRW/fJGK0Tj3vWzk/i99kMM3XCLE26iBXkqn7RsQvy83az5VRy4CnCZ/RbcU\nWZTALThWRFjOUcTGF6qeb365QIR8pUjATkohhgXCahT+8GIkl7SDlFu/lYWf+zIiCTh5h2lIxLbK\nnH3VzxN/4sPVLPHf7tfgQfjOj+HdAWg9DPI04AK7RzxXjmvww2WI4U9Pw6bTwjMy/nA3jd0T7Plh\nLf7OKC/s6OE7Kvzp30LNI5fz64ab6fa1irIA2zXQDNCB0K4tCCH2L/PYpDACFyEse5WV6aEig9sG\nY35Rl6XgGIHdwOsUpvxj/ikJOXpExmYp2acgOEzpyc1iI5V6vhx0wDaEmC909xgQKpzx/jG281P9\nnXz44zruuOlNTpouIqJ/EaMKPAKPT8HpVyDyKoxIyTpSy94hQQ9CO99xDbzrVh8EA2DU2H4H/PuD\n4jP1DNJOJ332RmRJY2yqJj0qSYl5K9CLeCrkgoLIiE9tVK4PhRSVNvAGIL4alnkqk/sY+fERFZYL\nQMhbyb32iRlhNhWptbieKbcIl4oiF2dkSi0iuiTb+iF2hAdPYvHLxYxwp6TurI0xsKvE0HOCbUi2\nO7jM2kJc1vNCDK4zvwJygjDQFQU1mvZu5EpqWlB6E3r+WqPv3+K87xLQD4KkQuheeOvRQdxf2c8N\n709gkKOMUSP2LfOZq0vuswKLZfsvSmp5qXDtlcwjyhI4LeALiWqJBUePGJp0UexVEy8AITeTWzUf\nG2JMXaqdkldkSZSeLXg1uxWQ8mlnM1BwkW7Blk3hvLlVDU0aKDAVKEeWVOwWP1ZphmpGORnfwrU/\n3g9TQpTynVdzagq6pyBwFmZOQMIPMRX0d8CV8Sj7fvIGvpAFyy1XpL+kMLtDkJH8qIQJEa7tRoj5\neI7L0a22YWAmfUCKNzRxHQu5hIgTy8Wv7Uh+txSdklcUWUxomorwspJI1/52svQgrAohPjayr3xq\nYcHE4nDMRDRuJIaeaZzoiVOvGyJ+jcZ7DsKLx2Aiz0k1kyl3cghO9QsNvKERbLdCf2Ub/Se2cbz5\nSvrZkP6ShLgtBjMWVI64zYLAxAo2yJJ8pW7ZXMXckmxIsSpWOayHpKEivOOWg5vcdsFCKU48zyjJ\nji/F6A9XEAJbxeKhhRLCby4hLPbl2AgpEc/CNoihZwYbZbpp1PdpbH8T+gYhGCq8NzYShw53C8c2\nXULv1ncwyA4mAvOE3FQihDaGeJjZEEk/Grm7WVKYSXdFSiAaPy8Ho07syKoJOYgT66Mk5HlFQYQb\n5FKPwYp4wpbIG6kejJYim2swIgRIjzCoFiIVAy4jJj+Xc1nZENa9nazz0UyEaZrpo/mFfrQxDc0O\nBnNhy30DqAqMXQWDjnYG5TrCiz2pqkiLdmrW1YxwS6ms3MuQcrUkksvzkn0CEohkIb0CsdUUcyvi\nQik+f/k6FXIdImthuVhIO+pK5AWlSEV8oSzLTIzJlwXhaVsuFkRlxIVCEO0qKOeHyznwsXv8EJf/\nwxucOgaGPdAzvfKgjsUw6KBlI7z/q7C3KsQgCSIIVw8gbimrCoGMp1iqzECmmJsQxcBS+hlgZRUQ\nUz75VEJltstKjfwCkVWKYoG08dhL/tJX88M6FfJcMCKmzudrllxi2SiycEEUm4jrEeJQSbqRw1wM\niPsxs3/mcjEiIjoWKqolAa0x0M0WchkVUyyMeTrISBgejkLs+Ry3YRnYLfCByyD4Cth39mIwVeOp\nvQqvvYJ4QpesihiHE3POZTVi0jPTn60nnTzVjXC5xFiZoG9EhCgGk+vLJiM/JebT+a7YtRhORKRc\nBxBexfUuzjoUconlD0IVRNZmaXIzL0gSlJmEmVcsyMwuHbvYZ1ZShSFlsLay+ITpHEtckjQkScOO\nj50jR9n10F7+o1NEkawGQR88+1N466cgy/2opiGU7+9g6gPvxDOT4SNPlRvJJHXc5svFSZU87yc9\nEarNs4xsSA2yBxEVFbPN/ZGkPDVuzhY7IpngCPkvyJ4b67DVmxVxEJfDdkp9NvNIhaW4RBxgKyI5\ndyE/tYTINNzJytIOLkq+FhNxgwY7IrOs8SrHGC5bOuTDVSUyLA2rlEwcRiRZJoBPbYMrfnMFrR/R\n0W7roMqRnG00q7Atcn73omoWHt2kqEcc253k5vXMpJbsE7XWrDWgCbiUws9sZEeR3Y3ZkKtFXqQx\nzesJSRIiriuOixcQV/BmxH013ym2IOogpRJ5crkMyhFiJrP0pWdRhUsl43M1zmEcZh8Vkpcqxpio\ncfHUJ66hMd7Lp9v6OPx5lWNDhS3TpAGKAndvhWf6YYh6QgYbMiqSlDSfJUTNlYsicNoAseTBkhHx\n9BIL1yrPNAkdiIddJmfJfgdTbi8ZYekvxtyWdatGLjpUONaZkDsgM+Z1SVIFL4rIh7teUWThTtEX\nycVrQ1htqY7085GqSJyLwVaX8T092dUmt6vQEBdimCQl4oqcQCGBnjgzeisdFa1UB4ax/BCGJwvv\nbXVIcJMNKv4fMCq7Sex0k5hvQJ5qCdcag7N6CCVVUke67+jg+V+bhcz5x7yR+f3ecUTVxbmeER3p\n6aylxFynQJl5lX3lIHZ0K9BJ/tO5lsc6E3I92VcnNDC7622JnNHJYCsSn3g5wr9tYGEXSUXyPSPC\nE6dj+TVV7Cyv/E6ZCjVxsKoocoKqMtFQ027yo8gJypjGkQxbS6Dgx86IXMPmjV3MHIB4gTUookGn\nBm2VYN7pYspQSyiZ6WQ1BqmwefHOZAQC2FRoiAmr3KuATxbHoxwhutl2EUqxUFKVirhNU0I+Tjog\nRI84lxowsMiyZUkkoWkm8IdXsWBhKnuqBehjLWuyFMGdWQgy84FLrAi9IhJ91jpbM5WgY2fxZ3kF\nwg1iRViFqcbH8wUr2VVwLDArN6FAZAkDwJ0Q/nAAq4qxIoTDPI0iqzgt6UpTZUzjwoMpw+5OoDCt\nlKG7VGJPN+w7AN4ChidHgBOqxCU2F+NyJSoSMipWArh0E2DVUOQ4qibj8ScnP8uSx8aggV0W1rlX\nSXcFAjHBuZLiMCm3TaruzSQiQSiAiIbxIm5jFTEButAkqpSsd++PsPqlZ12IDRtkrcIS15GQW8g+\n1MDE0rMzJRbFoIghq14RE0prgUTaN5tN6zUn4rS7ESJuTn4/hSsxy5drqgyweeo4m7o6OHXLVvq1\nRqZDZWiaBLIG4SViAWqEG8VsCGHShzHpQzits0sFOvCdJ+I64jjwgwyJdtj9IThxtnBC7jDApjqQ\n9khMNLnw6FxYCGIkjDXZdELTSRjsUVRVRlXnuM+sosRAaNIqxDaBcLinyIwzXw460pPTWxN86M1f\n4942wqtt7+TomZ1C0FVE9cVUItIEi4QmJsU8HFulcreZuBEbWhLyJUg1PlyKlNlWImf0iqghblyj\ny0OHsLplhG81m9iqVPmcCjC2htHXRdATx4EPOz56aSZQZ0CTJcyGEIocp848xEePPsCHf3ofP7ff\nxAvKFXRcdiOj8Vri1bBYaJnVGECSVMyEqbSOYzfNn85TxRiGpMoZiGJjhjKmcTOOVRdg6p1WXHV+\nan+qMt4F/gLoj9sJ7/8diP1XOG4EC0GCmDERJoEfIxEkNFRkgrKFGufwrO8biaAd9+EfMjBzaSuB\nqI3Aaadwt6Ti8FP6FWPxRswm0lNWRtIZt01x3n/4Ea6vfYpvVnyeDnM7EbMRZFkIuA9xfjWElT7f\nKEBClE2OxtdAyCE9DFxtX/26EXID2U1YpjrfNhZ2cy5kdDLYTcIiX230yZcVEeu9FJmVBltIFrTU\naLm0gzrzaZTBKTbbetjSPsYP+DT9E4041SmaPT1obpl6ywDbN5ykyjLKO+/6MZaLHsX9pJHnwr/D\nTFz4b+IJHRoSemW2ctSVD1KljFHd24E9EUYxGVEno2gTYcztJvxzjAkdcVxMsIF+qhjHxgx1DOHV\nu7BVBbmqUYUOOOITXX7yhdkI5c2gvQtizYAEG+gnhg5d0rSdohwJDTMheuaJHaxmlNYn7sP09Cl6\n/s8f0rnxHXTJ7URO2NHiEtTJ6WfeNMIFshCVzB4lSYiwRwl+8qcfxWTw42SCS6tfZahiA2f9bUKl\nOhH6uAHhJ1qsRrpOhoS2yrHlkJ59H2S1Jz/XiZDXkp2rpIaSiK+AVL1n3SqmF2RegVUsnlKfiYQQ\n7sz2bI3A1jjvdT3D1f/xLfr+qg/z1RXYf3s9O0xH2VZ5gru0e9n8R4d5/eY9jN3ZQGVsBH0DXHqd\nAdMvWjmCRpOllxBmNCQ8/koSmpyOtYZzAvgBfo3liw+h2+7Acs9GAo8MkfjxSS57aTNPSjee+7w6\nHaNeGcWh+VA1cNqDNNCPrKm0zPRiOq0i3Qg3SKB/CV4eyk+aiQxsa4b3Xy00TYqBTpegRXcGWVYJ\nJEs4e0N2rFEP8TLruX1LL0OlhhHchjF0rxzF/tlvYXrmS1iqg7zFxUTiRrQzBpiWQZXSrd6yIRUh\nszWKoiSQABWZK2b2c2Pst5yo2sVnGu8VIr4VOIEI71GYP3EphdMC3qCwzFedWsSOda/qWteJkJdY\nFVxWEWa4mrSTfZnYTBxA2/xvPcxt1Cl9fK7tW5h3eYk++Cu23HmSLtrYHj7GC5EApsRxbvruUUa/\nPMY/R8GwxUAdLbTQQxyFLtoJYMVl98yaOtMRp40uJDRMhLlOB11fOckbXzkFqkb95Rrt4S66zOmN\nG/nUfi7dPExwWpSqveY7IKGBCqaTGtIJ4CQwLWSgARF2vVJqSA9spAiYj6tcRDeRLWAoixJQhJD3\n/mgK6akA7Q9uF9uVQS3DtNHN1qs6qP8rmDEN0cE3iaPjv1X9C2+NX0y4OTkpPKyDoWVIilmFrVGQ\noNl9BpMuxDC12P7xLU5JTp74s5vSn5UQeYAnEVEuMrmXwb0AuYCEvInSBGeOKLJI9FktETcgwvtT\nvy83OtSFULvM72U8EMb8VTz4kTuIf9jIlhMvMvzJA8S/fIq7vtBNxVgYOqDu0TFGQjAa1fj0xRD/\nqsTzSEhoSGjUM8gINfhwnFuNkQiN9CGj0kQv7XTyenyGwaiGlhTAkcPwyA0BPvT0b+k1NVMXGcZc\nN82rD6i8NQ6bW8D8L+A1wo+/A8Tgro3wfC9IHpiJLh2mnS2jQE8vbPwRyDawbANpk8b9z8Du8CBb\n/lhC74LoIyonXtIwvP91PvOLTp43X4uW3Os9Lx3i7Pf6+cWLMXRBUKUQpu+d5sNmhfYXOwi6LJyZ\n3EggYoXquJhQnpHhzBIT5A4VWmLnzqEkaTTe/m1Ch95g5LNNeO7ew3CoTpzTNkSTntSJyKwtP7rA\n8p1m8IXFxOeq40ZsZNeqrXEdCHlyBmtRmsm9NvnbGINONIKQWB0Rd5B2I+aamu5GPK/n6kSqo/2g\njoQGQ431vFL2LqIOuPEnAzT/dpAnvx0jOg3jHpgYS5BQoXkruD8Fgd1Rmuklhp5KPIxQjZkg0xk5\n/zIqJsI00M8nIz/j2OcGGXguMCtgwxCB2k6NiscCmFt76Ph6hM6XVfrGIRSD8U547t+gS4apZKLL\nA9MwGQYSwluQL4dAAuiIiv3FC4oHOATDE+BPxOHbMKODg6OQCMLoCxFeun2C6+5/hkFLA3FJR2Vo\ngvBImJ7BlGZqSDMxnNtUWqUeupR2qp0jjE9X4Q/bRY0ZnQpblghj0WkYzBFqy0VA+rvkV/i9T5yl\ns8dHtdaDar+cYX01NMWga56Hgo50tu3w+W8jS2uYPqKQnghYHTFfB8pnZumJzmw+U2IWxqSIFzpT\nMxXTDeIUZZvPNR9uhB99sYdAWIKYRAwdCRR0DpnYNS3UVg+y+2V4sgdCEfFqk2GXDpQEmL1RNps6\naHp1GK1a4sn7zFhuupjguzcxk7HROhK00oNeF8V3NEx0YrY3OwqcnYZHvg7vuziE+0U4NAozSY/F\nRAj8g7MrWg8XMGItqIkXIFaaXPEEsPcsRDXwJt+PBmB0n0rNcx4cGyM8e69M34EQ4VOzS+wG4/Dr\nQZXb/ukh+j7fRLTOgGaXcVimCUasTAbKwTb/RKPD7MNuFhuhyCp2g58aRvhUz0+YfGIM7yAososR\npYaoTg8WTahUEyIpqCH5M4i4nha7FlKdhMJr4SvXsZptJNeBkC9FHbk5Wd/GGHUivLCQIl6LuLrK\nyC69fSlSIj439VtGWOjz7EqDOsCN8aeweL088wNhkWZWGwwAJ/uh8wGwnla5/IPT9H5rmgEbBF+C\nSyumqbzoNP1l9fToWoTPHA8OfAQUK7s/ovDmGIxkGF1xYCwC46+DsR+0qdkiGGGtk7nTDM8zWRgJ\nwHPfhHdfPMPUL0TnokyMQIMe6jZpVGydpMXUg4cKJINGCDNGXQSDTljjEhpuxmf53RMGGdUgYySC\nkyl0xGmnE82q4bxC4opOMNlDeLQJyphm0KiJ8roagE6c7zHSSZRWxLUxn79cn8yFyNsYZ7mkah0P\nUOgkpSIXchdLq0BVFp8pcY6UiOc7vNDI7MqDqVZe+cDFwglBCmmXZAblTFJ7+jiWXx+megoe/Q7s\nroKdLjg1CZMRIWTDHmAf2I4Bg9D1NAzERZhy8MlRaoyjVF7rIX6RjlGqqWASJ1NUxL00Xhqnv3q2\nkKfQEC6L9UYsCvufB0MH+Bboz2mywZUflXjp4y0YLWEq8BDGRAhzOoQUkKMxNn3/F8jRGLYP1xM9\n6SdYXk7wijZCmDARwckkLiboqW7ixjs78B8IAAHqGaRd10mw3MJMuQ2nNs0ZrZ3E5JxJFTOibMBC\nE596RbgQ1ySCRY+wMharL5AfilzIq1nc2q6gmCqQFTVGnUhjthrya4mbEQJrJf9zzRLi4dDAsuqe\nmPRhNpw5gun/7uflb8MVZti+G264GIwh8L4Ik3Nqhcz44OlHZ//vzNPQGIZNdVNs23yS6sQY+kmV\nmhPDzIwME+9IrKwZcRHz4gIzrhFgyADshrDejJkwDvxEmSCa4d6UUSlLeLj0wGN4Hvbg8G4i9PIE\nSq0Z219uY+bSNoaow4YfGzM48KHvTHCkCyzteiokLzd4n2H3wBsMbK+n/Xg332+6mzMHNhOb60/R\nI0Z+0/NssFEHCXWNhBzSNQiW28tueRS5kC+EjHC2trNud2G1kEhXh5PzNPtjIZ1t6WZ2gke+SJ3i\nFrLL7Exh0rDYAhj29hM96GFyTy2HyyLc9ekQuqYwHNXgGFkVfSpDaEDMF2DTcBexUBfe3wD/CU+d\nhmsaILgZxyDdAAAgAElEQVTSRsTrDAXhtiYAbf4eJs/GkJqDmMvbGcd9rnWckQjt5h7e9Z9lHB2a\nou8bpwiHwC5Bg32K+m/0M0ADY5KbOvMQG6R+7M+HkAegfHCaugPHcR/oYuNrPob+dw265zVevvsq\nhq3NxKxG4RdLBaSYEQ/7+YR8zUnVWT6FyGIqTCOKIlbBVD+uhd7bsYrbsk5JibhrhZMuMrOvlFZy\njzrJdn1zk33mQ+J8S70+jtdQRvcHPkDVJ1200UkZ3Rynk/ZQF47OMNn6KweBwb1wuQrvC4M3AT/+\ni3RZkd+cWcY+XSCYgMYIhF/TqJzsx/ulfjb8jzO03N5Hn72NieRT3UiEzYnTuLp6uf39ce47Az3d\nsFmGq4b8OJ89wU7lBH4VopvAboBY8hX9BQw/5mM0pFC1ywweHd/43D08PXgDoUq7uC6izHanyAhZ\nmC9YRpKEEbMmafsptgBHKVTj5iIW8m0UVi3eBuRDxCVE6Zr2fGxQlmS7PjMLiv0kTjozFqIhoek1\ntmtdoC2z+nc3aA8J3/BFwJusfn29YiEAHJ6E+N+K4xADxu4e5uKoRtsfiUqPAI64j6tHnuFfL4OP\nRyGe9Gy8noDoE3DjAZiugu8fF///3C54qAv6Z0Qi556PwqV/V8v9W27mt9rNPDF8MxwzwnFp/lIm\nRsTJSbVBysSsF0I+uXZlZgtNEQv5QtgR+bolFsSsF/VSVkqqxddqxuNWkLcqC37saTGPqfRf+jRj\nAxEcM6LkR7bu7cEJeOQl6JKEwfd2FfEUUdIinqLu7Bh1A+BsmMI2FMT54wG+/88QC8ADiDDHFCcS\n0O0BbTL9vx8cT0cJvqcB9IfgyF3DvOcP9vPbP7y58Du1KqSaUHjzvuR1KOTzjadLnMNiEBOaufrD\nN5KODsmIQFgVKhHRpNlclTaEXzST5tis+uIaEvHkwjbIQ7z3jijPflfDO7W8EMBxFbyRYuqZvrZo\nnF98UHlSpaZ6Ate1UyhTKr6ZKIFkeZq5BnQciKvMmvsLxMSld8kHofFj4n+dDyXY/9g41/7R80Tc\nRvbK18+aUD2PVEOwbs4/wYa16iKUiY5CtUleZ0LuILuyeG9DUsk9irz8LE096cNqZ20CgdyIEUC2\neV0Ks8MRm2PgVEHRcFqmZjV2kNAwaGH2P6cx7lu+IMdYWe+EtwMHT4P8vTjbD8bxuuDJJ5e/jHHA\nNApRi5OBq2tJOBLs/GUvT93zEjP//hE0aQnjREJMxM/3MUla3WJwC1KPeILl1yovQiFXEEH0823a\nYv293sbYjMKdslwBtyCsYIWFu8+vBqlkn5V4gxzqua71Rn0Eoz5tkulIUCuNYG9XGTkC4bU0yi5Q\nRoPw2mmYGgPscCaHql9+YPidTvq2XsQb5bvwXlVJvCzCm90OuqbaiQ8qK3uiKjLYjckuQmuFjULk\nvRShkMuIu3ruY9VOYeLc1imyBOak+WpZhivFSTo70szSZWwKTSXZdf/JJNXCDZKXS3zeUYSRCGVM\nowGqrFB3TwVXjHrwvxynZwrG3u7O7jwzqsGMFyw5GpvbPw7h36ulp72VM7TQWdbOmStbmNxawWS3\nG8bklSVppu6ZNRVyEBdvkMWLqi+PIhTyhXAg7vq3OYos/H1y0rrIlnLSz8jVKwGxOBUIn/hyyuSY\nERZ8OcICr1ChPn7uuW8xBjHqIhiJUIEXN+OYCdEptcFuuOJTM5gsccIvwNhiDRBK5ESA3JqdVQDv\nuAn8G6KMqQEkWWMq4WRqqhxPbzUM6rLPp3Ei2s8VrT+snKIR8ubmZhwOB4qioNfrOXDgAF6vl49+\n9KOcPXuW5uZmHnjgAZzO5YzZFeZv05ZKH3ybo8jpycysPk9atBspnjliGXGam1i+P96JuOt1GpSr\n0Ji+W02GEG77GE7jFBV4KWcSA1FcTDBCDW4miNykYLdCTRRqn4Ep31o05ioxl3Jg8jdgindT0aAR\n33gRQw11TExUwcA8MmVChBzOZ1w3IHRyISE3JlP213REZkL4NvMTEpmz91+SJF544QXeeOMNDhw4\nAMDXvvY1brjhBjo6Orjuuuv42te+tsylmhABxHPdBDW8rWuNK7KYqLFmKeKpiUAHIs56E8Ul4qmk\n3OWKuB5heihJEW9K36kGXZRGZx/lxkkqmaCcSWRU9MSQ0DASZQP9GKNRKIcrd8N7WktjvGKhG3jo\nl/CzT8O+35tg/MFJ4olFLtpqcjt5sgTlFjGiXVNciLnA/LAi14o2pyfeo48+yosvvgjAXXfdxTXX\nXJODmM9FZg0LC68tqVl6h2npRsiZh6mSfF4j+WO+9mzZIiPcMOWAK22Jy5KKjEpbZSebldNYY0ES\nskxCUbDhp44hDERp07rQx2OiG88JiB+D4THoz9e+lcgZBVCMoMZANSqM/+0H6fz932V0qrqQ5UmK\nAInFe9ZlT85CLkkS119/PYqicM899/CZz3yG0dFRqqtFW+3q6mpGR/NR/q2Vt63dVGlNFsjP4kG2\nkbRXqlife05E7ZRcaOK8gCWTPszGym7a6OZ6+WluDj9B9c+8HN14Ea9ds5sJXJgJs0nr4KbAU5gf\nVvnR/0rwbiN0TcDrJR/5mqMHdgJX/wUc+xkMfPkSznz8CsZjbphSlu40tK6pQCQJHV/xknIW8r17\n91JbW8v4+Dg33HADW7ZsmfW+JElICwrQCxm/NydfCyFTqCD6okSfTFyApcMJUwkQMsJfWKyHqQYx\nkkw1zV0uGxGWvARUJ3C0eKh0jWMmTLvcQTud3Bz+La///giX9SbY+V+PUhEbZ1RfhYSGdSiAenOM\nez0wPA6/liCqJpNSSqwpcURP5bqPNnL8U1dyvGoXfcYWtJgkfNiLnSN38ud8lRqbgT4Wnk+ssMBU\nCGKFKWKVHSmLfCHeQtRnWZqchby2VrQ7d7vd3HbbbRw4cIDq6mpGRkaoqalheHiYqqqqBb59zTz/\ns3F+sk8L2bfkXueY9OmaEEslLlhI96xcKAGiWKhF3HC5NHCSEZeELfl7TZympm52uo7QojtDFWOY\nJqfw3fkCro+PEjiSIJIA60iYxpFB3O5xpDDoTmioHeCJQlwr0iJ5bzNqZXhXI+g/r3Bmcx0HN13N\nUdMuTrOFiWxH4DoWnvsxsPgcjCIXyX1jQUwYdc7z3s7kK8VPF1xKTkIeDAZJJBLY7XYCgQBPPfUU\nX/7yl7n11lu59957+eIXv8i9997Lhz70oWUsVc/5fcBsFM8sXYGwJOuD6+Sl64Q7k6/1kBfVgLi6\n7CxfxE2kUwkcyZ81cZxNHq5xPc+lLz2M5bf9XNIc5PlXVKQn+3lmBLyDENVB+BfQ+1oM/+YYl18J\n9BSq5lyJXNggwzsboe3jEPmwQmddOxNU48PBDLZzpXBnkWq4PMz8FQ5zwWoELbLGVrmOfNzMOQn5\n6Ogot912GwDxeJw777yTG2+8kcsuu4w77riDH/7wh+fCD0vMIdXcIYVJn13qsBMxU29HuFGWO20w\nQqFKIafRIwQYhBWeS6q/ObmM8oz/VSfAncDiCKDpJYxlsFkbou4X08yIuXWOvyF+npBh5HUYPgih\nalAOQVsE9sUhVkoAKgqiGgRNoKuGSFRmSnOiSuIecOAjjg4vLjCr4E7AePJCKke0ecsklZ2cS4MP\no050xF5TIc8POQl5S0sLb7755nn/r6io4JlnnlnxRglyHY8XKYqc7NKDSKnPlnLEWaog3cjYlPx9\nOcxtzx4CZpa5jIUwIwZPBoQrJRdSmcsWZos4gDsORqHCZ2miztRGIwc5Mc9otDPTpzoMr4zApAIH\n1v+9esEwqsGBcdCeAHVaxfk3kzilaTYwgIzo/QmAWQNXhpCDMGgSpK1yC+LeWNedmhTERNIYuUaw\nFElmp5Hz0w2bWPf1yPVKOnVer2Qv4DLp0VYD4kK1g6E1glylEp7O6EDsJ7vEhvo5f08x++LXEP6H\nbJY1t7BWGemJp+VgZ3bI5NwRpoSooTJnwJI4FmTqqUnCWfTcntJg71p1+SqxIKMe+M0TYD6c4MOf\n7cShBDhz9DhS+xWM1GXkjCiauAZ8yYsgZcB4yI+LRa8IizyxljPfesSMvod1LuRO0kqzHmbwFkGn\npDfdZlw6/jsTBfHs0gNtyf+ZEGK+DRx7JjGVBxg72UA0roeEJArtB1h+llrK355CQ8y3ZCN6zeT2\njFVID7Kk5HIWcr/IgEWF1uh5Ql6/28wNf1OBv2eGH38piksCr1Z4z1GJ/CNFElgeHaBdHuD4V2HL\n//QzfUc9p+Ut6JUYMbMeWmLQmfSbh+W0mE8gsjdTCXC51Bm2GkBVIZAvx/tKsCAss+WLeZEIeSZ6\n4GLWjZBLpOO8JQlcluzivuciIyzvzMAdHSKMPhWNSBx3zSi2cj9nvC0kVB3IBjgtkZhR0OJS7rkF\nErkl6iyGwuzTWEZ2iUoSQsS3pG8uRU4gSWLnjrVt5edtt7H5yMtU/HMHvyvDDybAV1LydUdwGv7j\n0+m/P+g5StT/BicdW9FkGJxsIKFXYFvyWugwQFCCWkncMyOIwXwr54djp8Jd10WYqQRsB46Qi8+z\nCIV8nWHSgyPD1ZHr88fN7EYJEqLbXcY0QRtdfICHUIwqz9dcSw8bhU/6Oujf30bgsB0mKR5ayK1A\nl12F9tkWUl35IDZT+gIfw428fQ9/cLwDywmQPgrkI/+sxJqS+Avh5Sj/00k2mTqQKuDsRIZ1k7ou\n+vSQUIRQL5Seu4G02F/glIQ8F8rMogIhCOt7JYOHRoSlOtd6hfNcCocCl+GbtnBN2Qt8QvsZl0Ve\n53nztfTSzH2tnyAQtYAnuV1+8p9/LiOs9myTevQs/9iUJ6Axft73JGm270hDIqyYOVvRTMvOs2Ao\nhaRcCCj/BNWfmKCSCXw4MBuCtNWcP6u9/Vv/xqjSzP7rPg/9C8iYxLoZ2K+UIhNyM/MXzVpjZAmc\nGTNrOjn3VmqZNJGOSslEjxgqzvEfh1UTQ2odx9hOOVPcqv0apzbFtFRGTK+Hpriw4D2K8H8XomCk\nkcKdHlcCauOgn1+UK5k419zXTIhqxojJejQD3PFpeOt7cHKoFDO+Xnnf1WC7zI2+PEYLPSSQSUgK\nw8rsUKgqxvhAqJsp5yTeppOcPrtDGC2bgC6W70qxGMQcUbAY/ORtQA/LvYqLQMhdpEMeFipjuwYY\ndSLTEoTVbcglKHoeFNJ+cAfzn4FUhcB58EXL6PW3sMV2ilG1iov/8BAv/ON7hB/RpEFN0pqdUHKv\nO14TB4uG3ezjuujzXOI/wndr72FsOiPmcVgHoTwqemUCquNiH+ZQXTaKUR/BQBQ9MawE2MAArVo3\ntcERDGc16uthv6nUV3M9c7IHdntnaFAH0Clx2uhimDpe4wpGM6qf7uQtLr5lmMqDx/G8/DD/1LJV\n1Cu3kpuRkUt7xIJhJRdZLgIht1A0nQ4shvQJ1Sv5E+8UqTjrxbryGFk0FjsW1xOL6LAEx3n1K0Ec\nP53C8YURDK6kNWHWoCohfkaB0WWe4uq4SMIwalTYx9n6xj5ufPnn6P4yyL+Uf44JvxtNS7qTohJM\ny+nQsFxJibh5fkvcZprBLY8nrfBRqvcdxv/Q63jKJrn8Kj+KB176NZz15C/pr8Tq0zMIl7wRovHi\nQdyyh7H94D1oZmNFF63//cpzn7t67342DvdRVjPNxg3dUKmKJLLhtdv2taYIhDyFidyCkVeALKWt\nbpgt5PnEQjpVfald1LNo1qZRH8Fl9uAa9OD61RSeu53Yy/1UmD2MJ9xE4kYR8WFRhdCq0uyEivlQ\ngMpk3GF1AgwaVcYxGgwDKOYY0ZkZtv3kAS7/gz0c1K7AG3CRSCXtmGTQdODP8bhVLCzikqThtEwi\nywnqGaSeQdz73kD/z/sYe3AQuRwSx0TY4aGnwZ+fGv0l1hD5VfDGgwwHgoztg8EDYG0c572/P8VE\njROnx490fxdhm48zt29nb+uVomhWBSsTcn0yYS9SDEkHFYiOGdn3WioiIbcgCk6vAoqc7ji/nHZp\nuZBKOc8mpX6JGipGfYQq6ygbrT1s1I9xQ7vM/q+30m8xUM8Ak5QzHRHB4fGEjiAWaIgJpUvpZEAW\nAm/SRAo0iIfHhvQFbDEEuU56jsu6D+GY6GOy3sjkW1FapDNMOcrp0CSmguXEVR2UqSDFRfx8QsrO\nOpeBsmSsYH06a3PWRyQVu9lHddkodslP64uHeEfgLaZ+2kHvg0MAhKbgrYfF56PFcP+VWDH9R8D7\nGnRkVC1U/FHq//MtKj9YQ13XBA/ujRH8lInjO65kX2IPFmOQ4EqzwA06ULUiEfJqRAjiuhTyAqOT\n0/HdJn327dJWgpF0CddssLHos6zKNMYV6gGuGt5Pq9rJlF8i4jejmSTc8gQbrT1MWsuJoScYtSR9\n2hpsgZTzMHrWQmJaLyzhKiGmkqRh0odJqX1DWT83jT/DjQ8+zNkHgoxcqWfXd9t4Die1DBMss6Ci\nEIqKsMt4hULMYYCwBL1ZFDnTa9C6cENFRU5gNc1QXz6IiTAtnGHbQy/R/tJROgfSweLTGvymaPsy\nlsiFV+cJFYxMwt6/07h0cBi/B2JDEByzYjkyytayfejLVI5x6epvbBGxxkKuI9kfhIJVOUy5Shzm\n/Pu8F0OPiAufWzdkBdQxyAc6HueyI4c5eNUmQoMnuPKZl+m6tQWDPUotwygkmKASh8GHxR1MbopQ\nO2dwmjPNLYzFq1AzYgj1Sowmdy8AOuLUM8io3cWvgg3UNnWw58+dPGW8iFa66aQdPTFqyobRkg+H\n6WAZ474qsIJ2UXTxFl2LIMsqipTAYgxSVz6IhEYTZ2mml4pvVfDmPQ76ThRToHyJ1SAKHFPh2L+J\n8vseYNsDHuy/eYot752CL7iEkEuIUW3aJlnH6JKv7EYIayzkTQifgxsRb5dnZEl02ckl0zJXUqtq\nIX+lZpPL7KaNf9/9Gep3D9IwOcidf/EbXvy0xu7DrxDbqmccN2WIxsMz2OhnAwoJ2ulERuV/PvkV\nHtt5Ew+33opnnmGChEYjfZgIs+t7xzDd14H3IjAlgrTGO0VUl05DQqODTQSwoiFRZpnGaRFj4biq\no3ukDVU738WiZYQUSPPcaRVWD27H+LyfGaSe667qpvbwJK8cAk27AO7VEsumUxLnnl2g/dluTuz4\nEx7tv01cDDJwEXCMCyB8qQmxQ31ZffrCdK0YlHTc92qKuILIxpRZ/pGdm9mZyQbACd6ZCjqn29HK\nJMrKfDx2543s+V/P4jx+Fm99D3bHDA587OIIjfTRRRsDNKAQ51pewPXeYapMw7TTgUwb4xkzrxIa\nG+nBQJQ6Bpn5YyPTUy4mvuHhueuCnNIdR7V303T4JjbYBjARRqRtVKIiU5WsL6rICW6peYxerYUe\nNqYr2QH+sJ2RqVr0Soxm95nzdlOW0wHABqI004ucERQ8cHs1m2r81P+rlwO/Eg17S7y9uHMz7BuG\n8GOw/aWDXPvu53n0tjvyczEY9VAuweT6mzW/sITcrBeRJ7KUn4Sd5WBEDCoM5BbLKrNwASmdeF/V\nZKaC5XRpbVRPjnDZH7yG/ZcJDPdpvPPQQU6c1qMNKjRWqDj2RNh48REe/FczVz6+gR2xE7zwZ0Hs\nnzzK7hsclDHNFE7GSXdxMhClngHszDBurWT7H7l4V5OHiR9rHHkljsEf4Oa3nuXhf5LY8Xd+Jq/Y\nyhROfDiIo2cDfVR7zuK95VnaH/kkuto4vTQTSIaXOsw+zIYQEho6ZeEho5kQdQyhmzOsTJgVdBtl\nWnZB8E3oPpvDcS6xLpEV+OT/C4mr6wj95RSP7bmeb958O6/7rgSvlK0HYnEy6yatMy4cIbcYxCub\nJg15XzfCas6irOqyqZ+93ISqMBUq5y39xfzor+6i/p1DfOjwExz+hZ+y98CMBG88DvYTMNkS4A6X\nhPu30yj3B5l8XGX7jX00JHQ0K2eJo2OMKo6z7dzyrQRxM041Y1TXBKl8N9j64b2vwgthjWe/MMXk\nm3DdO/cxXh/mRP1mXHhpVPtoPX2IfX8+AwdH2RPdyww1BLEwQSV+7ChyAkVevLKVlQBVjGEkct57\nVUe89P/MR+/jYFmoF2OJCxJNhX0/h8Atl/HKVy/lDdvV9CkX4w/lcRJqHVMEQl7BiuLHrUZQJBE+\ntBYibkNEphQqIdXGeWcpoSqM6at54uZbaKcT360V+OuG2HTxBI7pcRxbx5Be9NHxGujaNDr+9wyx\nQ2AOQfXpMO63zpDoHMbrgyvvchHRG9GACSpxMkUzvVQziiEppmZJtOdSo9C5X2xDz889bNx5hCtq\nfPgVO86jg+j+oZu6F2HTR0BnHaOWYXw40JCwMUMIM1OzaufO3dUZXHgwEwKEu6eKMaoZZVvsBJ5f\nD3HqviixAbCvT8OpRJY0yrClCfTvgnidwmBjFS9wDYPl13G0/koGAg1E+qwQLJaMzEJQjugAM77U\nB4tByO3kPCtoMYgwwtV2o6SwIUI+F9am7HAw/yFws+AZSqDgwYWEhre9gpb2HnSMspnTlE3O4H3a\nR1kY+o+li79VAWOPg8sTItEdwq+H2ltU2uq6AGiPdeFkChcejESI7w3ieQIMJ+DoHEP69FGQfjjG\nVluAtkssqCeiTLwimhuHQ9AQnaRZ68UnOc6JuAcXOuIkUJhMhvM4mTrnQrESwJqMnVVIUIGXcrxI\n33uDydHTDDw0w2iyENhEaabzgiYB2Bug9RNmTu1oI1LfRje3M0g9o54qIhNmkYQWIl3xU0NUwFyJ\nm0WRhK4URd0VG0JcilrIy8ipO4EspZs12I1r59OyIkQ8HyO7suQrEym5/EXOkIbEOG4cyQI7fmx4\nqcBgMCPVwZW7INwHv/WKa3wMeOMQzPSKBkHT1TDySIitH+kETaPm4ChyTBV1ewxw5pdw+kcQtcHh\neTwiwWfBsClArT6AMgH2TXBqPzz/G7j2Rx7aPnSMeIueCWMFAWw48OHHzjRl5yJY3IyfC49MoSNO\nGVO4GcPODDVdpzn9kxlm3gblSEsIBlU4PQP1fiPd9U3sZw/D1BJP3RBTikg+CwHe5JdURHbnSurS\nK7IwDotCyLNnDYW8lmX7I1Ip9fZClPVbBhbE5q/UEl8ImfkLAEUkUahqTjp7CDMBrBiJ4MNO67WV\nXLJxGMMvw3T9AGGxJL8yimizBYAH9n4jyh3qEOVVwL/D4BhEbhT7GDkM4zNwzD//Zm6wgfEEeENg\nComii5sU0TfzuS/B74ycYvNnob7Nja8fps5aCFU1cnzrtnMPn/kwEaaGESwEqWGE6/8xzuEx6P4V\nTE8zj/e8xIXITDdM/CqK63YPw9RhIpwW8hKzWOOjYsh+E4pFxI2Iic18+cT1nB+tkuoMNBdvspB+\n82wLdog69MQwJktG+RU7Y0Y3png/v5inQXEmvh549a/h3e8DpuHRHhj9pnjvYgWcCgsOVfcOw97f\nwBYZ6mV4ds7nnvgW3NZ2ii2JUxz4AXA/3Pq7Lh7/mo9+GoihnxVbnkJGxUyIeobO/e+GG6DyGLx2\n+PxG6iUuTHp94DsLO9AwEmED/XTTil6JoxjjJHQyRVfyOu+k+j8ubr4UQUJQ1ZKfQgLMhsLXRVkK\nGSGw+YxOaWS2e0ZiycYNEhqSfH7RZZkEOuLo1DjxQyq+v1969QHgzQAkHhB/ZxrfR7Icop5SxWs+\ndA/Bwe/Bq8dg+3tFJx8DUTZyhm42EpuT0SuhYSF4TsRlVCQVYq9A79mSiL+dkBUw6IWIt9FJF+3o\niNNi66G/Jc64VgtjF7qF7kKI+dw+drNZH0fBZgTLGou4hMgaK3SJFgtLJrmajUE2uM5v/7OTo1wb\ne56akQn6OlQeyrIOSQR4I/l7vtsbhvZDNLXQfjC/GGLTJR3nLPEzNBPJmCux46c6KdcyKm10Ye2K\n8tCb0OWdu/QSFzKbd8AH7w7jD3XRaW4HoIlebn/uV3zX/sf8f/q713gLi4fiF3K7SbhU1nIEZUB0\nH8k12SdbHIjszoUscncC6uLCaJdmS24tQ2yKdtL37XFe/Y8EpmlYTj2pQvUtfiGS9sx0ngUeCvO+\n67uItkIzZ3nJeDXTcnqmV0ZFRsVAlCbtLO3hLn75+Sj9b0GiFKnytqLnFPzwrzUSXwkQMDzBu59v\nZ6SiicqzHi7ecoQ2ZxddoiJcfpFlqLSBNyAqIq4DilvIUyK+VuGFIMrQNlGYtmkbSHcCciKiVBaz\n+HXavG3Q6hjCjp9eXRObbhllj7kf42Go74DHX8n/Zi+HmYzNjURh6ojKzGcjPG4EiLD7W68S3lpH\nSJ59gKWkX9SoRXh3q8YTb8FwaFU3vcQaEwnD+ACAiqz42PHYSfqePMvxw358/z2MsTrDbxwFzpCf\nIaXE2uSkrIDiFfJiEHEroqzsAm3XVkQDIhcq5SI2QEZZkgWxGIK47B50xM/VN2mlCwtBfLKD8JYa\nbCYflcZpEWNYZHj98PQ+6FXhGh00nvYy1mRFts0eE+i0ONXhMQz9Gk0uMK+xZ63EGqNC/7eC+E4F\n4dNtDFzehi8z4iDB7AmetxlrKOTVzNviTZJEHKdFv7Z1D1LJPnPju/NFpog7WDoKxqlicc9Qbx+g\nzjiEQoJyJnExQQVeFBLY8SHLKiGLGezTWCW4XIHXE8VTKTCoQYcmjJ5pDTgILus05h0htIzmG4qW\noCw6je4kvHoIvKWOym9rNA1ePyQm5+PDYc6MVuApz7bQ/3rHxKL9H1lTIXdznpCnQgxta2x+pUS8\nEGUcZMREdGrkZkccisWEvEyF6jgbXV1cadqPk2kSyMxgo5IJJDRqGKEt0U3d/8/ee4dJctd3/q+q\n6hxmenLoyWl3Z7O0Wq3CriRYBSQhiSRZgAFJ3Pk4H9gGDnP2GQM/+yTzmPMdxuI4H2DZgCVACIko\nCaRdJRRW2pU2T84zPaFnpnOs+v3x7Z7uyWFndnpn+vU89exsd3VVdVX1uz7fz/cTfEPYWgKoxyDe\nDSyqp6gAACAASURBVKYMjc7SEElG1qch95yHinuh4MD0daQo4IJj78DEJra2NjtGBXYUwNuj0LQL\nzuvzGegrJaBlSKP2NccEac2n52KdhHye9vGKvP5x4hbEOVuLZB8lsd0qhElqTexrvt7TOaoQ/LIY\nRkeABl0b+3qfx9rmoqjJQJuzFgBrPEDe66cpVzqpCPjx/hzO/whcQ/BiJnSuWoAXjwPH4fZCKE2v\n1qACk4lltUNpslwS2K1Q7gRjhcLuPcWc7rBgvquYrnf/IaPDV0BbIrMzy3oJeQ2zgrElaW0aHy8H\nE8J3vVoNIdJREG6a2hn7cmigzOH4kIC6KOg09LoIZfZBZHOcrsc06r/6Cjv+WE/1J0Qd14rJPn7y\n+ShdeWAsh9PH4MW+NfgOa8jkm+CtBPs+Ul1eTkP8OBugSUCWlVCSD++6VUfkP5TwZPMtuKjnsdj1\ntLi2EvDliDmgEYR/fC3uEZ0C0Xiik0VmkxmTnRJg0kHuEmb71godQmTns45XipxYcoG6xH50mghn\n1AOVUXDMbXIqchwJjTLHEHXGDiQ0xsyl2AJ6nv5elPtMnQT9oLWD1gnPvwLPr/LhXyxePA7E4V0T\nIFUDYdDOQOCXoC69B22WDUSXCx49n4u5+V28yEHOso1WdxOhthyYSDP6/KxNl5F8C4z5hZhnOJkh\n5GYD5KyzS2UbK6rhtSilpOYpTMB2DXaGUz7yeXzYkqRRV9yBIseQpJRFYLu7gn137ueQ62Uij8M/\n/n8QCYG6AdwPkTMQ1sB0DcLCOsvaBbhnyXi2XAsH/xZ+TZR62mihab0PKWPJDCFfT3QIEV/tjM1a\nUrXEJcAOxh1BKq9tB4MI2+gbqyQUnf4AMxmCVOQLv4hOmZ7Sk4cbixyk01xPrbOX6g/28Een4ZHH\nYXwDRHWciIHaCu/ygLEYJs7BD8Mwkfkj2yxrQOvvwfuVELufeIcbQ89yxHRD6s1BYHTdDi3jWCch\n38JUcHaypvh6YEII7oVa4lZEck86ZoRfvAARapirIm2JkGdx86f8L/pxsv03LTyy96O8tuWKqY8p\nkopemTsnU0FFQsVDDq36BjQHOF/tozGochq41D0QYeB0GDwDUDsK4SCMq9m5zs1KJAgDzwcJHOzA\nKEcpebIbQ9ceQj5JjNQ21WhtYY/FOgm5NbVreZ0mOS2Iycbl+sSdzBZ+PXMnDRUmFlPifbOGiswk\nObz30d/gqbZR5hygydAyVad7LgoYw0k/OUwmdhclTxsnXxrH8AmNfT+C/rPgv/R6xs4ioEFHFEaj\nIkQxK+Kbm5hHxdsapfIfnXjM+cQDeoil+SO9COt8w7OwRq6va8WkTzWJuJgk27MtFJ1iZ+4QxPRE\nnvkoRpzZHBIirkJhnFhcx5i3AId9gsgj/Vi+Wkiloxs3eRiIYEpMvcdRGKWQIkaQUamkl1KGMBLG\nRIhydZD6YAf5/V4UK7SGwLeBrJMoMLboWlk2C3GrCffdV9Pu2UpYNYlGEsnhZ5hNndGZZH2F3KgD\n/Xyt49eIhdqzpYv0fEK+EBLClZIU8iQWDRxxYqqOCX8ekh3arq7B8XQn1fYTxLfqGKCcQCJHP45C\nPm6u4A0CWLDhnWpGnMskeT3dtP5gks5B4G14pQP82W4LWTYQuZebMB4qpo8KhhxOjno/zGBnBbGw\nToQdZuPHp7F+Qm5QLr5LZWZ7NoXpVnk5KyuOpUM8IOTENtK/llkDi3AQKHIcoynMK1xNw18VE/jL\nMbaeOU95aZgWRyMDlBPEhIzKVvUcjb/4NY4aB8H6XKJWcakc2gT24X7anga1D853b4yIlSxZ0qm4\nUkf+Vyppsd/G6fg+Xho8CH1GGJcurCfnBmX9hNxmFGJ+sTAjRDYXYXUbEaJdcwHb1COiXSwI3/lc\n5MehUPg99EqUYoeLfpzIxLnqSyVsf+k8vS934W+MkVPhxWUqJiYrlKv9uP+mj4PXdxD6w2K8TRbQ\ngz3ko6R4lL1fgPD34X/2QTgr5Fk2GFKLD17zMPFuB13hWvDJYtJkgKw1PgcbP/wwGRBTjXCVJGud\nzCe8S0VB1EhZQoMjEPXDdUrKlBiijJeM15BX66PvS91oXjc3/td3iO800WeqQFI1tjwQJ/Qm5Lw+\nTIkCfitIY6CbgPg4TJ4FbQP5xrNkSXLqtzBeaKf9UCN9Q5Vwfo7ItjhZ6zxB5gn5Uos8LRZbLCHE\ndjupb1mJEPELRUJY94ttK+27WE1+KvJTXX1q6MRNHmfr6thdPknbP7h52gcf+OsQObQha6Cdhh88\nAdf1wJY+ePsNGB+DO+8E91vwrXcyp6phliyrgSQDsoSKhEfOpVerSL0582YfBvov4sFlMJkl5GZY\nUvJWBJH1Nx92RDo8CDHfgnClrJYnp5KlTYRWRqEwTq5lgpJc15yrlIWH8IYm6I2B8RVouxuOyPDJ\nZnj5GHi98LMjcMNrkB8Bew7wGzB5YRdwik0WTptlQ3P756H/3mt5rPQDvMpB+nqrhTWuAedZrAfx\npmV9hNxhmR6tkodIY5dZPLQPxFE3L/B+DuKBICME3Yp4cl9InLWU2FayqfVi87TVUchTybOPU2Af\nRZFny20pLux4ce5Uce2E10+CKwwe4P96wBsQcdVqGF6KiK+tReANL8RVEXWVdY9n2Ui89H0Y2FZG\n29176PU2Eo8rqbjxGBd/CJprBk8QIpltLq2PkOtkUe2wBGE9G1lSd5yp+O/5cADVKrsq3+K+/H/h\nzwL/AGN6cfGdzPanRVja0ExHqi3bUlw/VULE8x1j5FndU5maBiI46aeSXpz0U087Yb2RWI2d8kov\neSdhMHGjDs5I0/Qmb+AYTGT9glk2GHfUgvUuePKy2zh51c0MqWVEJ4zQt85Og6RWZTjrd5aKEZOF\nSwn3y0MIvom5k3iS2ZMWoEoj2qTgzTVTEh5guKMMrV8P1jkuRozpwjyMEPckFkRsebKOuDMG8uIm\nwcGa54kYDOiMMYy61FjQho/LPa8Q/8uXOfCXk5TmT2JsiSEfieJrubABQ5YslzLdXvDcdCvvXP9+\nWrT9jPgKISKJaJUYMER2+LkA6yfkDhYX8QJE1Eku8/fNzCcl5IBRH0KyqrQYGsnTu1HrZNzDZcSj\nc3xVHeKBUhwTgj7K9NbzJqZ3CSqOzelS0StRci2TSGjk4+b9tsfRyXF6qMKT9uSx4aNBaqfA8hbb\nfqphugqUE9B6BE53kEjAz5Jl83Hinss5vu0uzmj7GfSVExy3pUrVqmQLZC3C+gh5Hov7wpN+84XE\n3oFwtaRFJpn1IYosw5QzwE7pJP12J8cLHASjNohKIjnHnGZVKwhLWwJbvQ9FjlGWKN4wmKg/q2oy\n3uD0oYDZEMSQsLaN+jDFtmFy8FDKENs5g/psD/q63YzUNxLGiI4Y+bgptHk4+CUwfRjazkJgCE72\nQmfW2siyyTDnQ/V7dJzTbaX3K3/AUMEWRtzFBCIWYYm7L3LW9yXMgkJ+//3388tf/pLi4mJOnjwJ\ngNvt5p577qG7u5uamhp+9KMf4XCIEI4HH3yQ7373uyiKwje+8Q1uuummuTfsZH6fuIwQ5zpSYh9m\nustDRljozeq0b2BV/BTbhyhmmEp/B7VvvEDHwUP4au2MyGUEAjZihRLkx4mryrQSshZDgOIcFzl6\nD4dbnsIiBXmp8UY85BCN62lTG6cdZr7Njc0kijwoxMllkvpEdXsZFe83x8j7gy6M9WZCPjOO4CTF\nBS4KtVHkQSAOLz4CPdk6EVk2IYYihbL322j6306OGe/hba5iPJJPTM2sQLpLhQXP2n333cenP/1p\nPvaxj0299tBDD3HjjTfyhS98gb/7u7/joYce4qGHHuLMmTM89thjnDlzhv7+fg4fPkxLSwuyPIcv\nIplVqScVEhhFiHW5BoellO9ar8EYKBNxDHIEmThRvYFIjQGaouiNESRJRUecBmsr++IvsePcq9S3\nv8abf9DPLUe6qNjZw4k9+2ijgfGEryQcNuKaKEWfeEKU5Q+iU2I46af6R2+xhfMU/qWfV6SrCSpm\njIVhDIl1Y+iIYEBCE5UIGWerdo5SaRCDGiHH52VbgcqI10V0KICza5KStlF87zIyECrF+31gDKJZ\nKzzLJsPiAGO+gvXmfBwP7+YHXEcP1cTQMTRZQjBiWXwjWWaxoJAfPHiQrq6uaa899dRTHD16FICP\nf/zjXH/99Tz00EM8+eST3Hvvvej1empqamhoaOD111/nwIEDs7arr/EjFRhFGdmkD7oXaNXgUxE4\nZUyFGZVGoT5Ovm2c+pw28hinLd5Ah6sOQlBj78KoD1PKEA5tAvmHp5j4zz/htcTHX9jXyaFj0Nvk\nxIcBHQHiKJTgwmQJUY34fh2BeqLoKaefXXeNsS/WTe7Yi3QbSxmknJt5g1q6UIgzQBkd1KMQo44O\ndsZPc3nkGN2mahqC7Zh+rvLoS9D1vSDXvTpMQR3EjoGpO0zx6W6++Vi2PkqWzYUMKHq45iPQ+Llc\nXq/dxVEO0k0N6syJp2z94mWz7HGMy+WipKQEgJKSElwukegyMDAwTbQrKiro7587tu8T37ueAnlc\nWN1Jy1tDZLY8yvQMFxmQNCRJQ5ZUJDR2IaOq4uLLsgZoSMklMjs275UburHIf88WZJoSO0yuLyfu\nmHpkNCRkVF5Qo7wMqHIrVv6e+sTrvYl1NSRKUZDQCKDyBirHtRiq1M6zmgoxiCaawb78A3hNTnyn\np0VKfVbEs2w2GoCrPgBlH4LOslwGKAfE6LadeuLp2XoDOnBlXSzL4YLOliRJSAvEWM733ve3HEYy\nGcAO+h1XYGjcL4rDd2pwfzSVyQVQGYPcOLmWSaptXeQySXe8hp6xKgCc+X0YdWGKGCWHSXKeeJOS\n//7jafu7/N+dPF97F/04cVOAikwhIxgJ40S0VUsO70pwcUv/b9kVO8n5slp+YbiVEUqoohsnvSjE\ncVFCD1UoxKmihy1qC7siJ+kzlVMT7MH4W5Wnvg19nbDvTri8CjgJ7IJACzzyc1CzufVZNhHdwMiv\n4ZoqqKj1UlLlopUGFOLU0EU31amVi+Pi9z+02cX8ReClJa257DNVUlLC0NAQpaWlDA4OUlwsqkY5\nnU56e1O1RPr6+nA6565MFax+EHIKhZ88jMgzDyH85EdU2CenLHWzBqMaE74oI7pd6IkRUCwEyixQ\nF8FnrkGWVIyEqbO0s/s+jZwrvNT2vM2pP3Fx+Q/L6bn+GsatjQzQwDj5aEB/RMHtKcCcKKXmyB9H\nllUC9NH9zDtYcNJy81UMyTvxY+VFLsNMEAmNMEYCWJDQOE+AE9oIJ9UzOJQJWuN+9pUf54a3u3Hd\nb0X/fisMeyloHCdwg4FJbxEfqe6HF+E352EkW8ktyyYgDIQn4ZUfgvWVSZR3neaKrxiJYKCfCqT0\nlE29trQM7w3PwcSS5KF511y2kN9xxx088sgj/Pmf/zmPPPIId91119TrH/7wh/nsZz9Lf38/ra2t\n7N+/f+6N+BL/ztUw+LQsMmMqk0cnQUAiGjIymeyxJpNoIaMnqBdX3A/E9SaipSZiNzmwT9jIN/fy\n5q1X80zwPQydq2QymEs0T4FclWhcjz+c6vOWM+GhKGcYdNBx9S5GpFpelW9klEKiqoG2yYZph5lr\nmcRi9DNOHi6phEGlTEStKFBd1sNApI9oYwm+5kbOl+nJq58gr3wMm+Zj6wMS+jYNc/tyz36WLJc2\n7j5w90Uxdw9TKb3G+//CxU8N76ODOgrto4x6iwhGlpLmnSWdBYX83nvv5ejRo4yOjlJZWclXv/pV\nvvjFL3L33Xfzne98Zyr8EKC5uZm7776b5uZmdDodDz/88Pxul2FSNcFnEgNaEdkxM2LEp1ARQn5G\nEQk9iaf3JPl06zUcVaOUObYx/JHraaeet85dSaDfDmFJdPO1pTmpFaA0hieYgySpTCoOfrf1fWhA\nl6cWgLgqMxGYXiUrpuqmHgQGXQTNIrzuBYyhIhN5Xy6D26oYo5ZgnhlDXoQiRmnSWog3tKHXw2Xv\nhYYRaD0JvUMLXYksWTYWwd4oA98eJjc0jkOnY+vnc2hx7GIy4CC4pHodWdKRNE27qN5aSZKgcASa\nCxfumQlCyI0kGhfPs04JqQxQwFzuo2nnKfaa3uJV7Somg7mMvFJJzD/PWE0BnFHhyplkemanEVEa\nIElBfM7MTqMuTJ7NjYxGMS4+Zf4WRjlMO3W4yZ9az4qfPYFj1Hz3V+z2aBivA/k0nPoe/P51GMhO\ngmbZpKh/ex1v3n8/p+1XMuQvITxmET5yjyzCks/M8aFBFq+V1M+FN2ceD0A4Ewoc5TKfXK/fbIIf\n4SOfy+JOkrRSCxFZnMluPOkkq8NaASMEI2b8XhsFOjfeiJ2hbifaXOn5ICJJPMB4QuQHINH/WGBD\nWPwS4qEjA8rsExnGzNCoM7GJCn5WeRdhgxH0TGsmkYOHwnA/537bgO2bfooK/RiNcWpPhgn0Rpns\nT/WUzZJlM9HwtaPkXVGG8To9b9kuY0grJRyzCCGXEb8/L9kC/POwfkLej7gojsRRLCToo4klBxF7\nPpPOxL8FgE4i3zzBPs9xvu79AvQahCsmzOzY1BBiOn0+fIlFj8g07dAvqfrhr7X3Qk6corwRHNZx\ndLIQcyNhonl6Gn9Wxcu000AbRZePUOF1kdc9SX5WyLNsUq4qBfvrjxLTgXe7TMx6Of26KjBpEJKg\nFjhNtiPQPKyPkGtaqv/eAMLiTgr0QkfkQQyxkjHnc/G8zOs513Jv07XiSb4F8ZDoJTXJulyiQEti\nWzqEO2YhQW/XQz2MSEWomkShfRRZUgljpIM6FOIoieLK1ZEebL2TnBkUp8KEcOVbdBCJpkowG5K7\nlEGniFMYiWXv6ywbgx+cB/47GK2Psu1rMPbxSgYd5agycG4hK2+NuUTihNdHyMf8kJ8HhsTuxxKL\nBdi2hM9HECGL85FDqkHrCUS7t7kmVpdDskMJCOtgMf9+hx5qwC0XomkyJbmzZzMHKMeHlbY3oe1N\nqJdhtwLPyvBfdsGRE/BWVATxXKsTSbCqA3bUgS8IL5wUXy+zS95nybJ0bv889NwxisvaiksqYcBT\nub4HNB6AaOb/wjIj4j750AsgEmeWsv5CD0ov8A7CIt+GEGCJ6SUBVkJynz2Iwl8LbUsDenVocZiQ\nHKiaRJljcMYqEoOGUpoNQ1TIE0wcgKY/hUoJdDq45klofRIO74Tmy+HNt2FiEnbfDNFTcOpkVsSz\nbCx+8w2IffdlGnQnkG+7iRcf+gr9WpVIEmwCOpg+j5UFyBQhT6KxOj35kj5xCeESARGXLiEmR/MQ\n0S4rJb3Q/UINmGMSxKVUGVxNojKvB4A+KtjLCWp6++kd9sENcN1nIHiVkSFDKUMq1Pf08b5wnNwD\noDsATR8BbUzsP7cC7j4Lj7Zki29l2Tj4x4HxIHqCFI52U6nvpd+QsMqn/IsJChC/wQuNStkAZJaQ\nrzYaKb94H+Ki2xE+bz/C3TJ38unihIGRxPZNLPpgmFk218EkV0TeoKqzH+0WKwM1pZxoLmXYVkhI\nMVIcc3HyqUHevTeO5+oC/NtMqAawBQPEXOPoY1DRDHIb2QJDWTYcTVeD9YM+3Lp2OgtrcdU6oWdG\nCLGBC3eZbhDWT8gDUdELT3+Risd7EVa0hvBvhxHRKOkiWMzyboxQYjEgfBwSIvY93WqYlMGkQF6c\nWFyH21tAsf0l6uhg9OEeuqv1BA5XM1zUQC8V+LGKeuayjfL3t3J2h5VgYz5hi7iJHdYJ3F6Nvu9N\nEGuFWFbEs2xAjDstON5lplAepdbaiSu/FHp1UCyJUXXWvTKN9RPyUBSMuosn5JDqpSaRssxdae9r\npITcwvRkoIWIILJVJYRfvpBUnXWfDBZNCLmqw+e3cYX9DezfaUXf7+P8dVvpL9pLHxX4Ev3sVGRa\n5Sau+pRKGzmYCGJIZCrlypOUmqPESiZQFNivgxNnIbgaLqksWTKEwfMw/i9uLLzAVns36gcVjpdc\nQVQywbiUFfIZrK9rJRoXQq6bI11yrZhECLbE7D6gw2l/O5g7ti8HmO/ZoyH8dTLC2rcgrPWQaCKr\n5ETJMU2Sj5uKR05j/5qdZ3fv4U0ux00+oURfuxg6XJTgogSFOJX0UsgIBiL4sGJt8nP917pxdPhQ\nngHtn+FkF/ij8xxXliyXGKNHAnCkBSMt1NXauOKTYfoaKnBNVhG36RJVuNb7KDOH9RXyQARkCWwX\n2dHlQbhUqpk/9X8iscykhtl9RJUZ20mmDZcgrHOPDLIOfV6AYscwHdSx5yNn6ZPLaAs0MmwrxkPO\ntHR+SPUMjSaKyRQxTAALvXIFHbYaGmrbsRcGuboEBgY2jpCnn04/2WS+zY4+FKLh9y/QsO8cE0ox\ngWKbGAUn3aNmUuHGm5R1EvIIQkllkdmiakLQLyY+RChTIwtnlc6ka47XbIjMz3R0CLeNhhDzCBAF\nHTGaOcszf3Q9+7/1Nj5DDq07U/1AJbRpaf0ALkqQUdETpYhhwhjpl5xYCFD5lS5e61MZ3yBxiHqg\nWIZGHUSj8KqWDbHczMhGCdkAJz/aRdHr/egNken1jnKZHp22YVnYnFknIT8F7AMc4I8kJiBnmrkX\ngSBwFtjJnMWwlowPEbeeThPCxz6cWHJk0BuYLM/lz6R/AKD7g9WiR2FarpDZEKC6SNQNkOa5eBYC\nNKqt1Ec6MNwB5/8NvHONHi4xJGCLArfmg8kJ7rNwLJwV8s2KJEH+lWauO+KkOtDNE5Y6wg1yIt72\nIrpjM4KFhxwbO/xwKUQRSUjbWJ5lvhgdpKJYSgAvhE+YaYvvgOYwSCIkcSahqJm2oUYkNGqLO1Dk\nlIy5yceKn4Z4O7Uj3Ug/hW9+Hybmqut+CbJXgUONYL4SpJgI97/vPDwagomsf2XTseVauPp/mHlB\n2sUvLLcRkNL8l2UILR+e79Obi8wQ8lCijKx9HaxyEGLeiki9X60m3knvyBCi/IAdtGqJ2HE9nFOE\nK0YHVMQgJxVDqGkSsbi4LN2j1Ugg0vuNIprF9/MhTn/5GJ7cOHe/D+7+I9C64BfPQf8lflNbmyDn\nepDqgDDIKlhbN5/tlUXQfQJ8D4N2jZlT0g5i6XKlMH/QwSYkM4Rc1SCYmKlbLzEPIlLvK5gdzXIh\nRNMWEBmmfllM1DgBSTe3VSFphGskUDSGPSUY7BEcpglyRwexDfvYc4uejttFFlKZewhDe4wDzVBb\nDK2n4Nhc9ZszmCuaYfftIF+JmEwOgaSB2QvS82TLQm5CiqxwMNdLvO1VJhoc2PARz1XorGogqNnB\nlXjEW4AqxO93NZkMXjKJGusk5P3AjukvqRpE1tkb6kOUICxl8aJYyyWKsMyTNV+8iFBFTZ4/ckYB\nZAgWSfRLFWznNFVXxbF+bS/D+wy01YoORjmVHux/2cIWUz/VsSB5v4IiN4yNwOuXiIO5ZB8UXo2Y\nGAbxMN0OOgmkV8kK+SbEF4SWNyLo/6qL9+79NU/05nDg3c/xqyvu5oj9MKP2EhFPPs7Scz6WQzgm\ngjEuAdZJyMcQYRwzUFURkmhZx7KVHlKZmast5jFEWr+CyCKdRPgNipjbpTOaGDvGIaDZ6DVU0b51\nP/at2ziHwgQO8nAj6TSq73QwHjuBcXIYmxzkiqifsVcgeg5OxDM3hG/vFrCVQckNENs+481i0A2T\nHUJvUtyT4D4G5hMxTC+1ow1AyTvH2X2lmzMHChktuRFCshDyTc46ulYmEGqW5kqJa+ANi+lqk078\nux7MzABdTeKIAUkBQqCSN+F8Yg4wJpTsvGkrQZOZMsMACnFy8CChkssk3VRj0QUIFxio2DtEjseP\nxQ2O8/Nsc52RgCYZDt0CukNWgpfrGC1Nva9oKvawF10+bK2Ht73gDazb4WZZR4IxeK5P/GTeegH0\nuhZKr+ugoyBAqHs1/aCZSgRwL7jGOgp5H8KHMcMnrmnCN4V5/cVcRbhBrGuwfR8iBjYp5slM0PkY\nU/AZcum2VBHMMWHXe5FR8dFAPe3kMMkEeYTVIQzhCATBo8HzGdZ5wggUyDCkQpMCxqth4HA+npzp\nP0i9FkVR4+i3BTl8SGOgOyvkmxlJgr318EYf5NxQRO3uAOcDo/Sv6oRWphJg4VZmmRwQMBlcfx+V\nF3H+1iIVuAPxsEj6sGNMb/w8D/6QjeHJEqLo6aaabqppo4EAFkxaCNvQMMoxN553YHylHZHWkEIz\n3FEPeU3wSi30NVgIWE2EMU5bApKFHnMl4ToJdxCiGfZAynJxkWRo/pwB+wEbBd88T+Wzp3FMDZ0R\nSrZOcRKZQGZErczHRBByzWBaWq/MNSGAaEyxndX31XYA9YiAaTdiBFDF/I9XjTlL1vZQRSU9NEfO\nMPGtTn72D2CPwdkMqEWhI9WZT5bB2AyF/1fmga0inPI50zVMyI6p9SU0JDSMhKmnHRWZHx1RGR5d\nr2+QZb2QFdAZAEUibtBx5gMN5PynevZ9600G62VULe2HYkTkgrzD6mSQXSIt3pJktpADeELCKl/P\nCdAIImloB2t7xiYRjaTr53nfpQNVEmUFZjCAk1ZjI4e+0MdV94zR/Tic/fLaHepSebcOXJqYcN1S\nDTe938zp3VVoMrTRQKvUMFUsDCAHDyWJMgStUiOY4cPfbOenn4nQdXwdv0iWi86WnXDHn8HETWae\nMd/I0Zw6oujZXXGWU/JOWnxNa7PjuAqj/ksmYgUy2bWSRNPAFxap/OtJDDjH2pbP1BBhdu0LvK9C\nMGKmd2x6L0MVmZPs4DnbjRzbciXKjlJu18+9mZkYgF2JZb5IyJVi2w+mBvG3UgnxG8y0KI20SE20\nSg0EMaMiTy0echhKdOmIoqddqie6w8Cdu2HrhbTpy3LJ0XEOHn/cRG9xPT25jZyXtvIc7+IzN3yd\nXzbeQlxbw3CmS0jEYd2FvB8Rj7cIqibCEv3r7CsIISzm1YxpHmB6+FSyTd0CqJpMNDZ7hBLChB/y\nlwAAIABJREFUjEey49NbUfbqKfmTxXdvBrZb4NCtcGg72NL8jFtluHwJv5U6Ga6eZ6Si3gG7vwV7\n7gd6QP2dhB8r7dQRxIw2w2emIuPDNiXmYYyoRonca6C2YuHOelk2FuEQuN0yAdlCG+Kh78POW+pe\nBlxVqfDcDc0Ei010wrq7VgIseSYxroquQgDWdezv5Ec8f8pYndDEILND6mNAL1DOdL+8Q4XC2Q7A\nYoYpYwAzIqzDGvdTILsxlsIt1fB0z/wGRm4lXHk/FNYAj8DhKASvAsygvgY9b89/6JcVQdV2yKsE\newgKO8FzAo7GxODh4McgeHMd7Tvy0QrAe9jEr6orOcGuqSYac2EgjIaEhEZx4kH/8nFoG9r01Uo3\nFU4b7CuH8cTD3YWY5A9FTUR9Bgiv18TZxSRCql/l/KyjkI8uf/dJMU+KktWwPuGJyaShZNu41UZF\nTH6WzXjdrIJt9mynHS8OJjAQwYafyOtjnP5XP8VnwRJmWjZQAVDlEAE53gK47ON61E84GNSggDHq\nwyrsAYzQqYK1BfZY4MQcAyd/DHIuh8pbQD4tkneOJPa1/08g96O1tDTtZVhXSGCXGfeuAsZx4KaA\nsYRtnY8b3YwOHnqiSGhoSMhovPawQusvYXIJg7csG4e8Gqi5RU8nDooYoY0G4tnssDlZZyHPXf7H\n4qrwmYMQcbP+4tcyh9VNGvIhYtVnGqluIJ95r5KERi6TKIlpegsB8nFjHvIx/AK0n5k+gZ8H7NwK\nO/dCSz/02aDh42aOV4rZU9OHK8nRvOQyiZ4o8q0htkaDGPsg+Ds4P+MZEtoJ44fMWC8zwVCU4VYf\ngwpcfgC2fy6XV5y7OSU348GOHytjFOLHShyF0alcfCHcACZCmAkSRc8YhcioqMjEvdVU3BhGORrE\n3brCc5zlkqJIgqoC0FXHKR4ZY1/RG7TRgCvZ5dyuQkASdYtySBlXhQhpWWnkSnrdp0uIDIhaCSSW\nFZQd9IbExVNk0S5Oucgu/2TbOJkLSxoaR8w4pgu5hvCf25nzKsmo5OChgTYa+jvRWvqorhvCGhjE\n3B+mogqea4PqcmiyQawV5DDUvBcsdxmRT+oxj8NwaT6tNAASg4ZSChmjjg4KGaXs1kFKG4NI34cD\nR+F8mguooha2f8ZB7NYq3tHlYG50odW2kj8BjdvBbS2iU66hjwrGySOCgQCWKUs8Sbqg5+ChkFFM\nhBLt7kqRUSn+8n4aIjEK/rqDc98JEh0Vd8vQpTUflWUZ5EpgHoLBp3zkvf0WZQ1DNJKH/tC1TBgc\nhItMRMNmIeRFCCGXEUXp0vMzlksszVBcd4IsdUIuA4Q8WXxkvpi7RfAkwkhsRhGieLGtcw/ipqll\nbRISQgiRTxtRypKKIz7ONSdfonpbDx998se8/P8mKXs/eNqg/YeQkwM12+CGHRL2242o/xzm31/X\nGN6lx7+/Av/VRZhQOE4R7WnnfijtlihiBAwQsYJbE0kZxXUw1gNXfMTG8OW7eVu3nSgGGve1Uv/N\nSVr/MMTpf/Zw5V/YGcwvZ5JcRilkcgmjLw85aEiU4MJABBWZfiowEeYtw2Ucel8Y62gX4adj5Lvh\niWwhrQ1Lmwpt5xE5HMRA6mFn7T/R9PNhLKYxjksHGDQ2ETLYWL8kk7VmFGHNLU4GCPkq4QuLGb3k\nROjFFHQ/0AZsZfXPaDeidnnCFy9JGnaTl8vH3uSLH/pbrE+EMf0ebtqv59SvZPK6ZPZu07DeEKd/\nv8J3HjRw/b/Usu3OFkwfCOGyO3HprqSNBibJZZDyqV3piFFFNy6KUYhRFBik8OQgg0fh51EwWuEj\n/2zk3z4HL994gMmaHUySywQOzilbqNuznbLftTGw9xleVA4xRgFDlOJN+J40TSKuKUho0xpmpOPF\nThyFSnqnXEZJBvYX0Vw0QVHtKOe+y/xhmlk2HLICH/oquO2vYfmPv8L57vdy/AP383bu5XjH8hff\nwAZn4wg5iFjzQAQMOshbrQ4RSySE6GC3kzWt1pdjnqQhr43CvFGee/kmDu/8Nca/j/HSzVcymFPE\nqFbA25KHOqmTVrme4IeqeFonoekU3vPYWZ417uYYV9BKIy5KpsL/JDQaaKOXKiropYRhOh8e480v\nQX7CNx5VdDx12U3kvqrjFeUaRihmhCLiKJTg4hQ7OF+0hbrORrr0TbTSiD/N5+QN2RkYd6JXotSX\ntM37HQNY6KaaOjpmvzkM/SfhnTneyrJxUWPwfz4Bn9zZi71b4wNHfsRff/Vxfnjw43zm3n9e78Nb\ndzaWkIPwLUdiMOITk6EFlosX2RIDTiNGerUsr0HFKOLYq+Z4rwdwQl6jm/rcNpz0Yff4uPnx33Fi\nIo5jRxUtefUMy0WMUMwkufwKIz1UockStXTSSxX/7aWvM7ytnJaaJkYoQk1LI9CQ6KSWKnoYpBzr\nt8MYvz+G7iBc+0ULRY01dMm1dNka6ZJrOcdWvNhRkdGQ8CSGDKqm8Ovx9xDVDMTQoUpycgeomoSm\nSURjetqGGlMj4oSvO9/mJt82Boj48XbqqaVz6hgrHnfR+u1xXn8FQln/+KYjHoMfntYIxaDxNpWW\n+/fy0u5rYDIK6C98hBaOJQr2XXqss5D3J/41IA6lenU2qyGiWwDcgVQbOf1FCF1KTgh2IyonLjUw\nJw7MVxgqBqhQJ3XysTP/xmVn3+b03irO/68AOx6Ep6oOcFLeiRc7wxQzSiFxFMIYkdBoTeT0f3H/\n39ASb6JnpHJa2yydEsOZ30cUPf04KWWIc7c1knOyg4LOTiSnBVdtEwPU0UYjLTRNuUAAvEE7bp+Y\nxNSQCMYWzg/VkIjGZ6edun35eIN2TIYgJbkuoqTWKWGId16YpOf3cbxZ3/impSgiZtWkbog+3ErF\noZ9w4/2FPKu/XYTttjFnq4MloWkZVGOlH3Atee11FvIQ4qxHgDWqURpN+Fm94ZTf3KgTYYtrSRAx\nT6EBjkXWXSLj5HG8dA9mQ5AdvI3ToHDufXtx2cvwYaMfJ27yCWMkFDUx5i2c9vlWqYngoJ2oxwC5\nKhSIcyNLKv3uiqn1InYDNn2Aa4vdjLwMx//FR83fdvGM4Ub8WAlixuUtIRwVs7uRmIFQ1CQSNPqX\ncEvpgKrZIV7RuJ5oXE8kbpjqVTpAORYC+P92jKbXPMT0Yn45y+ZBD9TIsP0eKJiA374BYzc4GH7X\nZfSXXo1bn/CRJ0tcXBrd2RYhxHLKrm4818p8RNLM3ZgKsbgox2ddw2JcfkTzZQ0RxL0YAUT/zuK5\n3x4Ml3HUeohAoZmizmH2F5ympVCPpgiR7wrWMhERQ4BozIA3ZBeW/qAulRQ0qUBIEktAPNhUHXjK\nUplNPVo1x6R96G+MoXMOYB46weTfD2L8izAd1OPyljLhz0tZ1T4ZxmWISeBewqhHQTTkBCiJg2G6\nFRSL65gIOJAkDdUu0yHVcX7LHgpKFeI/74YnRZNTuwSXJXb3Wmxty+BkuTjsLoSJMHR7U68Zc+Dq\nT4D+D4oo6RzH1B9DaYwyeVsdp+M30j1Qu27HmylkkJCHEIOmorXfVSwulplx52uRXJTMro0gTIuF\nJthDiCSgeYQ8GDEz6CunlUbOxaoo6lVxf7Od+Cf30m92MuArIxRJc2tEJRhRYGiOyxyQxQKg01JV\ndwrieEN2fml+D+cvb+Bgzm9415NvM6ApjGmFtAUaGPEVEVMT2/TJ4FJgfBluqziikmOS4jgYp4t5\nXFUY8xUgSyqqVab1g9cQoQJH2VuYva8RfG4Eqx2uulKsf+JFCGWV/JKnaQdIW6DUA+53oPU0aLl6\nQn9WQ2dlEe7qcZzP9VA17MfQepImZyungrsvfMfRuPCRZwRullvQKYOEPIDwRVwEIU8SV0VSURJN\nE4lFIP7VrZJP3ZdYjAjBXMjVEkMkNMzjWw9GzIz6ixgz53B6hw3rl0bw3WlipKAoJeJhSYh0WIKB\nJVzimAS9ifU0oCBOHxXodRGu6gWTW8b3N4d5JXgNg5NlaJoE3qQFLi9PxGfi0ok5jKLZYq5pEsOe\nYuxmL31yBX6s1N5qplwHjsJjFBd4MdzqR56EpiicfgMCWf/5JY16AGruN1EsGRh4FsaPGBgtruKp\nmmvFCuVw6EO/p+DMGSpfeY2W5qf5qfWDoubEhRCJZ1BGp4vlfqEMEPJwYlnHQlhJ0jO6TPrpbpfV\nmCgNA11AoqwrZmaHKoYQUSo7596ETo6h6cFXWM7BR/Ix3WPgJXM54WQ2UmQBK3wp9CU+Z1Nxq4X0\nN+7inaIgX/X+Nwbczunr+VcpkzZ5rMWz3SwAoaiJYbkEgxyhjwqkm67iypsUqtVWPIEOctpC3PYR\n8PRAe8f8c8ZZMpvifGCfgb7aUoZ1xQQbTNg/VcI7XEMfqbLN3pvtWKw+yn/nYsRbAnvkVNzEJiUD\nhHwYYZLVIGYpIogolnUmFBULCBdMflpcuiytPKQxWdccRDJrLrOLCWukXDEzdmM3eajK7cGuenEq\nA/zyl7cSlQxIo6qwkId0MKyIU7lSResQt8UkxfwrD/CvPCC6GUmJY1oLhhJNM8pjwtWTRr/bSVVh\nN3Gjkihza6eTGiRZQ2+N0tjYRuSoRmlM2DITa3SIWdaWG/aBwZnPeUWEt7aJAhT0UTFtvQgGdjz6\nGq/mb+ff33sPvKBL/WZWEnSiahlUfzzCSmZrM0DI0wkAbwP7yKi027gq4tKT5FlE5MuF0o7I2sxj\n+tdNdiTazbQrJKHhYIJdvMNV/J44Qtgq6cFCULhHkjWavZAWgr06KEAzS69iv9xLOKwI/3lNdM7P\nJhOYChgjgIU+nDRzFjT47tdhsneZ+8uSUTz2DNz14SHie7fTZqyfEvGZNev7cfILNZ9+TwMnu3eJ\nUN8QIodjJfgzoHHNFGdZStnamWSYkF8ipCcNmPSQcwFFVroRtceLIC1bfk6usL3ObblPYibIv8v3\n8nXz5whLRmLoGO0og5NKqknFWoRgxRH32VIFupbl10IbV4RVVbe4v9KkhqnzdWA6py2pcXWWzCf2\nORiOFDH6H8RcWTBips9dOWu97g88TNxtgN41DiO+RMgK+UpITxoIRVOx6hLCWl+O2yWeWEYQApZ0\nQ2uIgkH1TBXjGpCcPCsfJhQx0zrRRFg1QqsB2iEybhQDmrV2EC9n+91MnwOwM7vG+kxURIhkO1Cf\nUuehiTJKcocoNonQw0JGqT1/jJ98OM7746CNLeO4smQsyp8Ct8mMk0dnuJaBCSexeJpMdeohKBGL\nWMErLZwzM4DImN4EZKCQJ53IDaydQ3YVUTVQ04o7TaRZ61aDqPuyFKLAGCKRSI9Icg2SyhAFQpjo\nGallpK2MYMQiHgCnZBHlcqEWuIaYiF1K+c8qljaNMTOfIcr0YO8q5q5LEwc8ivDL10RBFklH8UTI\nY2N3O3ecfppo2yhvnoCfyRDYEEkgmw+THe78oiiK9ey3oKWhiTNlzXSGaun3OInEDCKMtjuhBV5Z\n3B+jiSWZSziXWy3MJTRS0xBpqSuLoc0QIfeQyoRJtse5RH+Z6bGoGqBL/F8nizK7C5FMclVIuS9K\nET7pEPiCOQQLrQRHbantT7CyCR4P02cFNYRbZimnXWb6nWNj4fj4JFFSDTlARBok/e15TK/pHke4\nWWTAGQN96kt6jk9y5n/24PeJ1dov0VslC0hmGcvHihnVFWOvUTi7bz/npK34YnZCUXMqhHYibWIm\nKeJJzYux8vDDYBTCKy1evtqMs9IhdYYIeRChKumZMIMIp3EGRLCslEgsVfdBJ4uZcUlaXNDjiOda\nkjEgF0KjFlHO1sjShDOdYaZb2z6mi+pycM/4vx9h/eiBwtmrL2k7UYQ/3UyqfZ6GmLwtjU0bnEXK\nTASrczA+vXiyvl0SzaHfzpTfahYACnKheTtwuY7zZQ20KQ2M3lNEDzVTBdgAYY2PpQ3b3EwX8Qsl\nHBPJgetKDKF3K7dIMkTI56IfoVaXsJCnE1NFvRdpRuiiQVm4s1FS0JM3rhchdsu1wgdYO/+5P7EY\nSY0k8lh6dAuIh8okwiqf2T5vQpmqC1OCi2rHEJXNOqo64PiMPp4VMuRIMKZCOB/2NMLOGESOQ2s8\nG2OeCeRLsLcErrkLAnfJ/EwWhd5aaWCMQsLJnJKwBJMzbqIRpot4mA1QfCeGSB5ZOQv+1O6//35K\nSkrYuTOVnfLlL3+ZiooK9u7dy969e/n1r3899d6DDz5IY2MjW7du5ZlnnrmgA9uwaJqIekkuwaiw\n3GOLPI3HEHVbPIm/O5e5XAwFC6ftbyJxrMv1UfoRxokXMWrQEMlHYwrhgJH8uBv9+SHOnLbivqkY\n88FCkKCqHkxmuMwKtzfBgWthzwfg0Ocg7164WQeGDIpo3czkSVChQTwOaq6GVfIjJSwTN/n4kvWf\n/bKoE5TEx2yj1Yv4XayEaBzUjeGXW9Aiv++++/j0pz/Nxz72sanXJEnis5/9LJ/97GenrXvmzBke\ne+wxzpw5Q39/P4cPH6alpQVZXqpZFmd2hmc48foG7pydzCY160V3o2QP0rlwJxYbIn8K1qa93GqQ\nbPxQCRSwvLFfABG1IiPmvM1An44xSvitdCPjt+dRd0c7XlzY/thL8Prf8r5PufnpN1SMGpg/Ac0f\nUgiVKvgDoHsVVl7bNMtq065CpB1u+h7o62Pk72jF3lCJSR/CSHhWZyggNRm/mgaJNyRS89cVldXw\nEy348zp48CBdXV2zXtfmyIJ68sknuffee9Hr9dTU1NDQ0MDrr7/OgQMHlngoHqAV2JH2WguwheU5\nXi9RglGx6JVUFul8YYw+RDciBXG65MSSiRZnL+JeLUYc33LcLSriFmhiSszb1K2M1hdSkjeEiRBN\nhS00nGogEPwW8k/dyEMakVKZIWchLkMxmMG4M0y1rQVDAIIh0Eki2CieKcl8m5BeFb7TAtwdR6GD\nw62glIl6Oy26bYzMVzkuHY35XYzqAu+BGBlnxPX3sfJMphQrKpbxj//4j+zevZsHHniAiQkR+jAw\nMEBFRSqVtqKigv7+TV4AYSVE4zDsE5mkixW5jyME/R1Wkgx28RhEHONK3YBtpIrBDemYbC2gdaiJ\ntjHR5KKNRp4x3cgNj+XR8B/hbNk2fmW4hd9wM0e4ji5nDaa34T99DUrK4c462LUJbINLAT2wB9j5\neBeXXfMDbvj541TTtbQPu5g77BDEiHB8nvcAxvyp/I8NwLInOz/1qU/xpS99CYC/+qu/4nOf+xzf\n+c535lxXmjcx5kja3zWk/ARz0UnKpNskJK0Fd0K9csxiUnQukvdiJykPVD6LJ95cTJJuyHHE91pu\n+WiVqXZ35IKmymiqTCBi4ZxrK53U0VNYxcsF12L7mI+QwUQAM2aClDDEOWULcplGzUe7uLO6h4K2\nMDUvguH38NpK/atZVoUYcAbo/rpKaDJC9DPHMHm2UPCBfbhyS8S90jlPPonG/IEei1rkF3DQq8Y4\nC/enewdRq2Nxli3kxcUpQf3kJz/Je9/7XgCcTie9vanHY19fH06nc9bnBdcvY48RNm2sQXIC1BsS\nhbpM+vk7G6W7gEcQk0Am5u4Bul7EEZOgnSxfzCOkIrQMMuh0aOUxwjETYUycHN+Fz2HDnBckj3Fy\n8BDBgAaUMsQZeStFjmGqr5bQFYDBA1Xt0DMkNptlfdAQwcfBZPRRX5CCB39JfaCY4Y8UMxrcyAZc\ncl5wPnYlliQ/mHfNZbtWBgdTt/0TTzwxFdFyxx138OijjxKJROjs7KS1tZX9+/cvc+thRCrjzMfl\nCJsm13YukkXv/ZGl1UyOIKYc3IgJoi4y51mYFPMulpZFms5UZ0BJxJen1VoPhC10T9YwGilkjAI8\n5BBDRwgzGhJ+rAxSRtiihzC83QpvDWS2R2ozUQXceDPc/jdwxed1lO5TMeoXmAQcY2HXyXwko8bW\nvTfnBCsPt5nNghb5vffey9GjRxkdHaWyspKvfOUrHDlyhBMnTiBJErW1tXz7298GoLm5mbvvvpvm\n5mZ0Oh0PP/zwAq6V+YghrtBMMzLZmWGTOzZjcSHmcVVY6IslFsVIPf9khOslHzFxuJ4kU6wVRObq\ncioxJEcauYm66zIiYQjwhWxIkgo2aSr9QE+UCRw00oaRMLpXNGI/ga6XoH0T2waZRhioeR+E7qqg\ns2g3BrmAotgI/jwbEyWFs+vr+5i/za+L+Q1djQxpIOFn5Rl5s5G0uUJQ1hAh7n+9wBp6YC+zQzAc\nCCfpKnUyvtSRJbAlQjVNy2hRV0SqIqER0pPo1oUyxDEtJ+/LhpgyyUE8DJxRkTCUmCMoyXWRbxvD\nTJBCRilngA9pP2br+VY8f+Mi8EyU02PQvzFCiDMKU2JZSU34K/8EdH+8g9bGfZxmB6000hGrY3S8\nmMmWAnhBSbkQuxED9bk4xfwRfaoGwxfaTuhC8SISHpdb6e3WOSMGISMzO5MFRGZ2XJhAHG5WyAFx\nQ3rS7talinn6zW9D+Jxl1k/Qk5665Yi5DyHaOQjrvkcPDhWU6Td5EDNjFOCkH5vqI/xdF0efjjKe\ntcTXhDwJam1gsMCrC1UlnIfX/jccyh+k+oFexpyF4AdnVwdtAzZO7LkL/8l8tFF5+S65JJqWIX05\nR1m+iC9MBgp5DBE8vIfZ7d9iiKmR9fYNZBhJQdcn0v2Xap37EKF9eqAx8ZqJFQalXgBJMS9m5QUv\ng9JUE+loXEcsrkOniB9tDB09WgXnnn0H3yWfzp2Z5BpgXyHs2w0jJdD2GxhdpgvYBDifGMOx4zj5\nt4zB8RiW75/jx331+J7azqmafcQ8spCAlRBTp/cSWBfWpiTjxf7JXiATCOXJMgtPSMTG+iPLb1sV\nRcSAnUGIeyyxXEzXwyBi7mep1pY6Y91WgyhxqoLbV8CYr2D66rKOD94PxYXiWbGcGz85vZBlfg7W\nwVX/WUb5so6yjyjc88nlb6MaMDRDTo6bPS+fwPLtUzzXlkfJLw5Srg6gLOWGjDF3aKFGhrRz62J+\nn9DKyUCLPMsF4Q+LG3alXYta0/4uY9GuRatKskJjzRLW9SJCcJvSXmsziM5CedOfBlF0tMVr+acH\nFbYNiZmWLqYXmFyIQoSjr3WxFTczt8Lwh/MYrCnC3usn9+jy++61AK2Pwk2vARr0K3DzJ/P4r/Fb\n+I3rPajqEqqGnmLuCK1IDMbnmx299LnELHIQJuPSguQ3LcGoyAxdyY2rpS0uxKle7d6fC+17/AL2\nN8PgsuNNtO9to07XzaFXq7jhG0a8W5bnoXQWwp0H4YFbYDuZWQnhYmJgVjtZBmqLOFfZSIvUxPHy\n3bz1Z9fywGugt8IH9VCepjTbFPijArg37SF8fzOUWUUNq6Pd4N0NO/9fGS/90QFUZFRNXloSTyYY\n3fNyjpXFTC5OBlvkrYjOxDObPqpkCyAtgpYoJKJqQszzlts4M0GyDV2yaVOSOtauuvBykoZCifVq\nmKWuuXiopYNG2qing0aplabifhyFUVTD8ubLdDVguwv8KnT/JsO1Yo2xANty4doHYF8t/OxrUPJf\nipE+WEerroERCkEHpvwQ0h740M9/T9WROG8+AnTDTgWuvg5K7oUCPdwXgWgNFNrgjufh6UegaBJM\nrdD6eTeHdz3Drs+fpah2mB9v+QNC0Rxom+NRmqy8OZf3JRQD32oVML8Qoqx8pnZhMljI/cz/paOI\ngVg9We/lAmiaGFJOBIWrZamToDOJMz1zJr0XZz6rH0iUnjRUxfzjRpXZscQuhVzbBDXP/5aqxx5B\nj4+wzUfzh7zYy3zIkaXLcAnQdBls+ShEbwFLCG75IvBT+FUHXFkKrRPQt4myiiKA2wSOd0t495ZS\nUFeKe08tQ6UNuCidqiVuJIxkgKqDvfz2m/2MjAt/x6AKZ2utTNxVyhCljEn5FOUOE2eQ6uNDWOQI\nObdDyW02zG8aKDnvBrOez/7gnxh8n5Pf+6/H32ObXYNcZf7sLlVdvEz0mtPBymdpFyeDhXwh0jNK\nKtkwzSfWAg3RIFqShCCa9aC7wIdfeh5DhFSzi9XM14oj/B8ywk+/1DvVJxP1G5AKLZh1Yew/OE+j\nFY51wnW7QedlyUHOYSAvB3JrzPz/7Z15dFxXnec/rzaVdpV2WZItWZbkXXbsOHQ2sncDHYeQAAkn\nmdAkp5swocnpNMNMn5luZvoASc/JHAIN3TQdSHqGdKCBYOjEaUMgxAkJTmxn8SZrtRZrq0W1r+/d\n+ePWU5Vk7aoqqZT6nFMnkfRc9W7de7/v3t/9LX2bq5iMlWH+M42q3RNcPWRn54jK+C9g6H10/h4j\nXnWtDLoqNzP54Va62cIYNYSTciobUekKNWP72wjDLwsK7t5M3nEXgao8uu5u42zlFoZowEKYUurZ\nQgkVe92IqgiercVwsI2ey7bw5vliBqs30LqpF7tWScxjlNaJ5ICfEPMXYV5VVGRmr5klulLLGhdy\nJ9LHYLaDO4F0c6glJ+SLIBg3R2lCRoSaU7ST0YNu85k+TitZ+WZJIMe/kSX5mQfChQzv2kf44au5\n3HaKzT747lNg6YQ8I7P6kecXwu59MPgajMcrCVUegJIPgmtDGafN2xkx11LW5MbeVEZrFKyvOOCN\n2Lp0pLqsCrrc4J1hxbQAVVGkqW0veMwlOCnHSQWhGfPUQRlol2O8L0LJZzcTfstFuNpG141b8VCC\nAY1K7GgYcFBBeK+JLa2gFEIPtTxf/yFOVezGrZZSccDO2d5dxMKzaEGEuTN4RNRVzjmuIstzpdcg\nt8aFfBRZ82s+D4xJpNjnxHxR6OHJhZaVr8ynvS/TU4oqyG4pZOVdM1/QkMas8WMTVDGxZxclO9tw\njXtonxxl/HXoGZUmU4AKBapKwNgAJdvhhrvgbAT68uD8u1D3sXJin7bRXdFAP5vwUowBgYaBErMH\n76AHj3f2ABMFaLMBfhiOgi9LDOsmM7TthJs6wP4ieGf4ghsBsx/O/Aiaq3p547or8BT2vRXnAAAg\nAElEQVSXYKcSP4VEVTPBiIzzMCDw/vUjKLrhuhHC5jzC5GElRCV23JTSShcbXGOMvhbB4gCLyYBd\nVHIuuo2zju3gNXBRNMIp86XWiTDzF16OxOSOdFWIka7DzZmscSFfDP3I5eBSqxG/jwlG5QKhKE8W\nhU4Heu7xehL1N00sv6LRXGKuIm3227lE5PuNG3nOeBsH6t7k9r8bxX4fjE4mhLzWAFdvgrp7IPYR\nA55GK3VGK3UNChPftnBi915eq9mOh9Kp9/RTyBZ6yNPC/Pa7GvZT0z/TCBSZoGwTfLQDvEfh3x0J\nITfHv4bVDksBKFOk1CQ/ZKyF8LH/ApE2K4WqQt3xCNFhFa9bamYQ6AyD93W4thoGOzYyULyJyVgZ\nqmbCFyrC7k3Y2AZpnP6ZhS7KCtwEKcCtlFFglqmaa8Z+QM/3w/S8DFXXVNFvbMIZLYeQAt1meWTW\nRyLOQcfL3OkrVU2+Vo0QmdquZYGQ6ye9860eF3NNjmmEokn+5goY0+RUl1xbpAx5eKmv1pfKCMTz\n0s49ck0CjAIjGjHM+MP5VHT3Efgx/GIAVAuYY3Iz0iPAbIJbq8HfYKGzuI3zt7USIY+LT9bRwxbc\nSSKuIMiLG2evi7xMYTW4CgyogYRYWIEdNrj5f0C03Uy/IwYnBRa/3OFXmKE+H84r4I2fNRRbIBgP\nwBKk1pJqBvIVQAGDWf7CEwCrgGvLwWOA33sgGAajBQo3K0x+JJ8LRU00/YuJ1v+4wMW/c/P6rxNd\nWWSC/9Su8OjXP8U75bvxaiXYvVW4A/HvSgNis4+nyUg5k65yMEBefojGigG6aOWftn6az33+CcKj\nA9TEBgjH+jFqKkQVOb11PewjccCte1TNhS+8igmyNNIRwTkXWSDk/SRm71x0I7+4tVRNIQsIx6S/\nudEAVUXp/7zJ+CuPREW/pT4/9OhP/YEwk00xsKmUCyf71OO0H/stL15/gSMmhYf/FxSNCZ75KbRt\ngkAQxo9D8Dug7bDQtb+VXjbThRTzYeqnCbmVEM300cMW/tl6Pzc99xKxT77LxR95piygfgXOlsDV\ndxroKmim9fAgl/23IC/+EI6PQ20L3HorTOTDP3wVEHDfdjjcD4obgiK1FtV2I9xiAYMVinYA2+Hb\nP4Vr/dD2eQVzFRT+UPD8UajarfCJ3+ZzuOCP0OJ2qhrFxbDRw4hRYNBvSih4ugz0qC2EyeOiawO+\nUHz8CGSEbdcC9rQyjXCLoHs0nhuiFgx/8/+oPDaE846tOIw11E2OM9DTPveXMYb8stYkLqb77KaX\nLBDyxXIBOcMbFrowx0xUTQp6ReHyXRSXQgRZ/AQS9TiXggP53E72Mz+P9G8HqkomuONff8QX/u83\nKb4mTPDrZs7d2cwvizdzVeB1tPfcDH+4mp2TKoYTDv7hLTD/OZT/LvF2QzTgnbIJScLk0UczzfTR\nTzPnaeMaUx/dRg/vxpfRdfvgIz8p4IWC64kpJs5b25i48Cb77xzjqknwRCH8iMy0+5/vhbxOKOiB\nOzuBc9BzRlYtWnpc5KXUAi3NUHwz8nveCpjgU38BY8X1nCspQDHAcMhNtdXP5me28U/5V065EAJE\nP2imcX8Pd5/spe5XdvyWfPoebKBfMdJV0UqXvQ1/pFBePGaCUePinkLuuNi3JU5TLzz3MA2Ro9Q+\n/gqWv/0dtQ/sl6vv3ln+/RDzR7pPBiG8WqvxcTIXRSdZR0KuktnkIOsMVQNnAMry02c31xEkdp09\nJA4pq+KvhdAPOJODhpJybHxEeZ6tvuf46VterAVlFD9yNb+p+CBh8rDluzhQ+B7vFW7l6F21tO9+\nj3ueP0vEEaGPAV7mOvrYjI8iBAouvw1NM1JRbEegTIk5wD6O83tVQXyhlcZPNRI4Mk70+W5GG5rp\nVOJhiwpE//cGTlvHKFS9xAS8VhWkllEMFRpNZYMoJRr5eYAdXIbUedI5kLqqFICogNB+AwPWRlSr\ngQvGTfgUKcCOe/Ox31aKo6KMTrZOe4/+vGY25LVQ2/tr8v7Fjn9vLZ21n+MCm3jPvptApAAxYJar\n8KgiX4tBAD4DnLdAa4RBx0Y22gaoMfvI/0IV7TEv4chLHBIfT/RtF9LsPBhv3Hw2KG01iytrZLqS\nS5YIebIP2nzYkVbBnIllWcRUmXyrOC917okLkRxwN4b0US9g4RwvetCQbjttQm6zC428V9aB+7aH\nUFtcbK64SGtFkGMcYMS1gVPaLlr+sotAQwEVtQ6qNzrZETpL71sRTt3di/97FvrDzfhVKXLhWB5C\nKFOeGDp1thFeMtxI1V9WU2SLYW4pJFYbQrvGw4ChmInksboFRglTjoM6RlBwEMJKHSMM2OppaB9B\neSrGidfhbWfq4pajwLlhsL4F1+0FrQAC+VaGlXr6aMYfj5p2VdkYq6oliJU+fVuThIMKfJ6z5LW0\nM/Bf7+G8doAuZyv+ThsiZpD2JF3AvcxfzMtGIoBMQ4p5j4Vwk4JWZsBChPca9vN64EqGTm+SfRpF\nbri9SHEOM79OekJyLK8KE6Sy8s9iyRIhD7K4c/4giSPsnJgvi0hMegYU5s1d8DldhOKvAHKiKkiv\nl7k2CLqYg1ylqUCPgTOW3fTWb8HSHKEbB8ffddDNdiZthQwZGji1dSdmY5SG0CBbhztpnzzJq391\nNa+Yd9PlvxxHtIKoNv00NqpO/9ng1phQqijasp2KfAcF+OW9zihTW4kdM1G8FOOjiEiSa40BjZ3q\nKYw+wZudcGIInCleRbp9MHQeOAGmA0ALDCiNOCjHQQVBCgiThwsbE1RdYk4CUDEycfXHCWz8OJM7\n9+B2lOHrKgenUYqxncQDOcT89fOiSX83I8NnJ2XWylufOcxl+9/mrdIP8MueDxHsLpShJBdJ9PMI\nc1cG0onEVqmU2zjyBv0Z/+QsEXKQve9CPtLnI4js/ZyQLxs9+X7UKN078jI8TCLIOaEgN2IK0rt0\nZnr6ZPQoUAFBpZDgYCFYYaw0rqwK0u/QAL74kjBcYeWQ7SADH2rkrXsv57xow+GrRAhFilR4bjOB\nmwqoUHFZbLhUG/mRAHnmMMXW6U7NCoJynFiIEMLKJDZK8aAgMKgaFW+7sZxR6XWkXsR1XC549TAo\nKuQ/Cl6rrGfqpxA3ZYSwTtU51YQBl6/8EqtEcMtWfBXF0G2UD8yJ+EPeyXQhX4gACSFOfjaOGDk/\nsp1AXzHHlCsJdhXLHdo4iQxn+s9zmb4FEIisYj1ON6tVBTaLhNyLdBReSMghnhGCnG/5CgjH5Etf\nlWdazEFOTN0rQUUeihYw9+HoBFKwRdK1KlLgy4CR6W0I+Es5aruBo9fecOnZ1KhJFquYDw2wCnyU\n4rOWkG/zo2lGDIpGcb5MBuKMj0FdzAEimHFho0q1Y3obOp8HbxpDzJ0heOksmIcEn/xzN6Wb3NiN\nlUSxTIm4Xa0kEC5E1QyMe2ZUrvcbpD+3zyAFXD+jAGlFWK4tKEpiA20w8Z3az8p+GENOdx+yTzUS\nK/N5LSZCuhyuSt7xSRb/NEs9WSTkS0E/6tbrgb3fE4+ugIgKIiy9WTJlN58N3exYiXw+m7g0MSbI\nFZtAPu+DyKFgZvYQA68BBuew2xTM8W+SSX4w2FSChgKCkXxMxhgGg1ScfEsQp5IQcx0DGgVaALUH\nfnkUHGkOADQDmxA0jo1SVz/KOeNWIljwUIJdrcQdKGPCM+MMym9IrL5d8S9DRSasGiA1CKRo60mw\nRpgeqZmcQG3e9xGrGIrvRRrxVy97WpYJuYo86Zhvj60TBs4iHZYLyYn5Coiq0p3LVpB+j5aFsMdf\nxciSMgqXDoeJ+Ks0fg1I75ilsBE5bEAK+kIzxWUETYFNUWIWEwN2+cEN5YMUWv04lXIMaFQzLkWc\nAI2xQSJHNCx+uWlIp89VoQLXmkBYINrnp6pxGE+htIf7QkWXinhYgQsmCCT1ty7iF5ZxAxFmb2AM\n6Y0y2yJaRVorFvLk00U844UjBHIVfp7VXI1D1hWWmGRpM1IF3mG1v+R1garJUnKrZn+cgRdZDeYc\nc2+33ciNmTrPNXMxgFwHnEWuEjUWVlq3AXrN064bcjbiCxUhhIJAQUOhQARpil7AaNQwPw5tNbIO\ndjqZFPADD0T/FCb2v0b0DfvcSxsV6LRMF3GB/D4XI+J6eGryqw/ZXzNf55i7NJuL2X3IZxJdDREH\n2dFrQ1+ybEWeY1URIhE4tNorc50oMrhoB7Mn5/LH/64gN2fLGfFO5BoiH9iywLV+A5zNgx2JPKsj\nrg1oZaMYCjTGqOWai7/jj1/4FbGQxrf/p0bQlc4EpwnCKvz9O/DpbfBu8SiTs2Wbiipw2nJpiL2D\nxUdR6kW9k1nqdmMCGfSTY1Gskdm4FPwsPfT1DPOnSMuxaES86lB0NVODzkBF7m5PMz1XOiRWh3qV\no9MsfSjo7xGIv8c55vecCClwOm9KDDVhYMJTjcNXgYaComl4L8Z48isaPqeseZDufY4V+ayLqvBv\n3fDOXW/SdyhGr68FuzduVgkqciU+U8THkWcU84nxReR3exq5+p65Il9KA0dJ7IIWIhQD92qsiEPI\n1fjaCELMwhW5xtK3MiHkHm0TqS9n8z5E1WTQhUGRhSqsy8mAlWL0ITGE9HqwcWn8mH7NIHLkV8Rf\niyV56F1AZoOYK1V+aLoYxlQTqmbCTxGdVe2cuvEybnrmBD+ZgGgGrFUFRXDdjbDvaujfWsdJSwfh\nnU1YrUGKYx5c/nIQyiX3DciH1lwBOAMk/P9XGsk0iDygDrH4fFNCrEKGQx/prvizVLJQyJeLDzlS\nNHJuiSlAX5HrodD5a0DMIRE7pgvBbMHAujk1hlydW5k/J9ts+JCr0Fpm954RwIAJGmJgSSh1DBNu\nawnu+gquKoGbzWDaA6+fh4k0eq4EInBsAG64FS5sbkKYWygjSgUO/OZCaQ+/OIsc2Ll0BxMjYfaY\nJDXR6ENIE9ZS0qOEY4mCKRnDg9SRtbXDz1Ih10fSBpZmHfKQ8F7JiXlKiKpSJWDtiDlIMR9HrhJN\nyAjCmejBKXkk/M1rWbyDkz6crEgPl5IZf3caoVadJuQAPoo4Y9tO04ODbLt4nsKYxtgEBFzpiwkM\nReDEaTD8I0z+VTHRKhMFBKbS8hIhXsMtCQeXBvtEkKaP+cLwl4KK3EFNsLSDgnAM/OEMuxy6mR5m\nunbIQhs5JMonLacT3WSqasf7hqgK/sgqVmKZAz1jwwiJwJLZCCddNx6/drELPTcJIVrkIi1AAefK\n2jnzJ20of2FAMYExmgEH2ShEfgANjiGqhB3LfMtfF/K7SBbxcPx34ym4lwgJN9HlTOVwbBX8xj2Q\nFAuwlsjSFbmOG2kMXWqgShi5N85ADu73CzEVvGFZ5Hk1okDnI4a05RqQQ6WQ2QtbaCTyx9YjvVSs\nLK6qkZ7QyUDC/3wOjKjkE8SvFqIcU+j/JfQ60h9OYlBgSxk0uc9QNKgwOmBgrMbC+IYaRmcmiRkh\n8TALkpgyqcgHpYv4XJV9FiKqroJd3M9asonPZI3NuKUgkD7lesDPUnAhZ20LS0+GnWNOVA3ceuCQ\ncW3FYAkSgSWNyEPO+Ua/Xg6nmkQh6YXi0HS7eeulfzIbo5iMMYyolOChkgkc0QrM/6zw/Emwp8EN\nulCRtx02QXkxaCH4cTc8+BvYGD1N++/70G4fI3RXHZ1sk/9Ij3FJtgbZSU1u3QiJMpbLFXGQYfjh\nTKaJTc6duzbJYiHX0ZMVL1U13EAn0ilrDdl2sx1NyMChikJZeUhR1pagg5yTAinSCxkXdVNCCVOF\nK+adNQI5JPVr4sOzothOZaGdKiZoFn0oikaruRvDQxq8Cnil6ApS59C2yyq7oW8DfPxqiJyA//Mm\nBB+Do2Yj7r/fQ94nthL1JzVId+UUST+n4oZUEgeaKyHjAWkx5IJx7dnFk8lSG3kynVzqPLxY/Ehf\n0DUSrbiecPhhwruKVVoWYJilmQk8yKFyhvmHSwAZcq7TZZGFMYEtdPHJ2LPcHX6W7dpZ/jB4BJNI\nrCw3sXTnmfmwXAN7n4I//i1EHkZWCDJA4efgwBsbcNx5JYf5EKen6u7NwgVSc7DZS2rMy65Ahlfj\nZ1jrIg7rYkUuWJkQR4DjwG5mDw3MsWwE0t98Lbkn6gikuUBj8dUB9cpGp+I/tzC762EEGdrfipxh\nAnZwmk9F/5WGH/2an/73MEWNR7E+qsHPAS98yARtV4BpE6hlYJ+EY8/IxfFSaUY+d44dBXtbCds+\nXI1/JMIbPxkgEgOMcNHSQJehjfO+Nsa9cZcePdxCn049yDXScqeXFv8eUlWH2OHPYMEIgQwJXo3Q\n/6WzDoR8peglR/SZN9vMzLFstKTUogVr7EGpIs2ePqT9u3n+y4HEcAFpczciTTTls1yjx6ABeYSx\ndXvYWGZAPF6J7T/G+P5nATu4PfCmCrZ90HIHiDxwvCgtEVYFDprg+Rj4FyGot22GhishNgGduzZx\n9k87eEmpxbApQunPuzn4wuuU+lVG/+w0tZ//HXVXbmfE2QidJrn61g84e5BPg+WaVcLx72emvX05\naAImMxlNHEHu9P1ky259nQj5CNJKVLrQhfPgQ468jTBLlZQcK0DVEr7ma03Mo/FXEJkqdSOLNzjq\nTgx6fu4ipKjrBJBqXGDipHUfT5Xcx679l6EUazS29hC9fpw6Rtj741G0V2Jo7TYuXFYACqhFIS7H\nwfHHoOVuOHgMjnSDY4Gg5toCsP0ROJvKCNS24GxpIYQVQ4FG7Np8BhtV3nZXkD/Uw20tr+JU93Mi\ncgB8ynSnDD/L8+71kvAJT5UbTkZT1OoDwbPAdWuLdSLkXhLLpJUwiZzF9Vwa3ZFjRcTWsJhDYnVu\nQMaZLWVmBJhe+SZZzL2Ay8DIwEYON36YY1X7KRR+Wnd10b6rk738Auez44QioBTXEChqwU8Bxdt9\nVD/QT3v5BMoBO20avDaysJBTDUoxjO2sZrh0Ay5seChBxcQ7hnLebdmNhxK2XXaWK9Tfow2ZYdiY\nsF+ryHXRcnTTQ6LuaqpQNbmjywh+UnMim3nWiZCDHD35rHw1ndyJOTFPKTFNBg7pTkb5ZunVslYQ\nTK/zvdTnTQgpZAaky6KOGzAbcFKNM1qOpTyEU6nAU1DCB3gD0+ZuOuvb8Ww9gJ8WVIyUKy4c9RW0\nPXQeNeDAvF1Q/qtLY3RmctEOoz+BWG8Aww0C5+5yHFTKmpyiismADSEUzrKNtz2XMTDcAmOGhPhq\n8TYsxaIwiVxHeUitiOsP/2AmDsx9yCdYqkJWM8s6EnInqRFy/b1Aqk3OzJJSVA28cSlSgDyzjFRZ\nS+g+zssR8zByUWdEWvoMJNyPFQMIExHFykVRh1GJcTJ/L3/weRVnzTWcKtghRRwHevZym+rC0A8E\noEDMPWEVoM0GnWfkR282+2GnAQeV2LVKLoQ3IYSBMXcNQiiMees4d7EDJgwJk0pyMevF4kH6zqf6\nTFAX8UAmcqn4SYT1ZifrSMhBHlKEWFwo3kI4kcuTZnIHoGnCHYLSeCToehLzGPKgcwtyHaCLuUCK\nuWYipsCgt4kjjbfwUv2NuEw2Iliw4ULFSIh8SvwebMddWI6B8iYM+SCvFJQoeJOE0wAUG+C2a+FI\nD2y8Bcb/pJWz29uwa5X0BFoYcyc5NgYMMGSSudNdJB40YZZW/cdPIvthqgnHMiTiQeSTNztX4jrr\nTMgn4v/dPO9Vi2cS6RS8jZxrYppwB6E0H6ymtWVmASnmAuncvdSZIpBDp5WEmOsbvTwDnLMQM1o4\nqnwQjLCheogCa4BxqrErlZQYPFT1T9DyUCfuj5SQvzuI9k6U6/cLHF44eRwiPgjEwGKAyypBPGTm\nw0746ZMmTp+so3d7M32h5ukiHlGgxyxLucVYui1cd8HU25cOl25NZKiAchTp4JD9uZfWmZDrLCfS\ncy58SH/SfSl8zxzTcAdBWNfmIegoUuw2srzu70KuzJPT4OvDU0UWoAAutjVCifT1sxW6MJRp/G7H\nlXzp3cdp5Tx/yBECr50lekeAq64QfPDf4NT3Zci91QZX/7XC6auaiOabsX6ikSD7cc5Mth6LF7zQ\nxfsi04+EFqOdPqRnXjrxh+VZSloRSJfjtZWOdrmsQyF3IE0iC9XkWgp60FAHuXD+NOENS+8EqxlK\nUmEaSyH6kFqMn/ls6F6tFchNngY0zbim1wybYmBTcQdK8YaKsZpCU4emRlTqn2vCk9/LBH1U1SZW\nkR4XfP1v4KpPwJv5fxD3VJlxthOKV/+ZawXuYeH6mIupZr9SPKEMHW6+S/qSBmeedSjkGqnf7+kR\nHqeBdnKJttKAiBeo0FPhriUx1/3E+1iemKvIlAAa0ubuRQpiU9I1MUXarVXQKkFTDQS0ArrsbVNi\nHis3MUktflHKvtp3oM4J3aBp4PPABbEJL8UM0UgPLTiTo5QEsh6nzhDTXaUXmjZ6zc50Rse7g9I2\nnlazSgw5j7Mn2GcxrINcK7MRID1LBz8y5C3dCUffx2hCirln9SuTT0P36OhjefM/QiJqUkWK6AWm\nR06GlWliqwkDoZiVIPmEsDLIRsaoYUhpYOyyKurvg1uvgJ3VcPsPFESpFYGBCGZCWInNt04Ls3hR\ntiNNTOl059ZFPK1JsULIpAc+1pOIw7oV8ihyCTFA6oujuuPvm12RX1mFLube0Nqab7oALxc3CQ8R\nbY73mjTIykIzGKeaScqIYSJMHo7qcmKXl7FhJ4QssK1DsME0Rg1j5BGhFDfFuv03pMDIMjffdqRX\nXrpScQshH9qhdIu4H/nkzP6DzdlYp0IOcrkxTnqUwIXcK+fEPG1oQrqf+cPylfFCAnOgkTgAXSoB\nZLCNfsCoIYdo8srYbwDfpdPSTem0FbaChnBpGLuh0ge//7qgubuP1mg3dYxQjpN8gljNIUoKPNNr\nrziYXgEpwOz65ojfb7ryRmkCfHFf8bSaU3zI+ZrdLobzsUpC3p+hzxHI0ZiOPA1O5OAY59IoiqNp\n+Ly1RIbaJ5AHoN6wnOyZEPPIy/P/Xbd321mevThAQk8E8qHgYNHZAQsIUoob7ZSTkWc8hLpg30EY\n88D4U3YC3+9l89A7tNHJRgaoNw9TVu6E6vjNnnpZeukmm0l8XFozQS/+kK6VuJ5/x59qe03y2PQg\n56de62/9ss6FXEMaNdPlyuRE+pddYLob06tp+ry1wiq0zx/JjJgvJOQ6esGYVDhYjLDooJqmwAV2\nDp7F9wM7rx4yMXRnCcHvVLPvH6s50VXEyLMONvziONcO/orbJg5xw8hvaDL005A/BMWaFPL5EEj9\nS1egD8g+DEbTlENFH5s+ZCd1kc0Rm4tlHXqtzEYYmac0Xc8tH/IQZTs5j5Y0ovsW51ukH7ZxlS2D\nen3PhcrGzUQjMSR1osjVvhG50o8oKHkaFmNiEWImSkfvGdoPnSLwBng+Xo7niV28QDMgaPpxP9vD\nZxn+mJMazwRX1L5N++g7VH+qi9N1f8CThgeImMV0f/gYiZ2FRqIwRrosHZpIo4jr6InV14eP+GJ4\nnwh5J7I8ykrS3C5ECFlC5vI0fkYO/BH5yjPJ2qCrjS7mNUv4Nz6gG1llUGcA6Wtejjzs1BTytnnY\nVNU/dUk9QxQN+3j116BsgI6HanmKm7iQ5Md4Xd7LNBpfoP7tES764MS/DzP+0nHMR/bRXNvL+aoY\nIk9AMK7mYyTSEYSAc0tox3IIRDKQzfAMa7lQcjpQhMhILGziA9daGHaOHDlyZAlzyXXGV+QZfm7k\nyJEjx7pnHbsf5siRI8f7g5yQ58iRI0eWk3Ehf/HFF9m6dSutra089thjmf74lNPU1MTu3bvZu3cv\nBw4cAMDpdHLzzTfT1tbGLbfcwuTkUrP1rx6f+cxnqKmpYdeuXVO/m689X/va12htbWXr1q0cOXJk\nNW55SczWvi9/+cs0NDSwd+9e9u7dy+HDh6f+lk3tGxwc5Prrr2fHjh3s3LmTb3zjG8D66b+52rde\n+m9FiAwSi8VES0uL6OvrE5FIRHR0dIgzZ85k8hZSTlNTk3A4HNN+98UvflE89thjQgghHn30UfGl\nL31pNW5tWbzyyivixIkTYufOnVO/m6s9p0+fFh0dHSISiYi+vj7R0tIiVFVdlfteLLO178tf/rJ4\n/PHHL7k229o3MjIiTp48KYQQwuv1ira2NnHmzJl1039ztW+99N9KyOiK/NixY2zZsoWmpibMZjN3\n3XUXhw4dyuQtpAUx4wD35z//Offddx8A9913Hz/72c9W47aWxTXXXIPNZpv2u7nac+jQIe6++27M\nZjNNTU1s2bKFY8eOZfyel8Js7YPZD+GzrX21tbXs2bMHgKKiIrZt28bw8PC66b+52gfro/9WQkaF\nfHh4mMbGxqmfGxoapjoiW1EUhZtuuon9+/fz3e9+F4CxsTFqaqRjcU1NDWNjY6t5iytmrvZcvHiR\nhoaGqeuyuT+/+c1v0tHRwf333z9lesjm9vX393Py5EmuuOKKddl/evs+8IEPAOuv/5ZKRoV8PfqQ\nv/baa5w8eZLDhw/zrW99i6NHp+chURRlXbV7ofZkY1sffPBB+vr6ePvtt6mrq+ORRx6Z89psaJ/P\n5+OOO+7giSeeoLh4eoGJ9dB/Pp+PO++8kyeeeIKioqJ113/LIaNCXl9fz+Dg4NTPg4OD056Y2Uhd\nXR0AVVVV3H777Rw7doyamhpGR0cBGBkZobq6ejVvccXM1Z6Z/Tk0NER9ff2q3ONKqK6unhK4Bx54\nYGr7nY3ti0aj3HHHHdx777189KMfBdZX/+ntu+eee6bat576b7lkVMj3799PV1cX/f39RCIRfvjD\nH3Lw4MFM3kJKCQQCeL0yn4Pf7+fIkSPs2rWLgwcP8vTTTwPw9NNPTw24bGWu9hw8eJBnn32WSCRC\nX18fXV1dU5472cTIyMjU/z/33HNTHi3Z1j4hBPfffz/bt2/n4Ycfnvr9eum/ufKD6MwAAAD4SURB\nVNq3XvpvRWT6dPWFF14QbW1toqWlRXz1q1/N9MenlN7eXtHR0SE6OjrEjh07ptrjcDjEjTfeKFpb\nW8XNN98sXC7XKt/p4rnrrrtEXV2dMJvNoqGhQXzve9+btz1f+cpXREtLi2hvbxcvvvjiKt754pjZ\nvieffFLce++9YteuXWL37t3itttuE6Ojo1PXZ1P7jh49KhRFER0dHWLPnj1iz5494vDhw+um/2Zr\n3wsvvLBu+m8lZDzXSo4cOXLkSC25yM4cOXLkyHJyQp4jR44cWU5OyHPkyJEjy8kJeY4cOXJkOTkh\nz5EjR44sJyfkOXLkyJHl/H/2z18sGfD0gAAAAABJRU5ErkJggg==\n", "text": [ "" ] } ], "prompt_number": 58 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using this simple code, you can generate some absolutely stunning images. The problem is, to go deeper you need **very high resolution** and **very many iterations**, both of which are extremely slow in Python!\n", "\n", "Your assignment is to make these faster:\n", "\n", "#### Part 1\n", "Speed up this example using either CTypes, F2Py, or Cython. With CTypes/F2Py, you'll have to write the above functionality as a C/Fortran routine, and use the tools above to wrap that functionality. \n", "\n", "For Cython, it will require adding appropriate type information to each of the variables used in the computation. Also, check the Cython documentation and make sure to make ``mandel()`` a ``cdef``-ed function. Once you're finished, use the ``cython -a`` trick to generate annotated code, and make sure there are no yellow-highlighted lines in the core of your algorithm: the loops should look completely clean!\n", "\n", "#### Part 2\n", "Speed up this example using Numba. Numba is much more of a black box, but you should be able to do it very quickly. You'll likely find that simply adding the decorator to the function doesn't work. I'll give you a hint: Numba doesn't like an array specified with ``dtype=float``. Look back to our previous example for clues about how to fix this.\n", "(Also, you might find [this notebook](http://nbviewer.ipython.org/f5707335f40af9463c43) to be a useful resource: it is all about doing the mandelbrot set with Numba).\n", "\n", "#### Part 3\n", "Use ``%timeit`` to compare your three implementations side-by-side for the values passed above.\n", "\n", "#### Part 4\n", "Once you have these working, choose one of the methods and use it to zoom-in on a portion of the above plot. Go to high resolution and a large number of iterations (say 256). Experiment with different matplotlib color schemes (see some of the options [here](http://wiki.scipy.org/Cookbook/Matplotlib/Show_colormaps)) and produce *your most beautiful visualization of the Mandelbrot set*." ] } ], "metadata": {} } ] }