{
 "metadata": {
  "name": "",
  "signature": "sha256:2367377a6fce3b59a11f38f4dd2b55cd70b0035b3fbff17a56eaba6ee57e897d"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Using DirectView and LoadBalancedView together"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We mentioned that you can use any number of views of any sort simultaneously.\n",
      "\n",
      "The most common model for doing this is to use a DirectView to initialise the engine namespaces\n",
      "(either defining functions and variables, or loading datasets) with a DirectView,\n",
      "and then submitting work as tasks via the LoadBalancedView.\n",
      "\n",
      "For this example, we are going to define a few functions for fetching data and getting a simple summary on some stocks,\n",
      "then we are going to run those functions on a series of stocks in a load-balanced way."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The usual boilerplate:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%matplotlib inline\n",
      "import matplotlib.pyplot as plt"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from IPython import parallel\n",
      "\n",
      "rc = parallel.Client()\n",
      "all_engines = rc[:]\n",
      "lbv = rc.load_balanced_view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Use matplotlib to fetch data from Yahoo\n",
      "\n",
      "Note that we are defining these functions on the engines"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%px\n",
      "\n",
      "import urllib2\n",
      "from io import BytesIO\n",
      "\n",
      "from matplotlib import finance\n",
      "\n",
      "def get_yahoo_data(name=None, start=None, end=None):\n",
      "    \"\"\"return historical stock data from yahoo as a recarray.\"\"\"\n",
      "    fp = finance.fetch_historical_yahoo(name, start, end)\n",
      "    return finance.parse_yahoo_historical(fp, asobject=True)\n",
      "\n",
      "def relative_close(records):\n",
      "    \"\"\"return the reative change in closing price over the interval\"\"\"\n",
      "    \n",
      "    close = records['aclose']\n",
      "    return (close[-1] - close[0]) / close[0]\n",
      "\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Setup the timestamps.  We want the past year"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%px\n",
      "import datetime\n",
      "end = datetime.datetime.now()\n",
      "start = end - datetime.timedelta(days=365)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Our ticker of stocks to check"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ticker = ['AAPL', 'GOOG', 'MSFT', 'ORCL', 'AMZN', 'INTC', 'ATVI', 'EA', 'NFLX']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And map a simple function (that depends on the code we have defined on the engines)\n",
      "onto our ticker."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def year_over_year(name):\n",
      "    \"\"\"fetch data, and return year-over-year change (relative to first close)\n",
      " \n",
      "    in percent\n",
      "    \"\"\"\n",
      "    stock = get_yahoo_data(name, start, end)\n",
      "    return relative_close(stock) * 100\n",
      "\n",
      "amr = lbv.map(year_over_year, ticker)\n",
      "amr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig, ax = plt.subplots()\n",
      "fig.set_figwidth(10)\n",
      "ax.bar(range(len(amr)), amr)\n",
      "ax.set_xticks(np.arange(len(amr))+0.4)\n",
      "ax.set_xticklabels(ticker)\n",
      "ax.axhline(0, c='k');"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Plotting stock data in parallel with Pandas"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%px\n",
      "%matplotlib inline\n",
      "import matplotlib.pyplot as plt"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from pandas.io.data import DataReader\n",
      "from datetime import datetime"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def plot_stock_since(name, since_when, key=\"Adj Close\"):\n",
      "    data = DataReader(name, \"yahoo\", since_when)\n",
      "    data[key].plot()\n",
      "    plt.title(name)\n",
      "    plt.show()\n",
      "    "
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "start = datetime(2011,1,1)\n",
      "plot_stock_since('GOOG', start)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%px from pandas.io.data import DataReader"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "all_engines.scatter('ticker', ticker)\n",
      "%px print ticker"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "all_engines['plot_stock_since'] = plot_stock_since"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%px\n",
      "from datetime import datetime\n",
      "start = datetime(2011,1,1)\n",
      "from pandas.io.data import DataReader\n",
      "\n",
      "for stock in ticker:\n",
      "    plot_stock_since(stock, start)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "amr = lbv.map(plot_stock_since, ticker, [start]*len(ticker))\n",
      "amr\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%time amr.wait()\n",
      "amr.display_outputs()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}