{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Quandl: S&P 500 Volatility Index (VIX)\n", "\n", "In this notebook, we'll take a look at data set , available on [Quantopian](https://www.quantopian.com/data). This dataset spans from January, 1990 through the current day. It contains the value for the index VIX, a measure of volatility in the S&P 500. We access this data via the API provided by [Quandl](https://www.quandl.com). [More details](https://www.quandl.com/data/YAHOO/INDEX_VIX-VIX-S-P-500-Volatility-Index) on this dataset can be found on Quandl's website.\n", "\n", "To be clear, this is a single value for VIX each day.\n", "\n", "## Notebook Contents\n", "\n", "There are two ways to access the data and you'll find both of them listed below. Just click on the section you'd like to read through.\n", "\n", "- Interactive overview: This is only available on Research and uses blaze to give you access to large amounts of data. Recommended for exploration and plotting.\n", "- Pipeline overview: Data is made available through pipeline which is available on both the Research & Backtesting environment. Recommended for custom factor development and moving back & forth between research/backtesting.\n", "\n", "### Limits\n", "One key caveat: we limit the number of results returned from any given expression to 10,000 to protect against runaway memory usage. To be clear, you have access to all the data server side. We are limiting the size of the responses back from Blaze.\n", "\n", "With preamble in place, let's get started:\n", "\n", "\n", "#Interactive Overview\n", "### Accessing the data with Blaze and Interactive on Research\n", "Partner datasets are available on Quantopian Research through an API service known as [Blaze](http://blaze.pydata.org). Blaze provides the Quantopian user with a convenient interface to access very large datasets, in an interactive, generic manner.\n", "\n", "Blaze provides an important function for accessing these datasets. Some of these sets are many millions of records. Bringing that data directly into Quantopian Research directly just is not viable. So Blaze allows us to provide a simple querying interface and shift the burden over to the server side.\n", "\n", "It is common to use Blaze to reduce your dataset in size, convert it over to Pandas and then to use Pandas for further computation, manipulation and visualization.\n", "\n", "Helpful links:\n", "* [Query building for Blaze](http://blaze.readthedocs.io/en/latest/queries.html)\n", "* [Pandas-to-Blaze dictionary](http://blaze.readthedocs.io/en/latest/rosetta-pandas.html)\n", "* [SQL-to-Blaze dictionary](http://blaze.readthedocs.io/en/latest/rosetta-sql.html).\n", "\n", "Once you've limited the size of your Blaze object, you can convert it to a Pandas DataFrames using:\n", "> `from odo import odo` \n", "> `odo(expr, pandas.DataFrame)`\n", "\n", "\n", "###To see how this data can be used in your algorithm, search for the `Pipeline Overview` section of this notebook or head straight to Pipeline Overview" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# import the dataset\n", "from quantopian.interactive.data.quandl import yahoo_index_vix as dataset\n", "# Since this data is provided by Quandl for free, there is no _free version of this\n", "# data set, as found in the premium sets. This import gets you the entirety of this data set.\n", "\n", "# import data operations\n", "from odo import odo\n", "# import other libraries we will use\n", "import pandas as pd\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "dshape(\"\"\"var * {\n", " open_: float64,\n", " high: float64,\n", " low: float64,\n", " close: float64,\n", " volume: float64,\n", " adjusted_close: float64,\n", " asof_date: datetime,\n", " timestamp: datetime\n", " }\"\"\")" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Let's use blaze to understand the data a bit using Blaze dshape()\n", "dataset.dshape" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "6651" ], "text/plain": [ "6651" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# And how many rows are there?\n", "# N.B. we're using a Blaze function to do this, not len()\n", "dataset.count()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", " | open_ | \n", "high | \n", "low | \n", "close | \n", "volume | \n", "adjusted_close | \n", "asof_date | \n", "timestamp | \n", "
---|---|---|---|---|---|---|---|---|
0 | \n", "19.750000 | \n", "21.160000 | \n", "19.540001 | \n", "20.980000 | \n", "0 | \n", "20.980000 | \n", "2016-02-23 | \n", "2016-02-24 08:01:58.351899 | \n", "
1 | \n", "22.280001 | \n", "22.870001 | \n", "20.260000 | \n", "20.719999 | \n", "0 | \n", "20.719999 | \n", "2016-02-24 | \n", "2016-02-25 08:02:33.397136 | \n", "
2 | \n", "20.540001 | \n", "21.260000 | \n", "19.100000 | \n", "19.110001 | \n", "0 | \n", "19.110001 | \n", "2016-02-25 | \n", "2016-02-26 05:01:23.226761 | \n", "
\n", " | \n", " | adjusted_close | \n", "close | \n", "high | \n", "low | \n", "open | \n", "volume | \n", "
---|---|---|---|---|---|---|---|
2013-11-01 00:00:00+00:00 | \n", "Equity(2 [AA]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "
Equity(21 [AAME]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(24 [AAPL]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(25 [AA_PR]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(31 [ABAX]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(39 [DDC]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(41 [ARCB]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(52 [ABM]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(53 [ABMD]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(62 [ABT]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(64 [ABX]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(66 [AB]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(67 [ADSK]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(69 [ACAT]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(70 [VBF]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(76 [TAP]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(84 [ACET]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(86 [ACG]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(88 [ACI]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(99 [ACO]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(100 [IEP]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(106 [ACU]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(110 [ACXM]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(112 [ACY]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(114 [ADBE]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(117 [AEY]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(122 [ADI]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(128 [ADM]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(134 [SXCL]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
Equity(149 [ADX]) | \n", "13.75 | \n", "13.75 | \n", "14.02 | \n", "13.28 | \n", "13.83 | \n", "0 | \n", "|
... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
2013-11-25 00:00:00+00:00 | \n", "Equity(45864 [CDX]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "
Equity(45865 [XNCR]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45866 [ZU]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45867 [EROS]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45873 [IR_WI]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45874 [ALLE]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45875 [HFIN]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45880 [CACQ]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45882 [TKF_WD]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45883 [IIF_WD]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45885 [EGF_WD]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45891 [OXFD]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45892 [TLOG]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45893 [VTL]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45894 [RTGN]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45895 [EMSH]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45896 [AMZG]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45902 [WBAI]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45903 [GOMO]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45904 [IPWR]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45905 [GFIS]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45906 [VNCE]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45907 [RITT_W]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45914 [EVGN]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(45915 [NVGS]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(48504 [ERUS]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(49010 [TBRA]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(49131 [OESX]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(49259 [ITUS]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "|
Equity(49523 [TLGT]) | \n", "12.26 | \n", "12.26 | \n", "12.91 | \n", "12.24 | \n", "12.69 | \n", "0 | \n", "
134806 rows × 6 columns
\n", "