{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "#Ranking Universes by Factors\n", "\n", "By Delaney Granizo-Mackenzie and Gilbert Wassermann\n", "\n", "Part of the Quantopian Lecture Series:\n", "\n", "* [www.quantopian.com/lectures](https://www.quantopian.com/lectures)\n", "* [https://github.com/quantopian/research_public](https://github.com/quantopian/research_public)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One common technique in quantitative finance is that of ranking stocks in some way. This ranking can be whatever you come up with, but will often be a combination of fundamental factors and price-based signals. One example could be the following\n", "\n", "1. Score stocks based on 0.5 x the PE Ratio of that stock + 0.5 x the 30 day price momentum\n", "2. Rank stocks based on that score\n", "\n", "These ranking systems can be used to construct long-short equity strategies. The Long-Short Equity Lecture is recommended reading before this Lecture.\n", "\n", "In order to develop a good ranking system, we need to first understand how to evaluate ranking systems. We will show a demo here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##WARNING: \n", "This notebook does analysis over thousands of equities and hundreds of timepoints. The resulting memory usage can crash the research server if you are running other notebooks. Please shut down other notebooks in the main research menu before running this notebook. You can tell if other notebooks are running by checking the color of the notebook symbol. Green indicates running, grey indicates not." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import statsmodels.api as sm\n", "import scipy.stats as stats\n", "import scipy\n", "from statsmodels import regression\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "import pandas as pd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting Data\n", "The first thing we're gonna do is get monthly values for the Market Cap, P/E Ratio and Monthly Returns for every equity. Monthly Returns is a metric that takes the returns accrued over an entire month of trading by dividing the last close price by the first close price and subtracting 1." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from quantopian.pipeline import Pipeline\n", "from quantopian.pipeline.data import morningstar\n", "from quantopian.pipeline.data.builtin import USEquityPricing\n", "from quantopian.pipeline.factors import CustomFactor, Returns\n", " \n", "def make_pipeline():\n", " \"\"\"\n", " Create and return our pipeline.\n", " \n", " We break this piece of logic out into its own function to make it easier to\n", " test and modify in isolation.\n", " \"\"\"\n", " \n", " pipe = Pipeline(\n", " columns = {\n", " 'Market Cap' : morningstar.valuation.market_cap.latest,\n", " 'PE Ratio' : morningstar.valuation_ratios.pe_ratio.latest,\n", " 'Monthly Returns': Returns(window_length=21),\n", " })\n", " \n", " return pipe\n", "\n", "pipe = make_pipeline()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's take a look at the data to get a quick sense of what we have. This may take a while." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", " | \n", " | Market Cap | \n", "Monthly Returns | \n", "PE Ratio | \n", "
---|---|---|---|---|
2013-01-02 00:00:00+00:00 | \n", "Equity(2 [AA]) | \n", "8.975170e+09 | \n", "0.032143 | \n", "135.1351 | \n", "
Equity(21 [AAME]) | \n", "6.228180e+07 | \n", "0.065580 | \n", "16.0772 | \n", "|
Equity(24 [AAPL]) | \n", "5.505680e+11 | \n", "-0.089110 | \n", "13.2626 | \n", "|
Equity(31 [ABAX]) | \n", "8.283930e+08 | \n", "0.008415 | \n", "35.8423 | \n", "|
Equity(52 [ABM]) | \n", "1.037840e+09 | \n", "0.053308 | \n", "16.7504 | \n", "|
Equity(53 [ABMD]) | \n", "5.296030e+08 | \n", "0.008621 | \n", "39.8406 | \n", "|
Equity(62 [ABT]) | \n", "4.916000e+10 | \n", "0.008154 | \n", "15.7978 | \n", "|
Equity(64 [ABX]) | \n", "3.455270e+10 | \n", "0.014778 | \n", "10.2987 | \n", "|
Equity(66 [AB]) | \n", "1.848950e+09 | \n", "-0.007688 | \n", "9.3284 | \n", "|
Equity(67 [ADSK]) | \n", "7.444310e+09 | \n", "0.066727 | \n", "31.3480 | \n", "|
Equity(69 [ACAT]) | \n", "4.985790e+08 | \n", "-0.115538 | \n", "14.9925 | \n", "|
Equity(76 [TAP]) | \n", "7.513120e+09 | \n", "0.034208 | \n", "13.6054 | \n", "|
Equity(84 [ACET]) | \n", "2.710310e+08 | \n", "0.013798 | \n", "14.3266 | \n", "|
Equity(88 [ACI]) | \n", "1.426500e+09 | \n", "0.089153 | \n", "14.7493 | \n", "|
Equity(99 [ACO]) | \n", "9.648460e+08 | \n", "0.023727 | \n", "14.8148 | \n", "|
Equity(100 [IEP]) | \n", "4.212790e+09 | \n", "0.107074 | \n", "6.7204 | \n", "|
Equity(106 [ACU]) | \n", "3.430780e+07 | \n", "0.008284 | \n", "10.1626 | \n", "|
Equity(110 [ACXM]) | \n", "1.319010e+09 | \n", "-0.012450 | \n", "26.5957 | \n", "|
Equity(112 [ACY]) | \n", "2.004690e+07 | \n", "0.084681 | \n", "4.3764 | \n", "|
Equity(114 [ADBE]) | \n", "1.710190e+10 | \n", "0.088099 | \n", "20.8768 | \n", "|
Equity(122 [ADI]) | \n", "1.223640e+10 | \n", "0.044177 | \n", "19.0476 | \n", "|
Equity(128 [ADM]) | \n", "1.758370e+10 | \n", "0.025843 | \n", "18.7266 | \n", "|
Equity(153 [AE]) | \n", "1.471520e+08 | \n", "-0.003438 | \n", "6.0277 | \n", "|
Equity(154 [AEM]) | \n", "9.589180e+09 | \n", "-0.059362 | \n", "64.5161 | \n", "|
Equity(157 [AEG]) | \n", "1.096140e+10 | \n", "0.125654 | \n", "4.2644 | \n", "|
Equity(161 [AEP]) | \n", "2.069590e+10 | \n", "0.000469 | \n", "11.3636 | \n", "|
Equity(162 [AEPI]) | \n", "3.346250e+08 | \n", "-0.012629 | \n", "14.5349 | \n", "|
Equity(166 [AES]) | \n", "7.938060e+09 | \n", "0.005162 | \n", "128.2051 | \n", "|
Equity(168 [AET]) | \n", "1.444710e+10 | \n", "0.072222 | \n", "8.2576 | \n", "|
Equity(185 [AFL]) | \n", "2.484740e+10 | \n", "0.003021 | \n", "8.7108 | \n", "|
... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
2015-02-02 00:00:00+00:00 | \n", "Equity(47858 [NMS]) | \n", "6.352770e+07 | \n", "0.075804 | \n", "10.6952 | \n", "
Equity(47873 [OMAM]) | \n", "1.854000e+09 | \n", "-0.067692 | \n", "74.7579 | \n", "|
Equity(47875 [VBTX]) | \n", "1.237870e+08 | \n", "-0.064286 | \n", "18.4815 | \n", "|
Equity(47876 [MOLG]) | \n", "1.302750e+08 | \n", "-0.372549 | \n", "29.9010 | \n", "|
Equity(47883 [DPLO]) | \n", "1.336120e+09 | \n", "-0.102920 | \n", "171.6673 | \n", "|
Equity(47884 [PLAY]) | \n", "1.146320e+09 | \n", "0.051592 | \n", "87.8222 | \n", "|
Equity(47888 [FCAU]) | \n", "1.627660e+10 | \n", "0.143104 | \n", "14.2440 | \n", "|
Equity(47894 [KIQ]) | \n", "2.109430e+08 | \n", "-0.228376 | \n", "42.2230 | \n", "|
Equity(47898 [GWB]) | \n", "1.199980e+09 | \n", "-0.112478 | \n", "11.4336 | \n", "|
Equity(47904 [SRSC]) | \n", "9.647810e+08 | \n", "-0.030177 | \n", "7.6246 | \n", "|
Equity(47913 [XENE]) | \n", "2.444860e+08 | \n", "-0.113924 | \n", "28.1404 | \n", "|
Equity(47921 [KEYS]) | \n", "5.652700e+09 | \n", "-0.012718 | \n", "14.3484 | \n", "|
Equity(47923 [KE]) | \n", "3.170760e+08 | \n", "-0.149042 | \n", "14.2148 | \n", "|
Equity(47935 [ABCW]) | \n", "3.094400e+08 | \n", "0.008735 | \n", "4.1449 | \n", "|
Equity(47949 [APTO]) | \n", "5.733000e+07 | \n", "-0.156303 | \n", "3.3201 | \n", "|
Equity(47980 [BOOT]) | \n", "5.036520e+08 | \n", "0.108791 | \n", "923.0552 | \n", "|
Equity(48002 [FFWM]) | \n", "1.465090e+08 | \n", "-0.017641 | \n", "16.5062 | \n", "|
Equity(48019 [TBK]) | \n", "2.346070e+08 | \n", "-0.039882 | \n", "9.1637 | \n", "|
Equity(48090 [NDRM]) | \n", "1.966550e+08 | \n", "-0.238976 | \n", "0.1525 | \n", "|
Equity(48091 [VA]) | \n", "1.559080e+09 | \n", "-0.223353 | \n", "17.8565 | \n", "|
Equity(48103 [STOR]) | \n", "2.611000e+09 | \n", "0.062500 | \n", "44.1732 | \n", "|
Equity(48124 [WF]) | \n", "5.441790e+09 | \n", "-0.127706 | \n", "11.2718 | \n", "|
Equity(48126 [HABT]) | \n", "8.303110e+08 | \n", "0.021336 | \n", "47.1418 | \n", "|
Equity(48129 [UBS]) | \n", "6.324900e+10 | \n", "-0.021688 | \n", "16.6622 | \n", "|
Equity(48131 [NEFF]) | \n", "2.372420e+08 | \n", "-0.176889 | \n", "7.5635 | \n", "|
Equity(48139 [CPHR]) | \n", "3.714800e+08 | \n", "-0.046610 | \n", "13.9360 | \n", "|
Equity(48220 [LC]) | \n", "7.025730e+09 | \n", "-0.257708 | \n", "860.1009 | \n", "|
Equity(48252 [AVOL]) | \n", "1.582620e+09 | \n", "-0.010644 | \n", "14.0303 | \n", "|
Equity(48255 [MPG]) | \n", "1.234820e+09 | \n", "0.053026 | \n", "21.7141 | \n", "|
Equity(48258 [JRVR]) | \n", "6.139030e+08 | \n", "-0.063867 | \n", "9.1169 | \n", "
2270710 rows × 3 columns
\n", "\n", " | Equity(2 [AA]) | \n", "Equity(24 [AAPL]) | \n", "Equity(31 [ABAX]) | \n", "Equity(52 [ABM]) | \n", "Equity(62 [ABT]) | \n", "Equity(64 [ABX]) | \n", "Equity(66 [AB]) | \n", "Equity(67 [ADSK]) | \n", "Equity(69 [ACAT]) | \n", "Equity(76 [TAP]) | \n", "... | \n", "Equity(43476 [BERY]) | \n", "Equity(43479 [FLTX]) | \n", "Equity(43493 [ANFI]) | \n", "Equity(43494 [SSTK]) | \n", "Equity(43495 [AMBA]) | \n", "Equity(43512 [FANG]) | \n", "Equity(43514 [SHOS]) | \n", "Equity(43572 [WWAV]) | \n", "Equity(43599 [RH]) | \n", "Equity(43627 [RKUS]) | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2013-01-31 00:00:00+00:00 | \n", "9263400000 | \n", "4.996960e+11 | \n", "828393000 | \n", "1085530000 | \n", "49412800000 | \n", "35048800000 | \n", "1833170000 | \n", "7943140000 | \n", "440098000 | \n", "7513120000 | \n", "... | \n", "1678870000 | \n", "749643000 | \n", "288533000 | \n", "871338000 | \n", "290937000 | \n", "707182000 | \n", "752136000 | \n", "2688420000 | \n", "1279280000 | \n", "975600000 | \n", "
2013-02-28 00:00:00+00:00 | \n", "9434150000 | \n", "4.277320e+11 | \n", "852731000 | \n", "1085530000 | \n", "53214500000 | \n", "31955400000 | \n", "2141330000 | \n", "8693570000 | \n", "476344000 | \n", "8200170000 | \n", "... | \n", "1987300000 | \n", "867043000 | \n", "245378000 | \n", "845533000 | \n", "290937000 | \n", "829238000 | \n", "752136000 | \n", "2800870000 | \n", "1279280000 | \n", "1736040000 | \n", "
2013-03-31 00:00:00+00:00 | \n", "9110660000 | \n", "4.145000e+11 | \n", "937567000 | \n", "1240500000 | \n", "53073200000 | \n", "30273500000 | \n", "2422140000 | \n", "8217940000 | \n", "478849000 | \n", "8022940000 | \n", "... | \n", "2172600000 | \n", "821391000 | \n", "256434000 | \n", "1092660000 | \n", "269269000 | \n", "839964000 | \n", "1029570000 | \n", "2705720000 | \n", "1465930000 | \n", "1587240000 | \n", "
2013-04-30 00:00:00+00:00 | \n", "9110660000 | \n", "4.161420e+11 | \n", "1046720000 | \n", "1215370000 | \n", "55059100000 | \n", "29433900000 | \n", "2309700000 | \n", "9231750000 | \n", "577001000 | \n", "8924830000 | \n", "... | \n", "2154440000 | \n", "876024000 | \n", "227074000 | \n", "1507590000 | \n", "423369000 | \n", "992719000 | \n", "932085000 | \n", "2953110000 | \n", "1328870000 | \n", "1560850000 | \n", "
2013-05-31 00:00:00+00:00 | \n", "9089870000 | \n", "4.156150e+11 | \n", "944303000 | \n", "1215370000 | \n", "57553300000 | \n", "19732700000 | \n", "2498490000 | \n", "8813240000 | \n", "594034000 | \n", "9411840000 | \n", "... | \n", "2148790000 | \n", "847847000 | \n", "219620000 | \n", "1397660000 | \n", "423369000 | \n", "971266000 | \n", "1030490000 | \n", "2925430000 | \n", "1328870000 | \n", "1435260000 | \n", "
5 rows × 3203 columns
\n", "\n", " | Equity(2 [AA]) | \n", "Equity(24 [AAPL]) | \n", "Equity(31 [ABAX]) | \n", "Equity(52 [ABM]) | \n", "Equity(62 [ABT]) | \n", "Equity(64 [ABX]) | \n", "Equity(66 [AB]) | \n", "Equity(67 [ADSK]) | \n", "Equity(69 [ACAT]) | \n", "Equity(76 [TAP]) | \n", "... | \n", "Equity(43476 [BERY]) | \n", "Equity(43479 [FLTX]) | \n", "Equity(43493 [ANFI]) | \n", "Equity(43494 [SSTK]) | \n", "Equity(43495 [AMBA]) | \n", "Equity(43512 [FANG]) | \n", "Equity(43514 [SHOS]) | \n", "Equity(43572 [WWAV]) | \n", "Equity(43599 [RH]) | \n", "Equity(43627 [RKUS]) | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2013-01-31 00:00:00+00:00 | \n", "9.0 | \n", "14 | \n", "3 | \n", "1.5 | \n", "1 | \n", "25 | \n", "2 | \n", "2 | \n", "4 | \n", "1 | \n", "... | \n", "1 | \n", "1 | \n", "9 | \n", "2 | \n", "2.5 | \n", "1 | \n", "18.5 | \n", "1 | \n", "1.5 | \n", "5 | \n", "
2013-02-28 00:00:00+00:00 | \n", "10.0 | \n", "7 | \n", "6 | \n", "1.5 | \n", "5 | \n", "24 | \n", "7 | \n", "7 | \n", "8 | \n", "3 | \n", "... | \n", "2 | \n", "4 | \n", "5 | \n", "1 | \n", "2.5 | \n", "2 | \n", "18.5 | \n", "3 | \n", "1.5 | \n", "25 | \n", "
2013-03-31 00:00:00+00:00 | \n", "7.5 | \n", "3 | \n", "11 | \n", "5.0 | \n", "4 | \n", "23 | \n", "14 | \n", "5 | \n", "9 | \n", "2 | \n", "... | \n", "5 | \n", "2 | \n", "7 | \n", "3 | \n", "1.0 | \n", "3 | \n", "23.0 | \n", "2 | \n", "5.0 | \n", "24 | \n", "
2013-04-30 00:00:00+00:00 | \n", "7.5 | \n", "5 | \n", "17 | \n", "3.5 | \n", "7 | \n", "22 | \n", "10 | \n", "11 | \n", "14 | \n", "5 | \n", "... | \n", "4 | \n", "5 | \n", "2 | \n", "5 | \n", "5.5 | \n", "5 | \n", "20.0 | \n", "6 | \n", "3.5 | \n", "23 | \n", "
2013-05-31 00:00:00+00:00 | \n", "5.5 | \n", "4 | \n", "14 | \n", "3.5 | \n", "12 | \n", "12 | \n", "18 | \n", "8 | \n", "15 | \n", "11 | \n", "... | \n", "3 | \n", "3 | \n", "1 | \n", "4 | \n", "5.5 | \n", "4 | \n", "24.0 | \n", "5 | \n", "3.5 | \n", "22 | \n", "
5 rows × 3203 columns
\n", "