{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Quickstart or _\"How to get 100% return per year\"_" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, do some initialization and set debugging level to `debug` to see progress of computation." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "%load_ext autoreload\n", "%autoreload 2\n", "%config InlineBackend.figure_format = 'svg'\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import seaborn as sns\n", "import datetime as dt\n", "import matplotlib.pyplot as plt\n", "\n", "import universal as up\n", "from universal import tools, algos\n", "from universal.algos import *\n", "\n", "sns.set_context(\"notebook\")\n", "plt.rcParams[\"figure.figsize\"] = (16, 8)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# ignore logged warnings\n", "import logging\n", "logging.getLogger().setLevel(logging.ERROR)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's try to replicate the results of B.Li and S.Hoi from their article [On-Line Portfolio Selection with Moving Average Reversion](http://arxiv.org/abs/1206.4626). They claim superior performance on several datasets using their OLMAR algorithm. These datasets are available in `data/` directory in `.pkl` format. Those are all relative prices (start with 1.) and artificial tickers. We can start with NYSE stocks from period 1/1/1985 - 30/6/2010." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# load data using tools module\n", "data = tools.dataset('nyse_o')\n", "\n", "# plot first three of them as example\n", "data.iloc[:,:3].plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we need an implementation of the OLMAR algorithm. Fortunately, it is already implemented in module `algos`, so all we have to do is load it and set its parameters. Authors recommend lookback window $w = 5$ and threshold $\\epsilon = 10$ (these are default parameters anyway). Just call `run` method on our data to get results for analysis." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "# set algo parameters\n", "algo = algos.OLMAR(window=5, eps=10)\n", "\n", "# run\n", "result = algo.run(data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ok, let's see some results. First print some basic summary metrics and plot portfolio equity with UCRP (uniform constant rebalanced portfolio)." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Summary:\n", " Profit factor: 1.89\n", " Sharpe ratio: 3.34 ± 0.54\n", " Ulcer index: 23.19\n", " Information ratio (wrt UCRP): 3.27\n", " Appraisal ratio (wrt UCRP): 3.05 ± 0.21\n", " UCRP sharpe: 1.16 ± 0.27\n", " Beta / Alpha: 1.55 / 165.350%\n", " Annualized return: 466.14%\n", " Annualized volatility: 56.69%\n", " Longest drawdown: 185 days\n", " Max drawdown: 46.29%\n", " Winning days: 58.3%\n", " Annual turnover: 349.1\n", " \n" ] }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "print(result.summary())\n", "result.plot(weights=False, assets=False, ucrp=True, logy=True);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That seems really impressive, in fact it looks too good to be true. Let's see how individual stocks contribute to portfolio equity and disable legend to keep the graph clean." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "result.plot_decomposition(legend=False, logy=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, almost all wealth comes from single stock (don't forget it has logarithm scale!). So if we used just 5 of all these stocks, we would get almost the same equity as if we used all of them. To stress test the strategy, we can remove that stock and rerun the algorithm." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Summary:\n", " Profit factor: 1.55\n", " Sharpe ratio: 2.48 ± 0.43\n", " Ulcer index: 12.67\n", " Information ratio (wrt UCRP): 2.36\n", " Appraisal ratio (wrt UCRP): 2.14 ± 0.21\n", " UCRP sharpe: 1.12 ± 0.27\n", " Beta / Alpha: 1.48 / 96.840%\n", " Annualized return: 192.81%\n", " Annualized volatility: 47.90%\n", " Longest drawdown: 202 days\n", " Max drawdown: 45.91%\n", " Winning days: 56.5%\n", " Annual turnover: 346.1\n", " \n" ] }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# find name of the most profitable asset\n", "most_profitable = result.equity_decomposed.iloc[-1].idxmax()\n", "\n", "# rerun an algorithm on data without it\n", "result_without = algo.run(data.drop([most_profitable], 1))\n", "\n", "# and print results\n", "print(result_without.summary())\n", "result_without.plot(weights=False, assets=False, ucrp=True, logy=True);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We lost about 7 orders of wealth, but the results are more realistic now. Let's move on and try adding fees of 0.1% per transaction (we pay \\\\$1 for every \\\\$1000 of stocks bought or sold)." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Summary:\n", " Profit factor: 1.35\n", " Sharpe ratio: 1.79 ± 0.34\n", " Ulcer index: 6.76\n", " Information ratio (wrt UCRP): 1.61\n", " Appraisal ratio (wrt UCRP): 1.41 ± 0.21\n", " UCRP sharpe: 1.12 ± 0.27\n", " Beta / Alpha: 1.47 / 62.608%\n", " Annualized return: 108.72%\n", " Annualized volatility: 47.17%\n", " Longest drawdown: 382 days\n", " Max drawdown: 50.09%\n", " Winning days: 50.2%\n", " Annual turnover: 346.1\n", " \n" ] }, { "data": { "text/plain": [ "[]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "result_without.fee = 0.001\n", "print(result_without.summary())\n", "result_without.plot(weights=False, assets=False, ucrp=True, logy=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Results still hold, although our Sharpe ratio decreased from 3.14 to 1.56 and annualized return from 466% to 109%. Now some of you trained in quantitative finance might start asking: \"_Isn't there some [survivorship bias](http://en.wikipedia.org/wiki/Survivorship_bias)?_\". Yes, it is. In fact, a huge one considering that we have almost 25 years of data and mean-reversion type of strategy." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Testing Yahoo data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's see whether the algo works on recent data, too. First download closing prices of several (randomly chosen) stocks from Yahoo." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from pandas_datareader.data import DataReader\n", "\n", "# load data from Yahoo\n", "yahoo_data = DataReader(['MSFT', 'IBM', 'AAPL', 'GOOG'], 'yahoo', start=dt.datetime(2005,1,1))['Adj Close']\n", "\n", "# plot normalized prices of these stocks\n", "(yahoo_data / yahoo_data.iloc[0,:]).plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instead of using fixed parameters, we will test several `window` parameters with function `run_combination`. It the same as `run`, just use it as classmethod and use lists for combination of values. `run_combination` returns list of results which can be used similarly to `result`." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Summary for window=3:\n", " Profit factor: 1.09\n", " Sharpe ratio: 0.61 ± 0.27\n", " Ulcer index: 1.07\n", " Information ratio (wrt UCRP): -0.13\n", " Appraisal ratio (wrt UCRP): -0.20 ± 0.25\n", " UCRP sharpe: 0.95 ± 0.30\n", " Beta / Alpha: 1.07 / -4.173%\n", " Annualized return: 14.99%\n", " Annualized volatility: 30.87%\n", " Longest drawdown: 649 days\n", " Max drawdown: 51.24%\n", " Winning days: 51.4%\n", " Annual turnover: 284.1\n", " \n", "Summary for window=5:\n", " Profit factor: 1.12\n", " Sharpe ratio: 0.74 ± 0.29\n", " Ulcer index: 1.55\n", " Information ratio (wrt UCRP): 0.07\n", " Appraisal ratio (wrt UCRP): -0.01 ± 0.25\n", " UCRP sharpe: 0.95 ± 0.30\n", " Beta / Alpha: 1.06 / -0.112%\n", " Annualized return: 19.64%\n", " Annualized volatility: 30.50%\n", " Longest drawdown: 806 days\n", " Max drawdown: 48.11%\n", " Winning days: 52.1%\n", " Annual turnover: 215.5\n", " \n", "Summary for window=10:\n", " Profit factor: 1.15\n", " Sharpe ratio: 0.88 ± 0.30\n", " Ulcer index: 1.89\n", " Information ratio (wrt UCRP): 0.27\n", " Appraisal ratio (wrt UCRP): 0.21 ± 0.25\n", " UCRP sharpe: 0.95 ± 0.30\n", " Beta / Alpha: 1.03 / 4.371%\n", " Annualized return: 24.43%\n", " Annualized volatility: 29.94%\n", " Longest drawdown: 401 days\n", " Max drawdown: 53.78%\n", " Winning days: 52.0%\n", " Annual turnover: 153.3\n", " \n", "Summary for window=15:\n", " Profit factor: 1.10\n", " Sharpe ratio: 0.66 ± 0.28\n", " Ulcer index: 1.28\n", " Information ratio (wrt UCRP): -0.08\n", " Appraisal ratio (wrt UCRP): -0.10 ± 0.25\n", " UCRP sharpe: 0.95 ± 0.30\n", " Beta / Alpha: 1.03 / -2.095%\n", " Annualized return: 16.60%\n", " Annualized volatility: 30.02%\n", " Longest drawdown: 627 days\n", " Max drawdown: 57.16%\n", " Winning days: 51.6%\n", " Annual turnover: 117.6\n", " \n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "list_result = algos.OLMAR.run_combination(yahoo_data, window=[3,5,10,15], eps=10)\n", "print(list_result.summary())\n", "list_result.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since we don't know the best parameters in hindsight, we will invest equal money in each of them in the beginning and let them run. This is called _buy and hold_ strategy. Portfolio equities in `list_result` can be regarded as stock prices and used as an input for new algo (_buy and hold_ in this case). This way you can chain algorithms however you like, for example OLMAR on OLMAR, etc.\n", "\n", "To compare it with individual assets or uniform constant rebalanced portfolio, use parameters `assets` and `ucrp`." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# run buy and hold on OLMAR results and show its equity together with original assets\n", "algos.BAH().run(list_result).plot(assets=True, weights=False, ucrp=True);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ok, so that was enough for the start. There are plenty of other algorithms in module `algos` collected across research papers about online-portfolios including famous [Universal portfolio](http://en.wikipedia.org/wiki/Universal_portfolio_algorithm) by Thomas Cover." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# How to write your own algorithm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Entire package is actually pretty simple. Algorithms are subclasses of base `Algo` class and methods for reporting, plotting and analysing are built on top of this class. I will illustrate it on this mean-reversion strategy\n", "\n", "1. use logarithm of price\n", "2. calculate difference $\\delta_i$ between current price of $i$-th stock and its moving average of $n$ days\n", "3. if $\\delta_i > 0$, assign zero portfolio weight $w_i = 0$ for $i$-th stock\n", "4. if $\\delta_i < 0$, assign weight $w_i = -\\delta_i$ for $i$-th stock\n", "5. normalize all weights so that $\\sum w_i = 1$\n", "\n", "The idea is that badly performing stocks will revert to its mean and have higher returns than those above their mean. Here is the complete code, comments should be self-explanatory." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "from universal.algo import Algo\n", "import numpy as np\n", "\n", "class MeanReversion(Algo):\n", " # use logarithm of prices\n", " PRICE_TYPE = 'log'\n", " \n", " def __init__(self, n):\n", " # length of moving average\n", " self.n = n\n", " # step function will be called after min_history days\n", " super(MeanReversion, self).__init__(min_history=n)\n", " \n", " def init_weights(self, cols):\n", " # use zero weights for start\n", " return pd.Series(np.zeros(len(cols)), cols)\n", " \n", " def step(self, x, last_b, history):\n", " # calculate moving average\n", " ma = history.iloc[-self.n:].mean()\n", " \n", " # weights\n", " delta = x - ma\n", " w = np.maximum(-delta, 0.)\n", " \n", " # normalize so that they sum to 1\n", " return w / sum(w)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's all. Now let's try it on nyse data." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Summary:\n", " Profit factor: 1.46\n", " Sharpe ratio: 2.13 ± 0.38\n", " Ulcer index: 4.84\n", " Information ratio (wrt UCRP): 1.94\n", " Appraisal ratio (wrt UCRP): 1.67 ± 0.21\n", " UCRP sharpe: 1.16 ± 0.27\n", " Beta / Alpha: 1.15 / 32.157%\n", " Annualized return: 60.46%\n", " Annualized volatility: 23.47%\n", " Longest drawdown: 808 days\n", " Max drawdown: 41.29%\n", " Winning days: 52.4%\n", " Annual turnover: 365.7\n", " \n" ] }, { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n" ], "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "mr = MeanReversion(n=20)\n", "result = mr.run(data)\n", "\n", "print(result.summary())\n", "result.plot(assets=False, logy=True, weights=False, ucrp=True);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Not bad considering how simple that strategy is. Next step could be performance optimization. To profile your strategy, you can use function `profile` in `universal.tools` which profile the code using fantastic [line_profiler](http://pythonhosted.org/line_profiler/). After identifying the most critical parts of the code, you have two options. Either optimize your `step` function (using tools such as [weave](http://docs.scipy.org/doc/scipy/reference/tutorial/weave.html), [numba](http://numba.pydata.org/), [theano](http://deeplearning.net/software/theano/) or [cython](http://cython.org/)) or subclass `weights` method if your code could be vectorized easily (beware the forward bias!). " ] } ], "metadata": { "kernelspec": { "display_name": "universal", "language": "python", "name": "universal" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 4 }