\n",
"\n",
"conda packages are available on anaconda cloud for python 2.7, 3.5 and 3.6 on MacOS, Linux and Windows.\n",
"\n",
"pip packages are available on [pypi](https://pypi.python.org/pypi/pymcef) for both python 2.7, 3.5 and 3.6 on MacOs and Windows.\n",
"\n",
"pypi doesn't support linux binary distributions, to install from pip on linux please download the wheel files from [this repo](https://github.com/hzzyyy/pymcef/tree/master/linux_x86_64_wheels) and install locally.\n",
"\n",
"
\n",
"\n",
"This package is only available on 64 bits OS, in addition, C++11 runtime library is alse required. \n",
"\n",
"As result, this package will **NOT** work on Redhat EL6 with default configuration.\n",
"\n",
"If you mainly work with Redhat you can either upgrade to EL7 or customize EL6 with [C++11 support.](https://access.redhat.com/documentation/en/red-hat-developer-toolset/) \n",
"\n",
"The implementation quality of PyMCEF is validated through several [benchmarks](http://nbviewer.jupyter.org/github/hzzyyy/pymcef/blob/master/Benchmark.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Efficient frontier\n",
"This package helps to find the efficient frontier based on (jointly) simulated returns of all investible assets.\n",
"\n",
"Alternatively speaking, this package tries to solve this problem numerically:\n",
"\n",
"> Given the joint distribution of the returns of all the investible assets, what should be the\n",
"> best choice of weights to assign on each assets, for all possible risk preferences.\n",
"\n",
"Efficient frontier is the set of all 'best' choices of asset weights (a.k.a. portfolio) with all possible risk preferences.\n",
"\n",
"Heuristically, the most risk-taking portfolio simply puts all weight on the asset with largest expected return. \n",
"\n",
"The most risk-averse portfolio is very diversified, with many positions hedging each other."
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"### Portfolio optimization\n",
"\n",
"Each portfolio on the efficient frontier is obtained by solving the following optimization problem on the choice of weight vector $w$:\n",
"\n",
"\\begin{align*}\n",
"\\underset{w}{\\mathrm{argmin}}\\quad & \\mathrm{Risk}\\left(w\\right)-\\lambda\\cdot\\mathrm{Reward}\\left(w\\right),\\\\\n",
"\\mathrm{subject\\ to}\\quad & \\sum_{i}w_{i} = 1,\\\\\n",
" & \\ \\ \\ \\quad w_{i} > 0.\n",
"\\end{align*}\n",
"\n",
"The Lagrangian multiplier $\\lambda$ here uniquely defines a risk preference and its corresponding solution. \n",
"With $\\lambda$ going to infinity the solution is the most risk-taking portfolio and with $\\lambda$ being zero, the solution is the most risk-averse portfolio."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Risk measures\n",
"\n",
"We haven't defined the formulas for the risk and reward functions in the above optimization problem. Not surprisingly, the reward function is the just the expected return of the whole portfolio. Suppose the returns of all the asserts is the random vector: \n",
"\n",
"$$ Y = \\{Y_i\\}. $$ \n",
"\n",
"Then the reward function is simply:\n",
"\n",
"\\begin{eqnarray*}\n",
"\\mathrm{Reward}\\left(w\\right) & = & \\mathbb{E}\\left[\\sum_{i}w_{i}Y_{i}\\right]\\\\\n",
" & = & \\mathbb{E}\\left[X\\right]\\quad\\mathrm{namely}\n",
"\\end{eqnarray*}\n",
"\n",
"The less obviously obtained risk function, formally known as risk measure, is a function of the portfolio return random variable $X$. \n",
"In the Markowitz mean-variance model, the risk measure is the variance of $X$, which is theoretically flawed ([check this example](http://nbviewer.jupyter.org/github/hzzyyy/pymcef/blob/master/examples/1.%20When%20mean%20variance%20models%20fail.ipynb)). In this PyMCEF, the following two more sophisticated risk measures are used:\n",
"\n",
"#### Absolute Semideviation\n",
"\n",
"\\begin{eqnarray*}\n",
"\\mathrm{Risk}\\left(w\\right) & = & \\mathbb{E}\\left[\\max\\left(\\mathbb{E}X-X,\\ 0\\right)\\right]\\\\\n",
" & = & \\mathbb{E}\\left[\\left(\\mathbb{E}X-X\\right)^{+}\\right].\n",
"\\end{eqnarray*}\n",
"\n",
"#### Fixed-target under-performance\n",
"\n",
"\\begin{eqnarray*}\n",
"\\mathrm{Risk}\\left(w\\right) & = & \\mathbb{E}\\left[\\max\\left(t-X,\\ 0\\right)\\right]\\\\\n",
" & = & \\mathbb{E}\\left[\\left(t-X\\right)^{+}\\right],\n",
"\\end{eqnarray*}\n",
"\n",
"where $t$ is given target. The riskless return is a sensible choice. \n",
"\n",
"According to axiomatized portfolio theory, these two risk measures are better than the variance because they are more consistent with the stochastic dominance rule. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Stochastic programming\n",
"\n",
"The (joint) distribution of the returns of assets $\\{Y_i\\}$ can be parametric. A convenient\n",
"choice is to assume Normal or log Normal distribution. More complicated ways of\n",
"parameterizations for sure exist, and they lead to different optimization problems. \n",
"In addition, with $\\lambda$ varying as a positive real number, we face a continuum of\n",
"optimization problems.\n",
"\n",
"Alternatively, we can work with the (finite) samples of $\\{Y_i\\}$.\n",
"In this case, we replace the expectation in the risk or reward function with statistical mean and work with a stochastic programming problem. \n",
"PyMCEF is implemented in this way, and there is huge advantage in terms of flexibility. \n",
"The input for PyMCEF is just Monte Carlo simulated returns, and no knowledge about the underlying distribution function is needed for this package to work. \n",
"In Bayesian inference, it is a common practice to directly simulate the posterior predictive\n",
"distribution with Markov Chain Monte Carlo, where the posterior density function is known only \n",
"up to a scaling factor. \n",
"In another word, in practice, samples of distribution is more available than its corresponding parametric form.\n",
"\n",
"Another benefit comes with the linearization of the problem. With slack variables, \n",
"the constraint and objective function can be linearized such that the solution is valid for an interval of $\\lambda$.\n",
"As a result, the (approximated) efficient frontier has only finite number of portfolios.\n",
"With stochastic programming, we solve an approximated problem whose solution is much easier to describe."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## One example\n",
"---------\n",
"\n",
"We take the daily returns of US stocks in Russell 3000 from 2015-01-02 to 2017-01-04.\n",
"Each asset has 505 returns, which are treated as simulated daily returns.\n",
"The direct use of historical data are not Monter Carlo, however it is good enough to demonstrate how to use PyMCEF.\n",
"We can consider this predictive distribution as a largely over fitted model with not much predictive power.\n",
"However, this is good starting point for better models.\n",
"\n",
"We removed stocks with prices lower than $5 (pink sheet) to avoid liquidity problem. The data is stored in this git repository and can be loaded into a pandas data frame:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import pandas as pd\n",
"returns = pd.read_json('data/Russel3k_return.json')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Instead of smoothing, we directly exclude those stocks with extremely large jump in prices (possibly erroneous).\n",
"Here, a gain or loss more than four times the 99% or 1% quantile is considered as extreme."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"returns_filted = returns[(returns > returns.quantile(0.01) * 4) & \\\n",
" (returns < returns.quantile(0.99) * 4)].dropna(axis=1)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"After this filtering process, the number of remaining assets is:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2083"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(returns_filted.columns)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next we use this return data as both training set and validation set to construct an efficient frontier with PyMCEF.\n",
"The risk measure is not specified and the default absolute semi-deviation is used."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"14.947410821914673\n"
]
}
],
"source": [
"from time import time\n",
"import numpy as np\n",
"from pymcef import SimpleEFp\n",
"tic = time()\n",
"frt = SimpleEFp(training_set=np.transpose(returns_filted.values),\\\n",
" validation_set=np.transpose(returns_filted.values),\\\n",
" asset_names=returns_filted.columns)\n",
"print(time() - tic)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As we can see, the full efficient frontier is obtained in less than one minute. Let's take a look at the efficient frontier:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"%config InlineBackend.figure_format = 'svg'\n",
"fig_ef = frt.plot_ef()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The performances of all portfolio is also visualized:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig_pf = frt.plot_performance()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And how the weights vary with different values of $\\lambda$:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig_ws = frt.plot_weights()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"All the weights are stored in the instance of SimpleEFp.\n",
"The first one is the most risk-seeking: "
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"EGRX 1.0\n"
]
}
],
"source": [
"from __future__ import print_function\n",
"prt0 = frt.frontier_in_sample[0]\n",
"for k, v in prt0['weight'].items():\n",
" print(frt.asset_names[k], v)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Starting from the second portfolio, there are more than one assets:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"STRP 0.0014850743788279803\n",
"EGRX 0.9985149256211721\n"
]
}
],
"source": [
"prt1 = frt.frontier_in_sample[1]\n",
"for k, v in prt1['weight'].items():\n",
" print(frt.asset_names[k], v)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"STRP 0.0051277953990925785\n",
"EGRX 0.983506565427417\n",
"NVDA 0.011365639173490446\n"
]
}
],
"source": [
"prt3 = frt.frontier_in_sample[3]\n",
"for k, v in prt3['weight'].items():\n",
" print(frt.asset_names[k], v)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And the number of assets contained in the most risk-averse portfolio is:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"51\n"
]
}
],
"source": [
"print(len(frt.frontier_in_sample[-1]['weight']))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"frt.frontier_in_sample stores all the main results of the efficient frontier construction,\n",
"the user can conduct their own studies with all the weights in it. "
]
}
],
"metadata": {
"anaconda-cloud": {},
"hide_input": false,
"kernelspec": {
"display_name": "Python [conda env:jupyter]",
"language": "python",
"name": "conda-env-jupyter-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}