{
 "metadata": {
  "name": "tsa_filters"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Filtering Time Series Data"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import pandas\n",
      "import matplotlib.pyplot as plt\n",
      "\n",
      "import statsmodels.api as sm"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dta = sm.datasets.macrodata.load_pandas().data"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "index = pandas.Index(sm.tsa.datetools.dates_from_range('1959Q1', '2009Q3'))\n",
      "print index"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "dta.index = index\n",
      "del dta['year']\n",
      "del dta['quarter']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print sm.datasets.macrodata.NOTE"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print dta.head(10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig = plt.figure(figsize=(12,8))\n",
      "ax = fig.add_subplot(111)\n",
      "dta.realgdp.plot(ax=ax);\n",
      "legend = ax.legend(loc = 'upper left');\n",
      "legend.prop.set_size(20);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Hodrick-Prescott Filter"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The Hodrick-Prescott filter separates a time-series $y_t$ into a trend $\\tau_t$ and a cyclical component $\\zeta_t$ \n",
      "\n",
      "$$y_t = \\tau_t + \\zeta_t$$\n",
      "\n",
      "The components are determined by minimizing the following quadratic loss function\n",
      "\n",
      "$$\\min_{\\\\{ \\tau_{t}\\\\} }\\sum_{t}^{T}\\zeta_{t}^{2}+\\lambda\\sum_{t=1}^{T}\\left[\\left(\\tau_{t}-\\tau_{t-1}\\right)-\\left(\\tau_{t-1}-\\tau_{t-2}\\right)\\right]^{2}$$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "gdp_cycle, gdp_trend = sm.tsa.filters.hpfilter(dta.realgdp)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "gdp_decomp = dta[['realgdp']]\n",
      "gdp_decomp[\"cycle\"] = gdp_cycle\n",
      "gdp_decomp[\"trend\"] = gdp_trend"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig = plt.figure(figsize=(12,8))\n",
      "ax = fig.add_subplot(111)\n",
      "gdp_decomp[[\"realgdp\", \"trend\"]][\"2000-03-31\":].plot(ax=ax, fontsize=16);\n",
      "legend = ax.get_legend()\n",
      "legend.prop.set_size(20);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Baxter-King approximate band-pass filter: Inflation and Unemployment"
     ]
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Explore the hypothesis that inflation and unemployment are counter-cyclical."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The Baxter-King filter is intended to explictly deal with the periodicty of the business cycle. By applying their band-pass filter to a series, they produce a new series that does not contain fluctuations at higher or lower than those of the business cycle. Specifically, the BK filter takes the form of a symmetric moving average \n",
      "\n",
      "$$y_{t}^{*}=\\sum_{k=-K}^{k=K}a_ky_{t-k}$$\n",
      "\n",
      "where $a_{-k}=a_k$ and $\\sum_{k=-k}^{K}a_k=0$ to eliminate any trend in the series and render it stationary if the series is I(1) or I(2).\n",
      "\n",
      "For completeness, the filter weights are determined as follows\n",
      "\n",
      "$$a_{j} = B_{j}+\\theta\\text{ for }j=0,\\pm1,\\pm2,\\dots,\\pm K$$\n",
      "\n",
      "$$B_{0} = \\frac{\\left(\\omega_{2}-\\omega_{1}\\right)}{\\pi}$$\n",
      "$$B_{j} = \\frac{1}{\\pi j}\\left(\\sin\\left(\\omega_{2}j\\right)-\\sin\\left(\\omega_{1}j\\right)\\right)\\text{ for }j=0,\\pm1,\\pm2,\\dots,\\pm K$$\n",
      "\n",
      "where $\\theta$ is a normalizing constant such that the weights sum to zero.\n",
      "\n",
      "$$\\theta=\\frac{-\\sum_{j=-K^{K}b_{j}}}{2K+1}$$\n",
      "\n",
      "$$\\omega_{1}=\\frac{2\\pi}{P_{H}}$$\n",
      "\n",
      "$$\\omega_{2}=\\frac{2\\pi}{P_{L}}$$\n",
      "\n",
      "$P_L$ and $P_H$ are the periodicity of the low and high cut-off frequencies. Following Burns and Mitchell's work on US business cycles which suggests cycles last from 1.5 to 8 years, we use $P_L=6$ and $P_H=32$ by default."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bk_cycles = sm.tsa.filters.bkfilter(dta[[\"infl\",\"unemp\"]])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "* We lose K observations on both ends. It is suggested to use K=12 for quarterly data."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig = plt.figure(figsize=(14,10))\n",
      "ax = fig.add_subplot(111)\n",
      "bk_cycles.plot(ax=ax, style=['r--', 'b-']);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Christiano-Fitzgerald approximate band-pass filter: Inflation and Unemployment"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The Christiano-Fitzgerald filter is a generalization of BK and can thus also be seen as weighted moving average. However, the CF filter is asymmetric about $t$ as well as using the entire series. The implementation of their filter involves the\n",
      "calculations of the weights in\n",
      "\n",
      "$$y_{t}^{*}=B_{0}y_{t}+B_{1}y_{t+1}+\\dots+B_{T-1-t}y_{T-1}+\\tilde B_{T-t}y_{T}+B_{1}y_{t-1}+\\dots+B_{t-2}y_{2}+\\tilde B_{t-1}y_{1}$$\n",
      "\n",
      "for $t=3,4,...,T-2$, where\n",
      "\n",
      "$$B_{j} = \\frac{\\sin(jb)-\\sin(ja)}{\\pi j},j\\geq1$$\n",
      "\n",
      "$$B_{0} = \\frac{b-a}{\\pi},a=\\frac{2\\pi}{P_{u}},b=\\frac{2\\pi}{P_{L}}$$\n",
      "\n",
      "$\\tilde B_{T-t}$ and $\\tilde B_{t-1}$ are linear functions of the $B_{j}$'s, and the values for $t=1,2,T-1,$ and $T$ are also calculated in much the same way. $P_{U}$ and $P_{L}$ are as described above with the same interpretation."
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "The CF filter is appropriate for series that may follow a random walk."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print sm.tsa.stattools.adfuller(dta['unemp'])[:3]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print sm.tsa.stattools.adfuller(dta['infl'])[:3]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "cf_cycles, cf_trend = sm.tsa.filters.cffilter(dta[[\"infl\",\"unemp\"]])\n",
      "print cf_cycles.head(10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "fig = plt.figure(figsize=(14,10))\n",
      "ax = fig.add_subplot(111)\n",
      "cf_cycles.plot(ax=ax, style=['r--','b-']);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Filtering assumes *a priori* that business cycles exist. Due to this assumption, many macroeconomic models seek to create models that match the shape of impulse response functions rather than replicating properties of filtered series. See VAR notebook."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}