{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Lasso regression with block updating\n",
"\n",
"Sometimes, it is very useful to update a set of parameters together. For example, variables that are highly correlated are often good to update together. In PyMC 3 block updating is simple, as example will demonstrate.\n",
"\n",
"Here we have a LASSO regression model where the two coefficients are strongly correlated. Normally, we would define the coefficient parameters as a single random variable, but here we define them separately to show how to do block updates.\n",
"\n",
"First we generate some fake data."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline\n",
"from matplotlib.pylab import *\n",
"from pymc3 import * \n",
"import numpy as np \n",
"\n",
"d = np.random.normal(size=(3, 30))\n",
"d1 = d[0] + 4\n",
"d2 = d[1] + 4\n",
"yd = .2*d1 +.3*d2 + d[2]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then define the random variables."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"lam = 3\n",
"\n",
"with Model() as model:\n",
" s = Exponential('s', 1)\n",
" tau = Uniform('tau', 0, 1000)\n",
" b = lam * tau\n",
" m1 = Laplace('m1', 0, b)\n",
" m2 = Laplace('m2', 0, b)\n",
" \n",
" p = d1*m1 + d2*m2\n",
" \n",
" y = Normal('y', mu=p, sigma=s, observed=yd) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For most samplers, including Metropolis and HamiltonianMC, simply pass a list of variables to sample as a block. This works with both scalar and array parameters. "
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"