{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### 一.算法推导\n", "Gibbs算法其实就是单分量MH算法中，建议分布取满条件概率分布的特殊情况，即： \n", "\n", "$$\n", "q(x_j^{(i-1)}\\rightarrow x_j^{'(i)} \\mid x_{-j}^{(i)})=p(x_j^{'(i)} \\mid x_{-j}^{(i)})\n", "$$ \n", "\n", "那自然的，我们接下来还会关心接收概率$\\alpha(x_j^{(i-1)}\\rightarrow x_j^{'(i)}\\mid x_{-j}^{(i)})$的情况： \n", "\n", "$$\n", "\\alpha(x_j^{(i-1)}\\rightarrow x_j^{'(i)}\\mid x_{-j}^{(i)})=min\\{1,\\frac{p(x_j^{'(i)}\\mid x_{-j}^{(i)})q(x_j^{'(i)}\\rightarrow x_j^{(i-1)}\\mid x_{-j}^{(i)})}{p(x_j^{(i-1)}\\mid x_{-j}^{(i)})q(x_j^{(i-1)}\\rightarrow x_j^{'(i)}\\mid x_{-j}^{(i)})}\\}=min\\{1,\\frac{p(x_j^{'(i)}\\mid x_{-j}^{(i)})p(x_j^{(i-1)}\\mid x_{-j}^{(i)})}{p(x_j^{(i-1)}\\mid x_{-j}^{(i)})p(x_j^{'(i)}\\mid x_{-j}^{(i)})}\\}=min\\{1,1\\}=1\n", "$$ \n", "\n", "接受率竟然始终为1，始终不会拒绝样本！所以Gibbs采样是一种特别高效的方法，下面直接说下采样流程" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 二.Gibbs采样流程\n", "输入：目标概率分布的密度函数$p(x)$，正整数$m,n,m（1）初始化样本$x^{(0)}=(x_1^{(0)},x_2^{(0)},...,x_k^{(0)})$ \n", "\n", ">（2）对$i=1,2,...,m,m+1,...n$： \n", "\n", ">>对$j=1,2,...,k$： \n", "\n", ">>>根据建议分布$p(x_j\\mid x_{-j}^{(i)})$抽样第$j$维数据$x_j^{'(i)}$，并令$x_j^{(i)}=x_j^{'(i)}$ \n", " \n", "\n", ">（3）返回样本集$\\{x_{m+1},x_{m+2},...,x_n\\}$ " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 三.案例\n", "例子就用前一节的，直观感受一下与单分量MH的不同...公式部分就不码了..." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "os.chdir('../')\n", "from ml_models import utils\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "#定义均值，协方差\n", "u=np.asarray([0,0])\n", "sigma=np.asarray([[1,0.5],\n", " [0.5,1]])" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import copy\n", "#采样的样本量\n", "nums=1000\n", "count=0\n", "points=[]\n", "#采样x0\n", "point=[np.random.randn(),np.random.randn()]\n", "points.append(point)\n", "while count" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "utils.plot_contourf(np.asarray(points[-400:]),lambda x:utils.gaussian_nd(x,u,sigma),8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 四.小结\n", "对比一下Gibbs采样和单分量MH直接的异同点： \n", "\n", "先说相同的地方：（1）两者都是要在满条件分布可用的前提下才能使用； （2）Gibbs采样是单分量MH的特殊情况，即建议分布取满条件概率分布的情况； \n", "\n", "再说不同的地方：（1）满条件概率分布好采样时用Gibbs，不好采用时用单分量MH；（2）单分量MH有拒绝抽样的情况，而Gibbs采样全部接受 \n", "\n", "下面再补充一副关系图对抽样这一章的内容做总结 \n", "\n", "\n", "![avatar](./source/12_MCMC总结.png)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" } }, "nbformat": 4, "nbformat_minor": 2 }