# Bayesian Inference 🔒 Full text at this link requires payment or institutional access. Unmarked links may be unchecked. ## Foundations and workflow - **Bayesian Data Analysis** - Third-edition reference on hierarchical models, computation, model checking and predictive evaluation; the linked electronic edition incorporates later corrections. [Gelman et al. 2013]( "Bayesian Data Analysis; book"). - **Bayesian Methods for Hackers** - Notebook-based introduction to Bayesian modeling and probabilistic programming through worked examples. [Davidson-Pilon 2015]( "Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference; book"). - **General Bayesian updating** - Weights the prior by exp(−w × cumulative loss). The scale w sets prior-data weighting and changes the target; it needs justification. Negative log likelihood with w = 1 recovers ordinary Bayes. [Bissiri et al. 2016]( "A General Framework for Updating Belief Distributions; JRSS B"). - **Visualization in Bayesian workflow** - Uses graphics throughout model building, inference and checking, extending diagnostic visualization beyond MCMC trace plots. [Gabry et al. 2019]( "Visualization in Bayesian workflow; JRSS A"). - **Bayesian workflow** - Connects iterative model construction, inference, checking, comparison and computational troubleshooting; posterior computation is one part of the analysis. [Gelman et al. 2020]( "Bayesian Workflow; arXiv"). ## Markov chain Monte Carlo - **Metropolis sampling** - Samples molecular configurations with Boltzmann probabilities using accepted or rejected random moves, avoiding direct simulation of the physical dynamics. [Metropolis et al. 1953]( "Equation of State Calculations by Fast Computing Machines; J. Chem. Phys."). - **Hastings' generalization** - Generalizes Metropolis sampling and examines implementation and Monte Carlo error estimation for Markov-chain methods. [Hastings 1970]( "Monte Carlo sampling methods using Markov chains and their applications; Biometrika"). - **Understanding Metropolis-Hastings** - Derives the algorithm and its implementation; shows how block updates and Gibbs sampling fit within the Metropolis-Hastings framework. [Chib and Greenberg 1995]( "Understanding the Metropolis-Hastings Algorithm; The American Statistician"). - **Parallel tempering** - Runs chains at different temperatures and proposes state swaps, helping explore separated modes. Samples from the target-temperature chain retain the intended distribution; temperature spacing and swap mixing affect efficiency. [Earl and Deem 2005]( "Parallel Tempering: Theory, Applications, and New Perspectives; Phys. Chem. Chem. Phys."). - **Introduction to MCMC** - Geyer's handbook chapter covers Markov-chain sampling, Monte Carlo error and practical analysis of dependent draws. [Geyer 2011]( "Introduction to Markov Chain Monte Carlo; book chapter"). - **Hamiltonian Monte Carlo** - Neal's review uses auxiliary momenta and Hamiltonian trajectories to propose distant states; explains volume preservation, discretization and acceptance corrections. [Neal 2011]( "MCMC using Hamiltonian dynamics; book chapter"). - **No-U-Turn Sampler** - Adapts HMC trajectory length by stopping expansion when paths turn back; also introduces dual-averaging step-size adaptation. [Hoffman and Gelman 2014]( "The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo; JMLR"). - **Geometric foundations of HMC** - Develops HMC through measures on smooth manifolds, connecting the theory to implementations and generalizations. [Betancourt et al. 2017]( "The Geometric Foundations of Hamiltonian Monte Carlo; Bernoulli"). - **Conceptual introduction to HMC** - Explains the geometry behind HMC, practical implementations and failure modes; useful when a sampler runs but explores the posterior poorly. [Betancourt 2017]( "A Conceptual Introduction to Hamiltonian Monte Carlo; arXiv"). ## Stochastic-gradient samplers - **SGLD** - Adds noise to minibatch gradient updates and anneals the step size toward posterior sampling. The limiting argument does not make finite-step implementations exact. [Welling and Teh 2011]( "Bayesian Learning via Stochastic Gradient Langevin Dynamics; ICML"). - **SGHMC** - Uses momentum and friction to account for stochastic-gradient noise. Simply replacing HMC's full gradients with minibatch estimates can give the wrong target distribution. [Chen et al. 2014]( "Stochastic Gradient Hamiltonian Monte Carlo; ICML"). - **A recipe for SG-MCMC** - Constructs continuous-time samplers through diffusion and curl matrices, unifying existing dynamics and motivating stochastic-gradient Riemannian HMC. [Ma et al. 2015]( "A Complete Recipe for Stochastic Gradient MCMC; NeurIPS"). ## Sequential Monte Carlo - **SMC Methods in Practice** - Edited reference on particle methods, convergence and applications to tracking, filtering, time series and model selection. [Doucet et al. 2001 🔒]( "Sequential Monte Carlo Methods in Practice; book"). - **SMC samplers** - Propagates weighted particles through distributions on a common space; supports sequential estimation and ratios of normalizing constants beyond state-space filtering. [Del Moral et al. 2006]( "Sequential Monte Carlo Samplers; JRSS B"). - **Introduction to SMC** - Unifies particle filtering and smoothing with Feynman-Kac models, importance sampling and resampling; covers particle MCMC and implementation in Python. [Chopin and Papaspiliopoulos 2020 🔒]( "An Introduction to Sequential Monte Carlo; book"). ### Particle MCMC and noisy likelihoods - **Pseudo-marginal MCMC** - Nonnegative, unbiased likelihood estimates can preserve the posterior as an invariant marginal on an extended space. The estimate is retained on rejection; finite chains still have sampling error and can mix poorly. [Andrieu and Roberts 2009]( "The pseudo-marginal approach for efficient Monte Carlo computations; Ann. Statist."). - **Particle MCMC** - Uses SMC likelihood estimates in particle marginal Metropolis-Hastings and conditional SMC in particle Gibbs. With valid support and resampling, finite particle counts can preserve the invariant posterior. [Andrieu et al. 2010]( "Particle Markov Chain Monte Carlo Methods; JRSS B"). ## Approximate Bayesian computation - **ABC in population genetics** - Uses simulated summary statistics and local-linear regression adjustment to approximate parameter posteriors without evaluating the likelihood. [Beaumont et al. 2002]( "Approximate Bayesian Computation in Population Genetics; Genetics"). - **MCMC without likelihoods** - Builds likelihood-free MCMC using simulated data; summary-statistic matching within a nonzero tolerance targets an approximation to the full-data posterior. [Marjoram et al. 2003]( "Markov chain Monte Carlo without likelihoods; PNAS"). - **SMC without likelihoods** - Introduces ABC-PRC with sequential particles. Read the appended 2009 correction: the original importance weights can bias posterior samples, particularly in the tails. [Sisson et al. 2007]( "Sequential Monte Carlo without likelihoods; PNAS"). - **Nonlinear regression ABC** - Fits nonlinear heteroscedastic regressions of parameters on summary statistics, then refines estimation with importance sampling. [Blum and Francois 2009]( "Non-linear regression models for Approximate Bayesian Computation; Stat Comput"). - **Likelihood-free MCMC review** - Reviews ABC-MCMC construction and practical choices, including summary statistics, discrepancy kernels and tolerance; these choices define the approximation. [Sisson and Fan 2010]( "Likelihood-free Markov chain Monte Carlo; arXiv"). - **ABC in practice** - Reviews ABC algorithms and applications in evolution and ecology; emphasizes model formulation, fitting and improvement within a Bayesian analysis. [Csilléry et al. 2010]( "Approximate Bayesian Computation (ABC) in practice; Trends Ecol. Evol."). - **Hamiltonian ABC** - Approximates ABC gradients using forward simulations and incorporates simulator random seeds into the chain state to support Hamiltonian-style exploration. [Meeds et al. 2015]( "Hamiltonian ABC; arXiv"). - **Random forests for ABC model choice** - Treats model choice as classification from simulated summaries, then uses a second random-forest stage to estimate the selected model's posterior probability. [Pudlo et al. 2015]( "Reliable ABC model choice via random forests; Bioinformatics"). ## Variational inference - **Variational parameter estimation** - Uses variational transformations for logistic regression with Gaussian priors and binary graphical models, yielding tractable posterior approximations. [Jaakkola and Jordan 2000]( "Bayesian parameter estimation via variational methods; Statistics and Computing"). - **Gaussian variational approximation** - Relates Gaussian VI to Laplace approximation and simplifies its parameterization for Gaussian priors with factorized likelihoods; applies it to GP regression with non-Gaussian observations. [Opper and Archambeau 2009]( "The Variational Gaussian Approximation Revisited; Neural Comput."). - **Stochastic VI** - Scales variational posterior approximation through stochastic optimization, demonstrated with large-corpus topic models. [Hoffman et al. 2013]( "Stochastic Variational Inference; JMLR"). - **Black-box VI** - Estimates variational-objective gradients from Monte Carlo draws and reduces their variance, avoiding a new analytic derivation for every model. [Ranganath et al. 2014]( "Black Box Variational Inference; AISTATS"). - **Doubly stochastic variational Bayes** - Uses stochastic optimization and joint-density gradients for non-conjugate inference in continuous parameter spaces. [Titsias and Lázaro-Gredilla 2014]( "Doubly Stochastic Variational Bayes for non-Conjugate Inference; ICML"). - **Stein variational gradient descent** - Transports interacting particles toward a target distribution using functional gradient descent on KL divergence and a kernelized Stein construction. [Liu and Wang 2016]( "Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm; NeurIPS"). - **VI for statisticians** - Reviews KL-based density approximation, mean-field methods, exponential-family models and stochastic optimization; discusses limits and open statistical questions. [Blei et al. 2017]( "Variational Inference: A Review for Statisticians; JASA"). - **ADVI** - Fits mean-field or full-rank Gaussians after unconstraining continuous latent variables. Requires differentiable log densities and suitable support transforms; discrete latent variables need marginalization or another treatment. [Kucukelbir et al. 2017]( "Automatic Differentiation Variational Inference; JMLR"). - **VI defaults** - On a 1D Gaussian test, PyMC 5.25.1 default ADVI overestimates posterior SD by about 5%; more gradient samples or Adam mitigate it. The study tests specific versions and simple targets, not general framework superiority. [Andersen et al. 2026]( "The Curious Case of the Default Settings: Evaluating Default Performance of Variational Inference Software; arXiv"). - **Advances in VI** - Organizes methods by scalability, applicability beyond conjugacy, richer approximating families and amortized inference networks. [Zhang et al. 2019]( "Advances in Variational Inference; IEEE Transactions on Pattern Analysis and Machine Intelligence"). - **Pathfinder** - Builds Gaussian approximations along quasi-Newton optimization paths and selects by estimated ELBO; multiple paths are combined with importance resampling. Produces approximate draws or initial states for MCMC. [Zhang et al. 2022]( "Pathfinder: Parallel quasi-Newton variational inference; JMLR"). ## Rich posterior approximations - **Bayesian backpropagation framework** - Uses local Gaussian posterior approximations and Bayesian evidence for neural-network regularization, model comparison and uncertainty estimates. [MacKay 1992]( "A Practical Bayesian Framework for Backpropagation Networks; Neural Comput."). - **Expectation propagation** - Iteratively matches selected moments of approximate belief states, linking assumed-density filtering with loopy belief propagation; supports hybrid discrete/continuous networks. [Minka 2001]( "Expectation Propagation for approximate Bayesian inference; UAI"). - **EP for partitioned data** - Updates local likelihood approximations while sharing information through the global approximation and prior; a framework for distributed inference, not one fixed local solver. [Vehtari et al. 2020]( "Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data; JMLR"). - **INLA** - Approximates posterior marginals in latent Gaussian models using nested Laplace approximations and integration over hyperparameters. The original method exploits sparse precision matrices and few hyperparameters; likelihoods can be non-Gaussian. [Rue et al. 2009]( "Approximate Bayesian Inference for Latent Gaussian models by using Integrated Nested Laplace Approximations; JRSS B"). - **VI with normalizing flows** - Transforms a simple density through invertible maps to construct flexible variational posteriors, extending beyond factorized approximation families. [Rezende and Mohamed 2015]( "Variational Inference with Normalizing Flows; ICML"). - **Normalizing-flow review** - Surveys invertible distribution transformations, expressiveness and computational tradeoffs for density estimation and approximate inference. [Papamakarios et al. 2021]( "Normalizing Flows for Probabilistic Modeling and Inference; JMLR"). - **Laplace Redux** - Reviews scalable variants of neural-network Laplace approximation and introduces a PyTorch implementation. [Daxberger et al. 2021]( "Laplace Redux -- Effortless Bayesian Deep Learning; NeurIPS"). - **Model-Informed Flows** - Connects Variationally Inferred Parameters plus a full-rank Gaussian to augmented autoregressive flows; builds prior information, translation and hierarchical ordering into the flow. [Ko and Domke 2025]( "Model-Informed Flows for Bayesian Inference; NeurIPS"). ## Neural simulation-based inference - **Classifier-based likelihood ratios** - Uses calibrated discriminative classifiers to approximate likelihood ratios from simulator samples, supporting hypothesis tests without explicit likelihood evaluation. [Cranmer et al. 2015]( "Approximating Likelihood Ratios with Calibrated Discriminative Classifiers; arXiv"). - **Neural posterior estimation** - Learns conditional posterior densities from simulations and uses preliminary inference to guide later simulations, replacing ABC's explicit tolerance test with density estimation. [Papamakarios and Murray 2016]( "Fast $ε$-free Inference of Simulation Models with Bayesian Conditional Density Estimation; NeurIPS"). - **Automatic Posterior Transformation** - Extends sequential neural posterior estimation to dynamically updated proposals and expressive flow-based density estimators. [Greenberg et al. 2019]( "Automatic Posterior Transformation for Likelihood-Free Inference; ICML"). - **Sequential Neural Likelihood** - Fits an autoregressive-flow likelihood surrogate using simulations concentrated in high-posterior regions; includes calibration and goodness-of-fit diagnostics. [Papamakarios et al. 2019]( "Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows; AISTATS"). - **SBI overview** - Reviews inference for scientific simulators with intractable likelihoods and the role of learned surrogates in solving inverse problems. [Cranmer et al. 2020]( "The frontier of simulation-based inference; PNAS"). - **Benchmarking SBI** - On 10 tasks at 1k-100k simulations, sequential neural estimators generally improve simulation efficiency. Rejection ABC can compete on cheap, low-dimensional simulators. On Two Moons, median-heuristic MMD obscures posterior differences. [Lueckmann et al. 2021]( "Benchmarking Simulation-Based Inference; AISTATS"). ## Diffusion priors and inverse problems - **Score-based SDEs** - Relates noise injection and reverse-time generation through learned scores; includes predictor-corrector sampling and applications to inverse problems. [Song et al. 2021]( "Score-Based Generative Modeling through Stochastic Differential Equations; ICLR"). - **Diffusion posterior sampling** - Combines diffusion sampling with measurement-informed gradients for noisy linear and nonlinear inverse problems; the posterior sampling procedure is approximate. [Chung et al. 2023]( "Diffusion Posterior Sampling for General Noisy Inverse Problems; ICLR"). ### Posterior accuracy - **Diffusion-sampling benchmark** - On 64D Lévy signals and four linear inverse tasks, three samplers tuned for reconstruction MSE yield 0-58% coverage at nominal 90% in Gaussian/Laplace cases, but 100% for Bernoulli-Laplace signals. Coverage is averaged over signal-measurement pairs. [Zach et al. 2026]( "A Statistical Benchmark for Diffusion-Posterior-Sampling Algorithms; ICLR"). ## Neural-network and Gaussian-process uncertainty - **Gaussian Processes for Machine Learning** - Reference on GP regression, classification, covariance functions, model selection and approximate inference. [Rasmussen and Williams 2006]( "Gaussian Processes for Machine Learning; book"). - **Sparse variational GPs** - Introduces globally shared inducing variables and stochastic variational inference for GP models, including non-Gaussian likelihoods. [Hensman et al. 2013]( "Gaussian Processes for Big Data; arXiv"). - **Bayes by Backprop** - Learns a distribution over network weights by optimizing a variational free-energy objective, allowing weight uncertainty to enter predictions and exploration. [Blundell et al. 2015]( "Weight Uncertainty in Neural Networks; ICML"). - **Dropout as Bayesian approximation** - Interprets dropout as approximate inference in a deep Gaussian process. Predictive uncertainty uses stochastic test-time passes, which add computation. [Gal and Ghahramani 2016]( "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning; ICML"). - **Deep ensembles** - Combines independently trained neural predictors for uncertainty estimation; a practical comparison method alongside explicit posterior approximations. [Lakshminarayanan et al. 2017]( "Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles; NeurIPS"). - **Aleatoric and epistemic uncertainty** - Combines input-dependent observation noise with model uncertainty for depth regression and semantic segmentation, distinguishing two sources of predictive uncertainty. [Kendall and Gal 2017]( "What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?; NeurIPS"). - **Bayesian view of generalization** - Studies marginalization across and within neural-network solution basins, function-space priors and tempering; connects ensemble predictions with approximate Bayesian averaging. [Wilson and Izmailov 2020]( "Bayesian Deep Learning and a Probabilistic Perspective of Generalization; NeurIPS"). - **Bayesian computation in deep learning** - Reviews SG-MCMC and VI for neural-network weights and latent variables in deep generative models, focusing on their computational challenges. [Chen et al. 2026]( "Bayesian Computation in Deep Learning; Handbook of Markov Chain Monte Carlo"). - **Gaussian moment propagation** - Derives exact means and covariances for Gaussian inputs through residual layers with GELU, ReLU and other activations. Replacing each resulting distribution by a Gaussian remains approximate across layers; the paper gives an explicit failure example. [Kuang and Lin 2026]( "Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method; NeurIPS"). ## Evidence and predictive model comparison - **LOO and WAIC** - Estimate predictive accuracy for a specified held-out unit. In hierarchical data, withholding an observation or an entire group answers different questions. Unstable PSIS weights can require explicit refits or K-fold validation. [Vehtari et al. 2016]( "Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC; Stat Comput"). - **Bridge sampling** - Estimates a marginal likelihood using posterior samples and samples from a proposal distribution. Unlike predictive LOO scores, this targets model evidence; overlap, normalized priors and posterior sampling quality affect reliability. [Gronau et al. 2017]( "A Tutorial on Bridge Sampling; Journal of Mathematical Psychology"). - **Dynamic nested sampling** - Adapts the number of live points during nested sampling to allocate computation between posterior estimation and evidence accuracy, rather than using a fixed live-point count. [Higson et al. 2018]( "Dynamic nested sampling: an improved algorithm for parameter estimation and evidence calculation; Stat Comput"). ## Diagnostics and calibration ### Computational diagnostics and reference tests - **Simulation-based calibration** - Checks inference implementations with repeated simulations from the model and posterior-rank diagnostics. It assesses computation under that model, not whether the model fits reality. [Talts et al. 2018]( "Validating Bayesian Inference Algorithms with Simulation-Based Calibration; arXiv"). - **SBC sensitivity** - Returning the prior can pass parameter-only rank checks. Data-dependent quantities, including the joint likelihood, can detect failures those checks miss. Passing a finite set of checks does not certify the posterior. [Modrák et al. 2025]( "Simulation-Based Calibration Checking for Bayesian Computation: The Choice of Test Quantities Shapes Sensitivity; Bayesian Anal."). - **Rank-normalized R-hat** - Improves convergence diagnostics for heavy tails and unequal chain variances; adds local efficiency measures and Monte Carlo error estimates for quantiles. [Vehtari et al. 2021]( "Rank-normalization, folding, and localization: An improved $\widehat{R}$ for assessing convergence of MCMC; Bayesian Anal."). - **Pareto-smoothed importance sampling** - Fits a generalized Pareto tail to stabilize importance weights; provides effective-sample-size, Monte Carlo error and tail-shape diagnostics. [Vehtari et al. 2024]( "Pareto Smoothed Importance Sampling; JMLR"). - **posteriorDB** - Pairs models and data with reference draws where available. Compares posterior accuracy against computation; reference samples are numerical approximations, and some difficult targets lack them. [Magnusson et al. 2025]( "posteriordb: Testing, Benchmarking and Developing Bayesian Inference Algorithms; AISTATS"). ### Predictive calibration - **The well-calibrated Bayesian** - Relates coherent Bayesian forecasting to long-run calibration; the theorem concerns a forecaster's probabilistic expectations, not automatic empirical calibration of every fitted model. [Dawid 1982]( "The Well-Calibrated Bayesian; JASA"). - **Multiclass probability calibration** - Transforms classifier scores into multiclass probability estimates. [Zadrozny and Elkan 2002]( "Transforming classifier scores into accurate multiclass probability estimates; KDD"). - **Predicting good probabilities** - Compares distortions in classifier probabilities and correction by Platt scaling or isotonic regression; calibration method and calibration-set size affect results. [Niculescu-Mizil and Caruana 2005]( "Predicting good probabilities with supervised learning; ICML"). - **Nearly-isotonic regression** - Relaxes monotonicity through a convex penalty and computes a path of fits; intermediate solutions help examine whether strict monotonicity is appropriate. [Tibshirani et al. 2011]( "Nearly-Isotonic Regression; Technometrics"). - **Ensembles of calibration models** - Studies ensembles of near-isotonic regression for binary probability calibration and compares them with linear-trend calibration; relaxes strict isotonicity. [Pakdaman Naeini and Cooper 2017]( "Binary classifier calibration using an ensemble of piecewise linear regression models; Knowl Inf Syst"). - **Beta calibration** - Introduces a parametric calibration family for binary classifiers that includes the identity map and accommodates skewed score distributions. [Kull et al. 2017]( "Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers; AISTATS"). - **Temperature scaling** - Evaluates modern neural-network calibration and a single-parameter post-processing baseline; effectiveness on the evaluated datasets does not guarantee calibration after distribution shift. [Guo et al. 2017]( "On Calibration of Modern Neural Networks; ICML"). - **Verified uncertainty calibration** - Combines parametric scaling with binning and studies finite-sample calibration-error estimation; distinguishes measurable calibration from apparently smooth reliability plots. [Kumar et al. 2019]( "Verified Uncertainty Calibration; NeurIPS"). - **Uncertainty under dataset shift** - Benchmarks predictive accuracy and calibration as input distributions change; shows that successful in-distribution post-hoc calibration need not survive shift. [Ovadia et al. 2019]( "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift; NeurIPS"). - **CRUDE** - Calibrates regression uncertainty using empirical errors; assumes a common residual-distribution shape after shifting by predicted means and scaling by predicted standard deviations. [Zelikman et al. 2020]( "CRUDE: Calibrating Regression Uncertainty Distributions Empirically; ICML UDL Workshop"). - **Label-free calibration transfer** - Projects predictions toward compatibility with a calibrated reference using unlabeled paired outputs. Proper-loss improvement requires a calibrated reference and failure of mutual calibration; reference errors can offset the gain. [Kong et al. 2026]( "Calibration without Ground Truth; NeurIPS"). ## Software ### Modeling languages and inference frameworks - **Stan** - Probabilistic modeling language with R, Python, Julia and command-line interfaces; supports hierarchical and time-series models and an ecosystem for validation and visualization. [docs](). - **PyMC** - Python probabilistic modeling with MCMC and VI, including NUTS and ADVI; uses PyTensor for computation. [code](). - **Pyro** - Python probabilistic programming built on PyTorch, integrating neural networks with probabilistic models and customizable inference. [docs](). - **NumPyro** - Probabilistic programming with Pyro-style primitives and JAX differentiation/compilation; includes HMC and NUTS. [code](). - **TensorFlow Probability** - Distributions, invertible transformations and probabilistic inference components for TensorFlow, with a JAX substrate. [code](). - **Turing.jl and DynamicPPL** - Turing supplies Julia inference interfaces; DynamicPPL is its backend for defining, conditioning and manipulating probabilistic models. [code](), [code](). - **NIMBLE** - Extends BUGS model syntax and compiles models plus user-written algorithms to C++. Supports customizable MCMC, particle methods and deterministic approximations; model specification and inference algorithms can be changed independently. [docs](). - **JAGS** - Cross-platform MCMC engine for Bayesian hierarchical models in the BUGS language, with extensible functions, distributions and samplers. [docs](). - **greta** - Defines statistical models directly in R and fits them by MCMC or optimization, using TensorFlow and TensorFlow Probability internally. [code](). - **Gen** - Probabilistic modeling and inference components for custom algorithms combining neural proposals, VI, SMC and MCMC. [docs](). ### Regression interfaces - **brms** - R formulas generate Stan models for multilevel and distributional regression, including nonlinear and multivariate responses. Unlike rstanarm's precompiled models, brms generates model-specific Stan code. [docs](). - **rstanarm** - Fits precompiled Bayesian regression models through R formulas and data frames, with additional prior specifications. [code](). - **Bambi** - High-level interface built on PyMC for Bayesian regression and mixed-effects models. [code](). ### Sampling and posterior approximations - **BlackJAX** - JAX sampling components for user-supplied log densities; integrates with probabilistic programs without providing a modeling language itself. [code](). - **nutpie** - Rust-backed NUTS sampler for PyMC and Stan models, with compiled log densities, sampler diagnostics and experimental flow adaptation. It supplies an inference backend rather than a separate modeling language. [docs](). - **laplace-torch** - Laplace approximations for full networks, subnetworks or final layers, with marginal-likelihood and predictive computations. [code](). ### Simulation-based inference and density estimation - **sbi** - PyTorch toolkit for amortized or sequential posterior, likelihood and ratio estimation from simulations. Includes calibration and coverage diagnostics. [Boelts et al. 2025]( "sbi reloaded: a toolkit for simulation-based inference workflows; Journal of Open Source Software"); [code](). - **BayesFlow 2** - Amortizes posterior, likelihood and ratio estimation across simulated datasets. Uses Keras 3 with JAX, PyTorch or TensorFlow backends and includes calibration diagnostics. [Kühmichel et al. 2026]( "BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python; arXiv"); [code](). - **Normalizing-flow libraries** - FlowJAX supplies distributions and bijections in JAX/Equinox; Zuko supplies trainable conditional flows in PyTorch. [code](), [code](). ### Diagnostics and reference targets - **ArviZ** - Convergence diagnostics, predictive checks, model comparison and posterior visualization across inference libraries. Separates statistical analysis and visualization from the sampler that produced the draws. [docs](). - **posteriorDB tools** - Versioned models, data, reference draws and interfaces for inference testing. Check which targets have reference samples before treating the collection as a ground-truth benchmark. [code](). ### Browser and graphical interfaces - **WebPPL** - Probabilistic programming for the web, also runnable through Node.js with external packages. [code](). - **StatSim** - Browser-based visual stochastic modeling and Bayesian inference, with simulation, MCMC and variational methods. [docs](). - **JASP** - Graphical statistics software offering classical and Bayesian analyses, with tables and plots for reporting results. [docs](). ### Historical systems - **BUGS** - Historical review of Bayesian inference using Gibbs sampling in BUGS, WinBUGS and OpenBUGS; discusses both modeling flexibility and limitations. [Lunn et al. 2009]( "The BUGS project: Evolution, critique and future directions; Statistics in Medicine"). - **Edward** - TensorFlow-based library for probabilistic modeling, inference and model criticism; the linked documentation describes the original Edward framework. [docs](). - **ZhuSuan** - TensorFlow probabilistic programming for Bayesian deep learning, including VI, HMC and stochastic-gradient samplers. Its installation guide specifies TensorFlow 1.13 or later; check this older API before reusing examples. [code](). - **Brancher** - Historical PyTorch-based symbolic probabilistic modeling and stochastic variational inference for differentiable Bayesian models. [code](). ## Related topics - [Bayesian Networks](https://mlpapers.org/bayesian-nets/) - [Missing Data](https://mlpapers.org/missing-data/) - [Optimization](https://mlpapers.org/optimization/) - [Neural Networks](https://mlpapers.org/neural-nets/) - [Generative Models](https://mlpapers.org/generative-models/)