# Bayesian Networks ## Foundations and model families - **Mixed graphical models** - Models associations among qualitative and quantitative variables; characterizes decomposable cases. Covers graphs with directed and undirected edges, beyond ordinary Bayesian networks. [Lauritzen and Wermuth 1989]( "Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative; Ann. Statist."). - **Dynamic network forecasting** - Extends static belief networks with contemporaneous and lagged dependencies, combining network refinement with time-series forecasting. [Dagum et al. 1992]( "Dynamic Network Models for Forecasting; UAI"). - **Gaussian networks** - Develops Bayesian scores for learning networks of normally distributed variables, using prior-network assessments and likelihood equivalence. [Geiger and Heckerman 1994]( "Learning Gaussian Networks; UAI"). - **Learning with Bayesian networks** - Heckerman's tutorial on prior knowledge, parameter estimation, structure scores and incomplete data. Original 1996 manuscript, revised in 2022; arXiv upload dates are later. [Heckerman 1998]( "A Tutorial on Learning With Bayesian Networks; Learning in Graphical Models"). - **Dynamic Bayesian networks** - Murphy's thesis covers factored temporal states, exact and approximate inference, and learning; includes hierarchical HMMs and Rao-Blackwellized particle filtering. [Murphy 2002]( "Dynamic Bayesian Networks: Representation, Inference and Learning; PhD thesis"). - **Introducing Bayesian networks** - Book chapter on DAGs, conditional probability tables, independence and evidence propagation; from the second edition of Bayesian Artificial Intelligence. [Korb and Nicholson 2010]( "Introducing Bayesian Networks; book chapter"). - **Copula Bayesian networks** - Combines a Bayesian-network factorization with copulas to model continuous multivariate densities while separating marginal distributions from dependence. [Elidan 2010]( "Copula Bayesian Networks; NeurIPS"). ## Exact and approximate inference - **Bucket elimination** - Unifies belief updating, most-probable explanations and expected-utility computation through variable elimination; combines elimination with conditioning to trade memory for computation. [Dechter 1996]( "Bucket elimination: A unifying framework for probabilistic inference; UAI"). - **Loopy belief propagation** - Tests message passing on graphs with cycles. It can produce useful approximations, but the experiments also show oscillation and inaccurate marginals; convergence alone is not an accuracy certificate. [Murphy et al. 1999]( "Loopy Belief Propagation for Approximate Inference: An Empirical Study; UAI"). - **Factor graphs and sum-product** - Expresses factorizations as bipartite graphs and computes marginal functions by message passing; connects belief propagation, forward-backward and Kalman filtering. [Kschischang et al. 2001]( "Factor graphs and the sum-product algorithm; IEEE Trans. Inform. Theory"). - **Sum-product networks** - Uses weighted sums and products to represent distributions with tractable marginals under completeness and consistency conditions; a related alternative to general graphical-model inference. [Poon and Domingos 2011]( "Sum-Product Networks: A New Deep Architecture; UAI"). ## Independence tests and Markov blankets - **Equivalence of causal models** - Characterizes when graphs encode the same conditional independencies. Observationally equivalent structures limit what independence-based discovery can orient. [Verma and Pearl 1990]( "Equivalence and Synthesis of Causal Models; UAI"). - **Grow-Shrink** - Grows a candidate Markov blanket by adding conditionally dependent variables, then removes redundant ones. Correctness relies on faithfulness and reliable independence tests. [Margaritis 2003]( "Learning Bayesian Network Model Structure from Data; PhD thesis"). - **IAMB and variants** - Discovers a target's Markov blanket for variable selection and network learning. Soundness assumes a faithful network and reliable conditional-independence tests. [Tsamardinos et al. 2003]( "Algorithms for Large Scale Markov Blanket Discovery; FLAIRS"). - **Fast-IAMB** - Uses speculative additions to accelerate Markov-blanket discovery; the target is a conditionally sufficient variable set for prediction, not a ranking of marginal correlations. [Yaramakala and Margaritis 2005]( "Speculative Markov Blanket Discovery for Optimal Feature Selection; ICDM"). - **PC in high dimensions** - Analyzes PC for sparse Gaussian DAGs under faithfulness and regularity assumptions; conditional-independence testing identifies an equivalence class rather than every causal direction. [Kalisch and Bühlmann 2007]( "Estimating High-Dimensional Directed Acyclic Graphs with the PC-Algorithm; JMLR"). ## Scores, search and classifiers - **Prior networks and structure scores** - Combines prior-network assessments, confidence and data into Bayesian scores; likelihood-equivalent structures receive compatible scores under the stated assumptions. [Heckerman et al. 1995]( "Learning Bayesian networks: The combination of knowledge and statistical data; Mach. Learn."). - **Tree-augmented naive Bayes** - Adds a tree of dependencies among predictors to naive Bayes. Learns that tree using class-conditional mutual information and a maximum-weight spanning tree. [Friedman et al. 1997]( "Bayesian Network Classifiers; Mach. Learn."). - **Sparse Candidate** - Alternates restricted network search with refinement of small candidate-parent sets, reducing the number of parent combinations considered on large datasets. [Friedman et al. 1999]( "Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm; UAI"). - **Greedy equivalence search** - Searches equivalence classes through edge additions and deletions; asymptotic correctness depends on the generating model and scoring assumptions, not a finite-sample global optimum guarantee. [Chickering 2002]( "Optimal Structure Identification With Greedy Search; JMLR"). - **Optimal Reinsertion** - Removes a variable and its incident edges, then optimizes its reinsertion. The operator's optimum does not imply a globally optimal learned network. [Moore and Wong 2003]( "Optimal Reinsertion: A New Search Operator for Accelerated and More Accurate Bayesian Network Structure Learning; ICML"). - **Max-min hill climbing** - Combines conditional-independence tests for a candidate skeleton with score-based hill climbing to orient edges; a hybrid of constraint- and score-based structure learning. [Tsamardinos et al. 2006]( "The max-min hill-climbing Bayesian network structure learning algorithm; Mach. Learn."). - **Networks with thousands of variables** - Uses inexpensive approximate scores to guide candidate parent-set exploration and improve ordering-based structure search. [Scanagatta et al. 2015]( "Learning Bayesian Networks with Thousands of Variables; NeurIPS"). - **Greedy-search computational cost** - Analyzes the cost of local distribution fitting within greedy search; counting score evaluations alone can misrepresent runtime across data types and network sizes. [Scutari et al. 2019]( "Learning Bayesian networks from big data with greedy search: computational complexity and efficient implementation; Stat Comput"). ## Continuous optimization of structure - **NOTEARS** - Recasts linear DAG estimation as continuous optimization with a smooth acyclicity constraint. Acyclicity restricts the graph; it does not establish causal identification from arbitrary data. [Zheng et al. 2018]( "DAGs with NO TEARS: Continuous Optimization for Structure Learning; NeurIPS"). - **DAG-GNN** - Uses a graph-neural variational autoencoder and a differentiable acyclicity constraint for nonlinear structure learning; accommodates discrete and vector-valued variables. [Yu et al. 2019]( "DAG-GNN: DAG Structure Learning with Graph Neural Networks; ICML"). - **DAGMA** - Introduces a log-determinant acyclicity characterization over M-matrices and a central-path optimization scheme, evaluated with linear and nonlinear structural equation models. [Bello et al. 2022]( "DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization; NeurIPS"). ## Bayesian structure uncertainty - **DiBS** - Uses a continuous latent representation of graphs for differentiable approximate inference over DAG structures and parameters; retains uncertainty across graphs instead of selecting only one. [Lorch et al. 2021]( "DiBS: Differentiable Bayesian Structure Learning; NeurIPS"). - **DAG-GFlowNet** - Learns to construct DAGs by adding edges with a generative flow network, targeting samples proportional to Bayesian graph scores; approximates a posterior over structures. [Deleu et al. 2022]( "Bayesian Structure Learning with Generative Flow Networks; UAI"). ## Parameter estimation and incomplete data - **EM for network parameters** - Alternates inference of expected sufficient statistics with parameter updates. Handles incomplete cases under the model; initialization matters because the likelihood objective can have local maxima. [Heckerman 1998]( "A Tutorial on Learning With Bayesian Networks; Learning in Graphical Models"). - **Bayesian Structural EM** - Alternates inference over missing or hidden variables with structure search using expected scores. Friedman extends parameter-only EM to Bayesian model selection; convergence does not imply finding the globally best graph. [Friedman 1998]( "The Bayesian Structural EM Algorithm; UAI"). - **Qualitative parameter constraints** - Combines qualitative domain constraints with incomplete-data likelihood through a penalty formulation and modified EM, restricting parameter estimates using expert knowledge. [Liao and Ji 2009]( "Learning Bayesian network parameters under incomplete data with domain knowledge; Pattern Recognition"). ## Software - **abn** - R package for additive Bayesian networks: local generalized linear models accommodate different response distributions; supports structure search and clustered observations. [Kratzer et al. 2023]( "Additive Bayesian Network Modelling with the R Package abn; Journal of Statistical Software"). - **bnlearn** - R tools for structure and parameter learning, discrete/Gaussian/conditional-Gaussian networks, prediction and simulation; includes constraint-based, score-based and hybrid searches. [docs](). - **deal** - R package for discrete and continuous variables restricted to conditionally Gaussian networks; combines conjugate parameter updating with score-based heuristic structure search. [Bøtcher and Dethlefsen 2003]( "deal: A Package for Learning Bayesian Networks; J. Stat. Soft."). - **pcalg** - R implementations of causal structure discovery and effect estimation, including PC, FCI, GIES and IDA; supported assumptions differ across algorithms. [code](). - **BNfinder** - Python software for static and dynamic network reconstruction, with classification and continuous-variable scoring extensions. The linked documentation describes a legacy Python 2 implementation. [docs](). - **BayesNets.jl** - Julia library for representing Bayesian networks, performing inference and learning networks. [code](). - **pgmpy** - Python tools for probabilistic graphical models, including parameter and structure learning, exact and approximate inference, sampling and causal queries. [docs](). ## Related topics - [Bayesian Inference](https://mlpapers.org/bayesian-inference/) - [Causal Inference](https://mlpapers.org/causal-inference/) - [Models](https://mlpapers.org/models/)