# Missing Data and Imputation ## Overviews - **Missing Data: Our View of the State of the Art** - Reviews missingness assumptions, likelihood inference, and multiple imputation. Pigott (2001) compares these with deletion and single-value methods. [Schafer and Graham 2002]( "Missing data: Our view of the state of the art.; Psychological Methods"), [Pigott 2001]( "A Review of Methods for Missing Data; Educational Research and Evaluation"). - **Missing-data imputation** - Chapter 25. Compares deletion, simple replacements, regression, and model-based imputation; explains why treating filled values as observed understates uncertainty. [Gelman and Hill 2007]( "Missing-data imputation; book chapter"). - **Flexible Imputation of Missing Data** - Missingness assumptions, multiple imputation, diagnostics, and applications. Online chapters and book source. [van Buuren 2018]( "Flexible Imputation of Missing Data, Second Edition; book"); [code](), [code](). ## Missingness Mechanisms and Assumptions - **Missing completely at random (MCAR)** - Missingness probabilities do not depend on observed or unobserved data values. Complete cases then form a random subsample, with reduced information. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **Missing at random (MAR)** - Given the observed data, missingness does not additionally depend on the missing values. MAR permits systematic missingness explained by observed variables; it does not mean entries disappear uniformly. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **Missing not at random (MNAR)** - Missingness still depends on unobserved values after conditioning on observed data. Distinguishing MAR from MNAR generally requires assumptions beyond the observed data. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **Ignorability** - MAR and distinct data/missingness parameters permit ignoring the missingness model in direct-likelihood inference. This does not justify arbitrary deletion or replacement rules. [Rubin 1976]( "Inference and missing data; Biometrika"). - **MNAR sensitivity analysis** - Delta-adjusted imputation shifts conditional distributions away from MAR through user-specified sensitivity parameters. Vary plausible departures; the observed data alone do not identify those shifts. [docs](). - **Graphical missing-data recovery** - Uses graphs to express missingness assumptions and derive recoverability and testability conditions, including some MNAR cases. Recovery depends on the graph and target quantity. [Mohan and Pearl 2021]( "Graphical Models for Processing Missing Data; Journal of the American Statistical Association"). ## Deletion and Simple Imputation - **Complete-case analysis** - Uses rows complete on the variables needed for an analysis. MCAR is a sufficient, not necessary, condition for validity; selection can bias estimates and discarding rows reduces information. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **Available-case / pairwise deletion** - Uses all observed values for each statistic, so covariances may use different subsets. The resulting covariance matrix need not be positive semidefinite. This is not deletion of columns. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **Mean, median, and mode replacement** - Simple column-wise baselines. Deterministic fills do not represent imputation uncertainty; mean replacement reduces variance and can distort correlations. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"); [docs](). - **Regression imputation** - Predicts a missing variable from observed predictors. Deterministic predictions suppress residual variation; stochastic regression adds residual draws. Multiple imputation also accounts for parameter uncertainty. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **Hot-deck and cold-deck imputation** - Hot-deck methods draw observed values from eligible donors in the current sample; cold-deck methods use an external source. Donor selection determines which relationships are preserved. [Andridge and Little 2010]( "A Review of Hot Deck Imputation for Survey Non-response; Int. Stat. Rev."). - **Predictive mean matching** - Selects donors with similar predicted outcomes, then copies an observed donor value. Preserves observed support; results depend on the prediction model and donor pool. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **K-nearest-neighbor imputation** - Fills each feature from nearby samples that observed it. Distances use jointly observed coordinates; scaling and sparse overlap affect which donors are selected. [docs](). ## Likelihood and Multiple Imputation - **Expectation-maximization (EM)** - Alternates expected complete-data log likelihood and parameter updates. Fits an observed-data model; it is not itself a procedure for producing multiple completed datasets. [Dempster et al. 1977]( "Maximum Likelihood from Incomplete Data Via the EM Algorithm; JRSS B"). - **Multiple imputation** - Generates multiple plausible completions, analyzes each, and combines estimates with within- and between-imputation variance. Pool analysis results, not completed datasets. [Rubin 1987]( "Multiple Imputation for Nonresponse in Surveys; book"); [docs](). - **Imputation-analysis compatibility** - Rubin's rules can fail when imputation and analysis models are uncongenial. Bartlett and Hughes find valid bootstrap-before-imputation variants under consistent point estimation; resampling cannot fix a biased point estimate. [Bartlett and Hughes 2020]( "Bootstrap Inference for Multiple Imputation under Uncongeniality and Misspecification; Statistical Methods in Medical Research"). - **Sequential regression multivariate imputation** - Cycles through variable-specific regression models and draws from predictive distributions, accommodating different variable types and logical constraints. [Raghunathan et al. 2001]( "A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models; Survey Methodol."). - **Joint-model multiple imputation** - Develops an alternative computational approach to multiple imputation for incomplete multivariate data. Amelia II uses bootstrap-based EM. [King et al. 2001]( "Analyzing Incomplete Political Science Data: An Alternative Algorithm for Multiple Imputation; Am Polit Sci Rev"), [Honaker et al. 2011]( "Amelia II: A Program for Missing Data; J. Stat. Soft."). - **Fully conditional specification** - Specifies an imputation model for each incomplete variable. Flexible conditional models need not correspond to one compatible joint distribution. [van Buuren 2007]( "Multiple imputation of discrete and continuous data by fully conditional specification; Stat Methods Med Res"). - **MICE** - Implements chained equations with variable-specific methods, predictor selection, and diagnostics. Azur et al. (2011) explain the workflow and assumptions. [van Buuren and Groothuis-Oudshoorn 2011]( "mice: Multivariate Imputation by Chained Equations in R; J. Stat. Soft."), [Azur et al. 2011]( "Multiple imputation by chained equations: what is it and how does it work?; Int J Methods Psych Res"). - **Conditional tests after imputation** - PAIR-CI compares predictors with real or conditionally permuted candidate inputs, sharing imputed covariates. Its calibration theorem needs consistent, proper imputations and stable learners; reported MNAR robustness is empirical. [Robinson and Lall 2026]( "PAIR-CI: Calibrated Conditional Independence Testing for Causal Discovery with Incomplete Data; NeurIPS"). ## Tabular and Low-Rank Methods - **SoftImpute** - Fits a low-rank matrix through nuclear-norm regularization and repeated soft-thresholded SVD. Suits shared latent structure; low rank alone does not resolve selective missingness. [Mazumder et al. 2010]( "Spectral Regularization Algorithms for Learning Large Incomplete Matrices; JMLR"). - **missForest** - Iteratively fits random forests to incomplete columns, supporting mixed numeric/categorical data. Out-of-bag error estimates imputation error, not inferential confidence intervals. [Stekhoven and Bühlmann 2011]( "MissForest—non-parametric missing value imputation for mixed-type data; Bioinformatics"), [Stekhoven and Bühlmann 2011]( "MissForest—non-parametric missing value imputation for mixed-type data; Bioinformatics"). - **Optimal-transport imputation** - Optimizes missing entries by matching distributions of imputed batches with Sinkhorn divergences. Can learn an imputation function for new observations. [Muzellec et al. 2020]( "Missing Data Imputation using Optimal Transport; ICML"); [code](). - **HyperImpute** - Extends iterative imputation with automatic selection of each column's prediction model and hyperparameters. Model search is part of fitting and must stay inside evaluation folds. [Jarrett et al. 2022]( "HyperImpute: Generalized Iterative Imputation with Automatic Model Selection; ICML"). ## Deep Generative and Pretrained Imputers - **Autoencoders for incomplete records** - Evaluates autoencoder imputation on ALS trial data under simulated missingness, measuring reconstruction and progression prediction separately. [Beaulieu-Jones et al. 2017]( "Missing Data Imputation in the Electronic Health Record Using Deeply Learned Autoencoders; PSB"). - **MIDA** - Uses overcomplete denoising autoencoders for multiple imputation of mixed data. A separate method from Lall & Robinson's MIDAS software. [Gondara and Wang 2018]( "MIDA: Multiple Imputation using Denoising Autoencoders; PAKDD"). - **GAIN** - A generator fills missing entries; a discriminator predicts which entries were observed, aided by a partial missingness hint. The paper's identifiability result assumes MCAR. [Yoon et al. 2018]( "GAIN: Missing Data Imputation using Generative Adversarial Nets; ICML"). - **Monte Carlo dropout autoencoders** - Uses dropout within autoencoder and variational-autoencoder models to generate repeated imputations; evaluates reconstruction error and predictive similarity. [Miok et al. 2019]( "Multiple Imputation for Biomedical Data using Monte Carlo Dropout Autoencoders; EHB"). - **MIWAE** - Trains a latent-variable model on incomplete data using an importance-weighted bound on observed-data likelihood under MAR. Supports Monte Carlo single and multiple imputations. [Mattei and Frellsen 2019]( "MIWAE: Deep Generative Modelling and Imputation of Incomplete Data Sets; ICML"). - **not-MIWAE** - Jointly models values and missingness with importance-weighted variational inference. MNAR modeling relies on the specified mechanism; neural capacity does not establish identification. [Ipsen et al. 2021]( "not-MIWAE: Deep Generative Modelling with Missing not at Random Data; ICLR"). - **ReMasker** - Randomly masks additional observed entries and trains a masked autoencoder to reconstruct them, then imputes naturally missing values; the training task uses available table entries. [Du et al. 2023]( "ReMasker: Imputing Tabular Data with Masked Autoencoding; arXiv"). - **MissDiff** - Trains diffusion models directly on incomplete data by masking the denoising score-matching loss. Avoids first filling every missing entry with a separate imputer. [Ouyang et al. 2023]( "MissDiff: Training Diffusion Models on Tabular Data with Missing Values; arXiv"). - **TabImpute and pattern-specific transformers** - Pretrains imputers on synthetic tables with configurable missingness mechanisms. The default TabImpute uses MCAR training; specialists target particular patterns. [Feitelberg et al. 2025]( "One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Data; arXiv"). - **FLOWGEM** - Generates complete samples through approximate Wasserstein gradient flows over observed-pattern distributions. Population identification assumes MAR and positive complete-case probability. This is distribution recovery, not row-wise imputation. [Kremling et al. 2026]( "Generative Modeling under Non-Monotone MAR Missingness via Approximate Wasserstein Gradient Flows; NeurIPS"). ## Time-Series and Sensor Imputation - **Last observation carried forward** - Repeats the latest observed value across a gap. Assumes no change during that interval; bias can operate in either direction. [Molnar et al. 2008]( "Does analysis using "last observation carried forward" introduce bias in dementia research?; CMAJ"); [van Buuren 2018]( "Flexible Imputation of Missing Data; book"). - **BRITS** - Learns missing values within bidirectional recurrent computation, combining temporal and cross-feature estimates. Uses observations on both sides of a gap, so its full form requires future context. [Cao et al. 2018]( "BRITS: Bidirectional Recurrent Imputation for Time Series; NeurIPS"). - **M-RNN** - Combines interpolation within temporal streams with imputation across variables using a multidirectional recurrent network. [Yoon et al. 2019]( "Estimating Missing Data in Temporal Data Streams Using Multi-directional Recurrent Neural Networks; IEEE Transactions on Biomedical Engineering"). - **GP-VAE** - Places a Gaussian-process prior over a low-dimensional latent trajectory and decodes it into observations. Combines temporal structure with probabilistic imputation. [Fortuin et al. 2020]( "GP-VAE: Deep Probabilistic Time Series Imputation; AISTATS"). - **CSDI** - Trains conditional score-based diffusion models to sample missing time-series values given observations. Self-supervised masking creates training targets from observed entries. [Tashiro et al. 2021]( "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation; NeurIPS"); [code](). - **GRIN** - Uses graph message passing and recurrent processing to impute multivariate sensor streams. Exploits relations between sensors as well as temporal dependencies. [Cini et al. 2022]( "Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks; ICLR"); [code](). - **SAITS** - Combines two diagonally masked self-attention blocks with learned weights, using reconstruction and masked-imputation objectives. [Du et al. 2023]( "SAITS: Self-Attention-based Imputation for Time Series; Expert Syst. Appl."); [code](). ## Prediction with Missing Inputs - **RNNs with missingness indicators** - Studies diagnosis prediction from irregular clinical measurements and their missingness patterns. The objective is supervised prediction, not recovering every absent measurement. [Lipton et al. 2016]( "Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series; MLHC"). - **Missingness indicators** - Binary masks preserve which inputs were missing after filling them. They can help prediction; adding indicators does not generally repair biased coefficient estimates. [van Buuren 2018]( "Flexible Imputation of Missing Data; book"); [docs](). - **GRU-D** - Incorporates observation masks, elapsed times, and learned decays of inputs and hidden states into a recurrent predictor. Uses informative missingness directly. [Che et al. 2018]( "Recurrent Neural Networks for Multivariate Time Series with Missing Values; Scientific Reports"). - **NeuMiss** - Derives a prediction architecture from a Neumann-series approximation under specified data models. Multiplication by missingness masks adapts computation to the observed pattern. [Le Morvan et al. 2020]( "NeuMiss networks: differentiable programming for supervised learning with missing values.; NeurIPS"). - **Trees with native missing-value handling** - Histogram gradient boosting learns which child receives missing values at a split. This supports prediction without a separate imputer; it does not reconstruct missing measurements. [docs](). ## Evaluation and Diagnostics - **Time-series masking** - Evaluate isolated gaps and contiguous outages separately. Distinguish retrospective filling from online prediction: bidirectional imputers can use future observations unavailable at decision time. [Tashiro et al. 2021]( "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation; NeurIPS"), [Cao et al. 2018]( "BRITS: Bidirectional Recurrent Imputation for Time Series; NeurIPS"). - **Probabilistic evaluation** - CSDI evaluates distributions with CRPS alongside point errors. For inferential multiple imputation, assess bias and interval coverage of the analysis target; low reconstruction error does not establish valid uncertainty. [Tashiro et al. 2021]( "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation; NeurIPS"); [docs](). - **Imputation for prediction: beware of diminishing returns** - Across 19 datasets, better reconstruction had limited predictive benefit with expressive predictors and missingness indicators. Results depend on the prediction setup. [Le Morvan and Varoquaux 2025]( "Imputation for prediction: beware of diminishing returns.; ICLR"). - **Leakage control** - Fit imputers and tune their settings within training folds. Split related records or temporal windows before fitting; held-out outcomes must not become imputation predictors in deployment evaluations. [docs](). - **Missingness benchmarks** - MissBench evaluates 42 datasets and 11 missingness patterns in the revised preprint. Report the simulated mechanism and rate; random masking alone does not establish performance under MNAR. [Feitelberg et al. 2025]( "One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Data; arXiv"). ## Software - **scikit-learn** - Simple, KNN, and iterative imputers plus missingness indicators. `IterativeImputer` produces one completion per call; posterior sampling and repeated calls can generate multiple imputations. [docs](). - **mice** - Chained equations, diagnostics, and pooled inference in R. Amelia implements joint-model multiple imputation; missForest provides iterative forest imputation. [Honaker et al. 2011]( "Amelia II: A Program for Missing Data; J. Stat. Soft."); [code](), [docs](). - **miceforest** - Multiple imputation with LightGBM-based chained models and mean matching in Python. HyperImpute adds automatic column-wise model selection. [code](), [code](). - **MIDASpy** - Denoising-autoencoder multiple imputation. [Lall and Robinson 2022]( "The MIDAS Touch: Accurate and Scalable Missing-Data Imputation with Deep Learning; Political Analysis"); [code](). - **PyPOTS** - Implementations for learning from partially observed time series, including BRITS, SAITS, and CSDI. [code](). - **Impute.jl** - Missing-data handling for Julia vectors, matrices, and tables, including interpolation and carry-forward/backward methods. [code](). ## Related topics - [Bayesian Inference](https://mlpapers.org/bayesian-inference/) - [Neural Networks](https://mlpapers.org/neural-nets/) - [Models](https://mlpapers.org/models/)