# Anomaly and Novelty Detection ## Detection targets and supervision - **Outlier versus novelty detection** - Outlier detection fits potentially contaminated observations; novelty detection learns a reference population and scores new inputs. The training contamination and availability of labels define different tasks. [docs](). - **Anomaly structure** - Unusual points, unusual values conditional on context, and unusual sequences or groups require different scoring rules. Ruff et al. organize methods by assumptions about normality and available supervision. [Ruff et al. 2021]( "A Unifying Review of Deep and Shallow Anomaly Detection; Proc. IEEE"). - **Scores versus alarms** - An anomaly ranking does not specify an operational threshold or a probability of failure. Detection interfaces separate scores from thresholded predictions; contamination settings encode a cutoff choice. [docs](). ## Neighborhoods, isolation and distribution tails - **LOF** - Compares local reachability density with neighboring densities, detecting points sparse relative to their neighborhood. Neighborhood size changes the scale of abnormality. [Breunig et al. 2000]( "LOF: identifying density-based local outliers; SIGMOD Rec."). - **Isolation Forest** - Random feature/split partitions isolate observations; short average isolation paths indicate anomalies. Isolation replaces explicit density or distance estimation. [Liu et al. 2008]( "Isolation Forest; ICDM"). - **Extended Isolation Forest** - Uses randomly oriented hyperplanes instead of axis-aligned splits to reduce score artifacts. The geometric change addresses partition bias rather than learning semantic anomaly labels. [Hariri et al. 2021]( "Extended Isolation Forest; TKDE"). - **COPOD** - Constructs an empirical copula and combines tail information to score extreme observations. Distribution-tail scoring differs from LOF's comparison against a local neighborhood. [Li et al. 2020]( "COPOD: Copula-Based Outlier Detection; ICDM"). - **ECOD** - Estimates empirical marginal cumulative distributions, converts observations to tail scores and aggregates across features. Its featurewise construction supplies a simple tabular baseline without learning arbitrary joint dependencies. [Li et al. 2023]( "ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions; TKDE"). ## Support and robust distribution models - **Robust covariance** - Minimum covariance determinant / EllipticEnvelope fit a robust location and covariance for distance-based detection. Useful when an approximately elliptical normal population is plausible; multimodal structure needs other models. [Rousseeuw and Van Driessen 1999]( "A Fast Algorithm for the Minimum Covariance Determinant Estimator; Technometrics"); [docs](). - **One-class SVM** - Learns a function separating an estimated support region from its complement using kernels. Kernel and regularization choices define the boundary; it is not a labeled anomaly classifier. [Schölkopf et al. 2001]( "Estimating the Support of a High-Dimensional Distribution; Neural Comput."). ## Learned one-class and reconstruction methods - **Deep SVDD** - Learns a representation concentrated near a hypersphere center. Architecture and training restrictions address trivial collapse; a compact representation alone is not evidence of useful anomaly separation. [Ruff et al. 2018]( "Deep One-Class Classification; ICML"). - **Deep SAD** - Extends deep one-class learning to unlabeled data plus labeled normal or anomalous examples. Its label-informed setting differs from training only on presumed normal data. [Ruff et al. 2020]( "Deep Semi-Supervised Anomaly Detection; ICLR"). - **Reconstruction and MemAE** - Ordinary autoencoders can reconstruct anomalies well. Gong et al. restrict reconstruction through a learned memory of normal patterns; low reconstruction error is not inherently a normality guarantee. [Gong et al. 2019]( "Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection; ICCV"). - **DROCC** - Trains against adversarially generated nearby negatives under a local low-dimensional-manifold assumption. The generated contrast examples define the learned one-class boundary. [Goyal et al. 2020]( "DROCC: Deep Robust One-Class Classification; ICML"). ## Classifier-based and out-of-distribution detection - **Maximum softmax probability** - Uses classifier confidence as a baseline for misclassification and out-of-distribution detection. This needs a trained classifier and does not turn confidence into a novelty guarantee. [Hendrycks and Gimpel 2017]( "A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks; ICLR"). - **Outlier Exposure** - Trains detectors using auxiliary outlier data to improve detection of unseen anomalies. Auxiliary-data choice and overlap with evaluated anomalies are part of the supervision protocol. [Hendrycks et al. 2019]( "Deep Anomaly Detection with Outlier Exposure; ICLR"). - **Likelihood failure** - Tested flows, VAEs and PixelCNNs trained on CIFAR-10 can assign higher likelihood to SVHN images. High generative density is not a general test of semantic in-distribution membership. [Nalisnick et al. 2019]( "Do Deep Generative Models Know What They Don't Know?; ICLR"). - **Likelihood-ratio OOD detection** - Compares a generative model with a background model to reduce nuisance-statistic effects. Ren et al. improve detection on genomic and image tasks; the result depends on what the background model removes, not simply on using a density model. [Ren et al. 2019]( "Likelihood Ratios for Out-of-Distribution Detection; NeurIPS"). - **Energy-based OOD scoring** - Uses log-sum-exp of classifier logits to score inputs instead of maximum softmax probability. Supports scoring a pretrained classifier or fine-tuning with outliers; those setups use different supervision. [Liu et al. 2020]( "Energy-based Out-of-distribution Detection; NeurIPS"). ## Sequences and streaming thresholds - **Bayesian online changepoint detection** - Maintains a posterior over time since the last change using a predictive model and hazard function. Detects changes in a sequence's generating regime; an isolated unusual observation need not imply a persistent regime change. [Adams and MacKay 2007]( "Bayesian Online Changepoint Detection; arXiv"). - **Matrix-profile discords** - Rank fixed-length subsequences by distance to their nearest nontrivial match. Window length and normalization define the anomaly; repeated faults may cease to be rare patterns. [Yeh et al. 2016]( "Matrix Profile I: All Pairs Similarity Joins for Time Series: A Unifying View That Includes Motifs, Discords and Shapelets; ICDM"). - **Extreme-value thresholds** - Siffer et al. adapt streaming thresholds from tail behavior and a specified risk parameter. Threshold estimation is separate from learning representations; tail-model fit matters. [Siffer et al. 2017]( "Anomaly Detection in Streams with Extreme Value Theory; KDD"). - **OmniAnomaly** - Uses stochastic recurrent representations and normalizing flows, scoring multivariate sequences by reconstruction probabilities. Per-channel scores localize discrepancies rather than identify causal fault sources. [Su et al. 2019]( "Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network; KDD"). - **Anomaly Transformer** - Contrasts learned temporal attention with a locality prior using association discrepancy. Its anomaly criterion assumes abnormal points have different temporal associations; this is a modeling hypothesis. [Xu et al. 2022]( "Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy; ICLR"). - **TranAD** - Combines transformer reconstruction, self-conditioning and adversarial training for multivariate time series. Detection scores and diagnostic rankings need evaluation under the chosen temporal and threshold protocol. [Tuli et al. 2022]( "TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data; arXiv"). ## Visual anomaly detection and localization - **PaDiM** - Fits multivariate Gaussian distributions to pretrained patch embeddings to detect and localize deviations from normal images. The reference distribution is spatially indexed rather than one global image density. [Defard et al. 2021]( "PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization; ICPR Workshops"). - **PatchCore** - Compares pretrained patch features with a representative memory bank of normal patches. Nearest-reference scoring supports image detection and local defect maps without training a classifier on defect classes. [Roth et al. 2022]( "Towards Total Recall in Industrial Anomaly Detection; CVPR"). - **WinCLIP** - Combines text prompts with window, patch and image features for zero-shot anomaly detection and segmentation. WinCLIP+ additionally uses normal reference images; these are different data-access regimes. [Jeong et al. 2023]( "WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation; CVPR"). ## Comparative evidence and evaluation - **Rare-event metrics** - Davis and Goadrich show why precision-recall curves reveal failures concealed by small false-positive rates on imbalanced data. They prove that ROC dominance and precision-recall dominance are equivalent; optimizing their areas is not equivalent. [Davis and Goadrich 2006]( "The relationship between Precision-Recall and ROC curves; ICML"). - **Dataset artifacts** - Wu and Keogh identify labeling and construction flaws in widely used time-series benchmarks and introduce the UCR anomaly archive. Their criticism concerns examined datasets; it does not invalidate all temporal detection. [Wu and Keogh 2021]( "Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress; TKDE"). - **ADBench** - Compares 30 detectors on 57 datasets. No tested unsupervised detector dominates across datasets; revealing 1% of anomaly labels lets most tested semi-supervised methods beat the best unsupervised baseline. [Han et al. 2022]( "ADBench: Anomaly Detection Benchmark; NeurIPS"). - **Point adjustment** - Crediting an entire true anomaly segment after one detected point can inflate F1, even for random scores. Report unadjusted scores and the exact event-credit rule. [Kim et al. 2022]( "Towards a Rigorous Evaluation of Time-series Anomaly Detection; AAAI"). - **MVTec AD 2** - Adds eight industrial scenarios, including small defects and lighting changes. Heckler-Kram et al. report tested methods below 60% average AU-PRO; earlier near-saturated benchmarks conceal these difficulties. [Heckler-Kram et al. 2026]( "The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection; IJCV"). ## Software - **scikit-learn** - Outlier detection includes Isolation Forest, LOF, one-class SVM and robust covariance. LOF's novelty mode scores new observations; its training-score behavior differs from ordinary outlier mode. [docs](). - **PyOD** - Provides a common Python interface for classical, ensemble and neural outlier detectors. Shared interfaces aid comparisons but do not standardize preprocessing, tuning or label access automatically. [Zhao et al. 2019]( "PyOD: A Python Toolbox for Scalable Outlier Detection; JMLR"). - **Anomalib** - Visual anomaly-detection toolkit with model implementations, data pipelines and evaluation support. Record backbone, preprocessing and reference-image access when comparing methods. [code](). ## Surveys - **Unifying review** - Ruff et al. connect density, support estimation and reconstruction approaches, including shallow/deep relationships. Their organizing assumptions help explain why the same score behaves differently across tasks. [Ruff et al. 2021]( "A Unifying Review of Deep and Shallow Anomaly Detection; Proc. IEEE"). ## Related topics - [Evaluation](https://mlpapers.org/evaluation/) - [Clustering](https://mlpapers.org/clustering/) - [Feature extraction](https://mlpapers.org/feature-extraction/) - [Generative models](https://mlpapers.org/generative-models/) - [Computer vision](https://mlpapers.org/computer-vision/) - [Missing data](https://mlpapers.org/missing-data/)