# Time Series Forecasting ## Forecast targets and information - **Forecast origin and horizon** - Forecasts use information available at a stated origin to predict later observations. Evaluate the horizons actually needed; one-step accuracy need not identify the best multi-step forecaster. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **Local and global forecasting** - Local models fit each series separately; global models share a function across series. Montero-Manso and Hyndman show that global models need not assume similar series; approximation capacity and estimation error remain separate concerns. [Montero-Manso and Hyndman 2021]( "Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality; International Journal of Forecasting"). - **Known and unknown future covariates** - Calendar variables may be known ahead; future weather measurements may not. Regression forecast intervals conditional on assumed covariates omit their uncertainty. Report what was available at each forecast origin. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). ## Statistical foundations and decomposition - **Naïve and seasonal naïve** - Predict the last observation or the last observation at the matching seasonal position. Essential reference forecasts: improvements over a weak neural baseline do not establish gains over these task-specific alternatives. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **Exponential smoothing and ETS** - Updates level, trend and seasonal components with decaying weights. State-space formulations specify error models and prediction intervals; additive and multiplicative components impose different data and variance assumptions. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **ARIMA and seasonal ARIMA** - Models autocorrelation after differencing, with autoregressive and moving-average terms. Differencing addresses stochastic trends; seasonal terms model recurring lag dependence. Residual diagnostics matter alongside information criteria. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **Automatic ETS and ARIMA selection** - Hyndman and Khandakar combine unit-root tests, information criteria and search to automate selection. Provides statistical baselines; automatic order selection does not guarantee adequate residuals or future accuracy. [Hyndman and Khandakar 2008]( "Automatic Time Series Forecasting: The forecast Package for R; J. Stat. Soft."). - **STL and decomposition-based forecasting** - Separates trend, seasonality and remainder; forecast the seasonally adjusted series and restore seasonality. STL permits changing seasonal patterns, but the forecasting step still needs an assumption about future seasonality. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **Dynamic harmonic regression** - Uses Fourier terms for seasonal patterns with ARIMA errors for remaining temporal dependence. Useful for long or multiple seasonal periods; the number of Fourier terms controls seasonal flexibility. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **Vector autoregression** - Predicts several series from their own and each other's lagged values. Captures linear cross-series dynamics; parameter count grows with series count and lag order. Predictive lag relationships do not by themselves establish causal effects. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **GARCH** - Generalizes ARCH by allowing past conditional variances into the current variance equation. Studies stationarity, estimation and testing for time-varying volatility. [Bollerslev 1986]( "Generalized autoregressive conditional heteroskedasticity; Journal of Econometrics"). - **Intermittent demand: Croston** - Smooths nonzero demand sizes and interarrival times separately, then takes their ratio. The original estimator is biased and supplies no model-based prediction intervals; sparse counts require more than ordinary error metrics. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). ## Global neural and probabilistic models - **Deep state-space models** - Uses a shared recurrent network to parameterize a linear state-space model for each series. Combines cross-series learning with structured latent dynamics and probabilistic inference, rather than predicting each horizon independently. [Rangapuram et al. 2018]( "Deep State Space Models for Time Series Forecasting; NeurIPS"). - **DeepVAR and Gaussian copulas** - Couples recurrent dynamics to a low-rank Gaussian-copula distribution for multivariate forecasting. Models cross-series dependence; marginal calibration alone cannot establish that joint scenarios or aggregate forecasts are accurate. [Salinas et al. 2019]( "High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes; NeurIPS"). - **ForGAN** - Uses a conditional GAN for one-step probabilistic forecasting. Learns a distribution over future values rather than only a point prediction. [Koochali et al. 2019]( "Probabilistic Forecasting of Sensory Data with Generative Adversarial Networks - ForGAN; IEEE Access"). - **DeepAR** - Trains an autoregressive recurrent network across related series with a likelihood objective. Samples future trajectories recursively; the likelihood and available covariates determine the forecast distribution's support and conditioning. [Salinas et al. 2020]( "DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks; International Journal of Forecasting"). - **N-BEATS** - Uses residual fully connected blocks with backcast and forecast branches. Generic learned bases and constrained trend/seasonal bases are distinct variants; an interpretable decomposition is an architectural choice, not a property of every N-BEATS model. [Oreshkin et al. 2020]( "N-BEATS: Neural basis expansion analysis for interpretable time series forecasting; ICLR"). - **N-HiTS** - Combines input pooling at multiple scales with hierarchical interpolation of forecast outputs. Extends residual forecasting toward long horizons without attention; pooling and interpolation control which temporal detail is retained. [Challu et al. 2023]( "N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting; AAAI"). - **Temporal Fusion Transformer** - Combines recurrent processing, variable-selection gates and attention for multi-horizon quantiles. Separates static, observed-past and known-future inputs; attention and gate weights describe model behavior, not causal effects. [Lim et al. 2021]( "Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting; International Journal of Forecasting"). - **TiDE** - Uses dense encoder-decoder blocks with past observations and dynamic covariates. Supplies a non-attention baseline for long-horizon forecasting; its linear-system analysis has assumptions beyond the nonlinear benchmark implementation. [Das et al. 2023]( "Long-term Forecasting with TiDE: Time-series Dense Encoder; arXiv"). - **TSMixer** - Alternates MLP mixing across time and features. Compares temporal-only and cross-variate processing; gains from mixing variables depend on the data's cross-series information, not the multivariate label alone. [Chen et al. 2023]( "TSMixer: An All-MLP Architecture for Time Series Forecasting; TMLR"). - **Reservoir forecasting** - Uses input-driven recurrent states or delayed nonlinear features with a fitted readout. See [reservoir computing](https://mlpapers.org/neural-nets/#reservoir-computing) for echo state networks and their nonlinear autoregressive alternative. ## Attention, patches and normalization - **Informer** - Uses sparse attention, attention distillation and a decoder producing multiple future steps together. Targets long-sequence computational cost. [Zhou et al. 2021]( "Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting; AAAI"). - **Autoformer** - Combines progressive trend/seasonal decomposition with an autocorrelation mechanism that aggregates recurring subseries. Encodes periodic structure explicitly instead of relying solely on pointwise attention. [Wu et al. 2021]( "Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting; NeurIPS"). - **Reversible instance normalization** - Normalizes each series instance and restores its scale at the output. Reduces sensitivity to shifting means and variances; it does not guarantee robustness to changes in dynamics or conditional relationships. [Kim et al. 2022]( "Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift; ICLR"); [code](). - **PatchTST** - Tokenizes local time windows as patches and shares transformer weights across channels processed independently. Preserves local patterns while shortening token sequences; cross-variable dependence is not explicitly modeled in the core architecture. [Nie et al. 2023]( "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers; ICLR"). - **Linear-baseline challenge** - Zeng et al. find simple linear models outperform the tested long-horizon transformers on nine datasets. Supports matched linear baselines and lookback tuning; does not show that attention cannot help forecasting. [Zeng et al. 2023]( "Are Transformers Effective for Time Series Forecasting?; AAAI"). - **iTransformer** - Treats each variate's history as a token and applies attention across variates. Reverses the usual time-token arrangement; differs from PatchTST's channel-independent processing rather than simply using a newer attention layer. [Liu et al. 2024]( "iTransformer: Inverted Transformers Are Effective for Time Series Forecasting; ICLR"). ## Pretrained forecasting representations - **Lag-Llama** - Uses lagged values in a decoder-only transformer with a probabilistic output head. Studies zero-shot and fine-tuned forecasting; Student-t outputs in the original model restrict distributional shape. [Rasul et al. 2023]( "Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting; arXiv"). - **TimesFM** - Pretrains a decoder-only model on time-series patches for zero-shot forecasting. Variable context and horizon support concerns the model interface; unseen-series evaluation still requires checking pretraining overlap. [Das et al. 2024]( "A decoder-only foundation model for time-series forecasting; ICML"). - **Chronos** - Scales and quantizes observations into tokens, then trains language-model architectures on time-series sequences. Sampling produces probabilistic forecasts; numerical tokenization differs from transferring a text-pretrained language model. [Ansari et al. 2024]( "Chronos: Learning the Language of Time Series; TMLR"). - **Moirai** - Uses multiple patch sizes, variate-independent attention and a mixture distribution head across frequencies and dimensions. Pretrains on LOTSA; the paper separates in-distribution evaluation from held-out-dataset forecasting. [Woo et al. 2024]( "Unified Training of Universal Time Series Forecasting Transformers; ICML"). - **Time-LLM** - Reprograms time-series patches into representations consumed by a frozen text language model, with prompt context. This transfer mechanism differs from Chronos; whether the language backbone adds value requires component ablations. [Jin et al. 2024]( "Time-LLM: Time Series Forecasting by Reprogramming Large Language Models; ICLR"). - **Language-backbone ablations** - Tan et al. remove or replace language backbones in three LLM forecasters; simpler variants match or improve forecasts with less compute. Evidence concerns those architectures and tasks, not all uses of language with time series. [Tan et al. 2024]( "Are Language Models Actually Useful for Time Series Forecasting?; NeurIPS"). - **TiRex** - Uses [xLSTM](https://mlpapers.org/neural-nets/#recurrent-sequence-models) for univariate zero-shot forecasting. Contiguous patch masking trains multi-patch prediction; ablations improve long-horizon forecasting without the short-horizon degradation seen with end-only masking. Precedes multivariate TiRex-2. [Auer et al. 2025]( "TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning; NeurIPS"). - **TiRex-2** - Combines recurrent xLSTM, cross-variate attention and covariates. Leads the compared zero-shot models on GIFT-Eval and fev-bench using a separately pretrained checkpoint per benchmark; overlap exclusions follow each benchmark's rules. [Podest et al. 2026]( "TiRex-2: Generalizing TiRex to Multivariate Data and Streaming; NeurIPS"). ## Forecast combinations and tabular learners - **Feature-based forecast model averaging** - Learns combination weights from time-series features and training-series forecast errors. Transfers a weighting rule to new series; differs from choosing one model or training a single global forecaster. [Montero-Manso et al. 2020]( "FFORMA: Feature-based forecast model averaging; Int. J. Forecast."). - **Global boosted trees and forecast averaging** - The M5 winner averages direct and recursive LightGBM forecasts trained on several data pools. Combines cross-series learning with different multi-step strategies; gains cannot be assigned to the tree learner alone. [In and Jung 2022]( "Simple averaging of direct and recursive forecasts via partial pooling using machine learning; Int. J. Forecast."). ## Coherence, uncertainty and scoring - **Proper scoring rules** - Scores such as log score and CRPS evaluate forecast distributions, rewarding truthful reporting in expectation. Assess calibration and sharpness together; point-error improvements do not establish better uncertainty estimates. [Gneiting and Raftery 2007]( "Strictly Proper Scoring Rules, Prediction, and Estimation; JASA"). - **Target-consistent forecast scores** - Choose the target before ranking forecasts: squared loss elicits a mean, absolute loss a median, and pinball loss a quantile. Asymmetric decision costs can require a different target; changing metrics can change the optimal forecast. [Gneiting 2011]( "Making and Evaluating Point Forecasts; Journal of the American Statistical Association"). - **Minimum-trace reconciliation** - Projects base forecasts onto aggregation constraints using forecast-error covariance. Minimizes total variance among linear unbiased reconciliations under its assumptions; covariance estimation affects practical performance. [Wickramasuriya et al. 2018]( "Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization; JASA"). - **EnbPI** - Builds sequential prediction intervals from ensemble residuals. Approximate marginal coverage relies on error dependence and estimation assumptions; it is not unconditional finite-sample coverage under arbitrary time-series shift. [Xu and Xie 2021]( "Conformal prediction interval for dynamic time-series; ICML"). - **Probabilistic forecast reconciliation** - Maps joint forecast distributions or samples onto aggregation constraints; learns weights with energy or variogram scores. Extends point reconciliation: coherent means alone do not specify a coherent joint distribution. [Panagiotelis et al. 2023]( "Probabilistic forecast reconciliation: properties, evaluation and score optimisation; Eur. J. Oper. Res."). ## Comparative evidence and evaluation - **Rolling-origin evaluation** - Fit preprocessing and models using only data available before each origin; score relevant horizons across multiple origins. Tune on earlier validation periods and retain a final test period. Random window splits can leak future information. [Tashman 2000]( "Out-of-sample tests of forecasting accuracy: an analysis and review; IJF"), [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **M4 competition** - Across 100,000 series, a hybrid of exponential smoothing and recurrent networks won the point-forecast competition; combinations were prominent. These mostly univariate tasks differ from covariate-rich retail forecasting in M5. [Makridakis and Petropoulos 2020]( "The M4 competition: Conclusions; Int. J. Forecast."). - **Monash forecasting archive** - Collects diverse datasets and baseline results for univariate and multivariate forecasting. Reusing its datasets requires matching horizons, splits and aggregation; dataset count alone does not make two reported averages comparable. [Godahewa et al. 2021]( "Monash Time Series Forecasting Archive; NeurIPS"). - **Scaled errors and aggregation** - MASE/RMSSE scale errors by training naïve differences; constant training series can give zero denominators. Report scaling, horizon weights and aggregation: product-level scores and aggregate-demand scores measure different errors. [Hyndman and Athanasopoulos 2021]( "Forecasting: Principles and Practice; book"). - **M5 accuracy competition** - Evaluates 42,840 hierarchical Walmart sales series with covariates. Top submissions predominantly used global LightGBM models and combinations; this retail result does not establish a universal ranking over statistical and neural forecasters. [Makridakis et al. 2022]( "M5 accuracy competition: Results, findings, and conclusions; Int. J. Forecast."). - **M5 robustness across test periods** - Re-evaluation finds global LightGBM bottom-up forecasts strong at product level but less robust after aggregation across test periods. Cross-series error correlations affect totals; one test window cannot establish stable rankings. [Ma and Fildes 2022]( "The performance of the global bottom-up approach in the M5 accuracy competition: a robustness check; Int. J. Forecast."). - **GIFT-Eval** - Benchmarks pretrained and conventional forecasters across domains, frequencies and horizons. Includes pretraining data intended to avoid test overlap; zero-shot results must be read with the evaluated model's actual training exposure. [Aksu et al. 2024]( "GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation; arXiv"). - **fev-bench** - Adds covariates, multivariate tasks and pairwise statistical comparisons to foundation-model evaluation. Results identify covariate support as valuable; rankings depend on task coverage, failures and known pretraining overlap. [Shchur et al. 2026]( "fev-bench: A Realistic Benchmark for Time Series Forecasting; NeurIPS"). ## Software - **StatsForecast** - Implements statistical forecasters including AutoARIMA, AutoETS and intermittent-demand methods, with forecasting and cross-validation interfaces. Useful for matched statistical baselines; inspect each model's exogenous-input support. [docs](). - **GluonTS** - Provides probabilistic forecasting models, dataset transformations and evaluation tools. Keep training windows, covariates, likelihoods and sampling settings explicit when comparing its models with point-forecast libraries. [docs](). ## Related topics - [Anomaly detection](https://mlpapers.org/anomaly-detection/) - [Continual learning](https://mlpapers.org/continual-learning/) - [Model predictive control](https://mlpapers.org/model-predictive-control/) - [Evaluation](https://mlpapers.org/evaluation/) - [Bayesian inference](https://mlpapers.org/bayesian-inference/) - [Modeling](https://mlpapers.org/models/)