# Symbolic Regression and Equation Discovery

## Search space and discovery targets

- **Expression discovery** - Symbolic regression searches mathematical expressions as well as coefficients. SRBench separates predictive accuracy, expression complexity and exact recovery; a compact accurate formula need not be the generating equation. [La Cava et al. 2021](<https://arxiv.org/abs/2107.14351> "Contemporary Symbolic Regression Methods and their Relative Performance; NeurIPS").
- **Grammar and complexity** - Operators, nesting constraints and expression costs define which equations are searchable. PySR exposes these choices explicitly; changing the grammar changes the problem, not just the optimizer. [code](<https://github.com/astroautomata/PySR/blob/master/docs/src/options.md>).

## Evolutionary and structured search

- **Genetic programming / Operon** - Evolves expression trees and optimizes numerical constants. Operon provides a C++ implementation; combining structural search with coefficient fitting differs from selecting terms in one fixed linear dictionary. [code](<https://github.com/heal-research/operon>).
- **AI Feynman** - Combines neural fitting with recursive tests for symmetry, separability and other simplifying structure. Udrescu and Tegmark's search exploits properties of physics formulas rather than treating every expression equally. [Udrescu and Tegmark 2020](<https://arxiv.org/abs/1905.11481> "AI Feynman: a Physics-Inspired Method for Symbolic Regression; Science Advances").
    - **AI Feynman 2.0** - Uses gradients of a fitted neural model to discover generalized symmetries and graph modularity. Returns accuracy-complexity tradeoffs and extends discovery to probability densities through normalizing flows. [Udrescu et al. 2020](<https://arxiv.org/abs/2006.10782> "AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity; NeurIPS").
- **PySR / SymbolicRegression.jl** - Uses multiple evolving populations with an evolve-simplify-optimize loop. Searches expression structure while fitting constants and retaining accuracy-complexity alternatives; Python wraps a Julia backend. [Cranmer 2023](<https://arxiv.org/abs/2305.01582> "Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl; arXiv").

## Probabilistic, exhaustive and shared-structure search

- **Bayesian machine scientist** - Samples equations with MCMC using an approximate model posterior and a prior learned from a corpus of formulas. Retaining alternatives represents uncertainty over expression structure rather than selecting only one best-fitting equation. [Guimera et al. 2020](<https://arxiv.org/abs/2004.12157> "A Bayesian machine scientist to aid in the solution of challenging scientific problems; Sci. Adv.").
- **Exhaustive symbolic regression** - Enumerates expressions within an operator set and size bound, then ranks them by minimum description length. Exhaustive structure search does not ensure a global optimum unless numerical constants are also optimized correctly. [Bartlett et al. 2024](<https://arxiv.org/abs/2211.11461> "Exhaustive Symbolic Regression; IEEE Trans. Evol. Computat.").
- **Multi-level symbolic regression** - Learns a shared expression structure with group-specific coefficients. Parallel searches are merged across groups; MSRBench tests this intermediate setting between one pooled equation and unrelated fits for every group. [Fong and Motani 2024](<https://proceedings.mlr.press/v238/sen-fong24a.html> "Multi-Level Symbolic Regression: Function Structure Learning for Multi-Level Data; AISTATS").

## Sparse dynamical-system identification

- **SINDy** - Selects sparse combinations of candidate functions to model observed state derivatives. Recovery depends on the state variables, function library, derivative estimates and excitation in the trajectories. [Brunton et al. 2016](<https://arxiv.org/abs/1509.03580> "Discovering governing equations from data: Sparse identification of nonlinear dynamical systems; PNAS").
    - **PDE-FIND** - Rudy et al. select nonlinear and spatial-derivative terms by sparse regression. Multiple trajectories can distinguish equations that fit one trajectory; derivative estimation and library choice affect recovery. [Rudy et al. 2017](<https://arxiv.org/abs/1609.06401> "Data-driven discovery of partial differential equations; Science Advances").
    - **SINDy-PI** - Searches implicit equations and rational dynamics through parallel sparse regressions. It avoids the noise-sensitive null-space approach of earlier implicit SINDy and extends recovery to studied ODEs, PDEs and conservation laws. [Kaheman et al. 2020](<https://arxiv.org/abs/2004.02322> "SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics; Proc. R. Soc. A.").
    - **Weak SINDy** - Uses integrated weak-form equations instead of pointwise derivative estimates. Messenger and Bortz recover dynamics more robustly under white measurement noise in tested systems; test functions and the candidate library remain part of identification. [Messenger and Bortz 2021](<https://arxiv.org/abs/2005.04339> "Weak SINDy: Galerkin-Based Data-Driven Model Selection; Multiscale Model. Simul.").
- **PySINDy** - Python implementation of sparse dynamical-system identification. Treat differentiation, candidate-library construction and sparse fitting as parts of the identification procedure, not interchangeable preprocessing details. [de Silva et al. 2020](<https://joss.theoj.org/papers/10.21105/joss.02104> "PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data; JOSS").
- **Universal differential equations** - Combines known dynamics with trainable unknown terms. Rackauckas et al. fit hybrid systems and use symbolic regression to extract missing mechanisms; the equation is constrained by supplied physics and observed trajectories. [Rackauckas et al. 2020](<https://arxiv.org/abs/2001.04385> "Universal Differential Equations for Scientific Machine Learning; arXiv").

## Neural search and pretrained equation models

- **Equation learning networks** - Sahoo et al. use a shallow network with mathematical operations, including division, to identify concise equations. Extrapolation succeeds in studied analytical systems; it must be tested beyond the fitting range. [Sahoo et al. 2018](<https://arxiv.org/abs/1806.07259> "Learning Equations for Extrapolation and Control; ICML").
- **Deep symbolic regression** - An RNN samples expressions; risk-seeking policy gradients emphasize high-reward candidates. Petersen et al. optimize the expression search distribution, rather than using the neural network as the final predictor. [Petersen et al. 2021](<https://arxiv.org/abs/1912.04871> "Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients; ICLR").
- **Neural-guided genetic programming** - A neural generator seeds repeated genetic-programming searches and learns from their results. Mundhenk et al. find loosely coupled seeding effective in their benchmarks; the method combines search mechanisms. [Mundhenk et al. 2021](<https://arxiv.org/abs/2111.00053> "Symbolic Regression via Neural-Guided Genetic Programming Population Seeding; NeurIPS").
- **Neural Symbolic Regression that Scales** - Pretrains a transformer on generated equations and sampled input/output pairs, then uses its predictions to guide equation search. The synthetic equation distribution supplies the learned search prior. [Biggio et al. 2021](<https://arxiv.org/abs/2106.06427> "Neural Symbolic Regression that Scales; ICML").
- **End-to-end symbolic regression** - A transformer predicts expressions including numerical constants, optionally followed by constant refinement. Kamienny et al. contrast this with predicting only an expression skeleton. [Kamienny et al. 2022](<https://arxiv.org/abs/2204.10532> "End-to-end symbolic regression with transformers; NeurIPS").
- **Φ-SO: dimensional constraints** - Generates expressions with consistent physical units during neural-guided search. Tenachi et al. reduce the candidate space through dimensional analysis; unit consistency alone does not establish a correct physical law. [Tenachi et al. 2023](<https://arxiv.org/abs/2303.03192> "Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws; ApJ").
- **LLM-SR** - Uses a language model to propose equation programs, fits their constants to data and evolves the candidates using fit feedback. Scientific descriptions provide prior information beyond numeric samples; the evaluation must distinguish discovery from recall. [Shojaee et al. 2025](<https://arxiv.org/abs/2404.18400> "LLM-SR: Scientific Equation Discovery via Programming with Large Language Models; ICLR").

## Comparative evidence and evaluation

- **SRBench findings** - Across 252 problems, strong real-data methods combine genetic search with parameter fitting or semantic search. Neural and genetic approaches perform similarly on noisy exact-recovery tasks in this study. [La Cava et al. 2021](<https://arxiv.org/abs/2107.14351> "Contemporary Symbolic Regression Methods and their Relative Performance; NeurIPS").
    - **Expanded SRBench comparison** - Adds methods and measures accuracy, expression size and energy. Some accuracy plots use the best test score across 30 runs: these describe an optimistic search envelope, not performance of a model selected without test labels. [Aldeia et al. 2025](<https://arxiv.org/abs/2505.03977> "Call for Action: towards the next generation of symbolic regression benchmark; GECCO Companion").
- **SRSD** - Rebuilds physics-formula datasets with realistic variable ranges and adds irrelevant inputs. Matsubara et al. show that sampling and evaluation choices change what a symbolic-regression benchmark measures. [Matsubara et al. 2022](<https://arxiv.org/abs/2206.10540> "Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery; arXiv").
- **Equation equivalence** - Evaluate prediction, complexity and symbolic recovery separately. Equivalent algebraic forms need not have identical expression trees; SRSD's edit-distance metric measures structural similarity, not a general equivalence proof. [Matsubara et al. 2022](<https://arxiv.org/abs/2206.10540> "Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery; arXiv").
- **LLM-SRBench** - Uses transformed and synthetic equations to probe discovery beyond recall. It contains 239 problems in four domains; the best tested system reaches 31.5% symbolic accuracy on the transformed-equation track. [Shojaee et al. 2025](<https://proceedings.mlr.press/v267/shojaee25a.html> "LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models; ICML").

## Related topics

- [Feature selection](https://mlpapers.org/feature-selection/)
- [Neurosymbolic learning](https://mlpapers.org/neurosymbolic-learning/)
- [Model predictive control](https://mlpapers.org/model-predictive-control/)
- [AutoML](https://mlpapers.org/automl/)
- [Interpretability](https://mlpapers.org/interpretability/)
- [Evaluation](https://mlpapers.org/evaluation/)