Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws

Wassim Tenachi; Rodrigo Ibata; Foivos I. Diakogiannis

Source-linked AI summary

Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws

Wassim Tenachi, Rodrigo Ibata, Foivos I. Diakogiannis

arXiv:2303.03192v2Mar 6, 2023astro-ph.IMcs.LGphysics.comp-ph

TL;DR

Physics symbolic regression must discover analytic expressions while respecting dimensional consistency, a constraint often neglected by general symbolic-regression methods. Φ-SO integrates physical units into recurrent expression generation and trains the generator with deep reinforcement learning. It achieves leading benchmark performance above 0.1% noise and remains consistent through 10% noise, while astrophysical examples recover known forms and identify simpler fits in isolated observational data.

Problem
Existing symbolic-regression methods often explore candidates with unphysical units, while physics requires equations to be dimensionally balanced and interpretable.
Method
Φ-SO generates symbolic expressions with an RNN that tracks expression structure and applies in situ units constraints, trained using deep reinforcement learning.
Results
Φ-SO achieved state-of-the-art performance above 0.1% noise and consistent performance up to 10% noise across 120 Feynman and textbook cases.
Takeaways & Limitations
Dimensional-analysis constraints improve exact symbolic recovery and can support interpretable physical-law discovery from synthetic or observational data.
Takeaways & Limitations
Φ-SO currently requires the physical units of the free parameters it may use, which may be unclear in novel empirical settings.

Abstract

from arXiv · show

Symbolic Regression is the study of algorithms that automate the search for analytic expressions that fit data. While recent advances in deep learning have generated renewed interest in such approaches, the development of symbolic regression methods has not been focused on physics, where we have important additional constraints due to the units associated with our data. Here we present $Φ$-SO, a Physical Symbolic Optimization framework for recovering analytical symbolic expressions from physics data using deep reinforcement learning techniques by learning units constraints. Our system is built, from the ground up, to propose solutions where the physical units are consistent by construction. This is useful not only in eliminating physically impossible solutions, but because the "grammatical" rules of dimensional analysis restrict enormously the freedom of the equation generator, thus vastly improving performance. The algorithm can be used to fit noiseless data, which can be useful for instance when attempting to derive an analytical property of a physical model, and it can also be used to obtain analytical approximations to noisy data. We test our machinery on a standard benchmark of equations from the Feynman Lectures on Physics and other physics textbooks, achieving state-of-the-art performance in the presence of noise (exceeding 0.1%) and show that it is robust even in the presence of substantial (10%) noise. We showcase its abilities on a panel of examples from astrophysics.

1. INTRODUCTION

Symbolic regression seeks compact mathematical expressions from data, but physics additionally requires dimensional consistency and interpretable models. Φ-SO incorporates physical units during expression generation to reduce the search space and support physically plausible symbolic discovery.

1. INTRODUCTION: Physics symbolic regression must recover functions while ensuring equations remain balanced in physical units.Expressions that fit numerical data can still be physically nonsensical when their units are inconsistent.
1. INTRODUCTION: Naive symbolic regression faces an enormous combinatorial search space, reaching approximately 1.5×10^41 candidates for 35-symbol expressions with 15 available variables or operations.Free-constant optimization further increases the computational burden, motivating strategies that prune poor guesses.
1. INTRODUCTION: Φ-SO applies units constraints in situ, proposing only expressions with balanced units by construction and thereby greatly reducing the equation search space.The framework stores and manages dimensional information while expressions are generated.
1. INTRODUCTION: The method combines a physics-oriented symbolic embedding with recurrent neural networks and reinforcement-learning strategies adapted from deep symbolic regression.Unlike dimensionless preprocessing, it anticipates the required units of subsequent symbols in partially composed expressions.
1. INTRODUCTION: The framework is presented as a first step toward general-purpose symbolic regression for astrophysics and other physical sciences, while concrete astrophysical applications are left to future studies.The paper emphasizes algorithmic capabilities and potential rather than claiming completed astrophysical law discovery.

2. RELATED WORKS – A BRIEF SURVEY OF MODERN SYMBOLIC REGRESSION

Modern symbolic regression spans evolutionary, neural, and reinforcement-learning approaches that generate or fit mathematical expressions. Deep reinforcement learning has become especially important for exact symbolic recovery, particularly under noise.

2. RELATED WORKS – A BRIEF SURVEY OF MODERN SYMBOLIC REGRESSION: Genetic programming traditionally evolves populations of mathematical expressions through selection, crossover, and mutation.Eureqa is a well-known example of this approach, alongside more recent genetic-programming methods.
2. RELATED WORKS – A BRIEF SURVEY OF MODERN SYMBOLIC REGRESSION: Neural symbolic-regression methods range from neural architectures augmented with symbolic components to supervised networks that map datasets directly to symbolic functions.This literature includes approaches for sparse symbolic fitting, end-to-end expression prediction, and neural-network interpretability.
2. RELATED WORKS – A BRIEF SURVEY OF MODERN SYMBOLIC REGRESSION: The deep symbolic regression framework of Petersen et al. has become a standard for exact symbolic function recovery, especially in noisy settings.Several subsequent studies build on this reinforcement-learning framework.

3. METHOD

Φ-SO generates symbolic expressions as recurrently sampled token sequences while tracking their tree structure and physical units. Local units constraints mask invalid tokens during generation, and reinforcement learning trains the generator toward accurate, physically valid expressions.

3.1. Generating symbolic expressions: The method represents mathematical expressions as binary trees and linearizes them in prefix notation so an RNN can generate token sequences.Operators precede their operands in prefix notation, which has a one-to-one relationship with the tree and infix representations.
3.1. Generating symbolic expressions: At each generation step, the RNN uses expression context, required token units, and dangling-number information to produce a token distribution that is masked by local units constraints.The resulting process forbids tokens that would lead to nonsensical expressions and samples a complete physically valid sequence.
3.1. Generating symbolic expressions: Φ-SO maintains the full expression-tree graph and grammatical information, including units, symbol types, free parameters, constants, and argument counts, while generation proceeds.This representation supports priors that depend on the entire tree without recomputing it from scratch at every step.
3.2. In situ physical units constraints: In situ units reasoning computes constraints on incomplete trees, although some node units can remain free until later tokens resolve the expression structure.This is harder than post hoc checking because partially sampled trees may not yet determine all dimensional requirements.
3.2. In situ physical units constraints: The units prior combines required and surrounding-node units with the RNN distribution to zero out forbidden symbols, so generated expressions satisfy physical units by construction.This prior can also be paired with dimensional-data symbolic regression approaches such as those contrasted with AI Feynman.
3.3. Learning: Because symbolic token choices are nondifferentiable, Φ-SO trains its RNN with reinforcement learning, rewarding sampled expressions according to their fit to the data.The method uses risk-seeking policy gradients and reinforces only the best 5% of candidates in each batch.

4. FEYNMAN BENCHMARK

On the 120-equation Feynman benchmark, PhySO was evaluated under SRBench-style procedures and compared with established symbolic-regression baselines. Dimensional analysis improved recovery under noise while maintaining accurate, compact expressions.

4.2. Exact symbolic recovery: 58.5% exact symbolic recovery for PhySO versus roughly 42% for DSR demonstrates the benefit of combining reinforcement learning with dimensional analysis.The comparison uses Figure 4’s Feynman-benchmark recovery results; DSR relies solely on reinforcement learning.
4.2. Exact symbolic recovery: PhySO maintained consistent recovery at 0.1% noise, when many high-performing methods nearly halved their recovery rates.The benchmark emphasizes noise resilience because real-world physics data are often noisy.
4.2. Exact symbolic recovery: At 10% noise, PhySO recovered accurate expressions over 53% of the time, while uDSR and AI Feynman 2.0 reached 10.7% and 0.7%, respectively.Most other methods fell below 20% at this noise level, whereas PhySO remained comparatively stable.
4.3. Fit quality: Normalizing accurate-expression rates by free-constant count favored PhySO because competing methods often achieved fits with highly complex, parameter-heavy expressions.AI Feynman 2.0, Operon, and MRGP averaged 18, 17, and 88 free constants, respectively, in the cited noisy setting.

5. ASTROPHYSICAL CASE STUDIES

Φ-SO is applied to astrophysical problems including relativistic energy, the Hubble diagram, galactic actions, and supplementary test cases. Across these studies, it recovers known expressions, explores accuracy–complexity trade-offs, and reveals how units-aware components and reinforcement learning affect recovery.

5.1. Relativistic energy: Φ-SO recovers the relativistic energy expression 100% of the time, while also recovering the lower-complexity classical approximation.The recovered expressions form a Pareto front balancing accuracy and complexity.
5.2. Expansion of the Universe: For the Hubble diagram, Φ-SO recovers ΛCDM from synthetic data but finds lower-complexity, more accurate solutions on Pantheon observations.The authors caution that the observational fit uses noisy, low-redshift-heavy data without other constraints such as the cosmic microwave background.
5.3. Isochrone action from galactic dynamics: The isochrone-action expression was not recovered directly within 10 million guesses, but separating its additive terms enabled Φ-SO to recover both components.The direct problem also resisted standard DSR and AI Feynman 2.0, while fitted-constant subtraction allowed recovery of the second term.
5.5. Ablation study: Ablations show that the external units prior alone is insufficient: local units information and deep reinforcement learning are essential to Φ-SO’s recovery performance.The study also tests removing the units prior, local units constraints, or the neural network, including random-search alternatives.
5.5. Ablation study: The units prior can slow convergence on simple cases such as NFW when its restrictive constraints are combined with random search or without local units information.In that case, a standalone random number generator can achieve a higher recovery rate than some units-guided variants.
5.4. Supplementary cases: Exact analytical expressions substantially outperform a multilayer perceptron on the supplementary test cases.The comparison uses a five-layer, 32-unit MLP trained to convergence with sigmoid activations.

6. DISCUSSION

Φ-SO uses learned dimensional constraints to restrict symbolic-regression search toward physically valid expressions, improving exact recovery and predictive performance while retaining several practical limitations.

Method and motivation: Dimensional constraints reduce the search space and let the neural network learn units rules, addressing the mismatch between fit quality and correct symbolic arrangement.The framework guides exploration toward physically sound expressions rather than optimizing numerical fit alone.
Advantages and extensions: Units guidance gives Φ-SO an edge in exact symbolic recovery and improves performance from a predictive standpoint, while its components can also be combined with other symbolic-regression methods.The authors describe applications to probabilistic, neuro-symbolic, and genetic-programming approaches.
Limitations: Φ-SO currently requires the physical units of its free parameters, so novel empirical problems may need one unit-matched free parameter per variable.This protocol can provide flexibility to combine parameters and recover physical constants.
Limitations: Adding many free parameters can expand the search space, although units constraints, dimensionless-operation rules, and complexity preferences continue to restrict candidate expressions.The authors state that inferred constant units generally remain close to the problem’s unit family.
Limitations: Each task is trained independently from a reinitialized network, so Φ-SO does not benefit from prior experience or pretraining on known physical forms.The authors frame this independence as potentially unbiased for discovering new physics.
Future improvements: Combining supervised knowledge with reinforcement learning and existing simplification procedures is proposed as a route toward recovering more complex expressions.The authors specifically identify separability preprocessing as straightforward to integrate.

7. CONCLUSIONS

The paper presents Φ-SO as a dimensional-analysis-guided symbolic-regression algorithm and evaluates it on textbook benchmarks and astrophysical examples. The results support its usefulness for recovering exact or non-obvious physical expressions, while emphasizing dependence on data range and priors.

Conclusions: Φ-SO generates symbol sequences while cumulatively tracking physical units, training a recurrent network through deep reinforcement learning to produce accurate and physically sound expressions.The method learns local units constraints during symbolic generation.
Conclusions: Benchmarking on 120 textbook cases against 17 baseline approaches showed state-of-the-art performance above 0.1% noise and consistent performance through 10% noise.The comparison also demonstrated the usefulness of dimensional-analysis constraints relative to a purely reinforcement-learning baseline.
Astrophysical applications: Astrophysical tests recovered special-relativistic particle energy, the standard Hubble-parameter form, and a galactic-dynamics equation after additive-separability preprocessing was applied.The Hubble-data test also found simpler alternatives that fit the supernova data better in isolation.
Scope: The tests demonstrate applicability to real-world data and analytic properties of mathematical physical models, but discovered laws depend on data range and prior choices.The authors present this as a step toward connecting observational data with theory.
Future work: Future work will add differential and integral operators to address ordinary and partial differential equations with physical-unit constraints and target unknown astrophysical relationships.The planned direction is motivated by large observational surveys.

A. DATASETS FOR THE ASTROPHYSICAL EXAMPLES

The astrophysical datasets use synthetic, noiseless inputs generated from target expressions at arbitrary scales. Supporting tables specify the target formulas, variable ranges and units, and constant values and units.

Dataset construction: Each astrophysical case contains 1000 noiseless data points generated under a random uniform law with arbitrary mock-data scales.The dataset appendix identifies the target expressions and the variables and constants used in them.
Dataset specification: Table 6 lists the target expressions and marks Φ-SO input variables and free parameters separately from fixed constants.Input variables are colored red, free parameters blue, and fixed constants black.
Dataset specification: Table 7 records the data ranges and units for output and input variables in the astrophysical examples.
Dataset specification: Table 8 records target values and units for constants appearing in the astrophysical examples.

B. DISCOVERING BOTH ANALYTICAL LAWS & CONSTANTS OF NATURE

Φ-SO is extended to discover both symbolic laws and the physical constants needed to make them dimensionally consistent. In a demanding setup with unknown parameter units, it recovers exact expressions and corresponding constants across several physics examples.

B. DISCOVERING BOTH ANALYTICAL LAWS & CONSTANTS OF NATURE: The proposed protocol supplies one free parameter per input variable and one for the output, assigning each the corresponding variable units.This addresses cases where appropriate free parameters and their units are not immediately evident.
B. DISCOVERING BOTH ANALYTICAL LAWS & CONSTANTS OF NATURE: Φ-SO recovers precise symbolic expressions and corresponding physical constants with accurate units in the demanding discovery setup.The experiments use the challenges listed in Table 9, with dataset details in Tables 10 and 11.
B. DISCOVERING BOTH ANALYTICAL LAWS & CONSTANTS OF NATURE: Φ-SO derives the ideal-gas equation of state P = C nT and rediscovers the ideal gas constant through the units of θnθT.The recovered constant has units M.L2.T −2.K−1.N −1 and is usually denoted R.
B. DISCOVERING BOTH ANALYTICAL LAWS & CONSTANTS OF NATURE: For terminal velocity, Φ-SO recovers the dependence on mass, surface area, and medium density, with a factor corresponding to the square root of Earth-surface gravity.The result identifies the relevant scale factor through the available free parameters.
B. DISCOVERING BOTH ANALYTICAL LAWS & CONSTANTS OF NATURE: Φ-SO identifies gravitational force as F = Cm1m2/r2 and recovers the units of the gravitational constant, while also recovering the black-body photon number-density expression.The photon result uses C = h/kb, where h and kb are the Planck and Boltzmann constants.