Source-linked AI summary
A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge
Liron Simon Keren, Alex Liberzon, Teddy Lazebnik
TL;DR
Symbolic regression faces large search spaces and overfitting challenges, while physical modeling also calls for domain knowledge. SciMED integrates scientist-provided knowledge with multiple SR components; experiments report improved success with domain knowledge and a specific advantage over AI Feynman in one case.
Problem
Symbolic regression faces large expression spaces, slow convergence, and overfitting, while physics-oriented systems can rely on restrictive assumptions and be sensitive to noise.
Source: Related work · p. 2, State‑of‑the‑art SR systems · p. 3Method
SciMED combines user-directed feature selection, AutoML, genetic-algorithm symbolic regression, and Las Vegas search, with opportunities to insert physical knowledge.
Source: Scientist‑Machine Equation Detector · p. 4Results
Domain knowledge raised experiment E’s success rate from 65% to 100%; in experiment C, SciMED found the correct equation within 0.76% error of the numerical prefactor while AI Feynman converged to a false equation.
Source: Discussion · p. 13Takeaways & Limitations
SciMED lets users direct physical-data search with domain knowledge, aiming for more credible results with fewer resources.
Source: Scientist‑Machine Equation Detector · p. 4False user hypotheses can overconstrain the search, and SciMED handles differential equations only when written in one-sided form.
Source: Discussion · pp. 13–14
Abstract
from arXiv · showhide
Discovering a meaningful symbolic expression that explains experimental data is a fundamental challenge in many scientific fields. We present a novel, open-source computational framework called Scientist-Machine Equation Detector (SciMED), which integrates scientific discipline wisdom in a scientist-in-the-loop approach, with state-of-the-art symbolic regression (SR) methods. SciMED combines a wrapper selection method, that is based on a genetic algorithm, with automatic machine learning and two levels of SR methods. We test SciMED on five configurations of a settling sphere, with and without aerodynamic non-linear drag force, and with excessive noise in the measurements. We show that SciMED is sufficiently robust to discover the correct physically meaningful symbolic expressions from the data, and demonstrate how the integration of domain knowledge enhances its performance. Our results indicate better performance on these tasks than the state-of-the-art SR software packages , even in cases where no knowledge is integrated. Moreover, we demonstrate how SciMED can alert the user about possible missing features, unlike the majority of current SR systems.
Related work
Symbolic regression offers a way to discover models, but its search and overfitting challenges are especially salient for noisy, nonlinear physical data. Existing approaches integrate physical knowledge in different ways, while SciMED centers the scientist in that process.
Related work
- SR faces combinatorially large expression spaces, slow convergence, and overfitting risks, while physical models also require dimensional homogeneity and can expose mechanisms or useful variable combinations.These demands motivate integrating physical reasoning into model search.Source: Related work · p. 2
SR methods.
- Brute-force search can test all equations but is computationally infeasible in naive form; sparse regression, neural methods, and genetic algorithms offer alternative search strategies.Genetic algorithms can encode prior knowledge, while neural SR is described as resistant to outliers.Source: SR methods · pp. 2–3
Knowledge integration for SR systems.
- Knowledge integration reduces the search space through structural constraints, physical laws, or user-applied mathematical and logical constraints.Structural assumptions include monotonicity, convexity, and symmetry; physical-law approaches encode first principles or more specific physical knowledge.Source: Knowledge integration for SR systems · p. 3
- The authors argue that the scientist should remain central to SR because knowledge can be lost during automated execution.This motivates a scientist-in-the-loop pipeline.Source: Knowledge integration for SR systems · p. 3
State‑of‑the‑art SR systems.
- AI Feynman combines neural-network fitting, recursive problem decomposition, simplifying-property detection, and dimensional analysis to solve physical SR tasks.Its reported success on 120 physical equations is accompanied by concerns about restrictive assumptions and sensitivity to noise.Source: State‑of‑the‑art SR systems · p. 3
- AI Feynman’s assumptions may exclude implicit mechanisms and prevent users from specifying suspected target-related dimensionless numbers; the system is also sensitive to noise.The cited examples include nonlinear drag and mechanical linkages.Source: State‑of‑the‑art SR systems · p. 3
- SciMED is compared with AI Feynman and GP-GOMEA, selected respectively for physics-oriented SR and accurate, simple models; GP-GOMEA prioritizes equation brevity without physical knowledge.SRBench reported Operon as strongest in accuracy and GP-GOMEA in expression simplicity, while physics-oriented systems averaged worse on its largely nonphysical datasets.Source: State‑of‑the‑art SR systems · pp. 3–4
Scientist‑Machine Equation Detector
SciMED combines four user-configurable components to search for symbolic expressions from physical datasets, with knowledge inputs intended to guide the search toward credible results with fewer resources. Its feature selection, AutoML, and two symbolic-regression components provide distinct ways to shape and evaluate candidate solutions.
Scientist‑Machine Equation Detector
- SciMED combines a-priori feature selection, AutoML, GA-based symbolic regression, and Las Vegas symbolic regression, each allowing users to direct the search with physical knowledge or assumptions.Users may choose the GA, LV, or both SR components; GA-SR is the default first step, with LV-SR initiated if accuracy or stability criteria are not met.Source: Scientist‑Machine Equation Detector · p. 4
A priori feature selection component.
- Grouped feature alternatives let SciMED test hypotheses about informative parameters without enlarging the feature search passed to symbolic regression.Users define groups of features that represent the same knowledge; one feature from each group proceeds to SR.Source: A priori feature selection component · pp. 4–5
Automatic machine learning extrapolation component.
- AutoML fits a black-box predictor, tests its stability and extrapolation, and can add synthetic samples to improve input-space coverage for symbolic regression.When a-priori feature selection is used, AutoML also scores feature subsets; it can provide physical insight into the selected feature groups before symbolic expressions are obtained.Source: Automatic machine learning… · p. 5
Genetic algorithm based symbolic regression component.
- GA-SR evolves S-expression candidates using selection, mutation, and crossover, then evaluates them with user-provided loss functions and k-fold evaluation.Its default loss combines L1, L2, and L∞ prediction errors with a size penalty; repeated runs are checked for performance convergence and recurring equations.Source: Genetic algorithm based symbolic… · pp. 5–6
Las Vegas symbolic regression component.
- LV-SR samples full binary-tree candidates, fits their coefficients by linear regression, and iteratively concentrates sampling around promising candidates, trading greater stability for computational cost.Users can guide its sampling strategy or constrain candidate structure; the component can be skipped when GA-SR results meet user-defined consistency criteria.Source: Las Vegas symbolic regression component · p. 7
Scientist‑in‑the‑loop integration points.
- Scientists can guide five parts of the search: feature grouping, GA-SR loss and feature weights, LV-SR sampling, and equation-structure constraints.Feature groups can also be used to explore educated hypotheses without increasing SR’s search space; non-dimensional datasets are encouraged after independent dimensional analysis.Source: Scientist‑in‑the‑loop integration points · p. 7
- Hyperparameter choices trade evaluation accuracy and computation time, while the LV-SR size range must include the optimal topology without making the search enormous.The authors suggest 5 or 10 cross-validation folds and synthetic data amounting to 5%–20% of the original dataset; default hyperparameters were trial-and-error tested on other equations and datasets.Source: Scientist‑in‑the‑loop integration points · pp. 7–8
Results
Across experiments A–E, SciMED was tested against AI Feynman and GP-GOMEA, including noisy and feature-rich cases; further experiments assessed domain knowledge and noise robustness. Results show SciMED’s comparative strengths, while also illustrating how AutoML and symbolic-regression performance can flag a potentially missing variable.
Experimental design.
- Experiments A–E compared SciMED’s AutoML and two SR components with AI Feynman and GP-GOMEA; the evaluation also tested domain-knowledge integration and resistance to noise.For A–E, results included AutoML predictions, component performance scores, and discovered equations.Source: Experimental design · p. 8, Experiments A‑E: Comparison of SciMED… · p. 9
Experiments A‑E: Comparison of SciMED to AI Feynman and GP‑GOMEA.
- In experiment D, good AutoML but poor LV-SR performance indicated that at least one dependent variable was missing from the dataset.The removed variable depended on other supplied variables, which may explain why prediction accuracy declined less than expected; AI Feynman and GP-GOMEA did not terminate within 12 hours.Source: Experiments A‑E: Comparison of SciMED… · p. 9
- In experiment B, all systems identified the two equation features among 33 and their relationship, but SciMED’s prefactor error was 0.02 versus 0.33 for AI Feynman and 0.05 for GP-GOMEA.SciMED also added a constant term of 0.03; the other systems did not.Source: Experiments A‑E: Comparison of SciMED… · p. 9
- In experiment C, SciMED and GP-GOMEA identified the equation and prefactor, while AI Feynman omitted a parameter and misidentified the relationships and prefactor.SciMED’s prefactor was within 0.76% error and linked to gravitational acceleration; GP-GOMEA’s prefactor error was 0.06, while SciMED’s was 0.1.Source: Experiments A‑E: Comparison of SciMED… · p. 9
- In experiment E, SciMED alone found the correct features and algebraic relationships without domain knowledge.The experiment used noisy measurements with 12 features, of which four were needed for the equation.Source: Experimental design · p. 8, Experiments A‑E: Comparison of SciMED… · p. 9
Experiment F: domain knowledge integration.
- Across 32 experiment F configurations, adding more domain knowledge increased the correct-equation rate and reduced normalized computational time.The tested knowledge included feature selection, an alternative loss function, feature weighting, candidate sampling, and an equation-structure constraint.Source: Experimental design · p. 9, Experiment F: domain knowledge… · pp. 10–11
Experiment G: noise analysis.
- In noise analysis, the more complex equation was more noise-sensitive, and LV-based SR outperformed GA-based SR at higher noise levels in both cases.Experiments A and C were repeated 100 times per noise type and level, varying noise in inputs, targets, or both.Source: Experiment G: noise analysis · p. 11
Discussion
SciMED combines scientist-guided domain knowledge with several search components, and the reported experiments show strong performance across varied cases. Its results also identify limits tied to user assumptions, equation complexity, and supported equation forms.
Discussion
- SciMED combines computational frameworks with a scientist-in-the-loop approach, using four independently switchable components and user-provided domain knowledge.The paper also describes feature groups as a way to explore theories involving complex interactions between groups.Source: Discussion · pp. 11, 13
- SciMED slightly outperformed AI Feynman and GP-GOMEA in experiment B by finding a more accurate numerical prefactor; in experiment E, it alone detected the correct features and relation.Experiments A–B highlight GA-based SR and feature selection; C–D highlight LV-based SR and AutoML.Source: Discussion · p. 13
- In experiment C, SciMED found the correct equation within 0.76% numerical-prefactor error; in D, it alerted users to a possibly missing variable while competitors reported equations over 30 terms.AI Feynman converged to a false equation in C; AI Feynman and GP-GOMEA failed to terminate in D despite significant computational efforts.Source: Discussion · p. 13
- In experiment E, SciMED alone accurately detected the correct features and their algebraic relation in a highly noisy dataset requiring selection of 4 of 12 similar-valued features.Experiments A–E compared SciMED with AI Feynman and GP-GOMEA.Source: Discussion · p. 13
- In experiment F, domain knowledge raised SciMED’s success rate from 65% to 100% while decreasing computational time expense.Figure 4 records correct-result percentages and normalized computational time across knowledge-insertion configurations.Source: Figure 4 · p. 12, Discussion · p. 13
- Across noise types, SciMED withstands high noise, but becomes more sensitive as equation complexity increases; the Las Vegas-based component outperforms the GA-based component at higher noise levels.Figure 5 compares correct outcomes for the SR components under noise in inputs, targets, or both, using experiments A and C.Source: Figure 5 · p. 12, Discussion · p. 13
- False user hypotheses can shrink the search space enough to make deducing the correct equation more complicated or impossible.Source: Discussion · p. 13
- SciMED grows more noise-sensitive with equation complexity, depends on its supplied elementary functions, and currently solves only one-sided differential equations.The authors contrast its differential-equation support with SINDy’s broader capability.Source: Discussion · pp. 13–14
Methods
Experiments A–E use generated tabular datasets whose target functions are unknown to SciMED; experiment D omits a variable needed to construct its true target equation.
Methods
- Each dataset stores variable samples in rows and variables in columns, with the target in the last column; the unknown generating functions are listed in Table 3.Source: Methods · p. 14
- Experiment D follows experiment C’s data generation but removes z4 after calculating the target, leaving no available columns from which to construct the true equation.Source: Methods · p. 14
- All but experiment B add noise to target values; experiment A has 400 samples with a 75/25% training/testing split, while the others have 104 samples and an 80/20% split.GA and AutoML components are tuned using five-fold procedures.Source: Methods · p. 14
- SciMED, AI Feynman, and GP-GOMEA were each run 20 times on experiments A–E, with the most repeated outcome selected as each system’s result.Source: Methods · p. 14
Data availibility
The datasets generated and analyzed in the study are available in the SciMED repository.
Data availibility
- Study datasets are available in the SciMED GitHub repository; supplementary material is available online.Source: Data availibility · p. 14
Additional information
The paper provides supplementary material online and identifies the corresponding author for requests.
Additional information
- Supplementary material is available online, and correspondence and materials requests should be directed to L.S.K.Source: Additional information · p. 17
- Reprints and permissions information is available on the Nature website.Source: Additional information · p. 17
- The article is open access under a Creative Commons Attribution 4.0 license, subject to the stated conditions for third-party material.Source: Additional information · p. 17