PaperlayerView exact revision on arXiv

Source-linked AI summary

Discovering Causal Relations and Equations from Data

Gustau Camps-Valls, Andreas Gerhardus, Urmi Ninad, Gherardo Varando, Georg Martius, Emili Balaguer-Ballester, Ricardo Vinuesa, Emiliano Diaz, Laure Zanna, Jakob Runge

arXiv:2305.13341v1physics.data-ancs.AIcs.LGstat.ME

TL;DR

Causal and equation discovery from observational data is important because interventional experiments are often infeasible. This review synthesizes methods, connections, taxonomy, and case studies across physical sciences, finding that the fields are advancing through growing synergy and interdisciplinary analysis of complex systems.

  • Problem

    Interventional experiments are often infeasible, motivating the use of observations, simulations, and domain knowledge to discover causal relations and governing laws.

  • Method

    The paper reviews causal and equation discovery methods, develops a taxonomy, connects the fields, and presents case studies across physical sciences.

  • Results

    The review finds that causal and equation discovery methods have illustrated performance across neuroscience, Earth and climate sciences, and fluid mechanics, with growing synergy between the fields.

  • Takeaways & Limitations

    Analyzing complex physical systems requires an interdisciplinary approach that combines methods with domain expertise.

  • Takeaways & Limitations

    Causal discovery methods commonly assume causal sufficiency, so unobserved confounders can produce spurious or incorrect relationships.

Abstract

from arXiv · show

Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventional studies in the system under study. With the advent of big data and the use of data-driven methods, causal and equation discovery fields have grown and made progress in computer science, physics, statistics, philosophy, and many applied fields. All these domains are intertwined and can be used to discover causal relations, physical laws, and equations from observational data. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of Physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for observational causal and equation discovery, point out connections, and showcase a complete set of case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is being revolutionised with the efficient exploitation of observational data, modern machine learning algorithms and the interaction with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems.

3.2.1 Reduced-order models

The paper includes a reduced-order-model case study on learning vortex shedding behind an obstacle. This topic appears within the case studies in the physical sciences.

  • 3.2.1 Reduced-order models: Reduced-order modeling is presented as part of the paper’s broader case studies in the physical sciences.The case-studies section begins on page 73.
  • 3.2.1 Reduced-order models: A case study examines learning reduced-order models for vortex shedding behind an obstacle.The case study is listed as section 4.7 on page 90.

1. Introduction

The paper reviews how causal and equation discovery from data can support scientific understanding by learning causal relations, physical laws, and governing equations. It frames these methods within scientific modelling, model identifiability, and collaboration among domain experts, data, and machines.

  • Scientific understanding: Scientific understanding combines observation, experimentation, model refinement, prediction testing, and reasoning about causal mechanisms and counterfactuals.The scientific method is presented as an iterative loop in which observations confront model predictions and reveal processes requiring model improvement.
  • Challenges: Identifiability limits discovery when different parameterisations are observationally equivalent and therefore cannot be distinguished from available observations.This situation is also described as equifinality, because distinct model parameterisations can produce the same result or state.
  • Models for discovery: Observational discovery spans mechanistic models with interpretable, invariant, and robust laws, structural causal models, and statistical models optimized for prediction.This spectrum connects first-principles equations to data-learned models with different explanatory and predictive properties.
  • Integrative perspective: The review advocates an integrative data-driven framework in which domain experts, data, and machines formulate causal questions and discover new laws.It positions causal inference and equation discovery as tools for examining what algorithms can learn, explain, and contribute to new science.
  • Paper aims and scope: The review develops frameworks and taxonomies for causal discovery and learning physical laws from data in the physical sciences.It covers concepts, methods, and prior work, including applications to improving knowledge of physical systems from observations.

2. Causal discovery in the physical sciences

Causal discovery is increasingly used to uncover the causal structure of physical systems. This section reviews diverse methods, with emphasis on time-series approaches developed under different assumptions and use cases.

  • Causal discovery has become increasingly popular for identifying the underlying causal structure of physical systems.The field draws on established expositions and applications to physical systems.
  • The literature contains a growing range of methods designed for different assumptions and use cases.The section reviews several of these methods, focusing particularly on time series.

2.1 A taxonomy of causal discovery methods

This section organizes causal discovery methods around structural causal models, their assumptions, and conceptual distinctions, while relating observational and interventional distributions. It also highlights challenges involving time series, latent confounding, and consistency assumptions.

  • Taxonomy of methods: The taxonomy structures causal discovery methods by formal framework, conceptual axes, and method-specific assumptions to guide application and identify open challenges.Table 2.1 summarizes methods and indicates when assumptions are relaxed, including distinctions involving time series graphs and summary graphs.
  • Graphical-model framework: Structural causal models represent data-generating processes through structural assignments, causal parent sets, mechanisms, and jointly independent noise variables.The associated causal graph connects each variable to its causal parents and may be acyclic or cyclic.
  • Graphical-model framework: Interventions replace a variable’s structural assignment, hold it fixed, and remove incoming causal edges, distinguishing intervention distributions from observational conditioning.Thus, p(·| do(V_k := v_k)) generally differs from p(·| V_k = v_k).
  • Time series: Time-series causal stationarity denotes time-invariant qualitative cause-and-effect relationships across time indices.The time-series formulation places time indices on variables, mechanisms, parent sets, and noise terms.
  • Assumptions and limitations: Strong faithfulness is required for uniform consistency of the PC algorithm, whereas ordinary faithfulness provides point-wise but not uniform consistency.For linear models with infinite samples, faithfulness fails only on parameter sets of Lebesgue measure zero.
  • Assumptions and methods: Latent confounding violates causal sufficiency, motivating methods such as FCI that infer maximal ancestral graphs over observed variables rather than DAGs.Methods without causal-sufficiency assumptions generally return less informative graphs.

2.2 Challenges

The section highlights Dynamic Causal Models as Bayesian tools for fitting and comparing causal models of coupled dynamical systems, especially brain connectivity. It also describes local independence and graphical-separation frameworks for multivariate processes, including partially unobserved systems.

  • Dynamic Causal Models: Dynamic Causal Models fit and compare Bayesian causal models for coupled dynamical systems, with primary applications estimating connectivity between brain regions from neuroimaging data.The framework was introduced and applied mostly in Neuroscience.
  • Local Independence: Local independence formalizes whether a process’s past provides equivalent predictive information for another process compared with a larger conditioning history.The definition is given for components indexed by sets A, B, and C at time t.
  • Graphical Representations: Directed graphs with δ-separation characterize local independence in multivariate counting processes, while mixed graphs with µ-separation extend this framework to partially unobserved systems.The cited work established equivalence between pairwise and global Markov properties for multivariate counting processes.

2.2 Challenges

Causal discovery in physical-science applications faces challenges from nonstationary, nonlinear, autocorrelated, and cyclic processes, incomplete observations, high dimensionality, limited samples, and computational demands. Addressing these challenges requires suitable assumptions, domain knowledge, and improved statistical inference after model selection.

  • 2.2 Challenges: Nonlinear causal discovery requires nonparametric independence tests or complex score functions, which can reduce finite-sample performance and statistical power.Examples include conditional mutual-information, Gaussian-process, and kernel-based tests, which are computationally more expensive than linear tests.
  • 2.2 Challenges: Causal discovery requires assumptions about mechanisms and stronger post-selection inference because statistical inference after data-driven graph selection remains underdeveloped.Sample splitting can be statistically inefficient, while randomized GES provides a recent finite-sample correction for classical confidence intervals.
  • 2.2 Challenges: Real-world causal discovery is challenged by nonstationarity, autocorrelation, nonlinear relationships, and possible feedback cycles that violate common modeling assumptions.Many methods assume causal stationarity, while unmodified algorithms can perform worse on autocorrelated time series; nonlinear tests are more expensive and less powerful, and feedback can produce cyclic graphs.
  • 2.2 Challenges: Unobserved variables, especially confounders, violate causal sufficiency and can make causal relations difficult to identify from available data.Causal discovery methods often assume that all relevant variables are observed, but omitted confounders can distort inferred relationships.
  • 2.2 Challenges: High-dimensional physical data, small samples, and large datasets create statistical and computational difficulties, including large conditioning sets and kernel-method costs scaling cubically.Spatiotemporal data can be high-dimensional with limited samples, while large sample sizes can make independence testing computationally unaffordable.

2.3 Opportunities for the physical sciences

Observational causal discovery offers physical sciences principled tools to test competing hypotheses, characterize complex systems, and inform interventions despite assumptions and statistical challenges. It also supports quantitative causal-effect estimation, pathway analysis, and understanding anomalies and cascades.

  • Hypothesis testing and causal discovery: Observational causal discovery can test hypotheses, evaluate interventions, attribute extreme events counterfactually, and derive causal pathways and robust forecasting models.The field is growing in methodologies, theoretical guarantees, and empirical evidence.
  • Hypothesis testing and causal discovery: Causal graphs provide a principled representation of assumed or learned relations and evidence for choosing between competing causal hypotheses.They can reveal causal links and pathways, including in competing hypotheses about Arctic climate teleconnections.
  • Quantitative causal effects: Causal-effect estimation targets p(Y = y | do(X = x)) rather than observational association, using identifiability, covariate adjustment, and do-calculus when assumptions permit.Confounding can make p(Y = y | do(X = x)) differ from p(Y = y | X = x), motivating de-confounding through adjustment variables.
  • Causal pathways: Mediation analysis examines how perturbations propagate through causal pathways and quantifies both indirect mechanisms and direct effects.In complex systems, pathway analysis identifies subprocesses through which an effect is mediated.
  • Anomalies and complex systems: Causal analysis extends anomaly detection by investigating what causes anomalous processes, including extreme events and cascades across complex systems.Examples include compound extremes in Earth sciences and travelling cascades such as El Niño-related events.
  • Validation challenges: Method development requires benchmark datasets with known causal ground truth, but many fields lack such datasets beyond expert-labelled relations among a few variables.Ground truth ideally comes from expert knowledge of real data or experiments that can falsify predicted causal relationships.

2.4 Perspectives

The section frames causal discovery in physical sciences as a field with substantial methodological, theoretical, and communication challenges. It highlights domain knowledge, clearer causal reasoning, and collaboration between methods and domain sciences as key avenues forward.

  • Perspectives: Major practical challenges include preprocessing variables, non-stationarity, contemporaneous causation, hidden confounding, and nonlinear or non-Gaussian dependencies.The review also identifies algorithmic and data challenges as opportunities for future research.
  • Perspectives: Causal discovery from purely observational data is theoretically impossible in general, making identifiability a central unresolved challenge.Domain knowledge and causal DAG specification can help mitigate assumptions about sufficiency and faithfulness.
  • Perspectives: Incorporating uncertain expert knowledge can address identifiability and faithfulness assumptions while supporting scientific understanding and physics-model improvement.The section connects this strategy to sparsity priors used in equation discovery.
  • Perspectives: Cognitive and language barriers between methodological and domain-science communities hinder the use and interpretation of causal concepts in physical sciences.The section identifies reluctance to use causal language and communication across communities as future challenges.
  • Perspectives: Researchers should state assumptions transparently when making causal claims, especially when analysing complex systems such as Earth and the brain.The section notes that causal language is often used without sufficiently explicit justification.

3. Learning physical laws from data

This section reviews how modern statistical and machine-learning methods automate the discovery of plausible, simple, and explainable physical equations from data. It situates these methods within the longstanding scientific process of observing phenomena, hypothesizing models, and testing predictions.

  • Scientific method: The scientific method links observations to hypothesized mathematical models whose predictions are tested, falsified, and refined through further experiments.This iterative process underlies the development of mechanistic models that explain and predict natural phenomena.
  • Historical motivation: Scientists have long observed nature to extract simple laws and equations that explain mechanisms, predict behavior, and guide interventions.Modern data-driven approaches extend this enduring objective with automated discovery methods.
  • Motivation: Equation discovery addresses complex systems whose unknown interactions or emergent coarse-scale behavior make conventional models inadequate.Automating model building helps researchers interpret the large volumes of data produced by modern instruments.
  • Equation discovery: Modern statistics and machine learning exploit regularities in data to discover plausible, simple, and explainable equations describing observed systems.The section introduces explicit equation discovery from data, including symbolic regression, and contrasts it with traditional trial-and-error law discovery.

3.1 Explicit equation discovery with symbolic regression

Symbolic regression discovers compact, human-readable equations by optimizing predictive fit against expression complexity. The section describes evolutionary, continuous, neural, and physics-informed search strategies, including Pareto trade-offs and benchmark-dependent method choices.

  • Search strategies: Methods span discrete evolutionary search, continuous relaxations, and pretrained models that use synthetic data to discover mathematical expressions.Genetic programming iteratively generates and selects programs, while neural approaches represent expressions as token sequences or differentiable networks.
  • Formulation and trade-offs: Symbolic regression seeks a low-complexity function that maps inputs to outputs while balancing fit against a complexity penalty such as term count.The weighting factor λ controls the trade-off, but solutions are more generally viewed as a Pareto family of fit–complexity compromises.
  • Formulation and trade-offs: The Pareto curve captures decreasing training error with increasing complexity, while test error can rise for overly complex expressions because of overfitting.This motivates selecting among expressions rather than assuming one universally correct complexity level.
  • Benchmark comparisons: NeSymReS finds well-fitting expressions remarkably quickly on unseen equations from the Feynman and Nguyen benchmarks as wall-clock time increases.Figure 3.8 compares the accuracy of different symbolic-regression methods over time.
  • Method selection: Method selection depends on prior structural knowledge: FFX and SINDy suit known functional modules, SINDy is specialized for dynamical systems, and other methods handle unknown or nested forms.Genetic-programming methods generally perform best on small problem settings, whereas EQL scales across dimensions without specific system-size adaptations.

3.2 Implicit equation discovery: dimensionality reduction and transfer operators

This section reviews implicit equation-discovery methods that learn compact representations or transfer operators capturing system dynamics rather than explicit equations. It covers reduced-order models based on modal decomposition and transfer-operator approaches, including Koopman analysis and dynamic-mode decomposition.

  • 3.2.1 Reduced-order models: Reduced-order models simplify high-fidelity nonlinear simulations by trading model accuracy for reduced computational complexity, using either model-based or data-driven constructions.Data-driven ROMs use input-output data from the original first-principles model and may be static or dynamic.
  • 3.2.1 Reduced-order models: ROM development first identifies compact coordinates or modes, then derives differential equations governing the temporal evolution of their amplitudes.Proper-orthogonal decomposition provides a widely used modal-decomposition framework for this process.
  • 3.2.1 Reduced-order models: POD/PCA decomposes spatio-temporal data into orthogonal spatial modes and temporal coefficients, with truncation yielding a low-dimensional representation that optimally captures explained variance.The method is equivalent to PCA or EOF and is based on singular-value decomposition.
  • 3.2.2 Transfer operators for learning nonlinear dynamics: The Koopman operator linearises complex temporal trajectories by lifting state variables into an infinite-dimensional function space, enabling data-driven models and analysis of nonlinear dynamics.Its eigenfunctions can serve as basis functions for models used in control and optimisation, without solving or fully understanding the underlying equations of motion.
  • 3.2.2 Transfer operators for learning nonlinear dynamics: Koopman analysis is computationally expensive because accurate finite-dimensional approximations of its eigenvalues and eigenfunctions can be difficult, limiting real-time applications such as model-based control.The operator is intrinsically infinite-dimensional, motivating approximation methods such as dynamic-mode decomposition.

3.3 Perspectives

Equation discovery for physical sciences faces ambitious goals and theoretical, practical, and data-related challenges, while offering opportunities in interpretability, evaluation, compression, and model selection. Progress requires models that balance simplicity, generalisability, extrapolation, predictive accuracy, robustness, and invariance.

  • Perspectives: Equation discovery seeks models that are simple, generalisable, robust, and capable of extrapolating beyond the sample space, creating ambitious theoretical and practical challenges.These objectives are framed within a sparsity–extrapolation–generalisation space.
  • Theoretical challenges: Theoretical challenges include equation identifiability, the preference for sparsity, evaluation criteria, and strong assumptions about observed variables, representations, bases, and manifolds.The section questions whether invariance and robustness alone suffice to evaluate discovered solutions.
  • Practical challenges: Practical difficulties arise from high dimensionality, nonlinear relationships, overfitting, large search spaces, noisy data, basis-function choices, and unresolved treatment of time-lagged and interventional information.These issues affect symbolic regression and other nonlinear methods, including challenges in performance evaluation and hyperparameter tuning.
  • Research opportunities: Three opportunities are model interpretability, model compression and evaluation, and model selection, with discovered equations offering interpretable predictive models and insight into system dynamics.Intervention and sensitivity analysis can further support interpretation when complexity cannot be traded for accuracy or implicit representations are learned.
  • Model compression and evaluation; Model selection: Evaluation should assess predictive accuracy, simplicity, robustness to distortions and changes, and invariance in space and time, while model selection should retain plausible solutions along a Pareto line.Alternative regularisation, uncertainty estimates, and extrapolation indicators are identified as important for avoiding unrealistically simple models.

4. Case studies in the physical sciences

The section presents case studies applying data-driven causal and equation-discovery methods to neuroscience, Earth and climate sciences, and fluid and mechanical dynamics. Across these domains, the studies recover interpretable latent dynamics, ecosystem causal structure, model fingerprints, and equations, while highlighting the need to combine causal approaches and improve differential-equation discovery.

  • Overview: Case studies span neuroscience, Earth and climate sciences, and fluid and mechanical dynamics, illustrating diverse data-driven causal and equation-discovery methods.Table 4.1 summarizes the cases and their principal methods.
  • Neuroscience: Neural methods recover latent trajectories, reconstruct single-trial spiking activity, reduce recordings to interpretable dimensions, and distinguish competing decision-making models.GPFA variants discern upstream-area contributions to recurrent dynamics, while diffusion maps preserve competitive prediction accuracy after dimensionality reduction.
  • Earth and climate sciences: Causal-network analyses show that ecosystems with similar meteorological conditions can share biosphere-atmosphere interactions across climate zones and vegetation types, including drought-driven shifts.The studies also identify contemporaneous links and use anomalous network structures as indicators of ecosystem change.
  • Earth and climate sciences: Earth-system causal analyses identify climate-dependent relationships among GPP, temperature, soil moisture, evaporation, and evapotranspiration, while causal fingerprints distinguish model behavior from observations.Model fingerprints are more consistent within ensembles than across models, and more realistic fingerprints affect projected land-surface precipitation changes.
  • Fluid and mechanical dynamics: Symbolic regression and equation-learning approaches recover analytical descriptions that extrapolate to unseen external potentials or mean densities in hard-rod and Lennard-Jones fluids.The broader case studies also expose a limitation: inferential methods may not uniquely identify differential equations, motivating combined causal approaches and interventional evidence.

5. Concluding remarks

The paper reviews causal and equation discovery, emphasizing their complementary goals, methodological assumptions, challenges, and applications across physical sciences. It concludes that growing integration between the fields, more efficient algorithms, and continued research could advance understanding of complex systems.

  • 5. Concluding remarks: Causal discovery identifies qualitative cause-and-effect relationships from non-experimental data under enabling assumptions, including structural causal models.
  • 5. Concluding remarks: Equation discovery seeks equations, state variables, and laws for predicting and understanding system behavior using symbolic regression, evolutionary algorithms, and deep learning.
  • 5. Concluding remarks: Both fields face unresolved challenges involving identifiability, equation generality, model selection, physical units, modularity, expressive power, and overfitting.
  • 5. Concluding remarks: Hybrid models combining causal and equation discovery could uncover causal relationships and equations, while more efficient algorithms may support further progress.
  • 5. Concluding remarks: Case studies in neuroscience, Earth and climate sciences, and fluid mechanics illustrated algorithmic performance, while differing assumptions and data characteristics limited cross-field method transfer.
  • 5. Concluding remarks: The fields are rapidly advancing despite remaining challenges, with growing synergy and continued prospects for uncovering the underlying structure of complex systems.

C. Darwin. On the Origin of Species. John Murray, London, 1859.

The supplied section contains related-work references spanning causal discovery, equation discovery, climate science, fluid dynamics, and neuroscience.

  • Related work: The cited literature includes equation and program-discovery approaches for learning mathematical models from data.The references include Eureqa, inductive logic programming, program synthesis, and limits on learning closed-form mathematical models.
  • Related work: The bibliography also covers fluid turbulence, reduced-order modeling, and neural population dynamics.Examples include turbulent vortex clusters, fluidic-pinball bifurcations, turbulence prediction, neural recordings, and recurrent neural-network state-space models.
  • Related work: The references cover causal discovery methods for time series, graphical models, Granger causality, and geoscientific applications.Examples include causal discovery from time series, local-independence graphs, Granger-causality testing, and causal discovery in the geosciences.

J. Hicks et al. Causality in Economics. Australian National University Press, 1980.

The cited works span causal discovery, nonlinear and nonstationary causal modeling, data-driven dynamical-system methods, and applications in turbulence, neuroscience, climate, and economics.

  • Applications: Applications and related methods extend to stock price-volume causality, neural population encoding, global evapotranspiration, symbolic regression, and automated scientific discovery.The cited studies include Granger causality, medial prefrontal cortex neurons, global land evapotranspiration, neural-network symbolic regression, and robot-assisted discovery.
  • Causal discovery: Causal-discovery references address nonlinear additive-noise models, high-dimensional PC algorithms, and time-dependent or heterogeneous/nonstationary data.These works include nonlinear causal discovery, high-dimensional directed acyclic graphs, state-space models, and heterogeneous/nonstationary causal discovery.
  • Fluid dynamics: The bibliography also covers turbulence and fluid mechanics, including turbulent flows, wall-bounded turbulence, eddy effects, and data-driven turbulence theory.References address eddies and convergence zones, subgrid-scale backscatter, cascades, coherent structures, and machine-aided turbulence theory.
  • Dynamical systems: Data-driven dynamical-systems research includes Koopman and transfer-operator approximations, control-oriented model reduction, and learning emergent partial differential equations.The cited works cover Koopman eigenfunctions, operator approximations, transfer operators, and emergent PDE learning.

J. Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669–688, 1995.

This section cites foundational and overview works on causal inference, including Pearl’s books and surveys, alongside related applications and methodological developments.

  • Related work: Pearl’s books and surveys provide foundational references on causality and causal inference.The cited works span Causality, causal diagrams, and statistical causal inference.
  • Related work: Applications represented in the bibliography include neuroscience, geoscience, remote sensing, and Earth-system science.The cited works address dynamic causal models, causal inference from geoscience observations, and causality in Earth systems science.
  • Related work: The references also cover causal models based on functional, structural-equation, invariant-prediction, and dynamical-system perspectives.Examples include functional-model identifiability, restricted structural equation models, invariant causal prediction, and causal models for dynamical systems.

H. Reichenbach. The direction of time, volume 65. Univ of California Press, 1991.

This section lists related work spanning causal discovery, dynamical and structural causal models, information transfer, and equation learning. The cited literature also includes applications to climate, neural, and fluid systems.

  • The bibliography includes work on directed cyclic graphs, structural causal models, and causal relations in autocorrelated nonlinear time series.
  • Several cited studies address information transfer, mediation, causal pathways, and multivariate transfer entropy in complex systems.
  • Applications represented in the bibliography include climate and Earth-system causality, neural dynamics, and fluid-flow analysis.
  • The references cover equation learning for extrapolation and control alongside approaches for distilling natural laws from experimental data.
Loading 2305.13341v1…