Source-linked AI summary
Learning the Geometry of Admissible Hypotheses through Inductive Bias in Training Distributions
James Crowley, Faez Ahmed, Anton van Beek
TL;DR
The paper addresses the challenge of representing competing PDE hypotheses whose discrete structures and continuous parameters are both unknown. It uses scientific principles to shape a training distribution for learning a unified latent manifold, finding that an 11-dimensional representation reconstructs benchmark PDEs and exhibits smooth transitions across equation families. The resulting manifold is presented as a foundation for future inference over competing governing equations.
Problem
Mixed-variable and combinatorial scientific hypothesis spaces are difficult to represent probabilistically because active components and continuous parameters are jointly unknown.
Method
The framework generates PDE hypotheses using sparsity, logical dependencies, PDE families, and physical admissibility, then learns their unified continuous representation with a gated variational autoencoder.
Results
An 11-dimensional latent space accurately represents benchmark PDEs with smooth transitions within and across equation families, while scientific-principle ablations reduce structural errors and active-coefficient MSE.
Takeaways & Limitations
Embedding scientific inductive bias in the hypothesis distribution yields a compact, geometrically meaningful manifold suitable as a foundation for future inference over competing PDEs.
Takeaways & Limitations
Extending the framework beyond PDEs requires identifying the scientific principles that define admissible hypotheses for each new hypothesis class.
Abstract
from arXiv · showhide
Scientific discovery often requires reasoning over competing hypotheses that are consistent with experimental observations. For mixed-variable and combinatorial hypothesis spaces, however, constructing probabilistic representations remains challenging because both the active model components and their associated parameters are unknown. In this work, we present a framework for learning continuous latent representations of admissible partial differential equations (PDEs) by embedding a scientific inductive bias directly into the training distribution. Progressively richer structural principles (e.g., sparsity, logical dependencies, common PDE families, and physical admissibility) are used to generate a structured distribution of hypotheses from which a gated variational autoencoder learns a continuous latent manifold. Experimental results show that the resulting 11-dimensional representation accurately reconstructs a broad collection of representative PDEs, while exhibiting smooth geometric transitions both within and across equation families. Through an ablation study we further demonstrate that introducing scientific principles reduces both structural misclassifications of equation forms and parameter estimation errors when reconstructing a representative benchmark set of admissible partial differential equations. These results show that embedding a scientific inductive bias in the training distribution enables the learning of compact and geometrically meaningful hypothesis manifolds, providing a principled foundation for future inference over competing governing equations.
1 Introduction
The paper targets probabilistic reasoning over competing PDE hypotheses in mixed-variable spaces, where discrete equation structure and continuous coefficients are jointly unknown. It embeds progressively richer scientific principles into the training distribution so a latent model can learn a unified, continuous representation of admissible parameterized PDEs.
- Mixed-variable scientific hypotheses combine unknown active components with unknown continuous parameters, making computational representation and exploration challenging.
- Classical equation discovery selects among predefined equation libraries, while sparse regression constructs equations from candidate terms.
- Grammar- and graph-based approaches incorporate structural priors but generally represent numerical coefficients separately from symbolic equation structure.
- The proposed framework learns a continuous latent representation of admissible PDEs by generating training hypotheses with sparsity, logical dependencies, PDE families, and physical admissibility.
- Unlike sequential structure-then-parameter methods, the model jointly represents equation structure and continuous coefficients in one probabilistic latent space.
- The learned representation accurately reconstructs benchmark governing equations, supports smooth transitions within and across PDE families, and organizes physically meaningful classes geometrically.
2 Representation and Learning of Scientific Hypothesis Spaces: An Application to PDEs
The framework constructs increasingly structured distributions of admissible PDE hypotheses using scientific principles, then learns a continuous latent representation of that space. It represents PDEs as mixed discrete-continuous objects and progressively imposes sparsity, logical dependencies, PDE-family membership, and physical admissibility.
- Overarching Framework: The framework adds progressively stronger scientific knowledge to transform an unstructured PDE hypothesis space into one with meaningful regularity.The structured distribution is designed before model training rather than leaving the learning algorithm to discover structure from arbitrary hypotheses.
- Overarching Framework: A variational autoencoder trains on the structured dataset to learn a continuous manifold that places related equations nearby and preserves smooth transitions between hypothesis classes.The dataset contains finite-dimensional realizations of admissible PDEs generated under the successive principles.
- Equation Representation: PDE hypotheses combine binary indicators for active terms with continuous coefficients, creating a mixed-variable combinatorial representation.The framework restricts attention to second-order equations and maps admissible hypotheses into a continuous latent space.
- Inductive Biases: Sparsity restricts equations to a relatively small number of active terms, using kmin = 2 and kmax = 6 in this work.The assumption reflects the common dominance of a small number of active terms, not an assertion that every physical system is sparse.
- Inductive Biases: Logical dependencies encode structural relationships among differential operators while retaining combinations that remain admissible within the considered PDE classes.Examples include requiring underlying linear operators for nonlinear transport terms and associated principal directions for mixed diffusion terms.
- Inductive Biases: PDE-family classification considers elliptic, parabolic, hyperbolic, steady first-order, and time-dependent first-order equations based on their operator components.The classification uses the presence of second- and first-order time dynamics, transport, and symmetric diffusion terms.
- Scope: The considered PDE families do not cover every possible PDE, because the framework prioritizes introducing regularity without constricting the learned manifold too strongly.This scope boundary accompanies the family-classification principle.
- Inductive Biases: Physical admissibility additionally constrains continuous parameters by requiring the diffusion matrix to be positive definite.The condition is expressed through λmin(As) > 0, while the first three principles constrain symbolic structure.
3 Geometric Analysis of Latent Representations
The learned latent space organizes admissible PDEs according to mathematical structure while supporting smooth transitions within and across equation families. Scientific inductive bias improves reconstruction by reducing structurally implausible hypotheses and coefficient errors.
- Latent dimensionality: An 11-dimensional latent space is sufficient for the dominant structure of the admissible PDE hypothesis space, with reconstruction metrics plateauing beyond this dimension.The unrestricted space instead has average reconstruction error exceeding 0.9 and approximately 0.8 erroneously active terms per equation.
- Latent geometry: Latent variables are weakly correlated, approximately zero-centered, and typically have standard deviations between 1.5 and 1.9, reflecting a trade-off between Gaussian regularization and reconstruction accuracy.Most pairwise correlations lie between −0.25 and 0.25.
- Inductive-bias ablation: Scientific principles reduce erroneously active terms from 0.85 to 0.02, while active-coefficient MSE decreases almost monotonically as constraints are added.The largest structural improvements follow sparsity and logical-dependency constraints; physical admissibility yields the lowest structural reconstruction error.
- Global organization: PDE forms separate more strongly than broad PDE families, with mean distance ratios of 1.950 and 1.48, respectively.Family structure is present but may not be the primary criterion used to distinguish equation forms.
- Global organization: Nearly inactive differential operators place higher-dimensional PDEs near lower-dimensional counterparts, indicating sensitivity to effective operator strength as well as symbolic structure.Removing diffusion terms smaller than 0.1 eliminates the observed overlap between different heat-equation forms.
- Continuity within and across families: Linear latent interpolation produces smooth changes in coefficients and activation probabilities, including continuous deformation between wave and first-order-time equations.Within-family interpolations have mostly less-than-0.05 reconstruction errors, while heat-equation diffusion terms switch gradually between spatial directions.
4 Concluding Remarks
The framework learns continuous latent representations of admissible PDEs by embedding progressively richer scientific inductive biases in the training distribution. An 11-dimensional representation reconstructs benchmark PDEs with smooth geometric transitions, while extending the approach requires domain-specific principles for admissible hypotheses.
- Sparsity, logical dependencies, PDE family structure, and physical admissibility progressively organize the hypothesis space for a gated variational autoencoder.
- An 11-dimensional latent space accurately represents benchmark PDEs and exhibits smooth geometric transitions within and across equation families.
- The ablation study shows that progressively introducing scientific principles improves the quality of the learned representation.
- The approach is not restricted to PDEs, but other domains require identifying their corresponding scientific principles defining admissible hypotheses.
- Future work will investigate Bayesian inference within the learned hypothesis manifold and analogous representations for other scientific hypothesis classes.