Source-linked AI summary
Physics-Transfer Learning: A Physics-Constrained Theory for Zero-Shot Generalization Along the Ladder of Complexity
Yingjie Zhao, Zhiping Xu
TL;DR
Reliable prediction in complex physical systems is constrained by costly high-fidelity models, sparse measurements, and distribution shifts that can reflect changing mechanisms. PT organizes transfer through progressively more complex anchor domains and physical indicators; reported tests include high-accuracy target prediction using low-fidelity training data.
Problem
Method
Results
For quantum excited-state prediction, target-domain R2 exceeded 0.96 despite significant label distribution shifts; the framework also reports high-accuracy target predictions using low-fidelity training data.
Source: §1 · p. 3, §3.1 · p. 9Takeaways & Limitations
PT offers a structured framework for assessing generalization reliability and diagnosing transfer failure risks in data-scarce physical systems.
Source: §1 · pp. 3–4PT indicator estimation requires sufficiently large domain samples, while sparse anchor chains and limited CAD availability constrain its resolution.
Source: §3.2 · p. 11, §4.1 · p. 14
Abstract
from arXiv · showhide
Reliable out-of-distribution generalization remains a central challenge in machine learning, particularly when models are deployed in data-scarce high-fidelity domains. Existing approaches often rely on statistical distribution alignment, or invariance assumptions that can fail when domain shifts reflect changes in underlying mechanisms. Inspired by a longstanding paradigm in physics that understands complex systems by starting from tractable idealizations and progressively incorporating additional mechanisms and levels of fidelity, we propose physics-transfer (PT) learning, a physics-constrained learning theory for zero-shot generalization along a ladder of complexity. PT constructs chains of anchor domains (CADs) that connect low-cost source domains to complex targets and introduces computable feasibility and reliability indicators for assessing transfer before target-domain labels are observed. Under physical-consistency and stable-representation assumptions, we derive an operational upper bound on target risk in terms of source empirical risk, the PT indicators, and finite-sample errors, and establish convergence as adjacent complexity gaps shrink and data coverage improves. We evaluate PT on excited-state energy prediction in quantum systems and dynamical-transition prediction in classical systems using neural networks trained without target-domain labels. Across substantial distribution shifts, PT yields accurate high-fidelity predictions, while the indicators identify incorrect physical mappings before target-domain prediction fails. These results provide a model-agnostic framework for designing, diagnosing, and controlling generalization in data-scarce learning problems.
1 Introduction
Complex physical systems pose a fidelity-efficiency-data trade-off, while shifts may reflect changing mechanisms rather than sampling noise. PT proposes a physically grounded theory for transferring knowledge from tractable domains toward complex targets.
1 Introduction
- The framework is motivated by energy and dynamics predictions, including excited-state energies and dynamical strain localization under non-equilibrium conditions.These predictions matter to applications such as materials design, catalysis, energy storage, and structural engineering.Source: §1 · p. 1
- Complex systems demand high physical fidelity, yet simulations can be prohibitive and measurements costly, sparse, or inaccessible.The resulting constraint is a trade-off among physical fidelity, computational efficiency, and data availability.Source: §1 · p. 2
- Equation-based physics-informed learning can be limited when governing equations are unavailable or mechanisms are implicit, hierarchical, or difficult to formulate.Large physical datasets and foundation-model generalization across regimes also remain constrained by coverage and problem complexity.Source: §1 · p. 2
- PT organizes domains into chains of anchor domains and uses physical mechanisms to support zero-shot predictions in unseen, high-fidelity targets.The framework draws on the physics practice of starting with tractable idealizations and progressively adding mechanisms and fidelity.Source: §1 · p. 3
2 The Concept of Physics Transfer
PT addresses distribution shifts in physical systems by grounding transfer in physical mechanisms and structuring domains along increasing fidelity and complexity. It uses anchor-domain chains and indicators to assess transfer feasibility and reliability before target-domain labels are available.
The Concept of Physics Transfer
- Physical mechanisms, rather than statistical distribution alignment alone, ground PT generalization under shifts driven by evolving mechanisms.The framework does not require target-domain labels or invariant conditional distributions.Source: §2 · pp. 4, 6
- PT constructs progressively more complex anchor domains to bridge low-cost sources and high-fidelity targets while preserving physically coherent representations.The learned latent representations align data across fidelities without adversarial alignment or explicit distribution-matching losses.Source: §2 · p. 4
- PT indicators quantify whether transfer from the source to an anchor is feasible and whether anchor-domain validation is informative for target prediction.The feasibility and reliability indicators provide predictive diagnosis of transfer risk rather than relying only on post hoc assessment.Source: §2 · pp. 4–5
- Feasibility is evaluated by training on the source, fixing network weights, and directly evaluating along the anchor chain.Convergence is assessed by independently training on anchors of increasing complexity and checking whether performance stabilizes on the most complex anchor before transfer to the target.Source: §2 · pp. 5–6
- PT is complementary to transfer learning and geometric deep learning, and its stated guarantees rely on physical continuity along the anchors and digital-library completeness.The authors contrast its physical-mechanism basis with conventional domain adaptation and domain generalization methods.Source: §2 · p. 6
- The examples cover excited-state energy calculations across basis-set fidelities and strain-localization dynamics across force-field fidelities.They illustrate the accuracy-efficiency challenge: high-fidelity models are costly, while efficient low-fidelity models have systematic bias.Source: §2 · p. 6
3 Results for Representative Problems
The representative tests apply physics-transfer to excited-state prediction in quantum systems and sliding resistance in classical systems, where transitions involve data scarcity or costly high-fidelity modeling. The examples test transfer across physical mappings and fidelity hierarchies.
3.1 PT Models for Quantum Calculations of Excited-State Energies
- For oscillator strength, source-trained predictions achieved R² above 0.98 even on the highest-fidelity anchor, while its feasibility indicator remained below 2 × 10^-3.The reliability indicator decreased from 15.11 ± 4.21 at A1 to 2.72 ± 0.06 at A3.Source: §3.1 · p. 7
- As training-domain chemical accuracy increased, target-domain R² exceeded 0.99, approaching interpolation-level accuracy.The analysis evaluates convergence by applying models trained on progressively higher-fidelity domains to A3 and the target.Source: §3.1 · pp. 8–9
3 Results for Representative Problems
- Target-domain R² exceeded 0.96 for oscillator strength, while mean target-domain R² was 0.9898 for sliding resistance across ten runs.The quantum result is reported under significant label distribution shifts; the sliding-resistance model was trained only on source-domain empirical force-field data.Source: §3.1 · p. 9, §3.2 · p. 12
3.1 PT Models for Quantum Calculations of Excited-State Energies
- Using the ground-state dipole instead of the transition dipole caused physical-transfer failure, which abnormal indicators and anchor-domain performance diagnosed before target prediction.The authors describe correcting the physical mapping or increasing training-data complexity as ways to guide successful transfer.Source: §3.1 · pp. 9–10, Figure 3 · p. 10
3.2 PT Models for Dynamical Transition in Classical Systems
- For sliding resistance, the physical mapping relates material parameters to resistance, with source empirical force fields, MLFF anchors, and DFT targets.The test includes interpolation and mild combinational generalization for several fcc metals, and more demanding parameter-space extrapolation for Ir and Rh.Source: §3.2 · p. 11
- In the sliding-resistance example, limited force-field samples make indicator estimates less reliable, while the sparse PES fidelity hierarchy restricts systematic convergence analysis.The indicators are therefore used primarily qualitatively, alongside computational-result verification.Source: §3.2 · pp. 11–12
4 Discussion
PT’s diagnostic value depends on sufficiently resolved anchor-domain chains and training data that faithfully encode physical mechanisms. The discussion frames PT as a way to assess generalization risk before deployment, while identifying data and CAD resolution as practical boundaries.
4.1 Challenges in CAD Construction
- Sparse anchor domains or large adjacent complexity gaps make feasibility and convergence difficult to characterize, limiting PT’s diagnostic and predictive power.The passage attributes this bottleneck to current constraints in computational and experimental data generation, not to the theory itself.Source: §4.1 · p. 14
4.2 Data, Model, and Algorithmic Perspectives on Enhancing PT
- Successful PT depends on source data that faithfully encode underlying physical mechanisms, supporting meaningful latent representations rather than spurious statistical correlations.The framework is model- and training-strategy-agnostic, and can incorporate approaches such as active learning, adaptive sampling, and uncertainty-aware training.Source: §4.2 · p. 14
- Future work will develop dimension-reduction theories for latent physical representations to reconstruct the physical laws transferred across domains.The discussion also points to finer-grained CADs from multi-fidelity models and experimental validation as ways to address the CAD-resolution bottleneck.Source: §4.1 · p. 14, §4.2 · p. 15
4.3 PT as a Physics-Constrained Learning Theory for Generalization Control
- PT uses physically interpretable indicators to assess generalization risk before deployment, rather than relying only on post hoc empirical validation.The framework quantifies feasibility, discrepancy, and convergence to support risk-aware transfer.Source: §4.3 · p. 15
5 Conclusion
The conclusion presents PT as a structured pathway for transferring knowledge from low- to high-fidelity domains using physically grounded mappings and hierarchical anchor-domain chains. It reports accurate cross-fidelity prediction and pre-deployment diagnosis across energy and dynamics problems.
5 Conclusion
- PT organizes domains into hierarchical chains of anchor domains on a complexity ladder to transfer knowledge from low-fidelity sources to high-fidelity targets.It reformulates conditional learning relations as physically grounded mappings.Source: §5 · p. 15
- Unlike methods centered on statistical distribution alignment, PT bases generalization on consistency of underlying physical mechanisms across domains.Feasibility and reliability indicators enable quantitative assessment of generalizability and convergence before deployment.Source: §5 · p. 15
- Neural networks trained under weak physical constraints learned stable latent representations that supported accurate cross-fidelity predictions for energy and dynamics problems despite distribution and label shifts.The conclusion describes PT as a framework integrating data fidelity, hierarchical modeling, representation learning, and risk-aware generalization analysis.Source: §5 · p. 15
A.1 Formalization of Physics-Transfer Theory
Physics-transfer formalizes zero-shot transfer as stepwise extrapolation through physically ordered anchor domains. Its operational bound and convergence result depend on physical consistency, stable representations, and sufficiently complete data libraries.
A.1 Formalization of Physics-Transfer Theory
- PT places intermediate anchor domains between source and target when their complexity gap makes direct transfer unreliable.The ordered chain is S → A1 → ··· → AK → T.Source: §A.1 · p. 16
- Physical consistency assumes system responses vary smoothly with complexity, while stable representation assumes physically meaningful latent variables change smoothly across domains.Weak physics constraints support the latter assumption.Source: §A.1 · p. 17
- Stable representations depend on sufficiently complete and diverse digital libraries; inadequate coverage can prevent the learned representation from capturing physical mechanisms.The representation is approximated from finite observations.Source: §A.1 · p. 17
- PT feasibility and reliability indicators provide operational surrogates for transfer feasibility and for whether anchor-domain validation informs target prediction.The ideal joint source-target error is generally unobservable without target labels.Source: §A.1 · p. 18, Definition 9 (PT reliability indicator) · p. 19
Definition 9 (PT reliability indicator)
- The operational generalization bound expresses target risk using source empirical risk, PT indicators, and finite-sample estimation errors.The theorem assumes physical consistency and stable representations; sample size and hypothesis-class complexity enter the bound.Source: Definition 9 (PT reliability indicator) · pp. 18–19
- When neighboring complexity gaps shrink and data coverage improves, target-domain prediction error converges to source-domain error.The convergence statement assumes physical consistency and stable representations.Source: Definition 9 (PT reliability indicator) · p. 19
- The framework casts domain generalization as controlled extrapolation along anchor domains under physical consistency and complete data libraries.The indicators serve as operational criteria for assessing transfer feasibility and reliability.Source: Definition 9 (PT reliability indicator) · p. 20
A.2 Complexity Ladders
The section describes computational fidelity ladders in quantum chemistry and atomistic simulation, alongside a broader example of increasing geometric complexity.
A.2 Complexity Ladders
- Increasing basis-set size raises both electronic-structure accuracy and computational cost, forming a fidelity hierarchy from STO-3G through def2-TZVP.The listed intermediate basis sets are 3-21G, 6-31G, and def2-SVP.Source: §A.2 · p. 20
- PES modeling spans a computational ladder from efficient EAM and MEAM force fields through ML force fields to more accurate, costly DFT calculations.The methods progress from empirical models to data-driven surrogates and first-principles calculations.Source: §A.2 · p. 20
- Geometric complexity can increase from spheres to ellipsoids and then to surfaces with nonzero Gaussian curvature, including saddle geometries and brain morphology.Source: §A.2 · p. 21
A.3 Neural Network Architectures
Both prediction examples use fully connected neural networks with a representation-learning block and scalar regression head, keeping the architecture simple to focus on descriptors and anchor-domain construction.
A.3 Neural Network Architectures
- The oscillator-strength network maps four physical descriptors through 64-unit and 32-dimensional representation layers, while the sliding-resistance network uses three 64-unit hidden layers.Both use a scalar output; the latent representation before the final regression layer supplies Z for PT-indicator analysis.Source: §A.3 · p. 21
A.4 Data Sources and CAD Construction
The experiments construct distinct fidelity ladders for oscillator-strength and sliding-resistance prediction, reserving the highest-fidelity domain as the target.
A.4 Data Sources and CAD Construction
- Oscillator-strength domains progress from STO-3G through 3-21G, def2-SVP, and def2-TZVP, while sliding resistance progresses from empirical force fields through MLFFs to DFT.The source trains the model, anchors support indicator analysis, and the highest-fidelity level is the target; a reduced statistical-learning control omits explicit non-equilibrium transition descriptors.Source: §A.4 · p. 21
A.5 Training Protocols
Both examples use source-only training and fixed-model transfer, with preprocessing fitted on source data and applied unchanged to anchor and target domains. Their validation and optimization setups are tailored to each dataset.
A.5 Training Protocols
- Preprocessing parameters are fitted only on source-domain data, then applied unchanged to anchor and target domains.This keeps target-domain information out of training and maintains zero-shot evaluation.Source: §A.5 · p. 21
- Oscillator-strength models use a 70/30 training-validation split and are trained over ten independent runs with mini-batch optimization and early stopping.Source: §A.5 · p. 21
- After source-domain training, models are fixed and evaluated directly on anchor and target domains in both examples.Source: §A.5 · pp. 21–22
- Sliding-resistance models use ten source-domain runs, each holding out 20% for validation, with MSE loss and ADAM at a 0.01 learning rate.The first- and second-moment decay rates are 0.9 and 0.999; training lengths differ between examples because of dataset size and numerical characteristics.Source: §A.5 · p. 22
A.6 Prediction Metrics and PT Indicators
The section specifies the accuracy metrics and distance-based PT indicators, then describes source-only evaluation on progressively complex domains without target-label training or model selection.
A.6 Prediction Metrics and PT Indicators
- Accuracy is reported using MAE, RMSE, and R^2, with means and standard deviations across ten runs; PT distances are estimated by MMD with an RBF kernel and median-heuristic bandwidth.Source: §A.6 · p. 22
- The feasibility indicator λ_Ak compares source-anchor distances in output and learned-representation spaces, while reliability indicator δ_Ak compares source-target and source-anchor latent discrepancies.Small λ_Ak indicates that the learned representation preserves the relevant physical relation across the source-anchor shift; the indicators assess whether anchor success evidences target generalization.Source: §A.6 · p. 22
A.7 Zero-Shot Extrapolations
- Models trained only on STO-3G are evaluated without retraining on four higher-fidelity oscillator-strength domains, while sliding-resistance models transfer from empirical force fields to MLFF anchors and DFT targets.For convergence analysis, models trained on progressively higher-fidelity source or anchor domains are evaluated on the most complex anchor and def2-TZVP target.Source: §A.7 · p. 22
- Target labels are excluded from training and model selection, and PT indicators are calculated before target-label-based performance assessment.This separates transfer-risk diagnosis from final target-error measurement.Source: §A.7 · p. 22