On the Identifiability of Hybrid Deep Generative Models: Meta-Learning as a Solution

1 Introduction

• Hybrid-DGMs combine physics-based expressions with neural components to model systems when prior physics knowledge is incomplete.Physics-based inductive bias may improve generalizability and interpretability, while neural functions can fill gaps in prior physics knowledge.Source: §1 · p. 1
• Identifying physics parameters from observed motion can support predictions of a pendulum’s future trajectory.The introduction uses this as an example of why accurate identification matters.Source: §1 · p. 2
• The identifiability of hybrid-DGMs remains unexplored, despite its relevance to prediction beyond observations and out-of-distribution generalization.The paper asks how general DGM unidentifiability applies to hybrid models and how to construct identifiable ones.Source: §1 · p. 2
• The paper presents meta-learning as a way to establish identifiability without observed auxiliary variables, using conditional independence given few-shot context samples.It reports theoretical results and empirical evidence on synthetic and real physics-system data.Source: §1 · p. 2

Hybrid-DGMs

• Existing hybrid-DGM work improves learning of physics and neural components, but has not established their identifiability theoretically or empirically.Approaches include regulating neural expressiveness and using expert augmentation for out-of-distribution robustness.Source: §2 · pp. 2–3
• This work takes the first step toward addressing the gap by probing unconditional hybrid-DGM unidentifiability and proposing meta-learning for identifiability.The paper states that the connection may also apply to general, non-hybrid DGMs.Source: §2 · p. 3

Nonlinear ICA and Identifiability of DGMs

• Identifiability research commonly uses conditional independence given observed auxiliary variables, such as time indices or class and domain labels.Other time-series approaches leverage temporal dependence or inferred domain labels.Source: §2 · p. 3
• Observed conditioning variables may be unavailable for hybrid-DGMs with unknown, often continuous, physics parameters.The paper presents meta-learning as a condition for identifiable DGMs without such observed auxiliary variables.Source: §2 · p. 3

3 Problem Formulation: Unidentifiability of Hybrid-DGMs

• A hybrid-DGM uses physics-based latent variables for known mathematical components and abstract neural latent variables for missing physics.The model combines the physics expression and neural function in a mixing function; the physics component can have unknown parameters.Source: §3 · pp. 3–4
• In the pendulum example, the neural ODE is intended to absorb damping and external-force effects absent from the ideal-pendulum physics expression.The abstract latent variable represents the remaining data-generating parameters.Source: §3 · pp. 3–4
• Perfectly matching the observed marginal density under an unconditional prior does not guarantee recovery of the latent posteriors or joint data-generating process.The identifiability goal includes recovering the likelihood, priors, and posteriors; matching the marginal density alone does not ensure a unique solution.Source: §3 · p. 4
• Nonlinear transformations of neural latent variables can preserve the observed-data fit while yielding non-unique likelihoods and posteriors.The paper identifies this as theoretical unidentifiability of the neural component.Source: §3 · p. 4
• The neural component’s expressiveness can overpower optimization of the physics component, producing trivial physics parameters or functions.The physics form itself cannot trivially accommodate arbitrary transformations of its latent variables without affecting the fit.Source: §3 · p. 4
• The paper identifies two challenges: neural-component unidentifiability and that component overpowering the otherwise identifiable physics component.It positions its approach as addressing the first challenge at its fundamentals, rather than only the overpowering issue targeted by prior work.Source: §3 · p. 4

4 Learn-to-Identify Hybrid-DGMs via Meta-Learning

• Meta-hybrid-DGMs generate query samples conditioned on k context samples sharing the same data-generating process.The context samples serve as the conditioning variable in place of observed auxiliary labels or domain indices.Source: §4 · p. 5
• The conditional prior for zN is a factorized exponential family with fixed sufficient statistics and context-dependent parameters produced by a neural network and averaged across context samples.The paper notes that exponential families have universal approximation capabilities, making this assumption not very restrictive.Source: §4 · p. 5
• Amortized inference approximates the physics posterior from x and the neural posterior from x together with context samples.The context-conditioned neural posterior uses the same network as the conditional prior, with x as an additional input.Source: §4 · p. 5
• The metaobjective maximizes query-sample marginal likelihood conditioned on support sets across M distinct data-generation processes, with k much smaller than l.Source: §4 · p. 5
• Training shuffles support and query samples within each process per episode, uses reparameterization for likelihood estimates, and calculates both KL terms analytically.Source: §4 · p. 5
• The identifiability construction conditions the neural component; physics-component identifiability does not depend on this construction, though the conditional prior can also be extended to it.Source: §4 · pp. 5–6

5 Identifiability Theory for Meta-Hybrid-DGMs

• Theorem 1 establishes identifiability of meta-hybrid-DGMs under its stated assumptions, up to the paper’s equivalence relation on parameters.Source: §5 · p. 6
• The equivalence relation allows recovery of the generative model’s sufficient statistics up to a linear matrix multiplication.Source: §5 · p. 6
• The theorem’s conditions include an injective hybrid mixing function, differentiable and linearly independent sufficient functions, and an invertible context-difference matrix.The theorem also requires the stated characteristic-function zero set to have measure zero.Source: §5 · p. 6
• The context-set condition requires de+1 distinct processes; for the pendulum example with d=3 and Gaussian statistics e=2, that is seven combinations of parameter values.Source: §5 · p. 6
• Table 1 compares latent-variable MSE and MCC with reconstruction and prediction MSE for the presented model, physics-integrated hybrid-VAE, and APHYNITY.Source: Table 1 · p. 7

6 Experiments and Results

• Experiments used three simulated systems and one real-world double pendulum, comparing meta-hybrid-VAE with physics-integrated hybrid VAE and APHYNITY using latent-identification and reconstruction/prediction metrics.For real data, the physics component used known arm lengths and assumed mass parameter ˜m = 1; the model did not explain the observed decline in total energy over time.Source: §6 · p. 7, §6.2 · p. 9

6.1 Results on Synthetic Data

• Meta-hybrid-VAE achieved the highest MCC for neural latent zN and the lowest MSE for physics latent zP, while baselines had substantially lower MCC despite reasonable zP estimates.All models had comparable reconstruction MSE, but baseline prediction MSE deteriorated by two magnitudes while meta-hybrid-VAE remained comparable to reconstruction.Source: §6.1 · p. 8, Figure 1 · p. 8
• Across three simulated systems, meta-hybrid-VAE showed significant improvement over baselines, including on the relatively easier advection-diffusion system and harder double-pendulum system.Baseline degradation in MCC and prediction MSE was smaller for advection-diffusion, potentially because its missing prior-physics component had a small effect.Source: §6.1 · p. 8
• As prediction horizons extended beyond training, all models deteriorated, but meta-hybrid-VAE showed a smaller performance drop and slower deterioration.Figure 2 shows prediction performance over longer time domains and a forced damped pendulum example.Source: §6.1 · p. 8, Figure 2 · p. 9
• For forced damped pendulum settings with either latent component outside the training distribution, meta-hybrid-VAE achieved the strongest identification performance.Figure 3 (left) presents the OOD results.Source: §6.1 · p. 8, Figure 3 · p. 9
• For the pendulum system, neural-latent MCC exceeded 0.85 with seven meta-learning tasks and stabilized with eight or more, matching the stated 3*2+1 task condition.This setting used three neural-component parameters and a Gaussian assumption with two-dimensional sufficient statistics.Source: §6.1 · p. 9, Figure 3 · p. 9
• Ablations found meta-hybrid-VAE identifiability minimally affected by the number of parameters when the theoretical condition held; its meta-formulation moderately improved physics-parameter accuracy.The physics-based component did not exhibit the type of un-identifiability discussed for the neural component.Source: §6.1 · p. 9

6.2 Real Data of Double Pendulum

• On real double-pendulum data, meta-hybrid-VAE prediction MSE was similar to reconstruction MSE, whereas both baselines’ prediction errors deteriorated by 2–3 magnitudes.Ground-truth generative factors were unavailable, so prediction MSE served as an identifiability surrogate; Figure 4 shows reconstruction and generation examples.Source: §6.2 · p. 10, Figure 4 · p. 10

• The paper probes unidentifiability in hybrid-DGMs with unconditioned priors and presents meta-learning as a solution, supported by theoretical and empirical evidence.Source: §7 · p. 10
• Comprehensive ablations of k-shot context size and zN dimension remain needed; comparisons with purely physics-based and purely neural DGMs were out of scope.In synthetic experiments, zN dimension was set to the number of true generative factors, identically across baselines.Source: §7 · p. 10
• The paper provides an initial evaluation of the meta-formulation on a general non-hybrid VAE using synthetic non-stationary Gaussian time series and comparison with iVAE.Source: §7 · p. 10
• The meta-formulation assumes context and query samples can be paired from the same data-generation process; without pairing information, it is not applicable.Preceding temporal samples were effective for pairing in the reported real-data experiments.Source: §7 · p. 10

A.1 Forced Damped Pendulum

• For the forced damped pendulum, 60,000 trajectories were split into 40,000 training, 10,000 validation, and 10,000 test samples; models used ω as partial physics and were expected to recover ξ, A, and ϕ.OOD ranges were ω ∈ [3.14, 3.5] and ξ ∈ [0.8, 1.0]; this setup induces additive error in prior physics.Source: §A.1 · p. 13

A.2 Advection-diffusion System

• For advection-diffusion, 20,000 training and 5,000 each validation and test samples were generated; models used a as partial physics and were expected to recover b.The setup induces additive error in prior physics, with a and b sampled from [0.01, 0.1].Source: §A.2 · p. 13

A.3 Double Pendulum

• For the double pendulum, models used the partial physics with m̃ = 1 and G as zP, and were expected to recover m̃ from 20,000 training and 5,000 each validation and test samples.The true parameters were sampled as G ∈ [5.0, 15.0] and m̃ ∈ [0.5, 1.5], inducing multiplicative error in prior physics.Source: §A.3 · p. 13

B The proof of Theorem 1

• Theorem 1 assumes a measure-zero zero set, an injective hybrid mixing function, differentiable and linearly independent sufficient functions, and an invertible context-set matrix.Source: B The proof of Theorem 1 · p. 13
• The proof begins by equating conditional densities under two parameterizations and deriving log-density equations across context sets.Source: B The proof of Theorem 1 · p. 14
• The proof uses Jacobian rank and matrix invertibility to conclude that the parameterizations are equivalent under the stated relation.Source: B The proof of Theorem 1 · pp. 14–15

C.1 Effect of the Dimension of Unknown Parameters

• Identifiability results for meta-hybrid-VAE were minimally affected by the number of unknown parameters when the theoretical identifiability condition was met.The ablation is reported for the pendulum system in Table 2.Source: §C.1 · p. 15, Table 2 · p. 15

C.2 Meta-component on cp

• The physics component did not show the discussed neural-component unidentifiability, while the meta-formulation moderately improved physics-parameter estimation accuracy.The comparison is reported in Table 3.Source: §C.2 · p. 15, Table 3 · p. 15

• The evaluation compares Meta-VAE with VAE and iVAE on synthetic non-stationary Gaussian time series.Source: D Results on General Non-Hybrid VAEs · p. 15
• Performance is measured by MCC between original sources and corresponding latent samples from the learned posterior.Source: D Results on General Non-Hybrid VAEs · p. 15
• Table 4 contains the results for the general non-hybrid VAE comparison.Source: Table 4 · p. 15

E Strong MCC results

• ∼P denotes the equivalence relation when A is a block permutation matrix.The section introduces this as a stronger identifiable relation for evaluation.Source: E Strong MCC results · pp. 15–16
• For e ≥ 2, the stated conditions require twice-differentiable sufficient statistics and second-order cross derivatives of Fθ.The theorem also assumes the hypotheses of Theorem 1.Source: E Strong MCC results · p. 16
• For e = 1, the stated conditions require non-monotonic sufficient statistics and continuous partial derivatives of Fθ.The theorem also assumes the hypotheses of Theorem 1.Source: E Strong MCC results · p. 16
• Strong MCC is reported against ground-truth variables and between model runs under ∼P, with results in Table 5.Figure 5 visualizes learned latent variables across runs for a forced damped pendulum system.Source: E Strong MCC results · p. 16, Table 5 · p. 16, Figure 5 · p. 16