Source-linked AI summary
Learning Reduced-Order Dynamics with Singularity via Latent-Augmented Neural Ordinary Differential Equations
Xiaorui Wang, Yu Zhou, Wenjie Mei, Dongzhe Zheng, Yang Bai, Masaaki Nagahara
TL;DR
Reduced-order modeling can create self-intersecting trajectories whose states have conflicting motion directions, exposing a limitation of standard NODEs. The paper introduces LA-NODEs with latent state augmentation, establishes a minimum-dimension criterion, and reports improved modeling of complex industrial reduced-order dynamics. The framework is validated on industrial systems including IPMSM drives and distributed energy systems.
Problem
Standard NODEs cannot accurately capture reduced-order trajectories with singularities because a single observed state may correspond to multiple distinct velocities.
Method
LA-NODEs introduce learnable latent dynamics and derive an explicit criterion for the minimum augmentation dimension needed to resolve velocity ambiguities.
Results
LA-NODEs consistently outperform state-of-the-art baselines in accurately capturing complex reduced-order dynamics across multiple industrial systems.
Takeaways & Limitations
Latent state augmentation provides an effective approach for modeling reduced-order industrial systems with trajectory singularities while balancing expressivity and computational tractability.
Takeaways & Limitations
The method relies on a finite-dimensional latent space and was primarily validated on low-dimensional manifolds, leaving scalability to high-dimensional states unresolved.
Abstract
from arXiv · showhide
This paper addresses the issue of self-intersecting trajectories (in phase space) in industrial reduced-order modeling and proposes the Latent-Augmented Neural Ordinary Differential Equations (LA-NODEs) framework. From the perspective of artificial intelligence, the proposed method augments conventional neural ordinary differential equations to enhance model expressiveness, enabling the representation of conflicting vector fields that may arise in reduced-order systems, thereby improving learning accuracy. Through theoretical analysis, the underlying mechanism of the framework is established, and a condition for determining the minimum required augmentation dimension is derived. From the perspective of engineering applications, the effectiveness of the proposed method is validated on the reduced-order system of two representative industrial models, namely an interior permanent magnet synchronous motor (IPMSM) drive and a distributed energy system (DES). Experimental results demonstrate that the proposed method can recover system features that are difficult to capture using conventional approaches and achieve superior performance in terms of prediction accuracy and modeling fidelity, thereby providing an effective approach for high-precision data-driven modeling of complex industrial systems.
1. Introduction
Reduced-order modeling makes industrial systems more tractable but can lose geometric and dynamical information, creating singularities that classical NODEs cannot represent accurately. The paper proposes LA-NODEs, which augment latent dimensions to resolve these ambiguities and derives a principled criterion for selecting the required augmentation dimension.
- Reduced-order models improve computational efficiency but may sacrifice the original system’s geometric structure, dynamical properties, and numerical robustness.
- Singularities arise when reduced observations assign multiple distinct motion directions to the same state, violating the single-valued vector-field assumption of autonomous NODEs.
- Classical autonomous NODEs incur unavoidable nonzero training error when observed reduced-order dynamics are singular and multi-valued.
- LA-NODEs introduce learnable latent dynamics that lift the system into a higher-dimensional space to resolve singularity-induced vector-field ambiguities.
- The framework derives a lower bound on augmentation dimension from the rank of the velocity set, balancing expressive capacity against computational efficiency.
2. Problem Formulation
NODEs learn continuous-time vector fields from observed states, but reduced-order observations can merge internal states with different dynamics. This makes it difficult for standard NODEs to recover the complete behavior of singular reduced-order systems.
- NODEs approximate continuous-time dynamical systems by parameterizing an unknown vector field with a neural network.
- Autonomous NODEs model dynamics without time-dependent driving terms.
- Partial observations map the high-dimensional state x into a lower-dimensional observed state y through y = h(x).
- Multiple internal states may produce the same observation while having different dynamics or velocities, causing NODEs to miss intrinsic reduced-order behavior.
3. Learning Limitations of Autonomous NODEs
Reduced-order projections can create singularities where one observed state has multiple velocities, preventing autonomous NODEs from representing the dynamics consistently and forcing nonzero training error.
- Geometric origin: Reduced-order projections may produce trajectory intersections even when the full phase-space trajectory is unique and non-intersecting.Non-injective observation mappings can cause distinct internal states to share one observation.
- Geometric origin: At a singularity, the same observation is associated with different instantaneous velocities, so no single-valued autonomous dynamics can represent its evolution.This occurs when projected velocities differ along an observation-map fiber.
- Observed limitation: NODE variants that omit internal dynamics cannot accurately capture reduced-order trajectories exhibiting singularities.In the example, CSODE learns a trajectory that deviates from the true reduced-order trajectory after projection introduces a singularity.
- Training limitation: For repeated states with distinct target velocities, an autonomous NODE assigns one velocity and therefore incurs an unavoidable error determined by their velocity differences.The loss lower bound is dataset-determined and independent of the parameters or specific architecture.
- Training limitation: The limitation arises because NODEs cannot distinguish states that reduction has made observationally identical.A suitable higher-dimensional augmented space can separate these states by encoding latent dynamic features.
4. Proposed Method: Latent-Augmented NODEs
LA-NODEs augment observed states with learnable latent dynamics so singular trajectories can be separated in a higher-dimensional space. The framework can achieve zero approximation error in theory when the latent dimension spans the velocity structure required by the singularities.
- Framework: LA-NODE couples primary observed-state dynamics with k-dimensional learnable latent dynamics to resolve vector-field ambiguities at singularities.The latent state evolves through a neural network and serves as a workspace during integration, then is discarded after inference.
- Framework: The learning objective remains focused exclusively on accurately modeling the observed state, without requiring ground-truth supervision for latent variables.The loss is defined only on the p-dimensional observed state.
- Theoretical trainability: With k≥1, the augmented system can achieve zero approximation error on datasets containing singularities, unlike the strictly positive lower bound of an autonomous NODE.Distinct latent states allow repeated observations with different velocities to be represented separately.
- Dimension selection: Theorem 1 requires the augmentation dimension to satisfy k≥max_y⋆{r(y⋆)}, where r(y⋆) is the rank of distinct velocities at singularity y⋆.If k is smaller than the velocity rank, velocity information is lost and the nonzero-error limitation returns.
- Dimension selection: For an example with velocity rank 2, k≥2 permits distinct latent states and local dynamics learning, whereas k<2 leaves non-collinear velocities inseparable.The global bound is governed by the most demanding singularity, while excessive augmentation mainly increases computational cost.
5. Experiments
The experiments evaluate LA-NODE on reduced-order IPMSM-drive and distributed-energy-system dynamics exhibiting two-trajectory and three-trajectory singularities.
- Industrial systems: The evaluation uses an IPMSM drive and a distributed energy system whose reduced-order dynamics exhibit different singularity structures.The studied cases include singularity points with two or three distinct velocity directions.
5.1. Experimental Setup
The experiments standardize the data, train models with Adam and MSE, and compare LA-NODE against Transformer, CSODE, and ANODE baselines.
- Training protocol: All data are standardized and split into training and testing sets with a 9:1 ratio, using MSE for both phases.Training uses Adam with learning rate 0.01.
- Training protocol: The IPMSM and DES systems are trained for 1000 and 700 epochs, respectively.The number of epochs is configured separately for each system.
- Baselines: LA-NODE is benchmarked against Transformer, CSODE, and ANODE frameworks.CSODE provides an unaugmented comparison, while ANODE represents a general-purpose augmentation framework.
- Baselines: The Transformer supplies a non-ODE neural-network baseline for sequential-data prediction using positional encodings and self-attention mechanisms.This comparison separates the proposed ODE-based framework from a classical neural-network approach.
- Evaluation: Comparative experiments are conducted to assess the performance of the proposed LA-NODE framework.
5.2. Industrial Systems for Experiments
The experiments examine reduced-order modeling for an IPMSM drive and a distributed energy system. Both systems are reduced through observation mappings, with hidden variables creating trajectory singularities that motivate latent augmentation.
- IPMSM Drive Model: The IPMSM experiment uses physical data from a strongly nonlinear, highly coupled electromechanical system.The study reduces the IPMSM from R^3 to R^2 using mapping h1.
- IPMSM Drive Model: Eliminating electrical angular velocity allows different full-order states to share one reduced observation.This produces a two-trajectory singularity with an associated velocity ambiguity set.
- IPMSM Drive Model: The IPMSM reduction requires minimum augmentation dimension k=1 under Theorem 1.The two velocity trajectories span a one-dimensional subspace.
- Distributed Energy System: The DES benchmark combines photovoltaic generation, energy storage, and dynamic loads to represent multi-energy coupling and energy balance.Its simulation includes a proportional-integral-controlled battery input and operational constraints on state of charge and angular frequency.
0.9 × 𝜔0 ≤𝑥2(𝑡) ≤1.1 × 𝜔0.
The DES experiment reduces internal states through an observation mapping, where multiple internal states can correspond to one reduced observation. Theorem 1 then determines the minimum augmentation dimension required for representation.
- DES Simulation Constraints: The DES simulation uses system parameters and operational constraints listed in Table 2.The supplied passage identifies Table 2 as the source of simulation parameters and constraints.
- DES Reduction: The DES internal state is observed through the mapping h2: R^4 → R^2.The reduced-order velocity is obtained from the reduction mapping and its Jacobian.
- DES Reduction: Multiple DES internal states may correspond to the same reduced observation, creating the trajectory ambiguity addressed by the framework.The passage states this correspondence is similar to the IPMSM case.
- DES Reduction: The minimum DES augmentation dimension is selected as k=2 under Theorem 1.This dimension satisfies the stated representational requirement.
5.3. Experimental Results and Analysis
Experiments on IPMSM and DES reduced-order systems show that LA-NODE more accurately captures singular trajectories than the comparison frameworks. It achieves lower errors, better trajectory tracking, and stronger modeling stability across the reported evaluations.
- IPMSM Drive: LA-NODE achieves the lowest IPMSM trajectory-wise error, remaining within [0.001,0.01], while CSODE remains at the level of 1 for most indices.ANODE is mostly on the order of 0.01, and Transformer remains around 0.1.
- IPMSM Drive: LA-NODE maintains narrow error fluctuations, whereas Transformer has the largest variance and CSODE combines low fluctuation with the highest error.The reported variance pattern indicates stronger consistency for LA-NODE and ANODE than Transformer.
- IPMSM Drive: LA-NODE remains close to the ground-truth IPMSM trajectories near singular regions, while CSODE progressively deviates in its vector-field direction.LA-NODE preserves dynamic evolution despite slight discrepancies between learned and true singular-point locations.
- IPMSM Drive: LA-NODE reaches lower IPMSM MSE than ANODE and Transformer, with training loss ranging from [0.0001,0.00001] and testing loss decreasing as epochs increase.ANODE and Transformer training losses remain in [0.001,0.0001].
- DES: LA-NODE provides the most accurate DES predictions, with scatter points tightly aligned to the y = x reference diagonal for x3 and x4.Transformer and CSODE show larger deviations and later-stage difficulty tracking system evolution.
- DES: DES test MSE for LA-NODE decreases from 0.01 to below 0.0001, while ANODE stabilizes around 0.0003 and CSODE remains around 0.01.Transformer reduces error gradually but remains within [0.01,0.001].
- DES: LA-NODE achieves superior DES modeling performance across accuracy, stability, and generalization compared with the evaluated frameworks.Table 3 reports lower MAE and RMSE than ANODE by approximately 56.1% and 58.9%, respectively.
6. Sensitivity Analysis of the Augmented Dimension
The sensitivity analysis shows that increasing augmentation dimension generally lowers error, but most of the benefit occurs at the first dimension increase. Further augmentation adds computational cost and can eventually worsen test loss.
- Error Reduction: 88.5% of total training-error reduction and 97.4% of total test-error reduction occur when augmentation increases from k = 1 to k = 2.Later stages contribute only 8.1% and 3.4% for training error, and 2.1% and 0.5% for test error.
- Computational Cost: Training time increases with augmentation dimension because larger augmented spaces introduce additional parameters and optimization complexity.Table 4 reports average total training time over five runs for 700 epochs.
- Trade-off: Increasing augmentation from k = 1 to k = 2 yields the major performance improvement, while further increases provide marginal test-loss reduction and additional computational cost.The trade-off analysis identifies k = 2 as balancing representation capability and computational efficiency.
- Error Reduction: Test loss slightly increases when augmentation changes from k = 4 to k = 5, so larger augmentation dimensions do not guarantee monotonic improvement.The analysis attributes this to additional dimensions becoming unnecessary after the velocity-ambiguity requirement is satisfied.
- Trade-off: Selecting the minimum feasible augmentation dimension avoids redundant latent variables and better balances representation capability with computational efficiency.The paper links excessive augmentation to unnecessary model complexity and performance fluctuations.
7. Conclusion
The paper concludes that LA-NODEs address singular reduced-order dynamics by augmenting latent state dimensions and selecting them through an explicit criterion. The framework outperforms evaluated baselines, but its scalability and robustness beyond the tested low-dimensional settings remain open questions.
- Conclusion: LA-NODEs augment latent state dimensions to learn reduced-order dynamical systems with trajectory singularities, addressing a limitation of standard CSODEs.The framework is designed to represent dynamics that standard methods cannot capture consistently.
- Conclusion: An explicit criterion determines the minimum augmentation dimension, balancing model expressivity against computational tractability.For new systems, the dimension should follow the maximum velocity ambiguity rank among singularities.
- Conclusion: Experiments on multiple industrial systems show that LA-NODEs consistently outperform state-of-the-art baselines in capturing complex reduced-order dynamics.The conclusion summarizes the reported modeling advantage across the evaluated systems.
- Limitations: The framework was primarily validated on low-dimensional manifolds, leaving scalability and generalization to high-dimensional state spaces insufficiently explored.Its stability under stochastic perturbations or out-of-distribution disturbances also requires more rigorous quantification.