PaperlayerView exact revision on arXiv

Source-linked AI summary

Dirac-Interconnected Neural Elements: Discovering Modularity in Physical Systems Without Reduction

Reiho Li, Razmik Arman Khosrovian, Takaharu Yaguchi, Hiroaki Yoshimura, Takashi Matsubara

arXiv:2610.02960v1cs.LG

TL;DR

Existing approaches to physical-system learning either assume interconnections in advance or reduce systems to ODEs, discarding algebraic constraints. DINEs learn a DAE with a Dirac-structure interconnection and neural component models; experiments show strong prediction results and accurate subsystem reuse and composition.

  • Problem

    Existing methods may require a known interconnection or reduce the system to an ODE, leaving algebraic interconnection constraints unrepresented.

    Source: §1 · p. 1, §2.1 · p. 4

  • Method

    DINEs jointly learn component characteristics and a separable Dirac interconnection in an unreduced DAE representation.

    Source: §1 · p. 2, §3 · p. 5

  • Results

    DINEs achieved the lowest MSEs and highest VPTs in Tasks 1–4, and high prediction accuracy for Task 5 isolation and composition.

    Source: §4.2 · pp. 8–9

  • Takeaways & Limitations

    Experiments show DINEs learn constraints and unobserved states beyond existing methods and compose or reuse learned subsystems across physical domains without retraining.

    Source: §5 · p. 9

    The interconnection parameterization requires selecting and sweeping nf, and the setting excludes gyrators and state-dependent constraints.

    Source: §3 · p. 6

Abstract

from arXiv · show

Deep learning has shown remarkable success in the data-driven modeling of dynamical systems. Much of its success is attributed not to the flexibility of neural networks but to inductive biases based on physical prior knowledge, such as energy conservation and symplecticity. However, existing methods do not fully exploit the fact that real-world physical systems are interconnections of components. Some methods require the interconnection to be known a priori, while others assume the system to be reducible to an ordinary differential equation (ODE) and learn only the reduced ODE, discarding the algebraic constraints imposed by the interconnection. Here, we propose Dirac-interconnected neural elements (DINEs), a neural network model that represents a physical system as a differential-algebraic equation (DAE), whose algebraic constraints are given by a Dirac structure in kernel representation. With DINEs, we simultaneously identify from data the interconnection among the components as a Dirac structure and learn the characteristics of the components as neural networks. This allows us to keep the learned subsystems in unreduced form and isolate or compose them to make a new system without retraining. Moreover, DINEs can handle partially observable systems. Experimental results demonstrate these capabilities on physical systems beyond the reach of existing methods.

1 INTRODUCTION

Physical systems are interconnections of components whose algebraic constraints matter, but prior methods either assume a known interconnection or reduce the system to an ODE. DINEs learn the interconnection and component characteristics together while retaining the DAE form, including for partially observable systems.

1 INTRODUCTION

  • Energy-based neural models use physical prior knowledge to promote energetic behavior, whereas black-box trajectory models can behave inconsistently in long-term predictions.The cited energy-based examples derive a vector field from an energy function defined on the state space.Source: §1 · p. 1
  • Identifying component characteristics and interconnections from data can make models interpretable and components reusable, while supporting parameter identification, diagnosis, and design.Interconnected physical systems include mechanical components such as masses, springs, and dampers, and electrical components such as inductors, capacitors, and resistors.Source: §1 · p. 1
  • Existing methods either assume the interconnection is known or learn it only after reduction, discarding algebraic constraints and excluding partially observable systems.The comparison identifies no existing model that both learns the interconnection and retains its algebraic constraints.Source: §1 · p. 2, Table 1 · p. 2
  • DINEs represent physical systems as DAEs with algebraic constraints given by a Dirac structure in kernel representation.They jointly learn the interconnection and component constitutive relations from data.Source: §1 · p. 2

2 BACKGROUND

Port-based modeling describes components through energy-exchanging ports and their interconnection through power-continuous constraints. Reduction to an ODE can mix component and interconnection information or fail when an ODE representation does not exist; prior learning approaches handle only parts of this modeling problem.

2.1 PORT-BASED MODELING AND DIRAC STRUCTURES

  • Port-based models describe storage, resistive, and source elements through flow and effort variables, with the interconnection routing energy between their ports.Storage elements exchange stored energy, resistive elements absorb power, and sources prescribe a flow or effort as a time-varying input.Source: §2.1 · pp. 2–3
  • A Dirac structure formalizes the interconnection as port-variable constraints that conserve power, and its kernel representation writes them as Ff + Ee = 0.Each row imposes a linear constraint; the entries of F and E encode the interconnection, including transformer and gyrator ratios.Source: §2.1 · p. 3
  • Reduction can mix resistive relations with their connections and assumes an ODE exists, excluding cases such as nonlinear friction or balanced forces from nonlinear springs.The cited reduced forms also do not apply to a capacitor connected in parallel with a voltage source.Source: §2.1 · p. 4

2.2 RELATED WORK

  • Single-ODE neural models do not identify individual elements or their interconnection, while several component-learning methods require the interconnection to be given in advance.N-pHDAEs use known circuit topology; PoDiNNs learn a reduced interconnection but assume all states are observed, and methods for unobserved states restrict which outputs can be observed.Source: §2.2 · pp. 4–5

3 DIRAC-INTERCONNECTED NEURAL ELEMENTS

DINEs jointly learn component constitutive relations and a separable, constant Dirac interconnection without reducing the system to an ODE. Training fits observations and DAE residuals, while the learned representation supports subsystem isolation and composition; its scope and parameterization impose specific assumptions and limitations.

3 DIRAC-INTERCONNECTED NEURAL ELEMENTS

  • DINEs learn each storage element’s energy and each resistive element’s relation alongside a separable, constant Dirac structure, preserving the system as a DAE.The number of independent flow constraints nf must be specified; the parameterization exactly represents separable Dirac structures with that number of flow constraints.Source: §3 · p. 5
  • The DAE’s energy change equals power at resistive and source ports; imposing a dissipative sign constraint on resistive relations bounds it by source-port power.Without that sign constraint, the displayed balance still holds.Source: §3 · p. 5
  • The trajectory network supplies continuous-time states and port variables, and joint training minimizes observation error and the DAE residual.It acts as a numerical integrator during training and is discarded afterward; prediction integrates the learned DAE from initial values consistent with its algebraic rows.Source: §3 · pp. 5–6
  • Because constitutive relations and the Dirac structure remain separate, learned elements and subsystems can be isolated, reused, and composed without retraining.Composition adds boundary ports and constraints; connected ports may alternatively be eliminated algorithmically before integration.Source: §3 · p. 6
  • The flow-constraint count nf must be selected and swept in practice; a misspecified value excludes the true Dirac structure and causes catastrophic failure.The interconnection is assumed separable and constant, and the parameterization does not automatically determine nf.Source: §3 · p. 6
  • The setting excludes gyrators, state-dependent interconnection constraints, and kinetic energies that jointly depend on momenta and positions.The cited examples include motors, a multi-link robot in Cartesian coordinates, and the same robot in generalized coordinates.Source: §3 · p. 6

4 EXPERIMENTS AND RESULTS

The experiments test whether DINEs learn interconnection constraints, handle unobserved states, and support subsystem composition and reuse. Across Tasks 1–4, they achieve the lowest MSEs and highest VPTs; Task 5 demonstrates accurate isolation and composition.

4.2 RESULTS

  • DINEs compose learned subsystems without retraining in Tasks 3–4 and isolate a learned spring for reuse in Task 5.Task 4 identifies the unknown piston area from a few measurements of the composed system.Source: §4.1 · pp. 7–8, §4.2 · p. 9

4.1 EXPERIMENTAL SETTINGS

  • The five tasks assess constraint learning, learning from unobserved states, subsystem composition, and element reuse; Task 2 uses observed currents while capacitor charges are unobserved.Task 1 tests a three-way junction, Tasks 3–4 test composition, and Task 5 tests spring reuse.Source: §4.1 · p. 7

4.2 RESULTS

  • DINEs achieved the lowest MSEs and highest VPTs in Tasks 1–4, with high prediction accuracy on Task 5.Table 2 summarizes numerical performance; Task 5 has no baseline, and its result supports accurate isolation and composition in a setting where other components are known.Source: §4.2 · pp. 8–9, Table 2 · p. 8
  • On Task 1, Neural ODE predictions grew inaccurate after leaving the interconnection-constraint manifold, while DINEs learned the interconnection and suppressed such errors.PoDiNNs also performed poorly because they cannot learn this kind of constraint.Source: §4.2 · p. 8
  • On Tasks 3–4, DINEs achieved MSEs nearly two orders of magnitude smaller and much longer VPTs than Coupled Neural ODEs.Coupled Neural ODE predictions looked reasonable, but DINEs’ Dirac structure provides the boundary-port relation up to gauge freedom.Source: §4.2 · pp. 8–9
  • Soft constraints failed to yield converged Newton initialization or prediction in most seeds on Tasks 1–2, supporting the use of hard constraints.The authors suggest a DAE solution may not exist when constraints are only soft.Source: §4.2 · p. 9
  • The number of flow constraints nf is unknown; misspecification produced nonconvergence or catastrophic prediction metrics on Task 1, but can be tuned using a small validation set.Training uses a trajectory network, so the reported nonconvergence for nf = 1 occurred during prediction, not training.Source: §4.2 · p. 9
  • The learned F and E matrices recovered the interconnection in Tasks 1–4, although their rows are non-unique and require readout for interpretation.The Dirac structure depends on the row spaces of F and E.Source: §4.2 · p. 9

5 CONCLUSION

The conclusion presents DINEs as a way to learn physical systems as unreduced DAEs with separable Dirac-structure constraints. The authors report learning constraints and unobserved states and composing or reusing learned components without retraining; automatic determination of nf and non-separable structures remain future work.

5 CONCLUSION

  • DINEs represent physical systems as DAEs with algebraic constraints from a separable Dirac structure parameterized exactly by an orthogonal matrix, without ODE reduction.The authors report learning constraints and unobserved states beyond existing methods, and composing or reusing learned subsystems and elements across physical domains without retraining.Source: §5 · p. 9

AI USE STATEMENT

  • The authors used large language models for manuscript editing, formatting checks, and related-work surveying, then reviewed the text and manually checked derivations.They take responsibility for the final content, claims, and artifacts.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The work uses simulated data from known equations of motion and involves no human subjects, personal data, or privacy-implicating data releases.The authors foresee no direct harmful applications.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper specifies where to find the model, loss, implementation, task settings, and baseline details, and says source code and data-generation scripts will be public after refactoring.The statement describes planned public release after refactoring.Source: REPRODUCIBILITY STATEMENT · p. 10

A ELEMENTS AND JUNCTION STRUCTURE OF PORT-BASED MODELING

The appendix outlines port-based modeling elements and junctions, then derives an input-state-output form from the Dirac-structure equations. It also explains why reduction can obscure component-interconnection structure or fail when an ODE representation does not exist.

A ELEMENTS AND JUNCTION STRUCTURE OF PORT-BASED MODELING

  • Port-based modeling represents a physical system as interconnected elements that idealize basic behaviors; the appendix catalogs variables, states, and elements across mechanical, electromagnetic, and hydraulic domains.The setting excludes multiport storage elements whose energy depends on several states.Source: A ELEMENTS AND JUNCTION STRUCTURE OF… · p. 14, Table 3 · p. 14
  • A 0-junction imposes common effort with balanced flows, while a 1-junction imposes common flow with balanced efforts; transformers and gyrators relate two ports while preserving power.The appendix gives examples including springs, dampers, levers, gearboxes, and DC motor pumps.Source: A ELEMENTS AND JUNCTION STRUCTURE OF… · p. 14

B REDUCTION OF EQ. (3) TO THE INPUT-STATE-OUTPUT FORM

  • Exchanging flow and effort variables for selected ports yields an input-output representation when the chosen resistive-port matrix is invertible; its matrix B is skew-symmetric.After eliminating remaining port variables through constitutive relations, this can yield an ODE, as in the form used by PoDiNNs.Source: B REDUCTION OF EQ. (3) TO THE… · pp. 14–15
  • The appendix places Hamiltonian, port-Hamiltonian, Lagrangian, and constrained neural models within or alongside the reduced form, while contrasting DAE methods that reduce or retain the DAE.N-pHDAEs reduce by index reduction; pH-DAE identification avoids reduction but learns only element parameters in the described experiments.Source: B REDUCTION OF EQ. (3) TO THE… · p. 15
  • In the input-state-output form, the reduced matrix R combines resistive constitutive relations with interconnection blocks, so it cannot be uniquely decomposed into those parts.For nonlinear explicit resistive relations, the dissipative term depends on state rather than factoring as a constant matrix times the energy gradient.Source: B REDUCTION OF EQ. (3) TO THE… · p. 15
  • Reduction assumes an ODE exists, but some systems require state elimination or state-dependent matrices, and some constraints determine state directly from the input.Examples include balanced forces from nonlinear springs connected to a massless point and a capacitor in parallel with a voltage source.Source: B REDUCTION OF EQ. (3) TO THE… · p. 15

C PROOFS

The proofs characterize the exact parameterization of separable Dirac structures, while the implementation learns component relations and interconnections and retains subsystems in unreduced form for composition.

C PROOFS

  • Theorem 1 shows that every separable Dirac structure with exactly nf independent flow constraints has the parameterized form D(Ω), and that this form defines such a structure.The construction uses orthogonal row blocks with full row rank and mutually orthogonal flow and effort rows.Source: C PROOFS · p. 16

D.1 LEARNABLE ELEMENTS AND INTERCONNECTION

  • Element networks represent storage energies and resistive relations, while the interconnection matrices use the exact Dirac-structure parameterization and are initialized without the true matrices.Nonlinear storage energies augment a quadratic term; resistive relations combine a neural network with a linear term.Source: §D.1 · pp. 16–17

D.2 LEARNING AND PREDICTION

  • The trajectory network predicts storage states, resistive flows, and source-port conjugate variables, and training minimizes observation error alongside the DAE residual.Training samples observation and residual terms from separate minibatches of trajectory-time pairs.Source: §D.2 · p. 17
  • Because observations do not determine the scale of an unobserved resistive flow, the loss regularizes its minibatch variance toward one.The variance term applies to the set of resistive elements whose flows are unobserved.Source: §D.2 · p. 17
  • Prediction initialization estimates the initial state and algebraic variables from observations subject to the learned DAE, then integrates the learned DAE from that state.When a history is available, its unknowns are estimated jointly using the DAE and the trapezoidal rule between samples.Source: §D.2 · pp. 17–18

D.3 ISOLATION AND COMPOSITION

  • Before composition, boundary-variable scale is calibrated from a known junction coefficient when possible, or from additional measurements otherwise.An unknown transformer ratio is identified by closed-form least squares after both subsystem scales are fixed.Source: §D.3 · p. 18
  • The composed DAE combines frozen subsystem DAEs with their connection, solving boundary variables alongside states without retraining on composed-system data.The boundary variables are unknowns at each Radau IIA integration stage.Source: §D.3 · p. 18

D.4 ABLATIONS

The ablations replace the exact Dirac parameterization with soft penalties or set the flow-constraint count incorrectly; the common settings define training and prediction evaluation.

D.4 ABLATIONS

  • DINEs (soft) learn F and E independently and add penalties for Dirac-structure conditions, using the learned matrices directly without projection.The row-orthonormality penalty targets deviations from orthonormal rows; exact row orthonormality implies the rank condition.Source: §D.4 · p. 19
  • With nf set incorrectly, the true Dirac structure lies outside the model family, which the authors describe as leading to catastrophic failure.The other settings remain unchanged.Source: §D.4 · p. 19

E.1 COMMON SETTINGS

  • Common settings use five training seeds and test trajectories of 10,000 steps, with MSE and valid prediction time averaged over ten test trajectories per seed.The training sets contain 100 trajectories of 500 steps, and predictions use double precision.Source: §E.1 · pp. 19–20

E.2 TASK 1: THREE-WAY JUNCTION

Task 1 tests learning a three-way mechanical junction from trajectories, with a massless junction imposing flow and effort constraints.

E.2 TASK 1: THREE-WAY JUNCTION

  • Three masses attach by springs to a massless junction, whose force balance constrains the system; the springs are nonlinear and there are no dampers or inputs.The system contains three C-type and three I-type elements, with two flow constraints and four effort constraints.Source: §E.2 · p. 20
  • Training and test initial states use the same distributions, and all six spring elongations and mass velocities are observed.The third elongation is solved from the force balance.Source: §E.2 · p. 20
  • DINEs assume linear I-type elements and nonlinear springs, while Neural ODEs and PoDiNNs serve as comparison models with one-step training.PoDiNNs receive the element list and linearity of I-type elements, but neither the true constraint nor the state-dependent reduced-ODE coupling matrix.Source: §E.2 · pp. 20–21

E.3 TASK 2: RC LADDER

Task 2 evaluates modeling of an RC ladder with nonlinear resistors, voltage inputs, and partially observed currents.

E.3 TASK 2: RC LADDER

  • The RC ladder has two voltage sources, four grounded linear capacitors, and seven nonlinear resistors connected across two nodes.Node voltages are algebraic variables determined by Kirchhoff current laws and nonlinear resistor relations.Source: §E.3 · p. 21
  • The two source voltages combine common and differential modes with frequency bands split between the modes and input levels varied across trajectory groups.The groups use upper frequency limits of 0.03 or 0.08 cycles per step and levels κ of 4, 12, or 20.Source: §E.3 · pp. 21–22
  • Ten port currents are observed, while capacitor charges, node voltages, and the currents of resistors R0, R5, and R6 are unobserved.Each test trajectory includes a 20-sample input and observation history before prediction.Source: §E.3 · p. 22
  • DINEs assume unknown linear capacitances and nonlinear resistors, whereas CT-SUBNET encodes a 20-sample history and predicts through a learned latent-state model.DINEs regularize flows of the three unobserved resistors; CT-SUBNET is trained to predict 50 outputs from each sampled window.Source: §E.3 · p. 22

E.4 TASK 3: RIGID COUPLING

Task 3 tests whether separately trained mechanical subsystems can be coupled through a rigid boundary without training on the composed system. The setup compares DINEs with Coupled Neural ODEs using subsystem training data and held-out composed-system trajectories.

E.4 TASK 3: RIGID COUPLING

  • Each subsystem is a mass–spring–damper chain with one boundary port; their recorded boundary velocities are v3 for A and vB1 for B.Subsystem A has three masses, while B has two masses and an additional external force input.Source: §E.4 · p. 23
  • The composed system equates the subsystems’ boundary velocities, with an unknown reaction force and no composed-system training data.Only the external force on subsystem B is known during composed-system prediction.Source: §E.4 · p. 23
  • Ten accepted test trajectories are selected from at most 500 candidates, requiring states, inputs, and boundary variables to remain within subsystem training ranges and coupling residuals below 2 × 10^-7.The same test set is used for both methods and all seeds.Source: §E.4 · p. 24
  • DINEs use linear I-type elements with unknown coefficients and nonlinear springs and dampers, while Coupled Neural ODEs learn subsystem vector fields from observed states and prescribed boundary inputs.The boundary velocity scale for DINEs is fixed from the learned interconnection; Coupled Neural ODEs separately train a boundary network after freezing each vector field.Source: §E.4 · p. 24

E.5 TASK 4: HYDRAULIC–MECHANICAL COUPLING

Task 4 couples a hydraulic subsystem to a mechanical subsystem through a piston with unknown area. The experiment compares DINEs and Coupled Neural ODEs on composed-system trajectories without training on those trajectories.

E.5 TASK 4: HYDRAULIC–MECHANICAL COUPLING

  • The hydraulic subsystem combines a pressure source, fluid resistor, fluid inertia, and reservoir; the mechanical subsystem is a mass–spring–damper chain with one boundary port.The hydraulic states are qh and Q1, and the mechanical states are q1, q2, q3, v1, and v2.Source: §E.5 · pp. 24–25
  • In the composed system, boundary flow and velocity are unknowns, pressure source is the only known input, and piston coupling enforces Qb = Apvb and ApP + Fb = 0.The true piston area, Ap = 1.3, is not given to the models.Source: §E.5 · p. 25
  • DINEs treat the reservoir as linear with an unknown coefficient and the fluid resistor as nonlinear, calibrating the mechanical boundary force from four subsystem observations.The piston area is identified by closed-form least squares from four fixed states of the composed-system trajectories.Source: §E.5 · p. 25
  • Coupled Neural ODE boundary networks predict hydraulic pressure and mechanical force, and identify an unknown transformer ratio from the same composed-system states used by DINEs.Both methods integrate the composed system as a DAE with boundary variables unknown and estimate its initial state from observations.Source: §E.5 · pp. 25–26

E.6 TASK 5: REUSE OF THE LEARNED SPRING

Task 5 isolates a learned spring constitutive relation from a trained subsystem and reuses it in a different, specified mechanical system. The receiving-system setup limits predictions to the donor’s observed displacement range.

E.6 TASK 5: REUSE OF THE LEARNED SPRING

  • The isolated second-spring energy network is frozen and reused as a constitutive relation, with its force scale fixed from the learned interconnection.No other donor components or constitutive relations are transferred.Source: §E.6 · p. 26
  • The receiving system connects a main mass to a wall by a linear spring and damper, and attaches an auxiliary mass through the reused spring and another damper.All coefficients are fixed and initial positions and velocities are zero.Source: §E.6 · p. 26
  • Prediction stops if the reused spring’s displacement argument leaves the donor training range, and the ten test inputs were selected using reference responses within the donor’s first-to-99th-percentile displacement interval.No candidate was rejected during this selection.Source: §E.6 · p. 26
  • Reference trajectories use the true second-spring relation e⋆(q) = 0.3q + 0.1q^3 in place of the reused constitutive relation.Five inputs are retained at each of two nominal input levels.Source: §E.6 · p. 26

F ADDITIONAL RESULTS AND DISCUSSIONS

The additional results read learned interconnection matrices as physical constraints and use them to reconstruct circuit structure. For the reported further VPT thresholds, DINEs’ superiority over other models is consistent.

F ADDITIONAL RESULTS AND DISCUSSIONS

  • The model comparison used the same data and evaluation procedures despite differences in the prior knowledge provided to each model.Table 7 summarizes the models’ prior knowledge and assumptions.Source: F ADDITIONAL RESULTS AND DISCUSSIONS · p. 26, Table 7 · p. 27
  • One reported result averages only two seeds because three of five seeds did not converge.Source: F ADDITIONAL RESULTS AND DISCUSSIONS · p. 27
  • DINEs’ superiority over the other models is consistent across the further VPT thresholds reported for Tasks 1–5.The passage refers to Table 8 for the threshold results.Source: F ADDITIONAL RESULTS AND DISCUSSIONS · p. 27, Table 8 · p. 27
  • The learned flow constraints recover the three-way junction’s spring-to-mass connections, with the reduced matrix expressing the shared junction velocity.The learned constraints imply each spring connects the common junction to its own mass.Source: F ADDITIONAL RESULTS AND DISCUSSIONS · p. 28
  • The RC ladder’s learned flow constraints express Kirchhoff current laws, while its effort constraints express independent-loop voltage laws and common voltages.Together, the relations reconstruct the circuit diagram, including the three resistors whose interconnection roles were unknown.Source: F ADDITIONAL RESULTS AND DISCUSSIONS · p. 29
Loading 2610.02960v1…