PaperlayerView exact revision on arXiv

Source-linked AI summary

Transfer Meta-Learning for State Space Autoencoders

Stefano De Carli, Daniele Masti, Davide Previtali, Mirko Mazzoleni

arXiv:2610.31318v1eess.SYcs.SY

TL;DR

System identification often requires costly experiments and separate model fitting, while in-context methods can predict zero-shot without necessarily providing compact state-space models. Transfer-Meta SSAEs use a fixed pretrained encoder and affine parameterization to instantiate compact models from context; on two held-out system families, they achieve higher median simulation FIT than individually fitted baselines.

  • Problem

    In-context system identification can provide zero-shot prediction but often lacks compact explicit state-space models, while transferring pretrained representations across system classes remains underexplored.

    Source: §1 · pp. 1–2

  • Method

    Transfer-Meta SSAEs combine a fixed pretrained temporal encoder, a context compressor, and affine parameter maps to instantiate system-specific state-space models without target-specific optimization.

    Source: §1 · p. 2, §3 · p. 2, §3.2 · p. 3

  • Results

    The resulting models achieve higher median simulation FIT than individually fitted baselines on both target families while exposing an explicit state-space model.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The approach transfers useful temporal representations across distinct system classes without updating the pretrained encoder and produces explicit state-space models.

    Source: §5 · p. 8

    Further work is needed to assess other encoders and compressor architectures, heterogeneous system families, and state-estimation and model-predictive-control applications.

    Source: §5 · p. 8

Abstract

from arXiv · show

Nonlinear system identification often requires costly experimental campaigns and separate model fitting for each different instance (target system) of a given system family. To reduce this burden, meta-learning approaches reuse information acquired across related systems. In particular, in-context meta-learning enables zero-shot prediction from a brief target system context (i.e., a short input-output trajectory), but generally yields large black-box sequence predictors without exposing an explicit state space model. Conversely, compact structured state space models are suitable for simulation, estimation, and control, but are usually fitted or adapted separately for each target system. To bridge this gap, we introduce a transfer meta-learning extension of state space autoencoders. The method reuses a pretrained encoder, whose weights remain fixed, to extract a compact representation of the target system from its context. An affine parameterization maps this representation to the parameters of a compact neural state space model without target-specific adaptation; the resulting model contains orders of magnitude fewer parameters than the reference in-context predictors from the literature. We evaluate zero-shot prediction and simulation performance on held-out system instances from Buck-Boost converter and Bouc-Wen oscillator system families, comparing performance against individually fitted baseline models.

1. INTRODUCTION

System identification often requires separate costly experiments and model fitting, while existing in-context methods can predict zero-shot but may lack compact state-space structure. The paper introduces Transfer-Meta SSAEs to transfer pretrained representations into compact models without target-specific optimization.

1. INTRODUCTION

  • Conventional identification fits a dedicated model to each system, making informative experiments costly and time-consuming across large populations of similar devices.Few-shot methods instead reuse knowledge from related systems; meta-learning optimizes a shared model and target-adaptation mechanism.Source: §1 · p. 1
  • In-context predictors can be large black-box sequence models that do not expose compact state dynamics for estimation and control, and transferring representations across system classes remains largely unexplored.The introduction frames these gaps as the motivation for its two research questions.Source: §1 · pp. 1–2
  • Transfer-Meta SSAEs map a brief measured input–output trajectory to model parameters using a fixed pretrained encoder, enabling zero-shot prediction and simulation without target-specific adaptation.The evaluation covers held-out Buck–Boost converter and Bouc–Wen oscillator instances against individually fitted SSAEs and a nonlinear input–output baseline.Source: §1 · p. 2

2. PROBLEM STATEMENT

The problem is to infer a compact state-space model from a finite context trajectory for an unseen member of a related nonlinear-system family. The inferred model should predict and simulate independent trajectories of that same system.

2. PROBLEM STATEMENT

  • The target is a family of discrete-time nonlinear systems whose members share state, input, and output dimensions but have instance-dependent dynamics.The dynamics depend on a time-invariant parameter vector, while the system maps and instance parameters are unknown or unavailable.Source: §2 · p. 2
  • For each target instance, the available context is a finite input–output trajectory.The objective is to infer model parameters from that context and use the compact state-space model on independent trajectories of the same system.Source: §2 · p. 2

3. TRANSFER-META STATE SPACE AUTOENCODERS

Transfer-Meta SSAEs infer system-specific state-space parameters from context while retaining the SSAE maps for prediction and simulation. The method keeps a pretrained temporal encoder fixed and trains the context compressor and parameter maps across related systems.

3.3 Zero-Shot Model Instantiation and Use

  • The model is instantiated from a target context without target-specific optimization, then used with fixed parameters for prediction and simulation.For simulation, the state is initialized from measured history and propagated under future inputs; for one-step prediction, it is reconstructed from measured history at each step.Source: §3 · p. 2, §3.3 · pp. 3–4

3.1 State Space Model

  • The SSAE represents latent state reconstruction, state propagation under input, and output reconstruction with separate encoder, transition, and decoder maps.The latent dimension need not equal the physical state dimension; the decoder reconstructs an output vector whose final output-length entries estimate the current output.Source: §3.1 · p. 2

3.2 Transfer-Meta Parameterization

  • A frozen pretrained temporal encoder and trainable context compressor produce a system embedding, which affine maps use to generate parameters for the SSAE maps.The compressor aggregates representations to capture dynamical variability across target-family instances; shared base parameters and variation directions are learned during meta-training.Source: §3.2 · p. 3

3.4 Meta-Training Problem and Learning Procedure

  • Meta-training jointly learns the compressor and affine parameter maps while keeping the pretrained encoder fixed and separating context from query trajectories.The query trajectories support generalization evaluation; the dynamic loss is evaluated over query windows with a specified simulation horizon.Source: §3.4 · p. 4
  • Embedding regularization promotes consistency across two context views of the same system while preventing collapse across systems.The unit target standard deviation fixes embedding-coordinate scale, with the affine parameter maps absorbing parameter variation.Source: §3.4 · p. 4

4. NUMERICAL RESULTS

The experiments test Transfer-Meta SSAE on held-out Buck–Boost and Bouc–Wen systems, comparing zero-shot prediction and simulation with individually fitted baselines. Transfer-Meta SSAE leads in median simulation FIT, while its compact instantiated models use far fewer parameters than reference in-context sequence predictors.

4.1 Benchmark Families

  • The two independently studied benchmark families are Buck–Boost converters and Bouc–Wen oscillators, with sampled physical parameters and multisine inputs.Buck–Boost instances vary five parameters; Bouc–Wen instances vary seven of eight, keeping hysteresis exponent ν fixed.Source: §4.1 · p. 5

4.2 Population Split and Data Generation

  • Each family uses disjoint populations of 8,192 training, 1,024 validation, and 1,024 test systems, with independent context and query trajectories.Context and training-query outputs contain Gaussian noise at approximately 26 dB SNR; test-query outputs are noise-free.Source: §4.2 · p. 5, Data Preprocessing · p. 5

4.3 Model Configuration and Training

  • The same compact SSAE configuration is used for both families, with 5,556 parameters per instantiated model and a frozen pretrained encoder.The encoder weights remain fixed during meta-training and target-system inference; the context compressor uses attention pooling.Source: §4.3 · pp. 5–6

4.4 Baseline Methods

  • The individually fitted SSAE shares the instantiated model's architecture, while NARX uses matching input and output history lengths and ridge-fitted output weights.Both baselines are fitted separately for each test system using its noisy training query.Source: §4.4 · p. 6

4.5 Evaluation Protocol and Metrics

  • All methods are evaluated on the same independent, noise-free test query; prediction reconstructs latent states from measured histories, whereas simulation propagates states without later output feedback.The first 20 test samples initialize history and are excluded from metrics.Source: §4.5 · p. 6

4.6 Performance Comparison

  • 84.4% (Buck–Boost) and 88.5% (Bouc–Wen) are Transfer-Meta SSAE's median simulation FITs, above NARX (27.7%, 78.7%) and individual SSAE (−2.7%, 47.6%).These are aggregate FIT statistics on held-out test systems.Source: Table 2 · p. 6, §4.6 · pp. 6–7
  • NARX prediction FIT is comparable to Transfer-Meta SSAE, but its simulation FIT drops with recursive feedback; 191 Buck–Boost NARX simulations diverged, leaving 833 finite cases for simulation statistics.Transfer-Meta SSAE's worst-case simulation FITs were 52.3% for Buck–Boost and 24.0% for Bouc–Wen; low FITs tended to occur in specified highly damped or strongly dissipative parameter regimes.Source: §4.5 · p. 6, §4.6 · p. 7

4.7 Computational Considerations

  • Each instantiated SSAE has 5,556 parameters, about three orders of magnitude fewer than the cited in-context sequence models; its context-to-parameter mapping runs once at instantiation, not at runtime.Meta-training took approximately 45 minutes per benchmark family, versus cited training times of 1.0–3.8 days and over 5 days for the reference predictors.Source: §4.7 · p. 7

5. CONCLUSION

Transfer-Meta SSAEs use a frozen pretrained temporal encoder and affine parameter maps to instantiate nonlinear state space models from input-output contexts. On both target families, they achieve higher median simulation FIT than individually fitted baselines while exposing an explicit state space model.

  • Transfer-Meta SSAEs combine a frozen pretrained temporal encoder, a context compressor, and affine parameter maps to instantiate nonlinear state space models from input-output context trajectories.Source: §5 · p. 8
  • The pretrained temporal encoder extracts useful representations across distinct system classes without parameter updates, answering RQ1 affirmatively.Source: §5 · p. 8
  • The resulting models achieve higher median simulation FIT than individually fitted baselines on both target families while exposing an explicit state space model.Source: §5 · p. 8
  • Future work will test alternative temporal encoders and compressors, ablate the affine parameterization and context length, extend to heterogeneous families, and evaluate state estimation and model predictive control.Source: §5 · p. 8

Appendix A. BENCHMARK MODELS

The appendix reports the continuous-time benchmark equations and points to schematics of the benchmarks.

  • The continuous-time benchmark equations are reported, with Fig. A.1 showing the corresponding schematics.Source: Appendix A · p. 8

A.1 Buck–Boost Converter

The Buck–Boost appendix subsection introduces the continuous-time state vector and dynamics using the duty-cycle input.

  • The subsection presents the continuous-time state vector and dynamics driven by the duty-cycle input.Source: §A.1 · p. 8
  • It defines time-varying effective resistance terms Rt and Reff(t).Source: §A.1 · p. 8

A.2 Bouc–Wen Oscillator

The Bouc–Wen appendix subsection gives a three-component continuous-time state vector and introduces motion equations driven by applied force.

  • The state vector is s(t) = [p(t), v(t), z(t)]^T in R^3, and the continuous-time equations of motion use applied-force input u(t).Source: §A.2 · p. 8

Related papers

Loading 2610.31318v1…