PaperlayerView exact revision on arXiv

Source-linked AI summary

Information-theoretic receding-horizon active learning of nonlinear dynamical systems

Juncal Arbelaiz, Anushri Arora, Jonathan W. Pillow

arXiv:2609.36712v1eess.SYcs.LGcs.SY

TL;DR

Learning nonlinear controlled dynamics from a finite experiment requires informative data, but the target here is the map over a prescribed state–input region from a single trajectory. The paper uses prediction-oriented Bayesian active learning with non-myopic receding-horizon input design; in a noisy multistable-system experiment, it reports lower predictive uncertainty and reconstruction error than common excitation baselines under comparable constraints.

  • Problem

    The paper addresses how to efficiently learn stochastic nonlinear controlled dynamics from a finite, single observed trajectory, reconstructing the state-increment map over a prescribed compact state–input region.

    Source: §III · pp. 2–3

  • Method

    It combines a fixed nonlinear feature model with Gaussian Bayesian updates and prediction-oriented information-seeking input design using non-myopic receding-horizon planning.

    Source: §I · p. 2, §IV · p. 3, B. Optimal adaptive design &… · p. 4

  • Results

    The proposed strategy reduces predictive uncertainty and reconstruction error more efficiently than common excitation baselines under comparable experimental constraints.

    Source: §I · p. 2

  • Takeaways & Limitations

    The acquisition objective targets information about predictions across the prescribed region, rather than parameter uncertainty alone.

    Source: §IV · p. 3

Abstract

from arXiv · show

Accurately learning nonlinear dynamics from a finite-duration experiment requires the efficient collection of informative data. We address this challenge for stochastic controlled nonlinear dynamical systems whose state is observed along a single trajectory. Our goal is to reconstruct the unknown controlled state-increment map over a prescribed compact subset of state-input space. We construct a parametric estimator of the map using fixed nonlinear features, so that the model is nonlinear in the state and input, but linear in the unknown parameters. A Gaussian prior over the parameters yields recursive Bayesian posterior updates as data stream in, enabling online quantification of predictive uncertainty in the reconstructed dynamics over the target set. We formulate an optimal adaptive-design problem over an information state, using a prediction-oriented acquisition criterion based on the mean marginal mutual information between candidate future trajectories and the reconstructed dynamics over the target set. We then approximate the resulting adaptive-design problem by a non-myopic receding-horizon formulation, evaluate its remaining expectation using a scenario-based sample average, and solve the resulting deterministic program with the cross-entropy method, leveraging parallel candidate-scenario evaluations. Numerical experiments on a noisy multistable system demonstrate that the proposed adaptive information-seeking strategy reduces predictive uncertainty and reconstruction error more efficiently than common excitation baselines under comparable experimental constraints.

I. INTRODUCTION

The paper frames online nonlinear-system identification as active learning: inputs can be designed to improve a model while data arrive along one trajectory. Its prediction-oriented framework selects inputs for information about reconstructed dynamics over a target region, rather than about parameters alone.

I. INTRODUCTION

  • Because nonlinear dynamics may be unavailable or vary across operating conditions, the paper studies sequential learning from a single state trajectory when external excitation permits adaptive input design.The setting couples inference from streaming observations with decisions about future inputs.Source: §I · p. 1
  • The acquisition objective measures informativeness about reconstructed dynamics over a prescribed state–input region, distinguishing it from exploration aimed at downstream control performance.The paper connects this prediction-oriented objective to goal-oriented Bayesian experimental design and expected predictive information gain.Source: §I · p. 1
  • Unlike policy-based Bayesian design methods that approximate the adaptive policy directly, the approach solves a constrained non-myopic receding-horizon input-design problem online and replans after each observation.The passage names DAD as an example of direct policy approximation.Source: §I · p. 2
  • The framework combines target-region predictive information gain with sequential Bayesian learning and constrained, non-myopic receding-horizon input design.The introduction reports that the resulting strategy reduces predictive uncertainty and reconstruction error more efficiently than power-matched excitation baselines.Source: §I · p. 2

II. MATHEMATICAL PRELIMINARIES

  • Mutual information measures the expected reduction in entropy after an observation, providing the information-theoretic interpretation used by the paper’s acquisition objective.The passages describe posterior uncertainty as reduced entropy after observing data.Source: §II · p. 2

III. MODEL OF THE SYSTEM DYNAMICS

The learning target is the unknown controlled one-step state-increment map restricted to a prescribed compact region, inferred from a finite, single controlled trajectory. A fixed nonlinear feature representation makes the model linear in its unknown weights.

III. MODEL OF THE SYSTEM DYNAMICS

  • The target is the restriction of the controlled state-increment map to a compact region of state–input space, learned from one finite controlled trajectory.The stated objective concerns the map on the region, not merely the collected trajectory.Source: §III · pp. 2–3
  • The estimator uses a fixed nonlinear feature map and unknown output weights, yielding a model linear in the weights.The paper states a realizability assumption: the map is representable by those features over state–input pairs used in learning and planning, including the target region.Source: §III · p. 2
  • The learning objective is reconstruction over the target region rather than fitting only observed data, with a finite evaluation grid available when computation requires discretization.The passage describes maintaining a weight posterior that induces a posterior predictive distribution over the restricted map.Source: §III · p. 3

IV. BAYESIAN LEARNING

A Gaussian posterior over model weights supports recursive online updates and predictive uncertainty estimates over the target region. The learning objective concerns predictions of the dynamics, since uncertainty in weights need not track uncertainty in those predictions.

IV. BAYESIAN LEARNING

  • The Gaussian weight posterior is updated as observations arrive, with the current mean and covariance serving as fixed-dimensional sufficient statistics for the collected data.The framework uses recursive updates online and batch updates for hypothetical multi-step observations during planning.Source: §IV · p. 3
  • The posterior over weights induces a Gaussian predictive distribution at each target location, whose covariance quantifies local epistemic uncertainty in the reconstructed map.The prior is Gaussian and its parameters encode prior knowledge and regularize estimation in the small-data regime.Source: §IV · p. 3
  • The objective is information about predictions over the target region, because reducing weight uncertainty need not comparably reduce uncertainty in the reconstructed dynamics.Different parameter directions can have different predictive relevance on the target region.Source: §IV · p. 3

V. INFORMATION-SEEKING PLANNING

The planning component selects inputs intended to produce observations informative about the controlled dynamics across the target region. It defines a prediction-oriented acquisition criterion, formulates causal adaptive design, and uses receding-horizon and sample-average approximations.

V. INFORMATION-SEEKING PLANNING

  • The planner designs control inputs to steer the system toward observations informative about the controlled state-increment map over the target region.The passage describes this as the objective of the planning component.Source: §V · p. 3
  • The planning formulation first defines a prediction-oriented information criterion as its acquisition function.The criterion is used to assess candidate inputs by the information they provide about the target quantity.Source: §V · p. 3
  • The paper formulates adaptive design over causal feedback policies and introduces receding-horizon and sample-average approximations for implementation.These are presented as the planning formulation and its implementation approximations.Source: §V · p. 3

A. Prediction-oriented information objective

The prediction-oriented criterion measures how much a candidate experiment is expected to reduce uncertainty in the reconstructed dynamics over the target region. It averages marginal mutual information across that region, with a grid approximation for numerical evaluation.

A. Prediction-oriented information objective

B. Optimal adaptive design & receding-horizon planning

The adaptive-design problem seeks causal feedback policies that maximize information about the reconstructed dynamics over the remaining experiment. Because exact dynamic programming is generally impractical, the paper replans from each updated information state using a finite open-loop horizon and applies only the first optimized input.

B. Optimal adaptive design & receding-horizon planning

  • The information state includes the physical state and posterior belief, which summarizes information from previous observations and yields a controlled-Markov process.Source: B. Optimal adaptive design &… · p. 4
  • The remaining-horizon objective chooses a causal feedback policy to maximize information about the reconstructed dynamics, with inputs depending only on information available so far.Source: B. Optimal adaptive design &… · p. 4
  • Exact dynamic programming is generally impractical because the information state is continuous and high-dimensional, dynamics are nonlinear and stochastic, and inputs are constrained.Source: B. Optimal adaptive design &… · p. 4
  • At each replanning step, a finite open-loop sequence approximates the remaining policy; only its first input is applied before updating the posterior and solving again.A horizon longer than one can steer toward informative regions that greedy planning may not reach, while limiting online computational cost.Source: B. Optimal adaptive design &… · p. 4

C. Sample-average approximation

The nonlinear stochastic planning problem is approximated by a deterministic sample-average problem. Parameter and process-noise samples form rollout scenarios, while the analytic information criterion means Monte Carlo is needed only for the outer expectation over future trajectories.

C. Sample-average approximation

  • The remaining expectation in the nonlinear stochastic program is approximated with a scenario-based sample-average approximation.Source: C. Sample-average approximation · p. 4
  • At each replanning time, parameter samples from the current Gaussian posterior are paired with independently sampled process-noise sequences to form rollout scenarios.The Cartesian product yields NwNξ scenarios.Source: C. Sample-average approximation · p. 4
  • The empirical average over sampled trajectories turns the acquisition evaluation, conditioned on the current information state, into a deterministic SAA problem.Source: C. Sample-average approximation · p. 5
  • The Gaussian model provides predictive information gain analytically through a log-determinant, so Monte Carlo is used only for the outer expectation over uncertain future trajectories.Source: C. Sample-average approximation · p. 5

VI. ALGORITHM FOR RECEDING-HORIZON BAYESIAN ACTIVE LEARNING

The algorithm repeatedly solves a deterministic SAA planning problem with the cross-entropy method, applies the first input, updates the posterior, and replans. A shared scenario batch scores candidates, whose rollouts can be evaluated in parallel.

VI. ALGORITHM FOR RECEDING-HORIZON BAYESIAN ACTIVE LEARNING

  • At each replanning time, one common batch of parameter–noise scenarios is reused to score all candidate input sequences throughout the CEM iterations.Source: §VI · p. 5
  • Candidate–scenario rollouts can be evaluated in parallel across CPU threads or GPU cores.Source: §VI · p. 5
  • CEM samples input sequences, retains the lowest-cost elite set, and refits its Gaussian sampling distribution.Source: §VI · p. 5
  • After planning, the sampling distribution is shifted forward to warm-start the next solve; only the first input is applied before posterior updating and replanning.Source: §VI · p. 5

VII. NUMERICAL CASE STUDY: ACTIVE LEARNING OF A NOISY BISTABLE SYSTEM

The case study evaluates online learning of a noisy, underactuated bistable system from a single trajectory, comparing adaptive input design with passive and power-matched excitation baselines. The adaptive controller reduces posterior uncertainty and improves reconstruction, with pink noise the strongest open-loop competitor.

b) Numerical results

  • The learner reconstructs the controlled flow field jointly over state and input in an underactuated system whose single actuator’s effectiveness varies with state.The autonomous drift and state-dependent input map are not supplied to the learner; metastable trajectories can otherwise remain within one basin and leave other regions poorly explored.Source: §VII · p. 6

a) Numerical setup

  • Experiments use 250 samples over 25 s, a 17×17×11 target grid, and planning horizons ℓ∈{2, 5, 10, 15}, with 20 stochastic realizations.The estimator uses 75 fixed random tanh features; each replanning step uses CEM with 200 candidate sequences, 25 elites, and 7 iterations.Source: §VII · p. 6

b) Numerical results

  • The comparison includes passive excitation and white-noise, pink-noise, and sinusoidal inputs, with driven baselines matched to the active run’s empirical input power and subject to the same input bound.The learning metrics are average posterior differential entropy over the target grid and offline relative RMSE of the controlled flow field.Source: §VII · p. 6
  • Relative RMSE is evaluated over the target grid, with a small constant preventing singular normalization near zero-flow locations.The reported qualitative RMSE trend accompanies the posterior-entropy reductions.Source: §VII · p. 6
  • The adaptive controller substantially outperforms spectrally white excitation; pink noise is the strongest open-loop competitor, and entropy reductions accompany improved flow-field reconstruction.Pink noise’s long temporal correlations can induce sustained forcing and excursions across the bistable landscape, whereas the controller adapts online to posterior uncertainty and observed state.Source: §VII · p. 6

VIII. CONCLUSION

The paper develops prediction-oriented Bayesian active learning for identifying stochastic nonlinear dynamics over a target state–input region. It uses non-myopic receding-horizon design and reports reduced predictive uncertainty and reconstruction error in a noisy bistable-system experiment.

VIII. CONCLUSION

  • The framework measures informativeness through mean marginal mutual information about the reconstructed dynamics over a prescribed state–input region.For models linear in unknown parameters, predictive information gain can be evaluated analytically.Source: §VIII · p. 6
  • Adaptive input design is formulated as a sequential decision problem over the physical state and Bayesian posterior, then approximated with non-myopic receding-horizon planning.A scenario-based sample average approximates the remaining expectation over future trajectories, and CEM optimizes the resulting program.Source: §VIII · p. 6
  • Numerical experiments on a noisy bistable system show that the closed-loop strategy reduces predictive uncertainty and reconstruction error over the target region.Source: §VIII · p. 6

APPENDIX

The appendix describes fixed random-feature construction over the state–input region and derives batch and recursive Gaussian posterior updates for the unknown parameters.

APPENDIX

  • Feature directions are normalized, scaled relative to the state–input region's characteristic width, and centered so their zero-level hyperplanes pass through sampled points in the region.The feature parameters are held fixed during learning; the experiments use ρ = 1.Source: Appendix A · p. 7
  • The construction is intended to reduce variability in feature norms and avoid widespread feature saturation across the region.The passage attributes these aims to normalizing directions and distributing feature transition regions throughout the region.Source: Appendix A · p. 7
  • The posterior over parameters remains Gaussian after a batch of future state observations, and a one-step update follows by applying the matrix inversion lemma.The derivation uses conditionally independent disturbances and Gaussian likelihood factors for state increments.Source: Appendix B · p. 7
Loading 2609.36712v1…