PaperlayerView exact revision on arXiv

Source-linked AI summary

Forward and Adjoint Sensitivity Computation of Chaotic Dynamical Systems

Qiqi Wang

arXiv:1202.5229v4physics.comp-phmath.DSnlin.CDphysics.flu-dyn

TL;DR

Sensitivity computation for statistical quantities in chaotic systems is difficult because the butterfly effect makes finite-time-average derivatives diverge. The paper develops forward and adjoint algorithms based on shadow-operator inversion, and reports accurate estimates from a single trajectory of length about 20.

  • Problem

    Sensitivity derivatives of statistical averages matter in applications such as turbulent-flow forces and global temperature, but the butterfly effect makes their computation in chaotic systems difficult.

    Source: §1 · p. 2

  • Method

    Forward and adjoint algorithms compute derivatives of statistical quantities under infinitesimal dynamical perturbations using shadow-operator inversion and a divide-and-conquer strategy.

    Source: §1 · p. 3, §8 · p. 21

  • Results

    A single trajectory of length about 20 lets both algorithms efficiently compute accurate estimates of all sensitivity derivatives.

    Source: §8 · p. 22

  • Takeaways & Limitations

    The algorithms yield accurate statistical-sensitivity estimates from a short trajectory despite the finite-time-average divergence described for chaotic systems.

    Source: §1 · p. 2, §8 · p. 22

    Adjoint estimates can differ from forward estimates because they treat time dilation differently, and solving the adjoint perturbation requires both initial and terminal conditions.

    Source: §6 · p. 13, §7.3 · p. 19

Abstract

from arXiv · show

This paper describes a forward algorithm and an adjoint algorithm for computing sensitivity derivatives in chaotic dynamical systems, such as the Lorenz attractor. The algorithms compute the derivative of long time averaged "statistical" quantities to infinitesimal perturbations of the system parameters. The algorithms are demonstrated on the Lorenz attractor. We show that sensitivity derivatives of statistical quantities can be accurately estimated using a single, short trajectory (over a time interval of 20) on the Lorenz attractor.

1 Introduction

The section motivates sensitivity analysis for long-time statistical averages in chaotic systems, where finite-time linearized sensitivities diverge and existing approaches can be costly. It introduces a shadow-operator method designed to estimate sensitivities from a single short trajectory.

1 Introduction

  • Sensitivity analysis computes output derivatives with respect to input parameters; forward methods suit many outputs and few inputs, while adjoint methods suit few outputs and many inputs.Source: §1 · p. 1
  • For ergodic systems, a statistical average is an infinite-time average that depends on the governing system rather than the particular trajectory.The section defines the average of J(x(t)) over an infinitely long interval.Source: §1 · p. 2
  • Sensitivities of statistical averages matter for quantities such as mean aerodynamic forces and global temperature.Source: §1 · p. 2
  • Finite-time sensitivities of chaotic statistical averages can diverge instead of converging to the desired derivative, while existing correct methods often require costly ensemble averaging.The butterfly effect causes the linearized forward and adjoint equations to grow exponentially.Source: §1 · p. 2
  • The method estimates statistical sensitivities using a single short trajectory by inverting the shadow operator and applying a divide-and-conquer strategy [2][11].The inversion idea draws on structural-stability proofs, while the strategy is inspired by numerical computation of Lyapunov covariant vectors [2][11].Source: §1 · pp. 2–3

2 The “Shadow Operator”

A shadow coordinate transform displaces each state along a smooth vector field, producing a nearby trajectory that satisfies perturbed dynamics. The shadow operator maps that displacement to the induced perturbation, which links the transform to statistical sensitivity.

2 The “Shadow Operator”

  • A shadow coordinate transform maps a trajectory to a nearby trajectory that remains uniformly close, as illustrated by the Lorenz trajectory pair.The figure marks the displacement εδx associated with an infinitesimal change in parameter r.Source: §2 · p. 3, Figure 1 · p. 4
  • The shadow operator S_f is a linear map from a coordinate displacement δx to the induced dynamics perturbation δf.In the distorted coordinates, trajectories of the original system satisfy the perturbed equation.Source: §2 · pp. 4–5
  • For an ergodic system, the statistical-sensitivity derivative is the average of ∂J/∂x · δx when δx satisfies δf = S_fδx.Computing this derivative requires numerically inverting S_f for a given δf.Source: §2 · pp. 5–6
  • The perturbed and original trajectories must start at specially aligned initial conditions for the infinitesimal-perturbation and long-time limits to commute.Other initial-condition pairs can diverge through the butterfly effect.Source: §2 · p. 5

3 Inverting the Shadow Operator

The inversion procedure decomposes perturbations along Lyapunov covariant vectors and solves for their scalar coefficients. This divide-and-conquer approach handles positive and negative exponents in opposite time directions, while the zero exponent requires a separate treatment.

3 Inverting the Shadow Operator

  • Parameter perturbations can be represented as perturbations to the dynamics, and the sensitivity formula requires finding δx by inverting S_f for a given δf.Source: §3 · p. 6
  • The Lyapunov spectrum decomposition enables a divide-and-conquer strategy for computing δx = S_f^-1δf by resolving perturbations along covariant-vector directions.The passage focuses on systems with distinct Lyapunov exponents.Source: §3 · p. 6
  • The decomposition reduces the inversion to scalar coefficient equations along each Lyapunov direction, with integration direction chosen by the exponent sign.For positive exponents, integration proceeds backward in time; for negative exponents, it proceeds forward, and the computed coefficient converges exponentially.Source: §3 · p. 7
  • The zero Lyapunov exponent prevents direct inversion of S_f, so its coefficient equation needs a separate treatment.The paper identifies the zero-exponent vector with the system vector field.Source: §3 · pp. 6–7

4 Time Dilation and Compression

The shadow operator is not invertible for perturbations violating a zero-mean condition in its neutral Lyapunov component; adding a global time-dilation constant places the perturbation in its range. The resulting shadow displacement yields the statistical sensitivity, with its neutral component’s mean adjusted for finite-time averaging.

4 Time Dilation and Compression

  • The zero-exponent component of any perturbation in the shadow operator’s range must have zero time mean, so some perturbations have no inverse displacement.The flow direction is the zero-exponent Lyapunov covariant vector; for example, δf ≡ f lies outside the range.Source: §4 · p. 8
  • Adding a global time-dilation constant η to δf can place the combined perturbation in the shadow operator’s range, enabling inversion for the displacement.The time-scaled system has the same statistical averages as the original, so this adjustment does not change the sensitivity to δf.Source: §4 · p. 8
  • After choosing η, the method inverts the shadow operator on ηf + δf and uses the resulting displacement to compute the sensitivity derivative.The passage describes η as making the perturbed field lie in the range before inversion.Source: §4 · pp. 8–9
  • For finite-time averaging, the arbitrary constant in the neutral displacement component is adjusted so that its mean is zero.The text says this controls error from finite-time averaging; the constant’s contribution vanishes in the infinite-time limit.Source: §4 · p. 9

5 The Forward Sensitivity Analysis Algorithms

The forward algorithm computes sensitivities of multiple statistical quantities to one perturbation by reusing a shared trajectory and Lyapunov decomposition. It decomposes the perturbation, computes the displacement, then averages the resulting sensitivity.

5 The Forward Sensitivity Analysis Algorithms

  • For one perturbation, the forward algorithm computes sensitivities of multiple statistical quantities by repeating only the final averaging step.Its preparation phase computes the trajectory and Lyapunov spectrum decomposition; this contrasts with the adjoint algorithm’s one-output, many-perturbation use.Source: §5 · p. 9
  • The buffer time must exceed the nonzero-exponent relaxation scales, while the averaging interval must exceed the dynamics’ decorrelation time.The trajectory is obtained over [−TB, TA + TB] after an initial condition on the attractor is obtained.Source: §5 · p. 10
  • The algorithm decomposes δf along the trajectory, computes η, solves for the displacement components, reconstructs δx, and averages to obtain the sensitivity.The component equations are integrated in different time directions according to the Lyapunov exponents, with the neutral component integrated separately.Source: §5 · p. 10
  • An alternative forward computation splits δx into stable, neutral, and unstable components and solves each separately, potentially avoiding all Lyapunov covariant vectors.Source: §5 · p. 10

6 The Adjoint Sensitivity Analysis Algorithm

The adjoint algorithm computes the sensitivity of one statistical quantity to perturbations by forming an adjoint vector field from the quantity’s state derivative. Once computed, that field can be reused across perturbations.

6 The Adjoint Sensitivity Analysis Algorithm

  • The primal and adjoint Lyapunov covariant vectors are conjugate: their matrices are transposed inverses at each state point.Source: §6 · p. 11
  • The adjoint field is assembled from scalar components obtained by decomposing the state derivative along the trajectory and solving component equations.For nonzero exponents, the integration direction is forward for positive exponents and backward for negative exponents; the neutral component is integrated and mean-adjusted.Source: §6 · p. 13
  • For one statistical quantity, the adjoint algorithm computes sensitivities to many perturbations by repeating only the final averaging step.Its preparation phase and trajectory/Lyapunov decomposition are shared; the algorithm then decomposes the quantity’s state derivative and computes the adjoint field.Source: §6 · p. 13
  • The adjoint field must satisfy both initial and terminal conditions, making conventional time integration difficult.An alternative splits it into stable, neutral, and unstable components and solves them in different time directions; this may help avoid computing all Lyapunov covariant vectors for large systems.Source: §6 · pp. 13–14

7 An Example: the Lorenz Attractor

The section demonstrates forward and adjoint sensitivity algorithms on the Lorenz attractor, using a single trajectory of length 20 for averaging and spin-up.

7.1 Lyapunov covariant vectors

  • The Lorenz example estimates λ1≈0.95 and λ3≈−14.6, while λ2=0; the first and third covectors are computed by forward and backward tangent integrations, respectively.The vanishing exponent's covector is proportional to the vector field and normalized to mean magnitude one.Source: §7.1 · p. 15

7.2 Forward Sensitivity Analysis

  • Forward analysis decomposes a parameter-r forcing in Lyapunov vectors and inverts the shadow operator for δx; windowed averaging improves accuracy, and estimates compare well with finite differences.In the Lorenz example, δx is interpreted as the shadow-coordinate perturbation induced by increasing r.Source: §7.2 · pp. 16–17

7.3 Adjoint Sensitivity Analysis

  • For J=x3, the adjoint method decomposes its gradient into adjoint Lyapunov vectors, combines their coefficients into an adjoint vector, and computes sensitivities to s, r, and b.The gradient is (0, 0, 1); the resulting adjoint vector is plotted over time and along the trajectory.Source: §7.3 · pp. 17–18
  • Adjoint and forward estimates can differ because the methods treat the time-dilation term differently, which may cause apparent discrepancies.The forward method estimates η, whereas the adjoint method sets the mean of its corresponding coefficient to zero.Source: §7.3 · p. 19

7.4 Comparison with the finite difference method

  • The finite-difference study used a total integration length of 1,000,000, compared with a length-20 trajectory for this method.The finite-difference comparison used 10 simulations; the paper reports that its method is much more efficient than ensemble-average approaches.Source: §7.4 · pp. 20–21
  • Across 200 random starts, both algorithms usually produce accurate sensitivities, though estimates can have heavy tails possibly linked to the unstable fixed point (0, 0, 0).The histograms compare estimates with finite-difference confidence intervals; the authors note that the tails could indicate unbounded second moments.Source: Figure 8 · p. 20, §7.4 · pp. 20–21, Figure 9 · p. 21

8 Conclusion

The paper presents forward and adjoint algorithms for sensitivity derivatives of statistical quantities in chaotic systems. On the Lorenz attractor, both produced accurate estimates from a single trajectory of about 20 time units.

8 Conclusion

  • The forward and adjoint algorithms compute sensitivity derivatives of statistical quantities to infinitesimal perturbations of chaotic-system dynamics.The forward algorithm starts from a perturbation and computes a shadow coordinate system; the adjoint algorithm starts from a statistical quantity and computes an adjoint vector.Source: §8 · p. 21
  • The algorithms were demonstrated on the Lorenz attractor at standard parameter values.The forward and adjoint methods were applied to different sensitivities: ∂⟨x2⟩/∂r and ∂⟨x3⟩/∂b, respectively.Source: §8 · pp. 21–22
  • A single trajectory of about 20 time units yielded accurate estimates of all sensitivity derivatives computed by both algorithms on the Lorenz attractor.The forward algorithm computed ∂⟨x2⟩/∂r, while the adjoint algorithm computed ∂⟨x3⟩/∂b.Source: §8 · p. 22
Loading 1202.5229v4…