PaperlayerView exact revision on arXiv

Source-linked AI summary

Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers

He Yin, Peter Seiler, Murat Arcak

arXiv:2006.07579v2eess.SY

TL;DR

The paper addresses the challenge of certifying stability and regions of attraction for feedback systems with neural-network controllers. It develops Lyapunov and quadratic-constraint methods for nominal and uncertain systems, using IQCs to model perturbations and activation slopes. The framework provides stability results and ellipsoidal ROA inner-approximations, illustrated on neural-network-controlled pendulum and vehicle systems.

  • Problem

    Neural-network controllers make formal stability and safety certification difficult because of their complex nonlinear architectures.

  • Method

    The paper combines Lyapunov theory with local sector QCs for LTI plants and IQCs for perturbations, including off-by-one IQCs for activation-function slope restrictions.

  • Results

    The two stability results prove local or asymptotic stability and compute ellipsoidal inner-approximations of the ROA using semidefinite programming.

  • Takeaways & Limitations

    The framework supports stability analysis for NN-controlled systems with nominal LTI dynamics and modeled uncertainties such as unmodeled dynamics, time delay, and slope-restricted nonlinearities.

Abstract

from arXiv · show

A method is presented to analyze the stability of feedback systems with neural network controllers. Two stability theorems are given to prove asymptotic stability and to compute an ellipsoidal inner-approximation to the region of attraction (ROA). The first theorem addresses linear time-invariant systems, and merges Lyapunov theory with local (sector) quadratic constraints to bound the nonlinear activation functions in the neural network. The second theorem allows the system to include perturbations such as unmodeled dynamics, slope-restricted nonlinearities, and time delay, using integral quadratic constraint (IQCs) to capture their input/output behavior. This in turn allows for off-by-one IQCs to refine the description of activation functions by capturing their slope restrictions. Both results rely on semidefinite programming to approximate the ROA. The method is illustrated on systems with neural networks trained to stabilize a nonlinear inverted pendulum as well as vehicle lateral dynamics with actuator uncertainty.

I. INTRODUCTION

The paper addresses the difficulty of obtaining formal stability and safety guarantees for neural-network-controlled feedback systems. It proposes two QC-based stability results covering nominal LTI plants and uncertain systems, with ellipsoidal ROA inner-approximations computed using semidefinite programming.

  • Motivation: Neural-network controllers complicate classical stability analysis because their architectures include varied activations, multiple layers, and many hidden neurons.Monte Carlo simulations lack the formal guarantees needed in safety-critical applications.
  • Main results: Theorem 1 combines Lyapunov theory with local sector quadratic constraints to analyze LTI plants and inner-approximate their regions of attraction.The method uses offset local sectors centered at equilibrium inputs, allowing analysis around non-zero equilibrium points.
  • Main results: Theorem 2 uses integral quadratic constraints to model perturbations including unmodeled dynamics, slope-restricted nonlinearities, and time delay.The IQC formulation supports uncertain plants that need not be LTI and provides robustness guarantees.
  • Evaluation: Both stability results rely on semidefinite programming to compute ellipsoidal inner-approximations of the region of attraction.Numerical examples include neural-network controllers for a nonlinear inverted pendulum and an uncertain vehicle model.
  • Main results: Off-by-one IQCs capture activation-function slope restrictions and sharpen the local description beyond the static QCs used in earlier related work.The framework uses these dynamic constraints to reduce conservatism in ROA analysis.

B. NN Representation: Isolation of Nonlinearities

The neural network is rewritten to separate its linear operations from its activation nonlinearities. Stacking these nonlinearities produces a combined map that can be bounded directly in the stability analysis.

  • Layer representation: Each activation input is formed by an affine layer operation vi(k) := Wiwi−1(k) + bi.The weights Wi and biases bi define the input to activation function φi.
  • Isolation of nonlinearities: The nonlinear operation of layer i is expressed as wi(k) = φi(vi(k)), isolating activation inputs and outputs from the network’s linear operations.The decomposition prepares the activation functions for quadratic-constraint analysis.
  • Combined nonlinearity: Inputs and outputs from all activation functions are gathered into stacked vectors with total dimension nφ := n1 + ··· + nℓ.The combined nonlinearity φ maps the stacked activation inputs to stacked outputs.
  • Network reconstruction: The scalar activation function is applied element-wise to the combined vector, while the network policy is rewritten using a matrix N determined by its weights and biases.The matrix partition aligns network inputs and outputs with x, wφ, 1, u, and vφ.
  • Equilibrium representation: Equilibrium values for the activation inputs and outputs are obtained by propagating the system equilibrium through each network layer.This yields equilibrium tuples (vφ, wφ) = (v∗, w∗) for the activation nonlinearities.

C. Quadratic Constraints: Scalar Activation Functions

Quadratic constraints bound scalar activation functions through global, local, and offset local sector descriptions. Local sectors tighten these bounds over restricted input intervals, while offset sectors support analysis around nonzero equilibria.

  • Global sector constraints: Sector constraints bound activation functions between lines with prescribed slopes, such as the global [0, 1] sector for tanh and ReLU.The global sector is centered at the origin.
  • Local sector constraints: Local sector constraints provide tighter activation bounds than global sectors when inputs are restricted to an interval.For tanh on [−¯ν, ¯ν], α = tanh(¯ν)/¯ν > 0 and β = 1.
  • Offset local sector constraints: Offset local sectors center the constraint around an arbitrary point (ν∗, ϕ(ν∗)) rather than the origin.This formulation supports stability analysis around nonzero equilibrium points and can tighten the upper slope bound.
  • Quadratic constraint form: The scalar sector condition is expressed as (∆ϕ(ν) − α∆ν) · (β∆ν − ∆ϕ(ν)) ≥ 0 over the specified input interval.Here ∆ϕ(ν) and ∆ν measure deviations from the offset point.

D. Quadratic Constraints: Combined Activation Functions

Scalar activation bounds are combined across neurons to form quadratic constraints for the neural network nonlinearity. Interval bounds are propagated through the network, and nonnegative multipliers aggregate the element-wise sector inequalities.

  • Combined activation functions: Element-wise scalar sectors are stacked into vectors αφ and βφ to describe the combined neural-network nonlinearity.The construction applies around the equilibrium activation input v∗ over the computed interval [v, ¯v].
  • Combined activation functions: Nonnegative multipliers λ aggregate the element-wise sector inequalities into a quadratic constraint for the combined activation function.Each summand is nonnegative when the activation input lies within its interval.
  • Interval propagation: Each activation input is bounded on an interval, and its scalar sector bounds are computed analytically or numerically.For tanh, interval bounds can be converted directly into output bounds.
  • Interval propagation: Bounds propagate layer by layer from activation inputs to outputs and then to the next activation inputs.The resulting intervals contain the corresponding equilibrium values.

E. Lyapunov Condition

The Lyapunov condition combines an ellipsoidal candidate set with activation-input bounds and sector constraints. Feasibility yields forward invariance and asymptotic stability, while interval selection creates a trade-off between sector tightness and ROA coverage.

  • Theorem 1: Theorem 1 establishes local stability and makes E(P, x∗) an inner approximation of the region of attraction when its conditions hold.The result applies to an LTI plant with a neural-network controller and an equilibrium point x∗.
  • Invariant ellipsoid: Feasibility of the interval constraints ensures that states in E(P, x∗) remain within the activation-input bounds required by the offset local sectors.This validates the sector conditions throughout the candidate ellipsoid.
  • Lyapunov decrease: The Lyapunov function V(x) = (x − x∗)⊤P(x − x∗) decreases strictly inside the invariant ellipsoid, proving asymptotic stability.The proof uses V(x(k + 1)) − V(x(k)) ≤ −ϵ∥x(k) − x∗∥2.
  • ROA approximation trade-off: Choosing narrower activation-input intervals sharpens local sectors but restricts where ROA inner approximations can lie; wider intervals do the reverse.The interval parameter can be searched over a grid to maximize the resulting inner approximation.
  • Extensions: The analysis can be extended to output-feedback controllers by modifying the neural-network input mapping.The paper also permits nonsymmetrical intervals with minor notational changes.

A. Problem Formulation

The paper formulates nominal and uncertain NN-controlled feedback systems and seeks asymptotic stability certificates with ellipsoidal inner approximations of the robust region of attraction. Uncertainty is modeled as an input-output perturbation constrained by IQCs over finite horizons.

  • A. Problem Formulation: The uncertain feedback system interconnects a nominal plant G, perturbation Δ, and NN controller π through plant state, control, and perturbation input-output signals.The perturbation is represented as a bounded causal operator without explicitly modeling its internal state.
  • A. Problem Formulation: The perturbation set S contains operators Δ satisfying the assumed equilibrium and zero-output conditions at the origin.The formulation assumes the equilibrium is at the origin and Δ(0)=0 for every admissible perturbation.
  • A. Problem Formulation: The objective is to prove asymptotic stability and compute the largest ellipsoidal inner approximation of the robust ROA.The robust ROA is defined for trajectories under perturbations in S.
  • B. Integral Quadratic Constraints: IQCs characterize the perturbation’s input-output behavior using a virtual LTI filter ΨΔ and a quadratic constraint on its output.The perturbation model covers saturation, time delay, unmodeled dynamics, and slope-restricted nonlinearities.
  • B. Integral Quadratic Constraints: The time-domain IQCs used in the Lyapunov analysis hold over every finite horizon N ≥ 0.These finite-horizon constraints are called hard IQCs; soft IQCs are associated with infinite-horizon formulations.
  • B. Integral Quadratic Constraints: The extended-system analysis replaces the exact perturbation relation with an IQC constraint on the filtered signal.The offset local sector QC is a special time-domain IQC that holds at each time step and therefore over finite sums.

C. Lyapunov Condition ψ

The uncertain-system theorem applies Lyapunov analysis to an extended state containing the plant and IQC-filter states. Its strict LMI condition yields local stability and an ellipsoidal inner approximation to the robust ROA.

  • C. Lyapunov Condition ψ: The extended state combines the nominal plant state with the IQC filter state, with dimension nζ = nG + nψ.The corresponding dynamics are used for the Lyapunov analysis.
  • C. Lyapunov Condition ψ: Theorem 2 states that feasibility of the IQC-based matrix conditions proves local stability for every perturbation satisfying the specified IQC.The theorem assumes NN sector bounds and an admissible IQC pair for the perturbation.
  • C. Lyapunov Condition ψ: The same theorem identifies the intersection of the extended ellipsoid with ψ = 0 as an inner approximation to the robust ROA.This intersection is represented by E(Px, x∗), where Px is the upper-left block of P.
  • C. Lyapunov Condition ψ: The proof uses V(ζ) := (ζ − ζ∗)⊤P(ζ − ζ∗) and combines the strict LMI with local-sector and IQC inequalities.Summing the resulting inequality over a finite horizon establishes the required Lyapunov decrease.
  • C. Lyapunov Condition ψ: If ζ(0) lies in E(P, ζ∗), the trajectory remains in that ellipsoid and converges to ζ∗.Because ψ(0)=0, membership of the extended ellipsoid is equivalent to x(0) belonging to E(Px, x∗).
  • C. Lyapunov Condition ψ: The ROA approximation is optimized by minimizing trace(Px) over P, multiplier variables, and admissible IQC matrices.The resulting optimization is convex in (P, λ, MΔ).

D. IQCs for Combined Activation Functions φ

The paper augments local sector bounds for NN activations with off-by-one IQCs that encode slope restrictions across consecutive time steps. This richer description can produce less conservative ROA inner approximations, at the cost of a larger Lyapunov matrix and longer computation.

  • D. IQCs for Combined Activation Functions φ: Local sector bounds alone do not capture the slope restrictions of activation functions.The paper therefore adds off-by-one IQCs to the existing Lyapunov-IQC framework.
  • D. IQCs for Combined Activation Functions φ: Activation-input bounds can be used to derive local slope bounds [mφ, Lφ] for each activation function.For tanh, the local slope interval depends on the restricted input interval.
  • D. IQCs for Combined Activation Functions φ: The off-by-one IQC relates activation values at consecutive time instances, such as φi(vφ,i(k)) and φi(vφ,i(k+1)).It is described as a special case of the Zames-Falb IQC.
  • D. IQCs for Combined Activation Functions φ: The combined analysis applies perturbation IQCs, offset local sectors, and φ ∈ IQC(Ψoff, Moff) to an extended system containing G, ΨΔ, and Ψoff.This incorporates both plant uncertainty and activation-function slope information.
  • D. IQCs for Combined Activation Functions φ: Adding Ψoff introduces nφ additional states, increasing the Lyapunov matrix size and computation time.The paper presents this as the computational trade-off for using off-by-one IQCs.

IV. EXAMPLES

The examples apply the framework to a reinforcement-learning-controlled nonlinear inverted pendulum with saturation and slope-restricted nonlinearities. The ROA approximation is visualized as an ellipsoid alongside admissible-state boundaries and closed-loop trajectories.

  • IV. EXAMPLES: The examples solve the optimization using MOSEK with CVX, and the implementation code is available online.These details identify the computational setup used for the numerical demonstrations.
  • IV. EXAMPLES: The inverted-pendulum example uses m = 0.15 kg, l = 0.5 m, friction coefficient µ = 0.5 Nms/rad, and input saturation umax = 0.7 Nm.The controller is trained through a policy-gradient reinforcement-learning process.
  • IV. EXAMPLES: The nonlinearity Δ(θ) = θ − sin(θ) is modeled with both slope-restricted and sector-bounded descriptions over a bounded angular interval.For θ̄ = −θ = 0.73, its slope and sector bounds are [0, 0.2548] and [0, 0.087].
  • IV. EXAMPLES: The pendulum analysis uses an off-by-one IQC for slope information and a local sector IQC for the nonlinearity’s local sector bound.The activation function is characterized only with a local sector IQC in this example.
  • IV. EXAMPLES: Figure 6 overlays the ROA inner-approximation with admissible-state boundaries and phase-portrait trajectories.The ROA is shown as a blue ellipsoid, while green and red curves represent trajectories inside and outside it.

B. Vehicle lateral control

The vehicle lateral-control analysis evaluates a neural-network controller under saturation and actuator uncertainty, then compares ROA inner-approximations obtained with and without off-by-one IQCs. Including the off-by-one IQC yields a larger certified ROA and greater tolerance to uncertainty.

  • Vehicle model and controller: The vehicle controller is a two-layer tanh neural network trained with discretized vehicle dynamics and curvature disturbances, followed by steering saturation.The network uses n1 = n2 = 32, sampling time dt = 0.02 s, and umax = π/6.
  • Vehicle model and controller: The robustness analysis considers zero curvature, a zero equilibrium, steering saturation, and norm-bounded LTI actuator uncertainty with ∥∆LTI∥∞ ≤ 0.1.The uncertain actuator contribution is q(k) = ∆LTI(usat(k)), so the actual input is upert(k) = usat(k) + q(k).
  • ROA comparison: The ROA-volume proxy det(Px^-1) increases from 3.2×10^5 to 1.1×10^6 when the off-by-one IQC is added.The larger value corresponds to the experiment combining local sector and off-by-one IQCs.
  • ROA comparison: The SDP remains feasible up to δv = 0.67 with only local sector IQCs and up to δv = 1.4 when off-by-one IQCs are also used.The off-by-one IQC therefore permits looser local sector bounds in this analysis.
  • ROA visualization: Figure 8 displays slices of the second experiment's ROA ellipsoid in the e–˙e and eθ–˙eθ spaces, alongside the admissible-input boundary and zero equilibrium.Blue ellipsoids show ROA slices, orange lines show the boundary of the polytopic input set, and brown crosses mark x∗.
Loading 2006.07579v2…