Dynamic NE Seeking for Multi-Integrator Networked Agents with Disturbance Rejection

I. INTRODUCTION

• Prior work often treats agents without dynamics or with single-integrator dynamics, while disturbances are not explicitly considered; related noisy-gradient work treats stochastic noise instead.The paper distinguishes its deterministic-disturbance setting from the stochastic noise considered in and.Source: §I · p. 1
• The paper’s motivating robot example requires agents to maintain neighbor communication while rejecting disturbances such as wind or terrain slope.A disturbance-free related formulation in discretizes the state space and treats the game as finite-action, ignoring individual agent dynamics.Source: §I · p. 1
• The paper extends Nash equilibrium seeking to single- and double-integrator agents subject to external deterministic disturbances, and discusses multi-integrator extensions.The target equilibrium is an a priori unknown Nash equilibrium, which the dynamics must reach irrespective of disturbances.Source: §I · p. 2
• Each agent combines gradient-play and estimate consensus with a dynamic internal-model component that acts as a reduced-order disturbance observer.Agents estimate others’ decisions through local communication; coupling is introduced only through the communication graph.Source: §I · p. 2
• The convergence proofs combine input-to-state stability with disturbance-observer design under strong monotonicity of the pseudo-gradient and Lipschitz continuity of the extended pseudo-gradient.The paper notes that individual passivity cannot be used as in typical multi-agent agreement problems.Source: §I · p. 2

B. Graph Theory

• The communication network is modeled as a fixed, static, undirected graph whose vertices are players and whose edges are communication links.For a connected graph, the Laplacian is symmetric, has a simple zero eigenvalue, and has positive second-smallest eigenvalue λ2(L).Source: B. Graph Theory · p. 2

C. Game Theory

• Each player controls an action and minimizes its own cost, and a Nash equilibrium is an action profile where no player can unilaterally lower its cost.The joint action profile is formed from the players’ individual actions and each cost may depend on the overall profile.Source: C. Game Theory · p. 3
• The game assumes differentiable costs convex in each player’s own action, with a strongly monotone and Lipschitz-continuous pseudo-gradient; these conditions yield a unique Nash equilibrium.The pseudo-gradient stacks the players’ partial cost gradients.Source: C. Game Theory · p. 3

D. Full-Information Gradient Dynamics

• Under full information, each agent’s gradient-play dynamics use the other players’ decisions, and the interconnected system is globally asymptotically stable at the unique Nash equilibrium.The overall dynamics are expressed as dot(x) = -F(x).Source: D. Full-Information Gradient Dynamics · p. 3

E. Partial-Information Gradient Dynamics

• In partial-information play, agents evaluate gradients using estimates of other players’ actions, updated through local communication over a static graph.The extended pseudo-gradient is Lipschitz continuous under Assumption 4.Source: E. Partial-Information Gradient Dynamics · p. 3
• Under Assumptions 1–4, if μ(λ2(L) − θ) > θ^2, the unique Nash equilibrium x = x* is globally asymptotically stable for all networked interconnected Σi, (7).Source: E. Partial-Information Gradient Dynamics · p. 4

III. PROBLEM FORMULATION

• The problem is to stabilize the Nash equilibrium while rejecting additive deterministic disturbances affecting multi-integrator agents.The paper treats single- and double-integrator agents and indicates extensions to multi-integrator agents.Source: §III · p. 4
• Disturbances have known form but unknown parameters, and each agent knows its system matrices but not the disturbance’s initial condition or trajectory.The disturbance generator is assumed marginally stable and observable, with its state in a compact set.Source: §III · p. 4
• With partial information, agents use local neighbor communication and maintain estimates of the other players’ decisions.The learning dynamics combine gradient play, estimate consensus, and a dynamic internal-model component for disturbance rejection.Source: §III · p. 4

IV. NE SEEKING FOR SINGLE-INTEGRATOR AGENTS

• The internal-model component and consensus-based estimates are combined with gradient play to seek the NE despite disturbances.For single-integrator agents, the extra state ξ_i acts as an internal model, while the Laplacian-based estimate dynamics drive estimates toward consensus.Source: §IV · p. 4
• Under Assumptions 1–4 and µ(λ2(L) − θ) > θ2, the unique NE and all players’ estimates are globally asymptotically stable for all disturbances w ∈ W.The result applies to the interconnected learning dynamics with partial information over a graph.Source: §IV · p. 4
• The single-integrator proof puts the closed loop in cascade form and uses ISS properties tied to strong monotonicity and Lipschitz continuity.The disturbance-related subsystem is asymptotically stable, supporting convergence of the NE subsystem.Source: §IV · pp. 4–5
• In the full-information case, the convergence result holds without Assumptions 3 and 4.The learning dynamics then do not require agents’ estimates of others’ decisions.Source: §IV · p. 5

V. NE SEEKING FOR DOUBLE INTEGRATORS

• The feedback evaluates gradients at predicted states, while agents share estimates of predicted actions with their neighbors.Each agent predicts x_i + b_i v_i and uses the estimates in evaluating its gradient.Source: §V · p. 6
• For double integrators, gradient feedback at x_i + b_i v_i and negative velocity feedback shape learning around the agents’ inherent dynamics.The section motivates this design for agents whose actions cannot be updated directly because of their dynamics.Source: §V · pp. 5–6
• Under μ(λ2(L)−θ) > θ2, the unique Nash equilibrium and each player’s estimates are globally asymptotically stable for all modeled disturbances.The result applies to the partial-information learning dynamics under Assumptions 1–4.Source: §V · p. 6
• The proof uses a cascade argument: the estimate subsystem is ISS with respect to the disturbance-related input, supporting stability of the full system.The velocity subsystem is also shown to be ISS with input (γ̃, ρ).Source: §V · pp. 6–7
• In the disturbance-free case, the convergence results do not require Assumptions 3 and 4.Source: §V · p. 7
• The design extends to r_i-th order integrators, with a reduced-order disturbance observer and consensus dynamics for estimates; the resulting scheme has a global convergence guarantee.For r_i = 2, the feedback reduces to the double-integrator design in (28).Source: §V · p. 7

VI. NUMERICAL EXAMPLES

• The evaluation covers an optical-network OSNR game and a sensor-network game.Source: §VI · p. 7
• The algorithms are compared with both full- and partial-information gradient-play.Source: §VI · p. 7
• Both application comparisons are conducted in the presence of disturbances.Source: §VI · p. 7

A. OSNR Game

• The example treats ten WDM channels as agents that adjust transmission power to maximize their channel OSNR.The setup specifies a cost function and pilot-tone disturbances with channel-specific frequencies and unknown modulation.Source: A. OSNR Game · pp. 7–8
• With full information about others’ actions, dynamics (23) reject sustained OSNR fluctuations and converge to the NE found in, whereas standard gradient-play (6) does not reject them.The comparison is shown in Figs. 3 and 4.Source: A. OSNR Game · p. 8
• For partial information over random graph Gc, dynamics (16) and Laplacian-based dynamics (7) are compared in Figs. 7 and 6, respectively.The passage says this comparison is similar to the full-information comparison but does not state its outcome.Source: A. OSNR Game · p. 8, Figure 7 · p. 8, Figure 6 · p. 8

B. Sensor Networks

• The example considers five mobile robots with position-dependent costs, two actuation types, and both full- and partial-information settings over a random graph.The preceding comparison to uses continuous-time gradient-play-inspired feedback rather than a discrete-time extremum-seeking algorithm, and distinguishes deterministic disturbance rejection from noisy feedback.Source: B. Sensor Networks · p. 8, Figure 8 · p. 8

1) Velocity-Actuated Robots:

• The velocity-actuated robot example uses five robots and a constant disturbance d_i = col(0.5, 0).The setup considers both full information and partial information communicated over a graph.Source: 1) Velocity-Actuated Robots · p. 8
• With disturbances, dynamics (23) converge to the same Nash-equilibrium values as disturbance-free gradient-play (6), whereas standard gradient-play (6) does not.Fig. 9 compares dynamics (23) and (6) for single-integrator agents.Source: 1) Velocity-Actuated Robots · p. 9, Figure 9 · p. 9
• With information communicated over graph Gc, dynamics (16) converge in the presence of disturbances to the same Nash-equilibrium values as the full-information case, whereas Laplacian-based dynamics (7) do not.Fig. 10 compares (16) and (7) for single-integrator agents.Source: 1) Velocity-Actuated Robots · p. 9, Figure 10 · p. 9

2) Force-Actuated Robots:

• For double-integrator agents under d_i = col(0.5, 0), Figs. 11 and 12 show full-information dynamics (34) and partial-information dynamics (28), respectively, against the disturbance-free algorithm.The passage identifies the plotted methods but does not state their comparative outcome.Source: 2) Force-Actuated Robots · p. 9, Figure 11 · p. 9, Figure 12 · p. 9
• Although noisy feedback was not specifically investigated, bounded feedback noise leaves the steady-state solution within a neighborhood of the Nash equilibrium for dynamics (16) and (28).This robustness statement follows from their ISS properties.Source: 2) Force-Actuated Robots · p. 9

• The proposed schemes converge to the Nash equilibrium irrespective of disturbance for (multi)-integrator agents.Source: §VII · p. 9
• The schemes address both full information and partial-decision information based on local observation and communication.Source: §VII · p. 9
• The agent dynamics combine gradient play with a dynamic internal-model component, adding consensus dynamics under partial information.The consensus component drives decision estimates toward the consensus subspace.Source: §VII · p. 9