Source-linked AI summary
Program-Level Adversarial Co-Adaptive Self-Play for Robotic Security
Leo Benaharon, Chi Zhang, Wei Zheng, Weifan Zhang, Yifan Wei
TL;DR
Robotic security must address attacks that affect physical behavior, and evidence is needed on whether agents can revise defenses as opponents adapt. This paper uses program-level adversarial self-play with tested opponent code and mission feedback. Two eight-generation studies show local recovery and defensive revisions alongside recurrent failures, rather than establishing sustained robustness.
Problem
Can agents attack an existing robotic stack and revise defenses effectively as opponents adapt, in a security setting where software access can affect physical behavior?
Source: §1 · p. 1Method
The framework has coding agents develop attack and defense programs around a robotic system, then uses tested opponent code and mission outcomes to guide revisions without model fine-tuning.
Source: §1 · p. 1Results
Takeaways & Limitations
The studies make program adaptation and its limitations observable through executable artifacts and task-level evidence.
Source: §6 · p. 7The studies do not isolate which feedback mechanisms most improve agent learning across generations.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Coding agents can adapt robotic software by revising executable programs, but successful responses to an evolving opponent may not remain effective. We present a framework for program-level adversarial co-adaptive self-play in which coding agents generate attacks and defenses from tested opponent code and mission feedback, without model fine-tuning. Environment-specific interfaces support external programs for autonomous driving and SDK-level defenses for humanoid locomotion. Two eight-generation case studies reveal concrete software responses: a driving defender introduces direct-actuation recovery, while a humanoid defender revises incoming-state validation. These responses provide local task recovery but do not establish sustained robustness. The framework makes such adaptation and limitations observable through executable artifacts and task-level evidence.
1 Introduction
Robotic security must address physical behavior as well as software access, motivating tests of attacks and defenses during ongoing missions. This work studies program-level co-adaptation and traces concrete responses and recurring failures in driving and humanoid case studies.
1 Introduction
- A disclosed Unitree Go2 vulnerability exposed an unauthenticated DDS interface that could let a network-adjacent attacker install code later executed through a controller binding.This motivates proactive testing of attacks and candidate defenses against their consequences for an ongoing robotic task.Source: §1 · p. 1
- The central question is whether agents can attack an existing robotic stack and revise defenses effectively as the opponent adapts.Prior work connects generated code to embodied control, executable skills, and code-mediated improvement [2]–[5].Source: §1 · p. 1
- The framework places attack-generating agents and a shared defense-generating agent around a running robotic system, using tested programs and outcomes to guide revisions without model fine-tuning.Adaptation occurs in software and retained context; environment-specific permissions define what programs may change.Source: §1 · p. 1
- The framework’s competing programs execute around an operating autonomy stack through robotic middleware and permitted software modifications.The humanoid evaluation combines clean, attack-only, defense-only, and repeated joint trials to distinguish attack effectiveness, defense viability, and normalized task recovery.Source: §1 · p. 2
- Two eight-generation case studies trace countermeasures and counterattacks in driving and humanoid tasks, revealing targeted recovery alongside persistent failures.The studies motivate evaluation beyond a favorable individual match.Source: §1 · p. 2
2 Related Work
Related work spans code-mediated robot control, adversarial code competition, cyber defense, and behavior-grounded robotic testing. This paper studies generated attack and defense software competing directly during a robotic mission.
From executable behavior to competitive adaptation
- Unlike work that evolves strategies or policies against changing opponents, these experiments keep the mission and simulator configuration fixed while revising attack and defense programs around an existing robotic stack.Related work includes CodeClash [6], FAMOU [7], and COvolve [8].Source: §2 · p. 2
From competition to security hardening
From software interference to robot behavior
- Unlike software tests focused on semantic failures or adversarial instructions to LLM-controlled robots, this work has language models develop attack and defense software for an autonomy stack that need not contain an LLM –[15].Mission outcomes reveal whether interference succeeds and how countermeasures alter robot behavior and become targets of later attacks.Source: §2 · p. 2
3 Methodology
The methodology fixes the mission to focus adaptation on opponent software, then uses task-grounded outcomes and tested code to inform revisions. Driving and humanoid configurations use distinct intervention permissions, attacker allocation, and evaluation procedures.
3.1 Attack-Defense Self-Play Loop
- A fixed mission focuses adaptation on opponent software, and one shared defender faces each attacker separately so it must handle different interference using the same submission.Agents develop concurrently in private workspaces, receive role-specific history and feedback, and do not see opponents’ current workspaces.Source: §3 · p. 2, §3.1 · p. 3
- Driving agents use external ROS 2 programs, with Blue permitted direct actuation; humanoid Red uses DDS messages while Blue can provide a monitor and permitted SDK replacements before the fixed locomotion policy.The humanoid monitor can also publish commands.Source: §3.1 · p. 3
- Submissions are checked, frozen, and evaluated without live language-model calls or mid-trial edits; an independent referee uses simulator ground truth, and tested code and outcomes return to agents for revision.The arena allows up to two automated repairs; middleware access is assumed, while direct simulator or referee manipulation is outside policy.Source: §3.1 · p. 3
3.2 Attacker Allocation and Evaluation
- Driving uses five fixed attacker specializations, whereas a humanoid orchestrator proposes five roles each generation from interfaces, task, and prior feedback while retaining role histories.Unused attackers remain available to Blue, and the orchestrator is instructed to retain effective pressure or explore other directions as needed.Source: §3.2 · p. 3
- Each humanoid generation runs 22 formal evaluations: one clean, one defense-only, five attack-only, and fifteen joint episodes, alongside unscored development sparring.Shorter episodes and up to sixteen parallel slots make more testing practical than sequential driving matches with Autoware restarts.Source: §3.2 · p. 3
3.3 Mission-Grounded Evaluation
- Driving quality combines clipped route progress with retained peak speed, using a 60 m route and 4.5 m/s clean-baseline speed; it informs revision but is not added to terminal reward.Goal completion sets progress to one, and the speed component discourages slow defensive takeover while encouraging preservation of nominal driving behavior.Source: §3.3 · p. 4
- Humanoid recovery compares joint-episode quality with clean and attack-only quality; Blue receives the median of three recovery scores and its lowest eligible reward guides revision toward its weakest defense.Recovery ranges from no improvement over attack-only to restoring clean quality; for gaps below 0.05, it falls back to goal completion.Source: §3.3 · p. 4
4 Experimental Results
Across two eight-generation studies, agents revised attacks and defenses in response to prior outcomes, producing targeted recovery alongside persistent failures. The evaluations span driving and humanoid locomotion, with a shared defender in each configuration.
4.1 Experimental Setup
- Each platform configuration used one eight-generation run and one shared defender; the studies trace program revisions and resulting mission outcomes.The evaluated platforms were Autoware-CARLA and Unitree G1-MuJoCo.Source: Table 1 · p. 4, §4.1 · p. 4
4.2 Driving; 4.3 Humanoid
- Driving produced 18/40 goals, 16 timeouts, and 6 collisions; humanoid episodes produced 51/176 goals, including 11/90 joint episodes against mission-disrupting attacks.Figure 2 maps successful completion to a defender win, crashes, collisions, or off-road events to attacker wins, and timeouts to ties.Source: Figure 2 · p. 5, §4.2 · p. 5, Figure 3 · p. 5
4.2 Driving
- In driving, generation 3’s direct-actuation recovery driver helped all five matches reach the goal, but later attacks targeted the bypass and other defenses.Red exploited the bypass’s stand-down behavior in generation 4; later attacks targeted recovery inputs and commands, while Blue added recovery and arrival checks.Source: §4.2 · pp. 5–6
4.3 Humanoid
- In humanoid trials, Blue tightened incoming-state checks after tilt spoofing still caused falls; two of three generation-4 joint episodes then reached the goal.Later goal-progress and IMU-drift attacks targeted validation thresholds, and Blue tightened both checks in generation 8; some attack classes remained unresolved.Source: §4.3 · pp. 6–7
5 Limitations and Future Work
The studies are limited by narrow scenario coverage, incomplete evaluation metrics, and unisolated agent feedback mechanisms. Future defender orchestration also needs safeguards for coordinating specialist changes.
5 Limitations and Future Work
- Each configuration has one eight-generation lineage and a simple task, so broad robustness requires testing more scenarios and task conditions.The stated limitation is about establishing broad robustness, not a claim that the observed adaptations fail in other settings.Source: §5 · p. 7
- The quality functions use peak speed and progress, not comprehensive safety, smoothness, or control provenance, and normalized values are configuration-specific.Clipping and median aggregation can also obscure residual failures.Source: §5 · p. 7
- The study does not isolate which feedback mechanisms most improve agent learning across generations.Future ablations are proposed for tested opponent code, prior-match history, development sparring, and continuous quality feedback.Source: §5 · p. 7
- A future defender orchestrator would need merge checks, rollback, and historical-attack tests to add protections without removing or conflicting with earlier ones.Specialist changes must share SDK hooks, state, and command authority.Source: §5 · p. 7
6 Conclusion
The paper presents program-level adversarial co-adaptive self-play using executable robotic software, opponent artifacts, and mission feedback. Its two eight-generation studies trace task recovery and SDK-based defensive responses alongside recurrent failures, motivating further study with explicit intervention authority and historical-opponent regression.
6 Conclusion
- The framework uses executable robotic software, opponent artifacts, and mission feedback for program-level adversarial co-adaptive self-play.Source: §6 · p. 7
- Two eight-generation studies trace direct task recovery and SDK-based defensive responses while revealing recurrent failures.Source: §6 · p. 7
- The results motivate studying opponent-program changes with explicit intervention authority, behavior-linked feedback, and historical-opponent regression.Source: §6 · p. 7