Source-linked AI summary
Understanding and Exploiting Initialization Anchoring Weakness in Feedback-Based Agent Planning
Chuanchao Zang, Jianing Wang, Wenyu Chen, Xiangtao Meng, Li Wang, Xinyu Gao, Peng Zhan, Zheng Li, Shanqing Guo
TL;DR
Feedback-based planning can revise plans using observations and corrective feedback, but existing evaluations report final attack success rather than how mitigation is distributed across rounds. The paper analyzes round-level correction and proposes INITANCHOR, a black-box attack using attacker-controlled external materials. It reports attack success across settings with and without target planning traces, including against defenses and real-world agent systems.
Problem
Existing evaluations report final attack success with and without feedback but do not show whether mitigation is distributed across planning rounds or concentrated in a few.
Source: §1 · pp. 1–2Method
INITANCHOR is a black-box framework that uses directional-shift, contextual-plausibility, and counterevidence-resilience signals to generate adversarial material with limited target access or a local shadow model.
Source: §1 · p. 2, §2.2 · p. 3Results
76.1% attack success rate with planning traces and 72.0% ASR without traces exceeded the strongest baselines by 24.2 and 19.5 percentage points, respectively.
Source: §1 · p. 2Takeaways & Limitations
The first feedback round reverses about 46% of adversarially induced planning directions, while the next two rounds reverse 13% and 7% of those remaining.
Source: §1 · p. 2The attacker controls external material but cannot modify agent internals or access fine-grained execution traces; without target access or coarse-grained traces, the approach uses a local shadow system.
Source: §2.2 · p. 3
Abstract
from arXiv · showhide
Feedback-based planning improves agent reliability by incorporating tool observations and corrective feedback. However, its protection may not be distributed uniformly across planning stages. We conduct a round-wise analysis of four representative feedback mechanisms and uncover an initialization anchoring weakness: the first feedback round corrects 46\% of adversarial directions, whereas the rates fall to 13\% and 7\% among directions surviving into the next two rounds. Our analysis attributes this weakness to three interacting factors: a contextually plausible shift in the initial plan, insufficient counterevidence, and the persistence of accepted directions in the accumulated trajectory. Based on these findings, we propose \textsc{InitAnchor}, a black-box framework for exploiting this weakness through attacker-controlled external materials. It operationalizes the three factors as directional-shift, contextual-plausibility, and counterevidence-resilience signals under either limited target access or no target access. Across 112 tasks from 16 domains, six agent architectures, and five backbone LLMs, \textsc{InitAnchor} achieves average ASRs of 76.1\% and 72.0\% under the two settings while reducing first-round mitigation rates to 21.0\% and 25.0\%, respectively. It also remains effective against six defenses and across six real-world agent systems. These findings show that feedback-based agents can retain early biases even when later correction is available.
1 Introduction
The introduction identifies uneven round-level correction as a weakness in feedback-based planning and presents INITANCHOR as a black-box attack designed to exploit it. It reports results across agent architectures, backbone LLMs, defenses, and real-world systems.
1 Introduction
- The attack targets planning agents that review third-party materials, where embedded adversarial content may steer decisions about jobs, credit, funding, or products.The evaluation also reports effectiveness against six defenses and six real-world agent systems, and proposes a defense that reduces attack success by more than 40 percentage points.Source: §1 · pp. 1–2
- 46% of adversarial directions are reversed in round one, versus 13% and 7% of surviving directions in the next rounds; suppression tests show the same ordering.Suppressing first-round mitigation increases attack success by 29.9 percentage points, compared with 9.0 and 3.8 points in rounds two and three.Source: Figure 1 · p. 1, §1 · p. 2
- INITANCHOR uses directional-shift, contextual-plausibility, and counterevidence-resilience signals to guide adversarial-material generation under limited access or no target access.When target traces are unavailable, a local shadow model approximates them.Source: §1 · p. 2
- Across six architectures and five backbones, 76.1% attack success rate with traces and 72.0% ASR without traces exceed strongest baselines by 24.2 and 19.5 percentage points, respectively.First-round mitigation rates fall to 21% and 25% in the two settings.Source: §1 · p. 2
2 Preliminaries
The preliminaries describe how agents form plans and revise them through execution feedback, then specify the attacker's control and access constraints. They distinguish limited target access from a no-query setting using a local shadow system.
2.1 Feedback-Based Agent Planning
- Planning converts a user request into ordered subgoals; unlike fixed reasoning structures, feedback-based planning revises plans as observations arrive.An agent generates an initial plan, acts through a tool, incorporates observations and optional verification, then updates its plan toward a final decision.Source: §2.1 · pp. 2–3
2.2 Threat Model
- The attacker controls domain-specific external material published through a legitimate channel, but cannot alter model parameters, prompts, agent components, tools, or feedback signals.The attacker also lacks access to fine-grained execution traces such as tool calls, observations, and exact verification feedback.Source: §2.2 · p. 3
- The threat model distinguishes limited target interaction with coarse-grained planning traces from no target interaction or traces, where a local shadow system approximates them.Source: §2.2 · p. 3
3 Round-Level Analysis of Feedback
This analysis tests whether feedback mitigates adversarial planning directions evenly across rounds and finds that mitigation is concentrated in the first round. It examines round interventions, entry timing, and factors linked to persistence.
3.1 Experimental Setting
- The study evaluates ReAct, Self-Refine, Reflexion, and CRITIC with GPT-5.1 on 112 instances across 16 domains, using ASR and round-level mitigation rate.The instances include tools that provide explicit counterevidence against the induced direction.Source: §3.1 · p. 3
3.2 Uncovering Initialization Anchoring
- Among directions surviving each preceding round, mitigation rates are 12.81% in round two and 7.08% in round three, indicating mitigation is concentrated in the first feedback round.Source: §3.2 · p. 4
- Suppressing the first round raises final ASR by 29.9 percentage points, versus 9.0 and 3.8 points for rounds two and three; cleaning round one reduces ASR by 13.7 points.The same ordering holds across all four feedback mechanisms.Source: Figure 3 · p. 4, §3.2 · p. 4
- Delaying adversarial content entry lowers ASR from 43.5% before planning to 38.7%, 34.4%, and 29.9% after rounds one, two, and three.Because each condition has the same number of post-entry feedback rounds, the authors infer that the weakness likely stems from initialization rather than first encounter timing.Source: Figure 4 · p. 4, §3.2 · p. 4
3.3 Dissecting Initialization Anchoring
- Reducing retained trajectory support lowers average ASR from 43.5% with Full History to 30.7% with Truncated History and 27.2% with Neutralized History.The authors conclude that supporting content retained in the accumulated trajectory reinforces persistence of the adversarial direction.Source: Figure 5 · p. 5, §3.3 · p. 5
- As initial plans move from counter-aligned to neutral to mildly biased, ASR rises from 6.0% to 17.3% to 45.1%, while first-round mitigation falls from 89.5% to 69.0% to 44.2%.Under strong bias, ASR is 37.4% and mitigation is 52.0%; the authors suggest mild bias persists better because it remains plausible, whereas strong bias conflicts more clearly with task criteria.Source: Figure 6 · p. 5, §3.3 · p. 5
- Average ASR falls from 58.8% with No Counterevidence to 43.5% with Autonomous Retrieval and 18.3% with Complete Counterevidence.The remaining gap suggests autonomous retrieval does not reliably expose all relevant evidence available from the tool backend.Source: Figure 7 · p. 5, §3.3 · p. 5
- The authors link initialization anchoring to retained and reused planning context, greater fit between the adversarial direction and that context, and insufficient subsequent counterevidence.These factors are presented as jointly associated with the weakness.Source: §3.3 · p. 6
4 Our INITANCHOR
INITANCHOR turns three factors linked to adversarial-direction persistence into calibration signals and reusable rules for attacker-controlled materials. It calibrates with limited target access or a shadow model, then applies rules to new instances without further target interaction.
4.1 Design Overview
- The design targets initial-plan shift, contextual fit, and resistance to counterevidence through three trajectory-guided signals.Source: §4.1 · p. 6
4.2 Trajectory-Guided Calibration Signals
- Directional shift measures added movement toward the target relative to the clean plan, while contextual plausibility rewards preserving the plan’s remaining task structure.Source: §4.2 · p. 6
- Counterevidence resilience is estimated with standalone calibration inputs containing hypothetical counterevidence; greater retained target-direction influence indicates more resilience.Source: §4.2 · pp. 6–7
4.3 Signal-Guided Rule Calibration
- During probe-task calibration, rule combinations are scored by the geometric mean of all three signals, and weak signals direct rule refinement.Source: §4.3 · p. 7
4.4 Instance-Level Adversarial Generation
- For unseen instances, the fixed calibrated library generates adversarial material before the victim task, without further target queries or feedback observation.Source: §4.4 · p. 7
5 Evaluation
The evaluation tests INITANCHOR across feedback-based agents, backbone models, attacker-access settings, defenses, and real-world systems. Its results show high attack success, lower first-round mitigation than baselines, and robustness across the tested settings.
5.1 Experimental Setup
- The benchmark covers six agent architectures, five backbone LLMs, and 112 tasks across 16 domains, with disjoint calibration probes and evaluation tasks.Evaluations are repeated five times; the section also examines content similarity, run-to-run stability, and planning rounds under attack.Source: §5.1 · p. 8, §5.2.1 · p. 8, Table 2 · p. 9
5.2.1 Performance of INITANCHOR
- 0.761 and 0.720 average ASRs under A1 and A2, respectively, contrast with a 0.090 natural target-direction rate without attack.First-round mitigation averages 21% under A1 and 25% under A2; INITANCHOR uses an average of 43.8 target-agent queries under A1, within the 50-query budget.Source: §5.2.1 · p. 8, Table 1 · p. 9
5.2.2 Comparison with Baselines
- 24.2 and 19.5 percentage-point ASR gains over the strongest baseline accompany lower first-round mitigation under A1 and A2.INITANCHOR’s average ASR is 0.782 under A1 and 0.735 under A2, versus 0.540 for CA; Figure 9 compares INITANCHOR with baselines.Source: §5.2.2 · p. 9, Figure 9 · p. 10
5.3.1 Ablation Study of INITANCHOR
- Signal-guided calibration reaches 0.82 and 0.73 ASR with 0.16 and 0.24 first-round mitigation under A1 and A2.Directional Shift leads single-signal ASR, while Counterevidence Resilience yields the lowest single-signal mitigation; using full trajectories instead of initial trajectories reduces ASR.Source: §5.3.1 · p. 10, Table 5 · p. 10
5.3.2 Robustness to Shadow-Model Choice
- Matched or same-family shadow models calibrate best, while cross-family ASRs remain 0.65–0.75 and exceed the strongest baseline’s 0.54 average.Self-shadowing yields A2 ASRs of 0.76 for Gemini-3 Flash and 0.81 for DeepSeek-V4 Flash, close to their A1 results.Source: §5.3.2 · p. 10, Table 6 · p. 10
5.3.3 Robustness to Attacker-Side Implementations
- Increasing the stopping threshold from 0.5 to 0.9 raises ASR and lowers first-round mitigation, at the cost of more A1 queries.A1 queries rise from 21 to 89; increasing probe tasks from 9 to 21 also improves both metrics while raising A1 queries from 19 to 53. Three selected rules outperform one, while seven degrade both metrics.Source: §5.3.3 · p. 11, Table 9 · p. 11, Table 8 · p. 11
5.4.1 INITANCHOR Against Defense
- Across tested defenses, INITANCHOR has the highest ASR, ranging from 0.53 to 0.82 under A1 and 0.49 to 0.78 under A2.CaMeL is the strongest evaluated defense, but INITANCHOR’s ASR remains above the strongest baseline under CaMeL.Source: Figure 10 · p. 11, §5.4.1 · p. 11
5.4.2 INITANCHOR Performence in Real-World Agents
- Across six real-world agent systems, INITANCHOR (A2) has the highest ASR on each, averaging 0.693 versus 0.428 for CA.The ASR range is 0.59–0.78, a 26.5-percentage-point average gap over the strongest baseline.Source: Figure 11 · p. 12, §5.4.2 · p. 12
6 Related Work
Prior work studies decision interference, prompt injection, and early-step effects, but the paper distinguishes its setting by focusing on attacker-controlled third-party material in feedback-based planning.
Decision Interference and Prompt Injection
- Decision-interference and direct prompt-injection attacks can be conspicuous and are often evaluated on a single response or action, unlike multi-round feedback-based planning.Trusted observations and verification can subsequently contradict injected content and reverse its influence [33] [44].Source: §6 · p. 12
Indirect Prompt Injection
- Indirect prompt injection contaminates tool-returned content, whereas this paper assumes attackers control only third-party material in the initial input and later tool feedback remains trusted.Prior evaluations include FITD [33], AgentDojo [13], and ASB [58].Source: §6 · p. 12
7 Discussion
The discussion tests trajectory compression as a potential mitigation and checks the reliability of AgentEvals against human judgments.
7.1 Potential Defense
- 0.82 to 0.35 and 0.73 to 0.28 ASR reductions under A1 and A2 accompany higher first-round mitigation after every-round trajectory compression.The preliminary Reflexion results suggest that restricting propagation of intermediate planning content can improve mitigation against initialization anchoring.Source: §7.1 · p. 12
7.2 Human Evaluation of AgentEvals
- On 1,344 outputs, human labels and AgentEvals judgments show Cohen’s κ = 0.757 for annotator agreement, while AgentEvals voting exceeds 0.88 accuracy.All single-model judges exceed 0.85 accuracy across six architectures.Source: §7.2 · p. 12
A Dataset Construction
The authors construct a benchmark because existing benchmarks lack a tool-feedback environment matching their setting, especially one with trusted, task-specific counterevidence.
A Dataset Construction
- The benchmark contains 112 instances across 16 domains, each pairing a query and benign third-party material with attacker-preferred and alternative directions, plus relevant tools.Domains include finance, healthcare, and online shopping.Source: A Dataset Construction · p. 16
- Tools provide domain-specific decision-relevant facts and sufficient evidence against the attacker-preferred direction; their outputs are fixed and outside attacker control.In the loan example, policy and affordability tools independently oppose direct approval, so missing corrective information alone cannot explain failure to change direction.Source: A Dataset Construction · p. 16
A.1 Probe-Query Construction
The authors construct disjoint calibration probes that mirror test-instance inputs, then test whether domain or material knowledge can specialize them. In one setting, domain- and material-matched probes outperform fully disjoint probes on ASR.
A.1 Probe-Query Construction
- Main-experiment probes are disjoint from test instances in queries, benign materials, attacker-preferred directions, and associated evidence.Each probe follows the same input structure as a test instance.Source: §A.1 · p. 16
- The loan example pairs a query and benign laptop material with an attacker-preferred recommendation and counterevidence about weight and battery life.Source: §A.1 · p. 16
- ASR rises from 0.68 with fully disjoint probes to 0.82 with domain-matched probes and 0.94 with material-matched probes.The corresponding bρ1 values are 0.26, 0.14, and 0.04, respectively.Source: §A.1 · p. 16
B Baseline Details
The baselines are adapted to change only attacker-controlled material, while the prompt templates preserve the agents’ native feedback workflows. Calibration and generation prompts assign reusable rules complementary roles in steering direction, preserving plausibility, and resisting counterevidence.
B Baseline Details
- All baselines may modify only attacker-controlled material and cannot change the user query, system prompt, tools, or subsequent feedback.GPTFuzzer uses target-agent initial plans under A1 or shadow-agent initial plans under A2, but does not observe later planning or feedback.Source: B Baseline Details · pp. 16–17
C Prompt Templates
- The agent prompt specifies an initial plan followed by the architecture’s native planning–action–observation loop and plan revision.A1 and A2 use the same calibration prompts and differ only in whether planning outputs come from the target or shadow agent.Source: C Prompt Templates · p. 17
- Security-enhanced variants add an instruction not to follow instructions embedded in third-party material.Source: C Prompt Templates · p. 17
- Rule selection chooses exactly K complementary rules based on task relevance and their roles in directional influence, contextual plausibility, and counterevidence resilience.The rule LLM selects rules but does not generate adversarial material.Source: C Prompt Templates · p. 17
- The generation prompt applies selected rules to attacker-controlled material while preserving its format, task-relevant criteria, and unrelated content.It forbids changing the query, tools, or agent instructions.Source: C Prompt Templates · p. 17
- When calibration candidates fall below threshold, the update prompt diagnoses weak signals and refines or adds reusable rules.It links weak directional shift to clearer target-direction support, weak contextual plausibility to preserving task-relevant content, and weak counterevidence resilience to more plausible support under opposing evidence.Source: C Prompt Templates · p. 17
D Adaptation to Real-World Agent Systems
The study adapts INITANCHOR (A2) to six real-world agent systems under a shared feedback-planning configuration. Because some commercial platforms do not expose complete trajectories, ASR is determined from final decisions alone across all six systems.
D Adaptation to Real-World Agent Systems
- The six systems share task inputs, system-level defense, planning-round limit, and tool outputs, while platform-specific mappings adapt prompts, planning states, and tool interfaces.Adversarial materials are generated before execution using the attacker-side shadow model.Source: D Adaptation to Real-World Agent Systems · p. 17
- The evaluation includes Qianfan, Coze, GPTs, the OpenAI Agents SDK, LangChain, and AutoGen without modifying platforms or accessing their internal planning implementations.Source: D Adaptation to Real-World Agent Systems · p. 17
- ASR is determined solely from final decisions because commercial platforms do not expose complete planning trajectories.The same output-only protocol is applied to all six systems.Source: D Adaptation to Real-World Agent Systems · pp. 17–18
E Trajectory Compression as a Potential Mitigation
Trajectory compression removes intermediate directional content from accumulated context while retaining execution history and tool evidence. In preliminary experiments on ReAct and Reflexion, it lowers ASR and raises first-round mitigation rates.
E Trajectory Compression as a Potential Mitigation
- Before each round, an auxiliary LLM preserves round order, completed operations, tool calls, and returned evidence while removing future plans, rationales, recommendations, and accepted directions.Source: E Trajectory Compression as a… · p. 18
- The intervention changes only the accumulated trajectory, leaving attacker-controlled material, tool outputs, and the underlying agent architecture unchanged.The evaluation compares raw and processed trajectories on ReAct and Reflexion with GPT-5.1, retaining the main evaluation’s data and procedures.Source: E Trajectory Compression as a… · p. 18
- ASR falls from 0.78 to 0.33 for ReAct A1 and from 0.74 to 0.26 for ReAct A2, while Reflexion falls from 0.82 to 0.35 for A1 and 0.73 to 0.28 for A2.First-round mitigation rates rise from 0.20 to 0.49 and 0.24 to 0.61 for ReAct, and from 0.16 to 0.46 and 0.24 to 0.58 for Reflexion.Source: E Trajectory Compression as a… · p. 18