PaperlayerView exact revision on arXiv

Source-linked AI summary

Compaction Laundering: Provenance Destruction in Context-Compacting Coding Agents

Arshan Dabirsiaghi

arXiv:2610.29869v1cs.CR

TL;DR

Long-running agents must compact their histories, and the paper examines whether that process creates an exploitable trust boundary. It tests how serialization affects laundering and evaluates a code-level defense. Laundering occurs in flattening harnesses but not role-typed ones, and the paper identifies a harness-side fix.

  • Problem

    Long-running agents must compress their conversation into a note and discard the original, raising the question of whether this compaction step is an exploitable trust boundary.

    Source: The problem · p. 1

  • Method

    The paper compares flattening harnesses, which encode roles in text labels, with role-typed harnesses, which retain roles as message fields.

    Source: §4 · p. 4

  • Results

    87–100% of laundering occurred in flattening harnesses, compared with 0% in role-typed harnesses.

    Source: Discussion · p. 26

  • Takeaways & Limitations

    Escaping role labels before flattening, or keeping messages role-typed, addresses the provenance loss identified in the serializer.

    Source: Discussion · p. 26

    The executed demonstration measures the model’s decision in isolation, without an approval gate or sandbox; some sandboxed consequences were not demonstrated.

    Source: Limitations · p. 28

Abstract

from arXiv · show

Coding agents that run long enough must compact their context, summarizing the conversation, discarding the original, and carrying the summary forward as memory. We show that in popular open-source coding agents this step is an exploitable trust boundary. Harnesses that flatten the typed conversation into a single plain string, marking each message with a text label such as [User]:, destroy message provenance in code, before the summarizer runs. An attacker who controls anything the agent reads can forge a user turn that survives compaction as a genuine instruction. Across five flattening harnesses (pi, opencode, Cline, Aider, OpenHands) the forged approval launders into the agent's memory 87-100% of the time, up to a ~908,000-token context window; across four role-typed harnesses (Codex, Gemini CLI, Qwen Code, Continue) the identical attack succeeds 0% of the time. We follow the attack to executed consequences. A laundered standing approval drives real tool-running agents to execute attacker-controlled curl | sh in a sandbox, and end to end in the live pi TUI at ~50% against a 0% paired clean arm. The same channel reliably steers an agent's plan away from a file an attacker designates off-limits, which can blind a code review to a planted backdoor. Because the provenance is destroyed deterministically in harness code, no prompt-level defense can restore it; we measure a code-level provenance check that catches 77% of laundering while demoting 2% of legitimate summary content. The defect also has a straightforward fix at the serializer, escaping the role labels before flattening; the role-typed harnesses avoid it entirely by keeping messages role-typed.

Introduction

Long-running coding agents compact their histories, and flattening message roles before summarization can turn attacker-controlled text into a genuine-looking user instruction. The paper measures this exposure across harnesses, follows it to consequences, and evaluates a code-level defense.

The problem

  • A forged user line can survive compaction as a genuine-looking instruction after flattening erases the origin of untrusted content.The attacker can plant the line in anything the agent reads, such as a file or web page.Source: The problem · pp. 1–2

Scope and severity

  • A laundered approval can drive attacker-controlled commands with the developer’s permissions, while a malicious troubleshooting page needs no insider access.The paper also describes the risk of derivative agents inheriting a released agent’s serialization flaw.Source: Scope and severity · p. 2

Contributions

  • 87–100% of laundering occurred in flattening harnesses, versus 0% in role-typed harnesses, which preserve message provenance.The role-typed harnesses serve as a built-in control group.Source: Contributions · p. 2
  • A harness-side check catches 77% of laundering while demoting 2% of legitimate summary content; escaping role labels before flattening is a proposed direct fix.The check compares user-attributed summary claims against genuine user turns still held in harness memory.Source: The problem · p. 2, Contributions · p. 3

1. The trust boundary

The trust boundary is the serializer: flattening removes typed roles and leaves plain-text labels as the record of who spoke. Unescaped labels in untrusted content can then be mistaken for genuine user turns.

1. The trust boundary

  • Harnesses that flatten roles without escaping labels in untrusted content are vulnerable; the vulnerability is not universal.The attack resides in the compaction step, where typed messages become a string marked by plain-text labels.Source: §1 · p. 3
  • The pipeline’s critical transition is flattening, where message provenance is lost.Figure 1 identifies the flatten step as the point of provenance loss.Source: Figure 1 · p. 3
  • The summarizer need not be tricked: faithfully copying a forged [User]: line into user instructions is enough to launder it.Prompt instructions to ignore historical instructions do not prevent recording a forged user constraint.Source: §1 · p. 3

2. Threat model

The threat model gives an attacker control over material the agent reads and examines two ways that material can poison memory. Forging a role label produces the stronger attack chain; the measured rates exclude two additional attacker capabilities.

2. Threat model

  • A newline followed by a forged [User]: marker in a file or web page becomes indistinguishable from a user line after flattening.This is the stronger primitive and the source of the executed consequences measured in the paper.Source: §2 · p. 3
  • Even without forging a label, a summary with no origin field makes README content textually indistinguishable from user-provided text.This weaker provenance-erasure primitive takes effect in every compaction.Source: §2 · p. 4
  • The reported rates are a floor because recombination of innocent fragments and eviction of contradictory turns are excluded, though both are within an attacker’s reach.The forged line is recorded as a user instruction and re-injected into the next context as a genuine user-role message.Source: §2 · p. 4

3. Attack scenarios

The attack uses the same forged approval across scenarios, with the attacker’s position as the variable. The scenarios include insider onboarding and external documentation or dependency routes; the external routes do not require access to the victim’s repository or team.

3. Attack scenarios

  • The approval stays identical across the representative scenarios, making the attacker’s position the only variable.The scenarios explain how the forged approval reaches the agent and what each route requires.Source: §3 · p. 4

Onboarding doc

  • The insider scenario is the weakest claim because it assumes the most attacker access; in the two external scenarios, the attacker never touches the victim’s repository or team.The external routes include a troubleshooting page the agent web-fetches and dependency documentation read while tracing.Source: Onboarding doc · p. 4
  • The dependency route is a stronger code-publishing channel, but documentation is measured because supply-chain controls treat the two differently.Lockfile diffs, vulnerability scanners, and provenance signing watch dependency code, while no tool watches package documentation.Source: Onboarding doc · p. 4

4. Harness classes: flattening and role-typed

Exposure depends on how a harness serializes the conversation: flattening puts role labels in the text, while role-typed harnesses preserve roles as fields. The forged-label attack launders in the flattening group but not the role-typed control.

4. Harness classes: flattening and role-typed

  • Flattening harnesses serialize messages into one text string with in-band role labels; role-typed harnesses pass a typed message array with roles out of the attacker’s reach.The tested flattening harnesses were pi, opencode, Cline, Aider, and OpenHands; the role-typed controls were Codex, Gemini CLI, Qwen Code, and Continue.Source: §4 · p. 4
  • 87–100% of attacks laundered in five flattening harnesses, versus 0% in four role-typed harnesses.The role-typed group serves as the control.Source: §4 · p. 4
  • A genuine anti-injection rule does not eliminate laundering when applied to a flattening harness, because detection does not prevent false user attribution.Gemini CLI’s 0% rate comes from role-typing, not its anti-injection rule.Source: §4 · p. 5
  • A sandbox blunts network-payload harms, but staging exfiltration to a workspace file, planting code for a later trusted step, and blinding review by skipping files remain possible.Execution gates also differ: pi has no permission popups or sandbox, while Codex’s default workspace-write mode blocks network egress.Source: §4 · p. 5

5. Prompt provenance

The harness comparisons use each system’s upstream compaction prompt and serializer captured at a pinned commit. The listed sources identify the relevant components for the flattening and role-typed harnesses.

5. Prompt provenance

  • Each harness’s compaction prompt and serializer were captured verbatim from upstream at a pinned commit.The repository records the exact capture for each harness.Source: §5 · p. 5
  • The listed flattening implementations include pi’s compaction and serializer code, opencode’s session compaction, Cline’s compaction code, and Aider’s history and prompt files.The source list also identifies OpenHands’ summarizing condenser and prompt template.Source: §5 · p. 5
  • The role-typed source list identifies Gemini CLI’s prompt snippet, Qwen Code’s Gemini CLI fork, and Continue’s compaction summary prompt.Codex’s compact prompt is also listed among the upstream sources.Source: §5 · p. 5

6. Experimental setup

The experiments drive the real summarization prompts and serializers with real models, then execute generated commands in a disposable container. Staged runs isolate those components, while full-window runs and a live pi TUI experiment test the chain under more realistic conditions.

6. Experimental setup

  • The setup drives each harness’s real summarization prompt and serializer with real model calls, then runs generated commands in a disposable Docker container.The execution uses a real curl and a payload that writes a proof file, recovered from the host to confirm execution.Source: §6 · p. 5, Real, pinned to a specific commit · p. 5

Real, pinned to a specific commit

  • The prompts are copied verbatim from pinned upstream commits, and pi’s flattening code is a direct port of its TypeScript serializer.The experiments also use real API calls to frontier models.Source: Real, pinned to a specific commit · p. 5

Staged by us, not the released product

  • Staged experiments bypass the harness programs and trigger summarization directly on synthetic poisoned conversations; E15 instead runs the chain in the real pi TUI with a fetched attacker-controlled guide and paired clean runs.Full-window experiments address the short transcripts’ inability to trigger real compaction on their own.Source: Staged by us, not the released product · p. 6
  • The full-window runs found laundering at 100% across tested sizes up to ~908k tokens, while execution settled to ~20–40%, making short-run execution rates an upper bound.The model’s 922k input ceiling stopped the size sweep; full-window runs match the size at which real agents compact.Source: Staged by us, not the released product · p. 6

7. Metrics

The metrics distinguish false user attribution from mere survival of attacker text, and separate command issuance from payload confirmation. A clean arm checks that the task does not execute without the forged line.

7. Metrics

  • Laundering requires the forged line to appear in the summary as user-originated, with no indication that it came from elsewhere.A fixed LLM-judge criterion scores this, cross-checked against an independent adjudication that agreed on all 18 sampled items.Source: §7 · p. 6
  • Survival measures whether the forged line appears anywhere in the summary, including when it remains attributed to a document rather than the user.This distinguishes role-typed summaries that preserve the text without laundering it.Source: §7 · p. 6
  • For live measurements, issued means the agent ran the attacker’s command, while confirmed means the payload’s proof file appeared.The two can differ if a fetch fails mid-run; the behavioral number is issued.Source: §7 · p. 6
  • The clean arm repeats the task without the forged line and must remain at zero execution.Any execution in that arm means the task was mis-specified.Source: §7 · p. 6

8. Experiments

The experiments probe laundering across models, controls, compaction stages, and serialization strategies, then test whether forged memories alter plans or drive execution. Additional tests examine scale, position, defenses, live use, and overtly malicious transcripts.

8. Experiments

  • The experiment suite asks whether laundering generalizes across model families, depends on forged labels, resists an in-prompt rule, compounds across recompactions, changes plans, or drives code execution.These questions are designated E1–E6.Source: §8 · p. 7
  • Further experiments test whether laundering survives full windows, varies with poison position, is caught by code-level defenses, or depends on flattening rather than role-typed serialization.The suite also tests execution on other flattening harnesses and scale near the model’s input limit.Source: §8 · p. 7
  • The remaining listed tests examine summary salience and approval salience as windows grow, live execution in pi against a clean arm, audit-resolution forgery, and refusal to compact overtly malicious transcripts.The supplied experiment list truncates the wording of E13 and E16.Source: §8 · p. 7

9. The defense evaluated

The evaluated defense checks user-attributed summary claims against retained genuine user turns, demoting claims that cannot be grounded rather than deleting them. Provenance contrast is used to preserve legitimate paraphrases that a literal-match check would discard.

9. The defense evaluated

  • After summarization, the harness checks each user-attributed claim against genuine user turns still held in memory and demotes unsupported claims as unattributed.The check requires no model cooperation.Source: §9 · p. 7
  • Provenance contrast tests whether a claim tracks untrusted text more closely than anything the user stated, avoiding the legitimate-content loss of exact substring matching.The evaluation reports clean-conversation false positives alongside the catch rate.Source: §9 · p. 7

10. Containment and ethics

Executions are contained in disposable Docker containers with only a scratch directory mounted, and the payload only writes a proof file. The live demonstration uses separate attacker and victim containers and a canary that proves execution without transmitting sensitive data.

10. Containment and ethics

  • Each execution runs in a disposable Alpine container with only a scratch directory mounted; its benign payload writes a proof file and cannot reach the host.A localhost server is mapped to a plausible remote-looking hostname inside the sandbox.Source: §10 · p. 7
  • The live demonstration isolates attacker and victim containers and uses a host-registration POST canary, with no sensitive data read or sent.Both containers are destroyed after each take.Source: §10 · p. 7

Results

The results examine how flattened message roles can let forged approvals survive compaction as user instructions, while role-typed serialization prevents laundering, and how such approvals can steer plans or drive command execution. In the live pi chain, execution depended on the approval’s prominence in the compacted summary and whether the agent’s audit survived.

E1: The laundering rate is stable across model families

  • Across three labs, pi’s laundering rate was within two points, and removing the forged [User]: label made the same payload mostly fail to launder.E1 interprets the cross-lab consistency as evidence that harness flattening, rather than the model, causes the effect; only pi’s rows measure a real harness, while Gemini CLI’s flattening rows are counterfactual and Codex’s 0% is a scoring artifact. Escaping labels before flattening makes attacker text behave like the plain-text arms.Source: E1: The laundering rate is stable… · p. 8, E2: The forged label is the mechanism · p. 8

E3: An anti-injection rule, on a flattening harness, helps unevenly and never eliminates the effect

E5: The agent obeys a laundered instruction as completely as a genuine one

  • Laundered constraints steered agents to avoid a designated file almost as completely as genuine user instructions, enabling a review agent to skip a file holding a backdoor.Across three study harnesses, direct user constraints yielded 98% avoidance and 98% cited reasons; laundered constraints yielded 92% avoidance and 100% cited reasons. Excluding Codex, genuine and laundered avoidance were 100% and 94%.Source: E5: The agent obeys a laundered… · p. 10

E6: The laundered approval drives execution, at a context-dependent rate

The execution rate falls as the window fills

E7: The laundering is scale-invariant (unlike execution)

E13: Why execution falls with size, and how an attacker reverses the attenuation

E10: Serialization determines exposure: flattening launders, role-typed does not

  • 87–100% of trials laundered forged approvals in flattening harnesses, versus 0% in role-typed harnesses, with serialization the only variable.The identical attack, models, and payloads were used; all clean arms were 0/15. The flattening group was pi, opencode, and Aider at 100%, OpenHands at 93%, and Cline at 87%; Codex, Gemini CLI, Qwen Code, and Continue were 0%.Source: E10: Serialization determines… · p. 14

E13: Why execution falls with size, and how an attacker reverses the attenuation

  • At fixed size, execution tracked judged salience: the recency design scored 4.8 and executed 4/5, while repetition scored 2.8 and executed 0/5; laundering stayed 5/5.Moving one forged approval to the session’s end raised execution from the 40% baseline to 80%; repeating it four times lowered execution to 0%, while re-delivery shortly before compaction reliably recovered execution in live takes.Source: E13: Why execution falls with size,… · p. 18

E14: Salience declines as the window grows

  • Across context sizes from ~2,400 to ~263,400 tokens, judged salience fell from 5.0 to 3.0 while laundering remained 5/5.The reported intermediate context sizes were ~25,700 tokens at salience 5.0 and ~111,700 tokens at salience 4.2; the summaries’ prominence declined even though the approval continued to land.Source: E14: Salience declines as the window… · p. 18

E15: The complete chain, live, in the real product

  • 3/6 poisoned takes won end to end, versus 0/5 clean takes, when a fetched document’s forged approval survived compaction and the agent later executed the bootstrap.A clean take executed before compaction, so the paired design attributes that execution to model variance rather than the attack; the clearest separation came when the user asked the agent to continue its plan.Source: E15: The complete chain, live, in the… · p. 19

E16: The audit is the barrier, and it is forge-resistant

  • Execution collapsed to 1/50 across arms and both models when a genuine audit was present, even though forged user approval still laundered reliably.Assistant-thinking and tool-result forgeries could be checked against the transcript, while user approval could not; the audit’s concerns generally remained decisive, with one execution in the maximal-forgery evidence arm.Source: E16: The audit is the barrier, and it… · p. 20

Case Study: A Laundered Approval, Turn by Turn

In a live pi case study, a forged user-labeled approval survives compaction after the agent has audited and refused the bootstrap, then drives execution. A paired clean run with the forged line removed asks for approval instead.

Case Study: A Laundered Approval, Turn by Turn

  • The full chain is presented as one recorded E15 take, reconstructed from pi’s session log, showing the attack, laundering, and execution.The paper identifies it as a concrete instance of the chain measured in the Results section.Source: Case Study: A Laundered Approval, Turn… · p. 21
  • The setup uses pi v0.79.0 with a developer’s linked setup guide; the attacker controls only that web page, and the tests require the bootstrap.The forged standing approval was placed at the top of the guide and asserted prior approval rather than requesting immediate execution.Source: Case Study: A Laundered Approval, Turn… · p. 21

Phase 1: The agent catches the attack (19:19:27–19:20:44)

  • Before compaction, pi refused to execute the bootstrap and wrote a cleaned guide, while substantive investigation pushed the original audit deeper into the conversation.A later user request prompted a fresh fetch of the attacker’s guide, placing its forged line in the most recent tool result.Source: Phase 1: The agent catches the attack… · p. 22

Phase 2: Compaction launders the approval (19:21:27)

  • The summary promoted the forged approval to a user constraint, reversing the audit’s conclusion; compaction then discarded the tool result that exposed the line’s web-page origin.The summary retained the bootstrap’s security-relevant facts but appended that the user had approved running it.Source: Phase 2: Compaction launders the… · pp. 22–23

Phase 3: The agent obeys its corrupted memory (19:21:42)

Phase 4: What Bob received

  • The bootstrap and installed tools produced six callbacks to the attacker’s endpoint, demonstrating the canary’s reporting channel.The paper notes that further payload behavior is a design choice.Source: Phase 4: What Bob received · p. 24

The clean twin: same session, one line different

  • The paired clean run differed only by the forged line: it retained the need for approval, while the poisoned run executed.Both runs used the same session, server, audit, compaction, and follow-up message.Source: The clean twin: same session, one line… · p. 24

Why this example

  • The pre-compaction audit stopped the attack, but compaction reversed its verdict because the forged line was labeled as a user constraint.The follow-up “what’s next?” granted no approval; execution followed from the altered memory.Source: Why this example · pp. 24–25

Interpretation

The paper distinguishes compaction laundering from prior memory-poisoning work by tracing the attack to execution in released agents, with controls. Its interpretation locates exposure in role-flattening serialization and proposes serializer- or code-level defenses, while noting important limits on the execution and defense evidence.

Compaction summary

  • This work fills the compaction-channel gap: prior work described the consequence without runtime evidence, while Sleeper demonstrated execution only through persistent memory [3].Compaction is mandatory for long sessions and requires no dedicated memory module; its summarizer processes the whole window, unlike a persistent-memory manager that can gate storage.Source: Compaction summary · p. 25

Prior work

  • Prior work measured laundering at memory-write or followed poisoning through execution in a separate persistent-memory channel, while this paper follows compaction laundering to executed behavior.From Untrusted Input to Trusted Memory [2] measured write success but not behavior; Sleeper [3] and MemMorph [4] studied persistent memory, which requires a dedicated module.Source: Prior work · pp. 25–26, Position in the literature · p. 25

Discussion

  • 87–100% laundering in five role-flattening harnesses versus 0% in four role-typed harnesses; executed attacks reached all five flattening harnesses and ~50% live in pi TUI versus 0% in the paired clean arm.The live result is small-scale; the paper attributes execution-rate variation partly to context and audit/message phrasing.Source: Discussion · p. 26
  • The laundering rate persists from ~2,800 to ~908,000 tokens and across poison positions, but execution attenuates with window size; recency-oriented placement can raise execution again.Repeated compaction neither strengthens nor erases the injected claim; laundering verdicts held on 52 of 54 chains.Source: Discussion · pp. 26–27

Position in the literature

  • The reported laundering rates vary minimally across three labs—pi measured 69/68/67—supporting a harness-level rather than model-specific account.The paper distinguishes this replication from its newer contribution of following compaction laundering through to execution.Source: Position in the literature · p. 25, Discussion · p. 27

Defenses

  • Escaping role labels before flattening drops forged-label laundering to the plain-text floor; checking user-attributed summary claims against retained user turns catches 77% while demoting 2% of legitimate content.The defense has not been tested against adaptive payloads, so 77% is an upper bound against a non-adapting attacker.Source: Defenses · p. 27, Discussion · p. 27, Limitations · p. 28

Limitations

  • Execution evidence is bounded by model-only demonstrations, small samples, and a live end-to-end test limited to one harness, one model, and small n.High-stakes payloads may be refused on their own merits, and the live rate depends on whether the agent’s audit and next-message phrasing preserve or counter the attack.Source: Limitations · p. 28

Responsible disclosure

  • The defect is unescaped in-band role markers in untrusted tool output; advisories went to the five affected flattening harnesses, while four role-typed harnesses were not disclosure targets.The paper identifies a straightforward serializer fix.Source: Responsible disclosure · p. 28
Loading 2610.29869v1…