PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent Flight Recorder: Tamper-Evident Audit Trails with On-Chain Anchoring for Long-Horizon Tool-Using Agents

Laurent Bindschaedler, Quentin Botha, Christoph Siebenbrunner

arXiv:2609.01931v1cs.CRcs.MA

TL;DR

Long-horizon agents produce consequential action sequences, but existing observability systems rely on mutable logs and operator-controlled telemetry rather than independently verifiable evidence. The Agent Flight Recorder combines semantic event records, canonical serialization, hash chaining, Merkle batching, and on-chain epoch-root anchoring. The full system reports 100% detection across attack classes, 48 µs median per-event latency, and $2.30 per 100K L2-anchored events.

  • Problem

    Long-horizon agent failures raise causality, authorization, and neutral-verification questions that existing mutable, single-party observability systems do not address.

  • Method

    The system records agent-semantic events, applies canonical serialization, hash chaining, Merkle batching, and periodically anchors epoch roots on a public chain.

  • Results

    100% detection across all attack classes, 48 µs median per-event latency, and $2.30 per 100K events at 100-event L2 epochs are reported for the full system.

  • Takeaways & Limitations

    On-chain anchoring enables neutral third-party verification without a pre-agreed intermediary, while the full integrity stack detects tampering under the stated evidence model.

  • Takeaways & Limitations

    The guarantees depend on instrumented gateway coverage, stated evidence assumptions, and workloads that are primarily synthetic, with some evaluations using single deterministic runs.

Abstract

from arXiv · show

Long-horizon agents execute thousands of actions, resulting in sequential failures rather than isolated errors. When a coding agent deletes a production database or a prompt injection spreads across agents, the incident raises questions of causality, authority, and non-repudiable third-party verification. The Agent Flight Recorder captures each agent action as a structured, canonically serialized event binding eight semantic fields from intent through execution to provenance. Hash chaining and Merkle batching provide tamper evidence and compact inclusion proofs. For cross-organizational disputes where no party's infrastructure qualifies as neutral ground, periodic on-chain anchoring of epoch roots lets any verifier with the disclosed payload and Merkle proof check the record independently, without pre-agreeing on a trusted intermediary. The on-chain footprint is minimal: each anchor stores a 32-byte epoch root and a back-pointer, and no event content touches the chain. We evaluate the system across five cumulative ablation configurations on synthetic agent workloads. The full system adds ~48 microseconds median per-event latency and 512 bytes per event. L2 anchoring costs $2.30 per 100K events at 100-event epochs. The full integrity stack detects edit, delete, reorder, and fork tampering at 100% with zero false positives. Structured forensic queries achieve 1.0 precision on guardrail and delegation lookups where unstructured text search yields 0.013 and 0.077 respectively.

I. INTRODUCTION

Long-horizon agents create consequential sequence failures that existing mutable, operator-controlled observability systems cannot independently verify. Agent Flight Recorder addresses this gap with semantic, tamper-evident records and evaluates their integrity, overhead, and forensic utility.

  • Motivation: Thousands of tool calls over hours or days make agent failures consequential sequences involving actions, approvals, provenance, and execution context.Examples include production database deletion, prompt-injection propagation, and unauthorized privilege escalation.
  • Problem: Current observability stacks support cooperative debugging but lack tamper-evident, externally verifiable records for disputes involving compromised operators.They assume mutable internal logs and a single party that produces and consumes telemetry.
  • Contribution: Agent Flight Recorder binds intent, policy evaluation, approval, execution, effects, context provenance, code provenance, and delegation provenance into canonically serialized, hash-chained events.Merkle batching and on-chain epoch-root anchoring support compact proofs and independent verification without a pre-agreed intermediary.
  • Evaluation: The evaluation uses five ablation configurations on synthetic workloads and real SWE-agent traces, measuring latency, storage, tamper detection, anchoring cost, and forensic queries.The study deploys the anchoring contract to Base Sepolia and measures gas from 100 on-chain transactions.
  • Results: 48 µs median per-event latency, 100% detection of edit, delete, and reorder attacks with zero false positives, and $2.30 per 100K L2-anchored events characterize the full system.On-chain anchoring is the only evaluated configuration enabling neutral third-party verification without a pre-agreed intermediary.
  • Positioning: The paper frames its contribution as an evidentiary settlement architecture for agent actions rather than an observability tool.It integrates standard cryptographic primitives into an agent-semantic system for cross-organizational disputes.

C. Threat Model and Assumptions

The threat model treats host compromise as enabling edits, deletions, reordering, and divergent histories, while the motivating scenarios require reconstructing causality and authorization across actions and organizations. The system’s positioning is bounded by explicit assumptions about anchored state, verifier infrastructure, and evidence availability.

  • Threat Model: The adversary may edit, delete, reorder, or equivocate by presenting divergent histories from a common prefix after compromising the agent host.These operations target payload integrity, sequence order, completeness, and consistency across verifiers.
  • Assumptions: At least one of immutable anchored roots, uncompromised verifier infrastructure, or an off-host raw-log copy must remain available.These assumptions support preservation and verification of pre-compromise history.
  • Guarantee Boundary: After full host compromise, the system cannot guarantee the truth of newly written entries; anchoring frequency only bounds the compromise window.It preserves pre-compromise history and detects tampering of entries written before compromise under the stated assumptions.
  • Scenarios: A destructive database action requires determining the executed tool call, guardrail evaluation, and human approval.The scenario illustrates why action records must preserve causal and authorization evidence.
  • Scenarios: Prompt-injection investigations require binding retrieved context provenance to each downstream action across multiple reasoning steps.The forensic target is which retrieved content influenced each harmful tool call.
  • Scenarios: Cross-organizational disputes require verifiable chains of delegated requests and approvals that neither operator can unilaterally control.A neutral third party must verify whether the action was authorized and how responsibility crossed organizational boundaries.
  • Positioning: Prior work provides hash chains, Merkle proofs, or agent-audit formalisms, but the paper combines these layers with evaluation and on-chain anchoring.The authors identify the system as the first evaluated combination of an agent-semantic schema, selective disclosure, and on-chain anchoring.
  • Design Goals: The design targets integrity, ordering, external verifiability, privacy, low overhead, and interoperability while excluding completeness as an explicit non-goal.The supplied passage introduces the goals and the explicit non-goal boundary.

B. Agent-Semantic Event Schema

The event schema records an agent action as a structured causal object spanning intent through execution, effects, provenance, and delegation. Canonical serialization and layered integrity structures make these records stable, orderable, batch-verifiable, and anchored for independent checking.

  • Schema: Each event binds eight semantic fields tracing an agent action from intent through execution to effects.The schema treats the action as a structured record rather than raw reasoning text.
  • Intent: Intent records the proposed tool name and structured arguments, while excluding raw model reasoning and chain-of-thought.The recorder captures the proposed action, not the thought process behind it.
  • Authorization: Policy evaluation records which guardrails assessed the proposed action, and human approval records authorization identity, prompt, timestamp, and ideally a cryptographic binding.The approval field is intended to bind the approver’s identity to the specific action rather than a metadata string.
  • Execution and Effects: Execution records the concrete dispatched command or API mutation, while effects record observable returns, state changes, external responses, or errors.Effects capture outcomes rather than the agent’s interpretation of them.
  • Serialization: Deterministic CBOR provides canonical map ordering and numeric encoding so subsequent hashes operate on a stable byte sequence.This avoids JSON canonicalization pitfalls involving floating-point representation, key ordering, and Unicode normalization.
  • Integrity Layers: Hash chaining orders events, Merkle batching provides O(log N) inclusion proofs, and epoch-root chaining detects gaps between anchoring points.The layers address distinct integrity and verification functions in the pipeline.
  • Payload Separation: Integrity structures operate on fixed-size event records, while full payloads are stored separately and encrypted when disclosure mode is enabled.The separation distinguishes sensitive event content from metadata used by the hash chain, Merkle tree, and epoch roots.
  • Architecture: Figure 1 shows tool calls becoming structured events, then serialized, chained, Merkle-batched, and periodically committed as 32-byte epoch roots for verifier inclusion checks.Encryption and selective disclosure are orthogonal to the integrity pipeline and omitted from the figure.

D. Verification Workflow

The verification workflow supports both integrity-only checks and content verification, using Merkle proofs, anchored roots, and optional per-event decryption. Verification is non-interactive, but payload availability and authoritative identity remain outside the cryptographic protocol.

  • Integrity verification: Integrity-only verification confirms that an event was committed in an anchored epoch without revealing its payload.The verifier recomputes the Merkle path, matches the epoch root to the on-chain anchor, and checks back-pointer continuity.
  • Content verification: Content verification additionally decrypts the disclosed ciphertext and checks that the plaintext hashes to the committed payload hash.This establishes both content integrity and authenticity.
  • Verification properties: Both verification workflows are non-interactive and require only disclosed material plus read access to the public chain.The verifier does not need a dispute window or counterparty interaction.
  • System boundary: Authoritative contract identity and off-chain evidence availability must be established separately from the anchoring protocol.The chain commits to evidence but does not guarantee access to payloads, proofs, or decryption keys.
  • Privacy and storage: The recorder stores hashes and structural metadata on-chain-linked logs while keeping full event payloads encrypted off-chain for authorized disclosure.This separates integrity-critical structures from sensitive content.
  • Selective disclosure: Selective disclosure derives per-event keys through a two-level HKDF hierarchy, allowing auditors to decrypt only investigated events during cooperative audits.The operator discloses event-specific keys, while withholding the epoch key prevents derivation of other event keys.
  • Limitations: If an auditor obtains the epoch key, they can derive every per-event key in that epoch, limiting selective disclosure under compelled disclosure.The construction therefore supports cooperative rather than compelled full-disclosure settings.

IV. ANCHORING ANALYSIS

The anchoring analysis positions public-chain commitments as a non-interactive verification mechanism for disputes lacking a pre-agreed neutral intermediary. The design uses batched epoch roots and back-pointers while leaving event content off-chain.

  • Anchoring protocol: Each epoch anchor stores the root Rt, the previous-root hash H(Rt−1), recorder identity, epoch number, and timestamp, but no log data.The contract records only a 32-byte root and back-pointer on-chain.
  • Anchoring protocol: The architecture combines local execution, batched commitments, and public settlement, analogous to Layer-2 rollups but without asserting state-transition correctness.The analogy concerns commitment architecture rather than execution semantics.
  • Verification model: On-chain anchoring provides non-interactive verification after finalization, allowing any party with the disclosed payload and Merkle proof to verify independently.Unlike challenge-based designs, it requires no dispute window or counterparty interaction.
  • Deployment trade-offs: Permissionless chains maximize external verifiability, while sequencer failure can delay new anchors without invalidating previously anchored commitments.Force inclusion can preserve liveness, and anchoring frequency remains constrained by chain and sequencer throughput.
  • Comparison of mechanisms: On-chain anchoring is the only compared mechanism that avoids pre-agreement on a trusted intermediary and supports independent verification after operator failure.Other mechanisms require prior trust in an operator key, transparency log, or timestamping authority.
  • Comparison of mechanisms: Internal audits may use WORM hash-chain storage, while transparency logs or timestamping authorities may be simpler for bilateral relationships with pre-agreed auditors.Public-chain anchoring is justified when disputing parties lack a pre-committed neutral intermediary.
  • System components: The local recorder serializes tool calls, computes chain and Merkle links, and preserves logical ordering with optional concurrency metadata.The anchorer batches events and the verifier returns verified, tampered, or incomplete outcomes.

VI. EVALUATION

The evaluation studies cumulative integrity layers across synthetic workloads, real SWE-agent traces, and live anchoring. It measures overhead, anchoring cost, tamper detection, forensic precision, and dispute verifiability, while acknowledging that alternative anchoring mechanisms were not implemented as baselines.

  • Research questions: The evaluation asks about per-call latency and storage, anchoring cost and scaling, four tamper classes, and forensic query precision and dispute verifiability.These questions correspond to RQ1 through RQ4.
  • Workloads: Synthetic workloads model 100, 1K, or 10K tool calls across eight categories, with approximately 15% concurrent events and seed 42 for reproducibility.The workload includes file, shell, API, database, retrieval, code, approval, and delegation actions.
  • Evaluation data: The primary evaluation uses synthetic workloads, with RQ1 validated on five real SWE-agent traces containing 38 events and RQ2 measured through a live Base Sepolia deployment.The experiments use 30 runs with different seeds and a single Apple M-series machine.
  • Experimental setup: Five cumulative configurations progress from plain JSON logging to schema, hash chain, Merkle batching, and full on-chain anchoring.The transition from Baseline to Schema changes both event structure and serialization format.
  • Anchoring cost: Anchoring cost is evaluated across epoch sizes of 10, 50, 100, 500, and 1000 events.This sweep addresses how cost scales with epoch length.
  • Evaluation scope: The study does not include signed-digest or TSA-timestamped logs as non-blockchain integrity baselines.Head-to-head empirical comparison with deployed alternatives is left for future work.

C. Performance Overhead (RQ1)

The evaluation measures per-event latency and storage across cumulative integrity layers, showing that the full recorder remains low-latency while adding structured integrity metadata. Real SWE-agent traces preserve the overhead pattern, with lower absolute latency because they populate fewer fields.

  • Latency: 48.8 µs median latency for Merkle batching adds negligible median cost over Chain’s 48.2 µs, but epoch sealing raises P99 latency to approximately 4.1 ms.The Full configuration has a 47.6 µs median, within noise of Merkle, while transaction preparation raises P99 to approximately 30 ms locally.
  • Storage: 512 bytes/event in Full versus 272 bytes/event in Baseline represents an 88% storage increase from schema, hash pointers, and epoch metadata.
  • Full system: 48 µs median per-event latency and 512 bytes per event summarize the full integrity stack’s overhead.Transaction signing adds approximately 26 ms at epoch boundaries, amortized over each epoch and intended to run asynchronously in production.
  • Real-trace validation: 14 µs median overhead over a 2.4 µs Baseline appears on five SWE-agent traces, with signing overhead at P95.These traces contain fewer populated fields and no delegation or approval events than the synthetic workload.

D. Anchoring Economics (RQ2)

The study evaluates anchoring cost against compromise-window duration and tests tamper detection across attack classes and cumulative configurations. L2 anchoring makes short windows inexpensive, while Merkle batching and anchoring extend detection from chain breaks to terminal edits and forks.

  • Cost-window trade-off: $22.95 for 100K events at epoch size 10 on L2 supports a 10-second compromise window under the modeled 1 event/s workload.At epoch size 1000, L1 cost drops to $689 while the compromise window widens to approximately 17 minutes; bursty workloads shift the wall-clock window.
  • Tamper detection: 90% detection for Chain on edit and delete attacks rises to 100% with Merkle, because the final event has no successor hash link but remains a committed Merkle leaf.
  • Tamper detection: 100% detection across all attack classes is achieved by the full system, with zero false positives across configurations and attack types.Merkle closes Chain’s terminal-event gap, while anchoring enables equivocation detection by comparing forked epochs with anchored roots.

F. Forensic Queryability (RQ4)

The evaluation tests forensic queryability across synthetic multi-step failures and finds that structured fields sharply improve precision, while on-chain anchoring uniquely supports neutral third-party dispute verification.

  • Scenario A (prompt injection): Structured fields achieve 1.0 precision and recall on all five prompt-injection queries, unlike baseline text grep.Baseline guardrail grep achieves 1.3% precision, while structured delegation lookup reaches 1.0 versus 0.077 for grep.
  • Scenario B (production mis-targeting): Structured context_provenance.environment lookup identifies exactly the one configuration-read event, while baseline grep returns two staging-related events.This comparison concerns the production mis-targeting scenario.
  • Scenario C (cross-organizational dispute): Only the Full configuration with on-chain anchoring enables neutral third-party verification in the cross-organizational dispute scenario.All configurations resolve the dispute from logs, but alternatives require trusting one party’s key or log server.
  • Scenario A (prompt injection): 1.0 versus 0.013 precision on guardrail queries and 1.0 versus 0.077 on delegation queries demonstrates structured fields’ forensic advantage.The delegation query follows delegation.parent_event pointers, while grep matches unrelated delegation events.

VII. RELATED WORK

Prior work provides tamper-evident logging, inclusion proofs, signatures, tracing, and governance mechanisms, but the paper identifies a gap in jointly binding agent semantics with external anchoring for disputes.

  • Tamper-evident logging: Hash chains provide forward-integrity-style resistance to modifying pre-compromise entries, while Merkle trees provide logarithmic tamper-evident proof sizes.The paper builds on the Schneier–Kelsey threat boundary.
  • Existing audit systems: AWS CloudTrail provides digest-based integrity but trusts AWS as operator, an assumption that fails when the operator is a disputing party.
  • Transparency and authority: Certificate Transparency and related systems generalize Merkle inclusion proofs and auditability through designated log ecosystems or authorities.The cited systems include Sigstore, Trillian, timestamping, and keyless-signature infrastructure.
  • Blockchain-assisted auditing: Prior blockchain-assisted systems use local hash chaining with periodic commitments or generic governance, rather than per-action agent-semantic auditing.
  • Agent tracing: Mainstream tracing platforms capture prompts, tool spans, and execution flows for debugging but assume a cooperative operator rather than an externally committed evidence object.
  • Positioning: The paper positions its novelty as jointly providing an agent-semantic schema binding intent through execution and external anchoring for cross-organizational disputes.The cited passage introduces this as a combined gap in prior work.

VIII. DISCUSSION

The discussion bounds the recorder’s guarantees around completeness, cryptographic erasure, workload realism, cost assumptions, and post-compromise conditions, while identifying several future research directions.

  • Limitations: If actions bypass the instrumented gateway, the recorder cannot detect missing events within an epoch, although broken back-pointers reveal missing epochs.TEE-backed remote attestation is proposed as future work to make bypass structurally harder, not to make completeness absolute.
  • Privacy and compliance: Cryptographic erasure requires destroying or re-wrapping epoch-key material, and GDPR sufficiency depends on jurisdiction, metadata linkability, and retention policy.The paper presents erasure as an operational mechanism rather than a legal sufficiency claim.
  • Evaluation scope: Primary workloads are synthetic; RQ1 uses 30 runs, while RQ3 and RQ4 use single deterministic runs, and reported detection rates depend on the stated evidence model.Gas-price variation also makes dollar figures illustrative under stated assumptions.
  • Future work: The paper proposes ZK proofs, TEE-backed completeness, gas-adaptive anchoring, on-chain bonds, and a cross-vendor transparency-log standard as future directions.
  • Conclusion: The paper identifies on-chain bond mechanisms that incentivize third parties to audit agent behavior proactively as the most promising next step.
Loading 2609.01931v1…