Source-linked AI summary
The Acknowledgment Point Is the System: Durable Policy-Decision Receipts for AI Audit Evidence
Neeraj Kumar Singh Beshane
TL;DR
AI audit systems need to state when evidence actually exists, because acknowledging decisions before durable writes cannot guarantee survival after an immediate crash. RuntimeGuard-AI binds decisions to policy sources, commits records at selectable synchronization boundaries, and returns signed receipts, while measuring the resulting durability-latency trade-off. Buffered evidence reaches 27,193 requests/s at 141.9 µs median latency, whereas synchronization reduces throughput to approximately 242 requests/s and raises median latency to 16.0 ms.
Problem
AI audit systems lack an explicit acknowledgment boundary that truthfully distinguishes evidence existence from pre-durability acknowledgment.
Method
RuntimeGuard-AI binds deterministic decisions to source-bound records, commits them at caller-selected synchronization boundaries, and returns signed receipts with strict recovery and epoch verification.
Results
27,193 requests/s at 141.9 µs median latency for buffered signed evidence falls to approximately 242 requests/s with 16.0 ms median latency under synchronization.
Takeaways & Limitations
Durability is a machine-checkable interface property whose storage cost must be stated and measured rather than treated as free asynchronous auditing.
Takeaways & Limitations
The prototype does not prevent a signer from suppressing unsealed records or forking views without client head comparison or an external witness.
Abstract
from arXiv · showhide
An AI audit record is useful only if its durability and trust boundary are explicit. Returning a guarded decision before any durable write minimizes latency, but it cannot guarantee that evidence survives an immediate crash. We rebuild RuntimeGuard-AI around this constraint. The resulting research prototype binds each deterministic policy decision to the exact policy source, commits a privacy-minimizing record at a caller-selected synchronization boundary, and returns an Ed25519-signed receipt that states whether that boundary completed. After restart, the engine validates framed records, manifests, shard placement, sequence continuity, and replay identity. A separate attestation path groups committed records into chained, signed Merkle epochs that an auditor verifies with an externally obtained key. On an Apple M4 Pro at four worker threads and 2,048-byte prompts, buffered signed evidence reaches 27,193 requests/s with 141.9 microseconds median latency. Per-record data and full synchronization reduce throughput to approximately 242 requests/s and raise median latency to 16.0 ms. Sealing a 100,000-record signed epoch takes 97.0 ms. The result is a measured durability-latency trade-off, not a "free" asynchronous audit path. The prototype does not prove model execution, prevent a compromised signer from forking history, or establish legal conformity.
1 Introduction
RuntimeGuard-AI V2 treats the acknowledgment point as the system boundary for truthful audit evidence: buffered responses may lose records after a crash, whereas synchronized responses make durability explicit. The prototype replaces unreproduced proof architecture with a testable protocol centered on signed commit receipts, strict replay validation, Merkle epochs, and benchmarked synchronization semantics.
- Motivation: A fast asynchronous response releases an action while its evidence remains volatile, so an intervening crash can erase the record.Waiting for storage synchronization makes the durability claim meaningful but adds storage latency to the request path.
- Design correction: The successor removes an unrelated proof circuit after the released prototype failed to reproduce the prior paper’s asynchronous zero-knowledge attestation and performance claims.It rebuilds the smallest implementation whose claims can be tested end to end.
- Protocol redesign: RuntimeGuard-AI V2 binds each signed commit receipt to the request identifier, commitment, committed record, sequence number, durability bit, and verification key.The receipt exposes the evidence boundary as part of the returned interface.
- Protocol redesign: In data- and full-synchronization modes, the engine returns the receipt only after the configured host synchronization call succeeds, while buffered mode marks the record as not durable.This makes durability a machine-checkable property rather than an implied one.
- Contributions: The work contributes source-bound deterministic commitments, explicit synchronization and replay semantics, strict restart validation, signed Merkle epochs, end-to-end verification, and a source-hashed component benchmark.The listed protocol features include idempotent replay, fail-stopped write errors, and chained Ed25519-signed Merkle epochs.
2 Problem and Guarantee Boundary
RuntimeGuard-AI cannot guarantee crash-surviving evidence while acknowledging before any durability operation. Its durability claims are conditional on the selected synchronization boundary and trusted components, with explicit exclusions for stronger adversaries and rollback scenarios.
- Problem: Acknowledging before volatile write and host synchronization can leave evidence lost after an immediate process or power failure.Therefore, crash-surviving evidence cannot be guaranteed when acknowledgment precedes both durability operations.
- Guarantee boundary: RuntimeGuard-AI offers buffered, data-sync, and full-sync modes, returning durable=false only for buffered operation and durable=true after the selected sync.The data-sync and full-sync guarantees depend on documented operating-system, filesystem, controller, and device semantics, not remote replication or rollback immunity.
- Trust assumptions: A retained signed receipt enables a client to challenge an operator for the matching committed record and inclusion proof.The trust model includes the loaded policy, executing binary, storage semantics, uncompromised receipt and epoch keys, and independently obtained verification keys.
- Limitations: The prototype excludes root adversaries replacing code, keys, logs, and verifier configuration together, older valid-snapshot rollback, signer forks without external observation, and distributed exactly-once execution.It also excludes guarded-call bypass and key revocation; hash commitments bind data but do not make low-entropy fields confidential.
3 Protocol
The protocol separates synchronous, caller-selected durable acknowledgment from asynchronous epoch attestation while binding each decision to its compiled policy source and a privacy-minimizing commitment. Recovery and audit verification validate framing, manifests, sequence continuity, signed epoch chains, and Merkle inclusion, but do not establish global absence of forks or omitted requests.
- Policy binding and privacy: Each compiled policy binds canonical source bytes to a descriptor containing its id, version, and SHA-256 digest, which the evaluator consumes directly.The compliance record stores commitments and selected metadata rather than raw prompts or input payloads.
- Request commit protocol: For each request, the engine evaluates policy, assigns a global sequence and shard, appends one checksummed frame, synchronizes, and then signs the receipt.Fixed binary encodings define cryptographic commitments, while JSON is only the storage encoding.
- Failure and ordering: Append or synchronization failure puts the engine in a fail-stopped state rather than consuming a sequence and creating a recovery gap.The commit mutex serializes sequence assignment and append so later records cannot overtake an incomplete append.
- Recovery validation: On restart, the engine validates the manifest and complete frames, then rejects corruption, invalid commitments, misplaced shards, duplicate identifiers, or any sequence gap.Only an incomplete final frame may be truncated; recovered records must form the exact global sequence prefix 0, . . . , N −1.
- Epoch attestation and audit: Signed Ed25519 Merkle epochs bind roots, ranges, policy descriptors, times, counts, and predecessor hashes; auditors require an external trusted key and verify chain adjacency and inclusion.Without an external witness, verification proves integrity of an observed epoch chain, not global absence of forks or omitted pre-seal requests.
- Synchronous commit and asynchronous attestation: The synchronous path returns only after its selected host synchronization boundary, while epoch construction and audit verification operate outside that path.This separation isolates expensive batch attestation without eliminating storage cost from durable acknowledgment.
4 Implementation
The Rust prototype separates inline policy-and-receipt handling from attestation, while security-property tests cover recovery, concurrency, replay, authorization, and Merkle-chain behavior. It omits the dummy Groth16 circuit, so signatures authenticate trusted statements but do not prove model inference or arbitrary policy execution.
- Architecture: The Rust workspace separates inline policy compilation, deterministic commitments, sharded logs, receipts, restart validation, and benchmarks from attestation’s Merkle-tree and signed-epoch functions.The inline crate also owns the writer lease and recovery benchmarks; the attestor crate handles chain and inclusion verification.
- Security-property tests: Security-property tests cover tail truncation, frame corruption, manifest and sequence validation, replay identity, concurrency, leases, source binding, receipt keys, Merkle proofs, signing keys, and epoch continuity.The tests also check receipt-to-epoch inclusion and statement mutation.
- Limitations: The implementation omits the dummy Groth16 circuit, and signatures authenticate trusted statements without proving model inference or arbitrary policy code executed correctly.Proving execution would require trusted evaluator measurement or a proof relation encoding it.
5 Experimental Method
The experiment measures signed-evidence and synchronization costs, scaling across worker counts, prompt sizes, epoch sizes, and recovery sizes. It uses repeated randomized trials, paired baselines, bootstrap confidence intervals, and strict artifact-integrity checks.
- 5 Experimental Method: Four research questions assess signed-evidence and synchronization costs, worker and prompt-size effects, epoch and proof scaling, and log opening and recovery.The study evaluates policy evaluation and closed-loop component paths alongside epoch construction, proof generation, inclusion verification, signature verification, log opening, and recovery.
- 5 Experimental Method: The full inline matrix crosses four modes, three worker counts, three prompt sizes, and 20 repetitions.Each repetition uses 200 warmup and 1,000 measured requests; epochs span 100 to 100,000 records, while recovery spans 1,000 to 100,000 records, with five warm-ups and 30 measured repetitions per scale.
- 5 Experimental Method: Per-request latency and whole-condition elapsed time yield p50, p95, p99 latency, wall-clock throughput, medians, and deterministic 10,000-resample percentile-bootstrap 95% confidence intervals.Evidence overhead is paired with the policy-only repetition sharing worker count and prompt size.
- 5 Experimental Method: Strict provenance controls prevent reused output directories, verify exact tool versions and source hashes, and invalidate the corpus when source drift occurs.The runner performs formatting, warnings-as-errors linting, all-target tests, and dependency audit before measurement, hashes every raw artifact, and the analyzer verifies those hashes before summaries or figures.
6 Results
Results show that durability synchronization dominates inline cost, while serialization limits synchronized throughput and recovery scales near-linearly with retained records. Epoch sealing grows with batch size, but proof and signature verification remain microsecond-scale.
- Canonical run validation: All 720 inline conditions, four epoch scales, and three recovery scales completed, with the end-of-run source digest matching the starting digest and all artifact hashes verified.Each Table 1 estimate is the median over 20 independent randomized repetitions under the preregistered four-thread, 2,048-byte condition.
- RQ1: durability dominates the cost: 141.875 µs median latency and 27,192.808 requests/s characterize buffered signed evidence, whereas data and full synchronization reach 16,018.646 and 16,009.136 µs and 240.123 and 244.416 requests/s.Policy evaluation alone is sub-microsecond at the selected condition.
- RQ2: serialization saturates rather than scales: 30,884, 27,193, and 30,094 requests/s are buffered throughputs at 1, 4, and 8 threads, while per-record synchronized throughput remains near 243 requests/s.Synchronized median latency grows from approximately 4 ms at one thread to 16 ms at four and 32 ms at eight because one commit lock preserves global order.
- RQ3: epoch construction scales with batch size: 96,954.958 µs (97.0 ms) is the build-and-sign time for a 100,000-record epoch, while inclusion-proof and Ed25519 statement verification take 3.979 µs and 25.75 µs.Seal time grows from 0.116 ms at 100 records to 97.0 ms at 100,000 records.
- RQ4: recovery is near-linear in retained records: 665.483 ms is the time to open and validate 100,000 records, with 295.487 ms for reading and globally sorting them, yielding a 961.0 ms combined recovery path.Corresponding open times are 6.8 ms at 1,000 records and 66.2 ms at 10,000 records.
7 Related Work
RuntimeGuard-AI V2 builds on established tamper-evident logging and broader AI accountability infrastructure rather than claiming those mechanisms as novel. It provides a local evidence mechanism with externally anchored verification keys, while treating transparency services and regulatory materials as context rather than proof of compliance.
- Secure and tamper-evident logging: Forward-secure logs, Certificate Transparency, and Nitro establish prior art for audit logs, Merkle commitments, signed tree heads, consistency mechanisms, and tamper-evident logging.The paper explicitly excludes novelty claims for Merkle inclusion, signed roots, and tamper-evident logging.
- Transparency and trust operations: Sigstore shows that operationally meaningful signatures require identity, trust roots, transparency services, and monitorable public state.RuntimeGuard-AI V2 instead uses externally anchored verification keys and predecessor-linked epochs without public service, gossip, witness, or revocation protocols.
- Transparency and trust operations: RuntimeGuard-AI V2 should therefore be read as a local evidence mechanism, not a transparency system.Its design includes externally anchored verification keys and predecessor-linked epochs but omits public transparency infrastructure.
- AI accountability infrastructure: Related AI accountability systems address infrastructure gaps, lifecycle-wide LLM audit trails, autonomous-agent governance, skill-artifact registries, and training-data property attestation.The cited systems include Ojewale et al., lifecycle-wide LLM audit trails, Aegis, SIGIL, and property attestation.
- Regulatory context: EU AI Act Articles 12 and 14 and the NIST AI RMF provide governance context, while logging alone does not satisfy oversight obligations or establish legal conformity.Article 14 covers human oversight, including competence, authority, interpretation, intervention, override, and automation-bias risks; the NIST AI RMF is not a conformance certificate.
8 Limitations
The prototype’s evidence guarantees are bounded by a narrow, single-host evaluation and trusted implementation assumptions. Durable ordering, identity assertions, and signed history remain vulnerable to specific operational and cryptographic limitations.
- Evaluation scope: The evaluation covers one deterministic regex policy fixture in a closed-loop, single-host component benchmark, excluding network latency, saturation, availability, energy, key management, and production SLOs.It does not evaluate arbitrary policy languages or model inference stacks.
- Ordering and identity: Writer leases and a commit mutex serialize durable ordering; shards distribute files without creating parallel sequence authorities.The system records, but does not authenticate, caller-supplied model identity, user identity, or timestamps.
- History integrity: Signed receipts and observed epochs expose some omissions, but privileged signers can suppress unsealed records or fork views without client head comparison or an external witness.Whole-directory rollback is out of scope, while key rotation, revocation, hardware isolation, and public checkpoint publication remain future work.
- Execution claims: No cryptographic relation proves policy or model execution, and exact-source binding does not protect against replacement of the trusted implementation.The binding prevents accidental or caller-driven mislabeling inside that implementation.
9 Conclusion
RuntimeGuard-AI V2 makes the acknowledgment boundary explicit as a signed interface: returns before durable evidence must disclose that status, while durability claims incur measured storage cost. It couples this boundary to strict recovery invariants and independently verifiable signed epochs.
- 9 Conclusion: RuntimeGuard-AI V2 turns the acknowledgment boundary into a signed interface for stating whether durable evidence exists.The system must disclose when it returns before durable evidence is available.
- 9 Conclusion: Durability claims require paying and measuring the corresponding storage cost.The conclusion frames durability as a resource commitment rather than an implicit property of early acknowledgment.
- 9 Conclusion: The prototype couples acknowledgment handling to strict recovery invariants and extends committed records into independently verifiable signed epochs.These mechanisms define the system’s accountability and verification boundary.