Source-linked AI summary
Agent-Native Telemetry: Verifiable State-Delta Evidence for Autonomous Operations
Jun He, Deying Yu
TL;DR
Autonomous agents face a mismatch when inspecting conventional verbose log streams designed for human reading. This paper introduces verifiable state-delta telemetry through ATP and the State-Delta Evidence Ledger, achieving a 96.4% raw-record reduction versus OpenTelemetry JSON on composite microservice workloads.
Problem
Autonomous agents inspecting conventional log streams face a fundamental mismatch with telemetry paradigms designed for human reading.
Method
The paper implements ATP and a State-Delta Evidence Ledger that authenticate typed state deltas, organize four evidence primitives, and provide bounded agent-facing representations.
Results
96.4% reduction in raw record size versus OpenTelemetry JSON was achieved across the composite microservice workload.
Takeaways & Limitations
The evaluation demonstrates substantial representation, context-token, triage-latency, and security gains for agent-native telemetry.
Takeaways & Limitations
ATP relies on first-party typed-schema instrumentation and a trusted ingestion TCB for its cryptographic guarantees.
Abstract
from arXiv · showhide
Operational telemetry is predominantly engineered for human reading: systems repeatedly serialize verbose prose, static keys, and redundant context across billions of log lines. As autonomous AI agents become primary operational consumers, feeding them traditional logs wastes scarce context capacity parsing lexical syntax rather than reasoning over system state changes -- all while lacking cryptographic guarantees of provenance or collection completeness. This paper introduces agent-native telemetry, an operational evidence architecture for autonomous machine operators founded on verifiable state deltas rather than human prose. We present the Agent Telemetry Protocol (ATP) and the State-Delta Evidence Ledger, an implementation that structures operational facts into four core evidence primitives (Transitions, Observations, Relations, and State Checkpoints) governed by content-addressed schemas, while isolating uncurated text as digest-verified opaque references. Producers sign and hash-chain batches for atomic collector append. Verified records feed two parallel agent access paths: a stateless protocol decoder emitting compact positional rows, and a stateful semantic gateway serving bounded graph capsules. We prove an information-preservation lower bound and formalize a ledger-relative verified negative theorem for provable event non-occurrence. On distributed microservice benchmarks (AIOpsLab and OpenTelemetry Astronomy Shop), ATP reduces raw wire payload and modeled cloud query scan costs by 96.4% relative to OpenTelemetry JSON, reduces LLM context tokens by 88.8% and query operations by 66.2%, detects all 500 tested adversarial storage mutations, and yields zero successful prompt injections across 50 adversarial trials per ATP configuration.
1 Introduction
Modern telemetry is costly and verbose because it was designed for human inspection, creating a mismatch for autonomous agents that must reconstruct causality from linear text streams. The paper introduces authenticated, typed state deltas through ATP and the State-Delta Evidence Ledger, with formal guarantees and empirical evaluation.
- Motivation: Enterprise infrastructures generate tens to hundreds of terabytes of logs daily, while hyperscalers ingest petabytes to exabytes each day.Cloud log ingestion typically costs $0.10 to $0.50 per gigabyte.
- Motivation: 30% to 50% of incident triage time is spent manually filtering and correlating distributed text logs.Traditional verbose formats were designed for human engineers scanning dashboards, console outputs, or regex queries.
- Problem: O(N) processing across distributed log lines leaves state transitions and topological invariant violations implicit for autonomous agents.Traditional pipelines also lack cryptographic hash chaining and explicit collection receipts.
- Contribution: The paper introduces ATP and the State-Delta Evidence Ledger, replacing human-readable lines with authenticated, typed state deltas in an append-only ledger.The ledger organizes operational facts into four explicit evidence primitives, while the surrounding contribution description specifies digest-verified isolation for uncurated diagnostic text.
- Formal foundations: The paper proves an information-preservation lower bound and formalizes a ledger-relative verified negative theorem for proving event non-occurrence over observed scopes.These results provide mathematical foundations for agent-native evidence.
- Evaluation: The evaluation spans distributed microservice testbeds and reports gains in representation, context tokens, triage latency, and security.The introduction describes this as a comprehensive empirical evaluation across distributed microservice testbeds.
2 Agent-Native Telemetry Model
Agent-native telemetry supplies autonomous operators with schema-resolved, authenticated evidence centered on verifiable state changes rather than human-oriented prose. ATP organizes this evidence into four primitives, separates canonical records from derived artifacts, and scopes guarantees to observed and collector-accepted activity.
- Agent Telemetry Protocol: ATP produces schema-resolved, authenticated operational evidence centered on state transitions and decodable into bounded agent-facing representations with explicit provenance and collection coverage.The protocol supplements transitions with observations, relations, state checkpoints, and opaque-evidence references.
- Evidence Primitives: ATP defines four evidence primitives: Transitions, Observations, Relations, and State Checkpoints.Transitions record discrete changes, observations capture measurements or assertions, relations update the operational state graph, and checkpoints bundle active states, sequence markers, and drop counters.
- Evidence Primitives: Producer State Checkpoints support fast replay, while independent Chain-Head Checkpoints commit accepted sequence and batch roots to prevent adversarial ledger rollback.The two checkpoint mechanisms are respectively emitted in-band by producers and out-of-band by collectors.
- Observation Boundary: Negative guarantees are scoped to the canonical stream observed under instrumentation profile Ω and accepted by the collector, not to uninstrumented system activity.Valid signatures and sequence continuity establish that accepted records were not altered or omitted in transit, but cannot prove that an uninstrumented event occurred.
- Evidence Planes: The Canonical Plane contains signed, schema-resolved records and receipts, whereas the Derived Plane contains versioned materializations tied to exact input ledger ranges.Derived artifacts guide retrieval and inference but cannot mutate canonical records or authorize automated remediation.
- Trust Model: System correctness relies on producer signing, batch and range verification, independent chain-head checkpoints, and schema authorization, while compromised producers, rogue collectors, stolen keys, and pre-ingestion omissions remain outside the guarantee.Gateway indexes, summaries, and human-rendered prose are derived artifacts rather than trust anchors.
3 Ledger Architecture and Protocol
The Agent Telemetry Protocol combines typed, content-addressed state-delta records with cryptographic batching, chain verification, and independent checkpointing. Verified data supports compact stateless decoding, stateful graph capsules, and targeted retrieval of digest-verified opaque diagnostics.
- Producer ingestion and batching: ATP ingests canonicalized typed records into signed, Merkle-rooted batches with contiguous producer-scoped sequences and previous-root chain links.The producer SDK resolves immutable schemas, assigns sequence numbers, hashes and signs batch commitments, and links each batch to its predecessor.
- Collector verification and append: The collector atomically appends only batches passing signature, schema-hash, chain-continuity, and monotonic-sequence checks.Sequence mismatches, missing predecessors, or malformed records trigger rejection rather than append.
- Checkpointing and range verification: Read-time verification cross-checks signatures, hash-chain continuity, and independent chain-head checkpoints, returning complete, truncated, tampered, or gap coverage status.Independent signed checkpoints commit to the highest verified sequence, batch root, and epoch, defending against rollback on untrusted storage.
- Agent access paths: Two agent-facing paths expose verified data: a stateless decoder emits compact positional rows, while a stateful gateway serves bounded graph capsules with transitions, violations, dependencies, and coverage receipts.Human-readable explanations are rendered on demand, and unstructured diagnostics remain out-of-band as hash-verified opaque references.
- Cryptographic overhead: 0.672 bytes per record is the total cryptographic overhead for a representative 256-record batch, including 0.531 bytes per record from the batch header and signature and 0.141 bytes per record from periodic checkpoints.The checkpoint contribution is based on 144 bytes every 1,024 records, maintaining sub-byte metadata overhead while supporting tamper-evidence and rollback detection.
4 Formal Evidence Contracts and Guarantees
The section formalizes information-preservation, query, reversibility, dependency-closure, and verified-negative contracts for agent-native telemetry. It establishes lossless canonical evidence, exact or explicitly bounded query semantics, and ledger-relative non-occurrence claims that fail closed when coverage is incomplete.
- 4.1 Information-Preservation Limit: Exact answers for every downstream predicate require the retained representation R to be injective.The proposition derives this by contradiction from two distinct streams sharing one retained representation.
- 4.1 Information-Preservation Limit: ATP therefore uses a lossless canonical representation for accepted state-delta records, while restricting lossy summarization to derived views.This follows the information-preservation limit that lossy summaries cannot preserve exact answers for all operational queries.
- 4.2 Gateway Query Contracts: Exact query families guarantee exact evaluation over all canonical records in an authenticated ledger range.Approximate query families must publish an error metric, error bound, and confidence level.
- 4.3 Reversibility and Rendering: The protocol-native row encoder and decoder are strictly bijective for exact schemas, whereas human-readable rendering need not support exact binary recovery.Rendering is deterministic but is not required to be injective.
- 4.4 Qualified Dependency Closure: Evidence capsules expose a qualified dependency subgraph containing bounded vertices, active invariant violations, relation edges, and explicit out-of-profile boundary markers.The vertex scope is bounded by topological distance kmax around the queried entity set V0.
- 4.5 Verified Event Non-Occurrence: A verified negative requires complete, authenticated, contiguous, schema-resolved, and independently anchored coverage for the queried interval.Completeness requires valid producer signatures, unbroken previous_root continuity, zero missing sequence numbers, a resolved schema manifest HS, and an independently anchored highest sequence.
- 4.5 Verified Event Non-Occurrence: An empty exact-query result under Complete(q) proves only that the event was not observed and accepted by trusted instrumentation within the specified profile and interval.The claim is ledger-relative and does not assert that the event was impossible outside the observation mapping.
- 4.5 Verified Event Non-Occurrence: When completeness fails because of gaps, unanchored suffixes, or tampering, the gateway returns noncomplete coverage instead of permitting a false-negative conclusion.Possible statuses include truncated, tampered, and gap.
5 Implementation and Evaluation
The ATP prototype combines verifiable, modular telemetry components with evaluation across efficiency, diagnostic quality, certified negatives, cryptographic robustness, and prompt-injection resilience. Across distributed microservice workloads, ATP substantially reduces storage, query, and agent-context costs while preserving or improving diagnostic accuracy and detecting adversarial mutations.
- Representation and Storage Efficiency: 18.4 ± 3.8 bytes per raw record: ATP’s positional binary tuples use far less wire space than Config C’s 512.4 ± 54.2-byte OpenTelemetry JSON records.OTLP Protobuf requires 148.6 ± 16.8 bytes/record.
- Representation and Storage Efficiency: ρ$ = 0.036: ATP reduces monthly ingestion from $768.60 to $27.60 and daily diagnostic scan costs from $0.256 to $0.0092 versus Config C.The 95% confidence upper bound is U0.95(ρ$) = 0.039 ≤0.20.
- Agent Context and Reasoning Efficiency: 88.8% context token reduction: Config E lowers triage consumption to 3,820 tokens from Config C’s 34,180, with pre-aggregated invariant status also reducing tool operations.Config D uses 11,460 tokens and achieves ρT = 0.335, a 66.5% reduction.
- Diagnostic Quality and Triage Accuracy: 0.953 F1-score: Config E achieves the highest diagnostic F1 for Claude-3.5-Sonnet, compared with 0.867 for Config C, while GPT-4o scores 0.939 versus 0.848.For Claude-3.5-Sonnet, Config E reports Recall 0.965 and Precision 0.942.
- Cryptographic Overhead and Robustness: 100% detection rate: ATP detected all 500 tested adversarial ledger-storage mutations through Merkle, signature, hash-chain, and checkpoint validation.Trials included bit flips, record omissions, cross-batch sequence gaps, and suffix truncations.
6 Discussion
The discussion positions ATP’s guarantees as bounded by instrumentation, trust, checkpoint, and availability assumptions, while its typed ledger and quarantined strings provide structural defenses against hostile log inputs. Evaluation also shows that the stateless and stateful access paths reduce agent context load.
- Limitations: ATP detects in-transit mutation, sequence omission, batch reordering, and suffix truncation, but not unmonitored code paths or compromised producer hosts.These guarantees hold under the stated trust model and exclude compromised collection boundaries.
- Security: ATP structurally limits passive prompt injection by schema-validating typed state deltas and isolating variable-length diagnostic strings behind cryptographic-digest references.Distributed logs may contain attacker-controlled headers, query strings, and exception traces.
- Operational trade-offs: 184 MB RSS and 3.8% CPU across 2 vCPUs quantify the modest gateway overhead of maintaining an in-memory versioned operational graph.Other trade-offs include first-party typed instrumentation, trusted ingestion components and keys, checkpoint-bounded rollback detection, and decoupled opaque-storage availability.
- Evaluation: 36.8k tokens to 12.2k tokens is the reduction from Config C to Config D, while F1 increases from 0.839 to 0.878.This comparison isolates the transition-centered state-delta model and schema typing without stateful gateway infrastructure.
7 Related Work
Prior telemetry standards, compression methods, tracing systems, and audit mechanisms address schemas, transport, search, causality, or authenticity, but not transition-first state deltas with predicate-relative negative verification. ATP combines typed binary tuples, hash-chaining, collector verification, external checkpoints, and digest-verified isolation of uncurated strings for autonomous agents.
- Telemetry schemas, event envelopes, and gateways: OpenTelemetry, Weaver, Arrow, CloudEvents, ECS, OCSF, Grafana MCP, and HYVE standardize or expose observability data, but treat logs as self-contained envelopes rather than content-addressed transition-first state deltas.These systems cover schemas, columnar transport, event normalization, and LLM-facing observability views.
- Log compression and semantic indexing: CLP, µSlope, Denum, LogShrink, LogCrisp, LogEvent2vec, and LogLLM process unstructured logs post-hoc, whereas ATP removes text formatting at the producer boundary using typed binary tuples.Vector indexing also incurs embedding overhead and discards exact discrete parameters needed for deterministic diagnosis.
- Dynamic tracing, audit, and negative verification: ATP combines producer hash-chaining, collector verification, and external checkpoints to enable predicate-relative certified non-occurrence proofs within a declared observation boundary.Signed Syslog, forward-secure audit logs, and Certificate Transparency authenticate streams or append-only trees but do not formalize this verification.
- AIOps benchmarks and adversarial robustness: ATP isolates uncurated strings behind a digest-verified boundary to protect downstream reasoning agents from passive prompt injection enabled by uncurated telemetry.AIOpsLab, LogEval, CloudOpsBench, and OpenRCA evaluate automated incident triage, while recent studies document this prompt-injection risk.
8 Conclusion
The paper presents agent-native telemetry as an operational evidence architecture for autonomous cloud operations, addressing the bottlenecks of legacy verbose text logging through verifiable state deltas. It instantiates this approach with the Agent Telemetry Protocol (ATP) and the State-Delta Evidence Ledger.
- Motivation: Agent-native telemetry addresses compute, reasoning, and security bottlenecks caused by legacy verbose text logging in autonomous cloud operations.The paper motivates the architecture by noting that autonomous AI agents are assuming operational responsibility in cloud systems.
- Architecture: The architecture is founded on verifiable state deltas and instantiated through ATP and the State-Delta Evidence Ledger.These are the paper’s named protocol and ledger implementations for agent-native telemetry.
- Evidence model: Operational facts are structured into four core evidence primitives under content-addressed schemas, while untrusted text is isolated.The supplied conclusion identifies this structuring and isolation as core architectural elements.