PaperlayerView exact revision on arXiv

Source-linked AI summary

Serving a Revisable World: Versioned Execution for Interruptible Agents

Yanxin Zhang, Rahul Sharma, Nitin Vegesna, Zheyu Fu, Chang Liu, Trivikram Krishnamurthy

arXiv:2610.01160v1cs.DC

TL;DR

Agent revisions leave servers needing to suppress obsolete effects while preserving state that remains useful to a successor. RETIRE coordinates version authority, bounded retirement, and certified inheritance; paired experiments report a median 17.1% lower revision-to-successor TTFT than native abort and cold restart.

  • Problem

    Revisions require servers to stop obsolete output and work while allowing a successor to reuse valid completed state.

    Source: §1 · p. 1

  • Method

    RETIRE separates request resource ownership from execution-version authority and coordinates revocation, bounded retirement, certified state inheritance, and asynchronous reclamation.

    Source: §1 · p. 2

  • Results

    Median 17.1% lower paired revision-to-successor TTFT than native abort and cold restart; recorded-burst replay publishes no obsolete output and advances every final version.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Versioned execution makes a plan change a coordinated handoff that retires obsolete work while preserving useful state for its successor.

    Source: §9 · p. 10

Abstract

from arXiv · show

LLM agents revise running tasks when users change instructions, tools fail, or new information changes a plan. Today's servers express a revision as aborting old requests and submitting replacements. Yet the old execution's buffered output and outstanding work must stop affecting the application, while completed KV state may still be useful to its replacement. Handling these obligations separately can leave obsolete effects publishable and force the successor to rebuild valid state. We present \retire, a serving control-plane redesign around \emph{versioned execution}. Requests own scheduling and memory resources; execution versions own \emph{authority}, the permission to publish output or install state for the current execution. \retire first revokes obsolete work, then bounds its remaining execution and certifies the completed prefix its successor can inherit. The successor runs from that state while isolated old resources are reclaimed asynchronously. This unifies fast invalidation and selective preservation in one version transition. We implement \retire in vLLM across output publication, GPU execution, KV handoff, tiered recovery, and distributed and multi-tenant serving. Correctness experiments verify current-version output and valid state inheritance across these paths. Combining invalidation with inheritance reduces revision-to-successor time-to-first-token by a median 17.1\% in controlled paired experiments. A replay of recorded coding-agent interruption arrivals emits no obsolete output and keeps every final version progressing through repeated revisions. \retire turns abort-and-restart into a coordinated handoff that stops obsolete work quickly and preserves useful work for its successor.

1 INTRODUCTION

Agent revisions require the server to revoke obsolete effects while preserving completed state that remains useful to a successor. RETIRE makes this a versioned handoff, coordinating invalidation, inheritance, and asynchronous reclamation.

1 INTRODUCTION

  • A revision must stop buffered output and obsolete model work while retaining a completed common-prefix state that can accelerate the successor.The server must carry out the runtime’s replacement decision across output, computation, and memory.Source: §1 · p. 1
  • Coarse abort-and-restart pays both for the predecessor’s remaining work and for rebuilding useful shared state, motivating one control plane for publication, retirement, and inheritance.The paper characterizes revisable execution through version transitions and their pressure, retirement horizon, reusable state, and scope width.Source: §1 · p. 2
  • Median 17.1% lower paired revision-to-successor TTFT than native abort and cold restart combines fast invalidation with certified inheritance.Under HBM churn, certified retention accelerates post-churn recovery by up to 6.87×/5.07× on H100 PCIe/GB200.Source: §1 · p. 2
  • Recorded interruption-burst replay publishes no obsolete output, advances every final version, and delivers 5.53× the current-version output per second of the best tested native quantum.Distributed and co-tenant experiments extend the same authority boundary across workers and shared resources.Source: §1 · p. 2
  • RETIRE separates request-owned scheduling and memory resources from version-owned authority, coordinating revocation, certified inheritance, and asynchronous reclamation.Its sequence is ADVANCE, FENCE, ALIGN, RESUME, then RECLAIM; old resources are reclaimed off the successor’s critical path.Source: §1 · p. 2
  • The design is implemented across vLLM’s output publication, GPU execution, KV installation, state transfer, and reclamation lifecycle.The evaluation establishes correctness for publication, inheritance, and tenant isolation.Source: §1 · p. 2

2 CONVERGING TRENDS AND RELATED WORK

Related work treats revisions, preemption, scheduling, and persistent state as connected serving concerns. RETIRE adds a version authority boundary to coordinate which work loses authority and which retained state a successor may adopt.

2 CONVERGING TRENDS AND RELATED WORK

  • Abort followed by cold restart incurs both obsolete-tail delay and prefix recomputation; RETIRE shortens the tail with FENCE and preserves the certified prefix through ALIGN.The figure describes the two restart costs and the corresponding retirement and prefix-preservation mechanisms.Source: Figure 2 · p. 3, §2 · p. 3
  • Agent-control studies treat replacement after new information as a recurring execution event, while state-sharing and tiering systems preserve state beyond individual requests.Matching tokens or bytes alone does not establish whether an in-flight writer or transfer still belongs to the selected execution.Source: §2 · p. 3
  • Revision complements preemption: instead of retaining the paused work’s identity, it lets a different successor take over after the old identity loses authority.FastServe, InferCept, and Agentix address preemption, state preservation, or agent-program scheduling across time.Source: §2 · p. 3
  • RETIRE unifies revocation, writer closure, state handoff, and asynchronous reclamation in one version lifecycle.The runtime’s replacement decision governs publication and retirement and determines which retained state the successor may adopt.Source: §2 · p. 3

3 REVISABLE EXECUTION AS A SERVING WORKLOAD

The paper defines an interruption as a runtime-selected replacement of a live execution and frames its serving costs in terms of revision pressure, retirement horizon, successor overlap, and scope width. Successful transitions suppress revoked output, preserve only valid state, and keep useful work off the successor’s critical path.

3 REVISABLE EXECUTION AS A SERVING WORKLOAD

  • An agent interruption replaces a runtime-selected scope at version e with version e+1, revoking the former version’s authority to publish output or install state.The successor runs from the revised context supplied by the runtime.Source: §3 · p. 3

3.1 The runtime selects the replacement

  • SUPERSEDE identifies the scope, current and replacement versions, revised context, and a contract governing acceptable inherited state.That contract includes numerical mode and required successor-output equivalence against clean execution; a pause retains the same authority, while terminal cancellation has no successor.Source: §3.1 · p. 3

3.2 A taxonomy of agent interruptions

  • A failed tool call can trigger a transition when the runtime abandons dependent work and selects a new plan; the server receives the replacement decision regardless of its trigger.Representative runtime decisions map to the same serving-level version transition.Source: §3.2 · pp. 3–4, Table 1 · p. 4

3.3 The workload exposed to the server

  • Revision pressure, retirement horizon, successor overlap, and scope width expose the server’s costs, from repeated arrivals and delayed safe boundaries to reusable state and coordinated workers.A long prefill or speculative step can delay replacement, while a retained repository prefix can save most of successor prefill.Source: §3.3 · p. 4
  • A successful transition prevents revoked work from publishing, gives the successor only contract-valid state, and keeps useful work and unrelated tenants isolated from obsolete work.These outcomes define current output, valid inheritance, and efficient handoff.Source: §3.3 · p. 4

4 FROM REQUEST-CENTRIC TO VERSIONED SERVING

Request identity tracks resources but cannot determine whether replaced work may still publish or install state. Versioned serving separates those roles and coordinates output suppression, bounded retirement, and state handoff.

4 FROM REQUEST-CENTRIC TO VERSIONED SERVING

  • Requests own schedulable work and allocations, while the current scope version determines which effects may become visible.The successor can inherit useful state without keeping the predecessor authoritative or waiting for all old resources to be released.Source: §4 · p. 4, §4.2 · p. 5
  • Abort and cold prefill serialize predecessor tail time and full successor prefill, whereas versioned execution uses a bounded safe point, alignment, and only the remaining suffix prefill.The decompositions expose the two levers: shorten obsolete work and avoid repeating valid work.Source: §4 · p. 4

4.1 Obsolete results can escape

  • Removing a request from the scheduler cannot retract tokens already queued for delivery, so the server must recheck the current version at publication.This prevents buffered output from an abandoned plan from reaching the application after the switch.Source: §4.1 · p. 4

4.2 Obsolete work can delay the replacement

  • Old GPU or distributed work may continue until a recognized boundary, but its output and state commits must lose authority immediately after revision.The successor needs writer-closed shared state and isolation from old-only state, not complete reclamation of predecessor allocations.Source: §4.2 · pp. 4–5

4.3 State handoff lacks versioned ownership

  • State handoff requires both eligibility and lifetime decisions: retain a completed compatible prefix until adoption, while recomputing unfinished or incompatible state.Releasing the old request and later seeking a cache hit is opportunistic because intervening requests may evict the blocks.Source: §4.3 · p. 5
  • Revocation, writer closure, certified handoff, and isolation connect publication, inheritance, and asynchronous reclamation in one version lifecycle.This coordinates the two performance levers without making old-resource cleanup determine successor output or state.Source: §4.3 · p. 5, §4.2 · p. 5

5 RETIRE: A VERSIONED SERVING CONTROL PLANE

RETIRE coordinates a version handoff around “revoke first, preserve second”: it closes obsolete publication and writers, certifies reusable state, then resumes the successor before reclaiming isolated resources.

5 RETIRE: A VERSIONED SERVING CONTROL PLANE

  • RETIRE’s rule is revoke first, preserve second: obsolete plans lose publication authority while useful state passes through an explicit handoff.Resources are released after their last users finish; only state the successor will consume must be ready before it starts.Source: §5 · p. 5

5.1 Three guarantees across the serving lifecycle

  • Three guarantees restrict visible output and installed state to the current epoch, inheritance to matching state with valid ownership, and retired writes from overwriting successor or tenant state.Source: §5.1 · p. 5
  • FENCE closes writers before ALIGN certifies the common prefix; when reuse is not profitable, a cold successor can bypass physical pinning.Source: §5.1 · p. 5

5.2 One five-stage transition

  • The five stages advance authority, fence old work, align certified state, resume the successor, and reclaim isolated resources after their final users drain.The dependencies are revoke before publication, close writers before inheritance, and isolate state before recycling; successor progress need not wait for every predecessor allocation.Source: §5.2 · p. 6

5.3 Resume before reclamation

  • The successor adopts the certified common prefix and runs with fresh suffix pages while old suffixes and buffered results remain unreachable and drain.The prefix stays pinned until adoption is verified; only immutable, writer-closed pages cross the boundary.Source: §5.3 · p. 6

6 A VERSIONED VLLM SERVING STACK

The vLLM implementation carries version authority through serving boundaries, gating publication and state adoption while bounding obsolete execution. Its KV handoff aliases certified prefix pages for the successor and reclaims isolated old state asynchronously.

6 A VERSIONED VLLM SERVING STACK

  • The vLLM implementation carries scope epochs across admission, scheduling, execution, publication, KV installation, and transfer, separating local resource IDs from result authority.Source: §6 · p. 6

6.1 Redesigning the execution lifecycle

  • Publication checks reject superseded output at delivery, while worker safe points bound obsolete launches and writer receipts establish the completed-state frontier.Long steps use additional device predicates and bounded checks to shorten retirement tails without per-layer checking costs on short steps.Source: §6.1 · p. 6
  • ALIGN certifies a writer-complete, block-aligned common prefix; RESUME aliases its HBM pages, while a failed certificate selects a cold, fresh-generation successor.The same certificate governs DRAM offload and restoration when HBM capacity must be returned before the successor arrives.Source: §6.1 · p. 6, Figure 4 · p. 7
  • Figure 3 summarizes the cross-layer transition, and the implementation extends it across speculation, multiple KV groups, memory tiers, tenants, and distributed participants.Source: §6.1 · p. 6, Figure 3 · p. 7

7 EVALUATION

The evaluation tests whether RETIRE suppresses obsolete effects, preserves valid successor state, and improves handoff performance across single-GPU, repeated-revision, distributed, and co-tenant settings. Correctness-checked experiments show faster handoffs and continued progress under revision bursts.

7.1 Experimental method

  • The evaluation uses vLLM, primarily Qwen3-32B in BF16, with GPT-OSS-20B for heterogeneous KV groups and trace replay; performance data pass correctness checks.The checker compares current-version output, KV ownership and transfer digests, and distributed receipts against a clean reference.Source: §7.1 · p. 7, §7 · p. 7

7.2 Correctness: closing the version lifecycle

  • Across 18 native publication trials, consumers receive 170 predecessor tokens after abort; RETIRE’s publication boundary suppresses them.Completed-predecessor pages match clean-prefix pages in ten seed–LCP configurations, while an injected content fault changes successor output in 3/3 trials.Source: §7.2 · pp. 7–8
  • RETIRE suppresses predecessor output and matches clean successor references across integrated prefill, speculative, and heterogeneous-cache trials.The tests also reject late completions independently of when the native request object disappears.Source: §7.2 · p. 8, Table 4 · p. 8

7.3 Performance: removing both transition costs

  • 17.1% median paired successor-TTFT reduction comes from combining invalidation and certified inheritance over native abort plus cold restart.The 8192-token prompt used a 4096-token prefill quantum, 90% overlap, and revision 1.1 seconds into prefill; every trial adopted 4096 certified tokens.Source: §7.3 · p. 8, Table 5 · p. 8
  • FENCE latency grows with execution granularity, while inheriting a certified 4080-token prefix reduces successor prefill from approximately 3965 ms to 117 ms.Across long prefill, draft–verify prefill, active speculative decode, and heterogeneous KV groups, FENCE improves by 24.9×, 51.3×, 9.99×, and 7.01×, respectively.Source: §7.3 · pp. 8–9
  • Under HBM churn, RETIRE retains all 7168 common-prefix tokens; recovery reaches 6.87× on H100 PCIe and 5.07× on GB200.Native caching loses the prefix at high churn; Figure 5 reports recovery after intervening HBM churn, with four paired quartets.Source: §7.3 · p. 9, Figure 5 · p. 9

7.4 Repeated revisions, distributed execution, and isolation

  • In the 120-second replay, RETIRE emits no obsolete output, advances all 9 final versions, and delivers 5.53× native’s current-version goodput.Cleanup closes in 1.08 ms; native abort advances only 3/9 final versions and leaves three streams at the 180-second cleanup limit.Source: §7.4 · p. 9, Figure 6 · p. 10
  • Distributed retirement publishes no obsolete output and matches clean oracles; across three topologies, FENCE improves 6.04–12.19× and inherited TTFT 5.27–7.08×.In a TP=4 co-tenant run, Tenant B’s KV digest and block IDs remain unchanged across three seeds.Source: §7.4 · p. 10, Table 6 · p. 10

8 IMPLICATIONS FOR REVISABLE SERVING

Revisions change which work remains useful, not merely which request runs next. The implications are a shared authority boundary for effects with different lifetimes, while policies adapt within that invariant.

8 IMPLICATIONS FOR REVISABLE SERVING

  • Fast invalidation stops spending resources on abandoned work, while inheritance reduces the work required by its replacement.A common authority boundary lets the server optimize both together.Source: §8 · p. 10
  • Output queues, issued computation, and reusable state need one logical version boundary, even when physical cleanup proceeds independently.Correctness depends on enforcing that boundary wherever effects become visible, not on making every object disappear at once.Source: §8 · p. 10
  • Revision pressure, prefix overlap, and admission delay can guide scheduling and memory policies while authority checks continue to govern publication and handoff.The paper explores the separation through rate sweeps.Source: §8 · p. 10
  • The runtime selects successor context and handles external actions; serving determines when it runs and which state it inherits.This division leaves publication, isolation, and handoff efficiency as serving responsibilities.Source: §8 · p. 10

9 CONCLUSION

RETIRE treats agent revisions as coordinated handoffs: execution versions carry authority while requests own resources. Its combined invalidation and inheritance improve handoff time and preserve progress across tested revisions and serving settings.

9 CONCLUSION

  • RETIRE coordinates fast invalidation, certified inheritance, and asynchronous reclamation under version authority distinct from request resource ownership.Combining invalidation and inheritance reduces paired revision-to-successor TTFT by a median 17.1%; replay emits no obsolete output and advances 9/9 final versions.Source: §9 · p. 10

A METHODOLOGY AND SUPPLEMENTARY EVIDENCE

The appendix records information needed to interpret and reproduce the main results.

A METHODOLOGY AND SUPPLEMENTARY EVIDENCE

A.1 Scope, Contract, and Integration

The runtime chooses the replacement, while the server enforces that decision across output, computation, and memory. RETIRE represents the authority change and state shared across versions in a serving transition.

A.1 Scope, Contract, and Integration

  • The runtime selects what replaces old execution; the server applies that choice across output, computation, and memory.The paper calls replacement of a live execution an agent interruption.Source: §A.1 · p. 12
  • The runtime supplies revised context and advances a monotonic epoch carried by requests, output, KV blocks, transfers, and worker operations.The identity also includes tenant, scope, and generation.Source: §A.1 · p. 12
  • The checker requires current authority at publication, matching writer closure and generation for inheritance, participant agreement for distributed transitions, and isolation of retired completions.It records ADVANCE, FENCE, ALIGN, RESUME, and RECLAIM milestones.Source: §A.1 · p. 12
  • EngineCore propagates version tags across scheduling, workers, attention, KV installation, and API output; prefills poll at bounded boundaries and tensor-parallel participants retire collectively.ALIGN checks the common prefix against writer frontiers, fingerprints, generations, tenants, numerical mode, and KV-group boundaries before the successor aliases certified blocks.Source: §A.1 · p. 12

A.1.1 Coverage of prior systems

The appendix evaluates versioned execution across reproducibility, recorded interruption traces, correctness checks, and supplementary performance campaigns. Its reported trials include zero obsolete publication, valid inheritance, and measured performance trade-offs.

A.2 Workloads and Reproducibility

  • Paired reproducibility trials use identical churn or arrival schedules across arms, and trace replay preserves recorded transition times while setting prefix overlap to zero.The revision-rate sweep crosses rates 0, 0.05, and 0.2 per scope per second with concurrencies 4 and 8.Source: §A.2 · p. 12, §A.2.1 · p. 12, §A.2.2 · p. 12

A.2.1 Recorded agent trace

  • Every versioned recorded-trace configuration completes cleanup, advances 9/9 final versions, and publishes zero obsolete tokens; both native configurations hit the 180-second cleanup limit and allow only 3/9 final versions to publish.The fixed one-month trace contains 751 transitions across 345 scopes; replay uses an 8192-token state with zero prefix overlap.Source: Table 9 · p. 12, §A.2.1 · p. 12

A.2.2 Revision-rate matrix

  • The revision-rate sweep spans 108 audited points, and its current-goodput-optimal native quantum shifts from 8192 tokens at zero or moderate revision pressure to 1024 at rate 0.2 for both concurrencies.The rate-only selector follows the same direction.Source: §A.2.2 · p. 12

A.3 Correctness Closure

  • Clean-prefix and completed-predecessor sources produce identical outputs in all ten seed–LCP configurations in both execution modes; the page-fault control changes output in 3/3 trials, and restoring certified bytes recovers exact output in 3/3.Batch-invariant execution also makes cold and reuse chosen-token log probabilities identical.Source: §A.3 · pp. 12–13
  • Versioned distributed trials publish zero obsolete output and match clean oracles; inheritance adopts the same 4080-token physical prefix, while superseding Tenant A leaves Tenant B's KV digest and block IDs unchanged.Across three TP=4 co-tenant seeds, Tenant B's median runtime and cross-interrupt frame gap change by 1.0% and 1.8%, respectively.Source: §A.3 · p. 13

A.4 Supplementary Performance Results

  • In the no-revision campaign, selective arming reduces the earlier always-armed decode overhead from 5.25% to 1.05% and safe points from 35,190 to 1,080.The campaign uses six loaded burn-in groups, two warmups, and ten paired groups per arm.Source: §A.4 · p. 13
  • All 288 executions produce identical greedy successor tokens, while HBM retention and DRAM offload recover the complete 7168-token prefix in every trial.The offload path returns 512 predecessor blocks and conditionally publishes 448 certified blocks under the successor generation after churn overwrites the old slots.Source: Table 12 · p. 14, §A.4 · p. 14
  • Across the distributed 2×1, 4×1, and 2×2 campaigns, RETIRE FENCE medians are 216.5, 130.6, and 443.6 ms, respectively, versus baselines of 2638.7, 1381.2, and 2680.0 ms.The campaigns use a 16,384-token retirement prompt and a 4080-token aligned inherited prefix.Source: §A.4 · p. 14

Related papers

Loading 2610.01160v1…