PaperlayerView exact revision on arXiv

Source-linked AI summary

DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback

Yunpeng Dong, Jingkai He, Shiqi Liu, Yuze Hou, Dong Du, Zhonghu Xu, Si Yu, Baochuan Yang, Yubin Xia, Haibo Chen

arXiv:2605.22781v2cs.OScs.AI

TL;DR

Frequent agent search and reinforcement-learning workloads need fast coupled checkpoint/restore, but existing full-state duplication is too slow. DeltaBox introduces DeltaState with change-based filesystem and process mechanisms, achieving millisecond-scale checkpoint/restore and substantially lower state-management overhead. Its scope includes coupled sandbox state management, while existing approaches remain constrained by VM-granular or incomplete rollback behavior.

  • Problem

    Frequent agent search and reinforcement-learning workloads require coupled checkpoint/restore of filesystem and process state, while existing full-state duplication is too slow.

  • Method

    DeltaBox introduces DeltaState with DeltaFS for layered change-based filesystem management and DeltaCR for incremental process-state handling with warm-template forking.

  • Results

    DeltaBox hides approximately 10.83 ms of checkpoint work under inference, restores from a template fork in approximately 1.86 ms, and reduces SWE-bench MCTS state-management overhead to 1–2%.

  • Takeaways & Limitations

    Change-based coupled checkpoint/restore enables DeltaBox to support high-frequency agent exploration with millisecond-level sandbox operations.

  • Takeaways & Limitations

    Existing alternatives remain constrained by incomplete coupled rollback or VM-granular operations whose cost grows with total memory.

Abstract

from arXiv · show

LLM-powered AI agents require high-frequency state exploration (e.g., test-time tree search and reinforcement learning), relying on rapid checkpoint and rollback (C/R) of the complete sandbox state, including files and process state (e.g., memory, contexts, etc.). Existing mechanisms duplicate the entire state, causing hundreds of milliseconds to seconds of latency per C/R, which severely bottlenecks deep search and large-scale fan-outs. This paper observes that subsequent checkpoints in AI agents are highly similar. Therefore, instead of full duplication, a sandbox should only duplicate the changes between consecutive checkpoints (Key Insight). However, it is non-trivial to realize the idea, mainly due to the missing OS supports. This paper proposes a new OS-level abstraction, DeltaState, to enable the change-based transactional C/R for AI agents with two co-designed OS mechanisms. First, DeltaFS enables change-based filesystem C/R by organizing the file states into layers and dynamically freezing the writable layer and inserting a new one during checkpoint, reducing file updates to copy-on-write, and making rollback a simple layer switch. Second, DeltaCR enables change-based process state C/R using incremental dumps, and accelerates rollback by bypassing traditional pipelines to directly fork() from a frozen template process. We then present DeltaBox, a novel agent sandbox achieving millisecond level C/R through the two new mechanisms. Evaluations on SWE-bench and RL micro-benchmarks show DeltaBox completes checkpoint and rollback in millisecond-level latency (14ms and 5ms, respectively), empowering agents to explore substantially more nodes under fixed time budgets.

1 Introduction

AI agents increasingly use search and reinforcement learning, but frequent coupled checkpoint/restore of filesystem and process state becomes a critical bottleneck. DeltaBox addresses this with change-based state management through DeltaFS and DeltaCR, reducing state-management overhead and enabling millisecond-scale operations.

  • Motivation: Tree search and reinforcement learning repeatedly require fast sandbox checkpoint/restore for exploration, backtracking, and parallel rollouts.MCTS backtracks across historical nodes, while RL launches multiple sandboxes from a shared warm state.
  • Problem: Existing systems duplicate complete sandbox state, making high-frequency checkpoint/restore prohibitively slow for agent workloads.Process checkpointing and restoration can take seconds for multi-GiB processes, while filesystem copying also incurs high latency.
  • Key Insight: Consecutive agent checkpoints differ only marginally, motivating change-based checkpoint/restore that duplicates only new files and modified memory pages.This is the paper’s key insight for reducing work between checkpoints.
  • Approach: DeltaFS manages filesystem changes with layered copy-on-write and constant-time layer switching, while DeltaCR uses incremental process-state handling and warm-template forking.The two mechanisms jointly support coupled filesystem and process-state checkpoint/restore.
  • Results: DeltaBox hides approximately 10.83 ms of checkpoint work under inference, restores from a template fork in approximately 1.86 ms, and reduces SWE-bench MCTS overhead from 23–48% to 1–2%.DeltaBox is implemented on a Firecracker microVM and evaluated on SWE-bench workloads.
  • Results: The paper reports orders-of-magnitude checkpoint/restore latency reductions over baselines on SWE-bench MCTS and reinforcement-learning fan-out workloads.The evaluation covers both test-time search and RL-oriented scaling scenarios.

2 Background and Motivation

Agent search strategies create coupled filesystem and process-state rollback demands that existing sandbox approaches handle inefficiently or incompletely. DeltaBox organizes these dimensions into a jointly managed state and uses layered filesystem storage plus process-state mechanisms for transparent rollback.

  • AI Agent Search Strategies: MCTS combines selection, expansion, evaluation, and backpropagation, while Best-of-N clones an initial sandbox and still requires intermediate rollback within each trajectory.Both strategies therefore depend on fast checkpoint/restore during exploration.
  • Agent Sandbox: A sandbox state includes durable filesystem contents and ephemeral process context, which must be restored jointly to preserve deterministic search behavior.Restoring only one dimension can leave stale memory or mismatched files.
  • Limitations of Existing Approaches: Existing filesystem-only tools discard process memory, while VM snapshots capture guest memory and device state without the block device and operate at VM granularity.These approaches require restart, replay, or separate filesystem snapshotting.
  • Limitations of Existing Approaches: Existing coupled approaches can require approximately 4 s per GiB of RAM for checkpointing, while CRIU restoration takes seconds for multi-GiB processes.Such costs are incompatible with high-frequency rollback in tree search.
  • Requirements: Efficient sandboxes require joint millisecond-scale state management, write amplification proportional to actual changes, O(1) arbitrary rollback, and agent transparency.Transparency includes avoiding code changes, forced restarts, and context loss.
  • DeltaBox Workflow: DeltaBox performs deltaCheckpoint on coupled filesystem and memory state, then restores by switching overlay layers and forking a warm process template or loading a CRIU chain.The StateManager coordinates this workflow for every search step.
  • System Architecture: DeltaBox’s architecture combines XFS reflink base storage, runtime-reconfigurable DeltaFS layers, and DeltaCR process-state management under a StateManager.Reflink defers physical block allocation until actual writes, supporting change-proportional storage work.

3 System Overview

DeltaBox implements change-based checkpoint/restore for coupled filesystem and process state, coordinated by a StateManager. Checkpoints persist only inter-checkpoint deltas, while rollback switches filesystem layers and restores process memory from a warm template or CRIU image.

  • DeltaBox duplicates only inter-checkpoint changes rather than the complete sandbox state.
  • The StateManager maintains each checkpoint as a consistent, atomic filesystem–memory pair.
  • During checkpointing, DeltaCR asynchronously dumps process state while DeltaFS synchronously installs a new filesystem layer.
  • Both checkpoint mechanisms observe the agent at the same SIGSTOP-quiesced instant, preserving filesystem–memory consistency while checkpoint latency is masked by LLM inference.
  • Rollback switches DeltaFS to the target layer configuration and restores process state through a live template fast path or CRIU lazy-pages fallback.

4 Detailed Design

DeltaBox combines runtime overlay-layer switching with fork-based process restoration to implement millisecond-scale, change-based sandbox checkpoint/restore. DeltaCR retains CRIU images as fallback while frozen templates accelerate restores, with NPD isolating external LLM I/O from forked state.

  • DeltaFS: DeltaFS reconfigures overlay layers at runtime without unmounting, freezing the current writable layer and inserting a fresh upper layer.
  • DeltaFS: DeltaFS uses generation tracking to handle files opened before checkpoints and reflink-enabled XFS to defer physical block allocation until writes.
  • DeltaCR: DeltaCR creates both a CRIU dump and a frozen template at each checkpoint, enabling fork-based restores with CRIU fallback.
  • DeltaCR: Template restoration shares pages copy-on-write, while an asynchronous warm thread pre-privatizes anonymous writable pages to absorb subsequent faults.
  • Network I/O: The Network Proxy Daemon keeps SDK threads and sockets outside the agent address space, making frozen templates safely forkable.
  • Network I/O: DeltaBox does not currently support network I/O rollback, which may cause external side effects.

4.3 StateManager Coupling Protocol

The StateManager coordinates host-side and guest-side components so checkpoint and restore preserve a consistent filesystem–memory pair. It also integrates DeltaBox with agent frameworks and deploys the system in isolated Firecracker microVMs.

  • The StateManager synchronizes DeltaFS and DeltaCR at a shared quiesced instant to ensure persisted filesystem and memory states match.
  • DeltaBox supplies physical-state checkpointing beneath logical checkpoints in frameworks such as LangGraph and LangChain.
  • Firecracker microVMs provide hardware-level isolation, while process-level checkpointing allows multiple independently checkpointable agents to share a kernel and read-only base layers.

5 Implementation

DeltaBox implements DeltaFS as a Linux 6.8 overlayfs extension and DeltaCR as a userspace daemon, coordinated by the StateManager inside Firecracker microVMs.

  • DeltaFS adds approximately 565 lines of C across four files to implement runtime layer management in Linux 6.8 overlayfs.

6.1 Experimental Setup

The evaluation uses SWE-bench Verified MCTS trajectories grouped into four archetypes and compares DeltaBox with baselines that capture both filesystem and process state.

  • Hardware: The hardware platform is a four-socket server with 96 physical cores, 192 hardware threads, and 760 GiB RAM, using a 400 GB NVMe SSD.RL fan-out additionally uses a single-node 4-GPU cluster with 96 GB per GPU.
  • Workloads: Experiments use four SWE-bench Verified MCTS archetypes: Django, SymPy, Scientific, and Tools/small repositories.Scientific includes Astropy, Matplotlib, scikit-learn, and Xarray; Tools/small includes pylint, requests, and pytest.
  • Baselines: The baselines include replay+cp, FC-Diff+dm, CRIU+cp, and E2B (diff), each coupling filesystem and process-state recovery.The baselines use copying, VM snapshots, CRIU, or incremental VM snapshots for the two state dimensions.

6.2 End-to-End Performance

DeltaBox keeps MCTS state-management overhead near the millisecond scale by masking checkpoint work and using template-fork restores, while its fork-based fan-out scales efficiently for RL workloads.

  • MCTS search throughput: DeltaBox hides asynchronous checkpoint dumping under LLM inference while keeping restore on the critical path in the millisecond regime.The checkpoint API's blocking interval excludes asynchronous dump completion, whereas restore must finish before the next MCTS iteration begins.
  • MCTS search throughput: E2B restore is ∼480× slower than DeltaBox, while FC-Diff reaches ∼1800× and replay reaches ∼15000×.DeltaBox restore combines an OverlayFS layer switch with template fork(), whereas other approaches perform reload, merge, or replay work.
  • MCTS search throughput: 23–48% of total time is state-management overhead for E2B (diff), compared with 1–2% for DeltaBox.These measurements come from end-to-end 30-iteration MCTS trajectory replays across four SWE-bench archetype groups.
  • MCTS search throughput: DeltaBox remains at 1.01–1.02× normalized end-to-end time, while E2B (diff) reaches 1.30–1.93×.Each system is normalized to its own LLM+action latency, where 1.0× represents free state management.
  • RL fan-out: Fork p50 grows sub-linearly from 0.57 ms at N=1 to 5.5 ms at N=64, with p99=14.7 ms at N=64.Children inherit pages copy-on-write, keeping per-child resident memory near 11 MB; aggregate footprint grows with the write working set.
  • RL fan-out: Across the measured range, DeltaBox is an order of magnitude or more faster than CubeSandbox and E2B for substrate-level fan-out.The comparison includes per-child filesystem isolation and inherited process-memory materialization.
  • RL fan-out: At N∈{16, 64}, DeltaBox sustains 95–97% GPU occupation, compared with 77–80% for CubeSandbox and 29–36% for E2B.In synchronous training, sandbox fan-out cost becomes GPU idle time alongside generation and training.
  • RL fan-out: At N=64, DeltaBox reaches staleness 0.81, while CubeSandbox reaches 1.25 and E2B reaches 5.04.The verl staleness threshold is below 1, so only DeltaBox is below that threshold at N=64 in this comparison.

6.3 Extended Studies

Extended studies show that DeltaBox’s fast checkpoint/restore paths overlap work with inference and absorb post-restore faults, while reflink-aware storage and reachability-aware GC reduce filesystem and snapshot overhead.

  • Checkpoint/Restore Latency: 10.83 ms of checkpoint local work overlaps the LLM-inference window, so the agent perceives no blocking.
  • Checkpoint/Restore Latency: Template-fork restore is the common fast path, while CRIU-lazy restore is reserved for first restore or template eviction.Fork-based restore shares pages with the frozen template; slow-path restoration handles template eviction.
  • Checkpoint/Restore Latency: Async-warm pre-pays copy-on-write faults off the critical path, preventing post-fork page-fault debt during realistic LLM idle windows.
  • Filesystem and Storage: Reflink-aware copy-up duplicates only dirtied 4 KB blocks, whereas ext4 and XFS without reflink recopy the whole modified file.Across real swe-search edits, reflink copy-up remains nearly constant as edited-file size grows.
  • Adaptive Optimization: Lightweight checkpoints skip the CRIU dump and layer switch for read-only, idempotent actions, but require replay during restore.The paper enables this optimization only when snapshot memory, rather than restore latency, is the bottleneck.
  • Filesystem and Storage: 46–63% reduction in end-of-trajectory dump storage versus retaining every checkpoint comes from asynchronous reachability-aware garbage collection.

7 Related Work

Related systems mainly optimize isolation, startup, or coarse-grained rollback, whereas DeltaBox targets repeated, fine-grained checkpoint/restore of complete agent state with OS-level mechanisms.

  • Existing agent sandboxes provide limited rollback granularity: Daytona misses live process state, while E2B and ZeroBoot retain VM-level snapshot or cloning costs.
  • A position paper identifies fast forking and fork-aware state management as gaps, while DeltaBox contributes a concrete millisecond-scale OS mechanism.
  • Crab uses an eBPF-based inspector for semantics-aware checkpoint granularity, whereas DeltaBox optimizes the full OS-level checkpoint/restore mechanism.
  • Serverless-oriented techniques accelerate one-shot instance restoration, while DeltaBox addresses repeated descent and backtracking within one task.

8 Conclusion

DeltaBox accelerates agent workloads by replacing full state duplication with diff-based checkpoint/restore across filesystem and process state.

  • DeltaBox targets test-time tree search and reinforcement learning with diff-based checkpoint and restore.
  • DeltaFS provides dynamic, unmount-free overlayfs layer switching for filesystem state, while DeltaCR uses CRIU dumps and warm-template forking for memory state.
Loading 2605.22781v2…