PaperlayerView exact revision on arXiv

Source-linked AI summary

Resuming Vision-Language Inference across Onboard Interruptions

Xiaolong Huang, Yinyuan Zhang, Daliang Xu, Yun Ma, Qing Li, Mengwei Xu, Shangguang Wang, Gang Huang, Xuanzhe Liu

arXiv:2610.29183v1cs.DCcs.AI

TL;DR

On-orbit VLM requests can be interrupted by power, thermal, fault, or competing-payload events, losing volatile inference progress. OrbitWeaver combines phase-aware checkpointing with dependency-aware recovery; evaluations report up to 4.07-fold end-to-end speedup, and a 72-hour deployment demonstrates resumed inference across repeated shutdowns.

  • Problem

    Onboard VLM requests may span execution windows, but bus-enforced shutdowns can interrupt inference and discard volatile states before a request completes.

    Source: §1 · pp. 1–2

  • Method

    OrbitWeaver checkpoints phase-specific inference states to persistent storage and dynamically schedules their restoration alongside computation after interruptions.

    Source: §1 · p. 2, §3 · p. 5

  • Results

    Up to 4.07× end-to-end speedup over three state-of-the-art baselines, while a 72-hour deployment on two satellites demonstrated sustained inference across repeated bus-enforced shutdowns.

    Source: §1 · p. 3, §10 · p. 13

  • Takeaways & Limitations

    OrbitWeaver retained inference progress and resumed execution across repeated bus-enforced shutdowns in a 72-hour in-orbit deployment on Sat-1 and Sat-2.

    Source: §1 · p. 3

Abstract

from arXiv · show

Executing vision-language models (VLMs) aboard low-Earth orbit satellites enables flexible processing of observation data without first downlinking it to the ground. On commercial off-the-shelf payloads, however, a request may span multiple execution windows: power and thermal constraints, fault-triggered reboots, and competing payload demands can interrupt inference before it completes. Each shutdown discards volatile activations and KV caches, forcing an unprotected request to repeat work after restart. We present OrbitWeaver, a framework that preserves VLM inference progress across onboard interruptions through coordinated checkpointing and recovery. Its phase-aware checkpointing selects recovery boundaries according to the state lifetimes and computational characteristics of vision encoding, language-model prefill, and decoding, balancing storage traffic against recomputation. Its dependency-aware dynamic recovery schedules restoration from partially completed states across the layer and token-chunk dimensions. By overlapping checkpoint and weight reads with recomputation, it reduces restart latency; the recovery planner computes an optimal execution order under its scheduling model within 100 ms. Experiments on satellite payload hardware using on-orbit telemetry traces show up to a 4.07-fold end-to-end speedup and a 3.56-fold reduction in SSD write volume over the evaluated baselines. A 72-hour deployment on two low-Earth orbit satellites demonstrates that OrbitWeaver retains inference progress and resumes execution across repeated bus-enforced shutdowns.

1 Introduction

On-orbit VLM workloads can be long, while bus-enforced shutdowns can interrupt them unpredictably and discard in-memory progress. OrbitWeaver preserves selected state through phase-aware checkpointing and dependency-aware recovery, with evaluations reporting end-to-end speedups and sustained in-orbit execution.

1 Introduction

  • VLM requests can involve long visual-token sequences, including Earth-observation workloads with image sequences of up to 30 frames and images reaching 44.93 megapixels [15] [45].A typical three-minute mission processes only eight frames (>32K tokens) in the reported Sat-1 and Sat-2 evaluation.Source: §1 · pp. 1–2
  • Bus-enforced shutdowns arise from power or thermal constraints, fault-triggered reboots, and resource contention, creating unpredictable interruptions to sustained VLM inference.Telemetry reports worst-case reboots as frequent as once per minute.Source: §1 · pp. 1–2
  • OrbitWeaver addresses the checkpointing trade-off between preserving fine-grained progress and limiting I/O, and the recovery challenge of overlapping weight reads with computation from irregular progress states.Fine-grained checkpointing can incur substantial I/O, while coarse-grained checkpointing can require expensive recomputation.Source: §1 · p. 2
  • OrbitWeaver’s phase-aware checkpointing adapts recovery boundaries to vision encoding, prefill, and decoding, while dependency-aware recovery schedules partially completed states across layers and token chunks.Its planner computes an optimal execution order under its scheduling model within 100 ms.Source: §1 · p. 2
  • Up to 4.07× end-to-end speedup over three state-of-the-art baselines, with a 72-hour deployment on Sat-1 and Sat-2 demonstrating sustained inference across repeated bus-enforced shutdowns.The evaluation uses five interruption traces, including telemetry from Sat-1 and Sat-2 and the Dellingr satellite.Source: §1 · p. 3

2 Background

On-satellite VLM inference uses onboard compute and storage within a satellite platform whose bus controls payload power and can interrupt execution. Because interruptions can discard volatile state at any inference phase, resumption requires persisted checkpoints and recovery.

2.1 On-Satellite Vision-Language Models

  • A satellite platform combines a computing payload, which uses SSD, RAM, CPUs, and GPUs, with a spacecraft bus that supplies power, monitors temperature, and manages communications and fault protection.The three-phase VLM pipeline encodes image patches, prefills visual and prompt tokens into the KV cache, then generates output autoregressively.Source: §2.1 · pp. 3–4

2.2 Bus-enforced Shutdowns in On-Satellite VLM Inference

  • Bus-enforced shutdowns can result from power or thermal limits, anomalies handled by FDIR, or resource contention from intermittent high-priority payloads.The reported Sat-1 inference reaches a 55°C enclosure threshold from 45°C in 48 seconds, and private telemetry reports worst-case reboots once per minute.Source: §2.2 · p. 4
  • A shutdown can discard hidden states and the KV cache at any inference phase, so resumption requires checkpoints in non-volatile storage and recovery without advance knowledge of shutdown timing.The stated scope assumes persistent weights and checkpoints remain intact, leaving persistent-storage corruption to orthogonal hardware research.Source: §2.2 · p. 4

3 Design Overview

OrbitWeaver makes on-satellite VLM inference resumable by persisting selected boundary states and restoring the latest checkpoint after a shutdown. Its checkpointing choices vary by inference phase, while dynamic recovery schedules reads and computation to reduce stalls.

3 Design Overview

  • OrbitWeaver persists selected boundary states to SSD and, after shutdown, validates and reloads the latest checkpoint to reconstruct execution context and continue the unfinished request.The design aims to preserve progress without restarting inference from scratch.Source: §3 · p. 5
  • Phase-aware checkpointing limits persisted state according to phase: block-wise prefill checkpoints, layer-boundary vision states, and batched decoding KV-cache updates.The design also uses phase-specific storage layouts for I/O interference, recovery reads, and crash-consistent persistence.Source: §3 · p. 5
  • Dynamic recovery schedules SSD reads and dependency-ready block computation, restoring states that unlock useful work and overlapping later reads with computation.The recovery planner reduces planning to layer-level I/O ordering and uses a lightweight dynamic programming algorithm.Source: §3 · p. 5

4 Efficient Checkpointing

OrbitWeaver uses phase-aware recovery boundaries and storage layouts to preserve the states needed for resumption while limiting checkpoint overhead. Prefill, vision encoding, and decoding require different checkpoint policies and access patterns.

4. Efficient Checkpointing

  • Checkpoint granularity trades checkpoint traffic against recomputation: finer units reduce lost progress but increase transfer, write, and metadata overhead.Layer-wise checkpointing minimizes lost progress but uses the most storage; chunk-wise checkpointing keeps storage near total KV size but has the highest recomputation overhead.Source: §4 · p. 5, §4.1 · p. 6

4.1 Block-wise Prefill Checkpointing

  • Block-wise prefill checkpoints retain KV tensors and sparse layer-group boundary hidden states, allowing recovery inside a chunk while limiting replay to the unfinished G×C unit.With G=4, lost progress is about 2.04 × 10^3 token-layers, with checkpoint sizes of 8.96 GiB for Qwen3-VL 2B and 13.82 GiB for MiniCPM-V 4.5.Source: §4.1 · p. 6

4.2 Checkpointing Vision and Decode

  • Vision checkpoints persist full-patch hidden states at layer boundaries, while batched decoding checkpoints KV slices, token IDs, and decode position.A reset between decode checkpoints requires replaying at most D decode steps.Source: §4.2 · p. 6

4.3 I/O-Efficient Storage Layout

  • Prefill records separate boundary hidden states from KV tensors, vision uses alternating complete buffers, and decode appends batched KV updates in sequentially written records.A checksum-protected commit footer marks a record committed only after it is durably written.Source: §4.3 · p. 7

5 Dynamic Recovery

OrbitWeaver models recovery as an I/O–computation scheduling problem and chooses a dependency-aware order to reduce exposed I/O. The planner reduces the block-level problem to layer loading and computes an optimal layer order within 100 ms for the specified setting.

5. Dynamic Recovery

  • Increasing recovery time from 5 s to 25 s raises 64K-token request completion time by up to 5.04× on AGX and 5.56× on Orin.The controlled experiment used Qwen3-VL 2B/4B/8B on Jetson AGX Orin and Jetson Orin NX.Source: §5 · p. 7

5.1 Dependency-Aware Recovery

  • Fixed chunk-first and layer-first orders can expose I/O stalls by selecting blocks whose required weights, KV cache, or hidden states are not yet available.Dependency-aware recovery instead computes ready blocks to hide reads needed by later blocks.Source: §5.1 · pp. 7–8, Figure 8 · p. 8

5.2 Recovery as a Scheduling Problem

  • The recovery model represents load and compute tasks with precedence constraints; a compute task is ready only after its weights, preceding same-layer KV cache, and previous-layer hidden state are available.Completed prefill blocks form a staircase frontier across layers and chunks.Source: §5.2 · p. 8, Figure 9 · p. 8

5.3 Optimal Recovery Plan

  • OrbitWeaver reduces the O(NL) block-level recovery problem to ordering O(L) layer I/O operations and uses subset dynamic programming to find an optimal order.For Qwen3-VL 8B, the algorithm finishes within 100 ms when the layer group size G≥2.Source: §5.3 · p. 9

6 Implementation

The prototype implements OrbitWeaver on transformers for single-request, single-GPU satellite-payload settings, with optional decode KV compression and crash-consistent checkpoint records.

6. Implementation

  • OrbitWeaver is implemented with 9,617 lines of Python and 1,741 lines of CUDA/C++ on transformers, processing one request at a time on a single GPU.Its checkpointing and recovery mechanisms can also be integrated into engines such as vLLM and SGLang; optional KV compression reduces decode-stage recovery latency by up to 5.02×.Source: §6 · p. 9

7 Evaluation

OrbitWeaver completes every evaluated request, with up to 4.07× speedup over SONIC-layer, and sustains inference across repeated on-orbit shutdowns. Its recovery, checkpointing, and storage designs reduce recovery latency, normal-path overhead, SSD writes, and energy costs.

7.1 Experiment setup

  • Experiments use two Jetson devices, four edge-sized VLMs, LRS-VQA, and interruption traces from Sat-1, Sat-2, and Dellingr.The evaluation measures end-to-end and recovery latency, inference latency, SSD writes, device-level write amplification, and energy.Source: §7.1 · pp. 9–10, §7.2 · p. 10, Table 2 · p. 10

7.2 End-to-end performance

  • OrbitWeaver completes every evaluated request, with speedups up to 4.07×, 2.89×, and 2.67× over SONIC-layer, SONIC-chunk, and HCache, respectively.Geometric-mean speedups are 2.95×, 1.98×, and 1.65×; in short Trace A, only OrbitWeaver completes two specified requests.Source: Figure 10 · p. 10, §7.2 · pp. 10–11

7.3 Recovery latency

  • OrbitWeaver achieves recovery speedups of 1.22×–6.97× across tested devices, models, and prefill progress points.Dependency-aware recovery prioritizes states that unlock computation and overlaps SSD reads with dependency-ready computation.Source: Figure 11 · p. 11, §7.3 · p. 11

7.4 Checkpoint overhead

  • Across tested devices and models, OrbitWeaver stays within 5% of no-checkpoint inference time and achieves geometric-mean speedups of 1.62×, 1.38×, and 1.08× over the three baselines.Block-wise checkpointing avoids persisting full-sequence hidden states after every layer, while contiguous records reduce transfers and fragmented writes.Source: §7.4 · p. 11, Figure 12 · p. 12

7.5 Disk wear

  • Across 2K–32K prompts, the baselines write 2.33×–3.56×, 1.08×–2.13×, and 1.10×–2.20× as much data as OrbitWeaver, respectively.OrbitWeaver also reduces DLWA by 33.6–45.4%, 14.3–46.4%, and 15.0–46.2% versus SONIC-layer, SONIC-chunk, and HCache, respectively.Source: §7.5 · p. 11, Figure 13 · p. 12

7.7 Case Study: Real Deployment on Sat-1 and Sat-2

  • Across 72 hours on Sat-1 and Sat-2, OrbitWeaver completes all 50 requests despite 91 bus-enforced shutdowns.Forty-one requests were interrupted; one illustrated request resumed from committed checkpoints across two shutdowns and completed in its third execution window.Source: Figure 15 · p. 12, §7.7 · p. 12

7.6 Energy consumption

  • OrbitWeaver consumes only 3.75%–4.74% more energy than no-checkpoint inference across tested prompt lengths.The comparison measures uninterrupted Qwen3-VL 2B inference on Orin; the no-checkpoint baseline loses in-memory progress after a reset.Source: §7.6 · p. 11, Figure 14 · p. 12

8 Discussion

OrbitWeaver targets orbital edge computing and is presented as complementary to hardware co-design. Its checkpointing and recovery principles are also framed as applicable to space-based cloud platforms.

8 Discussion

  • OrbitWeaver’s checkpointing and recovery principles are described as complementing hardware co-design by letting interrupted requests resume rather than restart.The discussion names Starcloud and Project Suncatcher as envisioned space-based cloud platforms.Source: §8 · p. 12

9 Related Work

OrbitWeaver is positioned at the intersection of satellite computing, intermittent computing, and efficient LLM/VLM inference. Its distinguishing focus is preserving VLM progress across resets during intermittent on-orbit execution.

Satellite Computing

  • Unlike satellite systems focused on onboard sensing and decision-making, OrbitWeaver preserves VLM progress across resets so long-running requests can complete despite intermittent execution.The passage contrasts it with onboard filtering and observation-scheduling systems.Source: §9 · p. 13

Intermittent Computing

  • OrbitWeaver extends checkpointing and task-based intermittent execution to diverse interruption causes, tens-of-seconds-to-minutes windows, and multi-GiB states persisted to SSD or eMMC.The resulting challenges include balancing checkpoint size against preserved progress and scheduling recovery I/O with computation.Source: §9 · p. 13

Efficient LLM and VLM Inference

  • Unlike LLM/VLM serving and long-context systems, OrbitWeaver targets intermittent on-orbit inference where volatile activations and KV caches are lost after resets unless made durable and recovered.Source: §9 · p. 13

10 Conclusion

OrbitWeaver provides efficient checkpointing and dynamic recovery for on-satellite VLM inference. The evaluation reports up to 4.07× end-to-end speedup with less than 5% runtime overhead, and deployment confirms sustained progress across bus-enforced shutdowns.

10 Conclusion

  • OrbitWeaver combines efficient checkpointing and dynamic recovery for on-satellite VLM inference, achieving up to 4.07× end-to-end speedup with less than 5% runtime overhead.Source: §10 · p. 13

Related papers

Loading 2610.29183v1…