PaperlayerView exact revision on arXiv

Source-linked AI summary

Iapetus: Content-Aware Hierarchical Scheduling for Collaborative ViT Inference in LEO Satellite Networks

Yan Chen, Yunxiang Zhang, Guanjun Jiang, Haiquan Wang

arXiv:2609.03318v1cs.DC

TL;DR

Collaborative ViT inference can pool satellite resources, but intermediate activations can make partitioning costly and pruning quality varies with input and model depth. Iapetus screens a bounded set of constellation options and jointly plans content-aware compression and layer offloading; at 5 tasks/s, it outperforms MARATD3 on task accomplishment, latency, and battery draw while meeting quality targets.

  • Problem

    ViT partitioning can incur substantial activation-transfer overhead, while pruning quality depends on content, model depth, and earlier pruning decisions.

    Source: §1 · p. 1

  • Method

    Iapetus hierarchically screens constellation candidates and refines each into a complete content-aware token-compression and layer-offloading trajectory.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    91.60% task accomplishment ratio (TAR), 26.1 percentage points above MARATD3, with 53.0% lower mean latency and 70.8% lower battery draw while meeting quality targets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    HIL validation and constellation replay show strong task accomplishment with favorable latency, battery draw, and task quality across evaluated ViT workloads and constellation settings.

    Source: §9 · p. 13

    The current work leaves unexpected node or link failures, concurrent and bursty multi-source arrivals, and emerging model architectures for future extensions.

    Source: §9 · p. 13

Abstract

from arXiv · show

Collaborative inference pools distributed resources to run compute-intensive Vision Transformers (ViTs) in satellite edge computing. Model partitioning enables such collaboration by assigning consecutive layer groups to different nodes, but the large volume of intermediate activation data incurs substantial transfer overhead that can erase its benefit. Token compression reduces downstream computation and activation transfer, but its quality impact depends on input content, model depth, and earlier pruning decisions, while layer offloading must adapt to time-varying contact and battery conditions. We present \sys, a content-aware hierarchical scheduler that screens constellation-wide options to retain a bounded candidate set, then refines each candidate into a complete token compression and layer offloading trajectory using quality prediction and joint planning. A unified objective balances per-task latency, energy, and quality loss against accumulated workload and battery pressures. We implement \sys on an NVIDIA Jetson AGX Orin hardware-in-the-loop testbed and use its validated execution model for constellation-scale trace replay across multiple ViT workloads and constellation settings. At \(5\)~tasks/s, \sys accomplishes 91.6\% of released tasks, 26.1 percentage points above MARATD3, the strongest baseline, while reducing mean latency and battery draw by 53.0\% and 70.8\%, respectively, and meeting quality targets.

1 INTRODUCTION

Collaborative ViT inference can use distributed satellite compute, but large activations, content-dependent pruning quality, and changing constellation resources make execution planning difficult. Iapetus addresses these challenges with content-aware, hierarchical planning of complete compression and offloading trajectories.

1 INTRODUCTION

  • ViT-L/16 requires approximately 190.7 GFLOPs per inference, straining onboard compute and energy budgets when time-sensitive tasks cannot be finished locally or immediately offloaded.Partitioning can shift the bottleneck to communication because ViT activations retain their sequence length and embedding dimension; the fastest dense split measured 1.33 s versus 0.64 s for local inference.Source: §1 · p. 1
  • The same pruning setting can cause sharply different quality loss across images, while each pruning decision changes later computation and transfer payloads, motivating joint compression and offloading trajectories.The measurements found no accuracy loss for Desert scenes but a 21.79 pp loss for Park scenes under the same pruning setting.Source: §1 · pp. 1–2
  • Finite contact windows, multiple execution candidates, and changing workload and energy states make constellation-wide planning costly, while local-only planning can miss dependencies needed for feasible execution.The paper identifies quality estimation along pruning trajectories, trajectory feasibility, and constellation-scale planning as three key challenges.Source: §1 · p. 2
  • Iapetus combines content-aware quality prediction and hybrid planning with a common Lyapunov score, screening a bounded set of constellation-wide candidates before refining plans into complete trajectories.The leader associated with the selected plan commits it once for fixed execution.Source: §1 · p. 2
  • 91.60% task accomplishment ratio (TAR), 26.1 percentage points above MARATD3, with 53.0% lower mean latency and 70.8% lower battery draw while meeting quality targets.The evaluation uses a trace-driven prototype calibrated on two NVIDIA Jetson AGX Orin devices and validated on a hardware-in-the-loop testbed.Source: §1 · p. 2
  • The trace-driven prototype is evaluated across three ViT backbones, two token compression strategies, and two constellation settings, with gains in accomplishment, latency, and battery draw while meeting quality targets.Its execution model is validated on the corresponding hardware-in-the-loop testbed.Source: §1 · p. 2

2 MOTIVATION

The motivation measurements show why fixed execution plans are inadequate: compression can make partitioning viable, but pruning tolerance and preferred execution plans vary with image content and runtime conditions.

2 MOTIVATION

  • At 0.1 Gbps, the ToMe split transfers 0.81 MiB and completes in 505.4 ms, while the best dense split transfers 12.31 MiB and takes 1326.3 ms; local inference takes 641.2 ms.Token compression reduces transfer volume and downstream Transformer computation; the table compares representative ViT-L strategies under a 4 pp accuracy-loss budget.Source: Figure 1 · p. 3, Table 1 · p. 3
  • At r=12, Park loses 21.79 pp accuracy while Desert shows no accuracy loss, illustrating that a fixed pruning setting has content-dependent quality effects.Figure 1(a) reports overall accuracy for representative AID scene classes under uniform pruning ratios across Transformer blocks.Source: Figure 1 · p. 3
  • At comparable task accomplishment, the exhaustive reference reduces mean latency and battery draw by 16.14% and 22.88%, respectively, relative to the best fixed plan.The exhaustive reference and fixed policies accomplish approximately 99.99% of tasks, whereas full peer offloading accomplishes 77.81%.Source: §2 · p. 3
  • The preferred pruning setting and partition point change as queueing, contacts, and available compute change, so pruning and layer offloading must adapt jointly to runtime state.The fixed policies each use one configuration, while the exhaustive reference selects the lowest-latency feasible plan for each task.Source: §2 · p. 3
  • Token compression enables ViT partitioning, and Iapetus uses content-aware prediction and joint trajectory planning to realize the opportunity identified by the exhaustive reference.The measurement methodology uses fine-tuned ViT-L on AID and compares local inference, raw-input peer offloading, and partitioning with intermediate activations.Source: §2 · pp. 2–3

3 SYSTEM OVERVIEW AND EXECUTION ABSTRACTION

Iapetus first screens coarse execution candidates, then constructs complete layer-wise trajectories that jointly determine token compression and offloading. Its plan representation accounts for realized execution outcomes and task constraints.

3.1 System Architecture and Workflow

  • Iapetus uses fixed, non-overlapping swarms for local coordination, screens reachable swarms from a shared task-and-state snapshot, and has leaders construct plans before the sensing satellite selects a feasible option.The selected plan’s associated leader commits the resources and coordinates fixed execution; ground stations are optional endpoints.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 Decision and Execution Abstraction

  • Each candidate selects a swarm and execution mode, and its trajectory specifies per-layer token-removal fractions and execution nodes; node changes transfer the current activation.Ground-assisted execution may enter one reachable ground endpoint, after which ground is absorbing.Source: §3.2 · p. 4
  • A completed plan determines latency, onboard battery draw, and quality loss, and is feasible only when it satisfies the mode, contact, reservation, battery, and deadline constraints.The task’s quality target is met when achieved quality is at least the minimum target; every feasible plan also preserves the stated battery reserve.Source: §3.2 · p. 4

4 HIERARCHICAL COORDINATION

Iapetus uses one score to compare plans while accounting for execution cost and accumulated swarm pressures, then screens reachable swarm–mode options to bound detailed planning.

4.1 Lyapunov Score of a Complete Plan

  • The common plan score combines execution cost with accumulated workload and battery pressures, and lower scores are preferred.Coarse estimates rank candidates; detailed accounting selects the final plan.Source: §4.1 · pp. 4–5
  • The virtual workload queue tracks accumulated swarm workload pressure for scheduling, rather than the exact sum of node execution queues.The battery pressure discourages persistent depletion, while each feasible plan preserves the reserve at every satellite.Source: §4.1 · p. 5

4.2 Coarse Candidate Generation

  • Reachability filtering removes swarms without a usable ingress path, then lightweight workload, battery, ingress-latency, and service-time summaries screen the remaining swarms.The system retains up to K_swarm swarms before expanding execution modes.Source: §4.2 · p. 5
  • For each retained swarm, Iapetus estimates satellite-only and reachable ground-assisted modes, rejects candidates failing cheap checks, and retains at most K_cand candidates for trajectory planning.When the candidate budget permits, it first keeps the lowest-scoring survivor from each available mode, then fills remaining positions by global score.Source: §4.2 · pp. 5–6

5 JOINT TRAJECTORY PLANNING

Iapetus refines each bounded candidate into a complete trajectory, predicting pruning quality and jointly choosing pruning controls and execution nodes. It selects and commits the feasible plan with the lowest Lyapunov score.

5.1 Content-Aware Pruning Prediction

  • A content-aware predictor estimates final quality for each prospective pruning control using input content, layer depth, and the accepted pruning prefix.It uses a conservative estimate for planning and is queried again as decisions are accepted.Source: §5.1 · p. 6
  • 1,134,000 isolated and 48,000 history-conditioned records train separate predictors for supported pruning implementations.The history-conditioned labels replay prior controls before dense continuation.Source: Figure 4 · p. 6

5.2 Hybrid-Action Trajectory Planning

  • A hybrid-action policy jointly selects a pruning ratio and execution node at each layer, masking choices that violate contact, resource, battery, deadline, or mode constraints.Its marginal Lyapunov-score rewards telescope along the trajectory; incomplete trajectories are excluded from plan selection.Source: §5.2 · pp. 6–7

5.3 Plan Selection and Commitment

  • Iapetus discards incomplete or infeasible trajectories, selects the complete plan with the lowest detailed Lyapunov score, and commits its reservations atomically before execution.Execution follows the chosen trajectory without switching to another candidate.Source: §5.3 · p. 7

6 IMPLEMENTATION

The implementation combines a trace-driven scheduling runtime with hardware profiling and a Jetson AGX Orin hardware-in-the-loop testbed.

6 IMPLEMENTATION

  • The Python prototype replays contact, compute, task, and battery events while leaders independently construct plans and commit the selected reservations atomically.Its components include a routing module, resource manager, and preloaded models and policies; the prototype comprises about 30K lines of code.Source: §6 · p. 7
  • Jetson AGX Orin profiling in 30-W and MAXN modes records per-layer latency, activation size, downstream FLOPs, and INA power.The HIL testbed uses two Orins and a desktop host for satellite, peer, and ground-station roles.Source: §6 · p. 7
  • HIL execution replays committed trajectories on physical devices, transferring serialized activations with trace-paced links and actual handoffs between satellite hops.Ground-assisted trajectories forward activations to the desktop for remaining ground-side execution.Source: §6 · p. 7

7 EVALUATION

Evaluation spans ViT workloads, token-compression strategies, and constellation conditions, assessing end-to-end performance, component contributions, robustness, and execution fidelity. Iapetus generally sustains strong task accomplishment while balancing latency, energy, and quality.

7.1 Evaluation Methodology

  • The default evaluation uses a 24-plane Starlink trace, ViT-L on AID, 30-s deadlines, and a 0.87 quality target; TAR counts tasks completed feasibly before deadline while meeting quality.The study also evaluates ViT-H and DINOv2, RESISC45, and OneWeb settings.Source: §7.1 · p. 8

7.2 End-to-End Performance

  • At 5 tasks/s, Iapetus reaches 91.60% TAR versus 65.54% for the strongest baseline at that rate, with 53.0% lower latency than MARATD3 and 66.2% lower battery draw than SPS-AO.Its cost is 0.122, 39.1% below SPS-AO and 59.8% below MARATD3, while it accomplishes more tasks than either.Source: Figure 5 · p. 8, §7.2 · pp. 8–9

7.3 Component Evaluation

  • Using content, depth, and pruning history improves predictor pair accuracy by 11.00–33.33 pp and reduces quality regret by 29.3–66.7%; ToMe and TopK both support the planning workflow.With ToMe, the system achieves 91.60% TAR and 1.74% accuracy loss; with TopK, 93.28% TAR and 0.87% accuracy loss.Source: §7.3 · p. 9, Table 3 · p. 9
  • Decoupled planning loses 15.15 pp TAR and raises cost by 6.0%, while the default (8, 4) candidate budget retains the exhaustive best plan for 97.5% of tasks with a 1.66% P95 score gap.The energy-aware objective reduces aggressive execution choices, while the default candidate budget balances selection quality and planning cost.Source: §7.3 · pp. 9–10, Table 5 · p. 10

7.4 Robustness and Applicability

  • TAR remains 90.88–91.67% across 10–60 s deadlines; under link fluctuations, Iapetus reaches 67.35% TAR, 10.54/10.97 pp above SPS-AO/MARATD3.Under compute fluctuations it reaches 91.64% TAR, 28.29 pp above SPS-AO, with 22.65-s P95 latency.Source: Figure 6 · p. 10, Figure 7 · p. 10, §7.4 · p. 10
  • With frozen policies, Iapetus achieves 91.34% TAR in the alternate seasonal trace and 86.91% on OneWeb; it leads baselines across ViT backbones and both datasets.Across backbones, TAR is 86.08–92.28%; across the dataset comparison it leads the strongest baseline by 25.20–50.23 pp.Source: Table 6 · p. 11, §7.4 · p. 11, Figure 8 · p. 11

7.5 Runtime Overhead and Scalability

  • HIL execution latency matches the model at 1.06% MAPE and battery draw at 7.64%; mean planning latency is 217.50 ms, or 0.73% of the 30-s deadline.Doubling the constellation from 528 to 1,056 satellites raises mean planning latency to 306.90 ms, while detailed planning remains limited to 40 satellites.Source: §7.5 · pp. 11–12, Table 7 · p. 12, Table 8 · p. 12

8 RELATED WORK

Prior work reduces token counts or distributes Transformer inference, but Iapetus combines content-aware compression with layer execution under satellite contact and energy dynamics.

Token-efficient ViT inference

  • Token-efficient ViT methods prune or merge redundant tokens, but do not jointly determine how content-aware compression interacts with distributed layer execution [3].The paper also notes visual-token reduction in vision–language and video systems.Source: §8 · p. 12

Collaborative Transformer inference in MEC

  • Terrestrial MEC systems distribute Transformer inference through parallelism and split inference, generally assuming relatively stable participants and edge paths.ViT-specific systems can combine model decomposition with token compression.Source: §8 · p. 12

Collaborative inference and scheduling in SEC

  • SEC work addresses offloading and CNN partitioning, while recent Transformer systems use compressed submodel chains or topology-aware token scheduling [46].The paper distinguishes Iapetus by planning content-aware compression and layer offloading together across depth and finite contacts.Source: §8 · p. 12

9 CONCLUSION AND FUTURE WORK

Iapetus schedules collaborative onboard ViT inference as complete content-aware token-compression and layer-offloading trajectories, using hierarchical planning under changing contacts, workload, and battery conditions. Hardware-in-the-loop validation and constellation replay show favorable task accomplishment, latency, battery draw, and quality; future work targets failure recovery, multi-source coordination, and additional model architectures.

9 CONCLUSION AND FUTURE WORK

  • Iapetus constructs complete content-aware token-compression and layer-offloading trajectories with hierarchical scheduling for dynamic contacts, workload, and battery conditions.Hardware-in-the-loop validation confirms execution fidelity, and constellation replay spans ViT workloads and constellation settings.Source: §9 · pp. 12–13
  • Future work would add backup trajectories, checkpoint recovery, and runtime migration when unexpected node or link failures invalidate a selected plan.The proposed failure-aware execution extends one-time commitment with recovery mechanisms.Source: §9 · p. 13
  • Future work would coordinate concurrent and bursty Earth-observation arrivals competing for shared swarms, ground links, and resource reservations.Source: §9 · p. 13
  • Supporting sparse mixture-of-experts and state-space models would require extending the content-aware abstraction to model-specific compression, placement, and routing.Source: §9 · p. 13

A LYAPUNOV DERIVATION AND APPROXIMATION RESIDUAL

The derivation turns conditional quadratic queue drift and task execution cost into a per-slot objective that penalizes accumulated workload and battery pressures. Its bound assumes bounded per-slot arrivals, service, and battery variation, while the deployed bounded search is treated as an approximation with a bounded residual.

A LYAPUNOV DERIVATION AND APPROXIMATION RESIDUAL

  • The drift bound assumes bounded arrivals, service, and battery variation in each slot, making the quadratic terms bounded by a finite constant.The battery term bounds the assignment-attributable component by admitted mean satellite battery draw; platform draw and solar input are independent of the current assignment.Source: A LYAPUNOV DERIVATION AND… · p. 15
  • Eq. (5) combines workload and battery penalties under accumulated pressures with the task execution cost.The current-decision terms are obtained by minimizing the part of the drift bound affected by the assignment.Source: A LYAPUNOV DERIVATION AND… · p. 15
  • The deployed search is an approximation, not an exact per-slot optimizer, with a bounded residual from screening, candidate truncation, estimation, learned search, and sequential commitment.The drift bound gains an additive bounded term when the residual is bounded.Source: A LYAPUNOV DERIVATION AND… · p. 15

B ADDITIONAL EVALUATION RESULTS

Objective-weight profiles trade improvements in their target metric against changes in other outcomes, while Balanced retains competitive task accomplishment without the reported one-sided degradation. The appendix also describes a decomposition of task accomplishment under link fluctuations.

Admission under link fluctuations

Sensitivity to objective weights

  • 21.3%, 1.1%, and 16.0%: latency-, energy-, and accuracy-oriented profiles improve their respective target metrics relative to Balanced.Energy weighting also loses 2.32 pp of TAR and raises mean latency and accuracy loss by 22.3% and 22.5%; latency weighting draws 1.6% more battery. Balanced is used as the default joint objective.Source: Figure 9 · p. 16, B ADDITIONAL EVALUATION RESULTS · p. 16
  • All profiles use the same AID+ViT-L ToMe configuration at 5 tasks/s and evaluation seeds, with a shared frozen actor and objective weights supplied as inputs.Energy-oriented and Accuracy-oriented profiles use weights (0.2, 0.6, 0.2) and (0.2, 0.2, 0.6), respectively.Source: B ADDITIONAL EVALUATION RESULTS · p. 16
Loading 2609.03318v1…