PaperlayerView exact revision on arXiv

Source-linked AI summary

Predictive Rolling-Horizon Optimization for Commitment-Aware Model-Parallel Inference under Spatio-Temporal Edge Dynamics

Minghui Liwang, Chenxi Xu, Wei Gong, Li Li, Wenbo Zhu, Xinlei Yi, Yuhan Su, Xianbin Wang

arXiv:2609.31018v1cs.DC

TL;DR

Edge-inference schedulers often optimize current or short-term metrics without explicitly coupling assignments to future commitment fulfillment. PROMISE jointly schedules edge-server mappings and committed completion times using predicted demand and resource rollouts. Numerical evaluations show adaptability across varying system scales and workloads, while Raspberry Pi experiments assess practical model-parallel deployment and system characteristics.

  • Problem

    Existing edge-inference designs largely optimize current or short-term performance metrics without explicitly modeling long-term evolution or enforceable service guarantees.

    Source: §1 · p. 1

  • Method

    PROMISE jointly determines edge-server mappings and committed completion times, rolling predicted future arrivals and workloads into certainty-equivalent server-state evolution.

    Source: §1.3 · p. 3, §4.2 · pp. 8, 10

  • Results

    Numerical experiments demonstrate adaptability across varying system scales, workload intensities, and prediction horizons, while a Raspberry Pi testbed validates practical model-parallel deployment and system characteristics.

    Source: §7 · p. 15

  • Takeaways & Limitations

    PROMISE evaluates current assignments against anticipated downstream resource impacts rather than only instantaneous system states.

    Source: §7 · p. 15

    The Raspberry Pi testbed does not explicitly model stochastic task arrivals or time-varying resource states, and its scale and controllability limit its focus to implementation and empirical performance.

    Source: §6 · p. 14

Abstract

from arXiv · show

Model-parallel inference over dynamic edge systems requires scheduling decisions that account for not only instantaneous resources but also future resource contention and reliable service commitments. Existing edge-inference designs, however, predominantly optimize performance metrics based on current or short-term system states, without explicitly coupling current assignments with future commitment fulfillment. To address this issue, we propose PROMISE, a predictive rolling-horizon framework for commitment-aware model-parallel inference under spatio-temporal edge dynamics. PROMISE jointly models stochastic task generation, mobility-induced communication variations, privacy-aware model partitioning, and load-dependent edge computing capability. We introduce committed completion time (CCT) as an endogenous service decision and formulate joint SD--ES mapping and CCT determination to balance commitment fulfillment and service reward. To address cross-timeslot coupling, PROMISE estimates future task arrivals and computational workloads over an adaptive horizon and embeds them into certainty-equivalent ES-state rollout. A progress-aware surrogate then evaluates feasible current-stage mappings, while the corresponding CCTs are analytically recovered from predicted completion times. Only the first-stage decisions are committed, and the optimization is repeated with newly observed states in a receding-horizon manner. Numerical experiments demonstrate robust scheduling performance under diverse system scales and workload dynamics, while Raspberry-Pi-based experiments validate the practical feasibility and key system characteristics of model-parallel edge inference.

1 INTRODUCTION

Existing edge-inference scheduling often assumes static or weakly dynamic resources and optimizes short-term metrics without explicit service guarantees. PROMISE couples commitment decisions with predicted workload and resource evolution in a receding-horizon framework.

1 INTRODUCTION

  • Existing designs optimize surrogate metrics such as latency or energy consumption, without explicitly modeling long-term system evolution or enabling enforceable service guarantees.Source: §1 · p. 1

1.2 Literature Review

  • Prior studies typically model edge dynamics individually or react to current states, whereas PROMISE predicts multi-slot demand and rolls future edge-server states forward.The literature review also distinguishes PROMISE by treating committed completion time as a scheduling variable rather than relying only on metric- or constraint-driven QoS.Source: §1.2 · p. 2

1.3 Spotlight and Contribution

  • PROMISE jointly models stochastic task generation, mobility-related communication variation, privacy-aware partitioning, and load-dependent edge capability while making committed completion time an endogenous service decision.Tighter commitments offer higher service rewards but risk fulfillment; only current-stage mappings and CCTs are committed, with future information refreshed as states change.Source: §1.3 · p. 3
  • PROMISE’s software evaluations examine diverse dynamics, and a Raspberry Pi testbed assesses practical model-parallel deployment and its system characteristics.The numerical and hardware studies address complementary scopes: large-scale scheduling dynamics and practical computation, communication, and privacy characteristics.Source: §1.3 · p. 3

2 SYSTEM MODEL

The system comprises service-demanding devices, distributed edge servers, and a platform that orchestrates model-parallel inference. Its scheduling model accounts for workload-dependent server capacity, privacy-constrained partitioning, stochastic demand and communication, and the commitment trade-off faced by each task.

2 SYSTEM MODEL

  • Devices run the front part of each DNN locally and offload the remaining computation to a platform-selected edge server, which receives intermediate features.The platform observes server states and announces a committed completion time when a task is ready for edge-side scheduling.Source: §2 · p. 4

2.1 Modeling of ESs and Platform

  • An edge server’s effective per-task computing capability depends on concurrent workload and declines as resource contention increases.The server state tracks hosted-task evolution and is updated after admissions to capture effects on future resource availability and projected completion times.Source: §2.1 · p. 4

2.2 Modeling of SDs with Privacy Preservation

  • DNN partition points determine which layers run locally and which run at the edge, and are constrained by task-specific privacy requirements.For image inference, reconstruction leakage is assessed by reconstructing inputs from intermediate activations and measuring SSIM; retaining more front layers locally can reduce reconstruction fidelity.Source: §2.2 · p. 5

2.3 Uncertain Factors under Spatio-Temporal Evolution

  • Uncertainty comes from stochastic task generation, task types and batch counts, and mobility-driven variation in transmission time.These factors produce changing computation, transmission, and communication conditions across tasks and timeslots.Source: §2.3 · p. 5

2.4 An Illustrative Example

  • A tighter feasible CCT yields a higher service reward but leaves less tolerance for future resource contention.Tasks completed within their announced CCT are served successfully; violations impair platform credibility.Source: §2.4 · p. 6

3 COMMITMENT MODELING AND PROBLEM FORMU-

PROMISE treats committed completion time (CCT) as a proactive service decision distinct from actual completion, and jointly optimizes task-to-edge-server assignments and commitments. Its formulation balances commitment fulfillment and service reward while accounting for how current assignments and future arrivals affect edge-server capability.

3.1 Actual and Committed Completion Times

  • Edge-side completion is refined within the terminal timeslot using the residual workload and the ES capability in that slot.The terminal slot is the first one in which accumulated processing capability suffices for the required edge workload.Source: §3.1 · p. 6
  • Actual completion time depends on future realized edge capability, whereas the PT announces CCT beforehand as a duration from edge-side arrival.CCT is a management-level commitment and need not match actual completion or the instantaneous completion estimate.Source: §3.1 · p. 6

3.2 Task Completion and Commitment Reward

  • A task is successful only when its edge-side inference finishes within the proactively announced CCT, rather than an externally imposed deadline.Commitment violations impair the platform’s service credibility.Source: §3.2 · p. 6
  • A tighter fulfilled CCT earns greater service reward, while a violated commitment receives neither completion credit nor reward.The stated mechanism links commitment tightness to fulfillment reliability and discourages unrealistically aggressive announcements.Source: §3.2 · p. 7

3.3 Problem Formulation

  • PROMISE jointly determines device-to-edge-server assignments and CCTs to balance commitment fulfillment and service reward over the scheduling horizon.The formulation includes assignment, server-capacity, task-generation, and residual-latency constraints; all generated tasks are assumed admitted.Source: §3.3 · p. 7
  • Current assignments alter future ES occupancy and effective computing capability, while stochastic future arrivals can affect whether current commitments are met.Scheduling therefore must account for potential subsequent arrivals as well as immediate completion feasibility.Source: §3.2 · p. 6, §3.3 · p. 7
  • The joint problem is challenging because discrete assignments and continuous CCTs interact with cross-timeslot resource effects and uncertain future conditions.The paper assumes generated tasks are admitted; rejection under server saturation is outside its scope, with admission control left to an upper layer.Source: §3.3 · p. 7

4 METHODOLOGY: DESIGN OF PROMISE

PROMISE uses predicted workload evolution to evaluate and rank feasible current-stage SD–ES mappings by completion progress, recover commitments from predicted completion times, and repeatedly implement only first-stage decisions as new system information arrives.

4 METHODOLOGY: DESIGN OF PROMISE

  • PROMISE uses two coupled modules: Module A predicts future workload evolution, and Module B uses those predictions to map devices to edge servers while accounting for future resource availability.This prediction-aware approach addresses temporal coupling between current assignments, future task arrivals, and workload variation.Source: §4 · p. 7

4.1 Module A. Future X-Timeslot Demand Estimation under Spatio-Temporal Coupling

  • Module A estimates future schedulable task arrivals and aggregate edge-computing workloads over an adaptively selected horizon, updating both predictions at each scheduling timeslot.The estimates characterize stochastic task generation and the associated local-computation and transmission processes; future demand is not directly observable at the current timeslot.Source: §4.1 · pp. 7–8

4.2 Module B. Forward-Looking Rolling-Horizon Optimization for SD-ES Mapping and CCT Determination with Implicit Time Decoupling

  • Module B treats predicted demand as certainty-equivalent input and rolls observed edge-server states forward to evaluate current decisions over the prediction horizon, committing only current decisions and re-optimizing with updated states.The predictive scheduling-epoch formulation preserves downstream effects of current assignments without exactly decomposing the original time-coupled problem.Source: §4.2 · pp. 8–9
  • For each feasible current-stage mapping, PROMISE applies a common predicted continuation with uniform future loading across edge servers and uses a completion-progress surrogate to rank candidates without branching over future assignments.Only the mapping and CCT decisions for currently schedulable tasks are committed; subsequent predictions are replaced as new system states become available.Source: §4.2 · pp. 9–10
  • The completion-progress return ranks feasible mappings by the workload fraction processed over the look-ahead horizon, favoring earlier completion and resource release.This criterion supports predictive commitment feasibility and tighter recovered CCTs for newly admitted tasks.Source: §4.2 · p. 11
  • PROMISE evaluates every feasible current-stage mapping without heuristic pruning, approximating uncertain future demand with finite-horizon certainty-equivalent prediction.It avoids recursively optimizing unknown future task assignments while retaining explicit look-ahead evaluation of implementable decisions.Source: §4.2 · p. 11
  • For each newly admitted task, PROMISE recovers CCT from mapping-induced predicted completion time, using that duration if within the maximum tolerable duration and otherwise tmax.The recovered commitment is the tightest successful one when completion is predicted in time; otherwise, it is the least aggressive admissible commitment.Source: §4.2 · p. 11
  • PROMISE implements only the selected first-stage mapping and corresponding CCTs, then updates predictions and ES states with newly observed information and repeats.Because commitments precede future workload realizations, prediction mismatch and stochastic arrivals can affect their eventual fulfillment.Source: §4.2 · p. 11

5 SIMULATIONS

The simulations assess privacy-aware partitioning and PROMISE under varying system scales and workloads. PROMISE maintains high task completion rates and objective values across conditions, while adaptive prediction horizons perform more stably across batch workloads.

5.1 Privacy Evaluation on Real-world Datasets

  • Distributed inference matches full-model Top-1 accuracy, Top-5 accuracy, and F1-score, with sample predictions numerically identical under controlled execution.This establishes a baseline separating partitioning from deployment factors such as compression, quantization, packet loss, and platform-specific configurations.Source: §5.1 · pp. 11–12, Figure 4 · p. 12

5 SIMULATIONS

  • The numerical evaluation uses a discrete timeslot-based simulation environment with parameter settings summarized in Table 3.The real-world dataset evaluation covers four model–dataset settings and multiple candidate partition points per model.Source: §5.1 · p. 11, §5.2 · p. 12, Table 3 · p. 12

5.2 Evaluation on Numerical Datasets

  • The five benchmark strategies span random and instantaneous decisions, statistical workload expectations, one-step forecasting, and iterative optimization without explicit future-trajectory prediction.PROMISE instead predicts multi-timeslot workloads and rolls forward ES states, committing only the first-stage decision before re-optimizing.Source: §5.2 · p. 12
  • PROMISE maintains high task completion rates and objective values across varying system scales and workload intensities.More data batches generally reduce all three metrics; increasing task-generation intervals generally improves completion rates and objective values but reduces cumulative reward.Source: §5.2 · p. 13, Figure 5 · p. 13, Figure 6 · p. 13
  • Adaptive X(PROMISE) selects horizon depth by the effective temporal span of scheduling decisions and performs more stably across varying batch workloads.Fixed horizons do not yield monotonic gains, and X = 0 omits future-state evaluation.Source: §5.2 · p. 13, Figure 9 · p. 13

6 HARDWARE EXPERIMENTS

Raspberry Pi experiments validate deployment and examine model-parallel and multicore inference behavior. Partition choice trades computation against communication, while multicore speedups remain below linear scaling.

6 HARDWARE EXPERIMENTS

  • The hardware experiments do not explicitly model stochastic task arrivals or time-varying resource states, which are evaluated in the simulations.The testbed nevertheless captures practical task contention, network fluctuations, and device heterogeneity.Source: §6 · p. 14

6.1 System Deployment and Experimental Setup

  • The Raspberry Pi testbed uses four heterogeneous nodes, with the host coordinating task dispatch and result aggregation through RESTful APIs.Each device runs ONNX Runtime for local inference.Source: §6.1 · p. 14, §6 · p. 14, Figure 10 · p. 14

6.2 Inference Accuracy Validation

  • All four models deploy and execute on Raspberry Pi devices after ONNX conversion, with a maximum absolute accuracy difference of 3.44% for M5.The other models show smaller deviations, which may reflect numerical precision, operator, preprocessing, or runtime differences.Source: §6.2 · p. 14, Table 7 · p. 14

6.3 Experiments on Model Parallel Inference

  • An appropriate partition point balances computation and communication overhead, reducing end-to-end latency; the best point varies by model architecture.Early splits transmit larger intermediate features, whereas deeper splits reduce communication but increase computation on the preceding node.Source: §6.3 · pp. 14–15, Figure 11 · p. 15

6.4 Evaluation on Single-Node Multi-Core Parallelism

  • Increasing single-node parallel processes reduces inference time, but ResNet18 speedup rises only from 1.71 with two processes to 2.10 with four.The reported speedups remain below ideal linear scaling, while accuracy remains unchanged across parallelism levels.Source: §6.4 · p. 15, Table 8 · p. 15

7 CONCLUSION AND FUTURE WORK

PROMISE couples committed completion times with predictive workload estimation and receding-horizon scheduling so current assignments account for anticipated downstream resource effects. Experiments examine its adaptability across system scales and workload intensities, and a Raspberry Pi testbed validates practical feasibility and key system characteristics.

7 CONCLUSION AND FUTURE WORK

  • PROMISE evaluates current SD–ES assignments against anticipated downstream resource impact, rather than only instantaneous system states.It couples CCT-based commitments with predictive workload estimation and receding-horizon scheduling.Source: §7 · p. 15
  • Experiments demonstrate adaptability across varying system scales, workload intensities, and prediction horizons.Source: §7 · p. 15
  • Raspberry-Pi experiments validate the practical feasibility of model-parallel deployment and its computation, communication, and privacy characteristics.Source: §7 · p. 15
  • Future work will extend PROMISE to larger hardware testbeds, online stochastic scheduling, richer correlated workload and mobility models, and scalable decision mechanisms.Source: §7 · p. 15

APPENDIX A DETAILED LITERATURE REVIEW

The literature review contrasts PROMISE with prior work in how it models system dynamics, incorporates temporal information, and represents service quality. It positions PROMISE as unifying spatio-temporal prediction with rolling-horizon scheduling and endogenous completion commitments.

View 1: partial dynamics vs. coupled spatio-temporal evolution

  • Prior distributed-inference studies often model dynamic factors individually, while PROMISE unifies stochastic workload, mobility-driven communication, and load-dependent service capacity and predicts their evolution.The review identifies their coupled evolution as insufficiently characterized in prior work.Source: Appendix A · p. 17

View 2: reactive/adaptive scheduling vs. prediction-aware rolling-horizon optimization

  • Adaptive partitioning, offloading, runtime rescheduling, and long-term control do not necessarily expose predicted multi-timeslot workload and resource trajectories to current decisions.PROMISE estimates future arrivals and computational workloads over an adaptive horizon and embeds them in predictive ES-state rollout.Source: Appendix A · p. 17

View 3: metric-/constraint-driven QoS vs. commitment-based service provisioning

  • Prior QoS designs primarily use performance objectives or external constraints, whereas PROMISE makes announced completion time an endogenous decision linked to service utility.Tighter feasible CCTs yield higher service rewards, and failure to meet an announced CCT constitutes a commitment violation.Source: Appendix A · p. 18

TIONS OF PROMISE

PROMISE models privacy-aware partitioning, load-dependent computing, stochastic spatio-temporal conditions, and commitment fulfillment while accounting for cross-timeslot effects. To keep online scheduling tractable, it abstracts selected within-slot and infrastructure overheads and refreshes predictions as new states arrive.

TIONS OF PROMISE

  • Privacy-aware partitioning keeps privacy-sensitive front layers local, evaluates reconstruction leakage using measures such as SSIM, and constrains partition points by each device’s privacy-risk tolerance.This couples privacy preservation with local computation and the residual workload offloaded to the edge.Source: TIONS OF PROMISE · p. 18
  • A nonlinear degradation model captures how concurrent execution reduces per-task computing capability, with β representing sensitivity to increasing concurrency.Admitting a task can affect its own execution rate and the future completion evolution of tasks already hosted at the same edge server.Source: TIONS OF PROMISE · p. 18
  • PROMISE models stochastic task arrivals, changing DNN types and batch numbers, mobility-driven communication conditions, and load-dependent edge capacity.These factors jointly shape future service availability.Source: TIONS OF PROMISE · p. 18
  • Committed completion time is an endogenous platform commitment whose fulfillment is evaluated against realized task completion time, balancing aggressive commitments and reliable fulfillment.Source: TIONS OF PROMISE · p. 18
  • Current assignments reshape future workload occupancy and computing capability, while subsequent stochastic arrivals can affect earlier commitments’ completion feasibility.PROMISE incorporates predicted future demand into rolling-horizon optimization.Source: TIONS OF PROMISE · p. 18
  • Transmission conditions are treated as constant within each scheduling timeslot while varying across timeslots.This averages short-term channel variation within a slot and captures mobility-induced changes through time-varying transmission conditions.Source: TIONS OF PROMISE · p. 19
  • Task arrivals are modeled as independent Bernoulli processes without explicit spatial correlations or event-driven bursts, and predictions are refreshed as new states become observable.The refresh allows deviations from estimated demand to be progressively corrected.Source: TIONS OF PROMISE · p. 19
  • Model-loading and warm-up latency and PT–ES control-plane latency are omitted, while SD-side wireless transmission delay remains explicitly modeled.The omitted model overhead can be incorporated into task workload or ES state; the wired-backhaul assumption applies only to orchestration signaling.Source: TIONS OF PROMISE · p. 19

APPENDIX C DETAILED DERIVATION OF X[τ ′]-TIMESLOT DEMAND ESTIMATION

The appendix derives future edge-side task-arrival and computational-workload profiles by mapping task generation to the timeslot when tasks become schedulable. The resulting expected profiles are recomputed each slot and used as future-demand inputs to predictive ES-state rollout.

APPENDIX C DETAILED DERIVATION OF X[τ ′]-TIMESLOT DEMAND ESTIMATION

  • Future demand is counted by the scheduling timeslot when a task becomes ready for edge processing, rather than by its generation timeslot.Local inference and intermediate-feature transmission occur before edge-side scheduling.Source: Appendix C · p. 19
  • Expected aggregate workload cannot generally be computed as expected task count multiplied by unconditional average per-task workload.Task attributes determine both edge workload and local-computation and transmission delays, which affect the scheduling timeslot.Source: Appendix C · p. 20
  • The predictive demand formulation accommodates discrete task attributes and continuous or empirically characterized transmission conditions without adding a distributional assumption beyond the system model.Source: Appendix C · p. 20
  • The expected task-count and aggregate-workload profiles are recomputed each scheduling timeslot and serve as certainty-equivalent future-demand inputs to predictive ES-state rollout.They describe aggregate demand and do not prescribe future SD–ES assignments.Source: Appendix C · p. 20

APPENDIX D EXPERIMENTAL SETTINGS FOR SEC. 6.3

The model-parallel experiments use two Raspberry Pi nodes to execute partitioned DNN inference, with candidate partition points used to vary computation split and feature-transmission overhead.

APPENDIX D EXPERIMENTAL SETTINGS FOR SEC. 6.3

  • Each DNN is split at a candidate point, with front-end layers executed on one Raspberry Pi and back-end layers on the other.The candidate points are summarized in Table 9.Source: Appendix D · p. 20, Table 9 · p. 20
  • Intermediate features are transmitted between nodes, reproducing the SD–ES partitioned-inference pipeline rather than its large-scale scheduling topology.Large-scale stochastic workloads and dynamic scheduling are evaluated separately in Sec. 5.Source: Appendix D · p. 20
  • Multiple partition points per DNN vary the computation split and intermediate-feature transmission overhead.The candidate partition points are listed in Table 9.Source: Appendix D · p. 20, Table 9 · p. 20

APPENDIX E PRIVACY LEAKAGE EVALUATION IN EDGE ENVIRON-

The appendix evaluates reconstruction-based privacy leakage from intermediate features using feature inversion on a Raspberry Pi model-parallel prototype. Leakage depends on partition depth, with deeper representations generally less structurally reconstructable.

Experimental setup

  • The attack reconstructs inputs from intermediate features at different ResNet18 partition points on CIFAR-10 using gradient-based optimization.It is evaluated on a Raspberry Pi-based model-parallel inference prototype.Source: Appendix E · p. 20

Feature inversion method

  • The joint attack uses features from multiple intercepted layers and matches surrogate-input features against observed intermediate representations.The front-end subnetwork is frozen at the corresponding partition point.Source: Appendix E · pp. 20–21

Privacy leakage results

  • Joint inversion increases average reconstruction SSIM by 12.3% on CIFAR-10 versus single-layer inversion, providing a more stringent privacy evaluation.The objective jointly uses intermediate representations across multiple network depths, with larger weights for shallower layers.Source: Appendix E · p. 21
  • SSIM is 0.4135 at conv1 and 0.1316 at the output, while deeper representations mostly yield 0.15–0.19, indicating reduced structural reconstructability.Layer3.1.conv2 is a local fluctuation at SSIM 0.1904.Source: Appendix E · p. 21, Table 11 · p. 21
  • SSIM, rather than feature-matching loss, is used as the primary indicator because feature loss does not directly measure input-space reconstruction fidelity.The output representation has the lowest feature loss but also the lowest SSIM.Source: Appendix E · p. 21

Related papers

Loading 2609.31018v1…