PaperlayerView exact revision on arXiv

Source-linked AI summary

Energy and Latency First: How Deployment Constraints Reverse Time-Series Model Selection

Shruti Bothe, Illyyne Saffar, Ayush K. Varshney

arXiv:2610.30965v1cs.LG

TL;DR

Accuracy-based selection overlooks hard per-inference deadlines and cumulative energy costs in time-series deployment. The paper evaluates eight architectures using tail-latency feasibility and measured full-pipeline energy, finding that LightGBM leads on accuracy, latency, and duty-cycled energy under the evaluated setting.

  • Problem

    Time-series models are typically ranked by accuracy despite deployment conditions where local predictions must meet deadlines and energy budgets that benchmarks do not measure.

    Source: §1 · pp. 1–2

  • Method

    The paper benchmarks eight architectures by screening tail latency for deadline feasibility, then ranking survivors by measured energy that includes feature computation.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    F1=0.974 for LightGBM, which also has the lowest latency at 0.381 ms p99 and the lowest duty-cycled energy at 4,947 µJ.

    Source: §5.1 · p. 5

  • Takeaways & Limitations

    Under the stated conditions, deployment constraints can invert the accuracy-only model ranking.

    Source: §6 · p. 7

    Energy is estimated from CPU utilization rather than hardware counters, and the reported relative rankings hold on one platform.

    Source: §6 · p. 7

Abstract

from arXiv · show

AI models, and time-series models in particular are typically ranked by accuracy and deployed under constraints absent from that ranking. In this paper, we study a real setting where these constraints are explicit and measurable: handover pre- diction in cellular networks, where inference runs on the User Equipment once per 40 ms measurement period and must complete before the network acts. The timing bound is hard: a late prediction is superseded by the measurement it was predicting, while energy is a cumulative budget spent for the life of the device. Neither appears in the objective these models are selected on. We evaluate eight architectures under a protocol that screens for deadline feasibility at the tail rather than the mean, then ranks the feasible models on measured inference energy, count- ing the feature computation that precedes the forward pass and is usually omitted. Gradient-boosted trees on handcrafted temporal features are the most accurate model (F1=0.974) and among the cheapest to run, while the neural architecture an accuracy-only benchmark would rank first (CNN-1D, F1=0.914) costs over 3 times more energy per inference under a realistic duty-cycled deployment pattern. We give three conditions under which this outcome should be expected, so the finding applies beyond telecommunications. Accuracy-only benchmarking does not merely under-report deployment cost. It can rank first a model that cannot be deployed.

1 Introduction

Accuracy-only benchmarks miss the hard timing and cumulative-energy constraints that shape local time-series deployment. This paper studies those constraints in cellular handover prediction and evaluates model selection under them.

1 Introduction

  • Time-series models are often benchmarked offline or in batches, while edge devices must predict locally, quickly, and within tight energy budgets.Parameter count and FLOPs correlate poorly with device energy, and mean latency does not show whether every inference meets a deadline.Source: §1 · pp. 1–2
  • Handover prediction asks whether the 3GPP A3 condition will be met within a short horizon so the network can prepare proactively.RSRP is reported every 40 ms, and the A3 rule requires a neighbour’s signal advantage to persist through a Time-to-Trigger window.Source: §1 · p. 1
  • Learned mobility management has converged as a response and is now being standardized rather than merely proposed [5] [9].The passage describes learned mobility management as responding to thresholds that do not adapt per deployment.Source: §1 · p. 2
  • The evaluation screens models for deadline feasibility at the latency tail, then ranks survivors by measured energy, including feature computation.It benchmarks eight architectures against accuracy-only selection.Source: §1 · p. 2

2 Related Work

Prior work studies energy accounting and efficient time-series models, but their benchmarks generally do not measure deployment energy and latency together. The paper uses handover prediction to examine that gap.

2 Related Work

  • Energy-accounting methods transfer from NLP and vision, but their benchmarks do not capture time-series input structures, computation patterns, and deployment requirements [15] [16] [17].At fleet scale, inference can dominate a model’s lifetime footprint [17].Source: §2 · p. 2
  • Time-series classifiers are commonly evaluated on accuracy and throughput, while reviews of deep models report accuracy without energy [24].Random convolutional features such as ROCKET [11] and compact interpretable feature sets [12] remain competitive and cheap.Source: §2 · p. 2
  • Compression methods optimize proxies such as parameter count or FLOPs, which correlate only loosely with measured device energy [28] [29] [30].Handover models have been evaluated on accuracy alone, while the energy and latency comparison remains uncommon.Source: §2 · p. 2
  • The paper treats handover prediction as an instrument for studying a regime that accuracy-only benchmarks render invisible, not as a mobility-management contribution.It focuses on the deployment constraints of time-series model selection.Source: §2 · p. 3

3 Problem Formulation

The paper formulates handover prediction as a binary forecast from recent RSRP measurements, then selects models by filtering on timing feasibility and ranking feasible candidates by energy.

3.1 Prediction task

  • The task uses the most recent eight RSRP timesteps across 21 visible cells to predict whether the A3 handover condition will be met within the next eight timesteps.The labels follow the 3GPP A3 rule with Hys = 1 dB, Off = 2 dB, and TTT = 8 steps (320 ms).Source: §3.1 · p. 3
  • Setting the forecast horizon equal to the trigger window makes the task forecast whether an emerging trend will persist, rather than detect a completed trigger.A shorter horizon would overlap more of the trigger; a longer one would make the output too late to act on.Source: §3.1 · p. 3

3.2 Deployment constraints & model selection

  • Inference must complete within the 40 ms measurement period, with latency counting preprocessing, forward, and postprocessing.The screening upper bound is the full period because acquisition and action times are model-independent; feature preparation remains part of model latency.Source: §3.2 · p. 3
  • The protocol screens at p99 latency because a late prediction forfeits its benefit while still consuming energy.A model exceeding the bound at p99 misses the deadline in one of every hundred inferences, or about once every four seconds at 25 Hz.Source: §3.2 · p. 3
  • Table 1 reports full-pipeline latency and per-inference energy under busy-loop and duty-cycled execution, with a ratio showing how looped measurement understates deployment energy.Its benchmark covers the 100 s dataset and reports mean ± standard deviation over five seeds.Source: Table 1 · p. 4
  • Duty-cycled energy includes idle intervals between inferences, whereas busy-loop measurement captures only active-phase energy.The active duty cycle ranges from 0.6% to 4.6% for the evaluated models.Source: §3.2 · p. 4
  • Energy is a cumulative per-inference cost against a device budget, since predictions run once per measurement period throughout deployment.At 25 inferences per second, 1 mJ per inference corresponds to a 25 mW continuous draw; unlike a deadline miss, excess energy shortens battery life rather than breaking function.Source: §3.2 · pp. 3–4
  • Timing is a feasibility condition, so candidates must meet the deadline; energy then orders the feasible models.With an unlimited energy budget and no feasibility filter, the formulation recovers accuracy-only selection.Source: §3.2 · p. 4

4 Experimental Setup

The experimental setup uses a ray-traced mobility dataset and compares neural, feature-based, and capacity-probe models. Cost measurements cover the full inference pipeline under continuous and duty-cycled execution, while ablations isolate representation, capacity, distillation, and early execution.

4.1 Data and models

  • The dataset contains 4.7 million measurements and 596 handover events from 100 UEs moving through a ray-traced Santa Clara scene; results average five seeds.The classification set is split 70/15/15 with stratification and balanced to about 30% positive samples.Source: §4.1 · p. 4
  • The comparison includes three neural baselines, three feature-based models, and two capacity probes; LightGBM uses 247 handcrafted features.MiniRocket and catch22 test domain-agnostic feature baselines, while the Efficient-HO variants probe whether compression alone can recover the neural route.Source: §4.1 · pp. 4–5

4.2 Inference cost measurement

  • Latency and energy are measured on a controlled 36-core Xeon CPU, with absolute values expected to differ on handset silicon.The CPU measurement is intended to represent UE application-processor inference under an isolated, stable idle floor.Source: §4.2 · p. 5
  • Reported latency and energy cover preprocessing, forward, and postprocessing rather than the forward pass alone.Preprocessing includes neural-model normalization or feature computation for feature-based models.Source: §4.2 · p. 5
  • Busy-loop runs back-to-back inferences, while duty-cycled measurement inserts idle pauses and averages energy across the burst schedule.Power is sampled from CPU utilization at 50 ms resolution, so absolute µJ values are estimates while relative comparisons on one platform hold.Source: §4.2 · p. 5

4.3 Ablation protocols

  • Ablations compare LightGBM and a two-layer MLP on engineered features against LightGBM and CNN-1D on the raw window, and sweep Efficient-HO-v2 across seven widths.Distillation uses CNN-1D as teacher and the 7,618-parameter Efficient-HO as student; another comparison tests standard A3 handover against LSTM-assisted early execution.Source: §4.3 · p. 5

5 Results

Across the benchmark, LightGBM leads on accuracy and deployment cost, while feature computation and realistic idle periods materially affect measured inference cost. Ablations indicate its advantage comes from model class rather than privileged features, and capacity increases or distillation do not close the neural models’ accuracy gap.

5.1 Energy-Accuracy Benchmark

  • F1=0.974 makes LightGBM the most accurate model; it also has the lowest latency (0.381 ms p99) and costs 4,947 µJ per inference under duty cycling.It is 2.2× below the next-cheapest model; CNN-1D, the best neural model on accuracy (F1=0.914), costs over 3× LightGBM’s per-inference energy, and every neural model is dominated.Source: §5.1 · pp. 5–6, Figure 1 · p. 6

5.2 Inference Cost Decomposition

  • 0.116 ms of LightGBM’s p50 inference is feature computation, 44% of its total; omitting it makes LightGBM appear roughly twice as fast.MiniRocket’s 10k-kernel transform dominates its otherwise negligible ridge pass; neural-model preprocessing is negligible beside the forward pass. Training is excluded as a one-off cost, with inference dominating lifecycle energy at roughly 10^9 inferences per device-year.Source: §5.2 · p. 6

5.3 Feature vs. Architecture Ablation

  • F1=0.974 versus 0.971 on raw input shows engineered features add only 0.003 for LightGBM; the tree’s advantage comes from model class, not privileged features.On the same engineered features, the two-layer MLP reaches F1=0.929, above CNN-1D on raw input (F1=0.919), indicating representation matters more than architecture for the neural models.Source: §5.3 · pp. 6–7, Table 3 · p. 6

5.4 Compression Cannot Rescue the Neural Route

  • F1=0.924 at 58K parameters is the best capacity-sweep result, still below LightGBM’s F1=0.974; distillation lowers student F1 from 0.438 to 0.407.The apparent +0.08 distillation gain in earlier 50 s experiments disappears on the full 100 s split; the reported secondary experiment finds an LSTM confidence gate can save up to 542 ms per handover at 100% success rate.Source: §5.4 · p. 7

6 Discussion: Generalizability and Limitations

The discussion argues that the ranking reversal should extend beyond telecommunications when decision boundaries are simple, sequences are short, and inference is duty-cycled. It also identifies limits on the energy estimates and the tested deployment setting.

Generalizability

  • Simple or known decision boundaries favor tree models because features such as min-margin-to-neighbour directly capture the sustained linear A3 rule.Neural models must learn that rule from raw sequences.Source: §6 · p. 7
  • With T = 8 steps, temporal inductive biases are barely exercised, leaving attention over eight positions little advantage over direct feature computation.Source: §6 · p. 7
  • Under deadline-driven inference once per period, energy is dominated by leaving and re-entering idle, a cost throughput benchmarking amortizes away.The passage therefore calls throughput inappropriate for this regime.Source: §6 · p. 7

Limitations

  • Energy estimates use CPU utilization rather than hardware counters; the authors say relative rankings hold on one platform, while other environments and mobility patterns remain untested.The dataset is an urban Santa Clara scene, and the ray-traced simulation has less temporal diversity than field deployment.Source: §6 · p. 7
  • The MiniRocket and catch22 accuracy could improve with a stronger classifier, but their feature-transform cost would not change.Source: §6 · p. 7

7 Conclusion

The conclusion evaluates eight architectures for time-series classification under hard deadline and cumulative energy constraints, using tail-feasibility screening and measured inference energy. It reports that feature-based gradient-boosted trees lead the benchmark and that measurement choices materially affect estimated costs and energy advantages.

7 Conclusion

  • Gradient-boosted trees on handcrafted features are the most accurate, fastest, and cheapest model; capacity sweeps and distillation do not bring a neural model close.Source: §7 · p. 7
  • Counting feature computation nearly doubles LightGBM’s apparent cost, while duty-cycled rather than tight-loop measurement narrows its energy advantage from 10.4× to 3.3×.Source: §7 · p. 7
  • The evaluation screens deadline feasibility at the tail, then ranks surviving models by measured energy, including feature computation.Source: §7 · p. 7
  • Under the paper’s three stated conditions, accuracy-only benchmarking can rank first a model that cannot be deployed.Source: §7 · p. 7

A Dataset generation

The dataset is generated from a ray-traced urban cellular simulation with vehicular user movement and 40 ms measurements. Labels are derived from full RSRP traces before window extraction and dataset splitting.

A Dataset generation

  • The Santa Clara simulation has 7 base-station sites with 3 sectors each, yielding 21 cells; 100 vehicular UEs are sampled every 40 ms over 100 seconds.Source: A Dataset generation · p. 11
  • Ray tracing produces 2,487 timestamps and 4.7 million raw measurements, containing 596 handover events.Source: A Dataset generation · p. 11
  • Labels are generated from full RSRP traces before stride-one sliding windows are extracted; negatives are subsampled to about a 30% positive rate, then data are stratified into 70/15/15 train, validation, and test splits.Source: A Dataset generation · p. 11
  • Neural models receive RSRP normalized from [−150, −40] dBm to [0, 1], while feature-based models use engineered statistics computed on unnormalized values.Source: A Dataset generation · p. 11

B Architectures and training

The benchmark spans neural, feature-based, and capacity-probe architectures, with LightGBM receiving 247 engineered features. Those features include statistics and quantities designed to encode the handover labeling rule.

B Architectures and training

  • LightGBM uses 247 engineered features, while the evaluated model set includes neural baselines, feature-based models, and capacity probes.Table 4 summarizes the evaluated models; tree parameter counts are total leaves across boosting rounds, while MiniRocket and catch22 counts are feature dimensions.Source: Table 4 · p. 11

B.1 Neural baselines

  • The neural baselines include LSTM, Transformer, and CNN-1D architectures applied to the measurement window.The supplied descriptions specify their layers, widths, pooling, and parameter counts.Source: §B.1 · p. 11

B.2 Feature-based models

  • Feature-based models include LightGBM over 247 features, MiniRocket with a ridge classifier, and catch22 characteristics with a ridge classifier.MiniRocket and catch22 feature counts are 9,996 and 462, respectively.Source: §B.2 · pp. 11–12

B.3 Capacity probes

  • Capacity probes include an early-exit recurrent model and an Efficient-HO-v2 depthwise-separable CNN whose channel width is swept from 16 to 128.Efficient-HO skips recurrent processing when its exit confidence exceeds the stated threshold.Source: §B.3 · p. 12

B.4 Training configuration

  • Neural models are trained with Adam, binary cross-entropy, up to 80 epochs, early stopping, and five specified random seeds on a fixed partition.Source: §B.4 · p. 12

C Engineered feature set

  • The 247-feature representation includes per-cell variability and trend, per-timestep maxima, and the serving-to-neighbour min-margin.The min-margin group evaluates the A3 quantity per timestep, while the trend group stands in for its sustained-over-TTT character.Source: C Engineered feature set · p. 12

D Ablation protocols

The ablations vary feature representation, model family, and channel width, and specify a CNN-to-Efficient-HO distillation setup. The section also describes LSTM-based early execution during the handover countdown.

Features versus architecture

  • The representation ablation crosses 247 engineered features and a 168-value raw window with LightGBM and a two-layer MLP, with CNN-1D included on raw input.All cells use the split and seeds of Appendix A.Source: D Ablation protocols · p. 12

Capacity

  • Efficient-HO-v2 is trained at seven channel widths with other settings fixed to isolate capacity from architecture.Source: D Ablation protocols · p. 12

Knowledge distillation

  • The distillation setup uses CNN-1D as teacher and the 7,618-parameter Efficient-HO as student, with TKD=4, α=0.3, 100 epochs, and patience 10.Teacher and student use the same split.Source: D Ablation protocols · p. 13

Early-execution handover

  • During the TTT countdown, an ONNX-exported LSTM predicts future RSRP for early execution; at TTT=640 ms, ping-pongs fall from 5 to 2.The 542 ms average is over 223 early-executed handovers, not a paired comparison with the baseline’s 489 A3 triggers.Source: D Ablation protocols · p. 13

E Training cost

The handover simulation reports LSTM early execution relative to baseline A3 at TTT=640 ms. Training-cost measurements compare LightGBM and CNN-1D on the server CPU.

E Training cost

  • LightGBM trains 40.6× faster and uses 37.7× less energy than CNN-1D while reaching the highest validation accuracy.At 25 Hz, the training-cost difference is amortized within seconds; training energy is not a deployment constraint in the selection protocol.Source: E Training cost · p. 13, Table 5 · p. 13

F Handover Simulation

  • At TTT=640 ms, LSTM early execution saves an average of 542 ms per handover at 100% success while reducing ping-pongs.Table 6 compares baseline A3 (Algo 0) with LSTM early execution (Algo 4) at the best configuration.Source: Table 6 · p. 13, F Handover Simulation · p. 13

Related papers

Loading 2610.30965v1…