PaperlayerView exact revision on arXiv

Source-linked AI summary

QEIL v2: Heterogeneous Computing for Edge Intelligence via Roofline-Derived Pareto-Optimal Energy Modeling and Multi-Objective Orchestration

Satyam Kumar, Saurabh Jha

arXiv:2602.06057v3cs.DC

TL;DR

Deploying LLMs on heterogeneous edge devices requires jointly optimizing energy, inference quality, and reliability, while QEIL v1 relied on static modeling, greedy optimization, and unverified selection. QEIL v2 replaces these mechanisms with physics-grounded metrics, Pareto optimization, and progressive verification, achieving consistent cross-benchmark gains including IPW=1.024 on a pre-quantized Llama-3.1-8B.

  • Problem

    LLM deployment on edge devices requires jointly optimizing energy efficiency, inference quality, and reliability, but QEIL v1 used static energy modeling, greedy optimization, and unverified candidate selection.

  • Method

    QEIL v2 combines DASI, CPQ, and Φ in a physics-grounded energy model with PGSAM multi-objective orchestration and EAC/ARDE verified selection with CSVET early stopping.

  • Results

    75.7% pass@k at 63.8W (IPW=0.9749) is reported across three benchmarks and seven model families, while a pre-quantized Llama-3.1-8B reaches IPW=1.024 at 54.8W.

  • Takeaways & Limitations

    QEIL v2 reports physics-grounded orchestration and verified selection as jointly supporting efficient, reliable edge inference, including performance above the IPW=1.0 empirical reference mark.

  • Takeaways & Limitations

    QEIL v2’s evaluation includes one externally prepared RAMP-quantized checkpoint alongside six full-precision models, so quantization itself is not a contribution of this paper.

Abstract

from arXiv · show

Deploying large language models (LLMs) on heterogeneous edge devices demands frameworks that jointly optimize energy efficiency, inference quality, and reliability. Our prior QEIL v1 (Kumar & Jha, 2026) achieved 4.82x IPW improvement but relied on static efficiency factors, greedy optimization, and unverified candidate selection. QEIL v2 replaces every static heuristic with physics-grounded, runtime-adaptive models. We introduce three device-workload metrics: DASI (roofline-derived compute utilization), CPQ (memory pressure from allocation theory), and Phi (thermal yield from CMOS leakage physics), forming a unified energy equation with every coefficient traceable to semiconductor physics. For optimization, PGSAM (Pareto-Guided Simulated Annealing with Momentum) simultaneously minimizes energy, latency, and device underutilization. At inference time, the EAC/ARDE selection cascade with CSVET early stopping provides progressive verification among repeated samples. Evaluated on WikiText-103, GSM8K, and ARC-Challenge across seven model families (125M-8B parameters, including one pre-quantized variant), QEIL v2 achieves 75.7% pass@k at 63.8W (IPW=0.9749), a 2.86x improvement over standard inference. When applied to a 4-bit Llama-3.1-8B, QEIL v2's physics-grounded routing achieves IPW=1.024 at 54.8W -- the first edge orchestration system to surpass the IPW=1.0 empirical reference mark, with the gain attributable entirely to QEIL v2's workload-adaptive device allocation on a model with reduced memory bandwidth requirements. Total energy drops 75.6% vs. standard with 38.3% latency reduction, zero thermal throttling, and 100% fault recovery across all benchmarks and model families.

1 INTRODUCTION

QEIL v2 targets edge LLM deployment under tight power, memory, thermal, and reliability constraints by replacing QEIL v1’s static, greedy, and unverified mechanisms. It combines physics-grounded metrics, Pareto optimization, verified selection, and safety measures, improving results across benchmarks and models.

  • Problem Statement and Motivation: Edge LLM deployment must jointly address 5–85W power envelopes, 8–128GB memory, thermal throttling, and reliable operation.
  • Problem Statement and Motivation: QEIL v1 achieved 4.82× IPW improvement and 47.7% energy reduction but used workload-agnostic energy modeling, greedy optimization, and heuristic candidate selection.
  • Contributions: QEIL v2 replaces static efficiency factors with DASI, CPQ, and Φ, which provide workload-adaptive and runtime-responsive characterizations grounded in semiconductor physics.
  • Contributions: PGSAM simultaneously minimizes energy, pipeline bottleneck latency, and worst-case device underutilization through Pareto dominance and momentum-modulated acceptance.
  • Contributions: +15.9pp accuracy gain is achieved by the EAC/ARDE selection cascade with CSVET early stopping while adaptively conserving energy.
  • Results: 75.7% pass@k at 63.8W (IPW=0.9749) is reported across three benchmarks and seven model families, including a 4-bit Llama-3.1-8B reaching IPW=1.024 at 54.8W.

2 RELATED WORK

Related work establishes foundations in heterogeneous hardware, inference-time scaling, quantization, thermal behavior, and multi-objective optimization. QEIL v2 positions itself as integrating these strands through physics-grounded, layer-level routing and verified repeated-sample selection.

  • QEIL v1: Foundations and Limitations: QEIL v1 combined inference-time scaling, heterogeneous orchestration, efficiency metrics, and safety mechanisms, but retained static energy modeling, greedy assignment, and unverified selection.
  • Inference-Time Scaling: Verified selection extends repeated-sampling methods by using a cascade to convert sample diversity into higher-quality outputs.
  • Hardware-Aware Metrics: QEIL v2 extends query-level routing to sub-query, layer-level routing that adapts to arithmetic intensity, thermal state, and memory pressure.
  • Intelligence Efficiency and Hardware-Aware Metrics: IPW=1.0 is an empirical reference mark, not a theoretical upper bound, corresponding to 1% benchmark accuracy per watt.
  • Foundations: DASI builds on roofline analysis of prefill/decode intensity differences, while PGSAM combines Pareto dominance with momentum-modulated simulated annealing.
  • Model Compression: Post-training quantization reduces memory footprint and bandwidth requirements, while GPTQ and AWQ use uniform 4-bit layer widths and RAMP learns mixed precision.
  • Thermal Modeling: Thermal research links throttling, energy, and mobile-device thermal behavior; QEIL v2 derives Φ from CMOS leakage physics.

3 METHODOLOGY

QEIL v2 uses a four-phase pipeline spanning physics modeling, multi-objective decoder placement, auxiliary routing, and verified inference-time selection. The architecture connects device–workload characterization to final candidate generation.

  • Physics Modeling Engine: Phase 1 computes DASI, CPQ, and Φ from device and workload information to characterize energy, bottleneck time, and minimum DASI.
  • PGSAM Optimization: Phase 2 uses PGSAM to evaluate energy, latency, and negative minimum DASI while selecting the decoder split through weighted Chebyshev scalarization.
  • Auxiliary Placement: Phase 3 routes embedding and LM-head layers to the minimum-energy device.
  • Inference Runtime: Phase 4 applies structural filtering, PEBVC verification, NEAR pool ranking, and CSVET early stopping to select the final best generation.

3.1 Notation and Symbols

Table 1 summarizes the mathematical symbols used throughout QEIL v2’s methodology to support reproducibility.

  • Table 1 summarizes all mathematical symbols used throughout the methodology for reproducibility.

3.2 Phase 1: Physics Modeling Engine

QEIL v2 builds a physics-grounded model of each device–workload pairing using arithmetic intensity, memory pressure, and thermal degradation before assigning layers. These characterizations feed a unified energy equation that captures workload and runtime effects rather than relying on static device factors.

  • Physics Modeling Engine: QEIL v2 characterizes every device–workload combination through arithmetic intensity, memory pressure, and real-time thermal degradation before placement decisions.The three characterizations are combined into the unified energy equation used by PGSAM.
  • Roofline Model: AI below a device’s ridge point indicates memory-bound execution, where compute units idle while still consuming leakage power.LLM decode has AI ≈1 FLOP/byte, far below the GPU ridge point ρGPU ≈218.
  • Roofline Model: 75.7% of GPU compute capacity is not the relevant figure here: LLM decode wastes >99% of GPU compute capacity because its arithmetic intensity is far below the GPU ridge point.The supplied result states that decode wastes >99% of GPU compute capacity during autoregressive execution.
  • DASI: DASI reports useful compute utilization, with CPU decode DASI = 0.125 versus GPU decode DASI = 0.005.The CPU value is 25× higher despite lower absolute throughput, indicating lower proportional idle-compute waste during memory-bound decode.
  • DASI: 1024 FLOPs/byte for FP16 prefill attention exceeds all device ridge points, giving DASI →1.0, whereas decode attention gives DASI ≈0.005 on GPUs.The contrast shows that arithmetic intensity and execution phase change hardware utilization.
  • DASI: 16 FLOPs/byte at batch size B =16 remains below the GPU ridge point but approaches the CPU’s, showing that batch size modulates hardware optimality.At B =1, decode has AI ≈1 FLOP/byte and is memory-bound.
  • Memory Pressure: At context length C =128K, KV-cache memory alone exceeds 6 GB per 24 layers, dominating other memory terms at long contexts.KV-cache memory grows linearly with context length C and batch size B.
  • CPQ: CPQ becomes infeasible at CPQ ≥1.0, while its cubic penalty is calibrated to add about 10% overhead at CPQ = 0.95.The cubic form is intended to remain mild at moderate pressure and steep near the capacity wall.

3.3 Phase 2: PGSAM — Pareto-Guided Simulated Annealing with Momentum

PGSAM formulates layer assignment as a three-objective optimization over energy, bottleneck latency, and minimum device utilization, then searches and selects among Pareto-optimal solutions. Its momentum and reheating mechanisms improve exploration, while experiments report a small gap from the ILP optimum.

  • PGSAM minimizes total pipeline energy, bottleneck latency, and negative minimum DASI to prevent severe device underutilization.
  • Nonconvex energy–latency trade-offs motivate Pareto optimization because weighted-sum scalarization can miss Pareto regions created by heterogeneous assignments.Adding a layer to a nearly full device can discontinuously affect CPQ penalty and bottleneck latency.
  • Boundary-vector encoding enforces contiguous layer assignments, while boundary shifts, block swaps, and rebalances provide local, medium, and large neighborhood moves.The moves have probabilities 0.5, 0.3, and 0.2, respectively.
  • Momentum raises effective temperature during consistent progress, while stagnation triggers reheating to prevent premature convergence.The momentum coefficient is µ = 0.3, and patience P = 30 triggers T ← T × 1.3.
  • After 500 iterations, weighted Chebyshev scalarization selects one deployment solution from the Pareto archive using configurable energy, latency, and utilization priorities.Default weights are w = (0.5, 0.3, 0.2), prioritizing energy, latency, and utilization.
  • 500 iterations achieve a gap below 5% from the ILP optimum, while runtime requires 500 × O(L · D) arithmetic operations and completes in <50ms on any CPU.

3.4 Phase 3: Auxiliary Stage Low-Power Routing

QEIL v2 routes auxiliary stages according to estimated device energy and memory feasibility rather than overall efficiency scores. This particularly targets memory-bound embedding and LM-head operations, whose repeated token-level execution makes savings compound.

  • 99.5% of GPU compute sits idle for the LM head’s memory-bound operations, which QEIL v1 typically placed on the highest-efficiency GPU.The embedding layer and LM head have near-zero arithmetic intensity; the LM head has AI ≈1 FLOP/byte at batch size 1.
  • QEIL v2 routes each auxiliary stage to the lowest-energy device that fits it, typically a 10W NPU or 25W Intel iGPU.The LM head’s V × d projection executes at every token generation step, so per-token savings compound over a generation.

3.5 Phase 4: Inference Runtime — EAC/ARDE with CSVET

The inference runtime generates diverse candidate outputs, progressively verifies them, and ranks survivors by quality and confidence while using CSVET to stop sampling when confidence justifies termination. This concentrates verification and generation energy on promising candidates.

  • QEIL v2 processes each prompt through the EAC inference loop after pipeline compilation.
  • PGSAM’s implementation maintains a Pareto archive, accepts non-dominated or annealing-approved moves, and returns a Chebyshev-selected deployment state.
  • The sinusoidal temperature schedule varies candidate diversity, combining high-confidence low-temperature outputs with exploratory high-temperature alternatives.The schedule is T(i) = Tbase + ∆sin(πi/N).
  • PEBVC progressively filters candidates using structural validity, entropy, self-verification, and cross-sample consensus before ARDE ranking.Structural filtering removes degenerate outputs, while ARDE prioritizes quality, then confidence, with energy as a tiebreaker.
  • The entropy, self-verification, and ARDE thresholds are derived from information-theoretic analysis rather than ad hoc tuning.The reported cuts retain the top 70% by entropy and top 60% by self-verification, with a 1.2-nat confidence margin.
  • CSVET terminates after generating only 10–15 of 25 possible samples on easy prompts, saving 40–60% energy on routine queries.Termination occurs after nmin = max(6, ⌈0.35 × k⌉) when the best candidate exceeds the adaptive confidence threshold.

3.6 Safety and Reliability Framework

QEIL v2 integrates thermal protection into the energy equation through a continuous thermal-yield signal. As devices heat, the optimizer is steered toward cooler devices instead of switching only at a binary throttling threshold.

  • The thermal constraint Ti ≤ 0.85T max i is integrated through Φ, whose decrease with temperature continuously redirects workloads toward cooler devices.This replaces binary “device is fine / device is throttled” behavior with a smooth signal for PGSAM.

4 ABLATION STUDIES

QEIL v2 ablations attribute its gains to physics-grounded routing, Pareto-guided placement, verified candidate selection, and adaptive stopping, while also testing stability, thermal behavior, and fault recovery.

  • Component contribution analysis: −38.1% energy is the largest single reduction from DASI, which routes memory-bound decode to low-power devices.PGSAM reduces power from 98.2W to 72.1W, while EAC/ARDE adds +7.7pp accuracy.
  • PGSAM comparison: PGSAM outperforms greedy by 5.2pp and reduces energy by 7.2%, while matching NSGA-II quality at 3× lower runtime.Its Pareto-dominance search discovers trade-offs in non-convex regions.
  • EAC/ARDE stage contribution: CSVET reduces per-query energy by 12.6% versus full sampling without accuracy loss.The cascade combines entropy filtering, self-verification, cross-sample consensus, and ARDE ranking.
  • PGSAM momentum coefficient: µ = 0.3 produces the largest Pareto archive, whereas µ ≥0.5 over-explores and reduces convergence precision.Without momentum, PGSAM loses 1.9pp accuracy and 3.7% IPW through premature convergence.
  • Power–accuracy trade-off: QEIL v2 strictly Pareto-dominates v1 and standard inference, placing 75.7% accuracy at 63.8W within the fanless thermal envelope.The v2 operating point is inaccessible through convex combinations of the other modes.
  • Thermal validation: Φ-guided allocation prevents thermal stress by shifting workloads toward cooler devices as thermal yield degrades.Without Φ, the NVIDIA GPU reaches 89°C and triggers 47 throttling events; with it, the GPU remains at 62°C in the reported snapshot.
  • Fault tolerance: All four simulated device-failure scenarios recover within 200 ms with zero queries lost.Recovery takes 78 ms for NPU-only loss and 124 ms for GPU loss.

5 RESULTS

Across WikiText-103, GSM8K, and ARC-Challenge, QEIL v2 consistently improves accuracy and energy efficiency across model families and scales, including an externally quantized model.

  • Model scale: 75.7% pass@k at 63.8W is achieved by GPT-2, while Llama-3.1-8B reaches 78.4% at 69.6W.For large models, PGSAM places compute-bound prefill on GPUs and decode on the NPU.
  • Pre-quantized model: IPW=1.024 at 54.8W is achieved by the pre-quantized Llama3-8B-RAMP-4bit under unchanged QEIL v2 orchestration.The passage attributes this gain to routing adapted to the model’s reduced bandwidth profile, not to quantization as a QEIL contribution.
  • GSM8K: −51.7% energy on GSM8K versus −52.2% on WikiText confirms the energy-reduction pattern across task types.Larger GSM8K models show +6.6–14.8pp accuracy gains over standard inference.
  • ARC-Challenge: −52.8% energy is achieved on ARC-Challenge, where LFM2-2.6B reaches 83.8% pass@k at 66.1W.EAC/ARDE contributes +5.2pp over v1’s heuristic selection on short-form outputs.
  • Cross-benchmark consistency: Standard deviation below 0.50pp for coverage and below 1% for energy indicates consistent improvements across three benchmarks.The consistency extends to externally quantized models.
  • Optimization efficiency: ∼218 Pareto-optimal solutions are generated in 42ms, with a <5% gap versus the ILP optimum.The runtime supports online reoptimization during thermal events.
  • State-of-the-art comparison: +10.5pp accuracy, 43.2% lower power, and +68% IPW are reported versus IPW-based routing.QEIL v2 operates at layer-granularity rather than query-level granularity.

6 CONCLUSION

QEIL v2 replaces static heuristics with physics-grounded, runtime-adaptive models and jointly optimizes energy, latency, and device underutilization. Across heterogeneous edge evaluations, it improves efficiency and reliability while identifying future validation and extension needs.

  • DASI, CPQ, and Φ replace static heuristics with physics-grounded, runtime-adaptive metrics whose coefficients are traceable to semiconductor physics.
  • PGSAM jointly minimizes energy, latency, and device underutilization, while EAC/ARDE and CSVET provide verified selection with early stopping.
  • 75.7% pass@k accuracy at 63.8W (IPW=0.9749), with 75.6% total energy reduction and zero thermal throttling events.
  • IPW=1.024 at 54.8W on a 4-bit Llama-3.1-8B, surpassing the IPW=1.0 empirical reference mark through DASI-guided routing.
  • Integrating thermal physics through Φ steers workloads away from hot devices before throttling occurs, supporting zero-throttling and zero-query-loss operation.
  • Future work targets cross-platform validation, dynamic reallocation, distributed inference, additional compression, nontransformer models, learned verification, and formal safety verification.
  • QEIL v2 argues that principled, physics-grounded optimization of the entire inference stack enables energy-efficient and reliable edge intelligence.
Loading 2602.06057v3…