PaperlayerView exact revision on arXiv

Source-linked AI summary

The Battery Price of edge AI: A study of the Environmental Impact of LLM Inference on Mobile Devices

Édouard Guégain, Tristan Coignion

arXiv:2609.11940v1cs.PFcs.LG

TL;DR

Local-first LLM inference promises privacy and lower latency but shifts computation onto battery-powered devices, creating energy and lifecycle concerns. The paper measures 18 configurations across two smartphones and server deployments, finding that local inference is less energy-efficient and more environmentally impactful than batched server inference. Its results support lifecycle-aware model selection while identifying important measurement and accuracy-assessment limitations.

  • Problem

    The study addresses limited evidence combining smartphone energy, battery-lifespan effects, lifecycle impact, mobile and server inference, and systematic model variation.

  • Method

    The paper measures energy, performance, accuracy, and lifecycle impact for 18 LLM configurations across two smartphones and sequential and batched server deployments.

  • Results

    On-device inference is on average 3× less energy-efficient per token than batched server inference, while local impact is dominated by device embodied carbon.

  • Takeaways & Limitations

    Local AI should not be assumed more sustainable than cloud inference; model selection should account for execution context, battery constraints, and lifecycle impact.

  • Takeaways & Limitations

    Energy per token excludes prompt-prefill energy, and equal weighting of four capped benchmarks may alter relative accuracy rankings.

Abstract

from arXiv · show

The rapid diffusion of generative artificial intelligence raises privacy, latency, and performance concerns that motivate a shift toward "local-first" AI, where inferences are performed on the user's device instead of on remote cloud servers. This paradigm also places a significant computational load on battery-powered smartphones, potentially shortening battery life and increasing the overall replacement rate of mobile devices. This paper presents a systematic study of the energy consumption, performance, and accuracy of on-device large language model (LLM) inference. We evaluate 18 models from different model families, sizes, and quantization levels, on two modern smartphones and on a server, using the respective state-of-the-art for such deployments. We measure the energy per generated token, inter-token latency, model accuracy, and battery-cycle consumption. Our results show that (i) on-device inference is on average 3 times less energy-efficient than batched server inference; (ii) the relationship between quantization bit-width and energy per token is non-monotonic, with energy sweet spots on both tested smartphones; (iii) eight out of 18 model configurations lie on the Pareto front of accuracy and energy-efficiency, allowing practitioners to build battery-aware model routers; and (iv) realistic modeling assumptions do not allow local inference to be less environmentally impacting per token than batched server inference, with 88--90% of that impact attributable to device embodied carbon rather than electricity consumption. These findings challenge the premise that local AI is more sustainable than cloud inference, and motivate the need for context-aware and life-cycle-aware model selection when deploying edge AI on battery-powered mobile platforms.

I. INTRODUCTION

The paper studies whether local LLM inference can meet privacy and latency motivations without imposing unacceptable battery and environmental costs. It measures energy, performance, accuracy, and lifecycle impact across mobile and server deployments.

  • Motivation: Local inference shifts computational load onto battery-powered smartphones, where accelerated battery degradation can contribute to earlier device disposal and manufacturing emissions.Users commonly replace whole devices rather than batteries alone.
  • Study scope: The study addresses a gap by jointly measuring consumer-smartphone energy, battery lifespan effects, mobile and server inference, and varied model configurations.The evaluation spans model families, sizes, and quantization levels.
  • Energy results: On-device inference is on average 3× less energy-efficient than batched server inference, although non-batched single-user server inference consumes 5.4× more energy per token than batching.This comparison distinguishes realistic shared-server deployment from sequential server execution.
  • Model selection: Eight of 18 configurations lie on the mobile accuracy–energy Pareto front, with 4-bit quantization identified as the energy sweet spot on both smartphones.The result supports battery-aware model routing.
  • Lifecycle impact: Local inference remains more environmentally impactful per token than batched server inference, with 88–90% of local impact attributed to device manufacturing.Under the study’s modeling hypotheses, realistic break-even is not achievable with current technology.

II. RELATED WORKS

Prior work measures pieces of mobile or cloud inference, but generally omits a combined treatment of consumer-device energy, battery degradation, lifecycle impact, and server comparison. This study positions itself as filling that integrated empirical gap.

  • Research gap: No existing work simultaneously covers actual consumer-smartphone energy, battery lifespan reduction, mobile and server impacts, and systematic model variation.The gap includes model families, sizes, and quantization levels.
  • Mobile inference: Existing mobile studies often measure accuracy, latency, memory, or energy without combining server comparison with battery-lifespan and lifecycle analysis.The cited studies also use older or heavier models in some cases.
  • Energy studies: Prior edge-energy studies on embedded boards generally lack both consumer-smartphone coverage and server comparisons.Some report that aggressive quantization can increase energy because of computational overhead.
  • Cloud versus edge: Cloud-versus-edge comparisons have omitted independent server measurement, battery depreciation, or manufacturing impact, while one reported over 90% edge savings using a non-batched baseline.The paper identifies these omissions as important differences in comparison design.
  • Reproducibility: The authors provide a replication package containing experiment code, collected measures, and analysis scripts to support independent verification.The study follows established empirical software-engineering and energy-measurement guidelines.

A. Goal and Research Questions

The study evaluates how execution environment and model configuration affect energy, performance, and accuracy of compact LLM inference. It systematically varies model family, size, quantization, device, and server mode.

  • Study design: The study deploys three model families in two sizes and multiple quantization levels across two smartphones and sequential and batched server environments.The design targets energy efficiency, performance, and accuracy trade-offs.
  • Quantization: Quantization uses Q2_K, Q4_0, and Q6_K for on-device evaluation, while F16 provides a server-only full-precision accuracy baseline.F16 configurations exceed mobile memory limits and therefore cannot complete mobile performance assessments.
  • Configuration space: The configuration space combines model family, parameter size, quantization level, and execution environment into 72 configuration–environment pairs.The environments are iPhone 14, Pixel 8, sequential server, and batched server.
  • Performance tasks: Performance and energy are measured with controlled prompts requesting 100, 500, or 1000 output words across subjects sampled from popular Wikipedia articles.The task design enables per-token energy and latency comparisons under controlled output lengths.
  • Accuracy tasks: Accuracy is evaluated separately on the server using CommonsenseQA, GSM8K, HumanEval, and TruthfulQA, each capped at 200 samples.Open-ended generation prompts are not used for reliable accuracy assessment.

D. Protocol

The protocol measures mobile and server inference under comparable sequential conditions while adding continuous batching to represent shared-server deployment. Mobile energy is measured from battery discharge, and latency is collected alongside inference outcomes.

  • Execution environments: The protocol compares Pixel 8, iPhone 14, sequential server execution, and continuous server batching, with sequential modes enabling direct mobile–server comparisons.Batching models a realistic multi-user deployment, while sequential server execution represents a self-hosted single-user scenario.
  • Mobile protocol: Mobile trials use PocketPal with llama.cpp, and the measured window runs from prompt submission until inference completes.The automated journey opens the app, selects the model, submits the prompt, and waits for output.
  • Measurement: Battery discharge is measured through Android system reporting or an external INA260 module on iOS, then converted to energy using nominal battery voltage.Time to first token and inter-token latency are extracted from application screenshots via OCR.
  • Mobile sampling: Each mobile configuration uses 18 trials per device across six topics and three answer lengths, producing 630 total mobile trials.The Qwen 3 4B Q6 configuration could not finish on iPhone 14.
  • Server protocol: Server energy is measured with CPU RAPL and nvidia-smi, while sequential runs submit prompts individually and batched runs submit 100 prompts simultaneously.The server uses an AMD EPYC 7513 CPU and NVIDIA A100-SXM4-40 GB GPU.

E. Collected metrics

The study evaluates energy, accuracy, inference time, and lifecycle impact per generated token across local and remote inference scenarios. Its environmental model combines operational electricity with hardware manufacturing impacts depreciated over device battery throughput or server lifetime.

  • Collected metrics: Energy, accuracy, and inference time are the three evaluation dimensions, with energy normalized as Joules per output token and latency reported through TTFT and inter-token latency.Accuracy is a composite mean across CommonsenseQA, GSM8K, HumanEval, and TruthfulQA.
  • Environmental impact model: The environmental model measures per-token CO2e from both inference electricity and depreciated hardware-manufacturing emissions.Local and remote scenarios share generating one token as their functional unit, while remote inference includes contributions from both the user device and server.
  • Environmental impact model: Local embodied impact is depreciated over total battery energy throughput, using device manufacturing impact, battery voltage and capacity, and rated battery-cycle count.The model treats battery deterioration as a recurring cause of mobile-device replacement.
  • Environmental impact model: Server embodied impact is depreciated over tokens produced during its operational lifetime at a specified throughput and usage rate.The server formulation includes server energy, facility overhead, manufacturing impact, request throughput, output tokens per request, lifetime, and utilization.
  • Parameterization: The environmental parameters draw device manufacturing impacts from product environmental reports and use a DGX A100 reference for server manufacturing impact.The model assumes 1,000 full battery cycles for both smartphones and three years of server operation at 80% utilization.

IV. RESULTS

The results section compares 72 scenarios formed by 18 model configurations across four hardware contexts. It analyzes collected metrics to identify performance patterns and discrepancies between servers and mobile platforms.

  • IV. RESULTS: 72 distinct scenarios compare 18 model configurations across four hardware contexts.The analysis examines collected metrics for patterns and discrepancies between servers and mobile platforms.

A. Patterns between mobile and server inferences

Hardware context strongly differentiates inference latency, energy efficiency, and measurement stability across smartphones and sequential or batched servers. Batched servers are generally most efficient, while mobile inference can outperform non-batched servers on energy but not latency.

  • Latency: Server batching reduces TTFT to 0.12–0.62 seconds, averaging 96–97% lower than other hardware contexts.Smartphone TTFT ranges from 0.24–1.13 seconds on iPhone 14 and 0.48–1.41 seconds on Pixel 8; sequential-server TTFT ranges from 0.72–2.59 seconds.
  • Energy variability: Server energy measurements are highly stable, with CI95% widths below 5%, while some mobile configurations exceed 100% relative CI95% width.Server widths are below 2% in 64% of scenarios, and mobile variability is correlated across the iPhone 14 and Pixel 8.
  • Energy efficiency: Server sequential inference is the least energy-efficient context, server batching the most efficient, and both smartphones lie between them on average.The comparison concerns per-token energy efficiency across fixed model configurations.
  • Hardware comparison: On-device inference is less energy- and time-efficient than batched server inference, whereas non-batched server inference uses more energy than mobile devices but remains faster.The non-batched server also offers better TTFT than mobile devices.
  • Practical implication: Practitioners deploying on-device inference should evaluate model energy and time efficiency across multiple consumer devices to account for user-device discrepancies.The recommendation follows the observed cross-device differences in performance and measurement variability.

B. Model configuration and the accuracy versus energy efficiency trade-off

Model family, parameter count, and quantization substantially affect accuracy and energy consumption, but accuracy and energy efficiency are only weakly correlated. This leaves multiple configurations on a Pareto front for battery-aware selection.

  • Energy consumption: Parameter count has the strongest effect on energy, with configuration transitions ranging from +132.2% to −53.3% and Cliff’s delta of ±100%.Moving from smaller to larger models increases energy, while the reverse transition decreases it systematically.
  • Model family: Model-family changes produce energy impacts from +27.2% to −19.5%, while Qwen 3 is reported as the most accurate family relative to Gemma 3 and Llama 3.2.Family effects are weaker than parameter-count effects, and their direction can depend on the transition.
  • Quantization: Quantization effects on energy range from +25.3% to −19.7%, with Q4 often preferable to Q6 and Q2-to-Q4 benefits not systematic.The Q4-to-Q6 transition has a weaker energy effect than transitions involving Q2.
  • Accuracy: Parameter count also changes accuracy strongly: moving from small to large configurations increases it by 88.5%, while the reverse decreases it by 43%, systematically.These effects use Cliff’s delta of ±100%.
  • Pareto trade-off: Eight of 18 configurations lie on the Pareto front of accuracy and energy efficiency on mobile devices.The weak relationship between the two objectives supports model selection based on battery level and required answer accuracy.
  • Pareto trade-off: Energy per token and accuracy correlate only moderately, with Pearson coefficients of 0.64 on Pixel 8 and 0.56 on iPhone 14.Some transitions, including Gemma 3 to Llama 3.2 and Q2 to Q4, reduce energy while increasing accuracy.

C. Environmental impact of local inferences

Server (Batch) has the lowest environmental impact per token, while smartphone inference is substantially more impactful and is dominated by device embodied carbon. Sensitivity analysis indicates this ordering remains robust under realistic assumptions.

  • Server (Batch) has the lowest footprint at 58–279mgCO2e per 1k tokens, while Pixel 8 and iPhone 14 inference is 7 and 5 times higher, respectively.Server (Seq.) is on average 2.6 times higher than Server (Batch).
  • 84–87% of local inference impact comes from phone embodied emissions, whereas client-side waiting contributes 48.3% of Server (Batch) impact.Client-side impact is only 1.6% on average for Server (Seq.).
  • The local-versus-batched-server ordering remains unchanged under realistic single-factor and joint sensitivity perturbations.The analysis varies electricity mix, battery longevity, server lifespan, and data-centre efficiency, including the two most influential factors jointly.
  • Some highly sustainable mobile configurations outperform the least sustainable server configurations, but equivalent configurations remain more sustainable when hosted remotely.This comparison motivates selecting an efficient model first and then running it remotely rather than locally.

V. DISCUSSION

The discussion explains that energy efficiency depends on hardware-specific quantization behavior and that Pareto-optimal configurations can support battery-aware routing. It concludes that remote inference remains more sustainable for a given configuration, while rebound effects and unmodeled obsolescence qualify the sustainability picture.

  • Unexpected findings: Q4_0 consumes less energy per token than Q6_K and Q2_K on both smartphones and the Nvidia A100, showing a non-monotonic bit-width relationship.On the A100, optimized INT4 kernels help Q4_0, while Q2_K incurs unpacking and control overhead; mobile savings are tied to reduced memory traffic.
  • Implications for mobile application developers: Eight configurations lie on the accuracy–energy Pareto front, enabling routing from high-accuracy models at high battery levels to efficient models as battery decreases.Examples include [Qwen 3, 4B, Q4] and [Llama 3.2, 1B, Q4].
  • Implications for mobile application developers: For a given model configuration, remote inference remains more sustainable than local inference, even though efficient local configurations can outperform inefficient remote ones.The recommended order is to choose the most energy-efficient model meeting user needs, then host it remotely.
  • Implications for end-users: 88–90% of local impact is embodied device carbon, and [Llama 3.2, 1B, Q4] on iPhone 14 reaches 0.301 mgCO2e/tok, comparable to mid-range unbatched server models.Privacy-focused users can narrow the local–remote gap by choosing an efficient configuration, but local inference remains more impactful for a given configuration.
  • Implications for researchers and tool builders: Mobile profiling requires platform-specific evaluation because server-calibrated models and single-hardware quantization results cannot be directly extrapolated across device classes.The discussion also recommends distinguishing inference regimes in benchmarking frameworks.
  • Sustainability considerations: Lower per-token efficiency may be offset by extra follow-up prompts, greater batched-server usage, or hardware obsolescence outside the paper’s scope.Device-longevity policies such as throttling inference as battery health degrades are suggested as a potentially important sustainability lever.

VI. THREATS TO VALIDITY

The study identifies limitations affecting energy comparisons, accuracy aggregation, deployment realism, hardware generalizability, and workload scope. These constraints qualify absolute cross-platform conclusions while leaving some within-platform rankings more robust.

  • Energy measurement reliability: Absolute cross-platform energy comparisons may be biased because mobile and server measurements use heterogeneous methods, although within-platform rankings are unaffected.The authors report low measurement variation, but the threat primarily concerns absolute values across platforms.
  • Energy metric excludes prefill: Reported J/token excludes prompt-prefill energy, potentially underestimating total inference energy when prefill is a substantial runtime fraction.The adopted measurement protocol provided no mitigation for this exclusion.
  • Accuracy as a composite score: The composite accuracy score equally weights four capped benchmarks, so reweighting tasks could change relative model rankings.Gemma 3 and Q2_K variants nevertheless rank consistently below others across individual benchmarks.
  • Backend heterogeneity as an explicit trade-off: Using llama.cpp on mobile and vLLM on servers means part of the mobile-to-server gap may reflect framework efficiency rather than hardware alone.The authors retain platform-specific production backends to represent realistic deployment conditions.
  • Scope of evaluated configurations: Results cover three model families, two 2022–2023 flagship smartphones, and one A100, limiting generalization to other architectures, devices, and server GPUs.The relative effects of quantization and parameter count are expected to transfer more reliably than their magnitudes.
  • Task type limited to text generation: Because measurements use controlled open-ended text generation, workloads with different input-output ratios or long contexts may exhibit different relative energy profiles.Speech-to-text and image-improvement use cases are outside the study’s scope.

VII. CONCLUSION

The paper measures energy, latency, accuracy, and lifecycle impact for local LLM inference across smartphones and a server. It finds that local inference is less energy-efficient and remains more environmentally impactful under the stated modeling assumptions.

  • VII. CONCLUSION: The study evaluates 18 configurations across three model families, three sizes, and four quantization levels on Pixel 8, iPhone 14, and an Nvidia A100.It characterizes energy per token, latency, accuracy, and full lifecycle environmental impact.
  • VII. CONCLUSION: On-device inference is on average 3× less energy-efficient per token than batched server inference, while local inference is 5–7× more environmentally impactful when manufacturing is included.Embodied carbon accounts for 84–87% of local impact in this conclusion’s accounting.
  • VII. CONCLUSION: Changing realistic assumptions about battery cycles, electricity carbon intensity, PUE, or server lifespan does not make local inference more sustainable than remote inference.The authors state that these results challenge the environmental benefits attributed to local inference.
Loading 2609.11940v1…