GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon

Rajeswari Kannan; Raj Firke; Shreya Bengle; Srushti Deshmukh

Source-linked AI summary

GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon

Rajeswari Kannan, Raj Firke, Shreya Bengle, Srushti Deshmukh

arXiv:2608.28667v1Aug 24, 2026cs.CLcs.AIcs.PF

TL;DR

LLM inference’s environmental impact is increasingly important, yet Apple Silicon lacks systematic energy evidence despite its consumer deployment. GreenBench benchmarks five open-source models across three NLP tasks on an M4 Pro using direct power and timing measurements, finding strong efficiency and model-size trade-offs that support deployment choices.

Problem
Inference accounts for 60–90% of deployed ML systems’ lifecycle energy, but Apple Silicon lacks systematic LLM inference energy profiling.
Method
GreenBench measures energy efficiency, throughput, and carbon footprint for five 3–9B open-source LLMs across three tasks on an M4 Pro.
Results
Smaller 3–3.8B models achieve up to 4.2× higher throughput and 62% lower energy per token than 7–9B models.
Takeaways & Limitations
Qwen 2.5 (7B) offers the best accuracy-efficiency Pareto trade-off, while Llama 3.2 (3B) suits latency-critical applications at 175 tokens/second.
Takeaways & Limitations
Package-power measurements use two-second powermetrics intervals and exclude DRAM power, requiring finer-grained validated measurements in future work.

Abstract

from arXiv · show

The rapid proliferation of Large Language Models (LLMs) has raised concerns about their environmental impact during inference. While Green AI research has focused on datacenter GPUs and embedded platforms, the energy profile of LLM inference on Apple Silicon, with its unified memory architecture, remains unstudied. This paper presents GreenBench, a benchmarking framework that evaluates the energy efficiency, throughput, and carbon footprint of five open-source LLMs (3-9B parameters) across three NLP tasks on an Apple M4 Pro with 48 GB unified memory. Using macOS powermetrics for direct power measurement and Ollama's nanosecond-precision timing, we find that the M4 Pro draws only 0.47 W of CPU+GPU package power during sustained inference, with total system power of 8-12 W, achieving 30-40x better energy efficiency per token than datacenter GPUs in single-user deployment. Smaller models (3-3.8B) deliver 2.6-4.2x higher throughput and up to 62% less energy per token than larger models (7-9B). Pareto analysis identifies Qwen 2.5 (7B) as the optimal accuracy-efficiency trade-off at 57% MMLU and 59 tokens/s, while Llama 3.2 (3B) suits latency-critical applications at 175 tokens/s. We provide per-token energy at package and system levels with CO2 estimates for India and US grids.

I. INTRODUCTION

LLM inference has become an important environmental concern as deployment expands beyond cloud APIs, but Apple Silicon remains largely unmeasured despite its consumer adoption and unified memory architecture. GreenBench addresses this gap with reproducible, fine-grained energy benchmarking and deployment guidance.

60–90% of deployed ML systems’ lifecycle energy is attributed to inference, and this share grows with daily query volume.
Over 70× energy variation across commercial LLMs and 2.61 J per token on Raspberry Pi 4 show why hardware- and model-specific inference measurements matter.Early stopping can reduce energy use by 44–89%.
Apple Silicon lacks systematic LLM inference energy profiling, leaving consumer deployments outside established datacenter-focused benchmarks.M-series processors power tens of millions of machines, while unified memory differs sharply from discrete-GPU designs.
Unified memory shares one physical pool across CPU, GPU, and Neural Engine, eliminating PCIe or NVLink transfers during inference.This permits zero-copy sharing and allows some quantized models to fit within one memory pool.
GreenBench combines direct powermetrics readings with Ollama timing to measure package- and system-level energy across five model families and three NLP tasks.The framework includes per-token energy, grid-specific CO2 estimates, and accuracy–speed–energy Pareto guidance.

B. LLM Inference Energy Measurement

Prior inference-energy research emphasizes datacenter GPUs and offers limited Apple Silicon evidence. GreenBench extends this landscape with broader Apple Silicon coverage and energy-per-token measurements relevant to consumer deployment.

25–55% energy-per-token variation occurs when swapping inference engines on a single H100, while configuration changes can cut energy by up to 29% at low concurrency.The configuration gains reverse under heavy load.
20–70% emission reductions have been demonstrated through carbon-aware scheduling, alongside protocols covering grid electricity and hardware-manufacturing carbon.
Apple Silicon evidence previously consisted of one tuned 1B-parameter M4 result without cross-model comparison or tokens-per-watt reporting.
All five benchmark models use Q4 K M quantization in GGUF format, matching Ollama’s default consumer deployment setup rather than a hand-tuned best case.
Apple’s UMA is especially relevant because LLM decoding is memory-bandwidth-bound, where eliminating data-transfer overhead should have its largest effect.Existing edge benchmarks cover Raspberry Pi, Jetson, and phone-class SoCs, but not LLM inference on Apple Silicon.

A. Hardware Platform

The benchmark runs on one 48 GB Apple M4 Pro system selected to represent reproducible consumer deployment across diverse open-source models. Its unified memory and default quantization support the study’s 3–9B model range.

One Apple MacBook Pro with an M4 Pro chip serves as the experimental platform.
48 GB of unified memory reaches up to 273 GB/s and lets CPU, GPU, and Neural Engine share data without PCIe transfers.The platform can run roughly 30B-parameter 4-bit models without manual memory management.
Five model families span 3–4B, 7B, and 9B parameter tiers, with distinct attention mechanisms.
Model selection prioritizes architectural diversity, consumer-relevant 3–9B sizes, memory fit, and direct availability through the Ollama registry.
All models use Q4 K M quantization, reflecting Ollama’s default and common consumer usage rather than a hand-tuned optimum.

C. Benchmark Tasks

GreenBench varies output length and computational pattern across question answering, summarization, and code generation. Energy-per-token is derived from power readings paired with request timing at package and system levels.

Three benchmark tasks vary output length and computational pattern to capture distinct inference workloads.
MMLU (Question Answering): MMLU uses 100 multiple-choice questions from 10 subjects, with short answers of around 15 tokens emphasizing recall and short reasoning.
Summarization: Summarization condenses 50 truncated CNN/DailyMail articles into 2–3 sentence outputs of roughly 100–200 tokens.
Code Generation: Code generation evaluates all 164 HumanEval problems, producing typically 150–300 tokens of structured code without executing test suites.
Energy measurement: Energy per token pairs hardware power readings with per-request timing to separate CPU+GPU package energy from whole-system energy.Wall-power meters and manufacturer TDP figures cannot provide this decomposition.
Power measurement: Powermetrics samples package power every two seconds, while sustained Qwen 2.5 (7B) inference raises GPU power to 115–167 mW.These readings exclude DRAM and peripheral power; system wall power is estimated at 8–12 W.

2) Timing Instrumentation:

The timing instrumentation uses Ollama’s per-call timing fields to isolate output-generation latency and compute throughput and energy at package and system levels.

Ollama provides nanosecond-precision timing for each inference call through its HTTP API.
eval_duration measures output-token generation time, while prompt_eval_duration measures input-prompt processing time.
eval_count records generated output tokens, enabling throughput calculation as θ = 1/ℓ tokens/second.
Per-token energy pairs measured power with per-request generation latency at both CPU+GPU package and whole-system levels.Package measurements use 0.47 W peak inference power; system estimates use approximately 10 W.

3) CO2 Equivalent Estimation:

Carbon footprint is estimated from total generated tokens, system energy per token, and country-specific grid carbon intensity.

The carbon footprint for a workload generating N tokens is computed from N, system energy per token, and grid carbon intensity.
India uses 708 g CO2/kWh, while the United States uses 390 g CO2/kWh for country-specific estimates.
Experiments use one model in memory at a time, with a warm-up inference before each batch and Q4 K M quantization for all tested models.

IV. RESULTS AND DISCUSSION

GreenBench evaluates throughput and energy across five open-source LLMs and three NLP tasks on the Apple M4 Pro. Smaller models are faster, while Qwen 2.5 offers a strong accuracy–efficiency trade-off and system-level efficiency substantially exceeds datacenter GPUs for single-user deployment.

A. Inference Throughput Analysis: 2.6–4.2× higher throughput is achieved by smaller 3–4B models than by 7–9B models across the evaluated tasks.
A. Inference Throughput Analysis: 175.9 tok/s on MMLU, 98.2 tok/s on summarization, and 91.3 tok/s on code generation make Llama 3.2 (3B) the fastest model on all three tasks.
A. Inference Throughput Analysis: 1.5–2× faster throughput for MMLU than code generation reflects shorter outputs and lower KV-cache memory overhead.
A. Inference Throughput Analysis: 52% is Qwen 2.5’s throughput decrease from 96.9 tok/s on MMLU to 46.2 tok/s on summarization.
B. Energy Efficiency Analysis: 11.6 mJ per token for Gemma 2 (9B) is 2.6× higher than Llama 3.2 (3B) at 4.4 mJ per token.The reported package power is 0.47 W, while total system power is estimated at 8–12 W.

C. Accuracy–Efficiency Pareto Analysis

The Pareto analysis compares MMLU accuracy with inference throughput, identifying Qwen 2.5 and Gemma 2 as frontier models while showing speed-oriented advantages for smaller models.

Pareto front: 57% MMLU accuracy at 59 tok/s places Qwen 2.5 (7B) on the Pareto front.Gemma 2 (9B) reaches 61% at 42 tok/s.
Pareto front: 40% higher throughput and 19% lower energy per token give Qwen 2.5 an edge over Gemma 2 despite four points lower accuracy.
Dominated model: 24% MMLU accuracy and 45.8 tok/s make Mistral (7B) a dominated model despite matching Qwen 2.5’s parameter count.The authors suspect a mismatch between Mistral’s instruction tuning and the zero-shot prompt at Q4 K M precision.
Smaller models: 91–175 tok/s makes Llama 3.2 (3B) and Phi-3 (3.8B) the fastest models, but their accuracy falls to 29–41%.
Smaller models: Llama 3.2 retains two-thirds of Qwen 2.5’s MMLU score while delivering three times the throughput and under half the energy per token.

D. Carbon Footprint Analysis

The carbon analysis translates cross-platform energy measurements into deployment-relevant comparisons and estimates the M4 Pro’s emissions under the Indian grid intensity.

Cross-platform comparison: Table VI compares M4 Pro inference energy with published Raspberry Pi 4 and NVIDIA H100 baselines.
Cross-platform comparison: 0.09–0.25 J/tok is the M4 Pro’s system-level energy cost, yielding a 10–29× efficiency gain over Raspberry Pi 4.The M4 Pro ran models 6–18× larger than the Raspberry Pi configuration.
Cross-platform comparison: 16–44× less energy per token than an H100 is measured for single-user inference on the M4 Pro.The comparison reflects low-volume local workloads, where datacenter batching is unavailable.
Carbon estimates: 0.18–0.48 mg of CO2 per 1,000 tokens is estimated for the M4 Pro using India’s 708 g CO2/kWh grid intensity.The exact footprint depends on the running model.

E. Implications for Apple Silicon Deployment

The deployment implications emphasize unified-memory capacity, predictable throughput scaling, and the energy cost of long generation, alongside measurement and evaluation boundaries.

Unified memory: 2.0 to 5.4 GB quantized models fit within the M4 Pro’s 48 GB unified memory pool.An 8 GB discrete GPU would struggle to load the 9B model while leaving room for the KV cache.
Predictable performance scaling: θ ≈ 340 · Nparams^-0.85 tok/s with R2 = 0.94 enables speed estimates for new model sizes without running them.Nparams is measured in billions; the fit is presented as useful for capacity planning.
Energy bottleneck: 5–8× more energy is consumed by HumanEval runs than by MMLU runs, making code generation the energy bottleneck.The authors attribute the overhead to long generation sequences that linearly increase wall-clock time and joules spent.
Limitations: Powermetrics reports CPU+GPU package power at 2-second intervals while excluding DRAM power.Because inference is memory-bandwidth-bound, the authors identify DRAM as a significant omitted energy component.
Limitations: Q4 K M is the only evaluated quantization level, leaving other accuracy–energy trade-offs unmeasured.The paper names Q3 K S, Q5 K M, Q8 0, and FP16 as future sweep options.
Limitations: The 100-question MMLU subset is smaller than the 14,042-question full benchmark, so larger samples could reduce accuracy-estimate variance.
Limitations: Single-request inference is evaluated, while batching may produce different power and throughput under shared-memory contention.
Limitations: Results are specific to the M4 Pro with 48 GB and may not transfer directly to other Apple Silicon variants.Those variants differ in core counts, memory bandwidth, and thermal envelopes.

VI. CONCLUSION

GreenBench establishes an energy baseline for open-source LLM inference on Apple Silicon and identifies model-size and accuracy–efficiency trade-offs. The findings support deployment guidance for consumer hardware while motivating broader future benchmarking.

0.47 W of CPU+GPU package power and approximately 10 W of system power were measured during M4 Pro LLM inference.System-level efficiency reached 4–12 tokens per watt.
30–40× greater energy efficiency per token than datacenter GPUs was achieved for single-user deployment.
3–3.8B models achieved up to 4.2× higher throughput and 62% lower energy per token than 7–9B models.The relationship followed a predictable power law, θ ∝ N^-0.85.
Qwen 2.5 (7B) offered the best accuracy-efficiency Pareto trade-off, while Llama 3.2 (3B) was optimal for latency-critical applications at 175 tokens/second.Qwen 2.5 achieved 93% of the largest model’s accuracy at 40% higher throughput.
The findings provide actionable deployment guidance and establish the first energy baseline for consumer hardware in the Green AI literature.Future work includes heavier M4 Max hardware, broader precision sweeps, and batched workloads.