GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon

• 60–90% of deployed ML systems’ lifecycle energy is attributed to inference, and this share grows with daily query volume.Source: §I · p. 1
• Over 70× energy variation across commercial LLMs and 2.61 J per token on Raspberry Pi 4 show why hardware- and model-specific inference measurements matter.Early stopping can reduce energy use by 44–89%.Source: §I · p. 1
• Apple Silicon lacks systematic LLM inference energy profiling, leaving consumer deployments outside established datacenter-focused benchmarks.M-series processors power tens of millions of machines, while unified memory differs sharply from discrete-GPU designs.Source: §I · p. 1, A. Green AI: Efficiency as a… · p. 2
• Unified memory shares one physical pool across CPU, GPU, and Neural Engine, eliminating PCIe or NVLink transfers during inference.This permits zero-copy sharing and allows some quantized models to fit within one memory pool.Source: §I · pp. 1–2
• GreenBench combines direct powermetrics readings with Ollama timing to measure package- and system-level energy across five model families and three NLP tasks.The framework includes per-token energy, grid-specific CO2 estimates, and accuracy–speed–energy Pareto guidance.Source: §I · p. 2

• 25–55% energy-per-token variation occurs when swapping inference engines on a single H100, while configuration changes can cut energy by up to 29% at low concurrency.The configuration gains reverse under heavy load.Source: B. LLM Inference Energy Measurement · p. 2
• 20–70% emission reductions have been demonstrated through carbon-aware scheduling, alongside protocols covering grid electricity and hardware-manufacturing carbon.Source: B. LLM Inference Energy Measurement · p. 2
• Apple Silicon evidence previously consisted of one tuned 1B-parameter M4 result without cross-model comparison or tokens-per-watt reporting.Source: B. LLM Inference Energy Measurement · p. 2
• All five benchmark models use Q4 K M quantization in GGUF format, matching Ollama’s default consumer deployment setup rather than a hand-tuned best case.Source: C. Quantization and Model Compression… · p. 2
• Apple’s UMA is especially relevant because LLM decoding is memory-bandwidth-bound, where eliminating data-transfer overhead should have its largest effect.Existing edge benchmarks cover Raspberry Pi, Jetson, and phone-class SoCs, but not LLM inference on Apple Silicon.Source: D. Edge AI on Consumer Hardware · p. 2

• One Apple MacBook Pro with an M4 Pro chip serves as the experimental platform.Source: A. Hardware Platform · p. 2
• 48 GB of unified memory reaches up to 273 GB/s and lets CPU, GPU, and Neural Engine share data without PCIe transfers.The platform can run roughly 30B-parameter 4-bit models without manual memory management.Source: A. Hardware Platform · p. 3
• Five model families span 3–4B, 7B, and 9B parameter tiers, with distinct attention mechanisms.Source: B. Models Under Study · p. 3
• Model selection prioritizes architectural diversity, consumer-relevant 3–9B sizes, memory fit, and direct availability through the Ollama registry.Source: B. Models Under Study · p. 3
• All models use Q4 K M quantization, reflecting Ollama’s default and common consumer usage rather than a hand-tuned optimum.Source: B. Models Under Study · p. 3

• Three benchmark tasks vary output length and computational pattern to capture distinct inference workloads.Source: C. Benchmark Tasks · p. 3

MMLU (Question Answering)

• MMLU uses 100 multiple-choice questions from 10 subjects, with short answers of around 15 tokens emphasizing recall and short reasoning.Source: C. Benchmark Tasks · p. 3

Summarization

• Summarization condenses 50 truncated CNN/DailyMail articles into 2–3 sentence outputs of roughly 100–200 tokens.Source: C. Benchmark Tasks · p. 3

Code Generation

• Code generation evaluates all 164 HumanEval problems, producing typically 150–300 tokens of structured code without executing test suites.Source: C. Benchmark Tasks · p. 3

Energy measurement

• Energy per token pairs hardware power readings with per-request timing to separate CPU+GPU package energy from whole-system energy.Wall-power meters and manufacturer TDP figures cannot provide this decomposition.Source: D. Energy Measurement Methodology · p. 3

Power measurement

• Powermetrics samples package power every two seconds, while sustained Qwen 2.5 (7B) inference raises GPU power to 115–167 mW.These readings exclude DRAM and peripheral power; system wall power is estimated at 8–12 W.Source: 1) Direct Power Measurement · p. 3

• Ollama provides nanosecond-precision timing for each inference call through its HTTP API.Source: 2) Timing Instrumentation · p. 3
• eval_duration measures output-token generation time, while prompt_eval_duration measures input-prompt processing time.Source: 2) Timing Instrumentation · p. 3
• eval_count records generated output tokens, enabling throughput calculation as θ = 1/ℓ tokens/second.Source: 2) Timing Instrumentation · pp. 3–4
• Per-token energy pairs measured power with per-request generation latency at both CPU+GPU package and whole-system levels.Package measurements use 0.47 W peak inference power; system estimates use approximately 10 W.Source: 2) Timing Instrumentation · p. 4

• The carbon footprint for a workload generating N tokens is computed from N, system energy per token, and grid carbon intensity.Source: 3) CO2 Equivalent Estimation · p. 4
• India uses 708 g CO2/kWh, while the United States uses 390 g CO2/kWh for country-specific estimates.Source: 3) CO2 Equivalent Estimation · p. 4
• Experiments use one model in memory at a time, with a warm-up inference before each batch and Q4 K M quantization for all tested models.Source: E. Experimental Protocol · p. 4

A. Inference Throughput Analysis

• 2.6–4.2× higher throughput is achieved by smaller 3–4B models than by 7–9B models across the evaluated tasks.Source: Figure 1 · p. 4, A. Inference Throughput Analysis · p. 4
• 175.9 tok/s on MMLU, 98.2 tok/s on summarization, and 91.3 tok/s on code generation make Llama 3.2 (3B) the fastest model on all three tasks.Source: A. Inference Throughput Analysis · p. 4
• 1.5–2× faster throughput for MMLU than code generation reflects shorter outputs and lower KV-cache memory overhead.Source: A. Inference Throughput Analysis · p. 4, Figure 1 · p. 4
• 52% is Qwen 2.5’s throughput decrease from 96.9 tok/s on MMLU to 46.2 tok/s on summarization.Source: A. Inference Throughput Analysis · p. 4

B. Energy Efficiency Analysis

• 11.6 mJ per token for Gemma 2 (9B) is 2.6× higher than Llama 3.2 (3B) at 4.4 mJ per token.The reported package power is 0.47 W, while total system power is estimated at 8–12 W.Source: B. Energy Efficiency Analysis · p. 4

Pareto front

• 57% MMLU accuracy at 59 tok/s places Qwen 2.5 (7B) on the Pareto front.Gemma 2 (9B) reaches 61% at 42 tok/s.Source: C. Accuracy–Efficiency Pareto Analysis · p. 5
• 40% higher throughput and 19% lower energy per token give Qwen 2.5 an edge over Gemma 2 despite four points lower accuracy.Source: C. Accuracy–Efficiency Pareto Analysis · p. 5

Dominated model

• 24% MMLU accuracy and 45.8 tok/s make Mistral (7B) a dominated model despite matching Qwen 2.5’s parameter count.The authors suspect a mismatch between Mistral’s instruction tuning and the zero-shot prompt at Q4 K M precision.Source: C. Accuracy–Efficiency Pareto Analysis · p. 5

Smaller models

• 91–175 tok/s makes Llama 3.2 (3B) and Phi-3 (3.8B) the fastest models, but their accuracy falls to 29–41%.Source: C. Accuracy–Efficiency Pareto Analysis · p. 5
• Llama 3.2 retains two-thirds of Qwen 2.5’s MMLU score while delivering three times the throughput and under half the energy per token.Source: C. Accuracy–Efficiency Pareto Analysis · p. 5

Cross-platform comparison

• Table VI compares M4 Pro inference energy with published Raspberry Pi 4 and NVIDIA H100 baselines.Source: D. Carbon Footprint Analysis · p. 5, Table VI · p. 5
• 0.09–0.25 J/tok is the M4 Pro’s system-level energy cost, yielding a 10–29× efficiency gain over Raspberry Pi 4.The M4 Pro ran models 6–18× larger than the Raspberry Pi configuration.Source: D. Carbon Footprint Analysis · p. 5
• 16–44× less energy per token than an H100 is measured for single-user inference on the M4 Pro.The comparison reflects low-volume local workloads, where datacenter batching is unavailable.Source: D. Carbon Footprint Analysis · p. 5

Carbon estimates

• 0.18–0.48 mg of CO2 per 1,000 tokens is estimated for the M4 Pro using India’s 708 g CO2/kWh grid intensity.The exact footprint depends on the running model.Source: D. Carbon Footprint Analysis · p. 5

Unified memory

• 2.0 to 5.4 GB quantized models fit within the M4 Pro’s 48 GB unified memory pool.An 8 GB discrete GPU would struggle to load the 9B model while leaving room for the KV cache.Source: E. Implications for Apple Silicon… · p. 6

Predictable performance scaling

• θ ≈ 340 · Nparams^-0.85 tok/s with R2 = 0.94 enables speed estimates for new model sizes without running them.Nparams is measured in billions; the fit is presented as useful for capacity planning.Source: E. Implications for Apple Silicon… · p. 6

Energy bottleneck

• 5–8× more energy is consumed by HumanEval runs than by MMLU runs, making code generation the energy bottleneck.The authors attribute the overhead to long generation sequences that linearly increase wall-clock time and joules spent.Source: E. Implications for Apple Silicon… · p. 6

Limitations

• Powermetrics reports CPU+GPU package power at 2-second intervals while excluding DRAM power.Because inference is memory-bandwidth-bound, the authors identify DRAM as a significant omitted energy component.Source: §V · p. 6
• Q4 K M is the only evaluated quantization level, leaving other accuracy–energy trade-offs unmeasured.The paper names Q3 K S, Q5 K M, Q8 0, and FP16 as future sweep options.Source: §V · p. 6
• The 100-question MMLU subset is smaller than the 14,042-question full benchmark, so larger samples could reduce accuracy-estimate variance.Source: §V · p. 6
• Single-request inference is evaluated, while batching may produce different power and throughput under shared-memory contention.Source: §V · p. 6
• Results are specific to the M4 Pro with 48 GB and may not transfer directly to other Apple Silicon variants.Those variants differ in core counts, memory bandwidth, and thermal envelopes.Source: §V · p. 6

• 0.47 W of CPU+GPU package power and approximately 10 W of system power were measured during M4 Pro LLM inference.System-level efficiency reached 4–12 tokens per watt.Source: §VI · p. 6
• 30–40× greater energy efficiency per token than datacenter GPUs was achieved for single-user deployment.Source: §VI · p. 6
• 3–3.8B models achieved up to 4.2× higher throughput and 62% lower energy per token than 7–9B models.The relationship followed a predictable power law, θ ∝ N^-0.85.Source: §VI · p. 6
• Qwen 2.5 (7B) offered the best accuracy-efficiency Pareto trade-off, while Llama 3.2 (3B) was optimal for latency-critical applications at 175 tokens/second.Qwen 2.5 achieved 93% of the largest model’s accuracy at 40% higher throughput.Source: §VI · p. 6
• The findings provide actionable deployment guidance and establish the first energy baseline for consumer hardware in the Green AI literature.Future work includes heavier M4 Max hardware, broader precision sweeps, and batched workloads.Source: §VI · p. 6