PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving the Energy-Efficiency of the Code Generated by LLMs through Effective Prompting

Ritika Rekhi, Bing Zhang, Md Arman Islam, Jaya Krishna Pasham, Yeswanth Chitturi, Akshay Paramesha, Isha Valiveti, Asif Imran, Bekir Turkkan, Tevfik Kosar

arXiv:2610.02571v2cs.SEcs.AI

TL;DR

Because runtime energy from LLM-generated code recurs each time it runs, this study tests energy-aware prompting across models and Python and C++. Prompts often lowered energy, but their effects varied by model and task.

  • Problem

    Whether prompting can improve the energy efficiency of LLM-generated code matters because its runtime energy recurs each time the code runs.

    Source: §I · p. 1

  • Method

    The study screens 21 prompting techniques, selects 8, and evaluates them across 10 models on 878 problems in Python and C++.

    Source: §I · pp. 1–2

  • Results

    Prompting often reduced energy; Python reductions reached about 50% for Granite-4.0-H-Small, while C++ reductions reached 57% for Granite-4.0-H-Small.

    Source: §V · p. 14

  • Takeaways & Limitations

    Energy-aware prompting can elicit lower-energy code in many models, but the results indicate that models do not reliably optimize for energy across tasks and models.

    Source: §V · p. 14

    The benchmark has more medium-difficulty problems than easy or hard ones, does not cover all practical programming tasks, and includes only Python and C++.

    Source: §V · p. 14

Abstract

from arXiv · show

As AI-assisted programming becomes increasingly mainstream, the environmental impact of AI-generated software has emerged as an important consideration. This motivates evaluating LLM-generated code beyond functional correctness by considering execution efficiency and energy consumption. However, despite substantial advances in code generation, frontier LLMs are rarely evaluated based on the energy efficiency of the code they produce. In this work, we conduct a comprehensive evaluation of 21 prompting strategies for energy-efficient code generation and identify 8 strategies for evaluation across 10 widely used open-weight and proprietary LLMs. We evaluate their effectiveness for both Python and C++ code generation relative to a baseline prompt. Across the evaluated models, the selected prompting strategies achieved energy reductions of up to 25% for Python and 17% for C++ code generation. At the model level, Python energy reductions reached up to 50% for Granite-4.0-H-Small, 39% for Claude 4.5 Haiku, and 28% for MiniMax M3, while C++ reductions reached up to 57% for Granite-4.0-H-Small and 6.4% for Qwen3-Coder-480B-A35B-Instruct. These results demonstrate that prompting strategies can substantially influence the energy consumption of LLM-generated code, although their effectiveness varies across models and programming languages. Our findings highlight the importance of incorporating energy efficiency into the evaluation and optimization of LLM-based code generation and provide practical insights into designing prompts for more sustainable AI-assisted programming.

I. INTRODUCTION

The introduction motivates energy-efficient LLM-generated code and examines whether prompting can improve it across models and languages. Prior work reports mixed effects from prompting, with most evidence focused on Python.

I. INTRODUCTION

  • LLM-generated code can consume far more energy than canonical human-written solutions, and its runtime energy recurs each time the code runs.The introduction attributes inefficiency in part to heavier algorithms, extra data structures, and recursive logic.Source: §I · p. 1
  • Prompting evidence for energy reductions is mixed and has focused almost entirely on Python, motivating evaluation across languages and models.Prompting is presented as an alternative to costly model retraining.Source: §I · p. 1
  • The study screens 21 prompting techniques, selects 8, then evaluates them across 10 models on Python and C++ problems.The extended EffiBench evaluation uses 878 benchmark problems.Source: §I · pp. 1–2

II. RELATED WORK

  • Green-software research spans energy-leak detection, code and library efficiency, programming-language comparisons, and calls to treat sustainability as a software quality property.The cited work includes guidelines for energy-efficient Python code and measurements across 27 languages.Source: §II · p. 2
  • Prior work finds LLM code often inefficient, while prompting, profiling feedback, and retrieval-based optimization have produced varying results across models, tasks, and metrics.Energy-specific improvements remain less consistent than some broader code-efficiency approaches.Source: §II · pp. 2–3

III. METHODOLOGY

The study uses EffiBench’s LeetCode-derived Python problems and extends the benchmark to support consistent C++ evaluation. The dataset also records problem publication dates for temporal analysis.

A. Problem Dataset Selection

  • EffiBench provides 878 Python-solvable LeetCode problems with standardized statements and test cases for evaluating correctness.The benchmark includes problems from 11 algorithmic categories and three difficulty levels.Source: A. Problem Dataset Selection · p. 3
  • The dataset adds each problem’s publication date, spanning 2015 through 2023, to support analysis relative to model training data.Dates come from LeetCodeDataset.Source: A. Problem Dataset Selection · p. 3
  • The C++ extension converts Python test cases into C++-compatible formats and standardizes representations for structures such as linked lists, graphs, and trees.It also supplies headers and boilerplate for consistent compilation and execution.Source: A. Problem Dataset Selection · pp. 3–4
  • Figure 1 depicts code generation for each prompting strategy across both evaluation stages.Source: Figure 1 · p. 4

B. LLMs Under Study

The evaluation covers three proprietary and seven open-weight LLMs. Code generation proceeds in two stages: screening 21 prompts, then evaluating 8 selected strategies on the extended benchmark.

1) Selected LLMs:

  • The study evaluates three proprietary and seven open-weight LLMs, documenting model configurations and knowledge cutoffs for context.The benchmark problems were published through the end of 2023, so the authors consider exposure during training likely for most or all problems.Source: 1) Selected LLMs · p. 4

2) LLM Configuration:

  • Model inference uses provider-recommended settings where available, with batch APIs and sampling used to reduce calls when supported.Qwen3-Coder-480B-A35B-Instruct uses a repetition penalty of 1.05 per its provider’s recommendation.Source: 2) LLM Configuration · p. 4

C. LLM Code Generation

  • The initial analysis evaluated 21 prompting strategies across selected models on 110 problems, using the EffiBench baseline prompt as reference; 8 strategies were then evaluated on the extended dataset with the baseline.Source: C. LLM Code Generation · p. 4
  • Each problem receives 10 independent code samples per model and prompt; multi-turn prompts apply later turns to base-prompt samples without intermediate reasoning context.Only the preceding response is passed to the next turn.Source: C. LLM Code Generation · p. 4
  • Responses from both stages are retained for correctness checks and fine-grained energy benchmarking, with the generation workflow shown in Figure 1.Source: C. LLM Code Generation · p. 5

D. LLM Code Evaluation

Generated Python and C++ solutions pass language-specific correctness checks before energy measurement. Measurements use idle-baseline subtraction, repeated executions, and cooldown intervals to improve reliability.

D. LLM Code Evaluation

  • Python solutions must pass all small tests and 100 large tests within 120 seconds; C++ solutions must pass every executable’s test within 10 seconds.Correct solutions are retained for energy measurement, using Python scripts and the C++ executables for large test cases.Source: D. LLM Code Evaluation · p. 5

E. Energy Measurement Setup

  • Energy measurement records a 3-second idle baseline, then runs each Python solution 30 times and each C++ executable 40 times, with 3-second cooldowns.The repetitions are intended to keep total measurement duration above one second for more reliable energy-counter sampling.Source: E. Energy Measurement Setup · p. 5
  • Average energy per execution is calculated after subtracting estimated idle energy, and the procedure is repeated twice before averaging results.All validated solutions are compared on the same hardware under identical execution conditions.Source: E. Energy Measurement Setup · pp. 5, 7
  • Table II summarizes the 21 initial prompting strategies and identifies the 8 selected for final Python and C++ evaluation.Figure 5 separates the five single-turn strategies from the three multi-turn strategies.Source: Table II · p. 6, Figure 5 · p. 7

F. Environment Setup

Energy was measured on specified hardware and software, with shared hardware configurations supporting comparisons; prompts were designed to test energy-efficiency guidance without external tools or feedback.

F. Environment Setup

  • Energy measurements used Intel Skylake and Cascade Lake instances, perf readings of package and RAM energy, Ubuntu 24.04, Python 3.12.3, and g++ 13.3.0 with C++20 and -O2.Solutions for each model and prompt were measured on the same hardware configuration.Source: F. Environment Setup · p. 7

G. Evaluating Prompting Strategies for Energy-Efficient Code Generation

H. Finalised Prompting Techniques for Energy-Efficient Code Generation

The authors screened 21 prompts on a stratified Python subset, then selected eight for broader evaluation; the finalists include single-turn and refinement-based multi-turn designs.

H. Finalised Prompting Techniques for Energy-Efficient Code Generation

  • The eight finalists comprised five single-turn prompts and three multi-turn prompts that add an energy-efficiency refinement turn to a baseline or single-turn prompt.Python and C++ prompt instances differed only in their language-specific reference implementation.Source: H. Finalised Prompting Techniques for… · p. 8
  • The baseline assigns a code-developer role and requests code passing sample tests, with a worked example and the task to solve.The example includes a task description, sample test cases, and a language-specific reference solution.Source: H. Finalised Prompting Techniques for… · p. 8
  • Single-turn variants combine efficiency guidelines and developer personas with examples of efficient and inefficient code drawn from LeetCode-style problems.Prompts 10 and 11 use examples outside the evaluation set; Prompt 12 adapts Ashraf et al. [12], while Prompts 4, 10, 11, and 18 add distinct guideline or example designs.Source: H. Finalised Prompting Techniques for… · p. 8
  • The multi-turn prompts extend base Prompts 0, 4, and 10 with a second instruction to generate energy-efficient code, following Niu et al. [10].Source: H. Finalised Prompting Techniques for… · p. 8

A. Initial Analysis for Prompt Selection

  • Screening tested 21 strategies on 110 problems, ten from each of 11 algorithmic categories, across Kimi K2.7 Code, MiniMax M3, and Granite-4.0-H-Small using Python.Selection was based on the assumption that prompt effectiveness would remain consistent across programming languages.Source: A. Initial Analysis for Prompt Selection · p. 8
  • Figure 7 presents average percentage energy reductions for the eight selected Python prompts relative to baseline, with negative values indicating lower energy use.Source: Figure 7 · p. 9
  • Only 59 problems where all three models returned correct solutions under all 21 prompts and the baseline were retained for prompt selection.Each retained problem also had to have at least two correct solutions from the model.Source: A. Initial Analysis for Prompt Selection · p. 9
  • In the initial analysis, the eight strongest prompting strategies reduced energy use by 34%–49% relative to the baseline, based on averages across 110 Python problems and three LLMs.These strategies were selected for evaluation across ten models and both Python and C++ code generation.Source: Figure 6 · p. 8, A. Initial Analysis for Prompt Selection · p. 9

B. Analysis of 8 Energy-Efficient Prompting Strategies Across all Models

Across the shared benchmark, prompt effects varied by model, language, and strategy: some models improved overall, but individual strategies could increase energy use.

  • The common correctness-filtered benchmark contained 537 Python and 607 C++ problems across all models and prompting strategies.The analysis retained problems solved correctly by every model under every prompt for that language.Source: B. Analysis of 8 Energy-Efficient… · p. 9
  • All evaluated models showed Python energy improvements, while five models showed higher overall reductions for C++; effects nevertheless varied by model, language, and prompt.Some prompts reduced energy while others increased it for the same model and language, and a model could show different trends across Python and C++.Source: B. Analysis of 8 Energy-Efficient… · p. 9
  • For Python, GPT-OSS-120B saved energy with Prompts 4, 10, 11, 12, 18, and 21, but used more energy with Prompts 13 and 20.Source: B. Analysis of 8 Energy-Efficient… · p. 9
  • Figure 8 covers Python prompt impacts by problem difficulty and algorithmic category, aggregated across evaluated models.Source: Figure 8 · p. 10

C. Evaluation of Prompts on Python Code Generation

Python energy reductions differed substantially across models and difficulty levels; aggregate reductions did not prevent model-specific prompt increases.

1) Dataset Distribution:

  • The Python benchmark comprised 537 commonly solved problems: 333 medium, 110 easy, and 94 hard, with Dynamic Programming, Sorting, and Greedy most represented.Source: 1) Dataset Distribution · p. 10

2) Overall Impact of Prompts on All the LLMs under Study:

  • Granite-4.0-H-Small reached a 50% reduction with Prompt 20, Claude 4.5 Haiku a 39% reduction with Prompt 21, and MiniMax M3 a 28% reduction with Prompt 13.Source: 2) Overall Impact of Prompts on All… · p. 10
  • Other models also showed reductions: Gemini 3.8 Flash reached 11% with Prompts 20 and 21, while Gemma 4 26B A4B IT reached 8% with Prompts 21 and 13.The passage also reports reductions for GPT-5.6 Terra, GPT-OSS-120B, Kimi K2.7 Code, Muse Glimmer 30B, and Qwen3-Coder-480B-A35B-Instruct.Source: 2) Overall Impact of Prompts on All… · p. 10
  • GPT-OSS-120B increased energy use by 74% with Prompt 13 and 33% with Prompt 20, while Claude 4.5 Haiku increased it by 9% with Prompt 12.The direction and magnitude of relative changes varied across models and prompts.Source: 2) Overall Impact of Prompts on All… · p. 10

3) Impact of Prompts on Problem Difficulty:

  • Across difficulty levels, all prompts reduced energy overall: hard-problem reductions reached 33%, medium-problem reductions reached 28%, and easy-problem changes ranged from 1% to 3%.Prompt 13 was an exception to the substantial hard-problem reductions, at 5%.Source: 3) Impact of Prompts on Problem… · p. 10
  • Difficulty-level aggregates concealed increases, including Claude 4.5 Haiku’s 67% increase on hard problems with Prompt 4 and GPT-OSS-120B’s 109% increase on medium problems with Prompt 13.Qwen3-Coder-480B-A35B-IT also increased energy use on easy problems for every prompt except Prompts 11 and 18.Source: 3) Impact of Prompts on Problem… · p. 10
  • Table IV describes C++ benchmark-problem distribution by difficulty and algorithm category; Figure 9 reports C++ energy reductions relative to baseline, with negative values indicating lower energy use.Source: Table IV · p. 11, Figure 9 · p. 11

4) Impact of Prompts on Algorithmic Category:

Across algorithmic categories, prompting usually reduced C++ energy consumption, but the size and direction of changes varied by category and model. Prompt 21 was consistently strong in several categories, while some prompts produced substantial model-specific increases.

4) Impact of Prompts on Algorithmic Category:

  • For Backtracking, Prompt 21 achieved −38%; for BFS, it achieved −30%, with Prompt 20 increasing Backtracking energy use for MiniMax M3 by +43%.For Dynamic Programming, Prompt 20 led at −32%, but Prompt 4 increased Claude 4.5 Haiku energy use by +68%.Source: 4) Impact of Prompts on Algorithmic… · p. 11
  • Prompt 21 reduced energy use by −47% for Bit Manipulation, while Prompt 20 increased it for MiniMax M3 (+41%) and GPT-OSS-120B (+11%).Across categories, reductions varied by algorithm and model.Source: 4) Impact of Prompts on Algorithmic… · p. 11, Figure 10 · p. 12
  • Greedy had the largest category-level reduction: Prompt 11 achieved −46%, while Prompt 20 achieved −38%.Prompt 12 increased energy use for Claude 4.5 Haiku (+29%) and Kimi K2.7 Code (+21%).Source: 4) Impact of Prompts on Algorithmic… · p. 11
  • More modest but consistent reductions were observed: Prompts 21 and 18 produced the largest reductions for Divide and Conquer (−9%, −8%) and Sliding Window (−12%, −11%), respectively.Source: 4) Impact of Prompts on Algorithmic… · p. 11
  • Two Pointers and Sorting showed small mixed changes, while DFS and Binary Search generally changed little or increased energy use under some prompts.Prompt 21 increased Two Pointers energy use by +7%; Binary Search increases reached +8% with Prompt 21 and +4% with Prompt 4.Source: 4) Impact of Prompts on Algorithmic… · p. 11

5) Overall Best Performing Prompts for All Models:

  • Across Python models, Prompts 21, 11, 18, and 10 reduced energy use by −15%, −12%, −9%, and −9%, respectively.Prompt 21 extends Prompt 10 with an additional energy-efficiency prompting step and achieved a larger reduction (−15% versus −9%).Source: 5) Overall Best Performing Prompts for… · pp. 11–12

D. Evaluation of Prompts for C++ Code Generation

C++ prompting effects varied across models: some models had large reductions, while others showed narrow changes or increases. The common evaluation set contained 607 problems solved by all ten models across every prompting strategy.

1) Dataset Distribution:

  • The common C++ evaluation set contained 607 problems solved correctly by all ten models under every prompting strategy.It included 373 Medium, 121 Easy, and 113 Hard problems.Source: 1) Dataset Distribution · p. 12
  • The common set represented several categories, led by Greedy (178), Dynamic Programming (176), and Sorting (174).Binary Search had 97 problems; Bit Manipulation and Two Pointers had 70 each.Source: 1) Dataset Distribution · p. 12

2) Overall Impact of Prompts on All the LLMs under Study:

  • Figure 9 presents average C++ energy decreases for each prompt relative to the baseline.The reported averages compare generated code against the baseline prompt.Source: 2) Overall Impact of Prompts on All… · p. 12
  • Granite-4.0-H-Small reached a 57% energy reduction across nearly all prompts, while Qwen3-Coder-480B-A35B-Instruct reached 6.4%.For Qwen3-Coder-480B-A35B-Instruct, Prompt 11 instead increased energy use by 4%.Source: 2) Overall Impact of Prompts on All… · p. 12
  • Muse Glimmer 30B, MiniMax M3, Gemini 3.8 Flash, and GPT-5.6 Terra showed narrow energy-change ranges around their baselines.Their respective ranges were −0.44% to 0.56%, −0.79% to 0.83%, −0.15% to 1.46%, and −1.06% to 1.8%.Source: 2) Overall Impact of Prompts on All… · p. 12
  • Gemma 4 26B A4B IT, Kimi K2.7 Code, GPT-OSS-120B, and Claude 4.5 Haiku generally had modest energy increases, usually within 5%.GPT-OSS-120B increased by 5.8% and 6.8% with Prompts 12 and 20; Claude 4.5 Haiku increased by 21% with Prompt 21.Source: 2) Overall Impact of Prompts on All… · p. 12

3) Impact of Prompts on Problem Difficulty:

C++ prompting effects depended on problem difficulty and algorithmic category, with substantial reductions in some settings and increases in others. Across models, Prompt 18 showed larger overall reductions than Prompt 20.

3) Impact of Prompts on Problem Difficulty:

  • For Medium problems, every evaluated prompt reduced energy use, with reductions reaching 25%; Easy problems instead increased energy use by up to 1.04%.For Hard problems, the largest cited reductions were 2.05% with Prompt 4, 1.27% with Prompt 13, and 2.45% with Prompt 21.Source: 3) Impact of Prompts on Problem… · p. 12

4) Impact of Prompts on Algorithmic Category:

  • Figure 11 summarizes the impact of eight prompting strategies on Python and C++ generation across their evaluation datasets and models.Source: Figure 11 · p. 13
  • Bit Manipulation prompts reduced energy use by as much as 47%, though several model-specific prompt combinations increased it.For example, Prompt 11 increased energy use for GPT-OSS-120B by +31%, Claude 4.5 Haiku by +14%, and GPT-5.6 Terra by +6%.Source: 4) Impact of Prompts on Algorithmic… · p. 13
  • Greedy energy use fell most with Prompts 18 and 13 (−25% each), while Prompt 20 increased it by +21%.Binary Search reductions were −3% to −5% for all prompts except Prompt 20, which increased energy use by +3%.Source: 4) Impact of Prompts on Algorithmic… · p. 13
  • Sorting reductions reached −5% with Prompts 4 and 18, while Sliding Window reductions reached −3% with five prompts.Prompt 10 increased Sorting energy use for Kimi K2.7 Code by +15%; Prompt 12 increased Sliding Window energy use by +14%.Source: 4) Impact of Prompts on Algorithmic… · p. 13
  • All evaluated prompts increased energy use for DFS and Two Pointers, with Prompt 11 reaching +53.4% for Two Pointers.The results characterize prompt effects as dependent on algorithmic characteristics as well as model and prompting strategy.Source: 4) Impact of Prompts on Algorithmic… · p. 13

5) Overall Best Performing Prompts for All Models:

  • Prompt 18 reduced energy use by 13% overall across six models, with an individual-model reduction reaching 56%.Prompt 20 reduced energy use by 2% across six models, with its largest individual-model reduction at 5%.Source: 5) Overall Best Performing Prompts for… · p. 13

E. Overall Impact of 8 Energy-Efficient Prompting Strategies For Python and C++ Code Generation

Across the evaluated datasets and models, the selected prompting strategies reduced energy use for both Python and C++ relative to the baseline, though outcomes varied by model and task. The study also notes scope limits and that prompt-token overhead adds inference energy.

E. Overall Impact of 8 Energy-Efficient Prompting Strategies For Python and C++ Code Generation

  • For Python, Prompt 20 achieved the largest overall reduction at −25%; for C++, Prompt 13 led at −17%.Prompt 20 yielded only −1% for C++ code generation.Source: E. Overall Impact of 8… · p. 14

V. DISCUSSION & CONCLUSIONS

  • Model-level reductions reached approximately 50% for Granite-4.0-H-Small, 39% for Claude 4.5 Haiku, and approximately 28% for MiniMax M3 in Python.For C++, Granite-4.0-H-Small reached 57% with Prompt 13, while Qwen3-Coder-480B-A35B-Instruct improved 6.4% with Prompt 12.Source: §V · p. 14
  • Energy-aware prompting varied considerably across models and tasks, and some prompts increased energy use relative to the baseline.Prompt 13 increased GPT-OSS-120B energy use by up to 74% for Python; Prompt 21 increased it by 22% for C++.Source: §V · p. 14
  • Prompting alone did not consistently elicit energy-efficient coding strategies across different tasks and models.The conclusion also notes that models may favor readability, generality, or commonly learned implementation patterns.Source: §V · p. 14
  • The benchmark had an imbalanced difficulty distribution and did not cover all practical programming tasks; the experiments included only Python and C++.Source: §V · p. 14
  • The selected strategies required significantly more tokens, increasing inference energy; overall pipeline efficiency therefore includes both inference and code-execution energy.The paper proposes future work on concise prompts, fine-tuning, and automated prompt optimization.Source: §V · p. 14
Loading 2610.02571v2…