PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach

Zhenlan Ji, Pingchuan Ma, Zongjie Li, Shuai Wang

arXiv:2310.06680v1cs.SEcs.AI

TL;DR

LLM code quality can fluctuate with prompt wording, while current evaluation and explanation approaches do not adequately capture prompt–code interplay. The paper applies causal analysis to prompt and code features and reports findings across several models and prompt adjustments. It presents the approach as a source of actionable prompt-calibration insights, while its evaluation is bounded by the studied models, dataset, language, and rephrasing instructions.

  • Problem

    LLM code-generation quality can fluctuate with prompt wording, and existing benchmarks and explanation methods do not capture prompt–code interplay or commonly accessible API use.

    Source: §1 · pp. 1–2

  • Method

    The paper uses domain-specific causal analysis to examine how prompts affect generated code and to identify prompts that facilitate higher-quality code.

    Source: §1 · p. 2, §10 · p. 15

  • Results

    Across three representative LLMs, the authors report that prompt concepts affect code patterns, some keywords create trade-offs, and selected prompts offer superior quality to the original.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The framework is presented as providing actionable insights for understanding generated code and improving its quality through prompt calibration.

    Source: §10 · p. 15

    The study focuses on GPT-family models, the APPS dataset, Python3 generation, and a limited set of rephrasing instructions.

    Source: §8 · p. 15

Abstract

from arXiv · show

While code generation has been widely used in various software development scenarios, the quality of the generated code is not guaranteed. This has been a particular concern in the era of large language models (LLMs)- based code generation, where LLMs, deemed a complex and powerful black-box model, is instructed by a high-level natural language specification, namely a prompt, to generate code. Nevertheless, effectively evaluating and explaining the code generation capability of LLMs is inherently challenging, given the complexity of LLMs and the lack of transparency. Inspired by the recent progress in causality analysis and its application in software engineering, this paper launches a causality analysis-based approach to systematically analyze the causal relations between the LLM input prompts and the generated code. To handle various technical challenges in this study, we first propose a novel causal graph-based representation of the prompt and the generated code, which is established over the fine-grained, human-understandable concepts in the input prompts. The formed causal graph is then used to identify the causal relations between the prompt and the derived code. We illustrate the insights that our framework can provide by studying over 3 popular LLMs with over 12 prompt adjustment strategies. The results of these studies illustrate the potential of our technique to provide insights into LLM effectiveness, and aid end-users in understanding predictions. Additionally, we demonstrate that our approach provides actionable insights to improve the quality of the LLM-generated code by properly calibrating the prompt.

1 Introduction

The paper addresses the difficulty of evaluating and explaining LLM code generation when prompt changes can affect output quality and existing approaches miss prompt–code interplay or require model internals. It proposes causal analysis of prompts and generated code, reports findings across three LLMs, and offers prompt-calibration guidance.

1 Introduction

  • Prompt quality can vary with natural-language phrasing, while existing benchmarks miss prompt–code interplay and many explanation methods require inaccessible model internals.The paper notes that code-generation behavior can be unstable or contain security vulnerabilities.Source: §1 · pp. 1–2
  • The approach represents prompts and generated code with linguistic and code features, then uses causal analysis and rephrasing-based treatment effects to identify potential drivers of code quality.The authors use DiBS to identify causal relations and estimate average treatment effects for rephrasing instructions.Source: §1 · p. 2
  • Across GPT-Neo, GPT-3.5-Turbo, and GPT-4, prompt concepts affect code patterns, some keywords create trade-offs, and selected prompts can outperform the original.The authors also report unreasonable model reactions and describe the results as actionable guidance for prompt calibration.Source: §1 · p. 2
  • The paper presents domain-specific causal-analysis designs, an approach for finding optimal prompts, and evaluations intended to yield actionable suggestions for understanding and enhancing generated code.Code, data, and supplementary materials are provided for reproducibility.Source: §1 · p. 2

2 Preliminary and Motivation

This section introduces prompts as a means of steering LLMs, places the study in code generation, and presents causal analysis as a way to examine relationships among variables. It distinguishes causal relations from correlation and states an assumption required for interpreting graph-based conclusions.

2.1 LLM and Prompt Engineering

  • Prompts provide task-specific knowledge and can steer LLMs, motivating prompt engineering as a way to elicit desired outputs.The section contrasts prompt-based steering with conventional pre-train-and-fine-tune practice and cites few-shot, chain-of-thought, tree-of-thought, and debate methods.Source: §2.1 · p. 3

2.2 Code Generation

  • The paper studies natural-language-to-code generation with Python3, while noting that its approach generalizes to other settings.Code generation is described as a longstanding software-engineering task, with earlier search-based approaches limited by low generalizability.Source: §2.2 · p. 3
  • Prompt instability and complex natural-language features make it difficult to determine which prompt characteristics affect generated-code quality.Compilers and operating systems have low tolerance for bugs and performance issues, further intensifying the impact of unstable prompts.Source: §2.2 · p. 3

2.3 Causality Analysis

  • Causal analysis represents inferred relations as a directed acyclic graph, whose nodes are variables and whose directed edges represent causal relations.Nodes are endogenous when their values are determined by parent nodes, and exogenous when they derive from external factors.Source: §2.3 · p. 3
  • Correlation can arise because a confounder affects both variables, so it does not establish that one causes the other.The section distinguishes graph-based qualitative analysis of causal paths from quantitative inference about counterfactual changes.Source: §2.3 · pp. 3–4
  • Graph-based conclusions require consistency with the real world; the global Markov assumption links graph separation to conditional independence in the distribution.The assumption is stated as graph separation of X and Y given Z implying conditional independence in P.Source: §2.3 · p. 4

3 Research Motivation and Pilot Study

The section motivates systematic analysis of how prompts affect code generation, rephrasing prompts to explore prompt changes, and causal methods to distinguish effects from misleading correlations. It also emphasizes that rephrasing can change code metrics without necessarily improving code.

3.1 Effect of Prompt in Code Generation

  • Subtle prompt changes can produce confusing, non-monotonic results, leaving prompt quality hard to assess and systematic design guidance lacking.The motivating example contrasts semantically equivalent prompts: one leads to code that capitalizes each word’s initial letter, while a more explicit version yields the intended first-letter program.Source: §3.1 · pp. 4–5

3.2 Prompt Adjustment via LLM-Based Rephrasing

  • LLM-based rephrasing can change prompt style while preserving semantics, and a pilot study found different instructions produced different code-metric trends.The pilot compared shorter, longer, and more formal rephrasings with a no-instruction control; changed metrics do not necessarily mean improved code.Source: §3.2 · p. 5, §3.3 · p. 5, Table 1 · p. 5

3.3 Analysis for Complex Relationships in Code Generation

  • Correlation can mislead prompt-effect analysis when question properties affect both prompt quality and code quality, especially with hundreds of variables.A regression may predict output accurately yet misrepresent the prompt’s effect; code generation’s stochastic outputs also motivate assessing quality as a distribution rather than from only a few executions.Source: §3.3 · pp. 5–6

4 Design

The design maps original and rephrased programming prompts to linguistic features and generated-code metrics, then analyzes their causal graph. It uses that graph to explain prompt effects and select prompt settings expected to generate high-quality code.

4 Design

  • Data collection extracts prompt features and generated-code performance metrics; causal-graph analysis then identifies principles for prompt design and seeks high-quality settings.The workflow covers original and rephrased questions and includes code correctness and readability among its metrics.Source: §4 · p. 6

4.1 Prompt Quantification

  • The framework quantifies prompts with 255 lexical, syntactic, and semantic linguistic features, following Lee et al. [37] [38].The authors presume these features sufficiently capture interactions between prompts and other variables, including rephrasing intentions and generated code.Source: §4.1 · p. 7

4.2 Rephrase Generation

  • Rephrasing with an LLM broadens prompt variation, while the meta-prompt combines user-provided programming questions with selected rephrasing intentions.The study designed 22 intentions but selected six instructions, three roles, and three scenarios for its experiments due to the cost of exploring all combinations.Source: §4.2 · p. 7

4.3 Causal Analysis

  • DiBS learns two causal subgraphs separately along the proposed meta-prompt → linguistic-feature → code-metric sequence, then combines them into a complete graph.The variables include binary indicators for selected rephrasing intentions, linguistic features, and code metrics.Source: §4.3 · pp. 7–8
  • The average treatment effect compares expected outcomes under two interventions on a treatment variable; the study notes these causal estimands cannot be estimated directly from observational data.The paper defines ATE as the difference between the expected outcomes under do(X = x1) and do(X = x2).Source: §4.3 · pp. 8–9
  • Double machine learning estimates causal effects by modeling treatment and outcome from confounders, then relating the models’ residuals.The paper uses the EconML implementation to estimate causal estimands.Source: §4.3 · p. 9

4.4 Establishing Prompt Effect on Code Generation

  • The prompt-effect algorithm ranks code metrics by their estimated treatment effects, then tests ancestor linguistic features as potential mediators to explain the strongest effects.It returns the three most affected code metrics with corresponding explanations; the meta-prompt variable flags whether a rephrasing intention was selected.Source: §4.4 · pp. 9–10

5 Experiment Setup

Experiments use 6K APPS Python problems with GPT-Neo, GPT-3.5-Turbo, and GPT-4. Generated code is evaluated across five metric categories covering different aspects of quality.

5 Experiment Setup

  • The implementation uses about 2.4K lines of Python 3 and runs on a server with an AMD Ryzen 3970X CPU and one NVIDIA RTX 3090 GPU.Source: §5 · p. 10

5.1 Datasets & Model

  • The experiments sample 6K Python problems from APPS and use GPT-Neo, GPT-3.5-Turbo, and GPT-4 for code generation.GPT-3.5-Turbo rephrases prompts; each model generates code for the 6K data points, with three solutions per point and a maximum length of 2,000.Source: §5.1 · p. 10

5.2 Code Metrics

  • Code quality is assessed through five categories—correctness, diversity, overhead, readability, and security—intended to capture orthogonal perspectives.The study uses existing metric implementations and reports the metrics in Table 2.Source: §5.2 · p. 10, Table 2 · p. 10

6 Evaluation

The evaluation finds that learned causal graphs generally predict code metrics well, while prompt effects vary across models and prompt groups. The analysis also identifies recurring prompt patterns and model-specific differences.

6.1 RQ1: Verification of the Causal Graphs

  • Most learned graphs achieve R^2 above 0.8 and MSE below 0.2, but GPT-3.5-Turbo and GPT-4 perform poorly on semgrep count.GPT-Neo instead scores R^2=1.0 and MSE=0.0 on semgrep count; the authors attribute the exception to low variance and omit the metric from subsequent analysis.Source: §6.1 · p. 11, Table 4 · p. 11
  • GPT-Neo’s learned graph behaves differently from the GPT-series graphs, with stronger prediction on similarity metrics that the authors associate with overfitting on APPS.They interpret its output as more likely to combine memorized snippets, making it less sensitive to meta-prompt changes; Figure 6 illustrates this behavior.Source: §6.1 · p. 12, Figure 6 · p. 12

6.2 RQ2: Analysis based on the Causal Graphs

  • In GPT-series graphs, Long and Formal tend to reduce similarity to ground truth, whereas Short and Fluent tend to help.The authors suggest that Long adds unnecessary or misleading information and Formal complicates programming logic, while Short extracts essentials and Fluent simplifies it.Source: §6.2 · p. 12, Figure 7 · p. 13
  • The code metric most affected by a meta-prompt variable differs across learned graphs, as illustrated by Short’s distinct effects in GPT-Neo, GPT-3.5-Turbo, and GPT-4.For Short, the most affected metric is syn err in GPT-Neo, gold sim B in GPT-3.5-Turbo, and pass rate in GPT-4.Source: §6.2 · p. 12, Figure 7 · p. 13
  • root det var is a frequent primary driver for GPT-Neo, while simp ttr and named-entity features occur most often for GPT-series models.The authors describe simp ttr as lexical variation and named-entity features as reflecting information complexity; they consider these drivers intuitive for the GPT-series models.Source: §6.2 · pp. 12–13
  • Role and scenario prompts tend to have more positive effects for GPT-series models but perform poorly for GPT-Neo.The paper recommends role-playing and scenario setting for models with sufficient capacity, while simple concise rephrasing can be more effective for tiny models.Source: §6.2 · pp. 13–14, Figure 8 · p. 13

7 Downstream Application

The paper uses a genetic algorithm to search for rephrasing instructions, scoring candidates with a causal-graph surrogate rather than repeatedly generating code. On APPS, the approach outperforms a single rephrasing instruction and the original prompt in most cases, with GPT-Neo as the exception.

7 Downstream Application

  • The causal graph supplies a ground-truth-free surrogate fitness score that estimates a candidate prompt’s objective metric without generating its code.The authors describe this surrogate as more efficient than direct metric evaluation and expected to be more stable than actual code generation.Source: §7 · p. 14
  • Candidate rephrasing instructions are encoded as binary strings and evolved using fitness-based selection, crossover, and mutation.The top-N candidates survive each generation, and two-point crossover generates offspring from parent vectors.Source: §7 · p. 14
  • On APPS, the approach outperforms the best single rephrasing instruction and the original prompt in most cases, with GPT-Neo the exception.The evaluation uses BLEU as the objective metric; the authors suggest GPT-Neo’s overfitting may explain the exception.Source: §7 · p. 14, Table 5 · p. 15

8 Discussion

The approach is described as language-agnostic and extensible to other languages and rephrasing instructions, subject to instructions being reasonably difficult for LLMs. The study’s findings are scoped to GPT-family models, APPS, Python3, and a limited set of rephrasing instructions.

8 Discussion

  • The approach can extend to other languages and rephrasing instructions, provided each instruction is reasonably difficult for LLMs to understand and implement.Examples include adapting function syntax for JavaScript or C and adding “make it more object-oriented” as a rephrasing instruction.Source: §8 · p. 15
  • The study focuses on GPT-family models, APPS, Python3 code generation, and a limited number of rephrasing instructions.The authors note that other models such as T5 and other datasets such as HumanEval were not investigated.Source: §8 · p. 15
  • The authors consider their findings representative of common usage and generalizable to other languages and rephrasing instructions, citing the approach’s extensibility.They qualify rephrasing extensibility by requiring instructions to be reasonably difficult for LLMs to understand and implement.Source: §8 · p. 15

9 Related Work

The related work distinguishes theoretical methods for learning causal relations from applications of causality analysis in software engineering. This paper focuses on applying causality analysis to evaluate and explain LLM-based code generation and the role of prompts.

9 Related Work

  • Causal-graph learning includes constraint-based methods such as PC, score-based methods such as BDe(u), and gradient-based methods such as NOTEARS [84] and DAG-GNN [81].The passage describes these theoretical advances as orthogonal to the paper’s focus on applying causality analysis.Source: §9 · p. 15
  • This paper applies causality analysis to evaluating and explaining LLM-based code generation, with a focus on the role of distinct prompts.Source: §9 · p. 15
  • Software engineering applications of causality analysis include debugging [22] [20], root cause analysis, data race detection, and DNN testing and repair [65] [82].Prior work also applied causality analysis to programming-language effects in code competitions [24] and factors affecting software productivity.Source: §9 · p. 15

10 Conclusion

The paper proposes a causality analysis-based approach to systematically analyze relations between LLM prompts and generated code. It aims to support evaluation and explanation and provide actionable insights for improving generated-code quality through prompt calibration.

10 Conclusion

  • The proposed approach systematically analyzes relations between LLM input prompts and generated code.Source: §10 · p. 15
  • The approach facilitates evaluation and explanation of LLM-based code generation.Source: §10 · p. 15
  • Properly calibrating prompts can provide actionable insights for improving the quality of LLM-generated code.The authors describe the work as a roadmap for users seeking to comprehend and improve generated-code quality.Source: §10 · p. 15

Related papers

Loading 2310.06680v1…