PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating and Explaining Large Language Models for Code Using Syntactic Structures

David N Palacio, Alejandro Velasco, Daniel Rodriguez-Cardenas, Kevin Moran, Denys Poshyvanyk

arXiv:2308.03873v1cs.SEcs.LG

TL;DR

Reliable mapping from code-LLM predictions to understandable programming-language concepts is needed to support both explanation and informative evaluation. ASTxplainer maps token predictions to AST-derived concepts and uses them for structural evaluation, loss-related explanations, and visualizations. The reported evaluation finds an advantage for monolingual models across syntax concepts, while the user study reports preferences among visualization formats.

  • Problem

    Explaining and evaluating code LLMs requires a reliable mapping from model predictions to fine-grained programming-language concepts.

    Source: §I · pp. 1–2

  • Method

    ASTxplainer aggregates token predictions into AST-derived Abstract Syntax Concepts used for structural evaluation, cross-entropy explanations, and prediction visualizations.

    Source: §I · p. 2

  • Results

    Monolingual models outperformed multilingual models at predicting all syntax-concept types, and Abstract Syntax Concepts influenced cross-entropy loss after controlling for code confounders.

    Source: §VIII · p. 10

  • Takeaways & Limitations

    The reported findings suggest the importance of fine-tuning strategies over model size and show that concept-level visualizations have utility for helping users understand predictions.

    Source: §I · p. 2, §VIII · p. 10

Abstract

from arXiv · show

Large Language Models (LLMs) for code are a family of high-parameter, transformer-based neural networks pre-trained on massive datasets of both natural and programming languages. These models are rapidly being employed in commercial AI-based developer tools, such as GitHub CoPilot. However, measuring and explaining their effectiveness on programming tasks is a challenging proposition, given their size and complexity. The methods for evaluating and explaining LLMs for code are inextricably linked. That is, in order to explain a model's predictions, they must be reliably mapped to fine-grained, understandable concepts. Once this mapping is achieved, new methods for detailed model evaluations are possible. However, most current explainability techniques and evaluation benchmarks focus on model robustness or individual task performance, as opposed to interpreting model predictions. To this end, this paper introduces ASTxplainer, an explainability method specific to LLMs for code that enables both new methods for LLM evaluation and visualizations of LLM predictions that aid end-users in understanding model predictions. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes, by extracting and aggregating normalized model logits within AST structures. To demonstrate the practical benefit of ASTxplainer, we illustrate the insights that our framework can provide by performing an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects. Additionally, we perform a user study examining the usefulness of an ASTxplainer-derived visualization of model predictions aimed at enabling model users to explain predictions. The results of these studies illustrate the potential for ASTxplainer to provide insights into LLM effectiveness, and aid end-users in understanding predictions.

I. INTRODUCTION

Existing code benchmarks and metrics assess task or code correctness but do not explain model predictions, while reliable mapping from predictions to programming-language concepts is needed for informative evaluation. ASTxplainer maps predictions to Abstract Syntax Concepts, supporting structural evaluation, explanations, and visualizations for users.

I. INTRODUCTION

  • Existing benchmarks and metrics often assess functional or syntactic correctness and task performance but do not explain model predictions or capabilities interpretably.The paper identifies reliable mapping from predictions to human-understandable programming-language concepts as a central challenge for both evaluation and explainability.Source: §I · pp. 1–2
  • ASTxplainer aggregates LLM token predictions into AST-derived Abstract Syntax Concepts, mapping predictions to programming-language concepts for fine-grained analysis.The framework is intended to help end-users reason about why a model made particular predictions.Source: §I · p. 2
  • AsC-Eval estimates structural performance, AsC-Causal links it to cross-entropy loss, and AsC-Viz visualizes prediction uncertainty within AST structures.These components respectively support structural evaluation, explanations of how code structure relates to loss, and practical interpretation of prediction reliability.Source: §I · p. 2
  • The empirical study evaluates AsC-Eval and AsC-Causal on 12 popular LLMs using about 10 million tokens excluded from model training, and introduces Galeras, a curated dataset of 50K Python samples.The paper also reports that the experimental data, datasets, source code, and statistical analyses are available in an open-source repository.Source: §I · p. 2
  • A user study examines whether AsC-Viz visualizations help developers understand and explain LLM predictions.The study assesses the utility of multiple visualizations for practitioners.Source: §I · p. 2

II. BACKGROUND & RELATED WORK

The background frames ASTxplainer as an approach to quantifying prediction uncertainty in code LLMs. It describes sequence-based language modeling and focuses on decoder-only models, which predict tokens from preceding context.

II. BACKGROUND & RELATED WORK

  • The paper frames ASTxplainer as an approach to quantifying prediction uncertainty in code LLMs.Its background covers LLM evaluation, ASTs used for probing, and explainability methods.Source: §II · p. 2

A. Large Language Models for Code

  • The paper represents software artifacts as a sequence of tokens and describes language models as estimating each token from preceding tokens.It uses a tokenization function to convert software-corpus data into discrete sequence elements and factorizes the sequence probability over next-token predictions.Source: A. Large Language Models for Code · p. 2
  • Transformer language models may be encoder-only, encoder-decoder, or decoder-only; this paper focuses on decoder-only models because of their popularity for code generation.For next-token predictions, the paper extracts and normalizes last-layer log-probabilities using softmax.Source: A. Large Language Models for Code · pp. 2–3

B. ASTs Probing Approaches

The section describes probing as a way to assess whether models encode programming-language syntax, then situates AST-based probing within formal syntax representations.

B. ASTs Probing Approaches

  • Probing is a supervised analysis of which model parameters influence learning, used to assess whether hidden representations encode properties such as programming-language syntax.Examples of parameters include input snippets, tokenization, hidden layers, and model size.Source: B. ASTs Probing Approaches · p. 3
  • Lopez et al. [40] used a linear classifier to show that pretrained models’ middle layers contain information about AST syntax.Source: B. ASTs Probing Approaches · p. 3
  • ASTs represent programming-language syntax under context-free grammar production rules, with terminal symbols and non-terminal nodes forming statements.CFG production rules combine terminal and non-terminal nodes; CFGs specify the language’s syntax and structural information.Source: B. ASTs Probing Approaches · p. 3

C. Explainability for Code Generation

Prior code explainability work includes token-level and uncertainty-based approaches, as well as structural analyses of encoder-only models. ASTxplainer instead maps token-prediction performance to programming-language syntax concepts.

C. Explainability for Code Generation

  • Prior code-generation methods identify influential tokens or highlight uncertain sequence fragments, while ASTxplainer targets syntax-structured interpretation.Liu et al. [15] identify tokens contributing to sequence predictions; Vasconcelos et al. [45] highlight fragments above an uncertainty threshold.Source: C. Explainability for Code Generation · pp. 3–4
  • Earlier structural analyses examined syntax alignment with attention, syntax information in hidden layers, and how pretrained models induce syntax, primarily in encoder-only models.Wan et al. [14] studied CodeBERT and GraphCodeBERT along these three dimensions.Source: C. Explainability for Code Generation · p. 4
  • Earlier structural explainability techniques were designed for encoder-only models below 1B parameters, whereas ASTxplainer targets decoder-only LLMs.ASTxplainer predicts AST terminal and non-terminal nodes and contextualizes cross-entropy loss using causal inference.Source: C. Explainability for Code Generation · p. 4
  • ASTxplainer formalizes φ as a mapping from token-prediction performance to human-understandable programming-language syntactic concepts.The model-space approximation uses normalized log-probabilities at different granularities; the mapped concepts are terminal and non-terminal nodes.Source: C. Explainability for Code Generation · p. 4

III. THE ASC-Eval COMPONENT

AsC-Eval maps next-token prediction performance onto syntax concepts, providing a way to examine which structural code elements LLMs predict effectively or erroneously. It uses AST-node categories, token-to-node alignment, and aggregation to make predictions interpretable.

A. Abstract Syntax Concepts (AsC)

  • AsC-Eval maps normalized next-token predictions to CFG terminal and non-terminal concepts, enabling evaluation of LLM prediction performance by syntactic element.It uses the last layer to calculate token-level predictions and organizes concepts into categories H.Source: A. Abstract Syntax Concepts (AsC) · p. 4, §III · p. 4
  • AsC groups AST symbols into semantic categories, allowing model performance to be associated with structural code attributes.Terminal and non-terminal nodes can carry different meanings; for example, identifier and string nodes belong to a Natural Language concept category.Source: A. Abstract Syntax Concepts (AsC) · p. 4
  • ASTxplainer parses snippets with tree-sitter, using leaves for terminal tokens and intermediate nodes for nonterminals.The parse provides node types, children, and locations for alignment.Source: A. Abstract Syntax Concepts (AsC) · p. 4

B. AST Alignment function (δ)

  • The alignment function δ assigns each AST node the snippet-token subsequence within that node’s offset range, including multi-token spans for intermediate nodes.A tokenizer may split a terminal node across tokens, while a parent node can align with a longer token subsequence.Source: B. AST Alignment function (δ) · p. 5, Figure 2 · p. 5

C. AST Aggregation function (θ)

The aggregation function turns aligned token predictions into AsC-Eval values at snippet or corpus scale. AsC-Causal then treats those values as treatments and estimates their relationship to cross-entropy loss while controlling for code-feature confounders.

C. AST Aggregation function (θ)

  • AsC-Eval aggregates aligned token predictions into confidence values for AST nodes, supporting local snippet-level and global corpus-level analysis.Local aggregation can also be averaged across snippets.Source: C. AST Aggregation function (θ) · p. 5

IV. THE ASC-Causal COMPONENT

  • AsC-Causal estimates how AsC predictions affect canonical LLM performance, using cross-entropy loss as the outcome.The approach is framed as a causal inference technique for explaining and contextualizing canonical metrics.Source: §IV · p. 5
  • The causal model treats each AsC as a treatment, code features as confounders, and canonical performance as the outcome; its target is the average treatment effect after confounder control.The text distinguishes p(Y|do(T)) from the unadjusted association p(Y|T).Source: §IV · p. 6, Figure 4 · p. 6
  • The study estimates treatment effects by controlling for sequence size, AST-node count, tree depth, and cyclomatic complexity.The effects represent the treatment-outcome slope after controlling for confounders.Source: §IV · p. 6

V. THE ASC-Viz COMPONENT

AsC-Viz places AsC-Eval values from a local code completion inside an AST, using node colors to distinguish stronger and weaker predictions.

V. THE ASC-Viz COMPONENT

  • The visualization maps AsC-Eval values for terminal and non-terminal AST nodes in a single local evaluation of generated code.The example shows an incomplete prompt, generated tokens, and the completed snippet represented as an AST.Source: §V · p. 6
  • Blue shades mark the highest aggregated values and red shades the smallest, while the AST nodes encapsulate generated code tokens.The linearly organized tokens are called a sequence representation.Source: §V · pp. 6–7

VI. EXPERIMENTAL DESIGN

The study evaluates AsC-Eval, AsC-Causal, and AsC-Viz through an empirical analysis of 12 code LLMs and a practitioner user study. Its setup uses a recent Python-snippet dataset and models grouped by fine-tuning strategy.

VI. EXPERIMENTAL DESIGN

  • The study asks how well LLMs predict syntactic structures, how Abstract Syntax Concepts relate to canonical prediction performance, and how useful AST-based evaluation is to developers.The three questions examine AsC-Eval, AsC-Causal, and AsC-Viz, respectively.Source: §VI · p. 7

A. Study Setup

  • Galeras contains 50,000 unique Python snippets from GitHub commits dated January 1, 2022, to January 1, 2023, selected to reduce training-data contamination.The authors avoided popular code datasets because model training may already have included their samples.Source: A. Study Setup · p. 7
  • The evaluation covers 12 popular open decoder-only LLMs, with the largest model at 2.7B parameters and four groups based on fine-tuning strategy.The groups include models trained mostly on natural language, codegen-architecture models, multilingual code models, and BigPython-fine-tuned or GPT-2 models.Source: A. Study Setup · p. 7
  • Experiments ran on Ubuntu 20.04 with an AMD EPYC 7532 CPU, an A100 GPU with 40GB VRAM, and 1TB RAM; inference used Hugging Face and PyTorch.Models were loaded onto the GPU to accelerate inference.Source: A. Study Setup · p. 7

B. RQ1 The AsC-Eval Empirical Methodology

The empirical methods derive AsC-Eval values from token-level predictions, estimate concept-level relationships with cross-entropy loss, and assess practitioners’ perceptions of AsC-Eval and AsC-Viz visualizations.

B. RQ1 The AsC-Eval Empirical Methodology

  • For RQ1, normalized token log-probabilities are matched to expected tokens, assembled into next-token prediction sequences, and aggregated into AsC-Eval values.The authors also compute snippet cross-entropy loss and bootstrap model medians using 500 samples for comparisons.Source: B. RQ1 The AsC-Eval Empirical… · p. 7

C. RQ2 The AsC-Causal Empirical Methodology

  • For RQ2, the study correlates 14 syntactic concepts with cross-entropy loss and estimates treatment effects for selected concepts while controlling for four confounders.The analysis focuses on best- and worst-performing models because correlations were similar across LLMs.Source: C. RQ2 The AsC-Causal Empirical… · p. 7

D. RQ3 Qualitative User-Study Methodology

  • Participants assess visualizations of predicted code tokens using AsC-Eval and next-token prediction values, with the model identity withheld to focus ratings on visualization usability.The example local evaluation is shown in Fig. 5.Source: D. RQ3 Qualitative User-Study… · p. 8, Figure 5 · p. 8
  • For RQ3, four surveys use a within-subjects design to rate AsC-Viz explanations across four Python samples and visual treatments.The treatments include AST-complete views with terminal and non-terminal nodes, AST-partial views with non-terminal nodes, and sequence-based token values.Source: D. RQ3 Qualitative User-Study… · p. 8

VII. RESULTS & DISCUSSION

AsC-Eval performance varies with model size and fine-tuning strategy: mono-lang [2B] has the highest reported global average, while performance differs substantially across syntax concepts.

A. RQ1 Empirical AsC Performance Evaluation

  • Scope reached 0.94 for Mono-Language-Type models, Iterations 0.82 for codegen-nl [2B], and Testing 0.85 for mono-lang [2B]; Natural Language and Data Types were below 0.5.The reported largest average medians for the latter categories were 0.46 for Natural Language and 0.47 for Data Types with NL GPT-3.Source: A. RQ1 Empirical AsC Performance… · p. 9
  • The authors suggest that context demands may contribute to low Natural Language and Data Types performance; mono-lang [2B] was the only evaluated architecture generally accurate on Data Types.They note that strings need a larger context window and that Python’s dynamic typing makes Data Types less frequent.Source: A. RQ1 Empirical AsC Performance… · p. 9
  • Iteration performance exceeded the 0.6 threshold for nearly all models, while gpt-3 [125M] had an average median AsC-Eval of 0.74.The exception was codeparrot-small-multi, with an average median of 0.6.Source: A. RQ1 Empirical AsC Performance… · p. 9
  • Models trained on natural language showed greater median variability than code-fine-tuned models; mono-lang [2B] had a 0.84 global average median AsC-Eval, versus 0.48 for gpt-3 [125M].The reported variability ranges were 0.2–0.9 for NL GPT-3 and NL Codegen, and 0.7–1.0 for mono-lang [2B].Source: A. RQ1 Empirical AsC Performance… · p. 9
  • 0.84 global average AsC performance was achieved by mono-lang [2B], the largest evaluated model, with the lowest variability.Fig. 6 groups AsC-Eval performance by LLM and density by model type.Source: A. RQ1 Empirical AsC Performance… · p. 9, Figure 6 · p. 9

B. RQ2 Empirical Causal Evaluation

AsC-Eval patterns were associated with cross-entropy loss across models, and AsC-Causal estimated effects after accounting for code features. Natural-language concepts, especially identifiers, showed notable effects, while the if-statement concept correlated with cyclomatic complexity.

B. RQ2 Empirical Causal Evaluation

  • All AsC categories influenced cross-entropy loss, with global-average effects of 1.78 for mono-lang [2B] and −1.60 for gpt-3 [125M].The paper reports these as the best and worst models, respectively.Source: B. RQ2 Empirical Causal Evaluation · p. 9
  • Natural Language had the largest reported impact: identifier effects were −1.78 for gpt-3 [125M] and −2.89 for mono-lang [2B].Functional Programming had the lowest impact; lambda had a positive effect of 0.2 for gpt-3 [125M].Source: B. RQ2 Empirical Causal Evaluation · p. 9
  • The if-statement concept had a Pearson correlation of ρ = 0.58 with cyclomatic complexity for both the best and worst models.The paper relates this observation to the metric’s consideration of control flows induced by conditional structures.Source: B. RQ2 Empirical Causal Evaluation · p. 10
  • Identifiers affected cross-entropy loss with average treatment effects of −1.78 for gpt-3 [2.7B] and −2.89 for mono-lang [2B].Source: B. RQ2 Empirical Causal Evaluation · p. 10

C. RQ3 User-Study on AsC Visualization Results

In a 40-response user study, participants favored the partial AST visualization for readability, while views differed in perceived usefulness and complexity. The paper concludes that AsC-based visualizations showed utility and that syntax-concept effects on cross-entropy persisted across models and settings.

C. RQ3 User-Study on AsC Visualization Results

  • The practical evaluation collected 40 responses, with participants randomly assigned to a control or survey group.The reported sampling confidence was 90%, with a 15% margin of error.Source: C. RQ3 User-Study on AsC Visualization… · p. 10
  • Participants found error-node values and color coding useful for understanding long-range dependencies and why the model generated an erroneous prediction.Participants also wanted workflow information; 57% found sequence visualization highly useful, although 29% considered it easy to read and use.Source: C. RQ3 User-Study on AsC Visualization… · p. 10
  • The partial AST visualization was the most readable representation, with 57% agreement, and was preferred over complete AST and sequence views.It was also judged less complex by 43% relative to control, while 67% judged the complete AST more complex; usefulness ratings varied across visualization types.Source: C. RQ3 User-Study on AsC Visualization… · p. 10

VIII. CONCLUSION & FUTURE WORK

  • The conclusion reports that mono-language models outperformed multi-language models across syntax concepts, and AsC effects on cross-entropy persisted across model sizes and fine-tuning strategies.The paper presents the user study as illustrating the utility of visualizations built on its Abstract Syntax Concepts.Source: §VIII · p. 10

Related papers

Loading 2308.03873v1…