PaperlayerView exact revision on arXiv

Source-linked AI summary

Robustness, Security, Privacy, Explainability, Efficiency, and Usability of Large Language Models for Code

Zhou Yang, Zhensu Sun, Terry Zhuo Yue, Premkumar Devanbu, David Lo

arXiv:2403.07506v1cs.SE

TL;DR

LLM4Code research has examined non-functional properties separately, but lacked a systematic review of how they are evaluated and enhanced. This paper reviews 146 studies, identifies seven properties beyond accuracy, and synthesizes findings, practices, and research opportunities.

  • Problem

    Studies separately examine LLM4Code non-functional properties, but no systematic review had synthesized how these properties are evaluated and enhanced.

    Source: §1 · p. 2

  • Method

    The paper conducts a systematic literature review of 146 relevant studies on LLM4Code properties beyond accuracy.

    Source: §1 · p. 2

  • Results

    The review identifies seven properties beyond accuracy and summarizes their definitions, evaluation approaches, enhancement practices, and key findings.

    Source: §1 · p. 2, §7 · p. 24

  • Takeaways & Limitations

    The review frames future work through data-centric, human-centric, and system-centric perspectives on LLM4Code development.

    Source: §7 · p. 24

    The review’s findings may not fully match current LLM4Code systems because architectures and training data continue to evolve.

    Source: §6 · p. 24

Abstract

from arXiv · show

Large language models for code (LLM4Code), which demonstrate strong performance (e.g., high accuracy) in processing source code, have significantly transformed software engineering. Many studies separately investigate the non-functional properties of LM4Code, but there is no systematic review of how these properties are evaluated and enhanced. This paper fills this gap by thoroughly examining 146 relevant studies, thereby presenting the first systematic literature review to identify seven important properties beyond accuracy, including robustness, security, privacy, explainability, efficiency, and usability. We discuss the current state-of-the-art methods and trends, identify gaps in existing research, and present promising directions for future study.

1 INTRODUCTION

LLM4Code can achieve high accuracy while still failing important non-functional requirements. This review examines those properties, how studies evaluate and enhance them, and opportunities for future research.

1 INTRODUCTION

  • LLM4Code has moved from experimental prototypes into developers’ workflows, supporting tasks such as code completion, summarization, and question answering [66] [146].Examples include GitHub Copilot, Amazon CodeWhisperer, and CodeGeeX.Source: §1 · p. 1
  • Despite high accuracy, LLM4Code can be perturbation-sensitive [237], leak sensitive information [240], generate insecure code [153], and lack explainability.The introduction notes that roughly 40% of programs generated by GitHub Copilot were found to contain vulnerabilities [153].Source: §1 · p. 2
  • The review examines 146 studies, identifies seven properties beyond accuracy, and surveys methods for evaluating and enhancing them.It also discusses research challenges and opportunities from three perspectives on incorporating non-functional properties into LLM4Code development.Source: §1 · pp. 2–3
  • The authors caution that findings from referenced publications may not fully reflect current LLM4Code systems because architectures and training data continue to evolve.The caveat particularly concerns commercial systems.Source: §1 · p. 2
  • The paper moves from review methodology and property definitions to evaluation and enhancement practices, research opportunities, validity threats, and conclusions.Source: §1 · p. 3

2 SURVEY METHODOLOGY

The review defines its LLM4Code scope, searches DBLP using property and code-related terms, and expands results through iterative snowballing. It reports the collected literature’s distribution and distinguishes its focus from earlier reviews.

2.1.1 Survey Scope.

  • The survey covers LLMs used for source code and related software artifacts, excludes binary, assembly, and domain-specific language work, and requires a property beyond accuracy.Its LLM4Code scope ranges from early pretrained code models such as CodeBERT to GPT-family models and commercial services; it also includes earlier non-LLM models when relevant.Source: §2.1.1 · p. 3

2.1.2 Search Databases.

  • DBLP was selected to capture relevant work across domains, including preprints, and provide timely access to contributions.The database choice reflects the prevalence of preprints and the reported breadth of DBLP’s returned corpus.Source: §2.1.2 · pp. 3–4

2.1.3 Search Items.

  • The search combines code, model, and property terms; authors used truncated property keywords and independently screened titles and abstracts, resolving disagreements through discussion.The process identified 146 relevant papers, with the keyword search conducted on Dec. 2, 2023.Source: §2.1.3 · p. 4

2.1.4 Snowballing Search.

  • Backward and forward snowballing followed references and citations over eight rounds, adding 23 papers before reaching a fixed point of 146 in-scope papers.The search used the Semantic Scholar API and repeated the relevance-labeling process each round.Source: §2.1.4 · pp. 4–5

2.2 Collection Results

  • The search process identified 146 papers on non-functional properties of LLM4Code, published from 2019 to 2024; robustness led with 27 papers, followed by efficiency (21), usability (20), privacy (16), and security (16).Figure 1(a) shows the accumulated paper count over the recent six years; Figure 1(b) shows the distribution across properties.Source: §2.2 · p. 5, Figure 1 · p. 5
  • LLM4Code and non-LLM4Code papers had similar counts for robustness and explainability, while security, privacy, usability, and efficiency represented new research demands for LLM4Code.Security had 16 LLM4Code papers versus 2 on non-LLM4Code; Figure 2(a) compares distributions by property and model type.Source: §2.2 · p. 5, Figure 2 · p. 6
  • 32.6% of collected papers were arXiv preprints, and around 10% appeared in emerging venues, defined as venues absent from CORE rankings or ranked below C.Figure 2(b) presents publication-venue distribution; its “Other” category includes venues with fewer than two relevant papers.Source: §2.2 · p. 5, Figure 2 · p. 6

2.3 Relevant Literature Reviews

  • Earlier reviews mainly address LLM4Code models and applications, whereas this review specifically examines how properties beyond accuracy are studied and identifies challenges and future work.Source: §2.3 · p. 6

3 NON-FUNCTIONAL PROPERTIES BEYOND ACCURACY

The section defines non-functional properties of LLM4Code beyond accuracy, explaining how they concern reliable, secure, private, understandable, efficient, and usable code models.

3.1 Robustness

  • Robustness is consistent, correct performance under slight input changes; instability can produce unequal output quality across users or let vulnerable code pass CI/CD checks.The examples include changes in prompt wording and code perturbations that alter vulnerability predictions.Source: §3.1 · pp. 6–7

3.2 Security

  • Security concerns malicious manipulation of LLM4Code, including poisoning training data to degrade performance or implant triggers that cause chosen outputs.A backdoored model behaves normally on clean inputs but produces attacker-chosen outputs when a trigger is present.Source: §3.2 · pp. 7–8

3.3 Privacy

  • Privacy covers undisclosed model information and personal information in training data; membership inference can identify training examples and support data extraction.The section also discusses unauthorized training on datasets without their owners’ permission.Source: §3.3 · p. 8

3.4 Explainability

  • Explainability includes understanding model decisions, learned information, and post hoc explanations, such as showing similar vulnerable-code examples alongside a prediction.The section describes research across multiple tasks and model types.Source: §3.4 · pp. 8–9

3.5 Efficiency

  • Efficiency concerns resource use and speed across development and deployment, including training and inference time, memory, data requirements, and energy.Model size, architecture, and complexity affect resource consumption; CodeBERT is reported at 481 MB with 1.5-second average inference latency on a desktop CPU.Source: §3.5 · p. 9

3.6 Usability

  • Usability concerns ease of use, learnability, and productivity; code completion can be inconvenient when it supplies too little or irrelevant code.The section frames use as iterative and interactive rather than a one-time output handoff.Source: §3.6 · p. 9

4.1 Robustness

Robustness research tests how code models respond to input transformations and develops defenses with and without retraining. Reported methods can improve robustness, sometimes alongside accuracy, while scalability remains a concern for large models.

4.1.1 Robustness Evaluation.

  • Robustness evaluation assumes predictions should remain stable under semantics-preserving changes, including code transformations and natural-language prompt perturbations.The survey includes methods studied on non-LLM4Code when their transformations may apply to LLM4Code.Source: §4.1.1 · p. 10
  • Test-input generation ranges from unspecified transformations to heuristic-, gradient-, search-, reinforcement-learning-, style-, and transferability-based methods.Gradient-based approaches assume white-box access; heuristics can guide search and reinforcement-learning methods.Source: §4.1.1 · pp. 10–11
  • Style transformations that preserve readability while removing identifying features show popular authorship-attribution systems are not robust to those changes.The survey also describes semantic-preserving transformation taxonomies and studies of unspecified transformations.Source: §4.1.1 · pp. 10–11

4.1.2 Robustness Enhancement.

  • Defenses without retraining include replacing variable names or filtering outliers, while retraining approaches include adversarial training and canonicalizing code.Replacing variables with an unknown token is 100% robust to variable renaming by construction, but sacrifices accuracy; outlier detection is reported as a better trade-off.Source: §4.1.2 · p. 12
  • Staggered adversarial training and SPACE improve both accuracy and robustness in reported downstream tasks and against variable renaming, respectively.SPACE defines perturbations in continuous embedding space, enabling gradient-based adversarial training.Source: §4.1.2 · p. 12
  • Robustness studies cover many tasks, but few assess recent large code-generation models, and retraining-based defenses are usually tested on small models and datasets.The survey calls for scalable methods that can protect models with billions of parameters.Source: §4.1.2 · p. 12

4.2 Security

The section reviews attacks that manipulate code models through training data or model access, alongside defenses for poisoned data. Existing defenses struggle with stealthy attacks, motivating more effective detection.

4.2.1 Security Evaluation.

  • Poisoned training data can degrade performance, elicit undesired outputs, or implant backdoors that activate on specific triggers across code-model tasks.Examples include generating vulnerable code, recommending fake APIs, and returning insecure code snippets in search.Source: §4.2.1 · p. 13
  • Attackers make triggers harder to detect through adaptive or concealed designs, while model-access attacks can prompt or tune systems to generate vulnerable code.Examples include variable-renaming triggers, malicious code in comments, and adversarial natural-language instructions.Source: §4.2.1 · p. 13

4.2.2 Security Enhancement.

  • Existing automated detectors often perform poorly; some reported defenses improve detection, but an improved spectral-signature method fails on adaptive triggers.For CodeBERT, spectral signature yielded 6.10% recall and 8.63% precision in one study; another defense detected a specific attack’s examples at accuracy close to 100%.Source: §4.2.2 · pp. 13–14
  • Human reviewers also struggle to identify poisoned code, and stealthy attacks remain a challenge despite automated detection and review strategies.Developers scored an F1 of 0.43 distinguishing poisoned from clean code; the survey calls for more effective defenses as training data can be accessed and altered.Source: §4.2.2 · p. 14

4.3 Privacy

The section surveys privacy risks from memorized training data and methods to detect or reduce unauthorized use and disclosure. It notes that the effectiveness and cross-property impact of data-cleaning strategies remain unmeasured.

4.3.1 Privacy Evaluation.

  • Code models can memorize and disclose sensitive training data, with studies reporting personal information, credentials, and other secrets in model outputs.One study found about 8% (43) of prompts elicited privacy leaks from Copilot; another extracted 2,702 hard-coded credentials from Copilot and 129 secrets from CodeWhisperer.Source: §4.3.1 · p. 14
  • Membership inference tests whether a data point was used for training and can support extraction by identifying generated outputs likely to belong to the training set.The survey describes membership-inference frameworks for code-completion models and heuristics based on perplexity.Source: §4.3.1 · pp. 14–15

4.3.2 Enhancing Privacy.

  • Privacy defenses include detecting and removing secrets from training data, using membership inference to identify unauthorized data use, and watermarking datasets to trace their use.One proposed mitigation is removing duplicates because memorization probability is highly correlated with a snippet’s occurrence in training data.Source: §4.3.2 · p. 15
  • Mitigations for membership-inference and data-extraction threats are scarce, model stealing is underexplored, and secret-removal strategies lack empirical effectiveness and cross-property evaluations.The survey mentions differential privacy as a potential technique, not as an established mitigation in the reviewed studies.Source: §4.3.2 · p. 15

4.4 Explainability

The review surveys methods for evaluating and enhancing LLM4Code explainability, while identifying conflicts among explanations and gaps in task coverage and end-user support.

4.4.1 Explainability Evaluation.

  • Attention analysis examines which code tokens models prioritize and how attention patterns vary across tasks.Other approaches identify prediction-contributing tokens or use counterfactual analysis, causal inference, and probing.Source: §4.4.1 · pp. 15–16
  • Model-agnostic methods including LIME, BreakDown, and SHAP are used mainly for classification, but their explanations can conflict.Applications include defect prediction and bug-report classification.Source: §4.4.1 · p. 16

4.4.2 Explainability Enhancement.

  • Enhancement methods include visualizing predictions, analyzing causal mechanisms, and reducing program inputs while preserving predictions.Other work targets interpretability in program synthesis and text-to-SQL generation.Source: §4.4.2 · p. 16
  • Existing studies largely explain classification rather than generative tasks, and current explanations do not yet meet end-user needs.The review calls for more reliable explanations because methods can produce conflicting results.Source: §4.4.2 · p. 16

4.5 Efficiency

The review describes efficiency measures and enhancement strategies for LLM4Code, including compression, input reduction, dynamic inference, and parameter-efficient fine-tuning. It notes performance trade-offs and unresolved effects on other properties.

4.5.1 Efficiency Evaluation.

  • Efficiency evaluation covers model size, FLOPs-based complexity, latency, and memory consumption, with training and inference latency distinguished.Model size is measured in MB or GB; inference latency is commonly measured in seconds per input or token.Source: §4.5.1 · p. 17

4.5.2 Efficiency Enhancement.

  • Compressed CodeBERT and GraphCodeBERT are 4.31× and 4.15× faster at inference, retaining 96.15% and 97.74% of original vulnerability-prediction performance, respectively.The models are compressed to 3 MB, 160× smaller than the original; Avatar reports up to 184× less energy and 76× faster inference with an average 1.67% effectiveness loss.Source: §4.5.2 · p. 17
  • DietCode reduces CodeBERT’s fine-tuning and inference computational cost by 40%, while prompt optimization cuts token cost by 62% on program repair tasks.Other approaches shorten tokenized inputs or simplify inputs, with one method reducing training and inference time by 20% to 40%.Source: §4.5.2 · pp. 17–18
  • Dynamic inference methods skip layers, reject low-return prompts, or use early-layer representations to reduce computation and latency.Reported outcomes include 11.2% speedup with 1.1% performance reduction, and 23.3% lower FLOPs with 80.2% of rejected prompts judged unhelpful.Source: §4.5.2 · p. 18
  • Parameter-efficient fine-tuning is increasingly used to lower training costs and offers benefits in low-resource settings while achieving performance comparable to full fine-tuning.PEFT updates only a small number of parameters and freezes most pretrained-model parameters.Source: §4.5.2 · p. 18
  • The review reports a trade-off between efficiency and performance, while effects of efficiency improvements on robustness and security remain unclear.Quantization and pruning receive relatively limited exploration compared with PEFT.Source: §4.5.2 · p. 18

4.6 Usability

The review finds that usability studies examine user interaction through observation, interviews, surveys, simulations, and online discussions. Reported productivity effects diverge, and proposed design changes have limited in-practice evaluation.

4.6.1 Usability Evaluation.

  • Usability research uses observation, interviews, surveys, simulations, and mining online discussions to examine LLM4Code use.The review also describes observational studies of novice programmers and controlled prompt-writing studies.Source: §4.6.1 · pp. 18–20
  • In one study, 95 professional programmers completed an HTTP-server task 55.8% faster with Copilot than without it (95% confidence interval: 21–89%).Other observations report difficulties understanding, editing, and debugging generated code, and more subsequent deletions despite increased lines of code.Source: §4.6.1 · p. 19
  • Programmers in one study spent 34.3% of session time double-checking and editing Copilot suggestions, with over half of task time on Copilot-related activities.The study analyzed 21 programmers’ interaction patterns.Source: §4.6.1 · p. 19
  • Surveyed developers cited fewer keystrokes, faster task completion, and syntax recall as motivations, while difficulty controlling outputs and unmet requirements were reasons not to use assistants.One survey analyzed 2,631 Copilot users; another received responses from 410 developers.Source: §4.6.1 · p. 19
  • Studies report that GPT-4-supported creative coding can help families understand game code, debug programs, and generate ideas.The simulation used 22 Scratch projects to produce 120 coding-support scenarios.Source: §4.6.1 · pp. 19–20

4.6.2 Usability Enhancement.

  • Suggested usability improvements include explanations, shorter recommendations for novice learners, and customization to fit users’ workflows.User-side guidance also includes providing clear, concise prompts.Source: §4.6.2 · p. 20
  • Evaluated design changes include uncertainty highlighting that speeds task completion and inline interfaces that increase IntelliCode usage.The review notes that few studies test design suggestions in practice.Source: §4.6.2 · pp. 20–21

5 CHALLENGES AND OPPORTUNITIES

The review organizes opportunities for improving LLM4Code’s non-functional properties into data-centric, human-centric, and system-centric perspectives. These perspectives address training-data quality, user needs, and deployment concerns.

5 CHALLENGES AND OPPORTUNITIES

  • The review proposes three complementary perspectives—data-centric, human-centric, and system-centric—for developing LLM4Code with non-functional properties in view.The paper presents these as ways to organize challenges and opportunities for enhancement.Source: §5 · p. 21

5.1 Data-Centric View

  • The data-centric view treats dataset quality as fundamental because issues such as memorization, class imbalance, mislabeling, and outdated data can be inherited by models.Proposed directions include identifying low-quality data, auditing and filtering datasets, augmentation, and data transparency.Source: §5.1 · p. 21

5.2 Human-Centric View

  • The human-centric view targets usability and developer wellbeing by identifying usability factors, building trust, personalizing tools, and benchmarking interactive use.Suggested trust supports include explanations, confidence scores, uncertainty indicators, and counterfactuals.Source: §5.2 · p. 22

5.3 System-Centric View

  • The system-centric view frames LLM4Code as part of broader practical systems, where effectiveness and user-friendliness must be considered in real-world settings.Opportunities include attack resistance, operational monitoring and updates, efficiency and scalability, and legal and ethical safeguards.Source: §5.3 · p. 23

6 THREATS TO VALIDITY

The review addresses selection bias through documented criteria and broad database searching, and uses iterative snowballing to capture recent studies, with searches ending February 15, 2024.

6 THREATS TO VALIDITY

  • Documented, reproducible inclusion and exclusion criteria and a comprehensive DBLP search aim to reduce selection bias and capture a broad range of relevant studies.The search includes preprints.Source: §6 · p. 24
  • To capture recent studies, the authors iteratively reviewed references of selected papers across multiple snowballing rounds, adding newly identified publications; the last search round was conducted on February 15, 2024.Source: §6 · p. 24

7 CONCLUSION

The conclusion synthesizes research on non-functional properties of LLM4Code beyond accuracy, reviewing 215 relevant papers. It summarizes definitions, evaluation and enhancement practices, and proposes research opportunities across three development perspectives.

7 CONCLUSION

  • The review identifies seven non-functional properties beyond accuracy and analyzes 215 relevant papers and their distribution and trends.The listed properties include robustness, security, privacy, explainability, efficiency, and usability.Source: §7 · p. 24
  • For each property, the review discusses definitions, evaluation approaches, current enhancement practices, and key findings.Source: §7 · p. 24
  • The paper frames LLM4Code development through data-centric, human-centric, and system-centric perspectives, discussing challenges and research opportunities for each.Source: §7 · p. 24
  • The authors identify benchmarks and tools for evaluating and enhancing non-functional properties as promising future work.They also plan to revisit relevant studies and update the review.Source: §7 · p. 24
Loading 2403.07506v1…