PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models for Software Engineering: Survey and Open Problems

Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, Jie M. Zhang

arXiv:2310.03533v4cs.SE

TL;DR

LLM-based Software Engineering is expanding rapidly, but hallucinations and nondeterministic outputs make reliable evaluation and deployment difficult. The paper surveys developments and empirical evidence, identifies open challenges, and concludes that traditional SE techniques such as execution, testing, and verification are central to addressing them.

  • Problem

    LLM-based SE lacks sufficiently thorough evidence and reliable ways to handle incorrect, nondeterministic, and hallucinated outputs across software engineering tasks.

  • Method

    The paper surveys developments, empirical results, gaps, and open problems in applying LLMs across Software Engineering.

  • Results

    More than 10% of LLM papers already concern LLM-based Software Engineering, while reported code-generation evaluations show success rates between 20% and 80%.

  • Takeaways & Limitations

    Code execution, automated testing, and established Software Engineering verification methods can help filter hallucinated LLM outputs and support reliable LLM-based SE.

Abstract

from arXiv · show

This paper provides a survey of the emerging area of Large Language Models (LLMs) for Software Engineering (SE). It also sets out open research challenges for the application of LLMs to technical problems faced by software engineers. LLMs' emergent properties bring novelty and creativity with applications right across the spectrum of Software Engineering activities including coding, design, requirements, repair, refactoring, performance improvement, documentation and analytics. However, these very same emergent properties also pose significant technical challenges; we need techniques that can reliably weed out incorrect solutions, such as hallucinations. Our survey reveals the pivotal role that hybrid techniques (traditional SE plus LLMs) have to play in the development and deployment of reliable, efficient and effective LLM-based SE.

I. INTRODUCTION

The paper surveys the emerging applications and evidence for LLM-based Software Engineering, identifies gaps and open challenges, and emphasizes hybrid approaches for reliable evaluation and deployment.

  • The survey reviews recent developments, advances, and empirical results across LLM-based Software Engineering applications.
  • LLM-based SE is rapidly expanding, although any survey of the field can provide only an early snapshot rather than comprehensive coverage.
  • Hallucination threatens the correctness, security, and reliability of LLM-generated software artefacts.
  • Software execution supplies automatable ground truth, while established SE testing and verification techniques can help check LLM outputs.
  • The Automated Regression Oracle benchmarks adaptations against an existing software version, but cannot detect functional incorrectness already present in that version.
  • The survey highlights prompt engineering, output artefacts, and explanations as important directions for further research.
  • LLM nondeterminism and prompt sensitivity complicate scientific evaluation, but empirical SE and SBSE provide mature techniques for robust conclusions under noisy results.
  • More than 10% of LLM papers already concern LLM-based Software Engineering, indicating rapidly rising research uptake since 2019.

II. PRELIMINARIES

The preliminaries introduce LLMs, their architectures and development, and their growing use in software development, especially code generation and completion.

  • LLMs are AI models trained on large amounts of data that generate human-like text and support diverse language-related tasks.
  • Transformer self-attention substantially influenced language modelling and triggered rapid activity around LLMs.
  • GPT evolved through successive releases, with GPT-3 and GPT-3.5 producing a notable step change in generative performance and attracting broad interest.
  • LLMs include encoder-only, encoder-decoder, and decoder-only architectures, with GPT, LLaMA, Claude, and PaLM cited as decoder-only examples.
  • Code completion is supported by code-trained models such as CodeX and Copilot, while AlphaCode ranked on average in the top 54% in simulated competitions.
  • GPT-4 achieved 67% zero-shot pass@1 accuracy on HumanEval, a dataset containing 164 programming problems.
  • GPT-4 showed comparable performance with human developers on a benchmark of 100 LeetCode problems.

III. REQUIREMENTS ENGINEERING AND DESIGN

Requirements engineering and design remain comparatively underexplored for LLMs, despite natural-language capabilities that fit traceability and related activities; the section also situates these topics within broader LLM-based SE work.

  • Requirements engineering links a system’s technical attributes to its purpose and has a mature research literature.
  • Preliminary studies have examined requirement retrieval, classification, and completeness using LLM- or BERT-based methods.
  • LLM-based requirements engineering and design have received relatively little attention, with practising engineers reportedly reluctant to rely on LLMs for higher-level design goals.
  • Requirements engineering offers opportunities for LLM support because requirements are commonly written in natural language.
  • Most LLM applications have focused on code generation, testing, and repair, while requirements engineering remains less developed.
  • Traceability links between requirements, code, and tests are challenging and are identified as a natural fit for LLMs.

A. Code Generation Models

LLM-based code generation builds on software’s predictable, reusable structure and has expanded from foundational models to prompt-engineered, hybrid, and collaborative systems. The survey reports promising improvements while emphasizing combinations with traditional software engineering techniques.

  • A. Code Generation Models: Software’s predictable and reusable structure helps LLMs produce effective code-generation recommendations.Prior findings connect code naturalness and reuse with generate-and-test approaches for repair and genetic improvement.
  • A. Code Generation Models: CodeX introduced HumanEval to measure functional correctness for programs synthesized from docstrings and outperformed GPT-3 and GPT-J.HumanEval subsequently became a widely used benchmark for LLM-based code generation.
  • A. Code Generation Models: AlphaCode combines extensive data, efficient transformer architectures, large-scale sampling, and behaviour-based filtering, reaching the top 54% on simulated Codeforces competitions.The reported ranking applies to competitions with more than 5,000 participants.
  • A. Code Generation Models: Textbook-quality training data enabled smaller LLMs to achieve performance comparable to much larger models.The finding contrasts with approaches based on large datasets containing little pre-filtering.
  • A. Code Generation Models: Prompt engineering improved code-generation performance across benchmarks, models, and programming languages.Reported examples include pass@1 gains of approximately 50%–80%, success-rate increases from approximately 1/4 to 3/4, and security improvement from 59% to 92%.
  • A. Code Generation Models: Hybrid methods combine LLMs with planning, search, testing, ranking, program analysis, synthesis, or multiple agents to select or improve generated candidates.Reported improvements range from approximately 11%–27%, approximately 20%, and approximately 30%–47% across cited approaches.

D. Scientific Evaluation of LLM-based Code Generation

Scientific evaluation of LLM-based code generation must address correctness, robustness, explainability, determinism, and security. The survey highlights strong reported results alongside variable outputs, failure analyses, and limited consideration of nondeterminism.

  • D. Scientific Evaluation of LLM-based Code Generation: Thorough scientific evaluation is needed because LLMs can fail to generate correct, secure, and reliable code.The survey also notes a need for substantial human validation.
  • D. Scientific Evaluation of LLM-based Code Generation: GPT-4 achieved 67% zero-shot accuracy on HumanEval, a modest improvement over earlier ChatGPT results.HumanEval evaluates functional correctness on generated programs.
  • D. Scientific Evaluation of LLM-based Code Generation: Reflexion generated correct code for over 90% of HumanEval tasks, although leaderboard values and model rankings are subject to change.The comparison uses pass@1, the test pass rate for the top-1 code candidate.
  • D. Scientific Evaluation of LLM-based Code Generation: Code-completion performance dropped by more than 50% on HumanEval when the context contained bugs.This result concerns the reported effect of buggy context on completion performance.
  • D. Scientific Evaluation of LLM-based Code Generation: Explanations accompanying generated code may support adoption, confidence, and faster understanding, but require further evaluation and optimisation.Studies report perceived helpfulness among students and potential value for engineers working with obfuscated or undocumented code.
  • D. Scientific Evaluation of LLM-based Code Generation: More than 60% of tasks had zero equal test output across ChatGPT requests, while only 21.1% of surveyed papers considered nondeterminism.The survey identifies nondeterministic outputs as a threat to evaluation.

5) Security Evaluation:

LLM-based testing and evaluation draw on established software-testing techniques to assess correctness, coverage, and failures. Hybrid systems report gains across fuzzing, differential testing, GUI testing, and natural-language bug reproduction, while benchmark validity remains a concern.

  • 5) Security Evaluation:: Code-generation evaluations commonly use course problems, synthetic sets, and online judges, with reported success rates between 20% and 80%.Results vary naturally according to the LLM and training sets.
  • 5) Security Evaluation:: Existing test suites can produce false judgements of code correctness, motivating benchmarks tailored specifically to LLM-based code generation.The survey cites high degrees of false-positive conclusions from existing test suites.
  • 5) Security Evaluation:: SWE-bench evaluates realistic software-engineering tasks from 2,294 GitHub issues, with Claude 2 solving 4.8% and GPT-4 solving 1.7%.The benchmark targets coding problems in a realistic software-engineering setting.
  • 5) Security Evaluation:: Execution provides ground truth for filtering hallucinated responses and can guide interactive reasoning and action.Automated test-data generation targets relevant regions of this runtime ground truth.
  • 5) Security Evaluation:: TitanFuzz achieved 30%/51% higher code coverage than state-of-the-art fuzzers on PyTorch and TensorFlow.The approach uses Codex to generate valid input deep-learning programs for testing libraries.
  • 5) Security Evaluation:: Hybrid differential testing with ChatGPT improved failure-inducing test effectiveness from 29% to 78%.The approach targeted buggy programs.
  • 5) Security Evaluation:: LLM-guided GUI testing achieved 32% higher activity coverage, while Libro reproduced approximately one third of failures from general bug reports.Both results use natural-language interaction or reports as inputs to test generation.

B. Test Adequacy Evaluation

Test adequacy evaluation uses established criteria to assess whether generated suites reveal faults efficiently. LLM-based approaches support test minimisation, execution-trace prediction, flakiness prediction, and mutant generation, but realistic fault modelling remains open.

  • B. Test Adequacy Evaluation: Adequacy criteria provide lower bounds on test-suite effectiveness because testing cannot exhaustively explore every possibility.Mutation testing is more stringent than statement- and branch-coverage criteria because it injects synthetic faults.
  • B. Test Adequacy Evaluation: CodeBERT-based mutant generation achieved better fault-revelation ability than PiTest, while vulnerability-oriented mutants matched only 17% of vulnerabilities whose tests failed.The cited comparison highlights both improved fault revelation and limited overlap with vulnerability failures.
  • B. Test Adequacy Evaluation: Embedding-based test minimisation achieved a 0.84 fault detection rate and averaged 26.73 minutes, faster than the baseline.The approach used CodeBERT, GraphCodeBERT, and UniXcoder embeddings of test code.
  • B. Test Adequacy Evaluation: CodeExecutor predicted execution traces with 76% output accuracy on the Tutorial dataset versus 13% for Codex.The model aims to imitate arbitrary real-world program execution behaviour.
  • B. Test Adequacy Evaluation: LLM-based flakiness prediction reached 73% F1 and 97% accuracy in reported studies.Flaky tests can pass or fail without an apparent tester-controllable execution-context change.
  • B. Test Adequacy Evaluation: The survey identifies many open problems in LLM-based test-data generation as compatible with existing software-testing techniques.It presents this area as a continuing research agenda.

1) Prompt Engineering:

LLM-based testing raises open problems in prompt engineering, test augmentation, correctness assessment, and adequacy evaluation. The survey emphasizes hybridizing LLMs with established testing techniques while managing flakiness, oracle limitations, and misleading outcomes.

  • 1) Prompt Engineering:: Prompt engineering should target test flakiness, fault revelation, mocking choices, realistic inputs, corner cases, and production-relevant behavior.These goals span both test quality and the behavioral focus of generated tests.
  • 2) Augmenting Existing Tests:: LLM-based test generation has emphasized novel suites, leaving augmentation and regeneration from existing suites comparatively less studied.Existing suites can support few-shot learning or fine-tuning using test data and historical faults.
  • 2) Augmenting Existing Tests:: Additional assertions should capture corner cases, historical faults, and programmer errors, while hybridizing LLMs with automated test-generation techniques.The survey identifies assertion generation and hybridization as productive directions.
  • 3) Test Correctness:: Generated tests create two correctness cases: passing tests may become regression tests, while failing tests require distinguishing wrong assertions from discovered bugs.Both outcomes require interpretation rather than automatic acceptance.
  • 3) Test Correctness:: Passing generated tests can enforce incorrect behavior through coincidental correctness, potentially inhibiting later fault remediation.A test may pass because both the code and the test encode the same incorrect behavior.
  • 3) Test Correctness:: High false-positive rates can make bug-revealing generated tests impractical because human assessment costs may outweigh true-positive discoveries.The survey calls for automatic confidence assessment, correctness checks, consistency checks, robustness checks, filtering, and augmentation.
  • 3) Test Correctness:: Hallucination threatens test correctness but may expose discrepancies between program semantics and programmers’ perceived semantics when conversational filtering is used.Conversational testing agents filter generated tests through programmer interaction.
  • 4) Mutation Testing:: LLM-based testing needs stronger empirical and search-based software-engineering foundations, including careful scientific analysis and reporting.Mutation testing is another direction: fault-model fine-tuning can suggest mutants coupled to real faults for adequacy assessment.

VI. MAINTENANCE, EVOLUTION AND DEPLOYMENT

LLMs are being applied across software maintenance, debugging, repair, and deployment, with gains from task-specific fine-tuning and conversational workflows. The survey also identifies persistent challenges in hallucination control, scalability, data leakage, and reliable evaluation.

  • VI. MAINTENANCE, EVOLUTION AND DEPLOYMENT: Maintenance and evolution apply LLMs to understanding, business-logic extraction, re-engineering, repair, and refactoring of existing codebases.These activities are language-rich, making them natural application domains for LLM-based techniques.
  • A. Debugging: GPT-4 achieved 47% higher fault-localisation accuracy than the state of the art, but performance declined dramatically with longer code contexts.The evaluation also examined stability and explainability.
  • A. Debugging: AdbGPT reproduced 81% of Android bugs from 88 reports, outperforming its baselines and ablations.RING instead frames multilingual debugging as localization, transformation, and candidate ranking.
  • A. Debugging: ConDefects contains 1,254 Java bugs and 1,625 Python bugs dated October 2021 through September 2023, enabling cutoff-aware evaluation of leakage threats.Researchers can select samples by creation period to compare models against their training-data cutoff dates.
  • B. Program Repair: LLM-based repair commonly uses generate-and-test workflows, but hallucination control, scalability, and build time remain technical challenges.ReAct-style deployment is suggested as a way to explore trade-offs between candidate generation and testing.
  • B. Program Repair: Existing software-repair and search-based software-engineering research provides experience for developing LLM-based generate-and-test repair approaches.The survey points to more than two decades of search-based software-engineering development.
  • B. Program Repair: LLM-based APR research includes cloze-style repair, Completion Engine interaction, conversational validation, and few-shot self-debugging.Reported studies found cloze-style superiority, baseline gains of 14 and 16 bugs fixed, and accuracy improvements up to 12%.
  • B. Program Repair: Natural-language repair benchmarks reached up to 21% Top-1 and 36% Top-5 accuracy, while fine-tuned models substantially improved repair success across tasks and languages.Reported results include 160% more repaired faults than state-of-the-art DL-based APR after fine-tuning and InferFix accuracy above 65% on C# and 76% Top-1 on Java.

C. Performance Improvement

Performance improvement has evolved from correctness-by-construction transformations toward broader, test-filtered searches that can trade semantic guarantees for substantial non-functional gains. LLMs extend this direction, offering potentially dramatic optimisations while increasing correctness risks and the need for testing.

  • C. Performance Improvement: Performance optimisation has progressed from compiler-based techniques toward source-to-source transformations tailored to specific problem domains.General compiler optimisation is broadly applicable but can be suboptimal for bespoke domains.
  • C. Performance Improvement: Wider transformation palettes can generate incorrect code, but automated testing filters candidates to preserve sufficient faithfulness to intended semantics.This approach relaxes correctness-by-construction in exchange for a broader search space.
  • C. Performance Improvement: 70x speed up was achieved for a non-trivial gene sequencing system using existing code as genetic material for performance improvement.The reported gains include execution time, memory, and power consumption improvements.
  • C. Performance Improvement: 3% reduction in compiler instruction counts was produced by a 7B-parameter LLM, with 91% compilable and 70% functionally correct outputs.The model was trained to generate instruction counts and optimised LLVM code, outperforming the state of the art.
  • C. Performance Improvement: LLM-based code optimisation may extend the relaxation of semantic constraints because generated code may be syntactically or semantically incorrect.Emergent behaviour and abundant training data can nevertheless yield useful changes in performance characteristics.
  • C. Performance Improvement: More permissive transformations increase reliance on testing for functional faithfulness and for regressions in non-targeted non-functional properties.High-coverage automated test generation therefore becomes increasingly important.

1) Open Problems in Performance Improvement:

Open problems include expanding LLM-based optimisation beyond execution time, developing reliable bespoke refactoring, and improving documentation generation and software analytics. Across these areas, evaluation, oracle availability, and human-facing usefulness remain central concerns.

  • 1) Open Problems in Performance Improvement:: LLM-based performance improvement should address power consumption, memory footprint, and multi-objective trade-offs in addition to execution time.The survey calls for Genetic Improvement-style techniques tailored to these broader objectives.
  • 2) Open Problems in Refactoring:: Automated API refactoring is comparatively less challenging because existing software provides an Automated Regression Oracle.The oracle compares changed software against the original system’s functional behaviour.
  • 2) Open Problems in Refactoring:: Bespoke refactoring remains an open problem because project-specific requirements are common and reliable few-shot generalisation from examples is still needed.The paper reports that up to a third of software engineering effort is spent on repetitive, tedious, and potentially error-prone refactoring activities.
  • A. Open Problems in Documentation Generation and Code Summarization: ChatGPT performed significantly worse than NCS, CodeBERT, and CodeT5 on BLEU and ROUGE-L for Python code summarisation.The study used CSN-Python and also reported METEOR.
  • A. Open Problems in Documentation Generation and Code Summarization: Retrieval-based summarisation is constrained by its training corpus, whereas LLMs may generate summaries beyond that corpus.Richer summaries create evaluation difficulties because existing metrics are often lexical.
  • Software Analytics: LLMs may support software analytics by identifying new Mining Software Repository questions and assisting with traceability.The survey notes that existing collection, curation, and analysis often rely on labour-intensive human work.

X. SOFTWARE ENGINEERING PROCESS

The survey frames LLMs as components of broader software engineering processes rather than isolated tools. It identifies opportunities for SE-specific models, adaptive prompting, hybrid workflows, and evaluation practices that address non-determinism, energy use, and reproducibility.

  • X. SOFTWARE ENGINEERING PROCESS: Most prior work treats LLMs as atomic components integrated into wider workflows, with customisation focused mainly on prompt engineering and some fine-tuning.The survey calls for workflow design patterns that place LLMs safely, efficiently, and effectively within SE processes.
  • X. SOFTWARE ENGINEERING PROCESS: Dynamic execution information improved downstream predictive-task accuracy by up to 25% for vulnerability detection, clone detection, and coverage prediction.Ding et al. trained a BERT-like model using execution inputs and dynamic execution traces.
  • X. SOFTWARE ENGINEERING PROCESS: SE-specific LLMs should exploit software’s distinctive properties, including dynamic information that is missing from most existing work.The survey expects future models to be tailored specifically for software engineering tasks.
  • X. SOFTWARE ENGINEERING PROCESS: Lighter models could widen adoption and improve deployability, but LoRA and quantization require empirical evaluation for specific applications.The paper links model size to capability and larger-model development to environmental impact.
  • X. SOFTWARE ENGINEERING PROCESS: Prompt-engineering results are highly problem-specific, and parameters such as temperature can substantially affect generated output.The survey recommends reporting settings for replication and studying adaptive prompt optimisation and parameter tuning.
  • X. SOFTWARE ENGINEERING PROCESS: Static and dynamic analyses for prompting and response post-processing may support hybrid SE processes that incorporate LLMs into Continuous Integration pipelines.The survey expects rapid development of these hybrid approaches.

D. Harnessing Hallucination

Hallucinations are both a persistent correctness challenge and a possible source of useful software-engineering suggestions when placed in suitable contexts. The survey therefore connects repurposing hallucinations with stronger evaluation foundations, reproducibility, benchmarking, and domain-specific theory.

  • D. Harnessing Hallucination: Hallucinations may be useful as “plausible futures” when placed in the right software-engineering context rather than treated as purely random errors.The survey describes hallucination as a property that can sometimes generate useful alternatives.
  • D. Harnessing Hallucination: Hallucinated test cases, summaries, and APIs may suggest new features, reveal possible human misunderstandings, or motivate refactoring.These uses repurpose incorrect outputs as prompts for further engineering activity rather than accepting them as correct.
  • D. Harnessing Hallucination: Only 33% of surveyed code-generation papers shared source code and 27% shared trained artefacts, limiting replication support.The review also examined whether training energy consumption could be assessed independently.
  • D. Harnessing Hallucination: Scientific evaluation of LLM-based SE remains insufficient, motivating clearer, repeatable methodologies with controls or baselines.A survey of LLM-based testing filtered papers using standard evaluation-quality constraints.
  • D. Harnessing Hallucination: LLMs’ explanatory capabilities require domain-specific theoretical and empirical foundations beyond generic software-engineering science.The survey recommends drawing on Empirical Software Engineering and SBSE while developing foundations specific to LLMs.
  • D. Harnessing Hallucination: Hallucination risk is unlikely to be completely eradicated, making robust automated verification especially important for LLM-based SE.The survey also identifies benchmarking platforms and longitudinal studies of developer behaviour as future needs.
Loading 2310.03533v4…