Security in LLM-Assisted Coding: A Systematic Literature Review

1 Introduction

• Generated code can be functionally correct yet exploitable, while patches may leave security issues unresolved or introduce new weaknesses.Evaluation is complicated by benchmark contamination, incomplete vulnerability coverage, and differences between isolated tasks and realistic development settings.Source: §1 · p. 1
• Agentic coding expands security risks beyond snippets: errors can propagate, tool use adds attack surfaces, and autonomy can make actions less visible to reviewers.Secure deployment therefore requires workflow-level risk analysis and controls governing agent actions.Source: §1 · p. 2
• Existing reviews provide limited synthesis of how code weaknesses, evaluation limits, human interaction, and increasing autonomy jointly affect coding-workflow security.The relationship between current evaluation measures and what secure deployment requires remains insufficiently integrated with analyses of oversight and agentic risk.Source: §1 · p. 2
• The review examines generation and repair weaknesses, evaluation limitations, agentic threats, human oversight, and governance conditions for trustworthy deployment.It also connects technical safeguards, evaluation practices, and deployment governance.Source: §1 · pp. 2–3
• The four-layer framework links system capabilities, failure modes, evaluation pipelines, and controls to organize analysis of risks and mitigation.It provides a common structure for examining how coding capabilities expose risks and how those risks can be assessed and mitigated.Source: §1 · p. 3

2 Related Works

• Anand et al. [4] review LLM use in APR and code generation, while Huang et al. [53] and Zhang et al. [150] survey APR techniques and applications.Zhang et al. [150] include repair scenarios involving security vulnerabilities.Source: §2 · p. 4
• Basic and Giaretta [11], Sheng et al. [122], and Zhou et al. [156] address generated-code vulnerabilities, vulnerability detection, or repair, but do not fully integrate broader deployment issues.Those issues include benchmark reliability, human-AI trust, governance, and increasing model autonomy.Source: §2 · p. 4
• Li and Chow frame oversight and governance risks, while Li and Chow describe agentic risks including tool misuse and compounding multi-step errors.These perspectives extend code-centric analysis to operational and organizational risks.Source: §2 · p. 4
• Unlike prior reviews focused separately on APR, vulnerability detection and repair, or AI safety, this survey integrates generation, repair, evaluation, oversight, agentic risk, and governance.It emphasizes contamination, the partial visibility of isolated static or dynamic analysis, and the mismatch between function-level benchmarks and deployment settings.Source: §2 · p. 4
• The review aims to clarify how technical, human, and organizational factors jointly shape secure use of LLM-assisted coding systems.This extends its scope beyond summarizing vulnerabilities and defenses.Source: §2 · p. 4

3 Review Methodology

• The review follows established software-engineering systematic-review guidance and proceeds through research questions, structured search and selection, and final snowballing.The stages are intended to ensure completeness.Source: §3 · p. 4

3.1 Research Questions

• The review is guided by three questions on security issues in code tasks, evaluation methods and benchmarks, and human oversight in secure development.These questions define the review's core objectives.Source: §3.1 · p. 5

3.2 Search Strategy

• The search spans major digital libraries and preprint archives, using combined terms for LLMs, code tasks, security, and human-AI interaction.The search initially covered 2020 to early 2025, with forward and backward snowballing applied to selected papers.Source: §3.2 · p. 5

3.3 Study Selection Criteria

• Selection screened titles, abstracts, and keywords before full-text review against predefined criteria for security-related LLM code tasks.Eligible work included peer-reviewed studies and, exceptionally, recent preprints; exclusions included non-English papers and work without a direct technical code-security component.Source: §3.3 · p. 6

3.4 Quality Assessment of Preprints

• Recent preprints without peer-reviewed counterparts were retained selectively and screened for author credibility, methodological transparency, evidence quality, and later archival versions.The additional checks address the field's rapid evolution and the risk of relying on unstable evidence.Source: §3.4 · p. 6

4.1 Inherent Security Challenges

• LLMs reproduce insecure training patterns and outdated practices; hallucinated APIs can make security failures plausible and difficult to detect.Source: §4.1 · pp. 7–9
• Up to 65% of code generated by some LLMs was initially insecure, while approximately 40% of GitHub Copilot snippets were susceptible to vulnerabilities.Source: §4.1 · p. 8
• LLM patches can be functionally plausible yet insecure; ChatGPT and GPT-4o correctly identified and fixed only 17 of 32 tested vulnerabilities.Source: §4.1 · p. 8

4.2 Human-Related Security Issues

• Vague prompts can elicit vulnerable code, and novices may obtain insecure outputs unless they identify and specify relevant security requirements.Source: §4.2 · pp. 9, 11
• AI-assisted programmers produced less secure code in four of five security-focused tasks while being more likely to believe their code was secure.Source: §4.2 · pp. 9–10

4.3 Attack-Related Security Issues

• Prompt injection can bypass safeguards, while poisoned data can implant backdoors or systemic weaknesses that reappear in generated code.Source: §4.3 · p. 11

4.4 Security Implications of Agentic Coding Systems

• Agentic systems expand attack surfaces through tools and retrieved content, adding trust boundaries across repositories, APIs, and development environments.Source: §4.4 · p. 12
• Agentic workflows can propagate early errors across steps and obscure intermediate actions, making audit and vulnerability attribution harder.Source: §4.4 · p. 12

5 Evaluating the Security of LLM-assisted Coding

• Current evaluations probe different aspects of risk, including functional correctness, known vulnerabilities, runtime behavior, repository context, and human review.The review highlights benchmark contamination, limited realism, and cross-framework incomparability as evaluation concerns.Source: §5 · p. 13

5.1 Datasets

• Coding datasets range from functional-correctness benchmarks to security-specific sets designed to probe known vulnerabilities rather than whether code works.The review distinguishes these two broad dataset purposes.Source: §5.1 · p. 13, §5.1.2 · p. 14

5.1.1 Functional Correctness.

• HumanEval tests function synthesis, while MBPP, APPS, ClassEval, DS-1000, BigCodeBench, and CoderEval span broader task types and contexts.Class-level generation is substantially harder than function-level generation, and practical, context-dependent generation remains challenging.Source: §5.1.1 · pp. 13–14

5.1.2 Security-Specific.

• Security-specific datasets include prompt sets for known weaknesses and CodeSecEval samples with secure and insecure versions for automated assessment.SecurityEval maps 130 prompts to 75 CWE categories; CodeSecEval contains 180 samples spanning 44 critical vulnerability types.Source: §5.1.2 · p. 14
• 51.24% of 112,000 FormAI programs were labeled vulnerable, while at least 63.47% of 331,000 FormAI-v2 programs were vulnerable under ESBMC.The two studies used different generated program sets; the reported values are not a direct model comparison.Source: §5.1.2 · p. 14

5.2 Metrics

• Functional evaluation uses pass@k, compilation and unit tests, similarity measures, and task-specific criteria, but token similarity does not reliably establish correctness.Pass@k measures the probability that at least one of k generated samples passes all unit tests.Source: §5.2.1 · pp. 14–15, §5.2 · p. 14

5.2.2 Security-Specific.

• Security metrics distinguish secure code from code that is both secure and functional, while false refusal rate measures refusals of benign prompts.SVEN-SR can overestimate practical security because trivially safe but useless code may score well; Secure-pass@k includes functional correctness.Source: §5.2.2 · p. 15
• Static analyzers such as CodeQL, SonarQube, Bandit, ESLint, and Infer detect known flaws but can produce false positives and false negatives.These tools underpin many security metrics and flag specific vulnerability classes and hotspots.Source: §5.2.2 · p. 16

5.3 Evaluation Tools

• Static analysis scales across samples and vulnerability classes but has false positives, false negatives, and limited runtime or repository context.Dynamic tests, fuzzing, and exploit execution provide behavioral evidence but depend on coverage, reproducible environments, and attack-scenario fidelity.Source: §5.3 · p. 16
• Repository-aware evaluation exposes integration failures, while expert or developer review captures usability and oversight that automated tests cannot measure.Results should be interpreted in light of their task, oracle, context, and security objective.Source: §5.3 · p. 16

5.4 Cross-Framework Comparison and Evaluation Gaps

• Studies can reach different conclusions about the same model because they observe different failure surfaces, not necessarily because one study is incorrect.Frameworks differ in whether they assess correctness, vulnerability patterns, repository integration, exploitability, or human interaction.Source: §5.4 · p. 16
• Six comparison dimensions—task granularity, oracle, objective, context, security realism, and human involvement—help explain why benchmark results are often not directly comparable.Table 4 summarizes framework families and their trade-offs.Source: §5.4 · pp. 16–17
• No single framework suffices: static analysis offers breadth, dynamic testing behavioral evidence, repository benchmarks context, and human-centered evaluation deployment realism.The review identifies an integrated pipeline combining these complementary signals as the main evaluation gap.Source: §5.4 · p. 17

6 Human Oversight and Human-Factor Risks

• Human oversight is both a safeguard and a risk factor: developers can identify flaws, while over-trust, automation bias, and weak review can introduce vulnerabilities.The review treats human-AI interaction as a dual-role factor in security.Source: §6 · p. 17

6.1 Limitation of Automated Feedback

• Self-repair cycles generate code, run tests, and use errors or diagnostics to revise outputs, sometimes through dedicated debugging agents or compiler feedback.Examples include Agents4PLC, OpenCodeInterpreter, and StepCoder.Source: §6.1 · pp. 17–18
• Models lack formal correctness guarantees for industrial applications, may misunderstand specifications, and can introduce defects while repairing previously flawless code.They can struggle with PLC safety constraints, complex tasks, and efficient algorithms; coarse error messages may not help when outputs significantly diverge from intent.Source: §6.1 · p. 18
• Self-repair gains can be slight, inconsistent, or negligible across tasks, and models are less efficient when feedback addresses multiple errors at once.The review links limited self-correction to difficulty introspecting and generating effective feedback on their own mistakes.Source: §6.1 · p. 18

6.2 Incorporating Human Feedback

• RLHF, HITL, crowdsourced review, and conversational repair use human input, but their security benefits depend on reviewer expertise, feedback quality, and workflow design.Human feedback can be costly or inconsistent; HITL review is selective, and conversational repair may require many LLM interactions.Source: §6.2 · pp. 18–19
• Conversational Cref reduced debugging time by 71.2% and costs by 69.9% in corporate scenarios; expert Socratic guidance solved up to 74% of initially failed problems.SoHF studies mainly involve expert programmers, and some do not account for debugging time, leaving broader applicability unclear.Source: §6.2 · p. 20
• On MBPP, ILF raised a 6.1B-parameter model’s pass@1 by 38% relatively and 10% absolutely, but human-feedback cost and simultaneous errors constrain the method.ILF incorporates human-written feedback during training rather than relying on it during testing.Source: §6.2 · p. 20
• Expert review can add defensive programming and resolve ambiguous requirements, but manual oversight is resource-intensive and difficult to scale.The review recommends treating LLMs as an enhancement to human programmers rather than a replacement.Source: §6.2 · p. 20

7 Open Challenges and Research Directions

• Probabilistic generation remains mismatched with deterministic security requirements.The review presents this as one of three persistent challenges for secure LLM-assisted coding.Source: §7 · p. 21
• Evaluation pipelines remain unreliable or incomplete, while human-AI collaboration is fragile under over-trust, weak prompting, and limited accountability.These challenges are synthesized into the paper’s research and deployment agenda.Source: §7 · p. 21
• The section outlines governance, practical recommendations, and broader implications for academia and industry.Source: §7 · p. 21

7.1.1 Probabilistic Models vs. Deterministic Security.

• LLMs can learn insecure patterns from public code, produce vulnerable code or incomplete patches, and recommend deprecated or cryptographically weak components.The review attributes these failures to pattern replication and notes that patches may appear functional while remaining exploitable or introducing subtler vulnerabilities.Source: §7.1.1 · p. 21
• Because statistical pattern prediction lacks security reasoning, cleaning training data alone will not resolve the semantic gap; model design, training goals, or symbolic reasoning may also need to change.The review characterizes the gap as one between statistical association and logical deduction, including reasoning about threat vectors and security invariants.Source: §7.1.1 · pp. 21–22

7.1.2 The Unreliability of Ground Truth.

• Security evaluation is unreliable when small functional benchmarks, imperfect static tools, limited tests, and isolated-code settings miss complex vulnerabilities and repository context.The review warns that these limitations can make benchmark scores overstate security robustness.Source: §7.1.2 · p. 22
• Newer adversarial benchmarks shift evaluation from whether code works to whether it withstands exploitation, using repository context, dynamic exploits, and adversarial inputs.The review names SecRepoBench [30], CodeLMSec [47], and BAXBENCH as examples.Source: §7.1.2 · p. 22

7.1.3 Human-AI Trust-Vulnerability.

• Human oversight can be undermined by overconfidence, uneven security expertise, vague prompts, and trust miscalibrated to a model’s task-specific reliability.The review describes both over-trust, which can lead to uncritical acceptance, and under-trust, which can inhibit use of the technology.Source: §7.1.3 · pp. 22–23
• Rather than relying on developer training alone, collaboration tools should help calibrate trust through uncertainty communication, verifiable evidence, and flags for high-risk code requiring review.The proposed aim is to make developers more effective critical reviewers, rather than passive reviewers.Source: §7.1.3 · p. 23

7.2.1 Towards Inherently Secure Models.

• The roadmap calls for security-aware model architecture and training, including security-driven objectives and hybrid neural-symbolic approaches that go beyond next-token prediction.It also identifies security-oriented fine-tuning and curated vulnerability-correction data as practical research directions, while warning that fine-tuning can degrade safety alignment.Source: §7.2.1 · p. 23
• LLM-assisted formal methods can use model-generated annotations or invariants as inputs to dedicated verifiers, rather than treating the LLM as the definitive code author.The review notes that direct generation of formally verified code remains unsuitable beyond the simplest programs.Source: §7.2.1 · pp. 23–24

7.2.2 Next Generation of Security Benchmarks.

• Future security benchmarks should use repository context, dynamic exploit-driven assessment, and evaluations of the complete human-AI workflow rather than isolated code snippets alone.Suggested workflow measures include generated-code security, reviewer effort to find and fix AI-induced flaws, and the effectiveness of AI-aided review tools.Source: §7.2.2 · p. 24

7.2.3 Calibrating Human-AI Collaboration.

• Human-AI collaboration research should calibrate trust to a model’s actual task-specific reliability, not simply maximize user trust.Suggested interfaces can communicate uncertainty, provide verifiable evidence, and flag high-risk or low-confidence code for heightened human scrutiny.Source: §7.2.3 · pp. 24–25
• Interactive repair, Socratic feedback, and IDE-integrated review tools are proposed to make collaboration iterative and strengthen developers’ security review.The review cites Cref [145], conversational APR [142], SoHF, and CODE-GUARDIAN [35] as examples.Source: §7.2.3 · p. 25

7.3 Accountability, Auditability, and Governance

• AI-assisted development can diffuse responsibility across model developers, providers, deploying organizations, engineers, and reviewers, creating accountability gaps.The review calls for explicit organizational policies that retain human accountability for the final software artifact.Source: §7.3 · p. 25
• Reconstructing how code was produced requires traceability of prompts, retrieved context, model outputs, tool use, validation, and human edits.Without these records, incident investigation, failure reproduction, and identifying where a vulnerability arose become difficult.Source: §7.3 · p. 25
• Regulated and safety-critical deployments need rigorous documentation, testing, and approval, while governance should cover data handling, tool boundaries, review, logging, and human override.The review presents governance as a first-class defense layer that connects technical properties to deployment conditions.Source: §7.3 · p. 26

7.4 Practical Insights and Actionable Recommendations

• Treat all LLM-generated code as untrusted input and require security assessment, peer review, testing, and human supervision before production deployment.The review also recommends ongoing developer training in secure prompting, model limitations, and critical assessment of generated code.Source: §7.4 · p. 26
• Use complementary SAST, DAST, and SCA checks in CI/CD, since static analysis, runtime testing, and dependency analysis address different risks.The review notes that LLMs may recommend vulnerable or outdated dependencies.Source: §7.4 · pp. 26–27
• Require senior or specialist context-aware review for security-critical code, including authentication, authorization, encryption, session management, and input verification.Review should account for the code’s interactions with application components and data pathways.Source: §7.4 · p. 27
• Set prompt and data-handling policies that prohibit sending sensitive or proprietary information to external models and protect retrieval knowledge bases against breaches and poisoning.The review includes API keys, internal algorithms, trade secrets, and personal information as examples of sensitive data.Source: §7.4 · p. 27
• Vet AI tools, plugins, and open-source models for vendor practices, data retention, training procedures, adversarial vulnerabilities, and bias before adoption.The review recommends a standardized intake process for public-repository models.Source: §7.4 · p. 27
• Organizations should define human and team responsibility for final software security and document oversight, amendments, and approvals.The review notes that liability rules remain unclear and continue to develop.Source: §7.4 · p. 27

7.5 Implications for Academia and Industry

• The section argues that academia and industry must collaborate to balance innovation with security and reliability.Source: §7.5 · p. 27

7.5.1 Implications for Academia.

• Academia should prioritize security-focused architectures, adversarial defences, and realistic, context-rich security benchmarks over evaluation centered on functional tests.The section allows that more secure models may perform worse on some functional tasks.Source: §7.5.1 · pp. 27–28
• Research on LLM-assisted coding should span computer science, HCI, psychology, cognitive science, law, ethics, and social sciences to address technical and human concerns.The section links the Human-AI Trust-Vulnerability Nexus to HCI and identifies liability, privacy, intellectual property, and fairness as cross-disciplinary concerns.Source: §7.5.1 · p. 28
• Foundational research should investigate why systems fail, including how models represent security concepts and how structure and training data relate to output assurances.The section identifies interpretability studies, including representation engineering, and theoretical work as approaches.Source: §7.5.1 · p. 28

7.5.2 Implications for Industry.

• Industry faces a strategic trade-off between LLM productivity gains and the security, legal, and reputational risks of deployment.Source: §7.5.2 · p. 28
• LLMs may lower attackers’ costs of finding vulnerabilities and crafting long-tail assaults, while safer, more trustworthy tools may offer vendors a competitive advantage.The section describes these as opposing economic incentives for organisations and vendors.Source: §7.5.2 · p. 28
• Industry should combine security technology, governance, and developer awareness in a proactive, defence-in-depth strategy, with human judgment and oversight remaining crucial.The section also notes that legal uncertainty around damages from AI-produced code may prompt validation, oversight, and documentation procedures.Source: §7.5.2 · p. 28

• The review identifies a triad of systemic risks: weak model security awareness, inadequate benchmarks, and cognitive biases in human-AI interaction.Its roadmap emphasizes security-driven training, stringent benchmarks, and collaboration frameworks, while practitioners are urged to use zero-trust checks on generated code.Source: §8 · p. 29