Source-linked AI summary
Instruction Tuning for Secure Code Generation
Jingxuan He, Mark Vero, Gabriela Krasnopolska, Martin Vechev
TL;DR
Existing instruction tuning does not adequately address code security, and instruction-tuned models still often generate unsafe code. SafeCoder combines security-specific training and standard instruction tuning using an automatically collected security dataset. Across diverse models and testing scenarios, it substantially improves secure code generation while maintaining utility.
Problem
Existing instruction-tuning schemes inadequately address code security, and instruction-tuned LMs frequently generate insecure code despite the importance of preserving utility.
Source: §1 · pp. 1–2Method
Results
Across 60 testing scenarios, SafeCoder reaches a secure code generation rate of ∼90%, surpassing pretrained and standard instruction-tuned counterparts by ∼30% while maintaining utility across benchmarks.
Source: §1 · p. 2Takeaways & Limitations
The results demonstrate SafeCoder’s effectiveness across popular LMs and datasets, with substantial security gains and minimal impact on utility.
Source: §7 · p. 9SafeCoder currently targets instruction-tuned LMs, not pretrained code-completion models or already instruction-tuned LMs requiring post-hoc security patches.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Modern language models (LMs) have gained widespread acceptance in everyday and professional contexts, particularly in programming. An essential procedure enabling this adoption is instruction tuning, which substantially enhances LMs' practical utility by training them to follow user instructions and human preferences. However, existing instruction tuning schemes overlook a crucial aspect: the security of generated code. As a result, even the state-of-the-art instruction-tuned LMs frequently produce unsafe code, posing significant security risks. In this work, we introduce SafeCoder to address this gap. SafeCoder performs security-centric fine-tuning using a diverse and high-quality dataset that we collected using an automated pipeline. We integrate the security fine-tuning with standard instruction tuning, to facilitate a joint optimization of both security and utility. Despite its simplicity, we show that SafeCoder is effective across a variety of popular LMs and datasets. It is able to drastically improve security (by about 30%), while preserving utility.
1. Introduction
Instruction-tuned LMs often generate insecure code, while security training must preserve utility across coding and other tasks. SafeCoder combines security-specific tuning with standard instruction tuning, supported by an automated dataset pipeline, and improves security while maintaining utility.
The security gap
- Instruction-tuned LMs generate secure code only around 60% of the time, and generated vulnerabilities can require costly fixes or reach production.The paper also reports that security-aware prompting alone does not significantly improve security.Source: §1 · p. 1
This Work: SafeCoder
- SafeCoder combines secure-program language modeling with an unlikelihood loss on unsafe programs, masking both losses to focus training on security-critical code.It mixes security data with standard instruction-tuning data to jointly optimize security and utility.Source: §1 · p. 2
- Its automated two-step pipeline screens GitHub commits with lightweight heuristics, then uses static analysis to verify vulnerability fixes and label pre-commit and post-commit programs.The pipeline treats the pre-fix program as unsafe and the post-fix program as secure.Source: §1 · p. 2
Effectiveness of SafeCoder
- Across 60 testing scenarios, SafeCoder reaches a secure code generation rate of ∼90%, surpassing pretrained and instruction-tuned counterparts by ∼30% while maintaining utility across benchmarks.The evaluation covers six state-of-the-art LMs and two instruction-tuning datasets.Source: §1 · p. 2
Main Contributions
- SafeCoder introduces security-focused instruction tuning, a diverse security dataset and coding scenarios, and an evaluation spanning varied LMs and datasets.The authors also report open-sourcing their code and datasets.Source: §1 · p. 2
2. Related Work
Prior work establishes that code-generating LMs can produce vulnerable code, but secure generation remains an early research area. SafeCoder differs from earlier approaches in its target models, joint security–utility training, and automated dataset collection.
LMs for Code Generation
- Code-generation LMs gain coding capabilities through pretraining on extensive code corpora, with synthetic coding instructions later used to improve functional correctness.The cited work covers both coding-tailored and general-purpose LMs.Source: §2 · p. 2
Program Security
- CodeQL is adopted as a static-analysis engine because it supports mainstream languages and supplies queries for common CWEs.The paper describes it as an industry-leading engine and notes its use in prior evaluations of generated code.Source: §2 · p. 3
- Earlier vulnerability datasets built from fix commits can mislabel code when changes are unrelated to security, while SafeCoder’s automated pipeline aims for broader CWE and language coverage.He & Vechev (2023) used expensive manual inspection to curate their training dataset.Source: §2 · p. 3
Security of LM-generated Code
- Studies of pretrained LMs report frequent vulnerabilities, and Khoury et al. (2023) found ChatGPT below minimal security standards in 16 of 21 cases.That study reports ChatGPT self-corrected 7 cases after further prompting.Source: §2 · p. 3
- SafeCoder targets coding-specific and general-purpose instruction-tuned LMs, whereas SVEN focuses on pretrained code-completion models.The paper says instruction-tuned LMs also require natural-language reasoning capabilities.Source: §2 · p. 3
- SafeCoder jointly optimizes security and utility, while SVEN’s instruction-tuning application is described as limited by a trade-off between them.SVEN uses manual data curation; SafeCoder uses automatic collection.Source: §2 · p. 3
3. Background and Problem Statement
The paper frames secure code generation as a dual objective: improve security while retaining utility, including functional correctness and natural-language task performance. It targets instruction tuning rather than pretraining from scratch.
Pretraining and Instruction Tuning
- The work focuses on supervised fine-tuning, while considering reinforcement learning as future work.Instruction tuning follows pretraining and trains LMs to follow task-specific instructions and align with human preferences.Source: §3 · p. 3
Instruction Tuning for Secure Code Generation
- The goal is to improve security and utility together, with utility including functionally correct code and natural-language tasks.The paper identifies solving natural-language tasks as an example of utility.Source: §3 · p. 3
- SafeCoder targets instruction tuning because it is more efficient than pretraining from scratch in compute and training-sample requirements.The paper situates this choice alongside prior instruction-tuning approaches for other harmful-content categories.Source: §3 · p. 3
4. SafeCoder’s Instruction Tuning
SafeCoder combines security-specific training with standard instruction tuning in one run. Its paired secure and vulnerable programs support masked positive and negative learning signals, while data-handling choices address class imbalance and limit training overhead.
Security Instruction Tuning
- Each security-dataset tuple pairs an instruction with secure and vulnerable programs that share basic functionality and differ in security-related aspects.The example dataset contains more complicated code changes, affecting approximately 9% of program tokens on average.Source: §4 · p. 4
- SafeCoder applies masked negative log-likelihood to secure programs and masked unlikelihood loss to vulnerable programs, providing positive and negative security learning signals.The masks focus training on security-related tokens.Source: §4 · p. 4
Combining Standard and Security Tuning
- Training samples are drawn from the combined standard and security datasets, with the loss selected according to each sample’s source.Standard samples use Lstd; security samples use Lsec + Lvul.Source: §4 · pp. 4–5
- The paper describes joint optimization as balancing security and utility across models, improving security without compromising utility.This is the reported outcome of combining the two tuning schemes in one training run.Source: §4 · p. 4
Handling Data Imbalance
- Oversampling duplicates minority CWE–language classes with fewer than k samples until they reach k, where k is 20 or 40 in the experiments.The authors report that this improves security and stabilizes training.Source: §4 · pp. 4–5
- The authors leave the standard-to-security dataset distribution unchanged despite standard data being 5 or 12 times larger, reporting high security and small training-time overhead.The security dataset is relatively small compared with standard instruction-tuning data.Source: §4 · p. 5
5. SafeCoder’s Data Collection
SafeCoder’s data pipeline mines GitHub commits for verified vulnerability fixes, then forms instruction–function triples for security tuning. The resulting dataset balances sample quality with coverage across vulnerability types and languages.
5. SafeCoder’s Data Collection
- The dataset is designed for accurate security labels and broad coverage, with secure and vulnerable programs differing only in security-related aspects.It should cover a wide range of vulnerabilities and programming languages.Source: §5 · p. 5
- The pipeline filters GitHub commits, checks repository versions with CodeQL, extracts changed function pairs from verified fixes, and uses GPT-4 to generate instructions.A fix is verified when a non-empty pre-commit vulnerability list becomes empty post-commit; verification is performed per vulnerability type.Source: §5 · pp. 5–6
- Keyword and change-size heuristics reduce candidate commits to make static analysis feasible, using thresholds of 40 lines and 2 files.Commits with unsupported file types or changes exceeding those limits are excluded.Source: §5 · p. 5
- For each verified fix, the pipeline pairs pre-commit vulnerable functions with post-commit secure versions and adds them with a generated instruction to the dataset.The instruction describes the functions’ common functionality without mentioning security-specific features.Source: §5 · pp. 5–6
- After rebalancing and cleaning, the dataset contains 465 high-quality samples spanning 23 CWE categories and 6 mainstream programming languages.The pipeline processed over 145 million commits; a vulnerability fix was verified for 1,211 samples, or 4.9% of successfully analyzed samples.Source: §5 · p. 6
6. Experimental Evaluation
SafeCoder is evaluated on coding and general-purpose LMs using code-security scenarios and utility benchmarks. It improves security while preserving comparable utility, but does not significantly improve security on unseen CWEs.
6.1. Experimental Setup
- On six open-source LMs, utility is measured with HumanEval, MBPP, MMLU, and TruthfulQA, while code security is evaluated across 60 scenarios.Source: §6.1 · pp. 6–7
- The security training set combines 1,268 samples covering 25 CWEs across six languages, with minority language-CWE classes oversampled.Source: §6.1 · p. 6
6.2. Experimental Results
- Across coding and general-purpose LMs, SafeCoder significantly improves code security while preserving utility scores comparable to standard instruction tuning.Source: §6.2 · p. 7, Table 1 · p. 7, Table 2 · p. 7
- Removing collected training data makes the model about 20% less secure; removing loss masks lowers security by about 10%, and removing unlikelihood loss lowers it by 5.1% on StarCoder-1B and 10.6% on Phi-2-2.7B.Source: §6.2 · p. 8, Table 3 · p. 8
- Unlike SafeCoder, SVEN exhibits a trade-off between security and functional correctness as its KL weight varies.Source: §6.2 · p. 8, Figure 3 · p. 8
- SafeCoder does not significantly improve security on CWEs absent from its training set, indicating limited generalization across CWEs.Source: §6.2 · p. 9, Table 4 · p. 9
- For coding LMs, oversampling minority classes to k=20 improves security and reduces variance, with diminishing returns above 20; k=40 is more beneficial for general-purpose LMs.Source: §6.2 · p. 9, Figure 4 · p. 9
7. Conclusion and Discussion
SafeCoder combines security-specific training with standard instruction tuning in one run, jointly optimizing security and utility. Experiments report substantial security gains with minimal utility impact, but the method has defined scope limits and no formal security guarantee.
7. Conclusion and Discussion
- Across popular LMs and datasets, SafeCoder achieves substantial security improvements with minimal impact on utility.Its unified training run combines masked language modeling on secure programs and unlikelihood loss on unsafe code with standard instruction tuning on other samples.Source: §7 · p. 9
- SafeCoder is designed for instruction-tuned LMs, not pretrained code-completion models or already instruction-tuned models needing post-hoc vulnerability patches.Extending the approach to reinforcement learning is identified as future work.Source: §7 · p. 9
- SafeCoder increases the likelihood of generating secure code but provides no formal guarantee that generated code is secure.The authors state that improved likelihood could reduce developers’ vulnerability-fixing effort and the risk of vulnerabilities reaching production.Source: §7 · p. 9
Impact Statement
The work aims to improve code-generation security and make its advances available to practitioners and LM users. The authors also note possible misuse and point to their security evaluation as a way to detect malicious behavior.
Impact Statement
- The authors plan to open-source the work so practitioners and LM users can benefit from it.They caution that misapplication could train LMs to generate unsafe code, and say the security evaluation can help detect malicious behavior stemming from such use.Source: Impact Statement · p. 9
A. Details on Experimental Setup
The appendix reports that security-aware prompts do not significantly improve code security for current instruction-tuned models. It also documents data post-processing, evaluation prompts, training settings, and an adaptation of SVEN for comparison.
A. Details on Experimental Setup
- The collected security dataset was post-processed by reducing overrepresented CWE-language pairs, removing likely CodeQL errors, and adding 10 CWE-476 samples.The additions addressed insufficient diversity among samples collected from GitHub.Source: A. Details on Experimental Setup · p. 13
- Instruction tuning generally runs for two epochs at a 2e-5 learning rate, while CodeLlama-7B uses five epochs and a 1e-3 rate.All models use batch size 1 with gradients accumulated over 16 steps; LoRA uses r=16, α=32, and 0.1 dropout.Source: A. Details on Experimental Setup · p. 13
- Coding evaluations use task descriptions and the expected programming language in an instruction prompt, with the program prefix as the response prefix for comparability with pretrained models.The same instruction template is used across all six evaluated LMs.Source: A. Details on Experimental Setup · p. 13
- MMLU and TruthfulQA use a 5-shot completion prompt across pretrained and instruction-tuned LMs; TruthfulQA options are shuffled to avoid a biased few-shot evaluation.The authors chose completion prompts because instruction-formatted question-answering increased incorrect output formats.Source: A. Details on Experimental Setup · p. 13
- SVEN is adapted to instruction tuning with full fine-tuning and the instruction-tuning data format, and its loss combines security, vulnerability, and KL-divergence terms.The KL terms align the fine-tuned model’s next-token probabilities with those of the original model and are weighted by wKL.Source: A. Details on Experimental Setup · p. 14
- Security-aware instructions do not significantly improve code security for current instruction-tuned models.Table 5 compares functionality-only prompts with generic and CWE-specific security instructions.Source: Table 5 · p. 15
B. Further Experimental Results and Details
The appendix reports that security-aware prompting does not significantly improve security for current instruction-tuned models. It also details prompt variants, testing scenarios, dataset statistics, and scenario-level result breakdowns.
B. Further Experimental Results and Details
- Security-aware prompting does not significantly improve code security for state-of-the-art instruction-tuned LMs.The tested variants add either a generic security instruction or one tailored to the test case’s CWE; the latter assumes users know which CWEs may arise.Source: B. Further Experimental Results and… · p. 15
- The collected security dataset’s programs average 367 tokens, with about 9% of tokens within code changes; GPT-4-generated descriptions average 24 tokens.Source: Table 6 · p. 16
- The listed security scenarios span multiple CWEs and programming languages, including Python, C, Java, Go, JavaScript, and Ruby.The appendix lists scenario identifiers and brief task descriptions for the collected-data evaluation.Source: B. Further Experimental Results and… · p. 17
- The testing setup includes scenarios whose target CWEs are absent from SafeCoder’s training set for a generalization experiment.Tables 7 and 8 distinguish the main testing scenarios from those reserved for that experiment.Source: Table 7 · p. 17, Table 8 · p. 18
- The appendix reports scenario-level security breakdowns and a no-collected-data versus full-method comparison for StarCoder-1B.The former is in Table 9 and the latter in Table 10; the supplied table descriptions do not provide their cell values.Source: Table 9 · p. 19, Table 10 · p. 20