Source-linked AI summary
GoodVibe: Security-by-Vibe for LLM-Based Code Generation
Maximilian Thang, Lichao Wu, Sasha Behrouzi, Mohamadreza Rostami, Jona te Lintelo, Stjepan Picek, Ahmad-Reza Sadeghi
TL;DR
LLMs used in informal vibe-coding workflows often generate functionally correct but insecure code, while existing security adaptations provide costly or coarse-grained solutions. GoodVibe identifies security-critical neurons through supervised gradient attribution and selectively fine-tunes them with clustering. Across six models and four languages, it improves security while preserving general utility and using far fewer trainable parameters and less computation.
Problem
Informal code-generation workflows rarely specify security, and existing full-parameter or parameter-efficient adaptations are costly, prone to forgetting, or coarse-grained.
Method
GoodVibe identifies security-critical neurons using gradient-based attribution from a supervised security task and selectively fine-tunes that subspace with activation-driven clustering.
Results
GoodVibe consistently improves security across six models and four languages, achieving up to a 2.5× increase over base models while using over 4 700× fewer trainable parameters and lower computational cost.
Takeaways & Limitations
Neuron-level optimization offers a fine-grained approach to strengthening security-aware code generation while preserving general coding capabilities and reducing fine-tuning cost.
Takeaways & Limitations
The evaluation excludes active attacks and repository-level structural vulnerabilities requiring multi-file or deployment-context analysis.
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly used for code generation in fast, informal development workflows, often referred to as vibe coding, where speed and convenience are prioritized, and security requirements are rarely made explicit. In this setting, models frequently produce functionally correct but insecure code, creating a growing security risk. Existing approaches to improving code security rely on full-parameter fine-tuning or parameter-efficient adaptations, which are either costly and prone to catastrophic forgetting or operate at coarse granularity with limited interpretability and control. We present GoodVibe, a neuron-level framework for improving the security of code language models by default. GoodVibe is based on the key insight that security-relevant reasoning is localized to a small subset of neurons. We identify these neurons using gradient-based attribution from a supervised security task and perform neuron-selective fine-tuning that updates only this security-critical subspace. To further reduce training cost, we introduce activation-driven neuron clustering, enabling structured updates with minimal overhead. We evaluate GoodVibe on six LLMs across security-critical programming languages, including C++, Java, Swift, and Go. GoodVibe substantially improves the security of generated code while preserving general model utility, achieving up to a 2.5x improvement over base models, achieving performance competitive with full fine-tuning while using over 4,700x fewer trainable parameters, and reducing training computation by more than 3.6x compared to the parameter-efficient baseline (LoRA). Our results demonstrate that neuron-level optimization offers an effective and scalable approach to securing code generation without sacrificing generality.
1 Introduction
GoodVibe secures code LLMs by locating security-relevant neurons and selectively adapting them, rather than updating the full model. Across multiple models and languages, it improves secure code generation while reducing parameter and computational costs.
- Motivation: Vibe coding prioritizes speed and convenience over explicit security review, allowing insecure LLM-generated code to be reused with minimal auditing.Prior work identifies injection vulnerabilities, hallucinated dependencies, and unsafe memory operations in generated code.
- GoodVibe framework: GoodVibe identifies a small subset of neurons that concentrates security-relevant reasoning, challenging uniform parameter-update assumptions.The framework treats these neurons as a security-critical subspace for targeted optimization.
- GoodVibe framework: Gradient-based attribution from a supervised secure-versus-insecure code task identifies security-critical neurons without requiring explicit semantic labels.Neuron-level gradient magnitudes from a single backward pass indicate which neurons influence security-related decisions.
- GoodVibe framework: Cluster-based fine-tuning groups security neurons with similar functional roles, reducing training complexity without compromising security adaptation.The mechanism performs structured updates within neuron clusters.
- Results: 2.5× increase in secure code generation is achieved on C++ and Java over base models, with performance competitive with full fine-tuning using over 4 700× fewer trainable parameters.GoodVibe also matches or exceeds LoRA across C++, Java, Swift, and Go at substantially lower computational cost.
2 Preliminaries
Code LLMs generate code autoregressively and generalize across coding tasks, but security properties are not explicitly supervised during pre-training. Fine-tuning adapts these models to security objectives, while PEFT reduces cost by restricting trainable parameters or updates.
- Code language models: Code LLMs are transformer-based autoregressive models trained to predict the next token over code vocabularies.Their pre-training uses large-scale corpora spanning multiple programming languages and development contexts.
- Code language models: Security-relevant reasoning emerges implicitly from training data because code LLM pre-training does not explicitly model security properties.Examples include identifying unsafe data flows or vulnerable API usage.
- Fine-tuning: Fine-tuning adapts pre-trained language models to downstream objectives such as vulnerability detection or secure code generation.It is presented as a standard approach for security-related adaptation.
- Parameter-efficient fine-tuning: Parameter-Efficient Fine-Tuning reduces adaptation cost by restricting optimization to a subset of parameters while freezing the pre-trained backbone.Adapter, prefix or prompt tuning, and low-rank adaptation are described as common PEFT approaches.
3 GoodVibe
GoodVibe secures code LLMs by identifying security-relevant neurons and selectively optimizing them while preserving the remaining model parameters. It further reduces optimization complexity by clustering related neurons and sharing structured updates within the security-critical subspace.
- Design Intuition: GoodVibe targets security-relevant reasoning localized in a small subset of neurons rather than uniformly distributed across model parameters.The framework aims to improve intrinsic security awareness during ordinary, non-adversarial code generation while retaining general model capability.
- Security Neuron Identification: Security neurons are identified using gradient-based attribution on a supervised dataset of secure and insecure code snippets.The security score is based on the expected magnitude of the security-loss gradient with respect to neuron-associated parameters, with selection performed independently per layer.
- Security Optimization: Neuron-selective optimization updates only parameters associated with identified security neurons while freezing the remaining parameters.This preserves pretrained representations for syntax, semantics, and general programming logic.
- Cluster-based Security Optimization: Cluster-based optimization groups security neurons with similar functional roles and assigns each cluster a shared update direction.The resulting trainable-parameter count scales with the number of clusters rather than the number of security neurons.
- Design Distinction: GoodVibe operates at neuron granularity, distinguishing it from parameter-efficient methods that typically constrain optimization at the layer or module level.Structure is imposed only within the security-relevant subspace, combining localized updates with clustered parameterization.
4 Implementation
GoodVibe identifies security-critical neurons through supervised security classification and gradient-based attribution, then evaluates security, efficiency, and utility preservation with an independently trained judge.
- Neuron Security Score Estimation: Security neuron identification formulates security analysis as binary classification using the model’s native token prediction mechanism.The final token is restricted to the vocabulary entries for “0” and “1” and optimized with cross-entropy loss.
- Neuron Security Score Estimation: Gradient hooks on transformer linear layers measure each neuron’s contribution through loss-output gradients.The method aggregates per-sample importance values and selects the top-k neurons in each layer.
- Cluster-Based Fine-Tuning: Cluster-based fine-tuning freezes the model except for selected neurons grouped by similar gradient-importance profiles.K-means clustering provides structured updates while retaining neuron-level selectivity.
- Security Evaluation and Metrics: Security effectiveness is measured as the safe response rate: the proportion of generated samples judged secure by an independently trained model.The judge labels each sample using a fixed binary-output prompt and is never trained on GoodVibe or baseline outputs.
5 Experimental Results
Experiments compare GoodVibe with pretrained, full fine-tuning, LoRA, secure-generation methods, and prompt steering across multiple models and languages. GoodVibe improves security while reducing computation and largely preserving utility.
- Evaluation Setup: Experiments evaluate six open-source models from Meta, Alibaba, and Google, including code-specialized and general-purpose instruction-tuned families.The main benchmark uses C++ and Java, with additional evaluation on Swift and Go.
- Security Performance: 87.5% safe responses on C++ and 76.0% on Java are GoodVibe’s cross-model averages, with consistently strong performance across both languages.Pretrained models show weak and variable security, while full fine-tuning and LoRA can exceed 90% in some settings but vary across models and languages.
- Security Performance: GoodVibe’s selective updates improve generated code security by adding safety checks absent from the base model in a buffer-overflow example.The example contrasts GoodVibe’s checked output with a base output that may overflow the buffer.
- Secure Code Generation Methods: GoodVibe achieves competitive security performance against existing secure code-generation methods under standard autoregressive generation.HexaCoder has the strongest overall average but uses modified inference, whereas GoodVibe uses targeted neuron adaptation without inference-time augmentation.
- Prompt-Based Security Steering: Security-oriented prompting improves baselines, but its gains are smaller and less consistent than GoodVibe’s and can complement model-level alignment.The comparison uses the same benchmark and judge pipeline for baseline and GoodVibe models.
- Securing More Programming Languages: GoodVibe outperforms LoRA on average for both Swift and Go, indicating security-aware adaptation generalizes beyond the main benchmark languages.Pretrained averages are 27.3% safe responses on Swift and 40.1% on Go.
- Utility Preservation: 1.03% average utility drop is observed after GoodVibe fine-tuning across GSM8K, ARC, MMLU, and LiveCodeBench.Reasoning benchmarks show small fluctuations, and the results suggest general code-generation capability is not substantially disrupted.
6 Ablation and Hyperparameter Studies
The ablations show that gradient-based neuron identification is critical for security performance, while clustering trades slight peak performance for substantially better efficiency and stability. Hyperparameter studies identify moderate clustering, neuron coverage, and two training epochs as effective settings.
- Security Neuron Identification: 86.6% safe responses on C++ with gradient-based identification versus 8.7% with activation-based selection on CodeLlama-7B.The results indicate that raw activation is a poor proxy for security relevance, whereas gradient-based attribution captures sensitivity to the security objective.
- Neuron Clustering: 57.1M versus 1.8M trainable parameters for CodeLlama when clustering is disabled versus enabled.Disabling clustering can slightly increase security performance in some settings, but independent neuron updates substantially increase optimization complexity and sensitivity to noise and dataset-specific artifacts.
- Neuron Clustering: Removing either component degrades practical utility: gradient-based identification reduces security performance when removed, while clustering’s removal undermines efficiency gains.Clustering provides structured regularization and low-dimensional adaptation, maintaining performance comparable to full fine-tuning and LoRA under stricter efficiency constraints.
- Hyperparameter Sensitivity: 87.5% average security performance is achieved with a silhouette threshold of 0.05, outperforming thresholds of 0.01 and 0.1.Lower thresholds can force heterogeneous neurons to share update directions, while higher thresholds increase the effective parameter count.
- Hyperparameter Sensitivity: 87.5% average secure code generation is achieved after two epochs, compared with 60.5% after one epoch; three epochs often degrade performance.The results suggest that minimal training may under-propagate security adaptation, while excessive optimization can introduce overfitting or interference.
7 Discussion
The discussion explains that evaluating security in short or context-limited code snippets is difficult, motivating an independently trained LLM judge as a consistent comparative instrument. The paper also frames GoodVibe as targeted internal control that can reinforce security before insecure patterns enter development workflows.
- Evaluation: Short or context-limited code snippets make security evaluation difficult because relevant properties may depend on intent, assumptions, data-flow origins, call sites, or invariants.Static analysis generally requires project context, while dynamic analysis requires executable programs and concrete inputs that are impractical to construct at scale.
- Evaluation: The independently trained LLM judge is treated as a consistent measurement instrument rather than an absolute oracle.It is applied uniformly across experimental conditions, and stable trends across models, languages, and ablations support its use for relative comparisons.
- Broader Implications: GoodVibe targets insecure patterns during generation, before they enter the development pipeline, using fine-grained internal control.The discussion suggests that this targeted adaptation strategy may extend beyond code security to privacy, compliance, and policy adherence.
8 Related Works
Related work shows that code LLMs can generate vulnerabilities despite functional correctness, especially in rapid, low-review workflows. Representation-level interventions influence outputs without parameter changes but generally lack explicit component identification and can vary more than fine-tuning methods.
- Security of Code LLMs: Code LLMs frequently generate security vulnerabilities even when their code is functionally correct, particularly during rapid or informal vibe coding.These workflows prioritize speed and convenience over careful review or explicit security reasoning.
- Representation-Level Interventions: Representation-level interventions such as SVEN manipulate internal activations without modifying parameters.These approaches typically lack explicit mechanisms for identifying internal components responsible for security behavior and often show greater variance than fine-tuning-based methods.
9 Conclusions and Future Work
GoodVibe selectively adapts gradient-identified, security-critical neurons with structured clustering to improve code security while reducing fine-tuning cost. Across six models and four languages, it remains effective under low-security-awareness usage, with stable results across model and language settings.
- Conclusion: GoodVibe improves generated-code security while substantially reducing fine-tuning cost through gradient-based attribution and structured clustering.The pipeline selectively adapts the security-critical subspace without disrupting general coding capabilities.
- Conclusion: Experiments across six models and four programming languages show consistently improved security performance relative to parameter-efficient baselines, using orders of magnitude fewer trainable parameters and lower computational cost.Ablation and hyperparameter studies indicate that the improvements remain stable across different LLMs and programming languages.
- Implications: GoodVibe is positioned as a way to reinforce security-aware default generation when explicit security guidance is absent, particularly in vibe coding.
Ethical Considerations
The work considers stakeholders affected by neuron-level security optimization and frames its benefits, ethical principles, and potential harms around responsible use of AI-assisted code generation.
- Stakeholders: Stakeholders include developers, organizations, end users, security engineers, model providers, researchers, and society at large.The analysis identifies parties who may benefit from reduced security risks or be affected by AI-assisted software development.
- Benefits: The intended benefits span model families, programming languages, and development contexts without targeting or disadvantaging specific user groups.The work uses open-source models so benefits are not limited exclusively to proprietary systems.
- Ethical Principles Considered: The ethical analysis follows Menlo Report principles and aims to reduce security vulnerabilities introduced by generated code.
- Potential Harms: GoodVibe could be repurposed to amplify harmful model behaviors, lower barriers to model manipulation, or encourage overconfidence in generated-code security.The authors emphasize that improved average security does not guarantee vulnerability-free code and that generated code still requires review.
- Mitigations and Responsible Use: The threat model is limited to benign usage and excludes adversarial prompting, jailbreaks, and intentional misuse.The authors also avoid releasing tools that let end users arbitrarily modify model parameters.
Open Science
The paper supports reproducibility through released artifacts and illustrates its neuron-level method with clustering visualizations and a Go code-generation example. The evidence links clustered security neurons to efficient updates and safer generated code.
- Open Science: The released artifacts include code for security-neuron identification, neuron-selective fine-tuning, and evaluation.Experiments use openly released models and publicly available datasets to support independent replication.
- Neuron Clustering: Figure 4 maps neuron importance vertically and cluster membership by color across q_proj, k_proj, and v_proj submodules.The visualization focuses on representative self-attention layers in CodeLlama-7B-Instruct.
- Neuron Clustering: Silhouette scores of 0.70–0.78 indicate clear neuron groups, supporting clustering before fine-tuning.Cluster number and structure differ across projection submodules, suggesting distinct security-related signal organization.
- Generated-Code Example: In the Go example, GoodVibe adds explicit length checks for a fixed-size buffer, while the baseline omits them and may cause a buffer overflow.Both models receive the same prompt without explicit security constraints, approximating a vibe-coding scenario.
- Benchmark Results: Table 12 reports per-model results and shows that effectiveness varies across model families and programming languages.HexaCoder has the strongest average performance, while GoodVibe remains competitive under standard autoregressive generation.
D Evaluation Across Different Judge Models
GoodVibe is evaluated with multiple judge models to test robustness, and the paper also examines how neuron clustering reduces training cost. Across judges and languages, security performance consistently improves over baseline models.
- Evaluation Across Different Judge Models: Qwen3-0.6B, Qwen3-14B, and GPT-5.2-based judges are used to assess robustness of the security findings.
- Evaluation Across Different Judge Models: GoodVibe consistently improves secure code generation over original baselines across all evaluated judge models and programming languages.Absolute security scores vary with judge capability and calibration, but the improvement pattern remains consistent.
- Efficiency Analysis: Neuron clustering shares update directions among similarly important neurons, reducing trainable parameters relative to independent neuron optimization.The method yields a more favorable cost–performance trade-off.