GoodVibe: Security-by-Vibe for LLM-Based Code Generation

• Vibe coding prioritizes speed and convenience over review and often omits security, allowing insecure generated code to reach production and propagate through reused components and dependencies.Source: §1 · p. 1
• Full-parameter fine-tuning is costly and can degrade general coding ability [10] [11], LoRA offers coarse-grained control, and prompting depends on consistent security guidance at deployment.GoodVibe aims to improve security by default without sacrificing coding capabilities or relying solely on explicit prompting.Source: §1 · p. 1
• GoodVibe treats security reasoning as localized to a small neuron subset, identifying security-critical neurons through supervised secure-versus-insecure code evaluation and gradient attribution.A single backward pass estimates neuron-level gradient magnitudes, avoiding the need for explicit semantic labels; optimization then updates only this subspace while freezing other parameters.Source: §1 · p. 2
• Activation-driven clustering groups security neurons with similar functional roles so they can share update directions, reducing training complexity while preserving security adaptation.Source: §1 · p. 2
• GoodVibe achieves up to a 2.5× increase in secure code generation over base models on C++ and Java, with performance competitive with full fine-tuning using over 4 700× fewer trainable parameters.Across six open-source models and four languages, it largely preserves functional correctness and syntactic validity and achieves comparable or stronger performance than LoRA across C++, Java, Swift, and Go at lower computational cost.Source: §1 · p. 2

2.1 Code Large Language Model

• Code LLMs predict the next code token, but security reasoning is not explicitly supervised during pre-training and can be insufficient for informal vibe coding.Security knowledge emerges implicitly from training data, making it important to understand which internal components influence security behavior.Source: §2.1 · p. 2

2.2 Parameter Efficient Fine-Tuning

• PEFT reduces fine-tuning cost by freezing the pretrained backbone and updating restricted parameters through adapters, prefix or prompt tuning, or low-rank adaptation.These approaches modify model behavior at the level of layers or submodules.Source: §2.2 · pp. 2–3
• PEFT retains much of a pretrained model’s general capability while improving task-specific performance, but provides limited visibility into knowledge encoded in individual neurons.The limitation follows from PEFT’s coarse-grained optimization of layers or submodules.Source: §2.2 · p. 3

3.1 Threat Model

• The threat model covers functionally correct but insecure code from benign, security-agnostic prompts; active attacks, malicious-generation requests, and repository-level structural vulnerabilities are out of scope.The assumed model is deployed as-is, with adversarial interaction limited to standard prompting.Source: §3.1 · p. 3

3.2 Design Intuition

• GoodVibe treats security awareness as an internal control on existing coding capability, selectively strengthening neurons that influence security decisions so secure practices are more likely without explicit requests.The method is motivated by the view that security improvements need not relearn coding and that retraining the entire model is unnecessary and potentially harmful.Source: §3.2 · p. 3

3.3 Security Neuron Identification

• GoodVibe identifies security-critical neurons by gradient-based attribution on a security-label prediction task, selecting the top-k neurons independently in each layer.Layerwise selection avoids domination by layers with larger gradient scales and reflects security reasoning across levels of abstraction.Source: §3.3 · p. 4

3.4 Cluster-based Security Optimization

• GoodVibe clusters security-critical neurons and shares update directions within each cluster, reducing trainable parameters and optimization complexity while freezing all non-security parameters.The number of trainable parameters scales with the number of clusters rather than the number of security neurons.Source: §3.4 · p. 5

4.1 Neuron Security Score Estimation

• GoodVibe estimates neuron importance by training the model’s native token predictor to classify code as secure or insecure, then collecting output gradients from transformer linear layers.The final-token logits for ASCII “0” and “1” are optimized with cross-entropy against the security label; hooks capture gradients used to score neurons.Source: §4.1 · p. 5
• Per-sample neuron importance scores are averaged to reduce selection variance, and each layer’s top-k neurons are selected and saved for reuse.Per-sample contributions are accumulated in CPU memory before averaging.Source: §4.1 · p. 6

4.2 Cluster-based Fine-tuning

• Cluster-based fine-tuning freezes the model except selected security-neuron weight rows, with per-layer k-means clustering skipped when silhouette scores fall below a threshold.Training uses secure-code samples for two epochs with AdamW at 1×10−4, cosine scheduling, 0.1 warmup, BF16, gradient checkpointing, and gradient accumulation; biases remain frozen.Source: §4.2 · p. 6

4.3 Security Evaluation and Metrics

• Security effectiveness is measured as the safe response rate: the proportion of generated code samples judged secure by an independently trained Qwen3-0.6B judge [19].The judge is not trained on GoodVibe or baseline outputs; Qwen3-14B and GPT-5.2 judges are also tested, with results in Appendix D.Source: §4.3 · p. 6
• Efficiency tracks trainable parameters, fine-tuning FLOPs, and FLOPs per second, while utility is assessed through ARC Challenge [20], GSM8K [21], MMLU, and LiveCodeBench accuracy.These benchmarks proxy reasoning, language understanding, and coding capability after security optimization.Source: §4.3 · p. 6

5.1 Evaluation Setup

• GoodVibe is evaluated on six models using held-out prompts in C++ and Java, with Swift and Go as additional generalization languages.Neuron identification and fine-tuning use 4,656 aligned secure/insecure code pairs; the default training uses two epochs, top-k 50 neurons per layer, and a 0.05 silhouette score.Source: §5.1 · pp. 6–7, §5.3 · p. 9

5.2 Performance Benchmark

• GoodVibe averages 87.5% safe responses on C++ and 76.0% on Java across models, outperforming LoRA and approaching or sometimes exceeding full fine-tuning.In the illustrated C++ buffer-copy task, GoodVibe adds size checks while the base model lacks them.Source: §5.2 · pp. 7–8, Table 1 · p. 7
• 2.4 PFLOPs for GoodVibe is 46% below full fine-tuning’s 4.5 PFLOPs and over 70% below LoRA’s 8.6 PFLOPs on CodeLlama-7B-Instruct-hf.The paper attributes LoRA’s higher computation to additional low-rank projections in forward and backward passes.Source: Table 3 · p. 8, §5.2 · p. 8
• GoodVibe is competitive with SafeCoder, Secure-Instruct, and HexaCoder on C++ and Java; HexaCoder leads overall but adds two-step inference, unlike GoodVibe’s standard generation.GoodVibe uses neuron-level adaptation without inference-time augmentation.Source: Table 4 · p. 8, §5.2 · pp. 8–9
• A secure-instruction prefix raises baseline security, but gains are smaller and less consistent than GoodVibe’s; prompting can complement model alignment, with additive benefits suggested under shared prompting.Both baseline models and GoodVibe were evaluated with the same security-oriented prefix and judge pipeline.Source: §5.2 · p. 9, Table 5 · p. 9

5.3 Securing More Programming Languages

• On Swift, GoodVibe averages 53.6% safe responses versus LoRA’s 51.9%; on Go, it reaches 54.0% versus 48.9%, showing generalization beyond C++ and Java.Full fine-tuning was omitted from this study to compare lightweight adaptation methods at comparable computational cost.Source: §5.3 · p. 9, Table 6 · p. 9

5.4 Utility Analysis

• After GoodVibe fine-tuning, average utility drops only 1.03%, with no consistent reasoning-benchmark degradation and competitive LiveCodeBench performance.The evaluation covers GSM8K, ARC, MMLU, and LiveCodeBench.Source: §5.4 · pp. 9–10, Figure 3 · p. 10

6.1 Ablation Study

• Gradient-based identification averages 87.5% safe responses on C++ and 76.0% on Java, versus 50.0% and 60.8% with activation-based selection.Replacing identified neurons with randomly selected neurons reduces performance by 24.0% on C++ and 14.9% on Java, supporting gradient attribution over arbitrary sparse adaptation.Source: §6.1 · pp. 10–11, Table 7 · p. 10
• Clustering trades occasional peak security for efficiency and stability: disabling it raises CodeLlama’s trainable parameters from 1.8M to 57.1M.Clustering provides structured regularization and low-dimensional adaptation; without it, gemma-3-12B’s Java safe response rate drops from 84.0% to 61.3%, illustrating reduced generalization in some settings.Source: §6.1 · p. 11, Table 8 · p. 11

6.2 Hyperparameter Study

• A silhouette threshold of 0.05 yields the best average performance (87.5%), balancing expressiveness against parameter efficiency.More aggressive clustering at 0.01 can force heterogeneous neurons to share update directions, while a 0.1 threshold yields sparser clustering and greater noise sensitivity.Source: §6.2 · p. 11, Table 9 · p. 11
• Selecting K=50 security neurons achieves the highest average security rate of 87.5%, compared with 82.9% for K=10; K=100 gives no consistent gains.The results indicate that too few neurons miss distributed security-relevant reasoning, while increasing neuron coverage can raise optimization complexity and overfitting risk.Source: §6.2 · pp. 11–12, Table 10 · p. 12
• Two fine-tuning epochs achieve the best average secure code generation rate of 87.5%, versus 60.5% after one epoch, while three often degrade performance.The study attributes the lack of further gains to potential overfitting or interference with pretrained representations.Source: §6.2 · p. 12, Table 11 · p. 12

Reliability of Automated Judge Models

• Automated code-security evaluation is difficult because snippets often lack semantic context, making static analysis unreliable and dynamic analysis impractical at scale.The judge is applied uniformly across experimental conditions and is treated as a consistent comparative instrument, not an absolute oracle.Source: §7 · p. 12
• False-positive rates fell from over 50% with the base judge to under 16% after SFT, while true-positive rates remained 95% across C++ and Java.The authors describe the fine-tuned judge as more discriminative and use it to support the validity of LLM-based security evaluation.Source: §7 · p. 12

Security Improvements in AI-Assisted Code Generation

• GoodVibe strengthens secure coding behavior during generation, increasing input validation, safer API usage, and conservative control flow even when prompts emphasize functionality alone.The discussion contrasts this internalized behavior with relying solely on prompts, post-processing, or downstream vulnerability scanners.Source: §7 · p. 12
• Because vibe-coding outputs may be reused with minimal modification, GoodVibe targets insecure patterns before they enter development pipelines, complementing codebase vulnerability-analysis tools.The discussion presents fine-grained internal control as a potential adaptation strategy for privacy, compliance, and policy adherence.Source: §7 · pp. 12–13

Weight adaptation

• Security-focused full fine-tuning methods such as SafeCoder [10] and Secure-Instruct [31] reduce vulnerable outputs but cost compute, require parameter access, and risk forgetting [11]; HexaCoder [32] uses PEFT but adapts broader representations.The paper contrasts these broader updates with targeted neuron-level adaptation under standard autoregressive generation.Source: §8 · p. 13

Prompt-based methods

• Prompt-based methods PromSec and SGCode, and in-context SecCoder [42], can improve security without training but depend on prompt phrasing, context length, and consistent user discipline.The paper notes that such external controls may be absent or inconsistently applied when vibe-coding prompts prioritize functionality.Source: §8 · p. 13

Representation-level interventions

• Representation-level work such as SVEN manipulates activations without parameter updates, but often lacks explicit attribution of security-relevant components and can vary more than fine-tuning methods.The passage characterizes this line of work as promising but limited in identifying which internal components drive security behavior.Source: §8 · p. 13

9 Conclusions and Future Work

• Across six models and four programming languages, GoodVibe consistently improves security under low-security-awareness use, with performance competitive with parameter-efficient baselines and orders of magnitude fewer trainable parameters.The improvements remain stable across LLMs and programming languages, and training computation is substantially lower.Source: §9 · p. 13
• GoodVibe uses gradient-based attribution to identify security-critical components, then selectively adapts that subspace with structured clustering to reinforce security-aware reasoning without disrupting general coding capabilities.The pipeline is presented as a fine-grained security optimization approach for code LLMs.Source: §9 · p. 13
• Future work includes integrating GoodVibe with deployment-time safeguards and extending it to other controllable behaviors, including privacy and policy compliance.The paper also suggests selective reinforcement can improve default generation quality when explicit security guidance is absent.Source: §9 · p. 13

Stakeholders

• Affected stakeholders include developers and organizations, end users, security engineers and auditors, model and tool providers, researchers, and society broadly.Source: Ethical Considerations · p. 14

Benefits and ethical principles

• GoodVibe seeks broadly applicable benefits without disadvantaging user groups, uses open-source models, and follows Menlo principles.The research uses publicly available or curated code samples, involves no human subjects or private personal data, and complies with applicable laws and licenses.Source: Ethical Considerations · p. 14

Potential harms

• GoodVibe could be repurposed to amplify harmful behavior, encourage overconfidence in generated-code security, or lower barriers to unanticipated model manipulation.Source: Ethical Considerations · p. 14

Mitigations and oversight

• The authors scope use to benign scenarios, disclaim vulnerability-free guarantees, withhold arbitrary parameter-modification tools, and call for code review, testing, auditing, and human oversight.Source: Ethical Considerations · p. 14

Open Science

• The release provides code for security-neuron identification, selective fine-tuning, and evaluation, with experiments using openly released models and public datasets.These artifacts support independent replication and further research.Source: Open Science · p. 14
• GoodVibe’s pipeline runs on consumer-grade GPUs, lowering the infrastructure barrier for researchers to validate, analyze, and extend the work.Source: Open Science · p. 14

A Neuron Cluster Visualization

• Across query, key, and value projections, security-critical neurons form well-separated clusters with silhouette scores of 0.70–0.78.Figure 4 visualizes neurons by gradient-based importance for security prediction and cluster membership.Source: A Neuron Cluster Visualization · p. 17, Figure 4 · p. 17
• Cluster number and structure vary by projection, suggesting distinct security-related signals and motivating clustering to capture shared neuron roles during fine-tuning.Structured parameter sharing reduces trainable parameters while preserving the expressiveness needed to reinforce secure coding behavior.Source: A Neuron Cluster Visualization · p. 17

B More Code Examples

• In an unconstrained Go buffer-copy prompt, GoodVibe adds length checks that the baseline omits, illustrating security improvements across programming languages.The baseline output may cause a buffer overflow, whereas GoodVibe’s output prevents writing beyond the fixed-size buffer.Source: B More Code Examples · p. 17

C Comparison with Existing Secure Code Generation Methods

• HexaCoder achieves the strongest average performance, while GoodVibe remains competitive under standard autoregressive generation with targeted neuron-level adaptation.Performance varies noticeably across model families and programming languages; HexaCoder’s results may benefit in part from its modified inference setting.Source: C Comparison with Existing Secure Code… · p. 18, Table 12 · p. 18

D Evaluation Across Different Judge Models

• The evaluation compares baseline models and GoodVibe using Qwen3-14B and GPT-5.2-based judges alongside the primary Qwen3-0.6B judge.Generated code was assessed under each judge.Source: D Evaluation Across Different Judge… · p. 18
• Absolute security scores vary across judges, reflecting differences in judge capability and calibration.This variation does not prevent GoodVibe from improving performance across the evaluated judges.Source: D Evaluation Across Different Judge… · p. 18
• GoodVibe consistently improves secure code generation over original baselines across all evaluated judges and programming languages.Table 13 presents the evaluation under different judge models.Source: D Evaluation Across Different Judge… · p. 18, Table 13 · p. 18

E Clustering Efficiency Analysis

• Without clustering, each selected security neuron is optimized independently, resulting in substantially larger trainable parameter counts.Table 14 presents trainable parameter reduction via neuron clustering.Source: E Clustering Efficiency Analysis · p. 18, Table 14 · p. 18
• Clustering shares update directions among neurons with similar importance profiles.This groups neurons rather than optimizing each selected neuron independently.Source: E Clustering Efficiency Analysis · p. 18
• Neuron clustering yields a more favorable cost–performance trade-off.Table 14 quantifies the efficiency benefit of clustering, but the supplied passage gives no specific values.Source: E Clustering Efficiency Analysis · p. 18, Table 14 · p. 18