PaperlayerView exact revision on arXiv

Source-linked AI summary

Llama-3.1-FoundationAI-SecurityLLM-Reasoning-8B Technical Report

Zhuoran Yang, Ed Li, Jianliang He, Aman Priyanshu, Baturay Saglam, Paul Kassianik, Sajana Weerawardhena, Anu Vellore, Blaine Nelson, Neusha Javidnia, Arthur Goldblatt, Fraser Burch, Avi Zohary, Assaf Eisenman, Mahdi Sabbaghi, Supriti Vijay, Rahim Dharssi, Dhruv Kedia, Kojin Oshiba, Yaron Singer, Amin Karbasi

arXiv:2601.21051v1cs.AIcs.CRcs.LG

TL;DR

Cybersecurity requires transparent, verifiable reasoning for complex, high-stakes analysis, yet existing models often rely on direct instruction following. The paper introduces Foundation-Sec-8B-Reasoning, trained with SFT and RLVR on cybersecurity, instruction-following, and mathematical reasoning data. It reports competitive specialized-task performance with strong general capabilities, while identifying reasoning and formatting failure modes addressed during training.

  • Problem

    Existing instruction-following models often struggle with the intricate, multi-step, transparent reasoning required for cybersecurity analysis and high-stakes decisions.

  • Method

    The model uses a two-stage pipeline combining SFT and RLVR with proprietary reasoning data spanning cybersecurity analysis, instruction following, and mathematical reasoning.

  • Results

    The model delivers competitive cybersecurity performance with significantly larger models while maintaining strong general capabilities, including 75.3% on CTIBench-RCM and 62.6% on AlpacaEval 2.0.

  • Takeaways & Limitations

    Domain-specialized reasoning models can achieve strong specialized-task performance while retaining broad general capabilities and effective multi-hop reasoning generalization.

  • Takeaways & Limitations

    SFT alone establishes domain knowledge but leaves substantial instruction-following and complex reasoning weaknesses, including 24.4% on 2WikiMultihopQA and 9.6% on HotpotQA.

Abstract

from arXiv · show

We present Foundation-Sec-8B-Reasoning, the first open-source native reasoning model for cybersecurity. Built upon our previously released Foundation-Sec-8B base model (derived from Llama-3.1-8B-Base), the model is trained through a two-stage process combining supervised fine-tuning (SFT) and reinforcement learning from verifiable rewards (RLVR). Our training leverages proprietary reasoning data spanning cybersecurity analysis, instruction-following, and mathematical reasoning. Evaluation across 10 cybersecurity benchmarks and 10 general-purpose benchmarks demonstrates performance competitive with significantly larger models on cybersecurity tasks while maintaining strong general capabilities. The model shows effective generalization on multi-hop reasoning tasks and strong safety performance when deployed with appropriate system prompts and guardrails. This work demonstrates that domain-specialized reasoning models can achieve strong performance on specialized tasks while maintaining broad general capabilities. We release the model publicly at https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Reasoning.

1. Introduction

Native reasoning models have advanced complex-benchmark performance, but cybersecurity still needs transparent, verifiable multi-step analysis. Foundation-Sec-8B-Reasoning addresses this gap with an 8B cybersecurity model trained to reason before answering.

  • Motivation: Cybersecurity tasks require transparent, verifiable reasoning because correct conclusions alone are insufficient for high-stakes decisions.Relevant functions include threat intelligence analysis, vulnerability assessment, and incident response.
  • Contribution: Foundation-Sec-8B-Reasoning is an 8B-parameter native reasoning model engineered for cybersecurity and built on the specialized Foundation-Sec-8B model.Foundation-Sec-8B was derived from Llama-3.1-8B-Base through continued pre-training on proprietary cybersecurity-focused data.
  • Approach: The model is trained to generate explicit reasoning traces before final outputs through a two-stage SFT and RLVR post-training pipeline.This establishes a “think before you speak” approach rather than directly generating answers like the instruction-tuned predecessor.
  • Results: Evaluations show substantial gains over Foundation-Sec-8B-Instruct, competitive cybersecurity performance with Llama-3.3-70B-Instruct, and comparable or better general reasoning performance.The authors describe the model as the first open-source native reasoning model explicitly architected for cybersecurity.

2. Related Work

Prior work includes instruction-tuned cybersecurity models and open-source reasoning models, but this paper adapts the established SFT-plus-RL paradigm specifically to cybersecurity’s high-stakes analytical demands.

  • Cybersecurity LLMs: Cybersecurity language models have commonly focused on secure code generation, vulnerability detection, or instruction-following assistance across security workflows.Examples include Lily-Cybersecurity-7B-v0.2, DeepHat-V1-7B, Primus, and Foundation-Sec-8B-Instruct.
  • Open-Source Reasoning Models: Recent open-source reasoning models use multi-stage training that combines supervised fine-tuning with reinforcement learning to improve complex reasoning.Examples include DeepSeek-R1, Nemotron, and GPT-OSS, which employ different RL or architectural strategies.
  • This Work: Foundation-Sec-8B-Reasoning adopts and specializes the established SFT-plus-RL methodology for cybersecurity’s distinct high-stakes challenges.Its goal is broader alignment of a general-purpose LLM with core cybersecurity tasks through instruction-following training.

3. Methodology

The methodology uses diverse supervised data to establish native reasoning, followed by verifier-based reinforcement learning focused on cybersecurity, instruction following, and mathematics. Training addresses long-response bias and reward hacking to preserve useful reasoning traces.

  • Base Model: The pipeline starts from Foundation-Sec-8B, a cybersecurity-specialized model continuously pre-trained on 8 billion cybersecurity-focused tokens.The base model originated from Llama-3.1-8B-Base.
  • SFT Dataset and Training: SFT trains explicit <think>...</think> reasoning traces on about two million diverse exemplars covering cybersecurity, mathematics, coding, and instruction following.Cybersecurity data exceeds one quarter of the dataset, while mathematics and coding comprise approximately one third together.
  • RL Training: The RL stage uses instruction-following, cybersecurity, and mathematical reasoning data to refine reasoning accuracy with verifiable rewards.The training uses GRPO, five sampled responses per prompt, task-specific binary verification, and KL regularization against the SFT checkpoint.
  • RL Challenges: Heterogeneous RL data creates loss-aggregation problems because long, low-quality responses can disproportionately influence policy updates.Weak-task outputs may contain gibberish and repetitive text, allowing undesirable failure modes to dominate optimization.
  • RL Challenges: Token-mean aggregation frequently produces degenerate failure modes, while alternative strategies prevent long, low-quality sequences from biasing optimization.The passage attributes the problem to differing output lengths and solving accuracies across tasks.
  • Reward Hacking and Format Degradation: Naive verifier-based RL can reward correct final answers despite empty or nonsensical reasoning traces because verifiers check validity rather than the reasoning process.The resulting reward hacking may also omit the required <think>...</think> tags, motivating a format penalty.

4. Evaluation Results

The evaluation spans cybersecurity benchmarks, general-purpose capabilities, safety, and an ablation of the SFT and RL stages, using multiple model groups and repeated trials. Foundation-Sec-8B-Reasoning performs strongly across cybersecurity tasks, including results competitive with much larger models.

  • Evaluation setup: The evaluation covers 10 cybersecurity benchmarks spanning threat intelligence, vulnerability classification, and broad security knowledge.The benchmark suite includes CTIBench tasks, CTI-Reasoning, CWE-Prediction, MMLU-Security, CyberMetric-2000, SecBench, and SecEval.
  • Evaluation setup: The study compares Foundation-Sec-8B-Reasoning with 18 baselines grouped by model size, specialization, training paradigm, and commercial status.The comparison includes smaller specialized models, Llama-family and cybersecurity-specialized 8B models, GPT-OSS models, frontier commercial models, and the reasoning model.
  • Evaluation setup: Evaluation uses five independent trials, with accuracy computed from extracted final answers; CTIBench-VSP reports average CVSS and CTIBench-ATE reports micro-F1.Sampling temperatures and top-p settings differ between instruct, reasoning, and GPT-OSS models.
  • Cybersecurity results: 75.3% on CTIBench-RCM is the model’s strongest highlighted result, outperforming all evaluated models on that benchmark.The model also achieves 69.1% on CTIBench-MCQA, 70.4% on CWE-Prediction, 41.1% on CTI-Reasoning, 78.2% on MMLU-Security, and 84.8% on SecEval.
  • Cross-benchmark analysis: Figure 3 places Foundation-Sec-8B-Reasoning among the top performers, especially on reasoning-intensive tasks, while larger models excel on knowledge-based benchmarks.The figure also indicates that cybersecurity-specialized models remain strong despite having significantly fewer parameters.
  • Cybersecurity results: 69.1% versus 69.2% on CTIBench-MCQA makes the model comparable to Llama-3.3-70B-Instruct, while 75.3% versus 68.4% on CTIBench-RCM exceeds it.On CTIBench-RCM, it also outperforms GPT-OSS-120B, 75.3% versus 71.2%, despite having 15× fewer parameters.
  • Cybersecurity results: The model outperforms Foundation-Sec-8B-Instruct on 8 out of 10 cybersecurity benchmarks, including gains of +13.3 pp on CTIBench-ATE and +8.7 pp on CWE-Prediction.It also outperforms Llama-3.1-8B-Instruct on 8 out of 10 benchmarks, with gains of +35.9 pp on CTIBench-ATE and +23.1 pp on CWE-Prediction.

4.2. General-Purpose Benchmarks

Foundation-Sec-8B-Reasoning maintains strong general-purpose capabilities across instruction following, reasoning, knowledge, mathematics, coding, and long-form question answering. It matches or exceeds comparable baselines on most benchmarks, with especially strong results on preference alignment and multi-hop reasoning.

  • Overall comparison: Foundation-Sec-8B-Reasoning matches or exceeds Llama-3.1-8B-Instruct across most general-purpose benchmarks.It achieves comparable performance on GSM8K and HumanEval while improving substantially on AlpacaEval 2.0 and 2WikiMultihopQA.
  • Instruction Following and Human Alignment: 62.6% on AlpacaEval 2.0 exceeds Foundation-Sec-8B-Instruct (33.1%) and Llama-3.1-8B-Instruct (25.4%).This benchmark measures length-controlled win rates against reference models.
  • Instruction Following and Human Alignment: 83.7% on IFEval remains close to Foundation-Sec-8B-Instruct (86.1%) and Llama-3.1-8B-Instruct (86.2%).The result indicates retained instruction-following capability despite the shift toward extended reasoning.
  • Reasoning and Knowledge: 69.9% on BBH exceeds Foundation-Sec-8B-Instruct (66.7%) and Llama-3.1-8B-Instruct (67.4%).The model also achieves 31.7% on GPQA and 68.3% on MMLU, remaining competitive across knowledge benchmarks.
  • Mathematical Reasoning and Coding: 82.3% on GSM8K and 43.3% on MATH show strong mathematical performance, while HumanEval reaches 79.9%.HumanEval is 2.9% below both Foundation-Sec-8B-Instruct and Llama-3.1-8B-Instruct, each at 82.3%.
  • Long-Form Question Answering: 60.5% on 2WikiMultihopQA exceeds Llama-3.1-8B-Instruct (49.6%), Foundation-Sec-8B-Instruct (45.4%), and Phi-4 (28.4%).HotpotQA performance is 54.8%, comparable to Llama-3.1-8B-Instruct at 54.1%.

4.3. HarmBench Evaluation

HarmBench results show that Foundation-Sec-8B-Reasoning’s safety depends strongly on deployment configuration. An appropriate system prompt substantially improves refusal performance, and adding Llama-Guard-3-8B improves it further.

  • System Prompt Effects: Without a system prompt, Foundation-Sec-8B-Reasoning achieves a 54.25% HarmBench pass rate.The evaluation measures the percentage of harmful prompts appropriately refused.
  • System Prompt Effects: 93.00% with an adapted system prompt brings Foundation-Sec-8B-Reasoning close to Foundation-Sec-8B-Instruct’s 95.00%.The system prompt was adapted from Foundation-Sec-8B-Instruct to accommodate extended reasoning.
  • Deployment Recommendations: The authors recommend additional safety layers such as automated content filtering or LLM-based moderation for deployment.They also recommend preserving core safety guidelines and domain focus when customizing prompts.
  • Defense in Depth: 98.25% with Llama-Guard-3-8B protection provides the highest reported HarmBench pass rate.Llama-Guard-3-8B analyzes user inputs and model outputs to flag or block policy-violating content.

4.4. Discussion

The ablation study separates the contributions of SFT and RL training. SFT establishes cybersecurity knowledge, while RL produces the largest gains on extended reasoning and instruction-following tasks, with generally small trade-offs elsewhere.

  • Role of Supervised Fine-Tuning: The SFT checkpoint achieves 68.4% on CTIBench-MCQA, 69.5% on CTIBench-RCM, and 85.3% on CTIBench-VSP.These results indicate that SFT establishes core cybersecurity knowledge and instruction-following capabilities.
  • Role of Supervised Fine-Tuning: 24.4% on 2WikiMultihopQA and 9.6% on HotpotQA show the SFT checkpoint’s limitations on multi-hop question answering.The authors attribute these low scores primarily to insufficient instruction-following ability and motivate subsequent RL training.
  • Impact of Reinforcement Learning: +5.8 pp on CTIBench-RCM, +9.7 pp on CTIBench-ATE, and +6.0 pp on CWE-Prediction are reported after RL training.These are improvements on reasoning-intensive cybersecurity tasks.
  • Direct and Indirect Benefits: RL training improves IFEval by +12.7 pp and MATH by +10.2 pp while producing direct gains on cybersecurity tasks.The RL data spans cybersecurity, instruction-following, and mathematical reasoning examples.
  • Generalization Beyond Training Tasks: +36.1 pp on 2WikiMultihopQA and +45.1 pp on HotpotQA demonstrate large gains on multi-hop tasks not explicitly covered in the RL data.BBH improves by +13.7 pp and AlpacaEval 2 by +6.3 pp, supporting transfer to diverse analytical tasks.
  • Maintained Performance with Minor Trade-offs: HumanEval decreases by 2.4 pp after RL, while MMLU-Security changes by -2.4 pp and SecEval improves by +0.5 pp.The authors describe these as minor trade-offs relative to the larger reasoning gains.

5. Conclusion

Foundation-Sec-8B-Reasoning is presented as an open-source native reasoning model specialized for cybersecurity. Its two-stage SFT-plus-RLVR training yields strong cybersecurity performance, broad general capabilities, multi-hop generalization, and improved safety with deployment safeguards.

  • Contribution: Foundation-Sec-8B-Reasoning is presented as the first open-source native reasoning model specifically designed for cybersecurity.It is trained directly as a reasoning model rather than adapted from an instruction-following model.
  • Training Method: The two-stage method combines SFT with RLVR using proprietary data for cybersecurity analysis, instruction following, and mathematical reasoning.KL divergence regularization and relatively small-scale RL limit deviation from the SFT initialization.
  • Evaluation Results: 75.3% on CTIBench-RCM and 62.6% on AlpacaEval 2.0 show strong specialized and general-purpose performance.The cybersecurity result is reported as outperforming 15× larger GPT-OSS-120B.
  • Evaluation Results: +36.1 pp on 2WikiMultihopQA and +45.1 pp on HotpotQA demonstrate substantial generalization to multi-hop reasoning.These gains are reported as indirect benefits of RL training beyond the specific training tasks.
  • Safety: 93.00% on HarmBench with system prompts and 98.25% with Llama-Guard-3-8B protection support safeguarded deployment.The reported safety results depend on system-level instructions and an additional guard model.

A. Appendix

The appendix documents the evaluation benchmarks and provides the HarmBench system prompt used for Foundation-Sec-8B-Reasoning.

  • The appendix describes all benchmarks in the evaluation suite, including two proprietary benchmarks and complete prompting strategies.
  • Appendix A.3 provides the system prompt used for Foundation-Sec-8B-Reasoning’s HarmBench safety evaluation.

A.1. Proprietary Benchmarks: CTI-Reasoning and CWE Prediction

The appendix defines CTI-Reasoning as a deep cybersecurity reasoning benchmark and CWE Prediction as a vulnerability-classification benchmark testing generalization to recent, unseen descriptions.

  • CTI-Reasoning: CTI-Reasoning contains 200 expert-curated questions derived from CWE and CAPEC documentation.96% are reasoning-intensive, with 77.5% requiring analysis-based processing and 22.5% requiring comprehension-based reasoning.
  • CTI-Reasoning: CTI-Reasoning tests multi-hop logical analysis, complex technical-document comprehension, and relationships within cybersecurity taxonomies.
  • Examples: The appendix includes representative CTI-Reasoning and CWE Prediction questions covering CAPEC attack scenarios and stored cross-site scripting.
  • CWE Prediction: CWE Prediction maps vulnerability descriptions to CWE identifiers using recent CVE and GHSA entries that are new to the model.The design tests the same vulnerability-classification ability as CTIBench-RCM while evaluating generalization to previously unseen descriptions.

A.2. Prompts for Cybersecurity Benchmark Evaluation

The cybersecurity evaluation suite uses standardized answer extraction and task-specific scoring across threat intelligence, vulnerability analysis, security knowledge, and reasoning benchmarks.

  • Evaluation protocol: Cybersecurity benchmark answers are extracted from the response’s final line using regular expressions for consistent automated accuracy computation.
  • CTIBench-VSP: CTIBench-VSP evaluates predicted CVSS v3.1 vectors using Score = 1 −|CVSSpred −CVSStrue|/10.The score compares the predicted and ground-truth CVSS numerical severity values, which range from 0 to 10.
  • CTIBench-ATE: CTIBench-ATE requires extracting adversary tactics, techniques, and procedures from threat reports and mapping them to MITRE ATT&CK technique identifiers.
  • Proprietary benchmarks: CTI-Reasoning evaluates multi-hop cybersecurity reasoning through candidate-answer questions requiring the final answer in a specified format.
  • Classification and knowledge: CWE Prediction maps vulnerability descriptions to one of 263 CWE weakness types, while MMLU-Security assesses knowledge across core security domains.
  • General security benchmarks: CyberMetric-2000, CTIBench-MCQA, SecBench, and SecEval provide broader multiple-choice coverage of cybersecurity concepts and operations.

A.3. System Prompts

The system prompt frames Foundation-Sec-8B-Reasoning as a cybersecurity specialist, specifies accuracy and safety expectations, and substantially improves HarmBench safety performance when present.

  • Prompt design: The system prompt positions the model as a cybersecurity reasoning tool emphasizing domain expertise, accuracy, professional context, and responsible use.
  • Safety evaluation: 93.00% pass rate with an appropriate system prompt compares with 54.25% without one in HarmBench evaluation.
  • Precision requirements: The prompt requires precise handling of security identifiers and cloud configurations, with source citation and explanation of logic.
  • Deployment customization: Organizations can customize the prompt for internal policies, operational focus, workflows, and organization-specific restrictions.
  • Deployment customization: The authors recommend retaining the core safety guidelines and domain focus during customization.
Loading 2601.21051v1…