gpt-oss-120b & gpt-oss-20b Model Card

1 Introduction

• The text-only models are Apache 2.0-licensed and designed for agentic workflows, with instruction following, web search and Python use, adjustable reasoning effort, customization, full CoT, and Structured Outputs.They are compatible with the Responses API and were developed with open-source community feedback.Source: §1 · p. 4
• Open-weight release creates distinct safety risks because attackers may fine-tune models to bypass refusals or optimize for harm, while developers and enterprises may need extra safeguards.The models are designed to follow OpenAI’s safety policies by default, but the systems built around them are maintained by diverse stakeholders.Source: §1 · p. 4
• gpt-oss-120b did not reach the Preparedness Framework’s indicative High capability thresholds in Biological and Chemical capability, Cyber capability, or AI Self-Improvement.These were the three Tracked Categories evaluated on the default model.Source: §1 · p. 4
• Even after robust adversarial fine-tuning, gpt-oss-120b did not reach High capability in Biological and Chemical Risk or Cyber risk, according to SAG’s review.The testing simulated potential attacker actions and used OpenAI’s training stack.Source: §1 · p. 4
• For most biological capability evaluations, existing open models’ default performance came near to matching adversarially fine-tuned gpt-oss-120b.The introduction concludes that releasing gpt-oss-120b would not significantly advance the frontier of biological capabilities in open foundation models.Source: §1 · p. 4

2.2 Architecture

• The models are autoregressive MoE transformers: 120b has 36 layers and 116.8B total parameters, while 20b has 24 layers and 20.9B total; both route each token to four experts.The 120b and 20b models have 5.1B and 3.6B active parameters per token, respectively. Their alternating banded-window and dense attention uses 128-token windows and extends dense-layer context to 131,072 tokens.Source: §2 · pp. 4–5, Table 1 · p. 5, §2.2 · p. 5

2.1 Quantization

• Post-training quantizes MoE weights to MXFP4 at 4.25 bits per parameter, allowing 120b to fit on one 80GB GPU and 20b to run with as little as 16GB memory.MoE weights account for more than 90% of total parameters.Source: §2.1 · p. 5

2.6.1 Reasoning, Factuality and Tool Use

• gpt-oss-120b surpasses o3-mini and approaches o4-mini on canonical reasoning, while nearing o4-mini on coding and tool use; 20b is competitive despite being six times smaller.The reasoning comparison covers AIME, GPQA, HLE, and MMLU; coding and tool-use results cover Codeforces, SWE-Bench, and τ-Bench Retail.Source: Figure 1 · p. 6, Figure 2 · p. 7, §2.6.1 · p. 9

2.4 Pretraining

• Pretraining uses trillions of text tokens focused on STEM, coding, and general knowledge, filters harmful content using GPT-4o CBRN filters, and sets a June 2024 knowledge cutoff; CoT reinforcement learning teaches reasoning and tool use.The 120b training run used 2.1 million H100-hours, while 20b required almost 10 times fewer; across training stages, the models use the 201,088-token o200k_harmony BPE tokenizer.Source: §2.4 · p. 6, §2.3 · p. 6, §2.5 · p. 7

2.5.1 Harmony Chat Format

• Harmony delineates roles and message channels to support instruction hierarchy and agentic workflows; models also support low, medium, and high reasoning levels and browsing, Python, and developer-defined tools.Evaluations report pass@1 at high reasoning with the default system prompt and cover reasoning, coding, tool use, multilingual ability, and health knowledge.Source: §2.5.1 · p. 7, §2.5.2 · p. 8, §2.5.3 · p. 8

2.6.1 Reasoning, Factuality and Tool Use

• Across AIME and GPQA, increasing reasoning effort smoothly increases accuracy as average CoT and answer length rises, generally with log-linear returns but greater latency and cost.The authors recommend balancing model size and reasoning level for the intended use case.Source: Figure 3 · p. 8, §2.6.1 · p. 9

2.6.2 Health Performance

• At high reasoning level, gpt-oss-120b nearly matches OpenAI o3 on HealthBench and HealthBench Hard, and significantly outperforms GPT-4o, OpenAI o1, OpenAI o3-mini, and OpenAI o4-mini.The gpt-oss models are not intended for diagnosis or treatment of disease.Source: §2.6.2 · pp. 9–10

2.6.3 Multilingual Performance

• In evaluating multilingual capabilities with MMMLU, a professionally human-translated version of MMLU in 14 languages, gpt-oss-120b at high reasoning comes close to OpenAI o4-mini-high in performance.Answers were parsed by removing extraneous Markdown or LaTeX syntax and searching for translations of “Answer” in the prompted language.Source: §2.6.3 · p. 10

3 Safety testing and mitigation approach

• Post-training uses deliberative alignment to teach refusals for a wide range of content, jailbreak robustness, and adherence to the instruction hierarchy [29, 30].Source: §3 · p. 11
• Open-weight risk assessment should test feasible malicious modifications, including fine-tuning, because downstream actors can both tailor capabilities and enhance harmful ones.Source: §3 · p. 11
• gpt-oss-120b’s default model did not reach indicative High capability thresholds in Biological and Chemical capability, Cyber capability, or AI Self-Improvement.These were the three Tracked Categories in OpenAI’s Preparedness Framework.Source: §3 · pp. 11–12
• Even after robust adversarial fine-tuning, gpt-oss-120b did not reach High capability in Biological and Chemical Risk or Cyber risk.The internal adversarially fine-tuned versions were not released; OpenAI’s Safety Advisory Group reviewed the testing.Source: §3 · p. 12
• On most biology evaluations, another open-weight model scored at or near gpt-oss-120b, so the release was considered unlikely to significantly advance open-model biological capabilities.The comparison evaluated other open foundation models alongside gpt-oss-120b.Source: §3 · p. 12

4.1 Disallowed Content

• Standard disallowed-content evaluations have saturated, motivating a more challenging Production Benchmarks set with more representative, multi-turn conversations.Completions are assessed with LLM-based policy graders using the not_unsafe metric.Source: §4.1 · pp. 12–13
• On standard disallowed-content evaluations, both gpt-oss models score within 1-2 points of OpenAI o4-mini and generally outperform it on the more challenging Production Benchmarks, except gpt-oss-20b underperforms on illicit/violent content.On that Production Benchmarks category, gpt-oss-20b still outperforms GPT-4o. Scores are expected to be lower on Production Benchmarks because the set is designed to be more challenging.Source: §4.1 · p. 13, Table 4 · p. 13, Table 5 · p. 14

4.2 Jailbreaks

• On jailbreak evaluations, gpt-oss-120b and gpt-oss-20b generally perform similarly to OpenAI o4-mini using StrongReject [31] prompts and policy grading.The evaluation inserts known jailbreaks into safety-refusal examples across harm categories.Source: §4.2 · p. 14, Table 6 · p. 14

4.3 Instruction Hierarchy

• Instruction hierarchy training teaches the models to prioritize system over developer messages and developer over user messages, with evaluations testing conflicting instructions and attempts to extract prompts or reveal protected phrases and passwords.The approach is intended to let inference providers and developers control guardrails at their respective levels.Source: §4.3 · p. 15
• Both gpt-oss models generally underperform OpenAI o4-mini on instruction hierarchy evaluations, despite performing at about parity on StrongReject jailbreaks [31].This suggests they are relatively robust to known jailbreaks but less effective at preventing users from overriding system messages; developers can fine-tune them against encountered jailbreaks.Source: Table 7 · p. 15, §4.3 · p. 16, Table 8 · p. 16

4.4 Hallucinated chains of thought

• The models were not directly optimized on chain of thought, preserving developers’ opportunity to use CoT monitoring, but unrestricted reasoning may contain hallucinations or unsafe language.Developers should not show chains of thought directly to users without filtering, moderation, or summarization.Source: §4.4 · p. 16

4.5 Hallucinations

• Without internet browsing, gpt-oss-120b and gpt-oss-20b underperform OpenAI o4-mini on both SimpleQA and PersonQA, and tend to hallucinate more.The paper attributes this to smaller models having less world knowledge and notes that browsing or gathering external information can reduce hallucinations.Source: §4.5 · p. 17, Table 9 · p. 17

4.6 Fairness and Bias

• On the BBQ evaluation [32], both gpt-oss models perform at about parity with OpenAI o4-mini.Source: §4.6 · p. 17, Table 10 · p. 17

5 Preparedness Framework

• The framework tracks and prepares for frontier capabilities that create risks of severe harm.Source: §5 · p. 17
• It commits to mitigating severe-harm risks, including with safeguards that sufficiently minimize risk for highly capable models.Source: §5 · p. 18
• The section provides details about evaluations conducted to inform the assessment.Source: §5 · p. 18

5.1 Adversarial Training

• Pre-training filtered certain harmful CBRN data, while post-training used deliberative alignment and the instruction hierarchy to teach refusal of unsafe prompts and defense against prompt injections.Source: §5.1 · p. 18
• The adversary was assumed to have technical expertise, strong post-training infrastructure, in-domain harmful-capability data, and substantial compute; the study focused on incremental reinforcement learning.Training used OpenAI’s o-series RL stack, and gpt-oss-120b was tested at the highest reasoning setting.Source: §5.1 · p. 18
• Adversarial fine-tuning combined helpful-only RL rewarding compliance with unsafe prompts and domain training: web browsing and expert biorisk data for biology, and capture-the-flag environments for cyber.Source: §5.1 · p. 18
• SAG concluded gpt-oss-120b did not reach High capability in Biological and Chemical Risk or Cyber risk, even after robust fine-tuning using OpenAI’s field-leading training stack.Source: §5.1 · p. 18

5.1.1 External Safety expert feedback on adversarial training methodology

• External reviewers submitted 22 recommendations; the authors acted on 11, including 9 of 12 high-urgency items, adding analyses and improving evaluation and reporting.Changes included more protocol-debugging data, a new uncontaminated protocol-debugging evaluation, an updated virology evaluation, stronger scaffolding tests, and clarifications on assumptions, baselines, refusal behavior, and task-level success rates.Source: §5.1.1 · p. 19

5.2.1 Biological and Chemical - Adversarially Fine-tuned

• gpt-oss-120b showed its greatest capability in biology, but did not significantly advance open-source biosecurity results; all evaluated models could synthesize information across the five threat-creation stages.The reported comparisons found no single open model consistently outperforming the others.Source: §5.2.1 · p. 19, §5.2.1.1 · p. 20

5.2.1.2 Multimodal Troubleshooting Virology

• OpenAI o3 scored highest on 350 held-out virology troubleshooting questions, and all models exceeded the 40% average-human baseline.Browsing results were filtered using a domain block list, and flagged rollouts were reviewed for cheating.Source: §5.2.1.2 · p. 21, Figure 6 · p. 21

5.2.1.3 ProtocolQA Open-Ended; 5.2.1.4 Tacit Knowledge and Troubleshooting

• OpenAI o3 was highest-performing on one benchmark, where all models fell below consensus and median expert baselines of 54% and 42%; on another, none beat the 80% consensus baseline, while all exceeded the 63% 80th-percentile PhD baseline.Source: §5.2.1.3 · p. 21, §5.2.1.4 · p. 22

5.2.1.5 TroubleshootingBench

• OpenAI o3 ranked highest on TroubleshootingBench, but all models scored below the 36.4% 80th-percentile human threshold on its non-public, experience-grounded protocol questions.The benchmark includes 52 expert-reviewed protocols, each paired with three troubleshooting questions.Source: §5.2.1.5 · pp. 22–23, Figure 9 · p. 23

5.2.1.6 Evaluations and Red Teaming by External Safety Experts

• SecureBio found adversarially fine-tuned gpt-oss-120b generally outperformed non-fine-tuned DeepSeek R1-0528, but trailed OpenAI o3 models in reliability and depth; Qwen 3 Thinking and Kimi K2 were competitive.The assessment covered static benchmarks, long-form biodesign, agent-based fragment and screening challenges, and manual red-teaming.Source: §5.2.1.6 · p. 23

5.2.2 Cybersecurity - Adversarially fine-tuned; 5.2.2.1 Capture the Flag (CTF) Challenges

• Cybersecurity results were comparable to OpenAI o3 and below the High capability threshold; o3 led collegiate CTFs, gpt-oss-120b followed, and cybermax scored just 3 percentage points below o3 on another CTF evaluation.The CTF results are likely lower bounds because additional scaffolding or stronger elicitation could raise observed performance.Source: §5.2.2 · p. 23, §5.2.2.1 · p. 25, Figure 10 · p. 25

5.2.2.2 Cyber range

• No model solved any cyber-range scenario, either unaided or with hints.The five scenarios assess end-to-end operations in emulated networks, while the reported primary metric is pass@12 without hints.Source: §5.2.2.2 · pp. 26–27, Figure 12 · p. 27

5.2.3 AI Self-Improvement; 5.2.3.1 SWE-bench Verified

• gpt-oss showed no improved performance on AI self-improvement tasks, where OpenAI o3 and o4-mini remained highest; SWE-bench Verified is a human-validated measure of real-world software issue solving.Source: §5.2.3 · p. 27, §5.2.3.1 · p. 27

5.2.3.1 SWE-bench Verified

• One percentage point separated OpenAI o4-mini from OpenAI o3 on SWE-bench Verified, where all models performed similarly.Evaluation used pass@1 on a fixed subset of n=477 verified tasks; models had to implement changes without seeing the correct tests ahead of time.Source: §5.2.3.1 · p. 28, Figure 14 · p. 28

5.2.3.2 OpenAI PRs

• Two percentage points lower than OpenAI o4-mini, the gpt-oss models replicated internal OpenAI pull-request contributions.Each rollout started from a pre-PR branch, used command-line tools and Python, and was graded by hidden task-specific tests.Source: §5.2.3.2 · p. 29, Figure 15 · p. 29

5.2.3.3 PaperBench

• PaperBench evaluates agents on replicating 20 ICML 2024 Spotlight and Oral papers, with a measured 10-paper subset requiring under 10GB of external data.The benchmark decomposes tasks into individually gradable subtasks; the subset reports pass@1 with high reasoning effort.Source: §5.2.3.3 · p. 29

• The appendix shows a low-reasoning Harmony input with a developer-provided function and an SF weather request, followed by a model response containing CoT and a tool call.Source: Figure 17 · p. 30, Figure 18 · p. 31

7 Appendix 2

• The appendix summarizes feedback on the adversarial testing methodology and the responses to that feedback.Source: §7 · p. 31

1. Clarifying Threat Model and Risk Categorization

• The paper clarifies low-resource actors’ compute, ML expertise, and data-access assumptions, while flagging future cost estimates for follow-up.Source: §1 · p. 31
• The paper clarifies preparedness criteria, retains ProtocolQA as required, and re-runs OpenAI o3 on ProtocolQA with a blocklist for consistency.Source: §1 · p. 31

• ProtocolQA results were checked for refusals, additional protocol-debugging training data, and performance on a similar uncontaminated troubleshooting evaluation.Source: §2 · p. 31
• Performance-scaling plots show how biology and cyber evaluation results vary with the number of trials.Source: §2 · p. 31
• Text-only versions of Multimodal Virology Troubleshooting improve comparability, while VCT results use the final 322-question dataset and include human-baseline comparisons.Source: §2 · p. 31
• Reporting specifies expert profiles and how expert baselines are calculated.Source: §2 · p. 31
• The evaluation separates refusal-based failures from other failure modes and reports refusal rates before and after naughtification.Source: §2 · p. 31

3. Improving Evaluation Setup

• Internal Best of K scaffolding was tested in cyber evaluations, RL datasets were aligned with ProtocolQA, and fine-tuning hyperparameters were checked with internal researchers.Analogous RL datasets were tested to confirm no harmful uplift; hyperparameters were changed when necessary to maximize performance.Source: §3 · pp. 31–32

7.0.2 Recommendations Not Adopted

• The recommendation to apply best-of-N scaffolding broadly was not fully adopted: experiments were conducted elsewhere, and full reruns were expected to yield limited additional gains.Source: §7.0.2 · p. 32
• ProtocolQA was retained despite its limitations because it provided a unique safety signal; removing it would leave a major gap, and broader preparedness-criteria changes were out of scope.Source: §7.0.2 · p. 32
• A proposed closed-model refusal comparison was not pursued because closed models generally did not refuse benign-proxy tasks, except Gryphon, so it would provide little signal about closing gaps on real malicious tasks.The recommendation was to combine performance by substituting non-refusal responses for refusals and treating refusals as zero.Source: §7.0.2 · p. 32