Source-linked AI summary
OCC-RAG: Optimal Cognitive Core for Faithful Question Answering
Maksim Savkin, Mikhail Goncharov, Alexander Gambashidze, Alla Chepurova, Dmitrii Tarasov, Nikita Andriianov, Daria Pugacheva, Vasily Konovalov, Andrey Galichin, Ivan Oseledets
TL;DR
Context QA requires faithful reasoning from supplied passages despite models’ tendency to rely on memorized knowledge. OCC-RAG addresses this with task-specialized small models trained on a large synthetic corpus, and matches or exceeds general-purpose models 2–6× larger across reasoning, faithfulness, and refusal benchmarks.
Problem
Context QA requires models to remain faithful to supplied evidence, yet capable models often favor parametric knowledge and fail on counterfactual, inconsistent, or unanswerable cases.
Method
OCC-RAG mid-trains Qwen3-based small models on synthetic multi-hop QA data using structured reasoning traces and context-grounded source citations.
Results
OCC-RAG models match or exceed general-purpose models 2–6× larger across multi-hop reasoning, faithfulness, and refusal benchmarks.
Takeaways & Limitations
Faithfulness, multi-hop reasoning, and calibrated abstention can be jointly learned in small models through targeted training curricula and supervision formats.
Abstract
from arXiv · showhide
Recent progress in the development of language models has been defined by scale, with each generation absorbing more of the world's knowledge into its weights. However, many practical applications benefit more from robust reasoning than from extensive parametric knowledge. In this setting, task-specialized small language models (SLMs) offer a principled design choice. We introduce Optimal Cognitive Core (OCC), a family of SLMs built around this premise. As a variant of OCC, we present OCC-RAG, optimized for faithful question answering (QA) grounded in the provided context. This task directly aligns with the OCC design approach, requiring multi-hop reasoning over supplied passages while ignoring memorized knowledge. To train OCC-RAG, we implement a novel pipeline for synthesizing multi-context, multi-hop QA data at scale, producing a corpus of over three million examples targeting multi-hop reasoning, strict context faithfulness, and calibrated abstention. We release OCC-RAG-0.6B and OCC-RAG-1.7B, both mid-trained on this corpus. The models produce structured reasoning traces with source citations grounded in literal quotes from the context. Through OCC-RAG, we demonstrate that compact, task-specialized SLMs can match or exceed general-purpose models 2 -- 6x their size across multi-hop reasoning (HotpotQA, MuSiQue, TAT-QA), faithfulness (ConFiQA), and refusal (MuSiQue-Un) benchmarks.
1 Introduction
OCC-RAG targets context QA with compact, task-specialized models designed to reason over supplied evidence while avoiding memorization. The released models use structured, citation-grounded reasoning and outperform or match larger general-purpose models across key benchmarks.
- Context QA requires answers strictly derived from provided passages while avoiding hallucinated content and conflicts with parametric knowledge.
- OCC prioritizes task-specific reasoning capabilities over knowledge capacity, with OCC-RAG designed for faithful context QA.
- OCC-RAG combines multi-hop inference, avoidance of memorization, and safe abstention when the context is insufficient or ambiguous.
- The models produce structured reasoning traces with source citations grounded in literal context quotes.
- 2–6× larger general-purpose models are matched or exceeded across multi-hop reasoning, faithfulness, and refusal benchmarks.OCC-RAG-0.6B and OCC-RAG-1.7B are evaluated on HotpotQA, MuSiQue, TAT-QA, ConFiQA, and MuSiQue-Un.
2 Model Design Principles
OCC-RAG’s design centers on learning context-grounded reasoning rather than relying on model scale or memorized knowledge. Mid-training uses structured traces and insufficient-context examples to jointly support multi-hop inference, faithfulness, and abstention.
- OCC-RAG is designed for multi-hop inference, avoidance of memorization, and safe abstention in context QA.
- Mid-training shapes reasoning through subquestion decomposition, information retrieval, and step-wise verification rather than surface answer patterns.
- Evidence-anchored reasoning traces train models to solve QA from supplied text without memorizing facts or hallucinating.
- Explicitly annotated unanswerable examples teach models to recognize when context does not support a confident answer and abstain appropriately.
3 Training Data
The training corpus is synthesized to teach context-grounded reasoning across increasing difficulty, from single-hop lookups to multi-context multi-hop fusion and refusal. It combines gold and distractor passages with structured reasoning traces and produces approximately 3.25M QA pairs.
- The corpus contains questions, gold context chunks, distractors, structured reasoning traces, and examples where insufficient evidence requires abstention.
- The dataset mixes inexpensive single-hop examples with progressively harder multi-hop single-context and multi-context fusion examples.
- Multi-hop QA Generation: Multi-hop generation samples explicit reasoning paths from knowledge graphs to ensure bridge entities and grounded evidence support.
- Multi-hop QA Generation: Generated multi-hop questions must be self-contained, answerable by literal spans in supplied context, and reachable through the sampled path.
- Reasoning Format: Every QA pair receives an explicit reasoning trace organized into Query Analysis, Source Analysis, Reasoning, and a closing answer section.
- Dataset Statistics: Approximately 3.25M QA pairs include 2.78M single-hop, 262k multi-hop single-context, 165k multi-hop multi-context, and 43k abstain pairs.The corpus uses roughly 8B Qwen3 tokens, with distractors consuming 35%–75% of tokens.
4 Mid-training
OCC-RAG is mid-trained from pretrained Qwen3 base models using supervised response-token fine-tuning and an evaluation-matched prompt format. Multi-hop examples are oversampled to emphasize the target reasoning capability.
- The released checkpoints are mid-trained from Qwen3-0.6B-Base and Qwen3-1.7B-Base rather than pretrained from scratch.
- Supervised fine-tuning applies loss only to response tokens, with questions and randomly ordered, source-tagged context passages in the prompt.
- The training prompt matches evaluation format and the response contains a reasoning trace with the final answer and ANSWERABLE/UNANSWERABLE verdict.
- Multi-hop subsets are oversampled three times per epoch because they exercise the reasoning capability targeted by training.
- Both models are trained on approximately 9 × 10^9 tokens using 8 NVIDIA H100 GPUs.
5 Evaluation
OCC-RAG is evaluated across multi-hop reasoning, faithfulness, and refusal, using diverse benchmarks and comparisons with open-weight model families. Its compact models remain competitive with or surpass larger baselines, especially on faithfulness and refusal.
- Evaluation dimensions: The evaluation covers multi-hop reasoning, context faithfulness, and refusal on question-answering benchmarks.HotpotQA, MuSiQue, and TAT-QA test multi-hop reasoning; ConFiQA tests faithfulness; MuSiQue-Un tests abstention on unanswerable questions.
- Faithfulness: ConFiQA measures counterfactual-context accuracy and Memorization Ratio across QA, MR, and MC subsets.Lower Memorization Ratio indicates stronger adherence to the supplied context rather than parametric knowledge.
- Refusal: MuSiQue-Un measures refusal accuracy when supporting passages contain no extractive answer.The reported metric is the fraction of predictions containing “Not enough information.”
- Baselines: OCC-RAG models are compared with Qwen3, Gemma3, SmolLM3, and Pleias-RAG checkpoints up to 32B parameters.The comparison includes both thinking and non-thinking modes where supported.
- Results: Although 2 −6× smaller, OCC-RAG models achieve competitive or superior performance relative to models up to 4B parameters.OCC-RAG-0.6B exceeds Gemma-3-4B and SmolLM-3-3B on each evaluated dimension, while OCC-RAG-1.7B achieves the highest faithfulness and refusal results in the described comparison.
- Results: OCC-RAG-1.7B reduces Memorization Ratio from 12.7 (8.3) for Qwen3-1.7B to 5.0 and reaches 87.2 R-Acc, matching models of 8B parameters or higher on refusal.Across evaluated scales, OCC-RAG achieves the best faithfulness performance, with highest ConFiQA accuracy and lowest memorization ratio.
6 Conclusion
The paper concludes that compact, specialized models can combine context-grounded reasoning, faithfulness, and abstention efficiently. OCC-RAG therefore offers a practical alternative to relying on scale alone when QA correctness must remain tied to evidence.
- Conclusion: OCC-RAG combines synthetic mid-training, explicit reasoning traces, and citation-aware formatting for faithful context-grounded QA.The released models are designed to answer from supplied context and abstain when evidence is insufficient.
- Conclusion: Faithfulness can be learned through an appropriate training curriculum and supervision format rather than scale alone.The conclusion identifies this as a key takeaway of the work.
- Conclusion: Multi-hop reasoning, context grounding, and calibrated abstention can be jointly trained in small models without sacrificing efficiency.The paper presents this combination as evidence for an optimal cognitive core.
- Conclusion: OCC-RAG provides a reusable recipe for compact QA systems that are transparent, efficient, and robust to missing or conflicting evidence.The authors connect this recipe to structured mid-training, evidence-anchored reasoning traces, and faithful abstention.
A OCC-RAG Prompt/Response Example
Figure 6 illustrates the prompt/response format used during mid-training and evaluation, including tagged context passages and structured reasoning, refusal, and answer outputs.
- The prompt wraps the question in query tokens and each context passage in source tokens with a numeric source identifier.
- The response separates reasoning, status, and answer using delimiter tokens.Reasoning composes evidence from sources 1 and 3 into a three-hop chain, while Status records refusal and Answer carries the final span.
- The example demonstrates multi-hop reasoning across multiple context passages, linking Karen Hayes to 24, Jack Bauer, and CTU.
B Mid-training Hyperparameters
Table 3 presents the mid-training hyperparameters for the released OCC-RAG checkpoints.
- Table 3 reports hyperparameters used during OCC-RAG mid-training.
- The listed settings concern the mid-training stage.
- The hyperparameters apply to the released OCC-RAG checkpoints.