Source-linked AI summary
Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
Zorik Gekhman, Roee Aharoni, Eran Ofek, Mor Geva, Roi Reichart, Jonathan Herzig
TL;DR
The paper asks why reasoning helps answer simple single-hop factual questions that do not require logical decomposition. Using controlled experiments, it identifies computational buffering and factual priming as mechanisms, while showing that hallucinated intermediate facts increase final-answer hallucinations and that trajectory selection can improve accuracy.
Problem
The effect of reasoning on simple single-hop factual questions remains unclear because these questions do not require step-by-step logical decomposition.
Method
The paper uses hypothesis-driven controlled experiments to separate content-independent computation from content-dependent factual priming in reasoning traces.
Results
Reasoning expands parametric recall through computational buffering and factual priming, while hallucinated intermediate facts increase the likelihood of hallucinated final answers.
Takeaways & Limitations
Prioritizing reasoning trajectories that surface factual statements while avoiding hallucinated intermediate facts can improve factual accuracy.
Takeaways & Limitations
Generative self-retrieval is fragile because model-generated intermediate facts may be hallucinated and shape the final answer.
Abstract
from arXiv · showhide
While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model's parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.
1. Introduction
The introduction shows that reasoning substantially expands parametric factual recall even for simple questions, then identifies computational buffering and factual priming as complementary mechanisms. It also highlights hallucination risks and an inference-time strategy for improving factual reliability.
- Reasoning substantially expands the model’s parametric knowledge boundary, unlocking correct answers otherwise effectively unreachable.
- Controlled experiments isolate a computational buffer effect in which generated reasoning tokens support latent computation independently of semantic content.Filler reasoning traces provide evidence for this content-independent effect.
- Factual priming enables generative self-retrieval by constructing contextual bridges from related facts to the correct answer.Conditioning reasoning-disabled runs on extracted facts recovers most reasoning-related pass@k gains.
- Hallucinated intermediate facts substantially increase the likelihood of hallucinated final answers, making generative self-retrieval powerful but fragile.
- Prioritizing reasoning trajectories containing hallucination-free factual statements can improve model accuracy at inference time.
2. Setup
The setup uses hybrid models, closed-book QA datasets, and pass@k to compare reasoning ON versus OFF while controlling for parametric knowledge. Figure 1 summarizes these comparisons across models and benchmarks.
- Hybrid models toggle reasoning ON or OFF, isolating reasoning’s effect while controlling for the model’s parametric knowledge.Reasoning ON generates a trace before the final response; OFF suppresses it.
- Figure 1 compares pass@k curves across two closed-book QA benchmarks and three LLMs under reasoning OFF versus ON.
- The evaluation uses SimpleQA and EntityQuestions, balancing realistic questions with template-based control over phrasing difficulty.
- Pass@k estimates the probability that at least one of k sampled answers is correct.Predictions are graded against ground truth using an autorater and unbiased estimation.
3. Reasoning Expands The Model’s Parametric Knowledge Boundary
Reasoning expands the model’s parametric knowledge boundary by making correct answers accessible at higher sampling budgets, especially when baseline recall is weak. The weighted effectiveness metric Ω summarizes this ON-versus-OFF advantage while emphasizing larger k values.
- The capability-boundary analysis asks whether reasoning discovers otherwise unreachable correct answers or merely improves sampling efficiency for already-likely answers.The metric emphasizes successful reasoning paths in the output distribution and is less sensitive to their exact ranking.
- Reasoning consistently increases pass@k across models and datasets, with gains often widening at higher k and nearly doubling for Qwen3-32B on SimpleQA-Verified.This pattern suggests reasoning exposes latent knowledge that is not reliably generated without it.
- Ω measures the relative pass@k improvement of reasoning ON over reasoning OFF across k=1 to N, weighting larger k values more heavily.The study uses N=100 samples and compares the full pass@k range rather than only top-1 accuracy.
- Ω decreases as model capability increases, indicating that less capable models benefit more from reasoning because they recall parametric knowledge less effectively without it.The high Ω of Qwen3-32B is interpreted as evidence of more hidden knowledge and greater compensation from reasoning.
- Ω is consistently higher on SimpleQA than EntityQuestions because SimpleQA has lower reasoning-OFF performance and therefore more headroom for recall compensation.The difference is attributed to the greater difficulty of fetching the targeted facts from model parameters.
- Figure 2 orders models from highest to lowest pass@1 effectiveness while displaying Ω across models and datasets.The ordering provides a capability context for interpreting how reasoning effectiveness varies with baseline model performance.
4. Question Complexity is a Poor Predictor of Reasoning Effectiveness
The study finds no evidence that reasoning produces larger gains on questions labeled complex than on simple questions. This weakens task decomposition as the primary explanation for reasoning benefits in these factual-recall settings.
- EntityQuestions uses predefined single-hop templates, while 90% of SimpleQA-Verified questions are not labeled as requiring reasoning or being multi-step.These datasets therefore predominantly contain direct factual questions rather than multi-hop reasoning tasks.
- SimpleQA-Verified labels questions by whether they require reasoning or multiple steps, defining Complex questions when either label is true.Questions with neither label are classified as Simple.
- Overlapping 95% confidence intervals provide no evidence that reasoning effectiveness is higher for Complex than Simple questions.For Gemini-2.5-Pro, the Complex-subset interval even crosses zero.
5. How Reasoning Improves Parametric Recall?
Controlled experiments identify two complementary mechanisms behind reasoning’s benefit for parametric recall: semantic-free computation from extra tokens and semantic factual priming. Hallucinated intermediate facts create a reliability risk, while selecting factual, hallucination-free traces can improve accuracy.
- Mechanisms: The study tests computational buffering and factual priming as explanations for reasoning gains on parametric knowledge recall.The experiments isolate reasoning effects with controlled trace and fact substitutions.
- Computational Buffer: Meaningless reasoning traces substantially improve recall, showing that R-LLMs can use extra tokens for latent computation independent of trace semantics.ON Dummy replaces the original trace with repeated filler tokens, while ON Single Dummy controls for ON/OFF mode bias.
- Computational Buffer: Dummy-trace length has a non-monotonic effect: increasing length initially helps, but performance saturates or can decline beyond a task-dependent point.On SimpleQA-Verified, pass@k improves up to 2048 tokens before showing diminishing or adverse effects.
- Factual Priming: Factual priming occurs when generated related facts provide contextual bridges that facilitate answer retrieval, even when reasoning is disabled.Fact-conditioned baselines outperform semantically matched dummy variants, and ON Facts can match ON reasoning on EntityQuestions with less compute.
- Hallucination Risk: Reasoning traces containing hallucinated intermediate facts are substantially more likely to produce hallucinated final answers.The study verifies intermediate facts across sampled trajectories and compares clean with hallucinated traces within questions.
- From Analysis to Practice: Selecting traces that recall facts improves relative accuracy by 8.2% on SimpleQA-Verified and 2.6% on EntityQuestions, rising to 12.2% and 5.1% when facts are verified correct.The selection strategy prioritizes traces with factual statements and no hallucinated intermediate facts.
6. Case Studies
The case studies illustrate the two mechanisms with concrete retrieval failures: dummy computation can recover an inaccessible answer, while related recalled facts can prime the correct answer.
- Computational Buffer: A dummy reasoning trace changes Gemini-2.5-Flash’s answer to Mary Engle Pennington’s induction year from incorrect “2019” to correct “2018”.A length-matched dummy trace succeeds, whereas a short single dummy does not.
- Computational Buffer: The Mary Engle Pennington trace contains no apparent useful information beyond restating the question and planning search, supporting an additional-compute explanation.The length-matched dummy reproduces the benefit without semantic content.
- Factual Priming: For the tenth King of Nepal, reasoning changes the prediction from incorrect “Jitari Malla” to correct “Birendra Bir Bikram Shah Dev”.The reasoning trace lists related facts about the kings rather than directly relying on the answer statement.
- Factual Priming: Providing the first nine kings as facts enables correct recall, whereas equal-length dummy facts fail and produce the seventh king’s name.The answer-disclosing fact is removed, so the case demonstrates priming from related facts rather than direct answer leakage.
7. Related Work
Related work frames this paper’s contribution around reasoning’s role in capability boundaries, computational buffering, and the content of effective reasoning traces.
- Reasoning in Closed-Book QA: Prior closed-book QA work mainly studies reasoning for complex multi-step questions, leaving the contribution of knowledge recall less clear.The paper examines whether reasoning gains persist when questions do not require decomposition.
- Capability Boundaries: Capability-boundary studies typically compare non-reasoning models with reasoning-tuned variants and report gains concentrated at small k.Those gains are interpreted as probability sharpening for answers already accessible to the model.
- Capability Boundaries: This paper instead studies reasoning’s effect on parametric recall using hybrid models with reasoning toggled ON or OFF.The design controls for the model’s parametric knowledge while isolating reasoning mode.
- Computational Buffer: Prior computational-buffer studies found mixed evidence, while later work trained models to exploit filler tokens or observed omitted and non-faithful reasoning steps.The paper tests whether modern reasoning models implicitly use reasoning tokens as computational buffers for factual recall.
- Effective Reasoning Content: Other work correlates reasoning content or length with correctness through behaviors such as review, verification, backtracking, and revision.This paper focuses specifically on trace properties relevant to parametric knowledge retrieval.
8. Conclusion
The paper finds that reasoning expands parametric knowledge recall even for simple factual questions, through computational buffering and factual priming, while hallucinated intermediate facts harm final-answer correctness.
- Reasoning increases pass@k at large k, unlocking correct answers that are effectively unreachable without it.
- The gains primarily reflect improved parametric recall rather than multi-hop question decomposition.
- A computational buffer uses additional reasoning tokens for latent computation, even when the tokens lack semantic content.
- Factual priming recalls topically related facts that provide a semantic bridge to the correct answer.
- Hallucinated intermediate facts systematically reduce final-answer correctness, motivating selection of factual, hallucination-free trajectories.
A.1. EntityQuestions Relations
The EntityQuestions analysis focuses on four relations that are both difficult to guess and well defined, using corresponding question templates.
- The “Hard to Guess” criterion indicates that the possible answer space is large, making default guesses less reliable.Person names are given as an example of hard-to-guess answers, unlike professions.
- The “Well Defined” criterion assesses whether the entity type and answer granularity are unambiguous.
- The study selects four relations meeting both criteria and presents them in Table 2.
- Table 2 pairs the selected EntityQuestions relations with their corresponding question templates.
A.2. Inference Parameters
Inference settings differ by model and reasoning mode, while Gemini-2.5-Flash uses identical sampling parameters for thinking ON and OFF.
- Gemini-2.5-Flash uses a maximum output token limit of 32,768 when reasoning is ON.
- Gemini-2.5-Flash uses temperature T=1.0 and top-p=0.95 for both thinking modes.
- Qwen3-32B uses different temperature and top-p settings for thinking ON versus thinking OFF, while keeping top-k=20 and min-p=0.Thinking ON uses T=0.6 and top-p=0.95; thinking OFF uses T=0.7 and top-p=0.8.
A.3. Simulating the Expected Accuracy With Test Time Selection
The test-time selection procedure extracts, filters, parses, and verifies facts from reasoning traces, then compares accuracy under subsets containing facts and verified-correct facts.
- The procedure compares all samples with subsets containing recalled facts and subsets whose recalled facts are all correct.
- Fact extraction uses a dedicated prompt, with Gemini-2.5-Pro used because this stage is critical to the analysis.
- Extracted facts that restate the question are removed before evaluating reasoning-derived information.
- Statements explicitly linking the target answer to the question are removed, while mere mentions of the answer are retained.
- The conservative baseline removes every fact containing the answer and underperforms OFF Facts, but still outperforms the dummy baseline.
- Each extracted fact is parsed for iteration and checked with search-enabled Gemini-2.5-Flash; manual checks found all 20 sampled labels accurate.
- OFF Facts supplies the facts list as context without reasoning, whereas ON Facts overrides the thought with that list before answer prediction.