PaperlayerView exact revision on arXiv

Source-linked AI summary

SIFT: Grounding LLM Reasoning in Contexts via Stickers

Zihao Zeng, Xuyao Huang, Boxiu Li, Zhijie Deng

arXiv:2502.14922v1cs.CLcs.AI

TL;DR

LLMs can misinterpret or overlook query facts, leaving them to reason about the wrong problem. SIFT uses model-generated Stickers and iterative refinement to ground reasoning in context. It reports consistent gains across models and benchmarks, including a 7.34% accuracy improvement for DeepSeek-R1 on AIME2024.

  • Problem

    LLMs may misinterpret, overlook, or hallucinate query information, creating factual drift even when their reasoning steps are logically correct.

    Source: §1 · pp. 1–2

  • Method

    SIFT uses model-generated Stickers to summarize query facts, compares Sticker-only and query-plus-Sticker predictions, and refines the Sticker when they differ.

    Source: §1 · p. 2

  • Results

    SIFT consistently improves reasoning across LLMs and benchmarks, including a 7.34% accuracy improvement for DeepSeek-R1 on AIME2024.

    Source: §1 · p. 2, §4.1 · p. 6

  • Takeaways & Limitations

    SIFT offers a practical way to anchor LLM reasoning to contextual facts without additional data or training.

    Source: §5 · p. 9

    Further studies are needed to establish SIFT’s effectiveness across a wider range of tasks.

    Source: Limitations · p. 9

Abstract

from arXiv · show

This paper identifies the misinterpretation of the context can be a significant issue during the reasoning process of large language models, spanning from smaller models like Llama3.2-3B-Instruct to cutting-edge ones like DeepSeek-R1. For example, in the phrase "10 dollars per kilo," LLMs might not recognize that "per" means "for each," leading to calculation errors. We introduce a novel, post-training approach called **Stick to the Facts (SIFT)** to tackle this. SIFT leverages increasing inference-time compute to ground LLM reasoning in contexts. At the core of SIFT lies the *Sticker*, which is generated by the model itself to explicitly emphasize the key information within the context. Given the curated Sticker, SIFT generates two predictions -- one from the original query and one from the query augmented with the Sticker. If they differ, the Sticker is sequentially refined via *forward* optimization (to better align the extracted facts with the query) and *inverse* generation (to conform with the model's inherent tendencies) for more faithful reasoning outcomes. Studies across diverse models (from 3B to 100B+) and benchmarks (e.g., GSM8K, MATH-500) reveal consistent performance improvements. Notably, SIFT improves the pass@1 accuracy of DeepSeek-R1 on AIME2024 from 78.33% to **85.67**%, establishing a new state-of-the-art in the open-source community. The code is available at https://github.com/zhijie-group/SIFT.

1 Introduction

LLMs can misread, overlook, or hallucinate query facts, so correct reasoning steps may still address the wrong problem. SIFT uses model-generated Stickers and iterative refinement to ground reasoning in context, with reported gains across models and benchmarks.

1 Introduction

  • Factual drift occurs when models misinterpret, overlook, or hallucinate query information, even when their reasoning steps are logically correct.For example, a model may read “10 dollars per kilo” as a total rather than a per-unit amount.Source: §1 · p. 1
  • Self-verification can help models correct context misunderstandings, but it is stochastic and not guaranteed to activate across reasoning scenarios.The paper points to Figure 1 when noting that the remaining risk can be significant.Source: §1 · pp. 1–2
  • SIFT summarizes query facts, essential conditions, and the core question into a model-generated Sticker, then compares predictions from the Sticker and the Sticker-augmented query.When predictions differ, it refines the Sticker through forward optimization toward the query and inverse generation toward the model’s reasoning preferences.Source: §1 · p. 2
  • SIFT improves reasoning across diverse LLMs and benchmarks, including a 7.34% accuracy improvement for DeepSeek-R1 on AIME2024.The paper also reports a 1.03% accuracy improvement over vanilla CoT (97.3%) on MATH-500.Source: §1 · p. 2

2 Related Work

Prior work improves reasoning through training, search and planning, or inference-time techniques. The paper distinguishes SIFT by prioritizing factual comprehension before answer generation.

2 Related Work

  • Reasoning methods described in this section include training-based alignment, search and planning, and inference-time augmentation.Examples include SFT and RL, MCTS-based approaches, Chain-of-Thought, and Self-Consistency.Source: §2 · pp. 2–3
  • Unlike approaches that primarily refine how models reason, SIFT prioritizes ensuring they address the correct problem through factual comprehension.The paper frames this as factual comprehension before answer generation.Source: §2 · p. 3

3 Method

The method addresses factual drift, in which reasoning fails because a model misunderstands the query context. SIFT uses prediction agreement and iterative Sticker refinement to align extracted facts with both the query and the model’s reasoning preferences.

3.1 Factual Drift in LLM Reasoning

  • Factual drift is a context-comprehension failure: models can neglect constraints, misread semantic relationships, or hallucinate conditions.Even a correctly generated Sticker may be misunderstood, particularly with complex questions or unfamiliar phrasing; self-verification in advanced models can help but is not guaranteed.Source: §3.1 · pp. 3–4

3.2 Stick to the Facts (SIFT)

  • SIFT has four operations: Sticker Generation, Consensus Prediction, Forward Optimization, and Inverse Generation.Sticker Generation extracts a Sticker; Consensus Prediction checks alignment between Sticker-only and query-plus-Sticker predictions; Forward Optimization aligns the Sticker to query facts; Inverse Generation derives a Sticker from the model prediction.Source: §3.2 · p. 4
  • Consensus Prediction checks whether Sticker-only and query-plus-Sticker predictions agree, treating agreement as a necessary but insufficient condition for correctness.It defaults to greedy decoding and checks the current Sticker without using historical reasoning traces; it tests semantic invariance rather than answer aggregation across diverse reasoning paths.Source: §3.2 · p. 5
  • Forward Optimization combines the query with the Sticker to refine it toward the source facts, while Inverse Generation infers a Sticker from the model prediction and is followed by Forward Optimization.The inverse step aims to fit the model’s preferences for representing facts, arranging conditions, or structuring questions.Source: §3.2 · p. 5

4 Experiments

Across diverse models and reasoning benchmarks, SIFT improves over Zero-shot CoT; iterative Sticker refinement and consistency-based variants further improve performance or efficiency. Experiments also examine how the stages affect prediction alignment and performance.

4.1 Enhancing LLM Reasoning with SIFT

  • SIFT improves over Zero-shot CoT across settings, including a 1.03% absolute accuracy improvement over vanilla CoT’s 97.3% on MATH-500 and a 7.34% absolute increase on AIME2024.The evaluation spans models from several billion to hundreds of billions of parameters, across dense and MoE architectures and reasoning paradigms.Source: §4.1 · p. 6, Figure 6 · p. 7

4.2 Iterative Optimization

  • Repeated optimization on GSM8K improves performance as average tokens per sample increase; Stage 2 saturates rapidly, while Stage 3 adds a noticeable boost.The largest gains occur on the first repeat; the authors suggest harder Sticker extraction may require more repeats in more complex conditions.Source: §4.2 · p. 6, Figure 7 · p. 7

4.3 Sample Augmentation

  • Across tested integrations with Self-Consistency, SIFT-Consistency provides the largest accuracy boost, and the figure comparison reports it outperforming standard Self-Consistency across sampling iterations.The test uses Llama3.2-3B-Instruct on GSM8K; Table 1 compares integration strategies across stages.Source: §4.3 · p. 7, Table 1 · p. 8, Figure 10 · p. 9

Evolution of Consensus Across Optimization

  • The proportion of correct, matching predictions from Sticker-only and Query + Sticker rises by 6.14% from Stage 1 to 2 and another 4.85% from Stage 2 to 3.The reported comparison attributes improved alignment to Forward Optimization and Inverse Generation.Source: Figure 8 · p. 8, Evolution of Consensus Across… · p. 8
  • Skipping Stage 2 causes a performance drop on GSM8K, while the tested Consensus Prediction strategy is reported as optimal among the compared strategies.The authors describe Forward Optimization as aligning the Sticker with the query before aligning it with model cognition.Source: Evolution of Consensus Across… · pp. 8–9, Table 2 · p. 8, Table 3 · p. 8
  • SIFT achieves comparable accuracy to Self-Consistency while using two-thirds as many output tokens, although its total token use fluctuates with formatting and example inputs.The figure distinguishes output-token counts from total-token counts, which include additional Sticker-generation inputs.Source: Evolution of Consensus Across… · p. 9, Figure 9 · p. 9

5 Conclusion

SIFT is a training-free framework that anchors LLM reasoning to contextual facts through iterative self-refinement.

5 Conclusion

  • SIFT anchors LLM reasoning to contextual facts through iterative self-refinement, without additional data or training.The paper presents this as a practical approach to factually grounded reasoning.Source: §5 · p. 9

Limitations

The work focuses on training-free SIFT and calls for further study across a wider range of tasks.

Limitations

  • The evaluation of SIFT’s effectiveness across a wider range of tasks remains for future study.The authors also identify internalizing SIFT into small LLMs through dedicated training as future work.Source: Limitations · p. 9

A More Results

The section reports DeepSeek-R1 performance as average token count increases.

A More Results

  • The section examines how DeepSeek-R1 performance evolves with increasing average token count.Source: A More Results · p. 12

B Prompting for SIFT

This section presents the prompt formats used in the SIFT process, including formats for prediction, Sticker generation, and Sticker forward optimization.

B Prompting for SIFT

  • The section presents the complete prompt formats used in the SIFT process.It directs readers to Figures 12 to 15 for details.Source: B Prompting for SIFT · p. 12
  • Figure 13 gives the prompt format for generating predictions.Source: Figure 13 · p. 12
  • Figure 14 gives the prompt format for generating a Sticker from the query.Source: Figure 14 · p. 12
  • Figure 15 gives the prompt format for forward optimization of the Sticker.Source: Figure 15 · p. 13

Related papers

Loading 2502.14922v1…