PaperlayerView exact revision on arXiv

Source-linked AI summary

Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information

Lukas Struppek, Dominik Hintersdorf, Hannah Struppek, Daniel Neider, Kristian Kersting

arXiv:2511.22176v1cs.CLcs.AI

TL;DR

Long chain-of-thought traces make LLM reasoning costly, while existing efficiency approaches often target the model through training or brevity instructions. F-CoT instead separates extraction from reasoning and supplies structured context; the paper reports 2–3x fewer generated tokens while preserving reasoning performance. The results present structured input as a training-free alternative for efficient reasoning.

  • Problem

    Long chain-of-thought traces increase inference time and computation, while existing token-reduction approaches typically intervene through training, fine-tuning, or explicit brevity instructions.

    Source: §1 · p. 1

  • Method

    F-CoT separates information extraction and structuring from reasoning, then prompts the model to reason over only the compact context.

    Source: §1 · p. 2

  • Results

    2–3x fewer generated tokens while preserving reasoning performance, according to the paper's conclusion.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Structured input representation offers a training-free alternative to model-centric optimization for efficient LLM deployment.

    Source: §6 · p. 10

    Smaller models, particularly the 0.6B variant, struggle to generate their own context, so a larger model may be needed to pre-compute it.

    Source: §5.1 · p. 9

Abstract

from arXiv · show

Recent large language models achieve strong reasoning performance by generating detailed chain-of-thought traces, but this often leads to excessive token use and high inference latency. Existing efficiency approaches typically focus on model-centric interventions, such as reinforcement learning or supervised fine-tuning, to reduce verbosity. In contrast, we propose a training-free, input-centric approach. Inspired by cognitive psychology, we introduce Focused Chain-of-Thought (F-CoT), which separates information extraction from the reasoning process. F-CoT first organizes the essential information from a query into a concise, structured context and then guides the model to reason exclusively over this context. By preventing attention to irrelevant details, F-CoT naturally produces shorter reasoning paths. On arithmetic word problems, F-CoT reduces generated tokens by 2-3x while maintaining accuracy comparable to standard zero-shot CoT. These results highlight structured input as a simple yet effective lever for more efficient LLM reasoning.

1 INTRODUCTION

Long chain-of-thought traces make reasoning costly in time and computation, and can be difficult to check for errors. F-CoT separates information structuring from reasoning, using a compact context to reduce generated tokens while preserving reasoning performance.

1 INTRODUCTION

  • Long chain-of-thought traces cost time and computation, and make errors challenging to locate because the full trace typically must be checked.Source: §1 · p. 1
  • The paper hypothesizes that intertwining information structuring with reasoning blurs relevant and irrelevant details, complicating reasoning and contributing to unnecessary tokens.Source: §1 · p. 1
  • F-CoT separates information extraction and structuring from reasoning, then prompts the model to reason over only the compact structured context.The approach is training-free and does not instruct or fine-tune the model to shorten its reasoning.Source: §1 · p. 2
  • 2–3x faster inference than standard chain-of-thought prompting while preserving strong reasoning performance is reported for F-CoT.Figure 1 illustrates the extraction-then-reasoning workflow and reports 43% fewer tokens for Qwen3 14B in its example.Source: §1 · p. 2, Figure 1 · p. 2

2 REASONING IN LARGE LANGUAGE MODELS

The section describes chain-of-thought and related inference strategies, then contrasts approaches that extend reasoning with methods intended to reduce its token cost. F-CoT instead structures input information to elicit shorter traces without training or explicit brevity instructions.

Prompt-Based Reasoning

  • Chain-of-thought prompting elicits intermediate reasoning steps, while tree-of-thought, self-consistency, and ReAct explore multiple paths before selecting or combining outcomes.Source: §2 · p. 2

Dedicated Reasoning LLMs

  • Dedicated reasoning LLMs use a reasoning stage followed by a concise summary, yet can produce thousands of reasoning tokens for relatively simple questions.Their high parameter counts and lengthy reasoning increase inference time and computational cost.Source: §2 · pp. 2–3

Test-Time Scaling

  • Test-time scaling extends or parallelizes reasoning to improve accuracy, whereas reducing generated tokens to save resources has received comparatively less attention.Examples of scaling techniques include reconsideration tokens and self-refinement steps.Source: §2 · p. 3

Efficient Reasoning

  • Existing efficiency methods reduce tokens through reinforcement learning, data processing or fine-tuning, or training-free instructions to keep reasoning concise.Source: §2 · p. 3
  • F-CoT targets input representation rather than the model, using compact structured information to shorten traces without additional instructions or fine-tuning.Unlike schema-based prompting, it structures inputs rather than outputs; unlike retrieval-augmented generation, it self-extracts information instead of querying an external database.Source: §2 · p. 3

3 FROM NATURAL LANGUAGE TO STRUCTURED REPRESENTATIONS

The section motivates replacing verbose natural-language inputs with compact representations that isolate relevant facts. It introduces a structured facts-and-question format and describes user-preformatted and two-step prompting pipelines for reasoning over that context.

3 FROM NATURAL LANGUAGE TO STRUCTURED REPRESENTATIONS

  • Natural-language math problems mix essential facts with irrelevant wording, which can lead to long, verbose reasoning traces.The example describes prices for apples and pears and asks for the cost of buying two of each.Source: §3 · p. 3
  • A compact representation isolates relevant details from full sentences, which the authors hypothesize can shorten LLM reasoning paths and reduce distraction.Source: §3 · p. 4

3.1 FROM TEXT TO STRUCTURED FACTS

  • The fixed context format separates enumerated information blocks from a question field that specifies the objective.Its XML-like syntax supports automatic parsing, syntax validation, and compatibility with downstream pipelines.Source: §3.1 · p. 4
  • Simpler enumerated lists yield comparable performance, suggesting the benefit comes from structuring information rather than the specific XML-like syntax.Source: §3.1 · p. 4

3.2 IMPLEMENTING STRUCTURED REASONING PIPELINES

  • Users can pre-format context themselves, reducing model workload but requiring extra effort; alternatively, an LLM can extract context before reasoning.Source: §3.2 · p. 4
  • For less capable LLMs, a larger model can generate context for a smaller reasoning model, combining extraction capability with lower reasoning cost.The paper's pipeline uses two steps and provides only extracted context, not the original question, during reasoning.Source: §3.2 · p. 4

4 EXPERIMENTS

Experiments compare F-CoT with zero-shot CoT across arithmetic benchmarks and Qwen3 model sizes, varying context source and design. They also examine reasoning traces, context generation, and sensitivity to prompt choices.

4.2 BOOSTING REASONING EFFICIENCY WITH PRE-COMPUTED CONTEXT

  • SVAMP token counts fell to roughly one-third of 0-CoT across model sizes, while GSM-Hard and MATH-500 counts fell by about half; Qwen3 32B used roughly 2.3k versus 4.4k tokens on MATH-500 at the same Pass@5.Accuracy was comparable across the three datasets except for slightly lower performance with Qwen3 0.6B; larger models outperformed 0-CoT on GSM-Hard.Source: Figure 2 · p. 6, §4.2 · p. 6

4.3 CONTEXT GENERATION AND REASONING WITHIN A SINGLE MODEL

  • Fewer than 2% of Qwen3 0.6B generations produced valid context, while context generation was stable for Qwen3 14B and larger and token savings remained substantial.The authors suggest precomputing context with a larger, slower model and using a smaller, faster model for token-intensive reasoning.Source: §4.3 · p. 7

4.4 QUANTIFYING AND CHARACTERIZING REASONING DYNAMICS

  • The overthinking score fell from 2.35 ± 1.5 with 0-shot CoT to 1.74±1.4 with context; it decreased in 50.2% of samples and was unchanged in 38.4%.The adapted 0–10 score measures emphasis on abstract deliberation versus concrete mathematical problem-solving.Source: §4.4 · p. 7
  • Reasoning sentences fell by more than half and filler sentences declined with F-CoT, while extraction-sentence counts and the relative token distribution across categories remained largely unchanged.The authors attribute a slight extraction-token increase to models explicitly referencing information blocks during reasoning.Source: Figure 3 · p. 8, §4.4 · p. 8

4.5 ABLATION STUDY AND SENSITIVITY ANALYSIS

  • Removing explicit instructions on using context raised average token count to about 3.1k (+747 versus default F-CoT) and slightly increased Pass@5 to 99.2% (+0.6pp).With the same prompt but standard questions instead of compact context, token count was 4.2k (−685 versus default 0-CoT), leaving a gap of more than 1k tokens.Source: §4.5 · p. 8
  • Enumerated-list context showed no Pass@5 or token-count difference; unnumbered lists and unstructured concatenation kept performance unchanged but increased token count, with numbering reducing it further.The authors conclude that the exact representation is not crucial, while added structure such as numbering can reduce token use.Source: §4.5 · pp. 8–9
  • Providing both context and the original question raised Pass@5 by 0.8 percentage points and token count by 19%, yet still used approximately 41% fewer tokens than 0-CoT.The authors identify this option as potentially useful when phrasing is unclear or interpretation may be ambiguous.Source: §4.5 · p. 9
  • Context quality improved with generator size but gains saturated at 14B; models of 4B or larger consistently generated valid XML-like context, while the 1.7B model did so about 95% of the time.Despite usually invalid strict XML from the 0.6B model, the 14B reasoner achieved Pass@5 of 41.2%, so perfect XML validity was not strictly necessary for correct reasoning.Source: §4.5 · p. 9

5 DISCUSSION

The discussion notes that extraction errors, misinterpretation, condensed inputs, and smaller models can constrain F-CoT. It proposes combining structured inputs with other methods, extending the approach to multimodal models, and using context as a dynamic notepad.

5.1 LIMITATIONS

  • Occasional extraction loss can omit answer-format instructions, yielding decimal instead of percentage outputs and potentially harming math-benchmark performance.The paper notes that such omissions may affect details critical to interpreting the input.Source: §5.1 · p. 9
  • Misinterpreted extracted information can affect downstream reasoning, although the structured context makes such issues identifiable and correctable.Source: §5.1 · p. 9
  • Context transformation does not improve correctness on highly condensed AIME questions and can degrade it; smaller models, particularly the 0.6B variant, struggle with self-generated extraction.Using a larger model to pre-compute context may achieve speedups without significantly harming the smaller model’s capabilities.Source: §5.1 · p. 9

5.2 FUTURE DIRECTIONS

  • Combining structured inputs with advanced prompting or test-time scaling, including tree-of-thoughts, is proposed as a future direction.Source: §5.2 · p. 10
  • The authors propose exploring structured information extraction for multimodal models and incorporating structured reasoning into model training or fine-tuning.They note that evaluated models were not trained with structured inputs, which may limit their native capacity to process them.Source: §5.2 · p. 10
  • A dynamic notepad could update structured context incrementally, but would likely require fine-tuning and recomputing cached key–value states after updates.Source: §5.2 · p. 10

6 CONCLUSION

This section summarizes F-CoT’s structured-context approach and its reported token reduction while preserving reasoning performance. It also specifies the prompts, context formats, labeling categories, and rubric for assessing overthinking in mathematical reasoning traces.

6 CONCLUSION

  • F-CoT reduces generated tokens by 2–3x while preserving reasoning performance by guiding models to reason over structured context rather than raw natural language.The conclusion attributes the reduction to less filler and fewer redundant reasoning steps.Source: §6 · p. 10

A.1 CONTEXT EXTRACTION WITH LLMS WITHOUT EXAMPLES

  • The context-extraction prompt asks for concise, relevant facts and a restated question, while explicitly forbidding solving or calculating.It also directs the model to retain given quantities and mathematical relations and ignore irrelevant background.Source: §A.1 · p. 14

A.2 CONTEXT GENERATION WITH GPT-5-MINI

  • In the experiment, Listing 2's prompt instructed the model to extract context information from GSM-hard and SVAMP data; Listing 3's prompt analogously targeted MATH-500 and AIME data.The prompts request directly relevant, concise information and a minimal restatement of the question, without solving the problem or performing calculations.Source: §A.2 · pp. 15–16

A.3 0-SHOT CHAIN-OF-THOUGHT PROMPTS

  • The 0-shot CoT prompts ask the model to reason step by step, including a shortened variant that limits reasoning to facts in the question.Both prompts direct the model to put its final answer in a box.Source: §A.3 · p. 18

A.4 BASELINE PROMPTS

  • The baseline prompts include Plan and Solve, which requests a complete plan and calculations, and CoRe, which asks for known conditions and the final goal.Both prompts also request step-by-step reasoning and a boxed final answer.Source: §A.4 · p. 18

A.5 F-COT REASONING WITH CONTEXT BLOCKS

  • The standard F-CoT reasoning prompt restricts the model to structured context, asks it to cite used information entries, and requires step-by-step reasoning.It directs the model not to reference or repeat the original question or outside information.Source: §A.5 · p. 19
  • The reasoning prompt directs models to use only structured-context facts, consult the original question only if context is unclear, show step-by-step work, cite information used, and box the answer.Source: §A.5 · p. 20

A.6 CONTEXT DESIGNS

  • The default context uses XML-like tags, while sensitivity-analysis alternatives use enumerated lists, unnumbered lists, or concatenated information.Source: §A.6 · p. 21

A.7 SENTENCE TYPE ANNOTATION

  • Sentence annotation distinguishes extraction from reasoning and filler: direct input restatement, inference or computation, and generic phrases without either.Source: §A.7 · p. 22

A.8 OVERTHINKING SCORE

  • The overthinking rubric scores solutions from 0–10 and flags analysis paralysis, rogue reasoning, and premature disengagement, contrasting focused problem-solving with extended unproductive reflection.Source: §A.8 · p. 23
  • The example assigns a score of 10 to a response that concludes without derivation or verification.The preceding example response says the answer is probably 2 or 3 because it looks symmetric.Source: §A.8 · p. 24
  • The prompt asks whether the model reflects excessively instead of reasoning concretely toward a mathematical solution.It requests a 0–10 overthinking score and a one-line explanation in JSON, wrapped in <answer> tags.Source: §A.8 · pp. 24–25

B.1 GSM-HARD RESULTS

The illustrated Qwen3 14B GSM-Hard output is shorter with F-CoT than with 0-CoT, while both traces grapple with an implausibly large muffin-egg quantity. Their differing answers show how the example’s ambiguous number complicates reasoning.

B.1 GSM-HARD RESULTS

  • 0-CoT questions whether 4,933,828 is a typo and ultimately gives $16.13 while expressing uncertainty.The trace considers alternative interpretations of the number.Source: §B.1 · p. 26
  • 6,401 tokens for 0-CoT versus 2,820 for F-CoT in the illustrated Qwen3 14B GSM-Hard outputs.The listings identify the outputs as shortened excerpts.Source: §B.1 · pp. 26–27
  • F-CoT also questions the 4,933,828 figure, but proceeds with the supplied numbers and reaches a negative answer.Its trace notes that the quantity exceeds the 16 eggs available per day.Source: §B.1 · p. 27

C.1 EXPERIMENTS WITH GPT-5 MINI PRE-COMPUTED CONTEXT INFORMATION

This section presents a five-benchmark comparison of reasoning models using GPT-5 Mini-precomputed context. It compares F-CoT with standard zero-shot CoT and two other prompting strategies.

C.1 EXPERIMENTS WITH GPT-5 MINI PRE-COMPUTED CONTEXT INFORMATION

  • The evaluated prompting strategies are F-CoT, standard zero-shot CoT, Plan-and-Solve, and CoRe.For F-CoT, the context is supplied before the reasoning step.Source: §C.1 · p. 28
  • GPT-5 Mini pre-computes the structured context used in the F-CoT comparison.This setup provides the reasoning models with context before they answer.Source: §C.1 · p. 28
  • The comparison covers reasoning-model performance on five benchmarks using context pre-computed by GPT-5 Mini.The table caption specifies its benchmark scope and context source.Source: Table 1 · p. 28, §C.1 · p. 28

C.2 SELF-COMPUTED CONTEXT INFORMATION

This section reports reasoning-model performance across five benchmarks when models compute their own context. The comparison concerns F-CoT with self-generated context versus F-CoT with GPT-5 Mini-precomputed context.

C.2 SELF-COMPUTED CONTEXT INFORMATION

  • The section compares F-CoT with self-generated context against F-CoT with context pre-computed by GPT-5 Mini.The two context sources define the comparison.Source: §C.2 · p. 28
  • The table covers reasoning-model performance on five benchmarks using context computed by the models themselves.The caption identifies the evaluated setting and benchmark scope.Source: Table 2 · p. 28
  • In the self-computed setting, the models generate the context used for their reasoning.This distinguishes the setting from one in which GPT-5 Mini supplies the context.Source: Table 2 · p. 28, §C.2 · p. 28

C.3 ABLATION STUDY AND SENSITIVITY ANALYSIS

The ablations examine prompt choices, context representations, in-context examples, and model size. Results indicate that F-CoT generally retains token savings across variations, while extraction quality can constrain smaller models.

C.3 ABLATION STUDY AND SENSITIVITY ANALYSIS

  • Alternative context representations produce similar accuracy, while prompt changes or including the original question can slightly improve accuracy with minimal efficiency impact.The tested representations include XML, lists, and concatenated text.Source: §C.3 · p. 29
  • Few-shot examples have negligible impact on Pass@5 and token efficiency, but slightly reduce the proportion of valid context blocks.This result concerns self-generated context extraction with Qwen3 14B.Source: §C.3 · p. 29
  • Smaller models, especially 0.6B, struggle to generate valid contexts, whereas larger models generally achieve high Pass@5 scores.The analysis reports that context quality and reasoning performance improve with model size, with gains saturating at 14B.Source: §C.3 · p. 29, Table 5 · p. 30
  • F-CoT retains its efficiency advantages across the examined prompt variations, context formats, and model scales.The overall sensitivity analysis reports robustness across these conditions.Source: §C.3 · p. 29

Related papers

Loading 2511.22176v1…