PaperlayerView exact revision on arXiv

Source-linked AI summary

A Four-Stage Decomposition of Word-Problem Solving and Mechanistic Fragility in LLM Math Reasoning

Zhongdi Qu, Carla P. Gomes

arXiv:2609.17804v1cs.AIcs.LG

TL;DR

The paper addresses the coexistence of orderly math-problem reasoning and sharp NoOp fragility by mapping a four-stage internal pipeline and using it to diagnose distraction. It localizes the failure to Operation Planning and validates the implicated attention heads bidirectionally.

  • Problem

    The paper asks how an orderly account of math word-problem reasoning can coexist with accuracy collapse when an irrelevant NoOp clause is inserted.

    Source: §1 · p. 1

  • Method

    The authors identify four residual-stream stages and use the pipeline scaffold with activation patching, per-head attribution, and bidirectional attention scaling to diagnose NoOp fragility.

    Source: §1 · p. 1, §6 · pp. 7–8

  • Results

    NoOp-induced failure localizes to Operation Planning and its attention heads, while a single global scalar recovers 0.51 of NoOp failures.

    Source: §1 · p. 1

  • Takeaways & Limitations

    Operation Planning is identified as a training-free lever for NoOp failure, while a targeted deployable repair remains future work.

    Source: §1 · p. 1, Limitations · p. 10

    The analysis centers on math word problems and GSM-style tasks; transfer results are probe-level replications rather than the full methods battery.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Large language models solve grade-school math word problems with high accuracy, yet a single irrelevant clause inserted into the problem can collapse it. We reconcile these observations with a mechanistic account. We show that the model's internal computation decomposes into a four-stage sequential pipeline, Schema Abstraction, Operation Planning, Operand Binding, and Computation, each stage producing a distinct intermediate representation in an identifiable band of layers. Using the same scaffold to diagnose distractor-induced failure, we localize the corruption to a single stage, Operation Planning, implemented by a set of attention heads whose causal role we validate bidirectionally. In short, we provide a mechanistic interpretation of math word problem reasoning in LLMs, and their failure when distracted.

1 Introduction

The paper reconciles an orderly multi-stage account of math word-problem solving with NoOp fragility by locating distractor-induced failure in Operation Planning and its attention heads.

1 Introduction

  • The proposed pipeline comprises Schema Abstraction, Operation Planning, Operand Binding, and Computation, each producing a distinct residual-stream representation.Three independent methods converge on the same four-stage decomposition.Source: §1 · p. 1
  • NoOp-induced failure localizes to Operation Planning, where bidirectionally validated attention-head interventions recover a substantial fraction of failures and disrupting either subset damages clean reasoning.The identified heads split into subsets that write toward and against the wrong operation plan.Source: §1 · p. 1
  • A single global scalar recovers 0.51 of NoOp failures, providing a training-free lever at the localized failure site.The authors identify this as a location where future repair can act.Source: §1 · p. 1

2 Background and Related Work

Prior work documents NoOp fragility and examines individual math-solving skills, while this paper investigates how a complete reasoning pipeline is organized and where distraction-related failure occurs.

2.1 Fragility of LLM math reasoning

  • Mirzadeh et al. (2025) report accuracy drops of up to 65% with irrelevant NoOp clauses, but behavioral studies do not localize the failure inside model computation.They argue the degradation reflects pattern matching rather than genuine reasoning.Source: §2.1 · p. 2

2.2 Mechanistic accounts of LLM math

  • Earlier mechanistic accounts examine individual skills, while Cheng et al. (2025) identify a two-stage pipeline limited to synthetic one-step direct prompting; this work studies multi-step GSM8K reasoning in direct and CoT regimes and localizes failure modules.The paper presents the four-stage account as extending that prior pipeline and addressing its scope limitations.Source: §2.2 · p. 2

2.3 Mechanistic methods

3 The Four-Stage Hypothesis

The paper hypothesizes that solving math word problems involves sequential transformations into distinct intermediate representations, and asks which stage breaks under distraction.

3 The Four-Stage Hypothesis

  • The four hypothesized stages transform a problem from stripped narrative structure, through an operation plan and bound values, to a computed answer.Figure 1 presents these as successive transformations of the residual stream.Source: §3 · p. 2, Figure 1 · p. 3
  • In the duck-egg example, Schema Abstraction preserves the relational structure while discarding story details, then Operation Planning forms R = (x − y − z) · u.Operand Binding substitutes 16, 3, 4, and 2, and Computation yields 18.Source: §3 · p. 3
  • The paper asks whether these stages appear as distinguishable residual-stream transformations and which stage fails under an irrelevant NoOp distractor.The subsequent analyses address the pipeline and the distractor-related failure, respectively.Source: §3 · p. 3

4 Experimental Setup

The experiments use GSM-style prompts and datasets to examine the four-stage account, with template-level aggregation intended to reflect variation across problem outlines.

4 Experimental Setup

  • The main target is Llama-3.3-70B-Instruct, with Qwen-2.5-14B-Instruct and Gemma-2-9B-it replications in Appendix F.The target has 80 layers, and prompts use the GSM8K format with #### before the final numeric answer.Source: §4 · p. 3
  • The dataset suite includes GSM-Symbolic, GSM-P1, recreated GSM-NoOp, length-matched GSM-Filler, and padded-Symbolic for Stage 2 patching experiments.Padded-Symbolic and P1 pairs match narrative, operand values, and token length while differing in required operations.Source: §4 · pp. 3–4
  • Metrics average instances within each template and then average across templates, with 95% template-bootstrap confidence intervals.Templates, rather than within-template instances, are the resampling unit.Source: §4 · p. 4

5 Identifying the Four-Stage Pipeline

Residual similarity, content probes, and activation patching support a four-stage pipeline with distinct layer boundaries. The analyses locate Operation Planning and Operand Binding at different transitions and trace what changes at each.

5 Identifying the Four-Stage Pipeline

  • L22, L36, L40, and L80 mark transitions associated with Schema Abstraction, Operation Planning, Operand Binding, and Computation, respectively.Similarity dynamics identify the boundaries; probes interpret three of them, while causal patching identifies Operation Planning at L36.Source: §5 · pp. 4–5, Figure 2 · p. 5

5.1 Stage 2: Operation Planning

  • At L36, question-span transfer falls to ρ = 0.21 while commit-readiness rises from 0.08 at L30 to 0.88 at L36, locating Operation Planning.Attention-output patching spikes with the commit-readiness rise, identifying attention as the active component during this stage.Source: §5.1 · pp. 5–6, Figure 3 · p. 6

5.2 Stage 3: Operand Binding

  • At L40, question-span transfer drops from ρ = 0.62 to 0.26, as attention moves operand information into the answer-prefix residual.The paired experiments separate operation structure from operand values: Operation Planning effects concentrate through L36, whereas Operand Binding effects appear at L40.Source: §5.2 · p. 6

6 Diagnosing NoOp Fragility

NoOp-induced failure localizes to Operation Planning, where a compact set of attention heads is implicated by head-scaling interventions. Ablations also produce predominantly planning-specific failures, supporting the stage diagnosis.

6 Diagnosing NoOp Fragility

  • NoOp patching collapses in the same L22–L36 band as clean Operation Planning, localizing distractor corruption to the operation plan rather than an earlier or later stage.At L36, question-span patching has little effect on the answer, while direct-mode residual changes rise across the planning band.Source: §6 · p. 7, Figure 4 · p. 7, Figure 6 · p. 8
  • Engagement-anchored DLA separates Operation Planning heads that write for the distractor-engaging token from heads that write against it.The analysis selects NoOp-wrong traces that engage the distractor and examines heads in layers 23–36.Source: §6 · pp. 7–8
  • Amplifying H− raises NoOp-wrong accuracy to 0.51, versus 0.19 for random-outside heads; ablating H+, H−, or their union drops Symbolic-correct accuracy to 0.06, 0.47, and 0.02.The Symbolic-correct random-outside controls retain at least 0.98 accuracy; H− amplification is about 4σ above baseline.Source: §6 · pp. 8–9, Table 2 · p. 9, Table 1 · p. 9
  • H+ is the dominant subset: its ablation damages clean reasoning 7–8× more than H−’s, and the DLA measures only direct clause-mediated writes, not each head’s whole-generation effect.Whole-head scaling can therefore show effects, including beneficial amplification of H+, that the DLA score alone does not establish.Source: §6 · pp. 8–9

6.1 Classifying the Induced Failures

  • Ablating the planning heads yields Operation Planning failures in 0.91 of cases, versus 0.10 for random outside-band heads, which instead produce misreadings and arithmetic errors.Two annotators independently labeled sampled failure traces and counted a planning failure only when both agreed.Source: §6.1 · p. 9, Table 3 · p. 9

7 Conclusion

The paper presents a four-stage account of GSM-style word-problem solving and localizes NoOp fragility to Operation Planning. Causal scaling tests support a role for a compact set of attention heads in that failure.

Conclusion

  • The authors report initial evidence that the decomposition generalizes to other reasoning domains.Source: §7 · p. 9
  • NoOp fragility localizes to Operation Planning, where a compact attention-head set has a bidirectional causal role validated by scaling.Ablating the heads collapses correct solutions against size-matched random controls, while amplifying the anti-engagement subset recovers a substantial fraction of NoOp failures.Source: §7 · p. 9
  • Residual cosine dynamics, presence probes, and cross-prompt activation patching independently converge on four stages: Schema Abstraction, Operation Planning, Operand Binding, and Computation.The framework extends from synthetic one-step direct-mode prompts to multi-step CoT reasoning and refines Cheng et al. (2025)’s two-stage account.Source: §7 · p. 9

Limitations

The analysis focuses on math word-problem reasoning and distractor-induced failures in GSM-style tasks; broader transfer and deployable repair remain open.

Scope

  • The study’s scope is math word-problem reasoning and distractor-induced failures in GSM-style tasks.Transfer experiments are probe-level replications, not the full battery of methods used in the main text; mapping other tasks with probes, patching, and causal interventions remains open.Source: Limitations · p. 10

Repair

  • Turning the identified repair location into a targeted, deployable intervention remains future work.The amplification result identifies where repair should act, but not a deployable repair itself.Source: Limitations · p. 10

A Implementation Details

The experiments use zero-shot prompting with the model’s native chat template and construct matched NoOp and filler controls. Additional controls test effects of numeric tokens and prompt length.

A Implementation Details

  • Prompts use the native chat template with a fixed system prompt and mode-specific direct-answer or step-by-step instructions requiring an integer after “####.”In cached-trace CoT experiments, the reasoning text is inserted after the assistant header and before the answer prefix.Source: A Implementation Details · p. 12
  • NoOp clauses are selected to be natural, irrelevant to the gold computation, and plausible as distractors, then instantiated across matched templates.A random sample is manually checked for naturalness, unchanged gold answers, and nontrivial ignorable status.Source: A Implementation Details · p. 12
  • GSM-Filler reuses NoOp numeric tokens in neutral contexts, while filler wording is length-matched to NoOp at the prompt level.The filler clauses are authored in parallel with NoOp using the same template-by-template pipeline.Source: A Implementation Details · p. 12
  • GSM-Filler-DF strips number tokens from the inserted clause, replacing them with words such as “some,” to test for residual soft-NoOp effects.Source: A Implementation Details · p. 12
  • Padded-Symbolic length-matches filler to GSM-P1 clauses and serves as a controlled substitute for GSM-Symbolic in Stage 2 patching experiments.Source: A Implementation Details · p. 12
  • Padded-Symbolic accuracy is within −0.004 direct and −0.016 CoT of GSM-Symbolic; digit-free GSM-Filler-DF is statistically indistinguishable from baseline.Digit-bearing GSM-Filler shows drops of −0.067 direct and −0.052 CoT, smaller than GSM-NoOp’s drops.Source: A Implementation Details · p. 12

B Prompting Mode Robustness

The appendix tests whether the four-stage signatures persist across prompting positions and controls, then examines layer-wide head activity and contrasts two ablation failure types.

B Prompting Mode Robustness

  • CoT answer-prefix readouts replicate the L22 entity-drop and operator-rise signatures, while trace attention reduces the L40 number jump and causes early answer-probe saturation.These shifts reflect attention to the externalized CoT trace, not a change in the staged structure.Source: B Prompting Mode Robustness · p. 13
  • Before reasoning begins, the L22 entity-drop/operator-rise persists but is attenuated; number and final-answer probes stay near chance as binding and computation occur during CoT generation.The L22 drop is 0.78→0.68 here versus 0.78→0.53 in direct mode, and entity decodability rises again at L40.Source: B Prompting Mode Robustness · p. 13

C Filler-DF Replications

  • On digit-free GSM-Filler-DF, the L22–L36 question-span patching cliff and the direct-mode full-residual rise both replicate.This control has near-zero behavioral effect versus GSM-Symbolic and checks whether NoOp’s extra numeric token drives fragility.Source: C Filler-DF Replications · p. 13

D Engagement-Anchored DLA: Full-Layer Scan

  • Across the layer sweep, pre-L22 activity is sparse, Operation Planning has the highest active-head density at 27.2%, and post-L36 heads include downstream binding and computation activity.Pre-L22 only 5.5% of heads exceed threshold; after L36, 12.9% do, with isolated activity at L57, L65, L74, and L78.Source: D Engagement-Anchored DLA: Full-Layer… · pp. 13–14

E Failure Classification: A Worked Example

  • Random-head ablation preserves the correct plan but yields an arithmetic slip, whereas ablating the identified heads prevents completion of the multiplication plan despite correct quantity reading.The example answers are 5570 after random ablation and 398 after identified-head ablation; the gold answer is 5,572.Source: E Failure Classification: A Worked… · p. 14
  • Figure 10 reproduces the digit-free control’s L22–L36 patching cliff and full-residual rise; Figure 11 shows head activity peaking within the Operation Planning band.Source: Figure 10 · p. 15, Figure 11 · p. 15

F Generalization to Other Models

The four-stage structure and its diagnostic signatures reproduce in Gemma-2-9B and Qwen-2.5-14B. Their stage boundaries and the dominant damage/rescue head subset vary by model.

F Generalization to Other Models

  • Across all three model families, Filler-DF is nearly unchanged, P1 causes a moderate drop, and NoOp causes the largest behavioral hit.Digit-bearing Filler produces a smaller soft-NoOp drop than full NoOp and a larger drop than digit-free Filler-DF.Source: F Generalization to Other Models · p. 15, Table 5 · p. 15
  • Gemma-2-9B and Qwen-2.5-14B reproduce the four-stage residual sequence, including cross-only drops for Schema Abstraction and Operation Planning, a joint drop for Operand Binding, and terminal Computation.The same models also reproduce the corresponding entity, number-token, and answer-decoding signatures.Source: F Generalization to Other Models · pp. 15–16, Figure 12 · p. 17, Figure 13 · p. 17
  • Stage boundaries shift with model depth: Qwen reaches Schema Abstraction’s boundary at 0.48 depth, while Llama allocates 50% of its layers to Computation.Operation Planning and Operand Binding occur later proportionally in Gemma and Qwen than in Llama.Source: F Generalization to Other Models · p. 16, Table 6 · p. 16
  • Presence probes reproduce the stage signatures: entity decodability falls to chance at Stage 1, number-token decodability rises at Stage 3, and answer decodability rises toward the final layer.The entity-decoding boundary agrees with the independently identified residual-dynamics boundary.Source: F Generalization to Other Models · p. 16, Figure 13 · p. 17
  • Question-span patching remains near 1 before Schema Abstraction ends, then collapses by Operation Planning’s boundary in both models.The full-residual commit-readiness trace rises across that band, with attention the leading sublayer component.Source: F Generalization to Other Models · p. 16, Figure 14 · p. 17, Figure 15 · p. 18
  • Within-template value-token patching shows an Operand Binding cliff at the Operation Planning-to-Operand Binding transition in both models.This CoT-mode test uses numeric-value-token positions, keeping operands at the same absolute prompt positions.Source: F Generalization to Other Models · p. 16, Figure 16 · p. 18, Figure 17 · p. 18
  • NoOp question-span patching reproduces an Operation Planning-stage cliff on the Filler control in both models.The cliff falls within each model’s Schema Abstraction-to-Operation Planning band.Source: F Generalization to Other Models · p. 16, Figure 18 · p. 18
  • +0.15: Gemma’s H+ amplification has the largest recovery gap; the dominant damage/rescue subset differs by model, but both subsets outperform random-outside controls across all three models.Ablating H+ collapses Symbolic-correct accuracy on Gemma, while ablating H− does so on Qwen.Source: F Generalization to Other Models · p. 19

G Generalization to Other Datasets and Tasks

The authors test whether the four-stage signatures transfer beyond GSM-Symbolic using unchanged probes on SVAMP and a non-math reasoning task. The signatures recur on SVAMP; PhantomWiki offers proof of concept, with task-shaped stages and a less distinct separation between binding and computation.

G Generalization to Other Datasets and Tasks

  • SVAMP probes use correctly solved problems and seed-disjoint cross-validation; operand-resampled variants provide within-problem contrasts for numbers and answers.Entity tokens are divided into decorative and operand-attached categories using a mechanical criterion.Source: G Generalization to Other Datasets and… · p. 19
  • On SVAMP, the GSM-Symbolic stage boundaries recover the four signatures without per-benchmark tuning: decorative entities drop, operators remain decodable, operand numbers rise, and answer decodability increases late.Operand-number and answer probes use same-problem variants as within-problem contrasts.Source: G Generalization to Other Datasets and… · p. 19, Figure 19 · p. 19
  • On PhantomWiki, relation words are decodable early, chain-membership decodability rises later, and answer entities reach saturation at the final sampled layer.The chain-membership contrast matches names on presence, adjacency, and difficulty; evaluation uses universe-disjoint cross-validation.Source: G Generalization to Other Datasets and… · p. 20
  • PhantomWiki’s stage contents are task-shaped: retrieval combines Operand Binding and Computation into a less distinct resolution cascade, so the result is a proof of concept, not a full replication.The task uses relation words and person names rather than math operations and numeric values.Source: G Generalization to Other Datasets and… · pp. 19–20

Related papers

Loading 2609.17804v1…