Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragile

1 Introduction

• The study asks whether repeating solutions makes reasoning more fragile to a model’s next training stage, which may be unrelated to reasoning.Recipes such as s1 and LIMO use small sets of worked solutions repeatedly, and handoff scores alone cannot show how much reasoning survives later training.Source: §1 · p. 1
• After one unrelated instruction epoch, the drilled model fell from 95.4% to 86.0%, while the once-trained model held its accuracy; harsher stages drove the drilled model lower.Twenty updates of instructions left it at 59.3%, and answer-only training left it at 22.5%.Source: §1 · p. 2
• A fresh-solution model visited the same problems as often as the drilled model but remained unharmed, implicating repeated solution texts rather than few problems.It received a different correct solution at each visit.Source: §1 · p. 2
• The break recurred with a stronger model’s traces, across further runs and models, and on an arithmetic search task, although its size varied widely.The authors therefore claim the direction of the effect, not its size.Source: §1 · p. 2
• Five updates of reasoning training or twenty updates showing reasoning format with almost no mathematics restored nearly all the lost reasoning.The authors interpret this as suppression rather than erasure.Source: §1 · p. 2
• Sharpening without repetition did not cause the same fragility, while fresh solutions and limited replay prevented damage in the tested settings.The claim concerns later training without replay; on a skill the base model could not perform within a token budget, repetition mainly reduced learning.Source: §1 · p. 3

Models

• Most experiments use Qwen3.5-9B-Base; Qwen3.5-35B-A3B-Base and Nemotron 3 Nano test whether the effect extends beyond that model.Both additional models are mixtures of experts with about 3B active parameters.Source: §2 · p. 3

Three ways to train on the same solutions

• Each model receives 140 updates of 32 examples at learning rate 10−4, with training texts varying across drilled, once-trained, and fresh-solution conditions.The drilled model cycles through 579 solutions for an average of 7.7 passes; the once-trained model sees 4,480 different solutions once, while the fresh-solution model gets a new correct sample on each visit.Source: §2 · p. 3

The later stages

• Later stages include one epoch of instruction tuning at 10−4 and harsher 20-update stress tests at 3 × 10−4, none of which teaches math problem solving.The stress tests use instructions or answer-only targets for program-synthesis puzzles.Source: §2 · p. 3

How we measure

• Forced accuracy is measured on 221 MATH-500 problems using four samples of up to 16,384 tokens; paired bootstrap intervals cover problems, not training runs.A harder check uses 59 AIME 2025 and 2026 problems.Source: §2 · p. 3

3.1 Handoff and one epoch of instruction tuning

• After realistic instruction tuning, the drilled model lost 9.6 points more than the once-trained model, while the base, once-trained, and fresh-solution models stayed within a point of their starting accuracy.At handoff, the drilled and once-trained models differed by no more than 1.8 points in any math setting.Source: §3.1 · p. 4

3.2 Harsher and broader later stages

• Under harsher stress tests, the drilled model lost 36.2 points more than the once-trained model with instruction tuning and 73.1 more with answer-only training; higher later-stage learning rates increased the loss.Sixty instruction updates at the reasoning rate cost 13.1 points, compared with 36.2 points for twenty updates at three times that rate.Source: §3.2 · p. 4, Figure 2 · p. 5

3.3 Repeated texts or few problems?

• The fresh-solution model stayed within a point of the once-trained model after each later stage, while the drilled model lost 36.4 and 72.5 points more in the stress tests.The fresh-solution condition held problems and visit frequency constant while changing the solution text at each visit.Source: §3.3 · p. 5

3.4 A stronger model’s traces

• With gpt-oss-120b traces, the drilled model lost 59.6 points more than the once-trained model under instruction tuning and reached 1.1% accuracy after answer-only training; fresh traces prevented the break.At the reasoning training rate, the extra loss after sixty instruction updates was 6.9 points, short of the predicted 10.Source: §3.4 · p. 5

3.5 Other runs, other models and an arithmetic task

• The break recurred across new runs, at 35B, in Nemotron, and on an arithmetic search task, but its size varied widely across runs and settings.On arithmetic, drilled models lost about 70 points in the answer-only stage, while the teacher, starting model, and a clone lost at most 4.2; Figure 2 collects the broader comparisons.Source: §3.5 · pp. 5–6, Figure 2 · p. 5

4 Suppressed, not lost

• After instruction stress testing, freely answering models often stopped opening their reasoning block, so forced accuracy exposed a much larger drilled-versus-once-trained gap than free accuracy.The gap was 35.4 points with reasoning forced, compared with 9.7 points when answering freely.Source: §4 · p. 6
• Appending “Wait” recovered 41% of the drilled model’s extra loss after instruction tuning but only 6.5% after answer-only training.Drilled models often stopped early or looped even when the reasoning block was forced open.Source: §4 · p. 6
• Five updates of reasoning training closed nearly the entire gap after instruction stress testing, with R5 = 0.98.Other broken drilled models recovered about as far, while the once-trained model barely moved under the same training.Source: §4 · pp. 6–7
• Twenty updates on chat responses showing reasoning format but almost no mathematics closed the gap, with R20 of 0.98 after instruction tuning and 0.97 after answer-only training.The authors conclude that later training suppresses reasoning that a small amount of training can restore.Source: §4 · p. 7

• 0.111 nats per token: under the drilled 9B model, base-model solutions lose this likelihood, while they lose almost nothing under once-trained and fresh-solution models.The loss grows with passes; training once on gpt-oss-120b traces moves the 9B model by 0.024.Source: §5 · p. 7
• 63% of the drilled model’s B comes from positions where it keeps the base model’s top token but assigns it greater probability, termed over-sharpening.Its top next token matches 90.7% of training-text tokens, versus 85.1% for the base model.Source: §5 · p. 7
• A model sharpened without repeated texts reached three-quarters of the drilled model’s B but showed no additional loss beyond two points in either stress test.The intervals included zero; the authors note that a threshold in B between the two models is not excluded.Source: §5 · p. 7
• B flags drilled models but does not identify the mechanism or determine the loss: high B without repetition did no harm, and a low-B exploratory model still lost 27.1 points.It requires one forward pass per base sample and does not predict how much accuracy a model will lose.Source: §5 · p. 8

• 6.25% replay reduced the drilled model’s extra loss to 1.1 points, versus 13.1 without replay, in the gentler stage.The same result held with gpt-oss-120b traces; replay was not tested in harsher stages.Source: §6 · p. 8
• Fresh solutions prevented damage in every tested later stage, using eight correct solutions per problem instead of one and no additional problems.This cost no accuracy on the 9B math task, where there was little accuracy to gain, but did cost some on the synthetic skill.Source: §6 · p. 8
• Within 2,048 tokens, the base model solved 0.2% of five-digit base-7 multiplication problems, while solver solutions fit within the budget and averaged less than half of it.The task tests learning an efficient procedure the base model lacks within the fixed token budget.Source: §6 · p. 8
• 37.9% versus 66.3%: at handoff, the drilled model solved fewer held-out synthetic-skill problems than the once-trained model, and the gap was already present before later training.The stress test wiped out accuracy for all three models; the authors conclude that repetition mainly cost learning on this new skill, with fragility across recipes, based on one synthetic task.Source: §6 · p. 8

• Earlier work found that repeated identical text can make learning less durable than paraphrases, and that repeated pretraining data can harm models disproportionately to its training-set share.Reasoning fine-tuning can also win immediately and lose after a later RL stage training the same skill (Kang et al., 2026).Source: §7 · p. 8
• The study fixes updates, solution source, and handoff accuracy while varying repetition; later stages do not train the skill, and fresh solutions serve as a paraphrase comparison.The authors also note that the later stage suppresses reasoning the drilled model had, rather than merely adding less reasoning.Source: §7 · pp. 8–9
• Prior studies track old skills during new-skill learning, whereas this paper follows a newly trained skill through the next stage; its sharpened-model result shows large divergence need not make a skill fragile.The paper relates B to forward-KL work on forgetting and notes prior findings that acquisition method can affect retention beyond immediate performance.Source: §7 · p. 9
• The authors connect their recovery result to prior work on restoring apparently lost capabilities and relate sharpening to research on reweighting reasoning and changes in reference-solution likelihood.They caution that their findings may depend on base models already being able to verify and backtrack.Source: §7 · p. 9

8 Discussion and limitations

• Test models after the next training stage with reasoning both forced and free: handoff accuracy did not distinguish drilled models in any math run.Fresh solutions prevented damage wherever tested, replay did so in one stage, and about 160 examples of reasoning training repaired it afterward.Source: §8 · p. 9
• The mechanism remains open: over-sharpening alone seems insufficient, while tighter fit to particular texts and adapter sensitivity are proposed accounts, not established explanations.A threshold in divergence is not excluded; the authors say relearning speed fits either account.Source: §8 · p. 9
• Full fine-tuning is untested, and the study’s realistic later stage and replay test each cover only one dataset/rate and one gentle stage, respectively.All training used LoRA; the synthetic skill is a single token-budget-defined task, and the 9B base was near ceiling on the math test.Source: §8 · p. 9

AI use statement

• The authors report that Claude assisted with code and paper text under their direction, while research decisions and responsibility remained theirs.Source: AI use statement · p. 10

Ethics statement

• The study used public datasets under their respective licences, synthetic arithmetic tasks, and no human subjects.Source: Ethics statement · p. 10

Reproducibility statement

• Experiments were registered before data collection, main results were independently recomputed from raw run files, and code, frozen outputs, and records are identified for reproducibility.Reported runs used about $2,437 of Tinker compute at list prices.Source: Reproducibility statement · p. 10

A Arms and training

• The math acquisition arms use the base model’s own correct forced solutions; the drilled arm cycles through 579 solutions while the once-trained arm uses 4,480.Table 2 documents the acquisition arms and notes shared learning-rate and batch settings.Source: A Arms and training · p. 14, Table 2 · p. 14

B Replications in detail

• Replication results vary by model and setting: at 3.3 passes, the 35B drilled model lost 16.6 points under answer-only training, while Nemotron lost 25.8 under instruction tuning.At 35B and on Nemotron, stress tests also reduced accuracy for the base and once-trained models; AIME results are described as descriptive given 59 problems and wide intervals.Source: B Replications in detail · pp. 15–16

C How the drilled models fail, and re-forcing

• After the answer-only stage, 93.2% of U’s (358/384) and 87.5% of F’s (336/384) forced searches reach the 16,384-token cap; verifier scans find accepted expressions in only 7 and 9, respectively.Source: C How the drilled models fail, and… · p. 16
• The drilled models’ response patterns vary: at 35B, the median is 270 tokens after answer-only training, while Nemotron’s responses under instruction tuning average 1,138 tokens.Across the broad mix and teacher-trace instruction tuning, drilled models also write shorter responses and close the reasoning block less often than once-trained models.Source: C How the drilled models fail, and… · pp. 16–17, E The later stage’s size · p. 19
• Under the primary budget rule, 74% of the drilled model’s instruction-tuned responses and 91% after answer-only training are continued, but the answer-only continuations run long without finding more answers.The answer rule recovers 37.6% after the answer-only stage; the cue itself costs other models 1.1–2.5 points and the base 9.3 points.Source: C How the drilled models fail, and… · p. 17, Table 6 · p. 17

D The decision to reason

• After instruction tuning, no 9B pilot model opens a reasoning block freely, yet every non-drilled model retains forced competence, with the base gaining 0.9 points and the once-trained model 0.1.On the arithmetic task, the teacher, clone, and M0 skip search on half of openings within about five answer-only updates, while forced accuracy remains 73.4–75.8%.Source: D The decision to reason · p. 18

E The later stage’s size

• At 3 × 10−4, the drilled model’s forced accuracy falls to 59.3% after 20 updates, while the once-trained model stays within 94.7–96.0%.At 10−4, the drilled model is at 82.8% after 20 updates; after 60 updates on the broad mix, D and D-T score 85.4% and 85.3%, versus 94.9% and 95.8% for O and O-T.Source: E The later stage’s size · p. 18, Table 8 · p. 19
• On instructions at 10−4, D-T’s mean loss falls from 1.85 to 1.74 and O-T’s from 1.80 to 1.74; on the broad mix, the models’ last-five-update losses converge to 0.96–0.97.The later stages train on distinct examples, with the default instruction stage’s first 640 examples replayed in the longer run.Source: E The later stage’s size · p. 18

F What the later stage does to the drilled model

• On M0’s less likely sampled tokens, drilled checkpoints retain only 0.25–0.41 of M0’s probability, versus 0.91–0.98 for the teacher and clone.Figure 4 traces forced accuracy across instruction-tuning updates and learning rates; Figure 5 tracks likelihood changes on handoff and base-model texts.Source: F What the later stage does to the… · p. 19, Figure 4 · p. 20, Figure 5 · p. 20
• After two instruction-tuning updates, the drilled model’s handoff texts become 0.067 nats per token less likely, versus 0.0005 and 0.0004 for once-trained and fresh-solution models.From update 5 onward, the drilled model assigns its own solutions lower likelihood than the base does; after 20 updates the negative log-likelihoods are 0.403 and 0.357 nats per token under the drilled model and base.Source: F What the later stage does to the… · pp. 19–20
• For traces from gpt-oss-120b, the drilled model’s own solutions lose 0.058 nats per token after two updates, versus 0.0004 for the once-trained model.The drilled model’s likelihood of base-model reasoning rises by 0.026 nats per token while it falls by 0.0056 and 0.0054 under once-trained and fresh-trace models; the drilled model moves toward base reasoning without regaining competence.Source: F What the later stage does to the… · p. 21
• The authors’ account that drilling narrows reasoning onto sharpened paths and the later stage flattens them is post hoc and untested.A model sharpened three-quarters as much without repetition was not fragile, and a few reasoning-training updates restored the drilled model.Source: F What the later stage does to the… · p. 21

G The anchor, an exploratory test

• After the answer-only stage, FA1 and FA025 score 48.2% and 27.9%, versus 11.7% for F-u140, while their B values fall to 0.022 and 0.0085.Their negative log-likelihood on drilled texts still falls by 85% and 99% as much as for F-u140; U2 collapses to 6.5% with B=0.154.Source: G The anchor, an exploratory test · p. 21, Table 9 · p. 22
• The registered protection test is not interpretable because anchored checkpoints fail the 4,096-token handoff parity check, though the gaps shrink at 16,384 tokens.Within 16,384 tokens, FA025 is level with F-u140 at handoff, 78.6% versus 81.5%, while FA1 remains at 75.3%.Source: G The anchor, an exploratory test · pp. 21–22
• FA1 has B=0.0085 and retains 0.95 of M0’s probability on sampled alternatives, yet loses 27.1 points within 16,384 tokens.The anchor trains on samples from the distribution measured by B, making its low B partly by construction; some effects of drilling lie outside what B measures.Source: G The anchor, an exploratory test · p. 21

H The arithmetic task in detail

• Over twenty answer-only updates, R-P-u140 falls from 61.2% to 2.6% within 4,096 tokens; R-S-u200 falls from 58.9% to 40.6% and keeps writing derivations.R-S-u200 is a counterexample to B as a warning: at 11.1 passes, its B is 0.068, yet it keeps writing derivations.Source: H The arithmetic task in detail · p. 22
• The registered predictions that forced accuracy would hold through update 80 and that trained checkpoints would keep their margin over M0 failed in the 480-update run.The paper claims the recurring decision clock and drilled collapse within twenty updates, not late half-lives.Source: H The arithmetic task in detail · p. 22
• Under the answer-only stage, the teacher, clone, and M0 reach half-skip after 4.1, 5.0, and 4.9 updates, while U reaches it after 2.7.Instruction tuning instead makes all checkpoints stop searching and answer with one-line derivations.Source: H The arithmetic task in detail · p. 22
• After twenty answer-only updates, 93.2% of U’s and 87.5% of F’s forced searches hit the 16,384-token cap, versus 16.1% for the clone.A four-times larger budget recovers little; the clone differs in prompts covered and completions per prompt as well as repetition.Source: H The arithmetic task in detail · p. 23
• Drilled checkpoints have B values of 0.097, 0.139, and 0.154 nats per token, compared with −0.0001 and −0.0034 for the teacher and clone.B rises with passes, and every 7.7-pass drilled checkpoint without added data exceeds every lasting checkpoint’s B by at least 0.046 nats per token.Source: H The arithmetic task in detail · pp. 23–24, Table 10 · p. 24

I Where the divergence sits, and the loss jump

• 0.93 rank correlation links B to forced points lost by update 20 across M0 and nine search-trained states, but within-group B does not rank the loss.Handoff accuracy also does not rank it; the group label alone gives 0.92 correlation.Source: I Where the divergence sits, and the… · p. 25
• One update into the answer-only stage, drilled checkpoints’ next-batch loss more than doubles while every other origin’s falls.Across eleven origins, update-2 loss correlates with forced accuracy change by update 20 at Pearson −0.84; the jump marks the start, not the full damage.Source: I Where the divergence sits, and the… · p. 25, Table 12 · p. 25

J Strict and resumed scores

• 68.5% to 4.9% strict accuracy is the drilled checkpoint’s collapse on correct solutions, while strict handoff accuracy can rank lasting checkpoints below drilled ones.The contrast between lasting and drilled checkpoints is weaker under the strict rule within 16,384 tokens, amid high placeholder-restatement rates.Source: J Strict and resumed scores · p. 25, Table 13 · p. 26, Table 14 · p. 26

K Token budgets

• After twenty answer-only updates, M0, the teacher, and the clone lose just 2.1–4.2 points within 16,384 tokens, while drilled states lose about 69.8.A four-times larger budget recovers little; at handoff, every state gains 19.5–27.6 points when moving from 4,096 to 16,384 tokens.Source: K Token budgets · p. 26, Table 16 · p. 27

L The math data pipeline

• The math pool comes from Tülu 3’s decontaminated NuminaMath-TIR subset, and evaluation excludes MATH-500 levels 1–2 and one source-overlapping AIME 2026 problem.Problems the base model solves within 256 tokens after an empty think block are skipped; retained training examples require a correct, uncapped forced sample.Source: L The math data pipeline · p. 27, Table 17 · p. 28, Table 18 · p. 28

M The main-phase arms

• D2 and rank-128 drilling produce similar reductions in negative log-likelihood on D’s texts, from 0.453 to 0.277 and 0.270, respectively.Their reductions differ by less than the registered 20% margin.Source: M The main-phase arms · p. 28
• The 35B noise floor is 0.0034 nats per token, close to the 0.0030 difference between independently created base samplers.On the common set, a second base sampler gives B=0.0000 [−0.00002, 0.00002], placing every state’s B above the floor.Source: M The main-phase arms · p. 28
• On teacher-written traces, D-T falls from 73.7% to 11.0% on AIME after instruction tuning, while O-T rises from 72.9% to 78.0%.The registered consolidation rival failed, though its reading conditions failed too and bound the cell because the student moved little toward the teacher.Source: M The main-phase arms · pp. 28–29

N The revision experiments

• 13.1 points without replay falls to 1.1 [−0.9, 3.3] with 6.25% replay for drilled models, a 91% reduction in excess over once-trained models.With gpt-oss-120b traces, replay yields −0.9 [−2.8, 0.8] excess, indistinguishable from zero.Source: N The revision experiments · p. 29, Table 21 · p. 30
• Five updates of reasoning training bring back almost all lost competence, and twenty updates on broad chat reasoning-format data do the same.The habit data contain only about 1–1.5% worked math word problems; the registered interpretation is suppression and restoration when R5 is at least 0.75.Source: N The revision experiments · p. 30, Table 22 · p. 30
• Sharpening without repetition leaves excess within 3 points after both instruction tuning and the answer-only stage, so sharpening alone does not break it.Its B reached 0.082 [0.077, 0.087] nats per token before evaluation.Source: N The revision experiments · p. 30, Table 23 · p. 30
• Across two new runs and both stages, drilled models’ excess over once-trained models remained positive, while fresh-solution models stayed within 1.4 points of once-trained models.Across four drilled sets, extra loss averaged 39.7 points (range 12.6–61.1) in the instruction stress test and 14.7 (13.1–17.8) at the gentler rate.Source: N The revision experiments · p. 31, Table 24 · p. 31
• After the gentler stage (60 updates at 10−4), D-S finishes 24.2 percentage points below once-trained O-S [20.4, 27.9] on synthetic-skill accuracy, meeting the registered definition of a break.After the stress test, every model is below half a percent.Source: N The revision experiments · p. 31, Table 25 · p. 32

O Registered predictions and outcomes

• 50 of 70 registered predictions held, 17 failed (7 partly), and 3 could not be read; the predictions behind the headline claims held.The paper follows the later relearning experiment’s reading that drilled competence is suppressed rather than lost.Source: O Registered predictions and outcomes · p. 32, Table 27 · p. 33, Table 28 · p. 34