PaperlayerView exact revision on arXiv

Source-linked AI summary

How Much Does the Hint Matter? Scope Conditions for Hindsight Self-Distillation in Financial Reasoning

Johnny Yu, Cyril Gorlla

arXiv:2610.36082v1cs.LGcs.AIcs.CL

TL;DR

Financial institutions face data-residency constraints that limit deployment to small open-weight models, while those models underperform on financial multi-step reasoning. Fin-SD localizes self-distillation at the first faulty step using a hint-conditioned copy of the student. It matches full-solution OPSD on accuracy with fewer evaluation tokens, while the paper reports no material regression on general reasoning or compliance.

  • Problem

    Data-residency constraints restrict institutions to locally hosted open-weight models, which underperform on the multi-step quantitative reasoning financial analysis demands.

    Source: §1 · p. 1

  • Method

    Fin-SD locates the first faulty step in a failed student trajectory and distills a hint-conditioned frozen copy of the student over a short rollout.

    Source: §1 · p. 2

  • Results

    Fin-SD matches OPSD’s accuracy (72.19% for Fin-SD vs OPSD’s 70.42%) while generating 9.5% fewer tokens at evaluation.

    Source: §1 · p. 2, §8 · p. 8

  • Takeaways & Limitations

    The reported recipe uses no external model gradient and shows no observable harm to MMLU-Pro reasoning or FinTrust compliance performance.

    Source: §8 · p. 8

    Training–evaluation overlap means 17 of the 295 corpus-seeding formulas appear in solutions for 22 of the 238 evaluation items, making this in-domain transfer despite differing problem-writing styles.

    Source: §7 · p. 8

Abstract

from arXiv · show

Financial institutions operating under data-residency constraints are restricted to open-weight models small enough to locally host, which trail frontier systems on multi-step quantitative reasoning. On-policy distillation narrows such gaps by supervising a student on its own rollouts, but traditional OPD requires a strong external teacher, and on-policy self-distillation (OPSD) conditions on a full reference solution. We ask how much of that signal is necessary. We present Fin-SD, a localized hindsight self-distillation recipe for multi-step reasoning. Given a problem the student fails, the student itself identifies the first incorrect reasoning step and writes a short critique of that step. We truncate the student's trajectory immediately before the error (aiming to obtain a correct prefix), condition a frozen copy of the student on the same prefix plus the hint, and distill that hint-conditioned teacher into the student over a short rollout under reverse KL. No external model contributes a gradient. We apply Fin-SD to GPT-OSS-20B on a custom multi-step financial reasoning benchmark that requires deriving formula inputs from realistic scenarios via accounting conventions. We find: (i) replacing the critique with a generic-feedback control matches Fin-SD's entire 3.03 pp accuracy gain over the base model on the FinanceReasoning Hard subset, suggesting the gain comes from where supervision is applied, not hint content; (ii) the model writing the hint does not need to be stronger than the model receiving it: hints from same-family GPT-OSS-120B, at roughly 6x the parameters, yield no materially greater benefits than self-written hints; and (iii) Fin-SD matches on-policy self-distillation from full reference solutions on accuracy while generating 9.5% fewer tokens. With Fin-SD, general reasoning on MMLU-Pro and a fourteen-metric compliance audit on FinTrust show no material regression.

1 Introduction

The paper asks how much privileged information self-distillation needs, aiming to improve small open-weight models’ multi-step financial reasoning while preserving behavioral assurance. Fin-SD localizes supervision to the first faulty reasoning step, and its reported comparisons find that generic feedback and a larger hint writer offer no material advantage over the localized recipe.

1 Introduction

  • Data-residency constraints push institutions toward small open-weight models, which underperform on financial multi-step reasoning, while domain fine-tuning can compromise behavioral assurance.The paper frames its question around improving performance while maintaining compliance and cost-effectiveness.Source: §1 · pp. 1–2
  • The feedback control does not materially degrade any of the three metrics and gains +3.36 pp accuracy over base, versus Fin-SD’s +3.03 pp, suggesting problem-specific guidance is unnecessary once mistakes are localized.Source: §1 · p. 2
  • A hint writer roughly 6× larger than the receiving model makes no significant accuracy difference on a corpus whose non-arithmetic failures are dominated by convention selection.The 20B model localizes errors with the ground-truth answer in hand.Source: §1 · p. 2
  • Fin-SD uses the student’s own weights to locate the first faulty clause, truncate the trajectory there, and distill a hint-conditioned frozen copy over a 100-token rollout.The recipe uses reverse KL and does not rely on an external model contributing a gradient.Source: §1 · p. 2
  • Fin-SD matches OPSD’s accuracy (72.19% for Fin-SD vs OPSD’s 70.42%) while generating 9.5% fewer tokens at evaluation.The comparison also reports directionally more reliable in-budget completion rate (+2.44 pp).Source: §1 · p. 2
  • The paper reports no observable harm to general reasoning on MMLU-Pro or compliance performance on FinTrust after Fin-SD training.The benchmark corpus derives formula inputs from realistic financial scenarios and conventions rather than stating them directly.Source: §1 · p. 2

2 Related Work

Related work motivates training on student-generated contexts and targeted supervision, then situates Fin-SD among self-distillation methods and financial reasoning benchmarks. The section also notes that the choice of KL direction remains contested.

2.1 From offline to on-policy distillation

  • On-policy distillation evaluates a teacher on student-generated prefixes, addressing the off-policy mismatch of training on teacher contexts; reverse KL is one proposed divergence choice.Prior work cited in the section also notes that partial rollouts can suffice to compute the signal.Source: §2.1 · p. 2

2.2 Self-distillation and privileged context

  • OPSD uses a reference solution as privileged context, while HINT-SD targets failure-relevant spans; Fin-SD adapts targeted supervision to the first faulty clause in a single reasoning chain.Context distillation more broadly trains a model using a context-augmented version of itself as teacher.Source: §2.2 · p. 2

2.3 Financial reasoning benchmarks

  • Existing financial benchmarks often emphasize extraction or few-step arithmetic, whereas FinanceReasoning Hard requires composing formulas and Fin-SD targets deriving function inputs from financial logic.FinanceReasoning Hard contains 238 problems and provides a library of 3,133 financial functions; FinTrust assesses seven compliance dimensions.Source: §2.3 · pp. 2–3

2.4 Choice of divergence

  • Prior work disagrees on KL direction: OPSD favors forward KL, while MiniLLM, OPD, and HINT-SD favor reverse KL, which Fin-SD implements and ablates.Source: §2.4 · p. 3

3 Method

Fin-SD finds failures in the student’s own reasoning, adds a targeted critique to a frozen copy of the student, and distills that copy’s behavior over a short rollout. The method uses student-generated prefixes and periodically refreshes its teacher and failure set.

3.1 Preliminaries and notation

  • The frozen teacher is a context-augmented copy of the student, re-frozen each epoch using the student’s weights from the previous epoch.Its target can therefore improve as the student does, rather than remaining fixed at initialization.Source: §3.1 · p. 3

3.2 Error localization and hint synthesis

  • For a failed trajectory, the hint model identifies the first departure from a valid solution path, writes a short critique, and classifies the error as arithmetic or conceptual.Arithmetic-only failures are excluded from weight updates, though they remain incorrect in reported accuracy; an informal classifier audit reached 96% accuracy (48 of 50 items).Source: §3.2 · p. 4

3.3 Prefix grounding and teacher augmentation

  • The student trajectory is truncated immediately before the localized error, and both policies receive that same student-generated prefix, while only the teacher also receives the hint.Using the student’s own prefix keeps training contexts within its native distribution.Source: §3.3 · p. 4

3.4 Distillation objective

  • Fin-SD aligns the student’s next-token distributions with the teacher’s over a short rollout, using reverse KL while masking the shared prefix from the loss.The teacher distribution is treated as a constant target, and only rollout positions contribute to the loss.Source: §3.4 · p. 4

3.5 Additional notes

  • The method recomputes the failure set each epoch and re-freezes the teacher, so training tracks current student weaknesses and uses an evolving target.The paper characterizes this as iterative self-distillation with a hint, rather than distillation from a fixed initial checkpoint.Source: §3.5 · p. 4

4 Experimental Setup

The experiments use a curated financial reasoning corpus whose problems require deriving formula inputs from realistic scenarios and conventions. They evaluate GPT-OSS-20B against distillation and supervised baselines on FinanceReasoning Hard, with additional general-reasoning and compliance evaluations.

4.1 Financial reasoning corpus

  • Each synthesized problem requires deriving unspecified inputs from finance conventions, market scenarios, or financial statements, executing the core function, then applying its intermediate metric to a domain-native endpoint.From 3,133 Python-formatted financial functions, difficulty screening (retaining scores ≥3) and viability screening yielded 295 formulas.Source: §4.1 · pp. 4–5
  • The final corpus contains 220 manually reviewed problems, split 80/20 for training and validation.A critique pass removed contradictory or truly underdetermined premises and quantities.Source: §4.1 · p. 5

4.2 Training configuration

  • GPT-OSS-20B is fine-tuned with all-linear LoRA and AdamW, with the epoch selected by in-distribution validation accuracy.The same epoch-selection treatment applies to OPSD and SFT baselines.Source: §4.2 · p. 5

4.3 Evaluation and statistics

  • Evaluation uses 238 FinanceReasoning Hard problems and reports accuracy, in-budget completion rate, and cumulative generation tokens.Fin-SD, OPSD, and ablations use five seeds; SFT uses three, and the base model has five independent runs, with means, standard deviations, and t-based 95% confidence intervals reported.Source: §4.3 · p. 5
  • MMLU-Pro and FinTrust provide single-run evaluations of general reasoning and compliance beyond the target domain.Decoding temperature is 0.0; run-to-run variation includes training seeds, inference non-determinism, and LLM-as-judge measurement noise.Source: §4.3 · pp. 5–6

5 Results

Fin-SD improves FinanceReasoning Hard accuracy over the base model and matches OPSD accuracy with fewer evaluation tokens. A generic hint matches the targeted critique, while MMLU-Pro and FinTrust show no material broad regression.

5.1 Fin-SD improves over the base model

  • 3.03 pp: Fin-SD reaches 72.19% accuracy versus 69.16% for the base model, while also improving completion rate and reducing token use.Completion rate rises by 1.01 pp, and token consumption falls by 17.3k (3.3%).Source: §5.1 · p. 6

5.2 Hints match full reference solutions at lower cost

  • 72.19% vs 70.42%: Fin-SD matches OPSD on accuracy while generating 9.5% fewer evaluation tokens.Fin-SD also has a 2.44 pp higher completion rate than OPSD.Source: §5.2 · p. 6

5.3 A vague hint matches a targeted critique

  • 72.52%: The generic-feedback placebo does not underperform the targeted hint, and its 0.33 pp accuracy edge lies within seed spread.The placebo also has slightly higher completion and lower token count; the results are consistent with gains coming mainly from applying supervision at the localized error.Source: Figure 3 · p. 6, §5.3 · p. 6

5.4 General capability and compliance

  • MMLU-Pro performance is near-identical overall, with a 5.47 pp chemistry drop; FinTrust shows no large regression across fourteen metrics.The authors conclude Fin-SD has no material deterioration in general capability or compliance.Source: §5.4 · pp. 6–7, Table 3 · p. 7, Table 4 · p. 7

6 Ablations

Self-written hints perform comparably to hints from a larger same-family model, reverse KL outperforms forward KL on the reported metrics, and shorter distillation rollouts provide a small gain.

6.1 Self-written hints match those of a larger same-family model

  • Holding KL direction fixed, Fin-SD’s self-written hint matched GPT-OSS-120B under reverse KL at 72.19% accuracy for both; no pairwise hint-writer accuracy contrast approached significance.Under forward KL, accuracy was 70.93% with self-written hints versus 71.26% with GPT-OSS-120B hints; the authors suggest self-correction, rather than hint information quality, is the bottleneck.Source: §6.1 · p. 7

6.2 Reverse direction outperforms forward

  • 1.26 pp: Reverse KL improves accuracy over forward KL from 70.93% to 72.19%, with completion rate higher and token use lower.The direction of the gain is the same across all three metrics under both hint writers; the authors characterize the effect as real but modest.Source: Table 5 · p. 7, §6.2 · pp. 7–8

6.3 Rollout truncation contributes a small gain

  • 72.19% vs 71.09%: Reducing the rollout cap from 8192 to 100 tokens raises accuracy by 1.09 pp and lowers token use by 5.0k.Completion rate rises by 0.42 pp; truncation also saves training memory because the backward buffer scales with rollout length.Source: Table 5 · p. 7, §6.3 · p. 8

7 Limitations

The evaluation is limited by training–evaluation formula overlap, a modest and selectively mined training corpus, and limited hyperparameter and prompt tuning; self-distillation also has a ceiling tied to what hint-conditioning can elicit.

7 Limitations

  • Self-distillation gains are bounded by what hint-conditioning can elicit from the student’s pre-training exposure, and self-diagnosis may fail beyond some difficulty threshold.Re-freezing the teacher at each epoch boundary raises but does not remove this ceiling.Source: §7 · p. 8
  • 17 of the 295 corpus-seeding formulas appear in solutions to 22 of 238 evaluation items, making the evaluation an in-domain transfer setting.The problem-writing philosophies differ greatly between the training corpus and benchmark.Source: §7 · p. 8
  • The study did not extensively tune hyperparameters or prompts, and its 15-step linear warmup occupied 15 of 20 optimization steps, likely limiting gains.The schedule converged well despite the extended warmup.Source: §7 · p. 8
  • The 220-problem training corpus is modest, and failure mining uses only a subset each epoch; tasks with multiple justifiable assumptions are excluded.Those tasks would require rubric-based grading.Source: §7 · p. 8

8 Conclusion

Fin-SD adapts targeted hindsight self-distillation to multi-step financial reasoning and investigates how much privileged information its teacher needs. It matches full-solution on-policy self-distillation on accuracy with fewer tokens, while generic feedback and a stronger hint writer offer no material advantage in the reported comparisons.

8 Conclusion

  • Fin-SD matches full-solution on-policy self-distillation on accuracy while using 9.5% fewer tokens.It also produces a shorter, more reliably executable solution style.Source: §8 · p. 8
  • A generic hint with no problem-specific information performs as well as one giving the correct next step, consistent with supervision site mattering more than feedback content.The proposed explanation is that supervision is applied at a repaired on-policy prefix at the first faulty step.Source: §8 · p. 8
  • On a corpus dominated by convention-selection failures, a 20B model can localize errors well enough that a same-family hint writer with roughly 6× the parameters makes no measurable difference.Fin-SD uses no external model to contribute a gradient, relevant to institutions unable to send data to a frontier API.Source: §8 · p. 8
  • Future work could test Fin-SD on student models of varying sizes or in other domains, such as law.Source: §8 · p. 8

Related papers

Loading 2610.36082v1…