Source-linked AI summary
Self-Specialized Teachers for Domain Post-Training
Yifei Li, Rongman Xu, Lingling Zhang, Muye Huang, Zihan Ma, Jiashuai Liu, Hang Yan, Heng Wang
TL;DR
Target-only post-training can improve specialized-domain performance while degrading general behaviors, especially when representative replay data are unavailable. SSTD trains a base-initialized domain teacher with base-aware objectives, then distills its distributions on student-generated prefixes. Across finance, medicine, and law, it retains most of direct fine-tuning’s target improvement while improving the evaluated general-suite mean by 4.8–5.0 points at the reported operating point.
Problem
Target-only post-training may improve a specialized domain while degrading general behaviors acquired by a broadly capable base model when representative replay data are unavailable.
Method
SSTD trains a copy of the base model into a domain teacher using key-token weighting and distribution alignment, then distills its token distributions on student-generated prefixes.
Results
Across finance, medicine, and law, SSTD retains most of direct SFT’s target gain while improving General Avg. by 4.8–5.0 points at the reported operating point.
Takeaways & Limitations
SSTD limits general degradation without requiring an external teacher or a broad general replay corpus.
Takeaways & Limitations
The evaluation covers selected held-out behaviors and cannot certify that no other general behavior changed.
Abstract
from arXiv · showhide
Target-only post-training can improve performance in a specialized domain while degrading behaviors that a general-purpose base model acquired before adaptation. We study this problem when target-domain data are available but a representative replay corpus is not. We propose self-specialized teacher distillation (SSTD), a two-stage procedure that first trains a copy of the base model into a domain teacher, then distills its token distribution to a student on prefixes sampled from the student itself. Teacher training combines standard target supervision with base-aware key-token weighting and distribution alignment to the frozen base model; on-policy distillation then places domain feedback on states the student can encounter at inference time. On financial numerical reasoning, medical question answering, and legal holding identification, SSTD retains much of the target improvement of direct fine-tuning while improving the mean score on the evaluated general suite by 4.8--5.0 points at the reported operating point. The pattern persists across Qwen3 sizes and on Gemma backbones. SSTD requires neither an external teacher nor general replay data.
1 Introduction
Domain post-training must add specialized expertise without treating unmeasured general behavior as expendable. SSTD addresses this setting by building a base-connected teacher and distilling it on student-generated states.
- Motivation: Target-only adaptation can improve domain expertise while moving general behaviors that target data and finite replay do not cover.Replay and reference matching constrain only covered regions, while parameter penalties restrict change without identifying useful domain changes.
- Teacher construction: A self-specialized teacher starts from the base model and upweights reference tokens that the frozen base finds unlikely.This directs teacher capacity toward domain gaps while preserving ordinary language and familiar response structure the base may already handle.
- On-policy distillation: SSTD queries the frozen teacher on prefixes generated by the student, placing domain feedback on states the student can encounter after its own earlier predictions.This supplements fixed gold-path supervision, which does not directly guide deviations from ideal prefixes.
- Contribution: Across finance, medicine, and law, SSTD retains most of direct SFT’s target gain while reducing degradation on held-out general evaluations.The method requires neither an external teacher nor a broad general replay corpus.
2 Related Work
Related work spans domain adaptation and retention, knowledge distillation, and supervision on learner-generated states. SSTD combines these lines through a target-specialized, base-compatible teacher queried on student trajectories.
- Domain adaptation and retention: Domain adaptation methods report in-domain gains, but their objectives do not directly test behavior outside the target distribution.Retention methods instead revisit stored examples or impose constraints, each covering only particular aspects of general behavior.
- Knowledge distillation and teacher construction: Knowledge distillation can use logits, representations, sequences, or teacher–student chains, including peers or identically initialized models.SSTD uses a same-architecture, same-initialization teacher for a different purpose: transmitting a constrained domain increment.
- Distillation at student-generated states: Student-state supervision queries an expert on states induced by the learner rather than relying only on fixed gold or teacher trajectories.SSTD adopts this on-policy principle while distinguishing itself through base-aware teacher construction and distribution alignment.
3 Problem Setting
The problem is a single focused domain update with target data but no representative replay corpus. Evaluation therefore measures both target learning and collateral change on a held-out general suite.
- Problem definition: The setting provides a base model, target-domain data, and a student initialized from the base, but no representative sample of the original pretraining distribution.The goal is to improve target-domain performance while keeping held-out general-suite losses small.
- Evaluation: Evaluation compares target performance and retained general performance because either axis alone cannot reveal the full adaptation outcome.Target measures use FinQA, MedMCQA, and CaseHOLD, while retention uses the arithmetic mean of MMLU, HellaSwag, and ARC-C scores.
4 Self-Specialized Teacher Distillation
SSTD constructs a domain teacher from the base model, emphasizing tokens where the base is weak while constraining distributional drift, then distills that teacher on student-generated prefixes. This separates domain acquisition from student-aware, on-policy supervision.
- SSTD trains a copy of the base model as a domain teacher while retaining a connection to the frozen student distribution.Shared initialization makes the teacher represent a domain increment over existing student behaviors rather than an unrelated capability set.
- 4.1 Base-Aware Key-Token Guidance: Base-aware key-token weighting gives extra teacher-training signal to target tokens that the frozen base predicts with low confidence.Weights remain positive for every token, so reweighting changes priority rather than removing full-sequence domain fitting.
- 4.2 Student-Distribution Alignment: Distribution alignment penalizes teacher probability mass on tokens that the base considers unlikely, limiting unsupported teacher preferences without requiring exact probability matching.The alignment term preserves room for target-domain updates when the evidence supports them.
- 4 Self-Specialized Teacher Distillation: The teacher objective combines ordinary domain SFT, key-token guidance, and alignment, with separate coefficients controlling key-token emphasis and departure from the base distribution.The three terms respectively teach the domain, select where the teacher should exceed the student, and constrain how far that excess moves.
- 4.4 On-Policy Distillation: On-policy distillation has the student sample trajectories while the frozen teacher supplies full next-token distributions at the student-reached prefixes.This addresses the mismatch between gold prefixes and states produced after the student’s own earlier choices, while full logits provide dense feedback over alternatives.
5 Experiments
Across finance, medicine, and law, SSTD preserves most target-domain gains while substantially recovering retained general performance. Ablations, scaling, backbone tests, and cost accounting support the method’s target–retention trade-off, though it costs more than Naive SFT.
- Domain Adaptation and General Retention: SSTD improves General Avg. by 4.8, 5.0, and 4.9 points over Naive SFT on Finance, Medical, and Legal, respectively.Its target scores are lower by 2.0, 1.6, and 1.9 points, respectively.
- Domain Adaptation and General Retention: On Finance, SSTD gives up 2.0 target points but gains 4.8 General Avg. points relative to Naive SFT.The paired scores are 59.5 versus 61.5 on target and 55.2 versus 50.4 on General Avg.
- Teacher Objective Ablation: Combining key-token guidance with distribution alignment yields the best ablation result, because the terms separately favor target gain and General Avg. recovery.Key-token guidance contributes +1.0 target gain versus +0.6 for alignment, while alignment contributes +1.1 General Avg. recovery versus +0.6 for key-token guidance.
- Student-Generated Supervision: Student on-policy prefixes improve the target score by 1.9 points and General Avg. by 1.4 points over offline teacher prefixes.They also exceed teacher rollouts by 0.8 target points and 3.1 General Avg. points.
- Student-Generated Supervision: Gold targets produce a +28.1 Finance gain but reduce General Avg. by 8.7 points, whereas student prefixes retain a +25.3 gain with a 2.6-point reduction.The comparison attributes the benefit to teacher feedback on prefixes produced by the student.
- Generalization Across Models: Across Qwen3 scales, SSTD reduces general-suite loss at every scale while remaining within 2.0 and 1.2 target points of Naive SFT at 1.7B and 4B.On Gemma, the target gap is 1.3 points at both sizes, while General Avg. increases by 3.9 and 4.2 points.
- Training Cost: SSTD costs 1.7× the GPU hours of Naive SFT but requires neither an external teacher nor general replay data.Compared with OPD, it uses 0.9× fewer GPU hours and avoids the external teacher.
6 Discussion and Limitations
SSTD’s benefits are bounded by the cost and capacity of its self-specialized teacher, by incomplete measurement of general capabilities, and by a trade-off between retention and needed domain change.
- Teacher construction may be unattractive when training is expensive, the target changes rapidly, or the base model lacks capacity to learn the domain.
- The held-out general suite measures selected behaviors and cannot certify that no other behavior changed.
- Stronger alignment can protect the general suite while preventing the teacher from expressing a required domain correction.
7 Conclusion
The paper studies single-domain post-training without broad replay data and proposes SSTD, which combines target-adapted teacher training with student-prefix distillation. It evaluates this procedure across finance, medicine, and law using controlled datasets, backbones, optimization budgets, and checkpoint selection.
- SSTD trains a target-adapted base-model copy with base-aware token weighting and distribution alignment, then distills it on student-generated prefixes.
- The evaluation covers financial numerical reasoning, medical multiple-choice question answering, and legal holding identification using official dataset splits.
- FinQA predictions are scored by executable program correctness, while MedMCQA and CaseHOLD use exact option-label accuracy.
- Experiments use Qwen3 and Gemma backbones, with student and teacher initialized from the same pretrained checkpoint and a frozen base reference.
- Teacher and student optimization uses fixed AdamW, bfloat16, sequence-length, batch, and update-budget configurations across methods within each domain.
- Checkpoint selection prioritizes target validation performance within 0.2 points of the best, then chooses the highest validation General Avg. and earliest tie.
A.4 Data Cleaning and Filtering
The appendix applies deterministic cleaning and filtering rules while preserving official labels, and fixes optimization configurations and update budgets before test evaluation.
- Duplicate training examples are removed after whitespace normalization, without filtering examples based on whether the base model answers correctly.
- FinQA examples are excluded for unparseable or unexecutable programs, missing table cells, or inconsistent released answers; other datasets require valid answer labels.
- The primary optimization configuration defines the candidate settings for Qwen3-1.7B experiments, while domain-specific update counts are reported separately.
- Teacher and student update counts are fixed before test evaluation.
- The financial prompt instructs the model to produce an executable reasoning program using only task-supported operators and constants.
B.1 Per-Domain and Per-Benchmark Results
The appendix decomposes General Avg. into its three component benchmarks and reports seed-level stability, supporting the main paper’s aggregate results and target–retention ordering.
- General Avg. is the arithmetic mean of MMLU, HellaSwag, and ARC-C, and the decomposed values reproduce the main-paper results.
- Variation across the three random seeds is small relative to the Naive SFT–SSTD differences, with unchanged target–retention ordering.
C.1 Ablation Results
The ablations show that alignment and key-token weighting require moderate settings: alignment initially improves retention without reducing target performance, while excessive values suppress domain adaptation.
- Student-generated prefixes are used throughout Stage II, while the teacher-objective panel varies only the objective used to construct the frozen teacher.
- Increasing the alignment coefficient initially improves retention without reducing the target score.Values above 0.2 increasingly restrict general drift but suppress the target-domain update, motivating λalign = 0.2 as the operating point.
- Moderate key-token weighting improves the target score while preserving the benefit of alignment.Excessive weighting overemphasizes a small number of high-loss positions and worsens both target and general performance.
D Computational Cost
The computational-cost analysis defines consistent accounting across methods and shows that SSTD costs more than Naive SFT but less than external-teacher OPD, without adding inference overhead.
- All computational costs are measured on Qwen3-1.7B Finance using eight NVIDIA A100 80GB GPUs under a common accounting convention.GPU-hours sum active device time, wall-clock time includes training through checkpoint writing, and peak memory is the maximum allocation on one GPU.
- SSTD uses approximately 1.7× the GPU-hours of Naive SFT and approximately 0.66× those of external-teacher OPD.Its additional cost comes from constructing the self-specialized teacher and querying it on student-generated states.
- The teacher is discarded after training, so SSTD does not change inference-time memory or latency of the final student model.
- Table 15 reports teacher queries at student-generated positions and counts all positions for which teacher logits are computed.
E.1 Positive On-Policy Training Trajectory
On-policy distillation can correct domain-critical decisions after student deviations, but token-level feedback cannot revise an incorrect operator already committed in the prefix and can be weakened by excessive alignment.
- Positive On-Policy Training Trajectory: The FinQA trajectory shows SSTD correcting a student’s premature subtraction by increasing probability on the required divide operation.The self-specialized teacher assigns divide probability 0.68 versus 0.17 for the frozen base, yielding the correct program and result 0.25.
- Positive On-Policy Training Trajectory: Student-generated prefixes expose the teacher to locally plausible mistakes, allowing guidance at the exact state reached after a deviation.
- Failure Cases: An early error can remove the evidence needed for the correct continuation, making the method less effective.
- Failure Cases: Token-level on-policy distillation can alter future decisions but cannot directly revise tokens already sampled into the prefix.Sequence-level correction, prefix rollback, and selective resampling are outside the current method’s scope.
- Failure Cases: With λalign = 1.0, the teacher can remain too close to an incorrect base distribution and fail to acquire a required medical correction.This preserves general behavior but reduces target-domain accuracy, consistent with stronger alignment improving General Avg. while reducing target performance.