Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation

1 Introduction

• QA-only answer supervision does not uniquely determine reasoning paths, so specialist optimization selects among trajectory distributions compatible with the same answers.The paper frames this selected behavior as the latent supervision passed to downstream students.Source: §1 · pp. 1–2
• Student distillation serves as an agnostic probe because students inherit generated trajectories, not the specialist’s parameters or optimization constraints.The controlled pipeline uses shared student initialization and training configurations, with equal amounts of sampled and filtered data.Source: §1 · pp. 2–3
• Across nine specialist–student pairs and 27 evaluations, specialization–generalization profiles align strongly (Spearman ρ = 0.9573, permutation test p −value = 0.0093), including across different model families.Students receive sampled trajectories rather than specialist parameters or optimization constraints.Source: §1 · p. 2
• Controlling specialist drift from its base model shifts both specialist and student between domain mastery and general-capability retention.LST provides implicit anchoring, while varying ASFT’s explicit anchoring strength moves both models along the same trade-off.Source: §1 · pp. 2–3

2 Related Works

• Prior specialist-distillation research studies domain adaptation, domain-specific supervision construction, and distillation-data design; this work instead examines trajectory-level ambiguity under QA-only specialist training.Source: §2 · p. 3
• Self-distillation studies reshape model outputs to transfer or preserve capabilities across continual learning, reasoning, multilingual, and code settings; this work complements it by studying QA-only specialist optimization.Source: §2 · p. 3

3 QA-only Specialist Distillation as Trajectory-Distribution Selection

• Specialist distillation adapts an origin model to a target domain, then uses the intermediate specialist to generate training data for a final model.The section studies this pipeline from a trajectory-learning perspective and reports that controlled optimization can induce high-quality reasoning trajectories under QA-only supervision.Source: §3 · p. 3

3.1 From QA-only supervision to trajectory learning in specialist distillation.

• QA-only teacher-forced SFT optimizes question–answer likelihood without explicitly optimizing or marginalizing reasoning paths, so multiple trajectory distributions can fit the same answer supervision.The analysis says higher-posterior answer-consistent trajectories contribute more to gradient updates, allowing repeated optimization to select a particular distribution.Source: §3.1 · p. 4
• Student distillation operationally probes the selected trajectories because students receive specialist-generated samples without its adapted parameters or optimization constraints.The resulting student behavior is largely determined by the specialist’s fixed trajectory distribution, without requiring direct characterization of the full distribution.Source: §3.1 · p. 4

3.2 Specialist Training as a Key Design Variable for Distillation Data

• Regulating drift from the origin model constrains specialist departure while allowing QA supervision to reshape trajectories, thereby shaping what students learn.The KL-constrained formulation characterizes reweighting of the base trajectory distribution; the paper says its training uses standard QA-only SFT without explicitly specifying the reweighting quantity or multiplier.Source: §3.2 · p. 5

4.1 Experimental Setup

• Specialists start from Qwen3-8B and use FFT, LoRA, or LST; students receive filtered specialist rationales and share equal data amounts and FFT training configurations.The setup also compares unconditional and answer-conditioned self-training baselines.Source: §4.1 · p. 6
• Experiments cover chemistry, university-level physics, and eight low-resource translation languages, with IT, ID, and OOD suites assessing task-matched, shifted, and broader performance.The chemistry, physics, and multilingual training and evaluation suites use different rationale-filtering criteria.Source: §4.1 · p. 6

4.2 Main Results

• Across 27 evaluations, specialist–student performance ranks align at Spearman ρ = 0.9573 (permutation p = 2/215 ≈ 0.0093).The relative ranking of tuning strategies is strongly reflected in downstream students.Source: §4.2 · pp. 7–8
• A chemistry cross-model transfer evaluation uses FFT- or LST-tuned Intern-S1-mini specialists to generate rationales distilled into Qwen3-8B.The supplied table caption identifies the evaluation setup, not its comparative outcome.Source: Table 3 · p. 7
• On all IT, scores of 29.54/92.75/34.17 surpass zero-shot performance of the 4× larger Qwen3-32B (25.35/92.45/33.78), highlighting latent-reasoning extraction as parameter-efficient versus scaling generalist models.Source: §4.2 · p. 7
• LST improves IT performance over the origin, modestly raises ID, and remains on par on OOD; FFT can sacrifice ID and OOD, while LoRA generally falls short of LST.The authors attribute these distinct profiles to different trajectory distributions induced by QA-only tuning.Source: §4.2 · p. 7
• On SMol, increasing filtered training data using LST Specialist rationales and FFT-distillation substantially improved IT performance, while ID remained largely stable and OOD showed only minimal degradation.Source: Figure 3 · p. 8
• In 12/27 splits, distilled students exceed their specialists, and distilled models consistently outperform both untuned self-training baselines.The authors also report that stronger specialists generally produce supervision leading to stronger students.Source: §4.2 · p. 8

5.1 Data scaling yields continuous in-task gains while preserving robustness.

• 19.3→32.5 IT as filtered distillation data grows from 50K to 360K; ID stays virtually unchanged and OOD degrades only modestly.The curated pool contains 360K examples, and the reported gains suggest scaling has not saturated at moderate data amounts.Source: §5.1 · p. 8

5.2 Students inherit their teachers’ generalization profiles across model families.

• The LST-S1m specialist yields a Qwen3-8B student with S1m-comparable robustness and +13.38 IT, while FFT-S1m maximizes IT but degrades ID/OOD.The student reflects its specialist’s profile despite the teacher and student belonging to different model families.Source: §5.2 · pp. 8–9

5.4 Explicit KL Drift Control with ASFT

• Increasing ASFT anchoring shifts specialists and distilled students toward better ID/OOD retention at the expense of task specialization in Chemistry and Multilingualism.The pattern appears in both teacher and student profiles; LST is shown as an implicit drift-control reference, not an ASFT setting.Source: Figure 4 · p. 9, §5.4 · pp. 9–10

5.3 Post-training strategies induce different degrees of KL drift.

• FFT produces the largest behavioral KL divergence from the origin model for both specialist and student; LoRA drifts less, and LST has the lowest divergence among adapted models.LST’s lower drift is consistent with implicit anchoring, but the authors do not claim it optimizes a KL-constrained objective.Source: §5.3 · p. 9

5.5 Structural constraint improves answer prediction without collapsing the trajectory.

• HR@1 = 0.94 for unconstrained FFT, versus approximately 0.85 for LST and LoRA, which provide calibrated answer-prediction gains without the same trajectory-distribution collapse.The measure uses teacher-forcing on 2,000 unseen SMol samples and suppresses intermediate reasoning steps.Source: §5.5 · p. 10

5.6 Structural constraint preserves the integrity of reasoning trajectories.

• 92.50% of LST trajectories were complete versus 8.90% for FFT, whose completed traces averaged 233 words versus 2,476 for the origin model.LoRA reached 41.39% completion; the self-rationalize baseline reached 16.18%, largely because it generated multiple reasoning blocks.Source: §5.6 · p. 10, Table 4 · p. 10

• Answer supervision leaves reasoning trajectories underdetermined, while specialist optimization determines the latent supervision passed downstream.Source: §6 · p. 10
• Across chemistry, physics, and low-resource multilingual tasks, distilled students closely inherit specialist specialization–generalization profiles, including across model families.Source: §6 · p. 10
• FFT induces larger drift and degraded generalization, while LST and explicit KL anchoring with ASFT provide implicit and explicit drift control along the same trade-off.The conclusion presents specialist optimization as a design variable for balancing domain specialization and general-capability retention when gold trajectories are unavailable.Source: §6 · p. 10

A Limitations

• Experiments focus on 8B models, leaving behavior at scales above 70B and in explicitly reasoning-optimized models open; exact-answer filtering can still admit wrong reasoning, especially in open-ended translation.Source: A Limitations · p. 18

B KL constraints enable controlled reweighting.

• The answer-level objective marginalizes over latent trajectories without specifying how probability mass is distributed among them.The trajectory-level lower bound relates answer likelihood to probability mass on answer-producing paths.Source: B KL constraints enable controlled… · p. 18
• KL-constrained updates reweight trajectories toward those supporting correct answers while keeping them close to the base policy.The constraint controls deviation from the base policy rather than specifying a particular trajectory distribution.Source: B KL constraints enable controlled… · p. 18

C.1 Ablation Study on LoRA Configurations

• Ranks 16, 32, and 64 perform relatively stably on ID and OOD, but rank 128 causes a noticeable OOD drop.Rank 32 has the highest IT score, while rank 64 has the best ID score and is nearly tied with ranks 16 and 32 on OOD; rank 64 is used as a capacity-balanced default.Source: §C.1 · p. 18
• At rank 64, standard LoRA yields a more balanced specialist and stronger distilled-model ID and OOD performance than RsLoRA.FFT distillation partly recovers RsLoRA's IT performance, but its OOD score remains close to zero.Source: §C.1 · p. 19, Table 6 · p. 19

C.2 Model Scaling: Using Qwen3-14B as Origin Model

• 23.80→59.10 IT for FFT contrasts with 23.80→35.76 for LST; FFT-distilled OOD falls to 3.46, while LST preserves ID and OOD capabilities.Both specialist variants and their distilled models outperform the Qwen3-14B origin on IT.Source: §C.2 · p. 19, Table 7 · p. 19
• FFT ranks above LST on IT and below it on OOD in both the 14B specialists and their distilled counterparts.The passage attributes this matching ranking to the trajectory distribution selected by the specialist tuning strategy.Source: Table 7 · p. 19, §C.2 · p. 20

C.3 Cross-scale Teacher–Student Gap

• Cross-scale 14B-to-8B distillation improves CHEM IT by +6.48 but reduces ID by −3.92 and OOD by −4.96.The comparison uses filtered, equally sized training data; the passage says same-base specialist distillation yields better overall trade-offs, suggesting teacher strength alone is insufficient.Source: §C.3 · p. 20, Figure 6 · p. 20

C.4 Additional Teacher-forcing Next-token Prediction Metrics: Probability and Rank

• 0.92 probability and 1.73 rank for the unconstrained FFT specialist indicate the reported extreme over-confidence and trajectory-distribution collapse.The text interprets this as answer-marginal overfitting rather than learning a generalizable reasoning process.Source: §C.4 · p. 20, Figure 7 · p. 21
• LST and LoRA specialist probabilities are around 0.83–0.84, with ranks around 10.4–11.8; the text links these calibrated values to smoother transfer and less downstream generalizability degradation.The passage attributes the controlled pattern to parameter-efficient tuning restricting the optimization space.Source: §C.4 · p. 20, Figure 7 · p. 21

D.1 Training

• Training uses a fixed effective batch size of 128 and seed 42 unless otherwise specified, with separate FFT, LST, and LoRA parameter configurations.FFT uses peak learning rate 1 × 10^-5; LST trains the bottom 4 and top 16 transformer layers; LoRA uses rank 64 and α = 2r.Source: §D.1 · p. 21

D.2 Test-time Decoding

• CoT generation uses vLLM and Transformers with sampling T = 0.6, TopP = 0.95, TopK = 20, and 3 candidates per CHEM/PHYS query or 1 per LRM query.The evaluation overview covers IT, ID, and shared OOD benchmarks; LRM evaluation uses eight low-resource languages and 16 en⇄xx subsets.Source: §D.2 · pp. 21–22, Table 8 · p. 22

E Details of Training Data and Evaluation Suites

• The evaluation suites span CHEM, PHYS, and LRM under Training, IT, and ID settings, plus shared OOD benchmarks for complex reasoning, math, and coding.Table 8 notes that SMol has 14 official subtasks and that LRM uses eight low-resource languages.Source: Table 8 · p. 22, E Details of Training Data and… · p. 22

E.1.1 Chemistry Training and IT Data — SMolInstruct

• SMolInstruct covers 14 molecular instruction tasks, and its rebalanced full training split contains about 3.6M instances.Rationale filtering applies task-specific checks, including exact canonicalized SMILES matches and an absolute error threshold of 1.0 for numerical predictions.Source: §E.1.1 · pp. 22–25

E.1.2 Chemistry ID Benchmarks

• Chemistry ID evaluation uses zero-shot accuracy on MMLU-Pro Chemistry, domain-relevant SuperGPQA subsets, and ChemBench.The listed benchmarks cover expert-level chemical knowledge, theoretical and applied problem solving, and multiple chemistry sub-disciplines.Source: §E.1.2 · pp. 25–26

E.2.1 Physics Training Data — MegaScience

• Physics data are drawn from MegaScience, with three independently seeded inferences per problem and sequential rule-based answer verification.The verification strategies include exact matching, numeric equivalence with 5% relative or 10^-8 absolute tolerance, and symbolic equivalence.Source: §E.2.1 · p. 26

E.2.2 Physics IT Benchmark — PHYSICS[Undergraduate/Postgraduate(Physics Major)]

• The evaluated PHYSICS subset targets undergraduate and postgraduate physics-major problems across five domains: mechanics, electromagnetism, thermodynamics, optics, and modern physics.It is selected to test expert-level physical reasoning rather than foundational concepts.Source: §E.2.2 · p. 27
• The hybrid Rule+Model framework applies answer-type-specific deterministic checks and falls back to an LLM judge when equivalence cannot be established confidently.Deterministic checks include numerical comparison, symbolic equivalence, MCQ matching, and bilingual True/False normalization; the judge handles open-ended or uncertain cases.Source: §E.2.2 · p. 27
• LLM-judge fallback rates ranged from 83.4% for Qwen3-14B to 85.9% for the LST specialist across 2,000 PHYSICS test problems.The similar rates across variants suggest fallback use is primarily determined by answer structure rather than model family.Source: §E.2.2 · p. 27
• A PHYSICS item receives full accuracy credit only when every sub-answer is judged correct against its corresponding reference.Partial credits are tracked, but strict overall Accuracy (Acc) is the primary reported metric.Source: §E.2.2 · pp. 27–28
• The unified prompt asks the model to act as an expert physicist, reason step-by-step, and provide final answers in LATEX boxed format.The judge prompt returns a binary CORRECT or INCORRECT verdict for mathematical or physical equivalence.Source: §E.2.2 · p. 28, Figure 10 · p. 28, Figure 11 · p. 28

E.2.3 Physics ID Benchmarks

• Physics in-domain evaluation uses zero-shot PIQA, AGIEval Gaokao Physics, and MMLU High School Physics, plus 5-shot MMLU-Pro Physics with Exact Match.The section introduces five rigorous benchmarks, though the supplied descriptions specify these four.Source: §E.2.3 · pp. 28–29

E.3.1 Language Selection

• The multilingual experiments select eight low-resource BenchMAX languages spanning five writing systems and six language families.The selection targets languages in which Qwen3 is relatively undertrained.Source: §E.3.1 · p. 29, Table 10 · p. 29

E.3.2 LRM Training Data— OPUS

• LRM training uses English↔target parallel pairs from Lego-MT for the eight languages, retaining the top 20% in each direction by spBLEU score.Training-set sizes vary across languages with data availability.Source: §E.3.2 · pp. 29–30

E.3.3 LRM IT Benchmark — Flores-101

• Flores-101 evaluates both English→target and target→English translation using its professionally translated devtest set.It is used as a clean, high-quality test set despite training data coming from OPUS.Source: §E.3.3 · p. 30

E.4 LRM ID Benchmarks

• Low-resource ID evaluation averages four IFEval accuracies across eight languages and reports GPQA performance averaged across those same languages.IFEval uses BenchMAX’s multilingual translation; GPQA tests graduate-level science reasoning in translated datasets.Source: §E.4 · p. 30

E.5 OOD Benchmarks

• The OOD score is a macro-average of reasoning, mathematics, and coding domain scores, giving each domain equal weight rather than averaging five benchmarks flatly.The component protocols are harmonic mean for BBEH, average accuracy for AIME, and Pass@1 for LiveCodeBench.Source: §E.5 · pp. 30–31

E.5.1 Complex Reasoning — BIG-Bench Extra Hard; E.5.2 Mathematics — AIME 2025 and 2026; E.5.3 Coding — LiveCodeBench

• OOD tests use BBEH for complex reasoning, 32-run average accuracy on AIME 2025/2026, and Pass@1 on LiveCodeBench V5 and V6.The AIME runs address variance in generated derivations; LiveCodeBench uses recent temporal splits to reduce contamination.Source: §E.5.1 · p. 31, §E.5.2 · p. 31, §E.5.3 · p. 31

NeurIPS Paper Checklist

• The checklist says the introduction’s claims are supported by the theoretical formulation and multi-domain evaluations, while noting their stated scope.The claims concern QA-only supervision underdetermining trajectory learning and optimization strategies implicitly resolving that ambiguity.Source: NeurIPS Paper Checklist · p. 33

2. Limitations

• The checklist marks limitations as discussed and directs readers to § 4.1 and Appendices D and E.The supplied checklist response does not specify the limitations themselves.Source: §2 · pp. 33–34
• Statistical tests are reported for § 4.2, while the authors say other reported accuracy values do not support error or significance calculations.This qualifies the statistical reporting available for results beyond § 4.2.Source: §2 · p. 35
• The paper describes its scope as theoretical analysis and empirical performance evaluation, without concrete real-world deployment analysis.The checklist uses this scope to explain why societal impacts are marked not applicable.Source: §2 · p. 36