CoTranslator: Reverse to Forward Chain-of-Thought Translator for Reasoning Distillation

• Up to 5.82 percentage points: CoTranslator improves average accuracy over semantic suppression; two 4B students retain 95.0% of RS average accuracy at 35.9% of its estimated compute.Source: Figure 1 · p. 1, §1 · p. 2
• Verified CoT data matters for distillation, yet large-scale teacher generation and checking are costly; forward rejection sampling may need many attempts, while reverse generation risks post-hoc rationalization and train-inference mismatch.Source: §1 · pp. 1–2
• CoTranslator trains on paired reverse and accepted forward responses from a small question subset, then translates reverse responses at scale using one RCG teacher call and one translator call per question.Source: §1 · p. 2
• Without retraining, the translator transfers to a new math dataset, a different teacher, and STEM problems, with average STEM gains of 4.88 and 4.31 percentage points for 4B and 8B students, respectively.Source: §1 · p. 2

2.1 Source and target distributions

• RCG conditions on the question and correct answer, whereas FCG uses the question alone and rejection sampling keeps responses whose extracted answer is correct.Source: §2.1 · pp. 2–3
• The translator maps RCG source responses to an output distribution intended to match rejection-sampled responses for the same question and answer.Source: §2.1 · p. 3

2.2 Data construction, training and inference

• For each question, paired training uses an RCG sample and the first question-only FCG response with a correct extracted answer; the RCG response is input and the accepted response is target.Source: §2.2 · p. 3
• The translator is conditioned only on the RCG response; question and answer are used for pairing and verification, preventing a direct question-answer bypass.Source: §2.2 · p. 3
• Student data uses one teacher call and one translator call per question, after which the student is fine-tuned to generate the translated responses from questions.Source: §2.2 · p. 3

3.1 Experimental setup

• The translator is trained on 35.7k paired examples and applied to approximately 285.5k records; evaluation covers six student models and six math benchmarks.Translator targets are rejection-sampled and verified against ground-truth answers; inputs use Semantic Suppression responses.Source: §3.1 · p. 3

3.2 Main result

• CoTranslator improves average math accuracy over SUP for every student model, with gains across both Qwen3 and Qwen2.5-Math families.The 4B Base student also exceeds its RS-trained counterpart in average accuracy.Source: Table 1 · p. 4, §3.2 · p. 5

3.3 Transfer experiments

• 4.76 and 1.66 percentage points are the respective average-accuracy gains over SUP for 4B and 8B students on the new math dataset.With Kimi-K3 as teacher, the gains are 18.73 and 19.02 percentage points, respectively.Source: §3.3 · p. 5, Table 2 · p. 5
• 4.88 and 4.31 percentage points are the respective average STEM-accuracy gains over SUP for 4B and 8B students using the math-trained translator.The transfer experiment uses 221.6k science problems and evaluates OlympiadBench, SciBench, and AA-Omniscience; paired STEM training could further improve performance.Source: §3.3 · pp. 5–6, Table 3 · p. 5

4.1 Similarity to RS responses

• CoTranslator has the highest MAUVE and cosine similarity and lowest MMD across semantic and lexical features at every difficulty level.Difficulty is indicated by rejection-sampling attempts; semantic features encode response tokens, while lexical features use TFIDF-weighted word and symbol counts.Source: §4.1 · p. 6, Figure 3 · p. 6

4.2 Post-hoc rationalization

• CoTranslator reduces expected-answer mentions and answer-dependent shortcuts, increases independent derivations, and delays the first answer for both teachers.The analysis also tracks suppression mentions and measures first-answer position relative to response length.Source: §4.2 · pp. 6–7, Figure 4 · p. 7, Figure 5 · p. 7

4.3 Translator and training data size

• The 4B and 8B translators learn effective trajectory translation, with performance varying more by translator size than training-data size.Both larger translators perform well with 35.7k examples, but removing paired training severely reduces performance; the 4B model outperforms the 8B model at matched data sizes.Source: §4.3 · pp. 7–8, Figure 6 · p. 7

4.4 Computational cost

• With 35.9% of rejection sampling’s main-experiment compute, CoTranslator achieves 95.0% of its mean accuracy across the two 4B students.Compute estimates include paired-data construction, translator training, teacher inference, and translator inference.Source: §4.4 · p. 8

CoT for reasoning

• Chain-of-thought prompting uses intermediate reasoning steps, while distillation trains smaller models on teacher responses through fine-tuning, specialization, or decomposition.Prior work also explores rationale supervision, bootstrapping, correctness filtering, and compact reasoning models.Source: §5 · p. 8

Reverse CoT generation

• Reverse CoT generation reconstructs reasoning from a question and known answer; prior approaches include STaR, Reinforcement-Learned Teachers, REER, and RAVR.The passage notes concerns about post-hoc rationalization and CoT faithfulness (Lanham et al., 2023).Source: §5 · p. 8
• SUP conceals the answer and SSR uses an answer-invariant skeleton, while answer-conditioned responses can still harm verifiable-reasoning distillation after final-answer filtering.The passage attributes these approaches to Wang et al. (2026), Peng et al. (2026), and Lee et al. (2026), respectively.Source: §5 · p. 8

6 Conclusions

• CoTranslator learns from a small paired set to translate reverse CoT into reasoning that approximates rejection-sampled forward CoT.The conclusion reports improved student performance across model sizes and families, with transfer to new datasets, teachers, and STEM.Source: §6 · p. 8
• The method saves 64.1% of compute while achieving 95.0% of rejection sampling’s mean accuracy for 4B students in the main experiment.It also saves 82.0% of compute in transfer experiments.Source: §6 · pp. 8–9

Limitations

• Experiments cover students up to 8B parameters and supervised fine-tuning; applicability to larger models, reinforcement learning, and on-policy distillation remains unexplored.Source: Limitations · p. 9

A.1 Datasets

• The main translator uses about 35.7k paired examples from a 1/8 subset of the cot data, while all cot problems supply student training data.The dataset contains 285,516 cot-subset records; its solutions are retained when final answers match references.Source: §A.1 · p. 12
• Transfer tests use all 459,646 ODA-Math-460k records and a 221,598-problem scientific vendor subset for STEM teacher responses.The science subset covers physics, chemistry, and biology.Source: §A.1 · p. 12

A.2 Models

• DeepSeek-V4-Pro is the main teacher and supplies verified FCG translator targets; Kimi-K3 tests transfer across teachers.DeepSeek-V4-Pro has 1.6T parameters with 49B activated, while Kimi-K3 has 2.8T parameters with 104B activated.Source: §A.2 · p. 12
• Qwen3-4B-Thinking-2507 is the main translator, and students include Qwen3 models plus Qwen2.5-Math-7B variants.Qwen3-1.7B and Qwen3-8B are translator-size alternatives.Source: §A.2 · p. 12

A.3 Benchmarks

• Mathematical reasoning is evaluated on competition and olympiad benchmarks, while STEM transfer uses OlympiadBench’s STEM subset, SciBench, and AA-Omniscience.The selected benchmark descriptions include diverse olympiad problems and college-level open-ended scientific problems.Source: §A.3 · p. 12

A.4 Hyperparameters

• Training uses seed 42 and NVIDIA H800 GPUs, with 32k-token student and 64k-token translator contexts; inference follows recommended or default sampling settings.RoPE scaling supports the required context lengths for models with shorter native windows.Source: §A.4 · p. 13

B Substantive AI assistance information

• AI tools assisted manuscript writing, some code implementation and debugging, and literature searches, but not research ideas, methodology, experimental design, or conclusions.Source: B Substantive AI assistance information · p. 13

C Mathematical analysis

• The population analysis assumes independent candidate sampling, independent source and target generation given context, independent forward attempts, and exact answer checking.It also assumes positive teacher acceptance probability on candidate support and finite population losses and entropies.Source: C Mathematical analysis · p. 13

C.1 Pairs preserve the target law

• An attempt limit changes which contexts are accepted but preserves the conditional law of accepted forward responses; paired samples therefore retain verified question-only targets.The source remains answer-conditioned even though the accepted target is question-only.Source: §C.1 · p. 14

C.2 Likelihood enables recovery

• Likelihood training has the forward-response conditional distribution as its unrestricted population optimum, and the recovery error separates source information loss from fitting error.When the source contains enough task information and population fitting is exact, recovery is exact for almost every accepted context.Source: §C.2 · p. 14
• Small training loss alone does not establish recovery on unseen pairs, and the stated response-probability bound is an average population guarantee.Source: §C.2 · p. 14

C.3 Recovery controls student risk

• The student-risk guarantee applies to bounded losses, not raw unbounded student NLL, and verified answers do not certify causal faithfulness.The bound accounts for response recovery, decoding, and question coverage.Source: §C.3 · p. 15

D.1 Translator and training data size

• The translator-size and data-size analysis uses a Qwen3-4B-Base student on the same held-out data, comparing untranslated SUP responses with responses translated by different models.Table 7 labels paired-data fractions as 1/8, 1/4, and 1/2, corresponding to 35.7k, 71.4k, and 142.8k examples; zero uses no paired training examples.Source: §D.1 · p. 15, Table 7 · p. 16

D.2 Similarity to RS responses

• Response similarity compares SSR, SUP, SUP-AUG, and CoTranslator with RS across difficulty groups using NLL, semantic and lexical features, MAUVE, MMD, and cosine similarity.Difficulty is based on a separate 16-attempt rejection-sampling run, with more than 16 denoting no success.Source: §D.2 · p. 15

D.3 Computational cost

• Compute budgets cover paired-data construction, translator training, teacher and translator inference for the main experiment; transfer budgets reuse the translator, and Math-Verify verification is excluded.Source: §D.3 · p. 15, Table 8 · p. 16

D.4 Consistency across training settings

• CoTranslator improves average accuracy over SUP across training settings, two model families, and 4B–8B sizes, with gains also reported for dataset, teacher, and STEM transfer.The same translator transfers without retraining.Source: §D.4 · p. 16

E Prompts

• The appendix provides prompt figures for SUP, SUP-AUG, the translator, and the post-hoc rationalization rubric.Source: Figure 9 · p. 17, Figure 10 · p. 17, Figure 11 · p. 17