Source-linked AI summary
Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis
Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev, Maksim Kuznetsov, Mathieu Reymond, Vladimir Aladinskiy, Alex Aliper, Alex Zhavoronkov
TL;DR
Single-step retrosynthesis is intrinsically one-to-many, but single-answer evaluation captures this poorly. The paper introduces Top-K, plausibility-aware C3LM training and reports improved performance over conventional models on URSA-expert-2026.
Problem
Single-step retrosynthesis is intrinsically one-to-many, yet single-answer evaluation and benchmarking poorly capture its diverse plausible predictions.
Method
The study trains C3LM with Top-K prompting on 45.6 million verified reactions, then applies ChemCensor-based and novelty-oriented rewards.
Results
The resulting C3LM improves performance over conventional single-step retrosynthesis models and other LLMs, including on URSA-expert-2026.
Takeaways & Limitations
Top-K, plausibility-aware training supports more diverse reaction generation and motivates ensemble-based retrosynthesis systems combining complementary model spaces.
Takeaways & Limitations
ChemCensor is only a proxy for chemical plausibility and does not fully account for practical laboratory parameters such as conditions, solvents, and purification.
Abstract
from arXiv · showhide
Single-step retrosynthesis is a central component of computer-aided synthesis planning, yet its intrinsically one-to-many nature is poorly captured by single-answer evaluation and benchmarking protocols. To address this, we introduce Top-K prompting as a robust training and inference paradigm to better capture diverse, plausible reaction predictions. We compile CREED-CCV-2+USPTO-XL, an ultra-large-scale dataset of ~45.6 million verified reactions to train the C3LM (Chemistry Constraint-Consistent Language Model). By integrating fine-tuning with ChemCensor-based and novelty-oriented rewards, our model achieves state-of-the-art performance on the OOD URSA-expert-2026 benchmark. Further analysis of reaction uniqueness shows that LLMs and conventional models explore complementary reaction spaces, motivating ensemble-based retrosynthesis systems. Overall, our results establish Top-K, plausibility-aware training as a practical new direction for robust future LLM-based synthesis planning.
1 Introduction
The paper presents Top-K prompting and plausibility-aware training as a way to improve diversity and chemical plausibility in single-step retrosynthesis. It introduces a 45.6M-reaction dataset and trains C3LM to outperform conventional models on URSA-expert-2026.
- Motivation: Single-step retrosynthesis proposes reactions and precursors, while multistep retrosynthesis assembles individual reactions into complete synthetic routes.These tasks form complementary components of computer-aided synthesis planning.
- Prior work: A URSA benchmark using ChemCensor-based chemical plausibility found LLM retrosynthesis models promising but below the best conventional solutions.URSA contrasts ChemCensor evaluation with Top-K accuracy.
- Contributions: ∼45.6M verified reactions comprise the newly constructed CREED-CCV-2+USPTO-XL dataset, derived from expert-coded templates.The dataset supports training the new C3LM version.
- Contributions: C3LM was trained on CREED-CCV-2+USPTO-XL and fine-tuned with ChemCensor-based and novelty-oriented rewards to improve plausibility and diversity.The study reports that this model outperforms the best conventional SSRS models on URSA-expert-2026.
- Evaluation considerations: ChemCensor-based filtering and rewards introduce partial circularity with ChemCensor evaluation, but directly target plausibility and precedent support for practical use.The authors explicitly acknowledge this evaluation-training overlap.
- Contributions: Top-K mode is proposed for LLM training and prompting to increase the diversity of generated reactions.The paper frames Top-K as a best practice for addressing diverse reaction predictions.
2 Approach
The approach evaluates LLM retrosynthesis under Top-1 and Top-K prompting paradigms while training on chemically verified, expanded reaction datasets designed to represent multiple synthesis routes per product.
- Prompting paradigms: Top-1 prompting asks for one most plausible reactant set from 15 MolInstructions-adapted templates and serves as the zero-shot baseline.Each target is prompted 15 times with randomly selected templates, yielding 15 collected reactant sets per molecule.
- Prompting paradigms: Top-K prompting appends “Give me 15 different answers” to each baseline template to simulate conventional models and capture one-to-many retrosynthetic disconnections.The mode is intended to represent cases where a target molecule can be synthesized through multiple independent disconnections.
- Training data: The decontaminated USPTO-full set contains unique ∼897K products and ∼951K reactions, while CREED-CCV contains unique ∼699K products and ∼6.4M reactions.CREED-CCV uses a virtual synthesis engine and ChemCensor v.0.5.2 for chemical plausibility verification.
- Training data: CREED-CCV-2 merges deduplicated reactions from ChEMBL compounds processed by the Virtual Synthesis Engine and reactions from CREED-CCV, then scores them with ChemCensor v.1.1.1.The resulting set contains unique ∼2.9M products and ∼36M reactions.
- Training data: USPTO-XL expands unique USPTO products with VSE-enumerated reactants and ChemCensor v.1.1.1 verification to address USPTO’s predominance of one reaction per product.The final set contains unique ∼859K products and ∼10.6M reactions.
3 Experiments
The experiments train C3LM with Top-K supervised fine-tuning and GRPO-based reinforcement learning, then benchmark C3LM, foundation LLMs, and conventional SSRS models using ChemCensor-based evaluation. Evaluation spans URSA-expert-2026 and USPTO-50K-test-mini, with reaction candidates generated and ranked under defined Top-K protocols.
- Training setups: C3LM is fine-tuned in Top-K mode on either CREED-CCV+USPTO or CREED-CCV-2+USPTO-XL.The latter dataset contains 3,680,906 unique products and 45,649,785 unique reactions after merging and deduplication.
- Training setups: C3LM-LFM2-CREED-CCV-2+USPTO-XL undergoes online reinforcement learning fine-tuning with single-reward GRPO.RFT uses group size 8, sampling temperature 1, and KL-regularization weight 0.1; its reward combines six components.
- Benchmarking: The benchmark evaluates foundation LLMs, conventional SSRS models, and C3LMs on URSA-expert-2026 and USPTO-50K-test-mini using ChemCensor v.1.1.1.The study benchmarks both proprietary and open-weight general-purpose LLMs alongside conventional SSRS models.
- Benchmarking: LLMs generate either 15 independent Top-1 responses or three responses from one random Top-15 prompt, while conventional SSRS models generate 15 reactions per product.The resulting reactions are evaluated with CC-aggregated metrics; the Top-K protocol uses K=15.
- Benchmarking: The benchmark uses public USPTO-full and combined USPTO-full with commercial Pistachio synthetic-precedent databases.These are described as major and supplemental databases, respectively.
- Benchmarking: To examine plausibility and diversity frontiers, reactions from all models are pooled, CC-sorted, and reduced to the best 15 reactant sets before metric calculation.The retained candidates are evaluated using CC-metrics as usual.
4 Results
Top-K prompting and plausibility-aware training substantially improve reaction diversity and ChemCensor-based performance, with scaling and reward fine-tuning producing strong benchmark results. Analyses also show that LLMs and conventional retrosynthesis models cover complementary plausible reaction spaces, while benchmark leakage and one-to-many evaluation limit definitive frontier estimates.
- Top-K vs Top-1 prompting: Most models benefit significantly from Top-1 to Top-K prompting, while model rankings shift between modes, with Gemini 3.1 Pro improving dramatically in Top-15.Av. PT-Top-10 is identified as the most sensitive CC-aggregated metric to generated-reaction diversity.
- Top-K mode for the C3LM training: Training in Top-K mode boosts all CC-based metrics, with the largest improvement exceeding 2.5-fold for Av. PT-Top-10.The matched Top-1 →Top-K transition improves Max/@3/@5/@10 by +0.30/ + 0.62/ + 0.70/ + 0.60, respectively.
- Scaling training set and RFT: Scaling from CREED-CCV+USPTO to CREED-CCV-2+USPTO-XL increases training-set size by > 6 times and improves Max/@3/@5/@10 by +0.12/ + 0.13/ + 0.17/ + 0.29.ChemCensor and Novelty reward fine-tuning then yields RFT-model dominance across all CC metrics on URSA-expert-2026.
- Scaling training set and RFT: Av. PT-Top-10 on USPTO-50K-test-mini is presented as less vulnerable to leakage, whereas conventional models’ Av. PT-Max CC values close to 5 warrant caution.481/497 product molecules have no more than 3 reactions in public USPTO-full, while values close to 5 indicate exact matches to synthetic precedents.
- Frontier of plausibility and diversity: LLMs and conventional models generate substantially different plausible reaction spaces, supporting ensemble systems to cover broader chemical space.Every LLM predicts some plausible reactions absent from MHNreact, and unique and intersecting reaction proportions generally increase for newer GP LLM versions; the ultimate frontier remains unknown.
5 Ethical considerations
CASP tools have significant dual-use potential because automated synthetic-route design could be repurposed to facilitate hazardous-compound synthesis. The study addresses these risks by focusing on medicinal-chemistry planning with novel molecules and using ChemCensor’s open patent-literature foundation.
- Dual-use potential: CASP tools carry dual-use risks because automated design of synthetic routes for complex small molecules could facilitate hazardous-compound synthesis.The paper frames this as a potential consequence of developing and deploying CASP technologies.
- Risk mitigation: The benchmarking targets legitimate medicinal-chemistry synthesis planning using novel molecular structures that do not overlap with known hazardous-compound databases.This focus is presented as a risk-mitigation measure.
- ChemCensor safeguards: ChemCensor is built on established open-scientific data from patent literature and does not inherently provide instructions for synthesizing toxic or regulated substances.The framework’s data source and stated limitations are central to the paper’s ethical positioning.
6 Limitations … E C3LM Reinforcement Learning Fine-Tuning Details
The paper acknowledges limitations in metric alignment and ChemCensor’s incomplete coverage of laboratory practicality, while documenting the C3LM model inventory, terminology, baselines, supervised fine-tuning, and reinforcement fine-tuning procedures.
- 6 Limitations: The evaluation criteria partly align with the optimization objectives, while ChemCensor remains an imperfect proxy for practical laboratory organic chemistry.The authors characterize this circularity as a deliberate design choice focused on practitioner utility.
- A C3LM Family Inventory: Table 2 distinguishes C3LM training datasets as TD-1, CREED-CCV + USPTO, and TD-2, CREED-CCV-2 + USPTO-XL.A prior-work model is re-scored under ChemCensor v1.1.1 without retraining for comparison.
- B Glossary: The glossary defines CREED-CCV as the ChemCensor-verified subset of ∼22.7M CREED reactions, retaining candidates with CC Score > 0.CREED is designed to expose multiple plausible single-step disconnections per product rather than a single ground truth.
- C Baselines: The evaluation compares proprietary and open-weight foundation models, including Grok, Gemini, GPT, Claude, DeepSeek, and Qwen families.The cited baseline inventory lists model versions and their associated organizations and years.
- D C3LM Supervised Fine-Tuning Details: Supervised fine-tuning packs multiple training sequences into GPU context windows totaling 524,288 tokens and uses specialized SMILES tokens.The vocabulary extension separates chemical tokens from natural-language tokens; SMILES tokenization is always used in outputs and with 0.5 probability for inputs.
- E C3LM Reinforcement Learning Fine-Tuning Details: Online reinforcement fine-tuning uses single-reward GRPO on the CREED-CCV-2+USPTO-XL model for 1000 steps with learning rate 10^-6, group size 8, and 64 groups per step.Sampling temperature is 1, KL-regularization weight is 0.1, and training uses the same CREED split as supervised fine-tuning.
- E C3LM Reinforcement Learning Fine-Tuning Details: The GRPO reward combines formatting, molecular syntax, ChemCensor plausibility, Top-K uniqueness and matching, and novelty, with ChemCensor and novelty each weighted 1.0.Novelty rewards plausible reactants absent from CREED, while Top-K uniqueness rewards distinct canonical SMILES solutions.
F Full Plausibility-Based Top-K Evaluation
The section evaluates single-step retrosynthesis in Top-K mode using full plausibility-based metrics derived from ChemCensor scores over targets and unique predictions.
- Evaluation metrics: Full plausibility-based evaluation uses per-target maximum and per-TM average ChemCensor scores for Top-K unique predictions.ChemCensor v1.1.1 is used, with USPTO-full as the source of synthetic precedents.
G Plausibility-Based Top-1 (Single-Answer) Results
Top-1 predictions come from the original benchmark and are re-scored with ChemCensor v1.1.1 rather than regenerated, enabling direct comparison with Top-K results. Table 4 evaluates plausibility using per-target maximum and per-TM average ChemCensor scores.
- Evaluation protocol: Top-1 predictions were taken from the original benchmark and re-scored under ChemCensor v1.1.1 for direct comparability with Top-K results.The predictions were not regenerated.
- Evaluation metrics: Table 4 reports Max as the per-target maximum ChemCensor score averaged over TMs and Av. PT-Top-K CC as the per-TM average over top-K unique predictions.Both metrics use ChemCensor v1.1.1.
H Reaction Intersection with the Conventional Model MHNreact
The analysis compares model-predicted reactions with MHNreact to assess complementary plausible reaction coverage. Most models produce fewer unique reactions than MHNreact, while two strongest C3LM variants extend the plausible reaction space beyond this conventional baseline.
- Intersection analysis: Predictions are partitioned into reactions shared with MHNreact, unique to each model, and the per-target difference between model-unique and MHNreact-unique reactions.Table 5 reports these quantities for every model on URSA-expert-2026.
- Intersection analysis: Most models, including all foundation LLMs, produce fewer unique reactions than MHNreact, yielding ∆ < 0.Figure 5 visualizes the intersection partition across models.
- Intersection analysis: Only C3LM-LFM2-CREED-CCV-2+USPTO-XL and C3LM-LFM2-RFT-CC-NR generate more unique plausible reactions than MHNreact, yielding ∆ > 0.These variants therefore extend the plausible reaction space beyond the conventional baseline.
- Representative example: Figure 6 illustrates the comparison for target X404-1768-5005 using reactions predicted only by each model and by both models.The example contrasts C3LM-LFM2-CREED-CCV-2+USPTO-XL with MHNreact.
I ChemCensor Metrics Calculated with USPTO ∪Pistachio Reference dataset
The study re-scores the same model predictions with ChemCensor v1.1.1 using the USPTO-full ∪ Pistachio Q3 2023 reference database. Although absolute plausibility scores change, the relative model ordering is largely preserved.
- Reference-database robustness: ChemCensor v1.1.1 re-scores the same predictions against USPTO-full ∪ Pistachio Q3 2023 to test reference-database dependence.The main results instead use ChemCensor’s default USPTO-full database.
- Reference-database robustness: Absolute plausibility scores change under the expanded reference database, but the relative ordering of models is largely preserved.This comparison supports the robustness of the study’s conclusions to the reference-database choice.
- Evaluation metrics: Table 6 evaluates Top-K retrosynthesis using per-target maximum and per-TM average ChemCensor scores over top-K unique predictions.Both metrics use the ChemCensor-U2P2 reference database and ChemCensor v1.1.1.
J Results under ChemCensor v0.5.2 · K Distribution of Reactant Sets per Product in USPTO-50K-test-mini · L CREED-CCV-2+USPTO-XL Details
ChemCensor v0.5.2 rescoring largely preserves model rankings despite slight absolute-score shifts, while USPTO-50K-test-mini products usually have one reference reactant set. CREED-CCV-2+USPTO-XL expands substantially over CREED-CCV in products and reaction candidates.
- J Results under ChemCensor v0.5.2: ChemCensor v0.5.2 rescoring slightly shifts absolute values but largely preserves the relative ordering of models.The evaluation setup matches Table 1, with only the plausibility scoring function changed.
- J Results under ChemCensor v0.5.2: Table 7 evaluates single-step retrosynthesis Top-K predictions using ChemCensor v0.5.2.It reports Max, the per-target maximum score averaged over TMs, and Av. PT-Top-K CC, the per-TM average score over top-K unique predictions.
- K Distribution of Reactant Sets per Product in USPTO-50K-test-mini: 420 of 497 USPTO-50K-test-mini products are associated with a single reference reaction.Reference reactant sets were collected by matching each product against the entire USPTO-full corpus.
- K Distribution of Reactant Sets per Product in USPTO-50K-test-mini: Only a small tail of USPTO-50K-test-mini products has two or more distinct reference reactant sets.The distribution therefore remains concentrated on products with a single reference reaction, despite searching USPTO-full.
- L CREED-CCV-2+USPTO-XL Details: CREED-CCV-2+USPTO-XL contains 3,680,906 unique products and 45,649,785 reaction candidates.The dataset is described as substantially larger and denser than the original CREED-CCV.
- L CREED-CCV-2+USPTO-XL Details: The expanded dataset provides roughly 5.3× more products and 7.2× more reactions than CREED-CCV.CREED-CCV contains 698,765 products and 6,368,986 reaction candidates, compared with the expanded dataset’s 3,680,906 and 45,649,785.
- L CREED-CCV-2+USPTO-XL Details: CREED-CCV-2+USPTO-XL averages ∼12.4 candidates per product, versus ∼9.11 for CREED-CCV.Both datasets use the same 0.8/0.1/0.1 product-disjoint split.