Source-linked AI summary
Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction
Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Costas Bekas, Alpha A Lee
TL;DR
The paper addresses predicting products from reactants and reagents, a central step in synthetic planning. It uses a multi-head attention Molecular Transformer and reports top-1 accuracy above 90%, while noting costs for ensemble-based improvements.
Problem
The paper focuses on predicting products given reactants and reagents, an important unsolved step in synthesis planning.
Method
The approach treats reaction prediction as machine translation and uses multi-head attention in the Molecular Transformer.
Results
Top-1 accuracy reaches 91%, while another reported result exceeds 93% top-1 accuracy.
Takeaways & Limitations
SMILES data augmentation significantly increases accuracy, supporting the model’s performance on reaction prediction.
Takeaways & Limitations
Model ensembles can improve accuracy and uncertainty estimation but add training and/or test-time cost, while template validity is questionable.
Abstract
from arXiv · showhide
Organic synthesis is one of the key stumbling blocks in medicinal chemistry. A necessary yet unsolved step in planning synthesis is solving the forward problem: given reactants and reagents, predict the products. Similar to other work, we treat reaction prediction as a machine translation problem between SMILES strings of reactants-reagents and the products. We show that a multi-head attention Molecular Transformer model outperforms all algorithms in the literature, achieving a top-1 accuracy above 90% on a common benchmark dataset. Our algorithm requires no handcrafted rules, and accurately predicts subtle chemical transformations. Crucially, our model can accurately estimate its own uncertainty, with an uncertainty score that is 89% accurate in terms of classifying whether a prediction is correct. Furthermore, we show that the model is able to handle inputs without reactant-reagent split and including stereochemistry, which makes our method universally applicable.
Introduction
The paper frames product prediction from reactants and reagents as a central, difficult step in organic-synthesis planning. It proposes a rule-free, attention-based SMILES model that predicts products accurately and estimates prediction uncertainty.
- Motivation: Accurate predictions and reliable uncertainty estimates are needed to assess synthesis-path risk and prioritize risky steps early.The stated goal is to fail fast and fail cheaply by placing riskier steps at the beginning of a synthesis.
- Related approaches: Template-based methods depend on templates whose validity, scalability, and atom mappings remain problematic.Automatic approaches still use reaction classes, pre-existing atom mappings, or expert rules, creating a circular dependence between templates and mappings.
- Related approaches: Sequence-based methods formulate reaction prediction as machine translation from reactant–reagent SMILES to product SMILES.This represents reactions as text sequences rather than requiring graph edits or handcrafted reaction templates.
- Contribution: The Molecular Transformer uses multi-head attention and SMILES input without atom mapping or handcrafted rules.The model is designed to handle mixed reactant–reagent inputs and stereochemistry while predicting subtle chemical transformations.
- Results: 90.4% top-1 accuracy and 93.7% top-2 accuracy were achieved on a common benchmark dataset, while uncertainty classification reached ROC-AUC 0.89.The model’s uncertainty score classifies whether a reaction prediction is correct, providing a risk estimate alongside the prediction.
Data
The experiments compare filtered USPTO-derived datasets, including versions that preserve stereochemistry, and evaluate separated versus mixed reactant–reagent preprocessing. Mixed preprocessing is harder because the model must identify reaction centers across more molecules.
- Datasets: Four datasets are used to compare the Molecular Transformer with previous sequence-to-sequence work, including USPTO_STEREO with stereochemical information retained.USPTO_STEREO underwent less filtering than the MIT and LEF datasets.
- Datasets: A non-public time-split Pistachio test set provides a comparison on more diverse reactions against a previous seq-2-seq model.The split is described as an additional test set rather than one of the four main comparison datasets.
- Preprocessing: Separated preprocessing marks reactants from reagents, where reactants contribute product atoms while reagents do not.The separation is represented with a > token between the two groups.
- Preprocessing: Mixed preprocessing removes the reactant–reagent distinction because that separation is not always obvious and can vary across tools.The model must determine the reaction center from a larger number of molecules, making the task significantly harder.
- Preprocessing: The inputs are canonicalized with RDKit and tokenized directly rather than replacing reagents with special reagent tokens.The tokenization follows a regular expression used in prior work.
The Molecular Transformer
The Molecular Transformer is a transformer-based, autoregressive encoder-decoder model that replaces recurrent processing with multi-head attention and positional encodings. Its attention architecture processes multiple tokens simultaneously while combining source information with previously generated target outputs.
- Architecture: The model is a step-wise autoregressive encoder-decoder composed of multi-head attention and positional feed-forward layers.The encoder creates a hidden representation of the input, while the decoder generates outputs autoregressively.
- Architecture: The decoder combines source-sequence information with the target sequence produced so far.Its first attention layer attends encoder outputs and the output of the first decoder attention layer.
- Attention mechanism: Scaled-dot attention uses queries, keys, and values, weighting value vectors according to normalized query-key alignment.The dot product measures how closely aligned keys are with queries, and softmax normalization emphasizes larger alignments.
- Architecture: Multi-head attention allows the encoder and decoder to attend to different tokens simultaneously.This is presented as the main advantage over seq-2-seq models previously used for reaction prediction.
- Positional encoding: Because the transformer lacks recurrent components, positional encodings add position-dependent trigonometric signals to token embeddings.These signals let the network identify where tokens occur in the sequence.
- Model configuration: At least 4 attention heads were required for peak accuracies, while the authors retained 8 heads because validation performance was superior.The model was reduced from 65M to 12M trainable parameters by using 4 layers of size 256 instead of 6 layers of size 512.
Results & Discussion
Results show that Molecular Transformer performance improves with SMILES augmentation and checkpoint averaging, exceeds prior methods across benchmark settings, and supports uncertainty estimation and mixed or stereochemical inputs.
- Training variations: SMILES augmentation significantly increases accuracy by doubling the training data with equivalent random-SMILES reaction copies.The augmented copy replaces molecules with equivalent random SMILES.
- Training variations: Averaging weights across multiple checkpoints improves results, while larger ensembles provide only marginal additional performance gains.The best single models average the last 20 checkpoints; ensemble gains are marginal despite their added training or test cost.
- Training variations: 91% top-1 accuracy is achieved by an ensemble of two models using averaged weights from 20 checkpoints.Ensembling improves accuracy and uncertainty estimation but increases training and/or test-time cost.
- Benchmark performance: Over 93% top-2 accuracy is obtained, substantially exceeding the corresponding top-1 accuracy on the reported datasets.The cited result describes the best single models using checkpoint-averaged weights.
- Benchmark performance: The Molecular Transformer outperforms literature methods across datasets, including when reactants and reagents are mixed, and it handles stereochemical information.Separating reactants and reagents gives the best model, but mixed-input models still outperform prior literature; prior graph-based methods cannot account for stereochemistry.
- Comparison with previous work: Performance is higher across all template-popularity bins than the compared model, with the accuracy gap increasing as template popularity decreases.The authors interpret this pattern as evidence that the model overfits common reactions less and requires fewer data points to predict well.
- Comparison with human organic chemists: 87.5% top-1 accuracy on 80 reactions exceeds the best human accuracy of 76.5% and the best graph-based model accuracy of 72.5%.The model remains accurate for less common reactions across the reaction-popularity bins.
- Uncertainty estimation and reaction pathway scoring: An uncertainty score yields an AUC-ROC of 0.89 for classifying whether a reaction prediction is incorrect.The model provides likelihoods for reactant-product combinations that can also score reaction pathways.
Conclusion
The Molecular Transformer outperforms prior reaction-prediction methods while requiring no handcrafted rules and predicting subtle chemical transformations. It also estimates prediction uncertainty, handles inputs without reactant–reagent separation, and supports stereochemical data.
- 90.4% top-1 accuracy and 93.7% top-2 accuracy were achieved on a common benchmark dataset.
- The model requires no handcrafted rules and accurately predicts subtle chemical transformations.
- 89% uncertainty-score accuracy was achieved when classifying whether a prediction is correct.
- The uncertainty score can be used to rank reaction pathways.
- 88.6% accuracy was demonstrated when no distinction was drawn between reactants and reagents.
- On the USPTO_STEREO dataset, top-1 accuracies were 78.1% for separated inputs and 76.2% otherwise.