PaperlayerView exact revision on arXiv

Source-linked AI summary

Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction

Zhengkai Tu, Connor W. Coley

arXiv:2110.09681v1cs.LG

TL;DR

The paper addresses molecule-to-molecule prediction tasks whose SMILES inputs are representation-sensitive and often require augmentation. Graph2SMILES replaces sequence encoding with permutation-invariant graph encoding while retaining Transformer decoding, improving Transformer baselines across reaction prediction and retrosynthesis benchmarks. Its reported gains are strongest for top-1 accuracy, while top-n performance and output-side augmentation remain limitations or open directions.

  • Problem

    SMILES representations are not efficient for molecular structures, and their non-bijective mapping motivates augmentation to improve prediction performance.

  • Method

    Graph2SMILES encodes molecular graphs with attention-augmented directed message passing and global graph attention, then uses a Transformer decoder to generate SMILES.

  • Results

    Graph2SMILES improves Transformer-baseline top-1 accuracy by 1.7 points on USPTO_480k, 1.9 points on USPTO_STEREO, and 9.8 points on USPTO_50k retrosynthesis.

  • Takeaways & Limitations

    The architecture is a drop-in replacement for Transformer-based molecular transformation methods and avoids input-side SMILES augmentation.

  • Takeaways & Limitations

    Top-1 accuracy may be less relevant when multiple retrosynthetic options are equally plausible or reaction products are legitimately ambiguous, making top-n performance important in some applications.

Abstract

from arXiv · show

Synthesis planning and reaction outcome prediction are two fundamental problems in computer-aided organic chemistry for which a variety of data-driven approaches have emerged. Natural language approaches that model each problem as a SMILES-to-SMILES translation lead to a simple end-to-end formulation, reduce the need for data preprocessing, and enable the use of well-optimized machine translation model architectures. However, SMILES representations are not an efficient representation for capturing information about molecular structures, as evidenced by the success of SMILES augmentation to boost empirical performance. Here, we describe a novel Graph2SMILES model that combines the power of Transformer models for text generation with the permutation invariance of molecular graph encoders that mitigates the need for input data augmentation. As an end-to-end architecture, Graph2SMILES can be used as a drop-in replacement for the Transformer in any task involving molecule(s)-to-molecule(s) transformations. In our encoder, an attention-augmented directed message passing neural network (D-MPNN) captures local chemical environments, and the global attention encoder allows for long-range and intermolecular interactions, enhanced by graph-aware positional embedding. Graph2SMILES improves the top-1 accuracy of the Transformer baselines by $1.7\%$ and $1.9\%$ for reaction outcome prediction on USPTO_480k and USPTO_STEREO datasets respectively, and by $9.8\%$ for one-step retrosynthesis on the USPTO_50k dataset.

1 INTRODUCTION

Retrosynthesis and reaction outcome prediction are fundamental computer-aided organic synthesis tasks, but SMILES representations are sensitive to equivalent atom orderings. Graph2SMILES addresses this with a permutation-invariant graph-to-sequence architecture that avoids input-side SMILES augmentation.

  • Retrosynthesis proposes reaction precursors from a desired product, whereas reaction outcome prediction predicts major products from reactants.
  • SMILES lack bijective mappings to molecular structures, motivating chemically equivalent SMILES augmentation that can improve performance by 0.8% to 4.3%.
  • Graph2SMILES combines an attention-augmented D-MPNN, a Transformer-based global attention encoder, graph-aware positional embedding, and a Transformer decoder.
  • The model represents inputs as molecular graphs and outputs as SMILES, guaranteeing permutation invariance and eliminating the need for input-side augmentation.
  • Graph2SMILES is presented as a drop-in Transformer replacement that outperforms Transformer baselines without input-side SMILES augmentation.

2 METHODS

Graph2SMILES represents molecular inputs as graphs and outputs SMILES sequences, combining permutation-invariant graph encoding with Transformer-based global attention and autoregressive decoding.

  • 2.1 GRAPH AND SEQUENCE REPRESENTATIONS OF MOLECULES: Graph2SMILES encodes molecule(s) as graphs with order-invariant atom and bond features, then generates output molecules as canonical-SMILES token sequences.The graph input may contain multiple molecular subgraphs, while the output is tokenized as a SMILES sequence.
  • 2.2 GRAPH2SMILES: The architecture replaces a standard Transformer encoder with a D-MPNN, a global attention encoder, and a graph-aware positional embedding.The D-MPNN captures local chemical context, while global attention exchanges information across atoms.
  • 2.2.1 ATTENTION AUGMENTED DIRECTED MESSAGE PASSING ENCODER: The D-MPNN updates directed-bond messages with GRUs and attention-based aggregation to avoid back-and-forth neighbor message passing.The attention-augmented variant is termed Directed Graph Attention Network, or D-GAT.
  • 2.2.2 GLOBAL ATTENTION ENCODER WITH GRAPH-AWARE POSITIONAL EMBEDDING: Graph-aware attention replaces sequence-relative positions with learnable embeddings based on shortest-path distance and whether atoms belong to the same molecule.The encoding buckets graph distances and assigns a separate category to atoms in different molecules.
  • 2.2.3 AUTOREGRESSIVE DECODER: A Transformer autoregressive decoder generates each output token by attending to encoded atoms and previously generated tokens.The decoder uses sequence-based relative positional embedding with maximum relative position 4.

3 RELATED WORK

Related approaches include template-based, graph edit-based, and translation-based methods for reaction prediction and retrosynthesis. Template-free translation methods simplify the formulation but commonly rely on SMILES sequence representations and augmentation.

  • Template-based methods: Template-based methods face a tradeoff between template generality and specificity and cannot generalize to unseen templates.These limitations motivate template-free approaches.
  • Template-free methods: Template-free methods broadly comprise graph edit-based approaches and translation-based approaches that formulate the tasks as SMILES-to-SMILES translation.Translation systems commonly use recurrent neural networks or Transformers.
  • Graph-enhanced encoders: Graph2SMILES differs from GET by excluding sequence representations from its encoder, thereby guaranteeing permutation invariance.GET concatenates SMILES sequence embeddings with learned atom representations.
  • Graph-aware attention: Graph Transformer methods have used atomic distances or shortest-path lengths to incorporate structural information into attention.Graph2SMILES draws on shortest-path approaches while explicitly separating graph-topology effects in its attention design.
  • Sequential graph and Transformer encoders: Prior sequential combinations of graph and Transformer encoders do not retain explicit graph-topology information before attention encoding, unlike Graph2SMILES.The paper reports that this distinction is important in its ablation study.

4 EXPERIMENTS

Experiments evaluate Graph2SMILES on USPTO reaction outcome prediction and one-step retrosynthesis benchmarks, comparing it with Transformer and existing methods under different feature and augmentation settings. Graph2SMILES improves Transformer baselines substantially while remaining competitive with methods using templates, atom mapping, augmentation, or pretraining.

  • Experimental setup: Experiments use four USPTO datasets and report top-n test accuracies, with reaction outcome prediction evaluated on USPTO_480k_mixed and USPTO_STEREO_mixed.The mixed datasets retain reactants and reagents without separating species by whether they contribute heavy atoms to the product.
  • Reaction outcome prediction: 1.7 points: Graph2SMILES improves MT top-1 accuracy on USPTO_480k_mixed, with additional gains of 0.5 and 0.4 points at top-3 and top-10.These improvements are measured against the Molecular Transformer baseline.
  • Reaction outcome prediction: 1.9 points: Graph2SMILES improves MT top-1 accuracy on USPTO_STEREO_mixed, while top-3 accuracy changes only slightly.The approach still trails Augmented Transformer and Chemformer, which use test-time augmentation or substantially larger pretrained models.
  • One-step retrosynthesis: On USPTO_full retrosynthesis, Graph2SMILES exceeds all reported methods except GTA in top-1 accuracy without templates, atom mapping, or output-side augmentation.The cited comparison notes that these additional techniques are orthogonal to the graph-to-sequence architecture and could potentially improve it.
  • One-step retrosynthesis: 9.8 points: On USPTO_50k, Graph2SMILES raises Transformer top-1 accuracy from 43.1 to 52.9 among methods without templates, atom mapping, or output SMILES augmentation.For n > 1, higher top-n accuracy can trade off against top-1 accuracy, so the reported model is selected by validation top-1 accuracy.
  • Ablation study: Ablations on USPTO_50k remove graph-aware positional embedding or the global attention encoder to assess their contributions within the D-GCN variant.The passage introduces the ablation design but does not provide the resulting numerical drops.

5 DISCUSSION

The discussion reports that Graph2SMILES was evaluated in a baseline configuration to isolate its advantage over a vanilla Transformer. It also identifies limitations of output-side augmentation and top-1 accuracy as an evaluation basis.

  • Discussion: The experiments emphasized the baseline Graph2SMILES model rather than additional features or performance-engineering techniques.Atom mapping and template-based filtering are discussed as possible integrations, but were not the focus of the baseline comparison.
  • Discussion: Output-side SMILES augmentation confused the implemented model when it was trained to generate two equivalent but syntactically different SMILES.The authors leave variational-inference-based approaches for future work.
  • Discussion: Top-1 accuracy is limited for retrosynthesis because multiple candidate reactions can be equally plausible.The authors note that top-n accuracy may be more relevant for some multistep planning applications.
  • Discussion: Performance-engineering techniques would be necessary to boost Graph2SMILES top-n performance in applications where top-n accuracy is more relevant.This boundary is stated specifically for cases such as some multistep planning applications.

6 CONCLUSION

The conclusion presents Graph2SMILES as a template-free model for reaction outcome prediction and retrosynthesis. Its permutation-invariant graph encoding removes the need for input-side SMILES augmentation while improving over Transformer baselines, with further gains left to future performance engineering.

  • Conclusion: Graph2SMILES addresses template-free reaction outcome prediction and retrosynthesis.The model is presented as a graph-to-sequence approach for these molecular transformation tasks.
  • Conclusion: Permutation-invariant D-MPNN encoding and graph-aware positional embedding eliminate the need for input-side SMILES augmentation.The conclusion connects this design to the model's representation of molecular inputs.
  • Conclusion: Graph2SMILES improves over Transformer baselines, especially for top-1 accuracy, and is presented as an attractive Transformer replacement.The architecture can replace Transformer models in molecular transformation methods with minimal pipeline modification.
  • Conclusion: Further gains may come from performance-engineering techniques orthogonal to the architecture, which the authors leave for future work.The conclusion identifies these techniques as a future direction rather than part of the reported baseline.

A APPENDIX: ATOM AND BOND FEATURES USED

The appendix summarizes the atom and bond features used by Graph2SMILES. Most features were adapted from GraphRetro, with chiral features added.

  • Atom and Bond Features: Table 5 summarizes the atom and bond features used in Graph2SMILES.The table is the appendix's feature summary.
  • Atom and Bond Features: Most Graph2SMILES features were adapted from GraphRetro.The adaptation applies to the atom and bond feature set summarized in Table 5.
  • Atom and Bond Features: Chiral features covering R/S and E/Z were added to the adapted feature set.These are the specific additions identified by the appendix passage.

B APPENDIX: OTHER METHODS FOR REACTION OUTCOME PREDICTION

The appendix compares methods excluded from the main reaction-outcome discussion on USPTO_480k_separated. Most reported results come from this less challenging separated-reagent dataset, and only NERF marginally exceeds Molecular Transformer among the listed methods.

  • Other Methods for Reaction Outcome Prediction: Table 6 lists excluded reaction-outcome methods on USPTO_480k_separated, sorted by top-1 accuracy.The table caption states that the best results are highlighted in bold.
  • Other Methods for Reaction Outcome Prediction: Most excluded methods report results on the less challenging USPTO_480k_separated dataset, where reagents were heuristically separated from reactants.This dataset distinction motivates the appendix's comparison context.
  • Other Methods for Reaction Outcome Prediction: NERF improves top-1 accuracy by 0.3 points over Molecular Transformer on USPTO_480k_separated.The passage identifies NERF as the only method showing marginal improvement among the listed methods.
  • Other Methods for Reaction Outcome Prediction: All other listed methods cannot perform as well as Molecular Transformer on that comparison.The authors therefore use Molecular Transformer as the baseline in Section 4.3.

C APPENDIX: SUMMARY OF FOUR USPTO DATASETS USED

Table 7 presents statistics for the USPTO datasets used in the study.

  • Table 7 reports statistics for the USPTO datasets used.
  • The table provides dataset-level information for the study’s USPTO data.
  • Its purpose is to summarize the datasets used in the experiments.

D APPENDIX: HYPERPARAMETER SETTING

Table 8 lists the hyperparameter settings used for experiments across different datasets, with validation-selected best settings highlighted when multiple values were tested.

  • Table 8 lists hyperparameter settings used in experiments for different datasets.
  • Best settings selected using validation are highlighted in bold when multiple values were tested.
  • The table distinguishes selected settings from alternative experimented values.
Loading 2110.09681v1…