PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Transformation Policy Network for Chemical Reaction Prediction

Kien Do, Truyen Tran, Svetha Venkatesh

arXiv:1812.09441v1cs.NE

TL;DR

Chemical reaction product prediction remains difficult because existing methods rely on handcrafted or heuristic rules and struggle with valid sets of bond changes. GTPN represents reactants and reagents as graphs, uses graph neural networks and reinforcement learning to learn iterative bond transformations without fixed length or order, and reaches state-of-the-art top-1 accuracy on USPTO benchmarks.

  • Problem

    Chemical reaction product prediction is important for synthesis planning, but many existing methods depend on handcrafted rules or heuristic templates that generalize poorly to unseen reactions.

  • Method

    GTPN jointly models reactant and reagent graphs and uses graph neural networks, policy learning, and constrained reinforcement learning to generate variable-length, unordered sequences of bond changes.

  • Results

    82.39% and 83.20% top-1 accuracy on USPTO-15k and USPTO, respectively, are reported as new state-of-the-art results.

  • Takeaways & Limitations

    GTPN learns reaction types directly from data without handcrafted or heuristically extracted reaction rules and can generalize to unseen reactions.

  • Takeaways & Limitations

    The discussion identifies dynamic graphs over time, full chemical planning, and structural reasoning with reinforcement learning as open directions.

Abstract

from arXiv · show

We address a fundamental problem in chemistry known as chemical reaction product prediction. Our main insight is that the input reactant and reagent molecules can be jointly represented as a graph, and the process of generating product molecules from reactant molecules can be formulated as a sequence of graph transformations. To this end, we propose Graph Transformation Policy Network (GTPN) -- a novel generic method that combines the strengths of graph neural networks and reinforcement learning to learn the reactions directly from data with minimal chemical knowledge. Compared to previous methods, GTPN has some appealing properties such as: end-to-end learning, and making no assumption about the length or the order of graph transformations. In order to guide model search through the complex discrete space of sets of bond changes effectively, we extend the standard policy gradient loss by adding useful constraints. Evaluation results show that GTPN improves the top-1 accuracy over the current state-of-the-art method by about 3% on the large USPTO dataset. Our model's performances and prediction errors are also analyzed carefully in the paper.

1 INTRODUCTION

Chemical reaction product prediction is important but existing rule- and template-based methods generalize poorly, while graph-based approaches struggle with valid, unordered bond-change sets. GTPN addresses this by iteratively learning graph transformations with graph neural networks and reinforcement learning, achieving higher top-1 accuracy on USPTO datasets.

  • Motivation: Chemical reaction product prediction supports planning syntheses, but handcrafted rules and heuristic templates are not well generalizable to unseen reactions.The paper frames product prediction as a longstanding problem in organic chemistry.
  • Challenge: Graph-based methods capture molecular structure but may ignore valid reaction-triple sets or impose a predefined transformation order.The configuration space is extremely large and non-differentiable, and small prediction errors can produce invalid products.
  • Method: GTPN combines a graph neural network, node pair prediction network, and policy network to iteratively predict reaction triples and update an intermediate graph.The process continues until the model decides to stop, and the final graph represents the predicted products.
  • Method: GTPN learns transformation length and order rather than assuming them, while combining graph neural networks and reinforcement learning in an end-to-end framework without handcrafted reaction rules.The generated reaction-triple sequence also provides an interpretation of how products are formed.
  • Results: 82.39% and 83.20% top-1 accuracy on USPTO-15k and USPTO, respectively, establish new state-of-the-art results in the reported evaluation.The paper also analyzes model performance and prediction errors.

2 METHOD

GTPN models reaction product prediction as sequential graph transformations over a joint reactant–reagent graph. It combines graph neural representations, top-K atom-pair selection, policy decisions, and recurrent updates within a Markov decision process.

  • Chemical reaction as graph transformations: A reaction is represented by predicting reaction triples that specify an atom pair and its new bond type.The predicted triples transform the input graph into a product graph.
  • Chemical reaction as a Markov decision process: GTPN treats each intermediate graph as a state and each tuple (ξ, u, v, b) as an action in a finite Markov decision process.The continuation signal ξ determines whether a bond change is applied and whether later actions are considered; γ is set to 1.
  • GTPN architecture: GTPN combines a graph neural network, node pair prediction network, policy network, and recurrent network to model and update the evolving reaction graph.The GNN updates atom representations, the NPPN identifies candidate pairs, and the PN predicts continuation, atom-pair, and bond decisions.
  • Candidate atom-pair selection: The model scores atom pairs and restricts reaction-triple prediction to the K highest-scoring pairs, where K is much smaller than |V|^2.This reduces the search space because only a small number of atom pairs typically participate in a reaction.
  • Policy decisions: The policy network predicts whether to continue, which top-K atom pair to modify, and the new bond type, then masks the selected pair from future candidate sets.The continuation signal supports reactions with varying numbers of transformation steps.
  • Training objectives: Training uses Advantage Actor-Critic loss together with constraints that address incorrect stopping and overly long predicted sequences.The sequence-length constraint penalizes predictions that continue beyond the ground-truth sequence length.

3 EXPERIMENTS

Experiments evaluate GTPN on USPTO datasets for atom-pair identification and full product prediction. The model uses reagent-aware graph representations and top-K selection, with performance stabilizing beyond k=10 and top-1 results outperforming major baselines.

  • Experimental setup: GTPN is evaluated on USPTO-15k and USPTO, whose reactions are represented as graphs converted from SMILES strings.USPTO contains reactant, reagent, and product molecules; USPTO-15k contains 15K reactions and USPTO contains 480K.
  • Reaction atom pair prediction: Coverage@k measures the proportion of reactions whose ground-truth atom pairs all appear among the top k predicted pairs.
  • Reaction atom pair prediction: GTPN’s GNN outperforms WLN and CLN across reaction atom-pair prediction cases.The authors attribute this to separately representing node features and structural information, while explicitly incorporating reagent information improves WLN by 1–7% depending on the metric.
  • Top-K atom pair extraction: Coverage@k and Recall@k rise rapidly for k<10 and stabilize for k>10, making K=10 an efficient choice.Experiments with k=10, 15, and 20 produced quite similar prediction results.
  • Reaction product prediction: GTPN with beam search and post-processing outperforms WLDN and Seq2Seq in top-1 accuracy.Its top-3 and top-5 performance is comparable to WLDN on USPTO-15k but worse on USPTO; USPTO hyperparameters were reused from the smaller dataset without tuning.

4 RELATED WORK

Related methods replace rule-based chemistry with machine-learning approaches based on templates, graph models, or sequence generation. GTPN is positioned as an order-free graph-transformation approach that avoids handcrafted reaction rules and predefined reaction topologies.

  • Learning to predict chemical reaction: Earlier chemical reaction predictors relied on handcrafted rules or extracted reaction templates, while machine-learning methods sought better generalizability and scalability.
  • Learning to predict chemical reaction: WLN-based methods predict reactive atom pairs, modify their bond types under chemical rules, and rank resulting product candidates.
  • Learning to predict chemical reaction: Seq2seq methods avoid multi-stage prediction by generating product SMILES end-to-end, but they do not properly handle sets of reactants, reagents, and products or provide concrete reaction mechanisms.
  • Learning to predict chemical reaction: Sequence-based graph transformation methods handle bond-change ordering by restricting evaluation to reactions with predefined topology.GTPN instead aims to operate without a fixed order and across almost any reaction type.
  • Background methods: Graph neural networks model molecular structure across chemistry tasks, while reinforcement learning supports discrete structural actions such as adding, connecting, or terminating graph generation.

5 DISCUSSION

The discussion presents GTPN as an end-to-end combination of graph neural networks and reinforcement learning for learning bond-change sequences. It also identifies broader graph-transformation applications and future extensions to dynamic graphs and chemical planning.

  • Discussion: GTPN represents reactants and reagents with graph neural networks and uses reinforcement learning to find bond-change sequences transforming reactants into products.Training uses Advantage Actor-Critic with constraints tailored to chemical reactions.
  • Discussion: GTPN is presented as generic beyond chemical reactions, with possible applications to relation reasoning and future extensions toward dynamic graphs and full chemical planning.These extensions remain open directions rather than demonstrated results in the passage.
  • Discussion: Structural reasoning concerns inferring or generating new structures, such as objects together with their relations.

A.1 GRAPH NEURAL NETWORK

The GNN represents molecular graphs with node and edge features, then recursively updates separate node state vectors through message passing. Neighbor messages are aggregated and transformed with a gated nonlinear update.

  • Notation: The GNN omits the recurrent hidden-state time index from its notation and uses t solely for the message-passing step.
  • Graph representation: The GNN input is a graph whose nodes have feature vectors and whose edges have feature vectors encoding molecular properties such as atom information and bond type.
  • Graph representation: Each node also maintains a state vector that stores information about the node and its surrounding context and is updated recursively.The initial state is obtained by a nonlinear mapping of the node feature vector.
  • Computing neighbor messages: At each message-passing step, node i receives messages from neighboring nodes j based on their representations and links to i.The message construction uses concatenation and a nonlinear function.
  • Aggregating neighbor messages: Neighbor messages are averaged into a single message vector for each node.The aggregation is over the neighboring nodes of i.
  • Node-state update: The node state is updated with a Highway Network whose gate controls information flow.The update combines nonlinear transformation and gated information propagation.

A.2 UPDATING STATES

After each predicted reaction triple, GTPN updates the molecular graph and relevant node representations so later predictions reflect prior bond changes. It retains modified-edge information and limits costly message passing when distant bonds are unlikely to be affected.

  • State memory: A GRU hidden memory preserves representations of modified edges, allowing the model to track graph changes accumulated across prediction steps.This memory supports more accurate later predictions by retaining the history of changes.
  • Graph structure updates: GTPN updates the neighbor sets of the two atoms after predicting each new bond change.The update incorporates information from the other atom and the predicted bond type.
  • Node-state updates: The model performs message passing for the affected atoms and then propagates their new structural information through the graph.Full-graph propagation is costly at every prediction step, so the model can avoid unnecessary updates for distant bonds except in cases such as aromatic rings.

A.3 MODEL CONFIGURATIONS

GTPN’s configuration separates hyper-parameter tuning across its prediction components and uses fixed neural-network design choices for experiments. The model combines graph, atom-pair, and policy networks with reinforcement-learning training objectives.

  • Tuning procedure: Hyper-parameters are tuned first for the GNN and NPPN, then the PN is optimized with those settings fixed.The first stage targets reaction atom-pair prediction, while the second targets reaction product prediction.
  • Dataset configuration: The USPTO model uses settings optimized on USPTO-15k rather than separately tuned on the larger USPTO dataset.The authors note that larger model sizes might improve performance but were not explored because USPTO training is time-consuming.
  • Network components: The GNN represents 72 atom types and 5 bond types, while atom features combine embeddings with five normalized attributes.The resulting atom feature vector has size 56, and atom state and neighbor-message vectors have size 99.
  • Network components: The NPPN computes atom-pair representations and unnormalized scores, while the PN uses a GRU and separate scoring networks for signals and bond types.The NPPN hidden sizes are 71 and 51; the PN GRU has 101 hidden units.
  • Training objective: Training uses rewards for correct or incorrect signal, atom-pair, and bond predictions, plus a terminal reward for correctly predicting the complete reaction-triple set.The objective combines A2C, value, atom-pair, sequence-length, and top-K terms.
  • Optimization: Adam optimization starts at learning rate 0.001, with dataset-specific decay schedules and minimum learning rates.USPTO-15k halves the rate after 1,000 stagnant validation steps, whereas USPTO uses a 0.8 decay after every 500 steps.

A.4 DECODING WITH BEAM SEARCH

GTPN decodes reaction-triple sequences by selecting a signal, atom pair, and bond type until a stop signal appears, then uses beam search to reduce sequence fragility. Beam width affects performance non-monotonically, with width 20 selected for subsequent experiments.

  • Greedy decoding: Greedy decoding selects the best atom pair and bond type at each step until the model emits the zero signal.The generated sequence includes the stop signal as part of the reaction-triple sequence.
  • Beam-search motivation: A single stepwise error can destroy the full generated sequence, motivating beam search during decoding.Beam search retains multiple candidate subsequences rather than committing to one sequence at each step.
  • Beam-search procedure: Because jointly computing all signal, atom-pair, and bond-type configurations is memory-intensive, the decoding distribution is decomposed into sequential components.At each step, beam search handles the signal, then the atom pair and finally the bond type.
  • Beam-search procedure: Beam search is applied to atom pairs and bond types only when the signal indicates that prediction is ongoing, and scores are normalized by sequence length.These choices prevent unnecessary branching and reduce the tendency to favor short sequences.
  • Beam-width analysis: Beam width 20 gives the best reported results across different k values on USPTO-15k, although larger width is not uniformly better.The authors therefore use beam width 20 in subsequent experiments.

A.5 USING REAGENT INFORMATION EXPLICITLY

GTPN explicitly incorporates reagent identity into atom-pair prediction. Reagent molecules constitute a substantial share of inputs, motivating their use as an additional atom-level signal.

  • Reagent representation: Reagent molecules account for about half of input molecules on average and occur in 60–80% of reactions containing reagents.The model appends a binary indicator to each atom representation specifying whether the atom comes from a reagent molecule.
  • Reagent representation: The model also excludes reagent-derived candidates during top-K atom-pair selection.This supplements the reagent indicator used before atom-pair scoring.

A.6 COMPARISON WITH ELECTRO

GTPN is presented as more generic than ELECTRO because it does not impose a transformation order or fixed number of bond changes. The section also describes beam-search decoding and analyzes prediction errors across reaction complexity and error sources.

  • Comparison with ELECTRO: GTPN assumes no specific order of transformations or fixed amount of valence change, making it more generic than ELECTRO.ELECTRO models reactions as ordered sequences alternating between removing and adding a single bond.
  • Beam-search decoding: Beam search decodes reaction-triple sequences using a beam width N and length-normalized joint probabilities.The algorithm initializes candidate subsequences, performs message passing, and searches over continuation signals, atom pairs, and bonds.
  • Errors by bond-change count: GTPN performs poorly on reactions with many bond changes, while erroneous sequences tend to be shorter than the ground-truth sequences.Reactions with many bond changes represent only a small proportion of the dataset.
  • Errors by sub-action: Atom-pair prediction causes nearly two thirds of the errors, exceeding errors from signal and bond-type prediction.The analysis identifies atom-pair prediction as the main target for improvement.
  • Errors caused by symmetry: Symmetry-related errors account for 5.7% of top-1 errors on USPTO-15k because different bond-change sequences can yield the same products.In the illustrated cases, the top-2 product is correct while the top-1 product is incorrect despite equal probability.
Loading 1812.09441v1…