Predictive Chemistry Augmented with Text Retrieval

1 Introduction

• Molecular-only chemoinformatics models still have room to improve, motivating the use of relevant scientific-literature information for prediction.The paper notes that such models operate solely on molecular encodings despite advances in neural molecular representations.Source: §1 · p. 1
• TextReact retrieves reaction-relevant descriptions and fuses them with the reaction input; an auxiliary masked-LM objective helps align reaction and text representations.For unseen reaction classes, training simulates novel inputs by removing the closest textual descriptions for reactions.Source: §1 · p. 1
• Retrieved passages can describe similar reactions with shared conditions, but the method does not assume the corpus contains structured catalyst, solvent, or reagent data.Source: Figure 1 · p. 2
• 58.4% top-1 prediction accuracy improvement for condition recommendation and 13.6–15.7% improvement in one-step retrosynthesis are reported for TextReact.The paper compares TextReact with state-of-the-art chemoinformatics models trained on reaction data alone.Source: §1 · pp. 1–2

Multimodal Retrieval

• TextReact retrieves text descriptions for reaction queries, the reverse direction of Text2Mol’s retrieval of molecules from natural-language queries.The paper relates its aligned embedding spaces to CLIP’s image-text retrieval.Source: §2 · p. 2

Retrieval-Augmented NLP

• Earlier retrieval-augmented NLP work uses relevant documents as additional context, while TextReact applies retrieval augmentation to chemistry prediction.Source: §2 · p. 2

Natural Language Grounding

• Natural-language grounding has been studied in vision, reinforcement learning, and robotics, including work on adapting to new tasks.Source: §2 · pp. 2–3
• Where reaction-extraction research creates structured databases for molecular-model training, TextReact directly uses unstructured literature descriptions to augment predictions.Source: §2 · p. 3

3.1 Problem Setup

• Training pairs each reaction-condition example with a reaction description and uses an unlabeled corpus of reaction paragraphs; the setup also applies to one-step retrosynthesis.For retrosynthesis, the input is a product molecule and the output is a list of potential reactants.Source: §3.1 · p. 3

3.2 TextReact Framework

• TextReact has a SMILES-to-text retriever that finds relevant corpus passages and a text-augmented predictor that combines them with the chemistry input.The predictor generates the chemistry output from both sources.Source: §3.2 · p. 3

3.2.1 SMILES-To-Text Retriever

• The retriever uses chemistry and text encoders to align reaction SMILES with descriptions, enabling retrieval of relevant paragraphs for a chemistry input.It is trained separately from the predictor; at retrieval time, an index of precomputed text encodings supports nearest-neighbor search.Source: §3.2.1 · pp. 3–4

3.2.2 Text-Augmented Predictor

• The predictor encodes the SMILES input with retrieved texts and is trained with an auxiliary masked-language-model loss to learn correspondences between chemistry and text.Dynamic sampling exposes it to random neighbors as well as gold text during training, while inference uses the top-k retrieved neighbors.Source: §3.2.2 · pp. 4–5

Text Corpus

• The text corpus contains 2.9 million patent-derived reaction paragraphs averaging 190 tokens, and the model learns from the text rather than the associated extracted structured reaction data.The corpus is constructed from USPTO patent data processed by Lowe (2012).Source: §4 · p. 5

Implementation

• The retriever uses ChemBERTa-initialized chemistry and SciBERT-initialized text encoders, with separate retrievers trained for each task.Its chemistry input is reactants and product for condition recommendation, but only the product for retrosynthesis.Source: §4 · p. 5

Evaluation

• Evaluation uses random splits and more challenging time splits that train on older patents and test on newer-patent data.For time-split retrieval, the experiments compare using the full corpus with using only years represented in training.Source: §4 · pp. 5–6

5.1 Reaction Condition Recommendation

• For reaction condition recommendation, TextReact substantially outperforms Transformer and ChemBERTa and retains strong performance on the time split.The retriever retains high accuracy in finding the corresponding paragraph for test reactions using historical data.Source: §5.1 · pp. 6–7, Table 3 · p. 7

5.2 One-Step Retrosynthesis

• TextReact raises one-step retrosynthesis top-1 accuracy by 13.6% in the template-free setting and 15.7% in the template-based setting.The baseline Transformer models perform comparably to previous models in both settings.Source: Table 4 · p. 7, §5.2 · p. 8
• Retrosynthesis is tested with template-free reactant generation and template-based reaction-template and reaction-center prediction.The template-free model generates reactant SMILES; the template-based model predicts applicable templates and reaction centers.Source: §5.2 · p. 7

5.3 Analysis

• With test gold texts removed, TextReact significantly outperforms Transformer under random and time splits, but hardly outperforms it with historical-only retrieval under the time split.The authors suggest reactions in new patents may differ sufficiently from historical reactions to create a predictive barrier.Source: Figure 5 · p. 8, §5.3 · p. 8
• Training α matters: RCR full-corpus evaluation favors smaller values, gold-removed evaluation favors larger values (α = 0.8 balances them), while RetroSyn favors much smaller α.The predictor receives gold text with probability 1 − α and randomly sampled paragraphs with probability α.Source: Figure 6 · p. 8, §5.3 · pp. 8–9
• Ablations favor combining chemistry and retrieved text, auxiliary MLM, scientific-text SciBERT pretraining, and jointly encoding neighbors over ensembling separate-neighbor predictions.The study evaluates TextReacttf for RetroSyn.Source: Table 5 · p. 9, §5.3 · p. 9
• Appendix analyses report that TextReact beats Transformer on RCR with 10% of the training data and benefits from more similar retrieved texts and jointly modeling neighboring reactions.Source: §5.3 · p. 9

• TextReact uses information retrieval to identify text descriptions relevant to chemistry inputs and supplies them as additional evidence for prediction.Source: §6 · p. 9
• Across two chemistry tasks, TextReact performs strongly with the full corpus and retains significant improvement when gold texts are excluded.Source: §6 · p. 9
• The results point to the potential of grounding retrieved patent and publication text in chemistry models to enhance predictive power.Source: §6 · p. 9

7 Limitations

• Experiments cover two representative chemistry tasks, leaving the method’s performance on other tasks and domains untested.The authors believe it may apply where literature knowledge is useful.Source: §7 · p. 9
• The retriever and predictor use simplified implementations, with advanced pretrained models and joint training identified as opportunities for improvement.Source: §7 · p. 9

A.1 Reaction Condition Recommendation

• Reaction condition recommendation suggests catalysts, solvents, and reagents; this work follows Gao et al.’s formulation with a Transformer architecture.Prior work also explored alternative reaction representations and learning formulations.Source: §A.1 · p. 13

A.2 One-Step Retrosynthesis

• One-step retrosynthesis proposes reaction precursors for target molecules, with template-based and template-free approaches forming two broad method categories.Template-based methods predict a reaction template before deriving precursors; template-free methods predict precursors directly.Source: §A.2 · p. 13
• Figure 7 shows USPTO example reactions and gold texts, with reactants, products, catalyst, solvent, and reagent marked in distinct colors.Reactants are yellow, products orange, catalyst red, solvent green, and reagent blue.Source: Figure 7 · p. 13
• TextReact is applied to both template-based and template-free retrosynthesis approaches.Source: §A.2 · p. 13

B Datasets

• Figure 7 presents example USPTO reactions alongside their corresponding gold texts.Source: B Datasets · p. 13
• Reaction condition recommendation data come from public USPTO data and use an 80%/10%/10% random train/validation/test split.Source: B Datasets · p. 13
• The reaction-condition time split trains on patents before 2015, validates on 2015 reactions, and tests on 2016 reactions.Source: B Datasets · pp. 13–14
• One-step retrosynthesis uses USPTO 50K with its original random split and a time split based on patent years.The time split trains before 2012, validates on 2012–2013, and tests on 2014–2015.Source: B Datasets · p. 14
• The authors state that the datasets will be publicly available to foster research in this direction.Source: B Datasets · p. 14

C Implementation Details

• The experiments use Hugging Face Transformers, PyTorch Lightning, Tevatron, and Faiss.Source: C Implementation Details · p. 14
• For reaction condition recommendation, the retriever is contrastively trained on batches of 512 queries, each paired with one positive and one randomly sampled negative paragraph.It trains for 50 epochs with a 1e-4 learning rate; query and paragraph maximum lengths are 256.Source: C Implementation Details · p. 14
• The reaction-condition predictor uses a 20-epoch training run, appends three neighboring text paragraphs by default, and applies random-order SMILES augmentation.It uses batch size 128, learning rate 1e-4, MLM-loss weight λ1 = 0.1, and a maximum encoder input length of 512.Source: C Implementation Details · p. 14
• Figure 8 plots top-1 accuracy against the percentage of available training data for TextReact and the Transformer baseline.It reports strong low-resource performance for reaction condition recommendation and evaluates TextReacttf for retrosynthesis.Source: Figure 8 · p. 14
• For retrosynthesis, the retriever trains for 400 epochs, and each product-molecule query has a maximum length of 128.Source: C Implementation Details · p. 14
• The template-free predictor generates output SMILES with a six-layer Transformer decoder, while the template-based predictor uses linear heads on atom and bond representations.Both use SciBERT and train for 200 epochs; the random sampling ratio is α = 0.2.Source: C Implementation Details · p. 14

D Further Analysis

• Retrieving three paragraphs performs best: accuracy improves from k = 1 to k = 3, then decreases for k > 3 on both datasets.The improvement from k = 1 to k = 3 is notable on RCR (RS) and smaller on RetroSyn (RS); k = 3 is used in the main experiments.Source: Figure 10 · p. 15, D Further Analysis · p. 15
• With 10% training data, TextReact reaches 67.7% full-corpus and 35.0% gold-removed accuracy on RCR, versus 30.0% for the full-data Transformer baseline.The models were evaluated on the full test dataset under the random split.Source: D Further Analysis · pp. 14–15
• TextReact predictions are more accurate when retrieved texts describe reactions more similar to the input reaction.Figure 9 plots top-1 accuracy against average input-neighbor distance and average distance between neighbors for RCR (RS).Source: D Further Analysis · p. 15, Figure 9 · p. 15