PaperlayerView exact revision on arXiv

Source-linked AI summary

RetroAgent: Harnessing LLMs to Search Over Structured Memory for Agentic Retrosynthesis Planning

Yanqiao Zhu, Jingru Gan, Xiaoqi Sun, Fang Sun, Yidan Shi, Md Mofijul Islam, Chao Shang, Wenhao Gao, Connor W. Coley, Yizhou Sun, Wei Wang

arXiv:2607.14512v2cs.AIcs.CLcs.LG

TL;DR

Multi-step retrosynthesis is a combinatorial, long-horizon search problem. RetroAgent trains an LLM to control structured-memory search and reports the highest USPTO-190 pass@1 alongside strong out-of-distribution results on ChEMBL-1000.

  • Problem

    Multi-step retrosynthesis is combinatorial and long-horizon, while value functions score molecules in isolation rather than reasoning over complete routes.

    Source: §1 · pp. 1–2

  • Method

    An RL-trained LLM uses structured memory and tool calls to control an explicit AND-OR retrosynthesis search.

    Source: §1 · p. 2, §4 · p. 10

  • Results

    RETROAGENT achieves the highest pass@1 on USPTO-190 and outperforms all baselines on out-of-distribution ChEMBL-1000, reaching 73.82% pass@1.

    Source: §1 · p. 3, §3.2 · p. 8, §4 · p. 10

  • Takeaways & Limitations

    The out-of-distribution results suggest the harness supports generalizable planning strategies rather than dataset-specific memorization.

    Source: §3.2 · p. 8, §4 · p. 10

    Template-valid routes do not capture full synthetic practicality, including yield, selectivity, reaction conditions, or protecting-group strategy.

    Source: D Limitations and Discussion · p. 16

Abstract

from arXiv · show

Multi-step retrosynthesis planning seeks to decompose a target molecule into commercially available building blocks through a sequence of feasible reactions. The vast combinatorial search space makes this task challenging even for expert chemists. Traditional methods combine tree search with offline-trained value networks that score candidates in isolation, without reasoning about complete multi-step routes. Recent work leverages Large Language Models (LLMs) for this task, but relies on simple interfaces that limit exploration of the full search space. We introduce RetroAgent, an LLM agent that bridges symbolic search and neural reasoning through a harness with structured memory. Through memory and chemistry tools, the agent observes the full search state, including explored routes, available alternatives, and properties of intermediates, enabling informed decisions grounded in both global progress and domain knowledge. Experiments on in-distribution and out-of-distribution benchmarks demonstrate that RetroAgent delivers strong performance and generalization.

1 Introduction

RetroAgent addresses the combinatorial, long-horizon retrosynthesis search problem by giving an RL-trained LLM control of a structured search process. It reports strong results on both in-distribution and out-of-distribution benchmarks.

1 Introduction

  • Multi-step retrosynthesis is a long-horizon problem with combinatorial search growth, where early choices constrain downstream options and may lead to regions without feasible routes.Source: §1 · p. 1
  • Prior value functions and heuristics are proxy objectives rather than direct route-completion objectives, while existing LLM interfaces limit exploration of alternative decompositions.Source: §1 · pp. 1–2
  • RETROAGENT couples an LLM to structured memory and chemistry tools, letting it choose search actions from the full context and training it end-to-end with an outcome-based reward.The agent can choose which molecule to prioritize and which and how many reactions to apply.Source: §1 · pp. 2–3
  • RETROAGENT achieves the highest pass@1 on USPTO-190 and surpasses all state-of-the-art methods on both ChEMBL-1000 measures, including at larger budgets.The authors report this advantage on ChEMBL-1000, an out-of-distribution benchmark.Source: §1 · p. 3

2 Method

The method represents retrosynthesis as an AND-OR search graph managed by a structured-memory environment, while an RL-trained LLM selects search actions. Its reward favors completed routes and efficient search, and policy optimization uses sequence-level clipping and KL regularization.

2.1 Problem Formulation

  • A valid route decomposes the target into commercially available building blocks through valid reactions, represented as an AND-OR graph with OR molecule nodes and AND reaction nodes.Source: §2.1 · p. 3

2.2 Environment and Tool Interface

  • The environment maintains graph state and frontier, propagates solved status automatically, and lets the agent flexibly expand molecules and apply multiple candidate reactions to explore alternatives.Multi-expand can hedge against dead ends while skipping unpromising candidates.Source: §2.2 · pp. 3–4

2.3 Training with Reinforcement Learning (RL)

  • The LLM is trained end-to-end with RL on complete multi-turn search trajectories; tool observations are masked so only agent tokens are trainable.Source: §2.3 · p. 5

2.3.1 Reward Function

  • The verifiable outcome reward gives positive reward only for complete routes, with no partial credit, and adds shaping terms that penalize excessive budget use and deep routes.Source: §2.3.1 · p. 5

1.0 −δN −δD + p,

  • The call-count and route-depth penalties are zero below 20, then rise by 0.02 per unit and cap at 0.3; minor penalties also discourage tool errors and invalid calls.Source: §1.0 · p. 5

2.3.2 Policy Optimization

  • GSPO combines a sequence-level clipped policy loss with KL regularization; grouped rollout rewards provide advantages without a separately trained critic, and TIS reuses batches across gradient steps.TIS clips sequence-level importance ratios and discards samples outside a fixed range to control off-policy variance.Source: §2.3.2 · p. 5

3 Experiments

The experiments compare RETROAGENT with search- and LLM-based planners on in-distribution and out-of-distribution benchmarks, then examine design choices and learned search behavior. Results favor RETROAGENT on pass@1 and especially on ChEMBL-1000, while ablations report gains associated with multi-expand and depth penalty.

3.1 Experimental Setup

  • Experiments use USPTO-190 as the in-distribution benchmark and ChEMBL-1000 as the out-of-distribution benchmark, comparing search-based and LLM-based planners.They report pass@1 and success rate within budget N; search-based methods use the full budget in one run, while LLM methods use multiple trajectories.Source: §3.1 · p. 6

3.2 Main Results (RQ1)

  • 53.26% pass@1 and 85.05% at N=500 on USPTO-190; on ChEMBL-1000, 73.82% pass@1 and 84.40% at N=500 surpass all search-based methods.On USPTO-190, the agent also has the highest pass@1 among methods; at N ≥400, exhaustive PDVN search surpasses LLM-based methods, with 92.11% at N=500.Source: Table 1 · p. 7, §3.2 · p. 8

3.3 Ablation Studies (RQ2, RQ3)

  • 4.42% pass@1 for zero-shot; multi-expand raises pass@1 from 40.63% to 53.26% with depth penalty, while depth penalty lifts N=500 success from 81.26% to 85.05%.The zero-shot baseline uses temperature 0.6 because greedy decoding produced degenerate outputs.Source: Table 2 · p. 8, §3.3 · p. 8
  • RETROAGENT finds 38.3 ± 3.1 shorter routes per run, the most among ablated variants; across runs, its routes are shorter than reference routes for 69 distinct targets.Removing either the depth penalty or multi-expand reduces shorter-route discovery, and removing both reduces it most.Source: §3.3 · p. 8, §3.2 · p. 8

3.4 Behavioral Analysis (RQ4)

  • After step 350, multi-expand dominates over 70% of expansions; chemistry-tool use falls from about 11% at step 0 to near zero by step 10.Prediction, expansion, and frontier retrieval remain the dominant tools throughout training.Source: Figure 3 · p. 9, §3.4 · p. 9
  • Multi-expand shifts the full model toward more nodes at shallower depth; removing it yields narrower, deeper trees, while the full model expands multiple OR branches.The agent applies selective local breadth: about 84% of expand calls apply one or two reactions, and solved targets expand slightly wider on average than failed ones on both benchmarks.Source: Figure 3 · p. 9, §3.4 · pp. 9–10
  • Zero-shot rollouts rarely exceed 15 steps or 75 nodes, while the trained agent routinely builds 50–100+ node trees; without depth penalty, solved cases often reach the 30-step cap.The paper describes this as unproductive deep exploration that wastes budget on unnecessarily long routes.Source: §3.4 · pp. 9–10

4 Conclusion

RETROAGENT is an RL-trained LLM agent that plans retrosynthesis over an AND-OR graph used as structured memory. The paper reports strong benchmark performance and identifies learned search behaviors and design choices supporting it.

4 Conclusion

  • With a 4B-parameter model, RETROAGENT achieves the highest pass@1 on USPTO-190 and substantially outperforms baselines on out-of-distribution ChEMBL-1000 despite using a smaller model and less training data.The authors suggest the out-of-distribution results indicate generalizable planning strategies rather than dataset-specific memorization.Source: §4 · p. 10
  • RETROAGENT combines an RL-trained LLM agent with an AND-OR graph as structured memory, with the environment maintaining search state and the agent making strategic decisions through tool calls.Source: §4 · p. 10
  • Ablations identify multi-expand and reward shaping as key to performance, while behavioral analysis finds that RL progressively shifts the agent toward trading depth for breadth.Source: §4 · p. 10
  • The authors propose combining LLM reasoning with structured environment management as a design principle for long-horizon planning beyond retrosynthesis.Source: §4 · p. 10

LLM Usage Disclosure

The authors disclose that LLMs assisted with editing and polishing the manuscript language. They state that the authors conducted the scientific work.

LLM Usage Disclosure

Supplementary Material for RETROAGENT

The supplementary material describes the agent’s system prompt and tool interface, then positions RETROAGENT among search-based, generative, and LLM-agent retrosynthesis methods.

A System Prompt

  • The system prompt frames retrosynthesis as solving a target into commercially available building blocks through valid reactions and explains OR molecule nodes, AND reaction nodes, and automatic status propagation.Source: A System Prompt · p. 14
  • The prompt’s core loop predicts templates, expands selected reactions, checks the frontier, and repeats until the route is solved, with a maximum depth of 30 reaction steps.Source: A System Prompt · p. 14

B Tool Interface Details

  • The interface separates graph tools using symbolic molecule, reaction, and prediction IDs from chemistry tools that take SMILES strings.The symbolic IDs are intended to prevent copying errors.Source: B Tool Interface Details · p. 15
  • Memory tools retrieve candidate reactions, expand one or multiple branches, expose open frontier molecules, and inspect node status and connections.Source: B Tool Interface Details · p. 15
  • Chemistry tools provide molecular properties, building-block availability, synthetic accessibility, and synthetic complexity information to help prioritize candidates.Source: B Tool Interface Details · p. 15

C Related Work

  • RETROAGENT uses a template-grounded single-step model for chemical validity but replaces fixed search algorithms with an RL-trained LLM agent that learns its own strategy.Source: C Related Work · p. 15
  • Traditional search-based methods use fixed heuristics and algorithm-determined expansion order, while RETROAGENT lets the agent control search strategy end-to-end through RL.Source: C Related Work · p. 16
  • Template-free generative approaches can leverage broad chemical knowledge but typically lack the structural guarantees of template-grounded methods.Source: C Related Work · p. 16

D Limitations and Discussion

The discussion cautions that template-valid routes do not establish full laboratory practicality and that LLM-based planning carries greater inference cost than traditional search.

D Limitations and Discussion

  • Template-valid routes ending in purchasable building blocks are chemically valid by the evaluation protocol, but that validity omits yield, selectivity, conditions, and protecting-group strategy.A shorter or template-valid route is not necessarily more practical in the laboratory.Source: D Limitations and Discussion · p. 16
  • A systematic, large-scale assessment of route practicality, ideally with experimental validation, remains an open problem for the field.Source: D Limitations and Discussion · p. 17
  • LLM inference makes RETROAGENT substantially more expensive per search step than traditional value-network or policy lookups, limiting its economy for latency-sensitive or very high-throughput screening.The paper describes planning as a one-time offline computation of seconds to minutes per target.Source: D Limitations and Discussion · p. 17

E Training Details

Table S1 lists RETROAGENT’s training hyperparameters, while the training curves show pass@1, reward, and episode length improving over 400 steps.

E Training Details

  • Table S1 lists the full training hyperparameters for RETROAGENT.Source: E Training Details · p. 17
  • Training pass@1 rises from ~30% to ~80% within 50 steps and reaches ~90% by step 400.Mean reward stabilizes around 0.75 over the same training trajectory.Source: E Training Details · p. 17
  • Mean turns per episode increase from ~12 to ~27 as the agent learns to sustain longer, more productive search episodes.The mean reward follows a similar trajectory, stabilizing around 0.75.Source: E Training Details · p. 17

F Evaluation Protocol

Evaluation uses iterative greedy rollouts with up to 120 turns and a 120K-token budget, and a same-backbone comparison tests the interface’s contribution. The reproduced Retro-R1 interface scores below RETROAGENT on USPTO-190.

F Evaluation Protocol

  • Evaluation uses greedy decoding and successive rollouts per target, with each rollout capped at 120 turns and a 120K-token budget.Ten independent experiments per target report mean and standard deviation; stochasticity comes from shuffled prediction and frontier orderings.Source: F Evaluation Protocol · p. 17
  • A rollout ends when the root is solved, a stopping budget or condition is reached, or a turn exceeds the 8,192-token response limit.Reaching the 30-step route-depth limit blocks only that branch, not the full rollout.Source: F Evaluation Protocol · p. 17
  • Table S1 covers full fine-tuning of Qwen3-4B-Instruct-2507 on 8×H200 GPUs using slime, Megatron, and SGLang.Source: Table S1 · p. 18

G.1 Same-Backbone Comparison with Retro-R1

  • 53.26% pass@1 for RETROAGENT exceeds the reproduced Retro-R1 interface’s 45.05 ± 0.29% on the same Qwen3-4B backbone.The reproduced interface also falls below the original Retro-R1 result of 50.00% on Qwen2.5-7B.Source: Table S2 · p. 19, §G.1 · p. 19

G.2 Reward Sensitivity

Reward sensitivity tests vary the relative weights and overall magnitude of budget and depth penalties. Keeping both terms performs best at pass@1, while stronger penalties trade first-attempt success for budget efficiency.

G.2 Reward Sensitivity

  • The sensitivity study varies the relative weighting and overall magnitude of budget and depth penalties across configurations otherwise identical to RETROAGENT.USPTO-190 success rates are reported as mean±std over five runs.Source: §G.2 · p. 19
  • 48.32% pass@1 results when both penalty rates and caps are doubled, while the N = 200 budget curve rises to 75.58%, approaching the full model’s 78.63%.Harsher penalties make the agent more conservative, trading first-attempt success for budget efficiency.Source: §G.2 · p. 19
  • 53.26% pass@1 is highest when both penalty terms are retained, versus 51.16% with budget-only and 50.84% with depth-only.Dropping either term lowers pass@1 only modestly, indicating robustness to their relative balance.Source: Table S3 · p. 19, §G.2 · p. 19

H Out-of-Distribution Analysis

ChEMBL-1000 differs from training targets mainly in scaffold novelty and molecular similarity, not synthetic accessibility. RETROAGENT’s success rates on scaffold-novel targets are close to its full-set rates.

H Out-of-Distribution Analysis

  • 96.2% of ChEMBL-1000 scaffolds are novel, versus 74.8% for USPTO-190; mean maximum Tanimoto similarity to training is 0.345 versus 0.483.85% of ChEMBL-1000 targets fall below the USPTO-190 median similarity.Source: H Out-of-Distribution Analysis · p. 20
  • Median synthetic accessibility is 3.18 for ChEMBL-1000 and 3.55 for USPTO-190, so the measured shift is driven by scaffold novelty and dissimilarity rather than raw complexity.Source: H Out-of-Distribution Analysis · p. 20
  • 73.8% pass@1 and 84.2% at N ≤500 (810/962) are achieved on scaffold-novel ChEMBL-1000 targets, close to full-set rates of 74.4% and 84.6%.The passage attributes this pattern to policy signals such as building-block proximity rather than memorized scaffold-to-template associations.Source: H Out-of-Distribution Analysis · p. 20

I Failure Case Analysis

Failures are overwhelmingly budget-driven rather than caused by exhausted candidate or action-space limits, while the example transcripts show both completed routes and stalled searches.

I Failure Case Analysis

  • ∼96% of USPTO-190 failures and ∼94% of ChEMBL-1000 failures end through max_turns or no_progress; dead_frontier and token_limit together stay under 7%.The environment logs four mutually exclusive termination reasons.Source: I Failure Case Analysis · p. 20, Table S5 · p. 21
  • Failed targets receive similar numbers of first-step predictions as solved targets: 15.0 vs. 17.9 on USPTO-190 and 16.6 vs. 17.0 on ChEMBL-1000.The analysis attributes failures to exhausting the search budget on hard, deep targets, not a lack of initial candidates.Source: I Failure Case Analysis · p. 20

J Example Transcripts

  • Two solved examples finish in 11 turns with 4 predict calls at depth 4 (18 molecule nodes), and 14 turns with 5 predict calls at depth 5 (22 nodes).Source: J Example Transcripts · p. 21
  • The two failed examples run 112 and 103 turns, reach depth 30, and end in no_progress, matching the hard, deep-target pattern.Both transcripts show sustained predict-expand-frontier cycles without reaching building blocks.Source: J Example Transcripts · p. 22

Related papers

Loading 2607.14512v2…