PaperlayerView exact revision on arXiv

Source-linked AI summary

Verification-Guided Retrosynthesis: Reaction Feedback and Evidence Timing

Hongxuan Shi, Hankz Hankui Zhuo

arXiv:2610.38245v1cs.AI

TL;DR

A route connected to purchasable materials can still fail if one reaction is infeasible, and post-search evaluation cannot redirect exploration. VGR feeds scheduled reaction-verification verdicts back into AND/OR search; experiments find that earlier verification and immediate guidance do not combine well, motivating early pruning with delayed guidance.

  • Problem

    Retrosynthesis search can produce topologically complete routes containing infeasible reactions, while evaluating routes after search cannot change subsequent exploration.

    Source: §1 · p. 1

  • Method

    VGR adds scheduled binary reaction verification to AND/OR search and uses acquired verdicts to exclude reactions or alter search priorities.

    Source: §1 · pp. 1–2

  • Results

    Earlier verification and stronger immediate guidance each improve coverage, but their combination performs worse than either change alone.

    Source: §7 · p. 5

  • Takeaways & Limitations

    Early pruning of rejected reactions with delayed positive guidance provides a response to the interaction between evidence timing and guidance.

    Source: §7 · p. 6, §8 · p. 6

    The reported certificate does not cover reaction conditions, stereochemistry, isolation, or wet-lab execution, which remain further evidence channels.

    Source: §7 · p. 6

Abstract

from arXiv · show

Retrosynthesis planning seeks synthesis routes from purchasable starting materials to a target molecule. Search over predicted reactions can connect a target to available materials, yet an infeasible reaction can invalidate the resulting route. Evaluating route quality only after search has finished cannot correct errors made during search. We introduce Verification-Guided Reflow (VGR), incorporating reaction verification as a decision action in AND/OR search. Verification results are represented as binary verdicts; queries are scheduled from the search structure, and acquired verdicts exclude reactions or adjust search priorities. We define verified solvability as the fraction of targets for which a complete route accepted stepwise by the verifier is found. On Retro190, feeding verification results into search increases verified solvability from 51.1% to 62.6% relative to route evaluation without feedback. Further controlled experiments show that earlier verification and immediate prioritization of accepted reactions each improve route discovery, yet combining them performs worse than either change alone. We therefore propose early pruning and delayed guidance: rejected reactions are pruned immediately, while accepted reactions gain search priority when processed as part of a complete candidate route. In a separate comparison with the verification scheduling rule fixed, this strategy increases verified solvability from 61.1% with immediate guidance to 71.1%, using fewer proposal calls and unique verifier queries.

1 Introduction

Reaction verification can improve route discovery only when its verdicts affect subsequent search decisions. VGR adds verification to AND/OR search and finds more stepwise-verified routes through reaction feedback, while tests of evidence timing motivate early pruning and delayed guidance.

1 Introduction

  • A topologically complete route may still contain an infeasible reaction, so topological solvability does not establish a feasible synthesis plan.Route evaluation after search cannot reallocate exploration unless verdicts change subsequent search decisions.Source: §1 · p. 1
  • On Retro190, verified solvability is 30.5% for the default first route and 51.1% when search continues without feedback, despite 90.5% topological solvability.Verified solvability counts targets with at least one complete route whose reactions all pass stepwise verification.Source: §1 · p. 1
  • VGR treats verification as an AND/OR search action: scheduled queries acquire binary reaction verdicts, and feedback changes reaction availability or search costs.The objective is to deliver complete routes with an acquired positive verdict for every reaction under expansion and search-query budgets.Source: §1 · pp. 1–2
  • 51.1% to 62.6% verified solvability on Retro190: guide improves on post-hoc checks while the policies solve identical topological target sets.Post-hoc, prune, and guide achieve 51.1%, 59.5%, and 62.6%, respectively.Source: §1 · p. 2
  • Earlier verification and stronger immediate guidance each improve coverage, but their combination performs worse than either change alone.The tested changes move verification to the first expansion of a non-stock precursor or reduce accepted reactions’ costs to zero.Source: §1 · p. 2
  • Early pruning with delayed guidance raises verified solvability from 61.1% to 71.1% versus immediate guidance, using fewer one-step calls and verifier queries.The comparison holds verification schedule and guidance strength fixed.Source: §1 · p. 2

2 Route Discovery with Reaction Verification

The section defines route-level evidence and formalizes verification acquisition and feedback as coupled decisions in AND/OR search. It distinguishes the immediate exclusion warranted by negative verdicts from the context-dependent value of positive guidance.

2.1 Candidate routes and observable evidence

  • A solution subgraph is a target-to-stock proof during search; its extracted reaction tree is a route, topologically closed when all leaves belong to purchasable inventory.A reaction observation is distinct from a complete route certificate; delivery requires all route verdicts to be acquired and the certificate to equal 1.Source: §2.1 · p. 2

2.2 Acquisition and feedback as coupled decisions

  • A policy pairs acquisition and writeback: acquisition selects reactions to verify from the graph, evidence, and budget, while writeback changes action availability or priorities.Experiments vary these rules while fixing the proposal model, verifier, and search backend; they compare feedback and positive-credit timing.Source: §2.2 · pp. 2–3

2.3 Why the two signs have different implications

  • A negative verdict excludes every route containing that reaction, whereas a positive verdict leaves other reaction checks and precursor-to-stock obligations unresolved.Thus negative evidence can prune immediately, while positive guidance depends on the surrounding search state.Source: §2.3 · p. 3

3 Verification-Guided Reflow

VGR separates verdict acquisition from its effect on search, retaining evidence as persistent updates to the planner. It controls when reactions are checked and when positive verdicts alter priorities, while negative verdicts can prune reactions.

3.1 Persistent feedback and positive-credit strength

  • Negative verdicts remove reactions; positive verdicts can lower reaction costs and change search priorities, with credit strength and release time controlled separately.Post-hoc writes neither sign into navigation, prune applies negative updates only, and guide also releases positive credit.Source: §3.1 · p. 3
  • Table 1 separates positive-evidence release controls from verification acquisition and negative-verdict activation.The section frames whether positive feedback helps, and when it should act, as empirical questions for controlled comparisons.Source: Table 1 · p. 3, §3.3 · p. 4

3.2 Structural acquisition and sign-asymmetric release

  • Closure-time acquisition checks reactions in a complete candidate; post-investment acquisition checks a parent reaction when search first spends a call on a non-stock precursor.Post-investment acquisition retains completion checks for unknown reactions on a complete candidate.Source: §3.2 · p. 3
  • Positive credit is released when a verified-positive reaction is processed on a complete target-to-stock proof, and can remain active even if that candidate later fails verification.A negative stops that candidate attempt and triggers reselection; the complete-candidate release trigger can occur before every reaction has passed verification.Source: §3.2 · p. 3

3.3 Preserved properties

  • Pruning a rejected reaction preserves all certified routes in the currently exposed graph, but future graph exposure and finite-budget coverage still depend on subsequent search.A route removed by pruning contains the rejected reaction and already has a zero certificate.Source: §3.3 · p. 4

4 Evaluation Protocol

The evaluation uses Retro190 as its primary benchmark, then tests transfer across proposal and search interfaces and reports additional resource and external-evidence analyses. Metrics distinguish verifier-certified delivery from topological success and track several forms of computational and verification cost.

4.1 Primary setup

  • Primary experiments use all 190 Retro190 targets, full stock, top-50 MLP reactions, retro*, and at most 500 one-step calls per target.A fixed ReactionT5 verifier generates five products; a positive verdict requires the target product after canonicalization without stereochemical markers.Source: §4.1 · p. 4

4.2 Transfer, resources, and external evidence

  • Transfer tests use RetroChimera proposals with Syntheseus PUCT and a 100-target Pistachio-derived task without its source starting-material constraint.Resource comparisons use unique-query caps of 100 and 500; URSA and FREA supply expert-labeled reaction audits, and a fixed-route audit uses a forward model absent from online route selection.Source: §4.2 · p. 4

4.3 Metrics and statistics

  • Verifier-certified success requires a closed route with every step accepted, while topological success ignores verdicts; the study separately reports call, query, feedback, delivery, and search-activity measures.Paired success uses exact McNemar tests; uncertainty uses paired target bootstrap and product-cluster bootstrap for FREA.Source: §4.3 · p. 4

5 Results

The results show that verification feedback and positive-release timing affect certified route discovery, with root-delayed positive credit outperforming immediate guidance under fixed acquisition. Earlier acquisition and stronger guidance do not combine additively, while additional experiments test transfer and verifier signal.

5.1 From route checking to negative and positive feedback

  • Negative feedback raises first-route certification from 58 to 113 targets, and adding positive credit raises it to 119, while all three policies topologically solve 172 targets.Mean one-step calls decrease from 302 to 277 and 261 across post-hoc, prune, and guide.Source: §5.1 · p. 4, Table 2 · p. 5

5.2 Earlier querying and stronger guidance do not compose additively

  • Earlier acquisition raises negative-only coverage from 113 to 116, and maximal credit raises closure-time coverage from 113 to 119; combining them falls to 109/153.The certified interaction is 13/190 with uncertainty spanning zero; the topological interaction is +7.9 points (95% CI [+4.2, +12.1]).Source: §5.2 · pp. 4–5, Table 3 · p. 5

5.3 Early negative pruning with root-delayed positive credit

  • Root-delayed release certifies 135 versus 126 targets with immediate release, a 10.0-point gain (95% CI [4.7, 15.8]), using 12.7% fewer one-step calls and 11.0% fewer unique search queries.With root-delayed release, certified targets also rise to 135 from 126 with neutral release, despite identical target-wise topology and first-closure call counts.Source: §5.3 · p. 5

5.4 Transfer and external chemical evidence

  • In PUCT, root-delayed release certifies 137 versus 125 targets under maximal credit, with 35.0% fewer simulations and 62.0% fewer nodes; milder credit yields 136 versus 132.RetroChimera preserves the post-hoc–prune–guide direction, with 162/174/180 certified targets.Source: §5.4 · p. 5
  • On the Pistachio-derived task, post-hoc certifies 61 targets, neutral release 64, and immediate and root-delayed release 66 each.The task contains 100 targets and does not retain the source starting-material constraint.Source: §5.4 · p. 5
  • ReactionT5 reaches 77.1% balanced accuracy on URSA reactions and 71.4% on evaluable FREA reactions; a separate fixed-route audit accepts 181/251 guide routes versus 117/251 paired first-topological routes.The fixed-route comparison is within the cohort of 251 targets successfully solved by guide, using a condition-aware ReactionT5 model absent from route selection.Source: §5.4 · p. 5

6 Related Work

The related work places VGR alongside retrosynthesis search, feasibility checking, and lazy-search methods, while distinguishing its focus on structural query selection and verdict activation.

6 Related Work

  • Modern retrosynthesis combines one-step models with best-first AND/OR search, learned values, or Monte Carlo tree search.Source: §6 · p. 5
  • Feasibility models enable filtering, round-trip checks, and route evaluation [8] [12], while ARP learns a reaction-conditioned query policy.VGR instead selects checks using downstream search investment and complete-candidate participation, then separates acquisition from activation of each verdict sign.Source: §6 · p. 5
  • LazySP and Generalized Lazy Search evaluate expensive edges, update state, and replan; VGR studies these decisions in an online-generated AND/OR synthesis graph.A positive observation supports one reaction but leaves precursor obligations unresolved.Source: §6 · p. 5

7 Discussion

The timing of verification and the release of positive feedback interact: early pruning can help, while immediate strong guidance can hurt when precursor synthesis remains unresolved. The discussion therefore favors pruning negatives early and releasing positive guidance after a reaction appears on a complete root candidate.

7 Discussion

  • Combining earlier verification with zero-cost guidance performs worse than either change alone, because a positive verdict supports one reaction while precursor obligations remain unresolved.The discussion identifies these as separate acquisition and guidance decisions.Source: §7 · p. 5
  • Root-delayed positive release preserves early negative pruning while allowing accepted reactions to guide search after participating in a complete root candidate.Positives released before a later failure remain available to alternative proofs.Source: §7 · p. 6
  • At αmin = .25, closure acquisition leads root-delayed release 126–113 under a 100-query ceiling, but the ordering reverses to 135–128 at 500.Full-on-use reaches 144 certifications but uses 2.26 times the unique queries and loses topology.Source: §7 · p. 6
  • Continuous local-maturity rules offer another way to vary positive-credit release, with definitions and the parameter grid reported in Appendices B and E.Source: §7 · p. 6
  • Search-control certificates support chemical relevance, but reaction conditions, stereochemistry, isolation, and wet-lab execution remain additional evidence channels.The paper frames experimental route selection as a next stage.Source: §7 · p. 6

8 Conclusion

VGR retains verifier verdicts as search-state updates, distinguishing reaction identity and query accounting from evidence activation and route certification. Its operational rules implement early negative pruning and controlled positive release across search backends and policy settings.

8 Conclusion

  • A complete predicted route can fail chemical checks, so the conclusion treats evidence selection and activation as explicit design variables for verified route discovery.VGR combines retained verdicts with AND/OR search; timing and feedback controls are assessed separately.Source: §8 · p. 6

A Operational Semantics

  • Operational semantics separate reaction keys, cached verdicts, search-feedback events, unique queries, and delivery checks, while enforcing immediate negative pruning and stepwise certification.The target-local cache reuses verdicts; delivery certification is accounted for outside the search-query ceiling.Source: A Operational Semantics · p. 8, §A.1 · p. 8, §A.2 · p. 8

B.1 Three-clock controller

  • The three-clock controller acquires verdicts on structural triggers, activates negatives immediately, and releases positives either through maturity updates or root-candidate processing.A failed root proof can still contribute reusable positive evidence from its accepted prefix.Source: §B.1 · p. 8

B.2 Additive-search realization

  • In retro*, negative evidence sets reaction cost to infinity and released positives scale its original cost by α, with native value propagation updating affected proofs.Equal priorities use graph-creation order; the planner is a tree and prevents repeated molecules on a path.Source: §B.2 · p. 8

B.3 PUCT realization

  • In PUCT, negative evidence masks reaction occurrences, while released positives alter sibling prior weights; unreleased positives retain their native weights.The implementation leaves visit counts, backed-up values, rewards, and the backup rule unchanged.Source: §B.3 · pp. 8–9

B.4 Continuous structural maturity

  • Continuous release bases positive credit on structural maturity, with the release fraction raised to β and the effective multiplier moving toward αmin.Maturity increases reuse cached verdicts without another verifier call, and the one-way maximum makes released credit monotone.Source: §B.4 · p. 9

C Policy Mapping

  • Policy labels map to acquisition and activation settings: root-delayed release processes positives on a closed root proof, stopping at a negative while retaining earlier releases.PI queries after a genuine downstream expansion; full-on-use queries before that expansion and can block it.Source: C Policy Mapping · p. 9

D Experimental Accounting

The accounting distinguishes query ceilings and delivery work, and the timing–credit experiment shows that earlier acquisition and maximal positive credit do not combine additively. Statistical tests and interval procedures are paired at the target level.

D.1 Shared-ceiling interpretation

  • The shared ceiling limits unique target-local search queries, not total search-plus-delivery queries; delivery added no unique keys in the uncapped release-timing run.In a separate Msearch = 500 rerun, PI-root-delayed used 43,355 unique search-time queries and closure used 3,872.Source: §D.1 · p. 9

D.2 Statistical protocol

E.1 Timing–credit interaction

  • +7.9 points is the topological interaction (95% CI [+4.2, +12.1], exact p = .0003); certification interaction is +6.8 percentage points (95% CI [−1.1, +14.7]).Maximal credit leaves closure topology unchanged but loses 15 post-investment proofs.Source: §E.1 · p. 10

E.2 Continuous-maturity results

Continuous-maturity release rules preserve certification while reducing search work in the target-only transfer set. Shared-ceiling and acquisition-sweep results show that earlier acquisition can trade greater verifier work for fewer proposal calls.

E.2 Continuous-maturity results

  • Both soft maturity-β = 1 and investment maturity-β = 1 certify 137 targets, with neither separated from root-delayed release.Each rescues five and loses three versus root-delayed release (+1.05 points, 95% CI [−1.58, 4.21]; p = .7266).Source: §E.2 · p. 10
  • On the Pistachio-derived target-only set, investment-β = 1 rescues three targets and loses none versus root-delayed release, using fewer calls, queries, and node visits.The comparison uses 917 fewer one-step calls, 862 fewer unique queries, and 28,878 fewer node visits; required starting materials are not supplied to search.Source: §E.2 · p. 10

E.3 Shared ceilings and the acquisition frontier

  • At Msearch = 500, PI-root-delayed certifies 135 targets versus 128 for closure acquisition, 115 for immediate, and 126 for neutral.At Msearch = 100, closure acquisition instead exceeds PI-root-delayed 126–113.Source: §E.3 · p. 11, Table 10 · p. 11
  • FU versus closure rescues 27 targets and loses four, while using 7.42 times PI’s feedback events and 2.26 times its unique queries.Only 1.19% of FU’s positive observations mature into released credit; earlier acquisition exchanges more verifier work for fewer proposal calls.Source: §E.3 · p. 11

E.4 Transfer statistics

Transfer experiments find directional gains from verification feedback and delayed release, while external reaction audits provide evidence of verifier signal. The reported endpoint contrasts and verifier evaluations have distinct scopes and qualifications.

E.4 Transfer statistics

  • With RetroChimera plus retro*, positive-plus-negative feedback certifies 180 targets versus 174 with negative-only feedback, but the endpoint contrast is individually inconclusive.It rescues seven and loses one (exact McNemar p = .0703); original-model PUCT counts are 121 versus 113, with p = .0768.Source: §E.4 · p. 11
  • 13 rescues and one loss distinguish root-delayed-α = 0 from immediate-α = 0 (p = .001831); paired AUSC differences also favor root-delayed release on both budgets.The AUSC target-bootstrap 95% intervals are [+.0091, +.0406] for one-step calls and [+.0061, +.0518] for simulations; root-delayed versus neutral has five rescues and no losses (p = .0625).Source: §E.4 · pp. 11–12
  • The first-30 pilot produced 78, 75, 78, 72, and 78 verifier-certified outcomes across five policies over 90 paired target–seed runs.Inferential tests use targets, not repeated runs, as the sampling unit.Source: §E.4 · p. 12

E.5 External evidence for the route verifier

  • The verifier reaches 77.1% balanced accuracy on 1,000 expert-labeled URSA reactions and 71.4% on 740 evaluable reactions in held-out FREA.The FREA product-cluster bootstrap 95% CI is [67.8, 74.9].Source: §E.5 · p. 12
  • In the selected 251-target PaRoutes cohort, condition-aware ReactionT5 accepts 181 guide routes versus 117 first-topological routes, a paired difference of +25.5 percentage points.Mean route length changes from 3.77 to 5.08 reactions; these estimates describe the selected paired cohort.Source: §E.5 · p. 12

F Reproducibility and Limitations

The experiment archive records target-level outcomes, certification events, routes, counters, seeds, versions, and manifests. The authors identify gaps in cost matching, verifier validation, and training-corpus overlap.

Reproducibility

  • The experiment archive records target-level outcomes, first-certification events, route records, separate search and delivery counters, seeds, software versions, and file manifests.The verifier environment uses deterministic execution with TF32 disabled and batch size one.Source: F Reproducibility and Limitations · p. 12

Limitations

  • Search-time query caps do not match total realized search-plus-delivery verifier cost, and the verifier has nonzero expert-label error with no reported wet-lab synthesis.Possible overlap in the underlying training corpora remains unmeasured.Source: F Reproducibility and Limitations · p. 12
Loading 2610.38245v1…