From What to Why: A Multi-Agent System for Evidence-based Chemical Reaction Condition Reasoning

1 INTRODUCTION

• Selecting solvents, temperatures, catalysts, and reagent ratios matters for reaction success, selectivity, and scalability, while traditional exploration relies on extensive human labor.The introduction also describes retrieval-based and reasoning-based LLM approaches to condition recommendation.Source: §1 · p. 1
• Existing reaction-condition approaches rarely explain why proposed conditions are appropriate, including which functional groups govern reactivity or which constraints rule out alternatives.The paper identifies this explanatory gap as limiting utility in high-stakes scientific workflows.Source: §1 · p. 1
• ChemMAS analyzes reactivity, retrieves condition exemplars, refines candidates through pairwise agent comparisons, and aggregates rationales with mechanistic, evidence, and constraint checks.The system parses SMILES to identify functional groups, balance stoichiometry, and infer plausible by-products before multichannel database retrieval.Source: §1 · pp. 1–2
• 20–30% Top-1 similarity gains over specialized chemical models and 10–15% average gains over leading general-purpose LLMs are reported for ChemMAS.The reported comparisons include RCR and Reagent Transformer, and GPT-5 and Gemini 2.5.Source: §1 · p. 2
• The paper recasts condition recommendation as producing both conditions and evidence, using chemistry-aware tools, multichannel recall, mechanistic reasoning, constraint verification, and debate-based aggregation.The stated goal is interpretable, falsifiable condition reasoning.Source: §1 · p. 2

2.1 PROBLEM DEFINITION

• Each proposed condition configuration must pass hard checks, meet an evidence-alignment threshold, and have a derivation coherent with mechanistic reasoning and evidence.Each configuration is paired with a rationale comprising domain reasoning, verifiable checks, aligned evidence, and a concise derivation.Source: §2.1 · p. 2

2.2 OVERVIEW

• The pipeline stores a Reaction Report in shared memory, retrieves condition hypotheses, ranks candidates through specialized-agent pairwise comparisons, and returns K verified configurations with rationales.Agents focus on condition dimensions such as catalyst, solvent, and reagent under context-aware constraints.Source: §2.2 · p. 3

2.3 GENERAL CHEMIST

• The General Chemist uses functional-group, constraint, and knowledge-base tools to identify motifs, infer balanced stoichiometry and by-products, and retrieve reaction-type evidence.Its outputs are written to memory for downstream condition prediction.Source: §2.3 · p. 4

2.4 MULTI-CHANNEL RECALL

• Multi-Channel Recall queries a structured reaction base by reaction type, reactants, and products, then deduplicates the matching entries into experience-driven proposals.The candidate pool can also include recombined conditions, with optional feasibility filtering and removal of infeasible or near-duplicate combinations.Source: §2.4 · p. 4

2.5 CANDIDATE PAIRING AND TOURNAMENT SELECTION

• Tournament Selection reduces 5,000 candidates to a Top-50 through shuffled pairwise matchups and majority voting, avoiding global-score calibration across heterogeneous conditions.Confidence-sum tie-breaks resolve ties; head-to-head comparisons are made under comparable context.Source: §2.5 · p. 4

2.6 MULTI-AGENT DEBATE

• Four agents assess each paired option by seeking evidence, consulting reaction memory and constraints, refining judgments through peer summaries, and voting on the winner.Agents can re-query when uncertain, and a facilitator manages debate and conflicts before the winning option advances.Source: §2.6 · p. 5

3.1 CHEMICAL TEACHING

• Chemical Teaching uses supervised fine-tuning on reaction-and-candidate prompts to teach stepwise tool invocation and structured judgments, yielding a cold-start model for chemical condition reasoning.Training trajectories include Chemical Knowledge Base and Memory searches, and judgments that critique responses and select a preferred option.Source: §3.1 · p. 5

3.2 TOOL INCENTIVIZATION

• Tool Incentivization applies GRPO with a hierarchical reward for valid format, answer correctness, and collaborative multi-tool use, aligning the policy beyond answer accuracy alone.The multi-tool bonus applies when both tools appear; the objective uses group-normalized advantages and KL regularization to a fixed reference policy.Source: §3.2 · pp. 5–6

4.1 TRAINING AND EVALUATION SETTING

• The AGen and multi-agent system are trained as separate models from Qwen3-8B-Instruct under a shared two-stage SFT-and-RL framework, with different objectives and tools.Top-k Similarity measures the maximum Tanimoto similarity of the ground truth to predicted candidates, averaged across path-based, MACCS, and Morgan fingerprints.Source: §4.1 · p. 6

4.2 DATASETS

• The private organic-reaction dataset contains 544,591 SMILES reaction entries, uses reactants and products as inputs and five condition fields as outputs, and is split 8:1:1.A 90-instance ChemCoTBench-RCR subset provides a lightweight public benchmark for evaluating generalization and stability under distribution shift.Source: §4.2 · p. 6
• Figure 4 presents intermediate-output accuracy against expert annotations, human-alignment scores using LLM-Score and BLEU-4, and the LLM-Score evaluation workflow.The center panel uses blue bars for LLM-Scores and green bars for BLEU-4 scores.Source: Figure 4 · p. 7

5.1 MAIN RESULTS

• ChemMAS surpasses specialized models by 70% to over 90% and GPT-5 and Gemini 2.5-Pro by 15–25% in relative Top-1 similarity across reaction types and Top-k settings.Table 2 reports Top-k similarity across five condition types on the private dataset.Source: §5.1 · p. 7, Table 2 · p. 8

5.2 GENERALIZATION EVALUATION ON OUT-OF-DISTRIBUTION DATA

• On ChemCoTBench, ChemMAS leads the second-best model by 16.5% for catalysts, 13.7% for solvents, and 11.1% for reagents in Top-1 prediction.Catalyst accuracy is 62.1%; the strongest solvent and reagent competitor is GPT5, while Gemini 2.5-Pro is second-best for catalysts.Source: §5.2 · p. 7, Table 1 · p. 7

5.3 EVALUATION OF MODEL INTERPRETABILITY

• The General Chemist reaches 95.8% MainFG, 90.2% by-product, and 92.5% reaction-type accuracy against human ground truth.The paper describes this alignment as a verifiable basis for downstream reasoning.Source: §5.3 · pp. 7–8
• ChemMAS earns an LLM-Score of 92.8 versus 62.5–77.2 for baselines and a BLEU-4 score of 0.26 on generated reasoning trajectories.The LLM-Score checks whether rationales logically support expert-derived QA pairs.Source: §5.3 · p. 8

5.4 ADDITIONAL QUANTITATIVE ANALYSIS

• Specialized agents raise average Top-1 by 8.5% for catalysts, 11.6% for solvents, and 18.4% for reagents; all three yield 16–19% macro-average gains.Figure 5 compares condition-type improvements from agents added to the AGen+AF ull base.Source: Figure 5 · p. 9, §5.4 · p. 9
• Removing Main FG lowers performance by an average of 8.4%, while removing multi-step reasoning decreases similarity by 12.3%.The ablations also include memory elements, multi-agent debate, and candidate pairing.Source: §5.4 · pp. 8–9
• Removing SFT degrades Top-k similarity more than removing RL; removing either task accuracy or the multi-tool bonus also degrades results.The authors interpret these ablations as support for complementary components in the two-stage training and hierarchical reward design.Source: §5.4 · p. 9, Table 4 · p. 9

• ChemMAS reframes reaction-condition recommendation as evidence-based reasoning grounded in domain knowledge, mechanistic constraints, and interpretable evidence.Source: §6 · p. 9
• ChemMAS achieves up to 30% Top-1 similarity gains over specialized models and outperforms general LLMs.Source: §6 · p. 9
• The authors identify materials design and bioinformatics as future extensions of the framework.Source: §6 · p. 9

Supplemental Material of ChemMAS

• The supplement covers methodology, training and evaluation settings, metrics, dataset details, prompts, and additional experimental results.It also includes reproducibility assets and qualitative analysis.Source: Supplemental Material of ChemMAS · p. 13

A THE USE OF LARGE LANGUAGE MODELS

• GPT-5 assisted with writing and Figure 2 images, while Google Nano Banana assisted with Figure 3 images; the authors designed both figures.Source: A THE USE OF LARGE LANGUAGE MODELS · p. 13

B.1 REACTION CONDITION PREDICTION

• Reaction-condition research spans large-scale neural prediction, role-specific ranking, reaction-center interpretability, missing-reagent imputation, and retrieval-augmented methods.The cited examples include Gao et al. (2018), Maser et al. (2021), Wang et al. (2023), Andronov et al. (2023), and Qian et al. (2023).Source: §B.1 · pp. 13–14

B.2 LLM-BASED MULTI-AGENT SYSTEMS

• Prior chemistry agents use tools for laboratory control, synthesis, and condition recommendation, while multi-agent research explores debate, role specialization, consensus, and communication.The review notes that factuality and reliability in reaction-condition prediction remain largely unexplored.Source: §B.2 · p. 14

B.3 LLM-BASED REASONING MODELS

• LLM reasoning work includes code-assisted logic, chain-of-thought, debate, and chemistry memory, but mechanistic or contextual rationales for reaction conditions are rarely addressed.Source: §B.3 · p. 14

• For each candidate pair, agents parse reaction-report keywords, seek knowledge-base citations, refine assessments using peer summaries, and consult constraints when uncertainty arises.The process yields per-agent decisions with confidence and citations.Source: §C.1 · pp. 14–15
• Agents compare candidates in repeated pairwise rounds; majority vote selects each winner, confidence sums break ties, and winners advance until the Top-50 remains.Final assessments and citations are stored in Memory.Source: §C.1 · p. 15
• Training first uses SFT on tool-annotated trajectories, then GRPO reinforcement learning with rewards for valid format, correctness, and collaborative multi-tool use.The second stage starts from the SFT policy and optimizes tool-augmented rollouts.Source: §C.1 · p. 16

D.1 TRAINING PIPELINE

• AGen and the multi-agent system use SFT followed by GRPO; training uses one epoch, batch size 128, and 8 NVIDIA A100 GPUs.SFT uses AdamW with learning rate 2 × 10−5; RL uses learning rate 1 × 10−6 and KL coefficient 0.04.Source: §D.1 · pp. 16–17
• The General Chemist learns tool use through step-wise supervision for functional-group tagging, constraint checks, and knowledge-base search, followed by a reward combining format, answer similarity, and collaboration.Its input is Reactant and Product SMILES, and its output is the predicted reaction type.Source: §D.1 · p. 17
• Role-specialized agents share a trained backbone and learn knowledge-base and memory searches from candidate-pair QA, with rewards aligning judgment quality and tool collaboration.The roles are AF ull, ACat, ASol, and ARea.Source: §D.1 · p. 17

D.2 EVALUATION SETTING DETAILS

• General-purpose LLMs rank the same Top-5000 pool and return Top-50 lists per condition head, controlling candidate access so comparisons reflect ranking and evidence integration rather than retrieval coverage.The pool combines reaction-base retrieval, functional-group cues, constraint heuristics, and memory lookup.Source: §D.2 · p. 17

E EVALUATION PROTOCOL DETAILS

• Predicted condition SMILES are compared with ground-truth annotations using molecular fingerprint similarity metrics.SMILES are canonicalized before fingerprint calculation, with stereochemistry excluded to focus evaluation on constitutional structure.Source: E EVALUATION PROTOCOL DETAILS · p. 17, §E.1 · p. 17

E.1 SMILES CANONICALIZATION AND VALIDITY

• Validity measures the share of predictions RDKit can parse as molecular graphs; invalid predictions receive a similarity score of zero across fingerprint metrics.The canonicalization procedure standardizes SMILES and excludes stereochemical information.Source: §E.1 · p. 17

E.2 TANIMOTO SIMILARITY

• Fingerprint similarities use the Tanimoto coefficient, comparing shared set bits with the bits set in either molecular fingerprint.The passages define a, b, and c as the set-bit counts in fingerprint A, fingerprint B, and both, respectively.Source: §E.2 · pp. 17–18

E.3 MOLECULAR FINGERPRINT TYPES

• The protocol uses path-based RDK, 166-key MACCS, and radius-2 Morgan fingerprints to capture different aspects of molecular structure.RDK encodes paths of lengths 1–7; MACCS represents substructures, while Morgan captures local chemical environments.Source: §E.3 · p. 18

E.4 AGGREGATE EVALUATION METRICS

• Fingerprint Tanimoto Score (FTS) is the validity-weighted average of the three fingerprint similarities.The reported mean similarity is computed separately for each fingerprint type across test samples.Source: §E.4 · p. 18

E.5 TOP-k SIMILARITY

• Top-k Similarity averages each reaction’s best similarity among its k ranked predictions across the dataset.Pairwise similarity averages three fingerprint Tanimoto coefficients, assigning invalid predictions a score of zero.Source: §E.5 · p. 18

F DATASET DETAILS

• The public ChemCoTBench RCR subset has 90 samples across 10 reaction types, while the private dataset contains 544,591 organic-reaction entries.The public subset allocates nine samples per reaction type; the private dataset supplements public benchmarks with a broader reaction collection.Source: F DATASET DETAILS · p. 18
• The private dataset uses reactants and products as inputs and five condition outputs: Catalyst1, Solvent1/2, and Reagent1/2.Its dataset is split randomly into training, validation, and test sets at 8:1:1; Figure 6 depicts heavy-atom and molecular-weight distributions across the splits.Source: F DATASET DETAILS · pp. 19–20, Figure 6 · p. 19

G PROMPT TEMPLATES

• Agent prompts specify tool schemas, interaction protocols, task objectives, and JSON output fields for reaction type, functional groups, by-products, and evidence.The prompt design standardizes tool use and verifiable outputs.Source: G PROMPT TEMPLATES · p. 20

H.1 ADDITIONAL QUANTITATIVE RESULTS

• Top-5 Similarity improves with specialized agents: ACat raises Catalyst by +10.1%, ASol raises Solvent1/2 by +16.4%/+13.4%, and ARea raises Reagent1/2 by about +18.7%/+13.9%.The Full System has the most consistent Top-5 lift across all five heads.Source: §H.1 · p. 20
• Top-10 Similarity gains include +13.1% for Catalyst with ACat, +10.8%/+13.6% for Solvent1/2 with ASol, and +17.2%/+9.8% for Reagent1/2 with ARea.The Full System achieves the highest Top-10 metrics in a macro sense.Source: §H.1 · p. 20

H.2 RESULT VISUALIZATION

• Some deviations are plausible substitutions, including potassium carbonate for cesium carbonate and ethanol for methanol, illustrating valid alternatives beyond exact training-example matches.The passage characterizes these alternatives as evidence of flexibility and generalization beyond exact memorization.Source: §H.2 · p. 20
• The General Chemist prompt defines a functional-group tagger that extracts key groups from reactant and product SMILES.Its expected output includes reactant and product groups and the main functional groups.Source: §H.2 · p. 21

Multi-Agent Debate

• The multi-agent prompt provides a Chemical Knowledge Base Search for reaction type, mechanism, and precedent conditions related to SMILES or candidate reagents and solvents.The example search results include evidence references and explanatory notes.Source: Multi-Agent Debate · p. 22
• Figures 9 and 10 chart Top-5 and Top-10 similarity improvements across Catalyst, Solvent1/2, and Reagent1/2 when specialized agents are added to the AGen+AF ull baseline.The figure captions identify the measured roles and candidate-set cutoffs but do not state individual values.Source: Figure 9 · p. 23, Figure 10 · p. 23

H.2 RESULT VISUALIZATION

• Table 5 compares predicted and ground-truth conditions; the reported examples align especially well for solvents and reagents associated with reaction patterns.The text cites polar solvents for reactions with polar functional groups and palladium-based catalysts in palladium-catalyzed cross-couplings.Source: §H.2 · p. 20, Table 5 · p. 24