Dynamic language model representations for multi-objective reaction optimisation

1. Introduction

• Reaction optimisation must navigate combinatorial component spaces while balancing objectives such as yield, selectivity, cost, and safety.The challenge arises in pharmaceutical process and preclinical chemistry as well as academic exploration of diverse reaction parameters; Bayesian optimisation has been applied in both low-data and parallel high-throughput settings.Source: §1 · p. 1
• Descriptor engineering depends on prior chemical intuition and often fails to transfer across distinct components, while one-hot encoding is sparse and captures no chemical similarity.The introduction notes that monodentate-phosphine descriptors only partly apply to bidentate ligands and that heterogeneous bases and additives may lack a shared descriptor space.Source: §1 · p. 2, Figure 1 · p. 2
• Retrospective benchmarks across nickel- and palladium-catalysed cross-couplings converged on high-performing conditions in fewer experiments than descriptor-library and one-hot baselines across sequential and high-throughput settings.The campaigns included sequential low-data optimisation and large-batch 96-well plate experimentation.Source: §1 · p. 3
• Alice encodes textual reaction conditions with a LoRA-fine-tuned language model, objective-specific projection heads, and independent Gaussian process surrogates trained jointly to refine embeddings at each optimisation iteration.The LLM-GP architecture is trained end-to-end by joint marginal log-likelihood maximisation.Source: Figure 1 · p. 2, §1 · p. 3
• The study shows that effective multi-objective optimisation can be achieved in reaction spaces without descriptor computation or feature engineering, even when constructing a shared descriptor representation would itself be substantial.Source: §1 · p. 3

2. Computational results

• The comparison used Kraken and COSMO-RS descriptors and one-hot encoding in the same qLogNParEGO loop and experimental budget; T5-base with mean pooling performed best among evaluated language models and pooling strategies.Performance was measured by mean iterations to 90% and 95% of maximum hypervolume across 20 random seeds.Source: §2 · p. 3

2.1. Sequential low-data reaction optimisation

• Sequential low-data optimisation evaluated nickel- and palladium-catalysed Suzuki couplings using yield and selectivity objectives; each campaign began with 5 Sobol-sampled experiments for diverse initial coverage.The nickel dataset contained 384 experiments from 88,000 combinations; the palladium dataset contained 404 experiments from 59,040 combinations.Source: §2.1 · p. 4
• Nickel Suzuki reached 90% hypervolume in ~13 LLM-GP iterations versus ~26 DFT and ~24 one-hot, and 95% in ~24 versus ~28 and ~30, respectively.Across both sequential case studies, LLM-GP also had a higher proportion of runs reach both thresholds within budget.Source: Figure 2 · p. 4, §2.1 · p. 5
• Palladium Suzuki reached both 90% and 95% hypervolume in ~65 LLM-GP iterations, versus ~80 for both DFT descriptors and one-hot.Across 20 random-seed runs, LLM-GP had a higher proportion of successful optimisations within the allocated budget.Source: Figure 2 · p. 4, §2.1 · p. 5

2.2. Highly parallel HTE reaction optimisation

• In 96-well plate HTE, LLM-GP reached practical convergence in ~1.2 plate iterations beyond initialisation, versus ~2.4 for both baselines.Each plate represented 96 conditions; campaigns typically required approximately one week, and LLM-GP had a higher proportion of successful runs. The same trend held for 24- and 48-well batches.Source: Figure 2 · p. 4, §2.2 · p. 5

3.1. Case study 1: Palladium-catalysed cyanation

• The cyanation search space comprised 30,000 conditions spanning 40 palladium catalysts, three cyanide sources, five additive options, ten solvents, two co-solvents, and three temperatures.The components included mono- and bidentate ligands and cyanide sources with differing toxicity, creating a feature-engineering challenge for conventional descriptors.Source: §3.1 · pp. 5–6, Figure 3 · p. 6
• With K4[Fe(CN)6] fixed for round two, cyanation yielded multiple conditions with >99% conversion and >99% selectivity across monodentate and bidentate ligand families.The first-round top hits relied on NaCN or Zn(CN)2, whereas K4[Fe(CN)6] initially gave mediocre outcomes.Source: §3.1 · p. 6, Figure 3 · p. 6
• After two rounds and 192 experiments, cyanation conditions scaled to gram scale delivered 94% isolated yield while evaluating less than 1% of the design space.The selected conditions used the lower-hazard cyanide source K4[Fe(CN)6].Source: Figure 3 · p. 6, §3.1 · p. 7

3.2. Case study 2: Asymmetric ketone hydrogenation

• Syn-selective hydrogenation outcomes rose from 5/96 to 59/96, and the Pareto front reached 87.1% de (syn) and 93.2% ee (syn).Within the productive catalyst family, de (syn) ranged from -67% to +87.1%; its sign inverted in 12 cases after changing temperature or base loading alone.Source: Figure 4 · p. 7, §3.2 · p. 8
• After two rounds and 192 experiments, hydrogenation conditions scaled to gram scale delivered 84% isolated yield and 99.6% enantiomeric excess.The campaign evaluated about 2.4% of its 8,064-condition design space.Source: Figure 4 · p. 7, §3.2 · p. 8

4. Outlook

• Conditions identified in two prospective wet-lab campaigns translated directly to preparative scale within two rounds of high-throughput experimentation.The approach is intended to let optimisation begin before system-specific mechanistic understanding is established.Source: §4 · p. 8

Representing reaction conditions

• Text prompts represent diverse reaction components in one framework, avoiding the need to select or compute system-specific descriptors.The work benchmarks this approach against one-hot encoding and molecular descriptors; descriptor coverage varies by component class.Source: §4 · p. 8, Representing reaction conditions · p. 9

Metrics for multi-objective optimisation problems

• Optimisation quality is assessed by normalised hypervolume percentage relative to the true Pareto front, using the common reference point (0, 0) for benchmarks.Hypervolume rewards convergence and diversity along the candidate Pareto front.Source: Metrics for multi-objective… · p. 9

Representing reaction conditions

• The language model converts reaction-condition prompts into pooled embeddings, using mean pooling for T5 and BART and last-token pooling for Qwen2.5.Mean pooling was also evaluated for Qwen2.5, and CLS pooling for BART.Source: Representing reaction conditions · p. 9

Overview of optimisation loop

• Bayesian optimisation starts with Sobol-sequence experiments, then iteratively fits surrogates and selects experiment batches using qLogNParEGO.Batch candidates are selected greedily, conditioning each selection on previously chosen pending points.Source: Overview of optimisation loop · pp. 9–10

LLM-based deep kernel Gaussian process surrogate

• Objective-specific projection heads and Gaussian processes are trained jointly with a shared language model, so representations adapt to predictive performance across objectives.Gradients from all objectives update the shared language model, while each projection head receives gradients only from its corresponding objective.Source: LLM-based deep kernel Gaussian process… · p. 10

Parameter-efficient fine-tuning of language models

• The framework updates language-model embeddings as observations accumulate, using LoRA to fine-tune only a small subset of pretrained parameters.LoRA injects trainable low-rank decompositions into pretrained weight matrices.Source: Parameter-efficient fine-tuning of… · p. 10

Competing interests

• J.W.S., D.M.S., M.D.R.L., A.A., R.P.B., K.P., M.J.N. and R.B. declare potential financial and non-financial conflicts as F. Hoffmann-La Roche Ltd. employees; the other authors declare none.Source: Competing interests · p. 11

• Sequential benchmarks compare the three representation methods on nickel- and palladium-catalysed Suzuki couplings using iterations to reach 90% and 95% hypervolume and the share of runs reaching those thresholds within budget.Error bars show standard error across 20 random seeds.Source: Figure 1 · p. 19
• High-throughput benchmarks compare the methods on palladium-catalysed sulfonamide coupling across 24-, 48-, and 96-well batches, measuring plate iterations and the share of runs reaching 90% and 95% hypervolume within budget.Error bars show standard error across 20 random seeds.Source: Figure 2 · p. 19

B. Influence of different pre-trained models and pooling strategies

• T5-base with mean pooling consistently reached 90% and 95% hypervolume thresholds in the fewest iterations across nickel- and palladium-catalysed Suzuki datasets.The comparison covered T5-chem, BART-base, T5-base, T5-small and Qwen2.5-0.5B with the pooling strategies specified in the figure caption.Source: Figure 3 · p. 20
• For Suzuki optimisation, the ablation also compared the percentage of runs reaching 90% and 95% hypervolume thresholds within the experimental budget.Runs used 20 random seeds and spanned the pre-trained model architectures and pooling strategies described in the Methods.Source: Figure 4 · p. 21
• Sulfonamide-coupling HTE ablations assessed plate iterations and budget-limited threshold attainment for 24-, 48- and 96-well batches.The comparisons covered 90% and 95% hypervolume thresholds and 20 random seeds; the supplied captions do not report a winning model or pooling strategy.Source: Figure 5 · p. 22, Figure 6 · p. 23

• 40 mono- and bidentate Pd catalysts, 10 solvents, 3 cyanide sources, 2 co-solvents, 5 additives, and 3 temperatures define the prospective Pd-catalysed cyanation search space.Source: Figure 7 · p. 24
• 32 chiral catalysts, 7 solvents, 9 bases, 2 base mol%, and 2 temperatures define the prospective asymmetric ketone hydrogenation search space.Source: Figure 8 · p. 25

1 High-throughput experimentation (HTE) platform

• A custom UnchainedLabs platform integrated two Big Kahuna systems, a LiCotel system, and a glove box; products were analysed by LC-MS and chiral HPLC.The setup is shown schematically in Supplementary Figure 1; LC-MS used a Waters ACQUITY UPLC I-Class with QDa, and chiral HPLC used a Chiralcel OJ-3 column.Source: §1 · p. 27, Figure 1 · p. 28
• Solids were dispensed by Mettler balance, liquids were added manually, and reactions ran in 96-position blocks or a Screening Pressure Reactor.Targets below 0.4 mg were dispensed as coated ChemBeads.Source: §1 · p. 27
• HTE OS and SpotFire were used to tag chromatographic peaks, excluding “other SM”, “solvent”, and “ignore peak” signals from LCAP calculations.Ligand and precatalyst-component peaks were tagged as “ignore peak”.Source: §1 · p. 27

2.1 HTE experimental procedure

• HTE reactions were run in sealed 96-well plates at 400 rpm for 20 h, then diluted and analysed by LC-MS.Vials contained palladium catalysts, cyanide sources, solid additives, methyl 3-bromo-5-fluorobenzoate, solvents, cosolvent, and NEt3 where applicable.Source: §2.1 · p. 29

2.2 Scale-up and synthesis of methyl 3-cyano-5-fluorobenzoate

• At 21.5 mmol, cyanation with [Pd(allyl)(Xantphos)]Cl in DMC/H2O at 75 °C for 4 h afforded methyl 3-cyano-5-fluorobenzoate in 3.61 g (94%) yield.The scale-up used 0.50 eq K4Fe(CN)6 · 3 H2O and 1.0 mol% palladium catalyst.Source: §2.2 · p. 29

3.1 HTE experimental procedure

• In nitrogen-filled glovebox plates, 1 mol% Ir/Ru catalysts and bases were screened overnight under 20 bar H2, then samples were analysed by chiral HPLC.Catalysts and solid bases were dispensed into 1 mL vials in 96-well plates; each received 20 mg (95.1 µmol) of 1,2-diphenylpropan-1-one in 200 µL solvent, followed by liquid bases where applicable. After reaction, solvent was removed and residues were redissolved in 200 µL EtOH.Source: §3.1 · p. 32

O OH [Ir(Cl)H2((S)-DTB-SpiroPAP-3-Me)] (1 mol%)

• 1,2-Diphenylpropan-1-ol was isolated as a purified crystal in 1.68789 g (83.6%) yield.The reaction used 1 mol% [Ir(Cl)H2((S)–DTB–SpiroPAP–3-Me)], 19.2 equivalents tAmOH, 0.5 equivalents DBU, and 20 bar H2 at 60 °C; reaction completion was reached within 4 h.Source: O OH [Ir(Cl)H2((S)-DTB-SpiroPAP-3-Me)]… · p. 32

3.3 Enantiomeric data augmentation

• Enantiomeric data augmentation added a reflected training observation for each reaction, inverting ee (syn) while retaining conversion and de (syn).All design-space solvents and bases were achiral, and the 32 chiral catalysts were present as enantiomeric pairs; augmentation was used to fit surrogate models only.Source: §3.3 · p. 35

3.4 Model sensitivity to initial training data

• The initial-data sensitivity test removed the top 1, 3, or 5 Plate 1 conditions ranked by de (syn), then refit surrogates without running new experiments.Because evaluation used only suggestions already measured in round 2, reported recovery values are lower bounds on performance at each ablation level.Source: §3.4 · p. 35
• With all five initial syn-selective conditions removed, the model recovered conditions reaching 83.4% de (syn) and 93.3% ee (syn).No syn-selective examples remained in the training data. With the top condition removed, recovered conditions reached 99.4% conversion, 75.4% de (syn), and 89.5% ee (syn); with the top three removed, they reached 64.2% de (syn).Source: §3.4 · p. 35