MFG: Multi-turn Feedback Guided Molecular Optimization

• MOMO aims to improve multiple properties while maintaining a similar overall molecular structure, addressing issues such as poor solubility, insufficient selectivity, or toxicity.Source: §1 · p. 1
• Existing LLM methods explore iterative feedback but lack specialized training to leverage it, while trained MOMO methods primarily generate molecules in a single turn.Source: §1 · p. 1
• MFG uses reinforcement learning to train an LLM to iteratively optimize molecules using prior-turn feedback on property changes and source-molecule similarity.Its feedback-aligned reward encourages progressive improvement across turns.Source: §1 · pp. 1–2
• 81.50% average similarity-constrained success rate across all tasks places GeLLM3O-MFG above GeLLM3O (70.81%) and BioMedGPT-Mol (77.81%).Source: §1 · p. 2

• Existing molecular-optimization RL methods primarily optimize source molecules in a single turn, treating each optimization as independent.Source: §2 · p. 2
• GeLLM3O instruction tuning may limit exploration beyond training examples, while sparse high-reward molecules can challenge effective RL exploration.Source: §2 · p. 2
• MFG develops multi-turn RL for iterative molecular optimization, which the authors describe as the first such approach for molecular-optimization LLMs.Source: §2 · p. 2

3.1 Problem Formulation

• MFG treats molecular optimization as iterative generation conditioned on the source molecule, target properties, and feedback from previous turns.The feedback records property changes and similarity to the source molecule, allowing the model to adjust its strategy across turns.Source: §3.1 · p. 2

3.2 MFG Training

• The optimization objective distributes rewards across turns to guide progressive improvement.The reward function is designed for molecular optimization and is used to train the policy.Source: §3.1 · pp. 2–3

3.2.1 Candidate Advancement Criterion

• Advance(·) carries forward candidates by prioritizing source similarity, then improvement across all properties, and, when none achieve that, candidates improving more properties.This hierarchy prioritizes the breadth of property improvements over their magnitude while preserving structural similarity.Source: §3.2.1 · p. 3
• The reward encourages property gains and source similarity while rewarding progress over the best molecule from previous turns.Below the similarity threshold, the reward penalizes similarity without accounting for property improvements; above it, the base reward combines normalized property improvement and similarity.Source: §3.2.1 · pp. 3–4

3.3 MFG Inference

• At inference, MFG selects an intermediate candidate and its feedback for the next turn, then chooses the final molecule from all generated candidates.Advance(·) determines which intermediate molecule and feedback are carried forward, helping retain promising progress for subsequent optimization.Source: §3.3 · p. 4

4.1 GeLLM3O-MFG

• GeLLM3O-MFG is based on GeLLM3O-P(6)Llama, a single-turn model previously trained on molecular optimization and structure-property relationships.The authors identify that prior training as making the base model suitable for MFG training.Source: §4.1 · p. 4

4.2 Dataset

• MuMOInstruct covers five in-distribution and five out-of-distribution tasks using combinations of six target properties.The out-of-distribution tasks assess zero-shot generalization to property combinations unseen during training.Source: §4.2 · pp. 4–5

4.3 Evaluation Metrics

• Evaluation uses success rate under a similarity threshold, average Tanimoto similarity, and average relative property improvement.A source molecule counts as successfully optimized only if all property objectives improve and the candidate meets the similarity threshold.Source: §4.3 · p. 5

4.4 Baselines

• Baselines include the GeLLM3O base model, general-purpose LLMs, and chemistry-trained LLMs, with identical inputs, inference settings, and output processing.GeLLM3O-P(6)Llama is included to assess MFG while keeping the underlying architecture fixed.Source: §4.4 · p. 5

4.5 Inference Settings

• Methods are evaluated in single-turn and multi-turn settings, with up to 20 generated molecules per source and turn counts from one to four.Single-turn baselines are extended to multi-turn inference using MFG’s feedback vector and selection criterion; the reported result uses the turn count with the highest success rate for each task.Source: §4.5 · p. 5

5.1.1 Overall comparison with baselines

• 81.50% vs. 77.81% average SR0.5 and 2.63 vs. 2.08 RI: GeLLM3O-MFG exceeds BioMedGPT-Mol, with the highest SR0.5 on 6 of 10 tasks and highest RI on 9.Its average Sim is 0.60, matching GeLLM3O and BioMedGPT-Mol.Source: §5.1.1 · p. 5, Table 1 · p. 6, Table 2 · p. 6

5.1.2 Comparison with GeLLM3O

• 10.69% higher average SR0.5 and 0.47 higher RI: under MT, GeLLM3O-MFG outperforms GeLLM3O across IND and OOD tasks.Moving from ST to MT raises MFG’s average SR0.5 by 5.27%, versus 0.76 for GeLLM3O, while average Sim falls by only 0.02.Source: §5.1.2 · p. 6

5.1.3 Comparison with other LLM baselines

• 81.50% vs. 13.57% average SR0.5 and 2.63 vs. 0.66 RI: GeLLM3O-MFG exceeds LlaSMol; its SR0.5 also exceeds Qwen3.5-9B’s 51.34%.The authors suggest broad chemistry-task training alone may not develop strong molecular-optimization or feedback-use capabilities.Source: §5.1.3 · pp. 6–7

5.2.1 Overall comparison of GeLLM3O-MFG and GeLLM3O-RL

• 75.87% vs. 70.05% average SR0.5: GeLLM3O-RL exceeds GeLLM3O under ST inference, indicating that ST RL training provides a useful learning signal.Both models share a backbone and inference setting.Source: §5.2.1 · p. 7
• 81.50% vs. 79.23% average SR0.5 and 64.45% vs. 61.78% SR0.6: under MT inference, GeLLM3O-MFG exceeds GeLLM3O-RL.The authors attribute the additional gains to MFG’s multi-turn training with intermediate feedback and the Rbest reward component.Source: §5.2.1 · pp. 7–8

5.2.2 Performance under different similarity thresholds

• At SR0.6, MFG’s average advantage over GeLLM3O-RL is 1.77% vs. 0.37% under ST and approximately 2.67% vs. 2.28% under MT, compared with SR0.5.Both models use S(·) in their reward functions; the authors suggest the stronger gains under stricter similarity constraints reflect better use of intermediate feedback.Source: §5.2.2 · p. 8

5.3 Case Study

• In the BDPQ case study, the first turn improves B, P, and Q while DRD2 activity rises from 0.03 to 0.05; the second turn prioritizes D and further improves B, P, and Q.The first-turn modifications reduce molecular weight from 718 to 461; the authors suggest a cyclic amine introduced in turn two may contribute to the D improvement.Source: §5.3 · p. 8

6 Conclusion

• MFG trains LLMs to iteratively optimize molecules using previously generated candidates and their property feedback.The framework is applied to GeLLM3O to develop GeLLM3O-MFG.Source: §6 · p. 8
• Across 10 MOMO tasks, GeLLM3O-MFG outperforms GeLLM3O and achieves competitive performance against existing baselines while improving properties and preserving structural similarity.Source: §6 · p. 8
• The analyses indicate that MFG leverages multi-turn feedback for progressive optimization and helps LLMs adapt molecular-optimization strategies across turns.Source: §6 · p. 8

Limitations

• Experiments use MuMOInstruct tasks involving six target properties, leaving effectiveness on additional property objectives unexplored.The framework shows OOD generalization to unseen property combinations within the evaluated tasks.Source: Limitations · p. 9
• MFG relies on property oracles for intermediate feedback and rewards, and the reported improvements have not been experimentally validated.Feedback reliability depends on oracle accuracy.Source: Limitations · p. 9
• Reward normalization uses MuMOInstruct target molecules, potentially limiting training on datasets without corresponding targets..Source: Limitations · p. 9
• MFG is evaluated on GeLLM3O, so its generalizability to other molecular-optimization LLMs remains unestablished..Source: Limitations · p. 9

Ethical considerations

• MFG can generate or optimize potentially unsafe compounds, and property gains plus source similarity do not establish practical safety.Generated molecules should undergo expert safety evaluation before synthesis or experimental testing and follow relevant safety regulations and guidelines.Source: Ethical considerations · p. 9

A.1 Related Work

• Transformer methods Chemformer and Prompt-MolOpt translate molecule strings, while Modof (Chen et al., 2021) and MIMOSA (Fu et al., 2021) edit molecular substructures as graphs..Source: §A.1 · p. 11
• Earlier graph- and string-based methods target specific objectives and may generalize only to a limited extent across diverse optimization tasks..Source: §A.1 · p. 11
• LLM approaches include inference-time dialogue refinement in Chat-Drug (Liu et al., 2023) and Re2DF (Le et al., 2024), instruction tuning in DrugAssist and GeLLM3O, and multi-task optimization in BioMedGPT-Mol (Zuo et al., 2025).This work fine-tunes a molecular-optimization LLM via reinforcement learning for repeated generation of progressively improved molecules.Source: §A.1 · p. 11

A.2 Selection Criterion

• Advance(·) ranks intermediate candidates according to the criteria listed in Table A.1..Source: §A.2 · p. 11, Table A.1 · p. 11
• The criteria are considered in descending priority order..Source: §A.2 · p. 11, Table A.1 · p. 11
• A lower-priority criterion is considered only when higher-priority criteria cannot identify a unique candidate..Source: §A.2 · p. 11

A.3 Reward Function

• Property improvements are normalized against a MuMOInstruct reference molecule, and average normalized improvement is computed across property objectives.The similarity component uses a threshold-based design intended to encourage structural similarity without letting it dominate optimization.Source: §A.3 · p. 11
• The reward adds a bonus when a candidate improves every target property, with a larger bonus for higher similarity; invalid outputs receive −2.Invalid outputs include malformed or unparsable SMILES and duplicates; valid molecules have a minimum reward of −1.Source: §A.3 · pp. 11–12

A.4 Implementation Details

• Training uses GRPO groups of four, up to three resampling attempts for uninformative rewards, and 100 source molecules per task excluding the five OOD tasks.Training took approximately 36 hours on two NVIDIA A100 GPUs; inference sets N to 20/T.Source: §A.4 · p. 12

A.5 Evaluation Metrics

• RI reports relative improvement by property, with its direction coefficient set to +1 for properties to increase and −1 otherwise..Source: §A.5 · p. 12

A.6 Prompt and Feedback Templates

• In multi-turn prompts, feedback gives source-to-candidate property changes and similarity, then either directs improvement of unmet objectives or provides the best prior molecule.Invalid generations receive corrective feedback identifying the failure reason.Source: §A.6 · p. 12

A.7 Additional Analysis of Experimental Results

• From T = 1 to T = 4, GeLLM3O-MFG’s average SR rises from 76.24% to 79.89%, while GeLLM3O’s falls from 70.05% to 61.64%.GeLLM3O-RL changes from 75.87% to 75.94%; the optimal number of turns varies across tasks.Source: §A.7 · pp. 12–13, Table A.2 · p. 13, Table A.3 · p. 14

A.8 Artifact Sources and Licenses

• The appendix reports artifact sources and licenses, using provider-specified licenses and noting when an artifact has no explicitly stated license..Source: §A.8 · p. 13