PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Language Models Dream of Binding Molecules? Benchmarking LLMs under Spatial Constraints

Thomas MacDougall, Maksim Kuznetsov, Roman Schutski, Rim Shayakhmetov, Maxim Malkov, Vladimir Aladinskiy, Alex Aliper, Alex Zhavoronkov

arXiv:2607.18144v1cs.LGcs.AIcs.CL

TL;DR

Whether general-purpose LLMs can satisfy complex, heterogeneous 3D constraints in molecular generation remains underexplored. This paper introduces 3D-Fit to benchmark that capability and finds emerging constraint-following abilities, although LLMs remain behind specialized diffusion models.

  • Problem

    Whether general-purpose LLMs can explicitly reason about physics and 3D environments while satisfying multiple heterogeneous spatial constraints remains systematically underexplored.

  • Method

    3D-Fit benchmarks protein-pocket-conditioned ligand generation under pocket-only, single-constraint, and multi-constraint settings using token-efficient textual conditions and structured molecular outputs.

  • Results

    LLMs show emerging ability to follow spatial constraints, perform particularly well on anchor fragments and pharmacophore points, and remain behind specialized diffusion models in docking scores.

  • Takeaways & Limitations

    LLMs possess emerging instruction-following capabilities for 3D molecular generation, especially when constraints can be directly copied, mirrored, or locally reconstructed in outputs.

  • Takeaways & Limitations

    The experimental results lack statistical significance, limiting the strength of the benchmark’s conclusions.

Abstract

from arXiv · show

Structure-based drug design (SBDD) leverages the 3D structure of protein targets, often complemented by other spatial constraints, to generate candidate binding molecules. While diffusion models have dominated as a leading paradigm for high-quality 3D molecule generation, LLM-based methods are rapidly emerging in molecular design and have shown competitive performance in pocket-conditioned molecular generation. However, their ability to reason about physics and 3D spatial environments is largely underexplored. In this work, we systematically analyze whether current general-purpose LLMs are capable of navigating complex 3D constraints compared to established baselines such as specialized diffusion models. We consider 3D ligand generation conditioned on protein pockets together with ligand- and interaction-derived spatial constraints, including anchor fragments, pharmacophore points, and mandatory pocket-ligand interactions. To enable this evaluation, we introduce 3D-Fit - a token-efficient benchmarking strategy for assessing LLM performance on multi-conditioned spatial molecule generation. Our findings reveal a clear pattern in LLM spatial capabilities: while they still lag behind state-of-the-art approaches, they are promising and can handle multiple spatial constraints simultaneously, enabling scaling to heterogeneous setups.

1 Introduction

This section frames multi-constraint 3D molecular generation as a difficult computational-chemistry problem and introduces 3D-Fit to assess whether current LLMs can satisfy geometrically grounded ligand-generation conditions. The benchmark compares LLMs with specialized diffusion methods across pocket-only, single-constraint, and multi-constraint settings.

  • Motivation: Realistic drug discovery requires 3D generative models to satisfy multiple heterogeneous spatial requirements simultaneously.These constraints may differ in scale, rigidity, and importance, and may conflict with one another.
  • Motivation: Specialized diffusion models lead standard pocket-conditioned generation, but balancing multiple heterogeneous constraints within one process remains non-trivial.The challenge arises because constraint types can differ in scale, rigidity, and importance.
  • Motivation: LLMs can process complex instructions and multiple constraints, yet their ability to reason explicitly about physics and 3D environments remains largely underexplored.Recent LLM success spans computational chemistry and drug discovery tasks.
  • Benchmark objective: The work assesses whether state-of-the-art LLMs understand, reason about, and generate ligands satisfying geometrically grounded structural constraints.The benchmark extends beyond pocket conditioning to explicit interaction patterns, pharmacophore features, and fragment anchoring.
  • 3D-Fit benchmark: 3D-Fit compares diffusion methods with proprietary and open-weight foundation LLMs across pocket-only, single-constraint, and multi-constraint protein pocket-conditioned generation.Single-constraint settings include anchor fragments or pharmacophore points, while multi-constraint settings examine LLM success rates and failure modes.

2 Related Work

Related work establishes curated protein–ligand structure resources and benchmarks for structure-based generation, while pocket-conditioned diffusion models lead current approaches. Recent methods extend diffusion generation with auxiliary geometric conditions such as fixed anchor fragments.

  • Structure-based Molecular Benchmarks: Curated resources including CrossDocked2020, PDBbind, and Binding MOAD support structure-based molecular benchmarks for protein–ligand generation.CBGBench evaluates de novo generation, linker design, and scaffold hopping using CrossDocked-style data and interaction-geometry metrics.
  • Pocket-conditioned diffusion models: Pocket-conditioned diffusion models are leading approaches and use SE(3)-equivariant processes to jointly generate molecular coordinates and atom types.TargetDiff and MolDiff introduced this paradigm, followed by methods such as PMDM and IPDiff aimed at improving binding accuracy.
  • Pocket-conditioned diffusion with auxiliary conditioning: Auxiliary-conditioning methods enhance pocket-conditioned diffusion with geometric constraints, including rigid anchor fragments preserved while generating new molecular components.DiffDec introduced a mechanism for generating new components around fixed scaffolds, while FDC-Diff is described as achieving SOTA.

3 Spatial Condition Descriptions

This section defines four spatial condition types for molecular generation: protein pockets, mandatory protein–ligand interactions, anchor fragments, and pharmacophore points. Together, these conditions represent pocket geometry, critical contacts, preserved ligand substructures, and essential binding features.

  • 3 Spatial Condition Descriptions: Protein pockets are cavities defined by residues lining the cavity, whose atoms determine its geometry and physicochemical environment.In practice, pockets are usually determined by residues within a given distance cutoff from a ligand.
  • 3 Spatial Condition Descriptions: Mandatory protein–ligand interactions specify key binding contacts that generated ligands are expected to reproduce with target-pocket residues.These interaction points provide an interpretable, target-specific binding-mode description that can guide generation or evaluation toward preserving critical contacts.
  • 3 Spatial Condition Descriptions: Anchor fragments are chemically significant ligand substructures expected to be preserved or approximately reproduced in generated molecules.They are typically extracted from a reference ligand in its bound conformation and constrain generation around observed ligand geometry.
  • 3 Spatial Condition Descriptions: Pharmacophore points describe the essential spatial arrangement of interaction features responsible for recognition and binding within a target pocket.The work uses ligand-based pharmacophores that abstract molecular structure into key functional features derived primarily from the ligand.

4 3D-Fit Benchmark

3D-Fit benchmarks LLM-based molecular generation under multiple spatial conditions using protein-pocket complexes from CrossDocked2020 and PLINDER. It uses compact textual condition descriptions and Simplified SDF outputs to reduce token use and formatting errors.

  • Benchmark scope: 3D-Fit evaluates satisfaction of multiple 3D molecular-generation conditions using protein-pocket complexes, focusing on CrossDocked2020 and PLINDER test data.The framework can use any test set of 3D pocket-ligand complexes as condition sources.
  • Condition representations: Spatial conditions are provided as compact text describing pocket-ligand interactions, anchor atoms with tolerance spheres, and pharmacophore points with 3D positions.Pharmacophore directionality is inferred from the protein-pocket description rather than specified explicitly.
  • Condition representations: The benchmark represents protein pockets compactly, removing redundant PDB information and hydrogens to reduce LLM input-token consumption.Proteins are described sequentially by residues, atoms, and coordinates, while hydrogen-depleted structures preserve the geometry emphasized by heavy atoms.
  • Output representation: Simplified SDF specifies atom indices, symbols, coordinates, charges, and bonded atom pairs with bond types, reducing textual tokens and format fragility.It consistently outperformed SMILES+XYZ on nearly all metrics for general-purpose LLMs in the reported ablation.
  • Test-set construction: The final test sets contain 948 CrossDocked2020 complexes and 505 PLINDER complexes after validity, binding-quality, molecule-size, and pocket-cutoff processing.CrossDocked2020 used RMSD < 0.5 Å and Vina score < −6, while PLINDER retained all test-set complexes.

5 Experiments

Section 5 benchmarks closed- and open-weight LLMs against specialist diffusion models using modular combinations of 3D molecular constraints. It evaluates generated structures through molecular validity and condition-success metrics across pocket-only, partially constrained, and fully multi-conditioned settings.

  • Experimental design: The benchmark compares recent closed- and open-weight LLMs with diffusion models, while 3D-Fit allows evaluation conditions to be added or removed by model capability.The experiments analyze both model success and failure modes.
  • Conditioning settings: Pocket+Fragment and Pocket+Pharmacophore conditioning are supported by only a small number of diffusion models and are evaluated in Tables 2 and 3.The specialist comparisons include models conditioned on these respective combinations of 3D constraints.
  • Evaluation metrics: The evaluation reports 3D Molecular Validity and 3D Condition Success, using Success Rate (SR, %) for binary pass/fail metrics across all outputs.Validity includes Parsed, N Hvy, and RO5, while condition success is assessed before and after UniDock pose optimization.
  • Conditioning settings: Pocket-only conditioning is the most widely supported setting, with results reported in Table 1 for LLMs and diffusion models.Diffusion models supporting additional constraints generally also support pocket-only conditioning.
  • Conditioning settings: Pocket+Mandatory Interaction+Pharmacophore+Fragment conditioning is unsupported natively by known diffusion or other models, so Table 4 compares only LLMs.The setting remains easy to prompt for LLMs, enabling evaluation of all four constraint types together.

6 Discussion

LLMs show emerging ability to follow multiple spatial constraints, especially as conditions increase, but generated conformations and raw binding poses remain inconsistent. Local optimization improves docking and intermolecular validity, although intermolecular filter success alone does not reliably indicate strong binding poses.

  • Increasing Spatial Conditions for LLMs: LLMs increasingly follow spatial constraints as the number of conditions grows, while successful parsing indicates general understanding of the required output format.PoseBusters intra-molecular filters reveal variability in generated-conformation quality, despite particularly strong performance on Anchor Fragment and Pharmacophore conditions.
  • Increasing Spatial Conditions for LLMs: Additional conditions substantially improve Pocket metrics relative to Pocket-only generation, suggesting seed-ligand information helps molecules fit inside protein pockets.The models may perform limited exploration beyond molecules directly satisfying the conditions, so added constraints may encourage more accurate chemical-space exploration.
  • Docking Scores of Generated Molecules: −6: Before local UniDock optimization, all LLM UniDock scores exceed the −6 threshold, often becoming strongly positive because of steric clashes or unfavorable placement.Local UniDock optimization substantially improves these poses, bringing most LLM scores to the moderate range.
  • Diffusion Models and Intermolecular Filters: LLMs have generally very low raw intermolecular pass rates, but local optimization can substantially improve both docking scores and intermolecular validity.Diffusion models show mixed behavior, with several methods achieving high optimized intermolecular pass rates without correspondingly strong UniDock scores.
  • Diffusion Models and Intermolecular Filters: Intermolecular filter success detects physically implausible poses but is insufficient as a standalone proxy for pose quality or binding strength.DiffSBDD, DiffPharma, SeFMol, IPDiff, and BindDM can pass many filters despite relatively weak optimized UniDock scores, whereas MolSnapper combines high validity with the best docking scores.

7 Limitations and Impact

The study acknowledges limitations from statistically non-significant results, random anchor-fragment selection, and use of a single pharmacophore point. Off-the-shelf LLMs may reduce drug-development costs but also create dual-use risks requiring safeguards.

  • Limitations: The experimental results lack statistical significance because the study prioritized analyzing more available models.This tradeoff was made to enable broader model analysis.
  • Limitations: Using random anchor fragments overlooks fixed-fragment approaches that target important substructures such as scaffolds.The study treats fragments as 3D constraints independently of chemical validity and the protein environment.
  • Limitations: Using one pharmacophore point is another acknowledged limitation of the evaluation.
  • Broader Impacts: Off-the-shelf LLMs could reduce drug-development costs, but their broad availability also raises risks of generating harmful or toxic compounds.The authors call for misuse-aware filtering, access controls in high-risk settings, and chemical-structure safeguards.

8 Conclusion

3D-Fit benchmarks 3D molecular generation under increasingly complex spatial conditions using textual representations and structured outputs. Results show that frontier LLMs can satisfy explicit local constraints and handle multi-condition prompts, but remain weaker than diffusion models in physical plausibility and binding quality.

  • 3D-Fit evaluates 3D molecular generation under increasingly complex spatial conditions with textual condition representations and a structured output format.
  • Frontier LLMs can parse spatial-constraint instructions and generate molecules satisfying explicit local 3D constraints, especially anchor fragments and pharmacophore points.These conditions can be directly copied, mirrored, or locally reconstructed in the output.
  • Adding more spatial conditions often improves pocket-related metrics, indicating that LLMs can handle multi-condition generation prompts.
  • LLM-generated poses frequently have non-optimal geometry, with poor pre-optimization docking scores often indicating severe steric clashes in protein pockets.Local UniDock optimization substantially improves scores, but they remain weaker than those of the best diffusion-based generators.
  • LLMs do not yet match diffusion models in physical plausibility and binding quality, motivating domain-specific training, improved 3D representations, and more reliable evaluation.

A Benchmarking templates

The benchmarking template prompts LLMs to generate a 3D ligand conditioned on a protein pocket, anchor fragments, pharmacophore points, and mandatory residue interactions. It prioritizes valid, plausible molecular structures and requires the result in a simplified SDF format.

  • Generation conditions: The template conditions ligand generation on protein-pocket binding, requiring favorable interactions and no clashes with pocket atoms.Pocket atoms are grouped by chain and residue and referenced with PDB-style atom names.
  • Generation conditions: The generated ligand must contain specified anchor atoms with 3D coordinates within 0.5 Angstroms of their targets.The anchor-fragment constraint is expressed directly as atom identities and coordinate tolerances.
  • Generation conditions: The template additionally requires specified pharmacophore points and interactions with particular protein-pocket residues.These constraints separately encode pharmacophore features and mandatory residue-level interactions.
  • Validity and output: Molecule validity takes priority when constraints conflict, requiring plausible bond geometry, no pocket overlap, and a valid connected 2D and 3D structure.The prompt explicitly prioritizes validity if all conditions cannot be satisfied simultaneously.
  • Validity and output: The output must enclose a simplified SDF/MolBlock representation within <sdf> and </sdf> tags.The format represents atoms and bonds explicitly, with bond order and stereochemistry encoded in the bond block.

B Generated Structures and Unidock Score Distributions

Generated poses from LLMs undergo substantially greater optimization than diffusion-model poses, and their final optimized Unidock scores differ. Figure 4 illustrates these pose changes, while Figure 6 compares score distributions across models and pose types.

  • Generated Structures: LLM raw poses differ significantly from their final optimized poses, unlike diffusion-model poses, which remain similar after optimization.Figure 4 presents examples generated for the same target.
  • Unidock Score Distributions: Figure 6 compares Unidock score distributions across models for both raw and optimized poses.The distributions show optimization effects within models and differences in final optimized scores between LLMs and diffusion models.
  • Unidock Score Distributions: LLM-generated poses are optimized more substantially than diffusion-model poses, with different final optimized scores between the model groups.Figure 6 visualizes both the degree of optimization and the final-score difference.

C Alternative Format: SMILES+XYZ

The paper benchmarks Enumerated SMILES+XYZ as an alternative 3D ligand representation and compares it with Simplified SDF under full conditioning. Simplified SDF consistently outperforms Enumerated SMILES+XYZ on this task.

  • Alternative Format: SMILES+XYZ: Enumerated SMILES+XYZ specifies a molecular graph with SMILES followed by heavy-atom 3D coordinates.The coordinates correspond to atoms in the SMILES representation.
  • Alternative Format: SMILES+XYZ: The format indexes atoms and coordinates while allowing arbitrary coordinate ordering if all heavy atoms are covered.This extension relaxes the fragility of requiring coordinates to follow the original SMILES order.
  • Comparison with Simplified SDF: On the full-conditioning task, Simplified SDF consistently outperforms Enumerated SMILES+XYZ.The task conditions on the pocket, anchor fragments, pharmacophore points, and mandatory interactions.
Loading 2607.18144v1…