PaperlayerView exact revision on arXiv

Source-linked AI summary

Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective

Jiatong Li, Yunqing Liu, Wenqi Fan, Xiao-Yong Wei, Hui Liu, Jiliang Tang, Qing Li

arXiv:2306.06615v2cs.CLcs.AI

TL;DR

Molecule-caption translation needs methods that reduce expert dependence, computational demands, and limited generalization. MolReGPT retrieves molecularly similar examples for in-context learning with ChatGPT, avoiding domain-specific pre-training and fine-tuning. It achieves competitive translation performance, including 0.585 and 0.593 Text2Mol scores for Mol2Cap and Cap2Mol.

  • Problem

    Existing molecule-caption translation methods rely on domain experts, costly pre-training and fine-tuning, and often generalize poorly to unseen examples.

  • Method

    MolReGPT retrieves similar molecules and captions using molecular similarity, then supplies them as context examples for LLM-based Mol2Cap and Cap2Mol without fine-tuning.

  • Results

    MolReGPT achieves 0.585 and 0.593 Text2Mol scores in Mol2Cap and Cap2Mol, respectively, comparable to MolT5-large and higher in Cap2Mol.

  • Takeaways & Limitations

    In-context few-shot molecule learning provides a paradigm for applying LLMs to molecule-caption translation without domain-specific pre-training or fine-tuning.

  • Takeaways & Limitations

    Increasing the number of context examples can trigger LLM input-length limits, causing performance to converge unless the maximum input length expands.

Abstract

from arXiv · show

Molecule discovery plays a crucial role in various scientific fields, advancing the design of tailored materials and drugs. However, most of the existing methods heavily rely on domain experts, require excessive computational cost, or suffer from sub-optimal performance. On the other hand, Large Language Models (LLMs), like ChatGPT, have shown remarkable performance in various cross-modal tasks due to their powerful capabilities in natural language understanding, generalization, and in-context learning (ICL), which provides unprecedented opportunities to advance molecule discovery. Despite several previous works trying to apply LLMs in this task, the lack of domain-specific corpus and difficulties in training specialized LLMs still remain challenges. In this work, we propose a novel LLM-based framework (MolReGPT) for molecule-caption translation, where an In-Context Few-Shot Molecule Learning paradigm is introduced to empower molecule discovery with LLMs like ChatGPT to perform their in-context learning capability without domain-specific pre-training and fine-tuning. MolReGPT leverages the principle of molecular similarity to retrieve similar molecules and their text descriptions from a local database to enable LLMs to learn the task knowledge from context examples. We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation. Experimental results show that compared to fine-tuned models, MolReGPT outperforms MolT5-base and is comparable to MolT5-large without additional training. To the best of our knowledge, MolReGPT is the first work to leverage LLMs via in-context learning in molecule-caption translation for advancing molecule discovery. Our work expands the scope of LLM applications, as well as providing a new paradigm for molecule discovery and design.

1 INTRODUCTION

Molecule-caption translation supports both describing molecules in text and generating molecules from captions, but existing approaches depend on expert-designed architectures, costly pre-training and fine-tuning, and limited generalization. MolReGPT addresses these challenges through retrieval-based in-context learning with LLMs, achieving competitive performance without fine-tuning.

  • Task definition: Molecule-caption translation comprises Mol2Cap, which describes molecular features in text, and Cap2Mol, which generates a molecule from a caption.Mol2Cap captions can cover structure, IUPAC names, molecular families, and chemical features; Cap2Mol generates the corresponding SMILES string.
  • Research gap: Existing approaches rely on domain experts, costly pre-train-and-fine-tune pipelines, and struggle to generalize to unseen examples.These limitations motivate a new paradigm for molecule-caption translation.
  • Motivation: LLMs offer potential for molecule-caption translation because their in-context learning can generalize to unseen tasks without fine-tuning, reducing computational cost.Their natural-language understanding, generation, and reasoning capabilities support this opportunity.
  • Proposed approach: MolReGPT uses molecular-similarity retrieval to provide in-context molecule-caption examples to ChatGPT for Mol2Cap and Cap2Mol without domain-specific pre-training or fine-tuning.The framework retrieves similar captions and molecules to guide translation.
  • Results: 0.585 and 0.593 Text2Mol scores are achieved by MolReGPT in Mol2Cap and Cap2Mol, respectively, with performance comparable to MolT5-large and higher in Cap2Mol.The reported gains over MolT5-large are 0.5% for Mol2Cap and 6% for Cap2Mol.

2 RELATED WORK

Prior molecule-discovery research uses deep learning and domain-specific pre-training to connect molecular representations with text. Related work includes retrieval, machine reading, self-supervised language–SMILES modeling, and graph–text comparative learning.

  • Molecule Discovery: Molecule-discovery methods have used CNNs, RNNs, and Transformers for advanced deep molecular representation learning.These approaches adapt architectures developed in other fields.
  • Molecule Discovery: Text2Mol retrieves molecules from natural-language descriptions by learning a shared semantic embedding space from paired molecule–text data.Its dataset pairs molecules with corresponding text descriptions.
  • Molecule Discovery: KV-PLM links SMILES strings with biomedical text through a domain-corpus-pre-trained machine-reading system.The method inserts SMILES strings into its processing pipeline.
  • Molecule Discovery: MolT5 pre-trains on unlabeled language text and SMILES strings to enhance molecule captioning and text-based molecule generation.Its pre-training supports both directions of molecule-caption translation.
  • Molecule Discovery: MoMu bridges molecular graphs and natural language by comparatively pre-training molecular graphs with semantically related text.It uses comparative learning to connect the two modalities.

3 MOLREGPT

MolReGPT equips LLMs for molecule-caption translation without fine-tuning by retrieving chemically similar examples, presenting them as context, and calibrating generated responses. Its retrieval uses molecular fingerprints and Dice similarity for molecule queries and BM25 for caption queries.

  • MolReGPT Workflow: The workflow comprises molecule-caption retrieval, prompt management, in-context few-shot learning, and generation calibration.Retrieved molecule-caption pairs are incorporated into system prompts before querying the LLM.
  • Molecule-Caption Retrieval: Morgan Fingerprints encode molecular substructures as binary vectors, enabling retrieval based on structural similarity.Each bit represents the presence or absence of a particular substructure, and the representation handles varying molecule sizes and small structural changes.
  • Molecule-Caption Retrieval: Dice similarity compares Morgan Fingerprints by measuring shared substructures between two molecules.The method explicitly exposes detailed structural similarities that are often stated directly in molecule captions.
  • Caption Retrieval: For Cap2Mol, BM25 ranks caption examples by query-term relevance within the local caption corpus.The ranking considers term frequency, inverse document frequency, caption length, and query-term position.
  • In-Context Few-Shot Molecule Learning: MolReGPT teaches LLMs molecule-caption translation without additional model training by using retrieved context examples.The framework targets both Mol2Cap and Cap2Mol through in-context few-shot learning while retaining the LLM’s original parameters.
  • Generation Calibration: Generation Calibration validates response formats, applies correction strategies, and re-queries when outputs remain invalid.For maximum-input-length errors, the longest example is removed before retrying.

4 EXPERIMENT

MolReGPT improves molecule-caption translation across Mol2Cap and Cap2Mol without additional fine-tuning, with performance shaped by model choice, retrieval strategy, and the number of in-context examples. Its main limitation is sensitivity to input-length constraints and incomplete caption-pattern coverage.

  • Performance Comparison: 59% improvement over zero-shot and 2.4% over MolT5-base in Text2Mol are achieved by 10-shot GPT-3.5-turbo for Mol2Cap.The result indicates that in-context examples substantially improve GPT-3.5-turbo’s molecule-caption translation performance.
  • Performance Comparison: MolReGPT remains model-agnostic: Llama-2-7B reaches 0.409 BLEU-4 and 0.466 Text2Mol without further fine-tuning, close to T5-base.The authors report consistent caption-generation improvements across three different language models.
  • Performance Comparison: 15% and 20% Text2Mol gains over MolT5-base are obtained by GPT-3.5-turbo and GPT-4-0314, respectively, for Cap2Mol.GPT-4-0314 additionally achieves a 7% improvement over MolT5-large and an average 8.1% fingerprint-score improvement.
  • Qualitative Analysis: MolReGPT captures molecular structure more accurately than competing models in qualitative examples, but limited examples reduce its ability to learn caption vocabulary and grammar.The reported GPT-4-0314 Text2Mol score is 0.593 versus 0.554 for MolT5-large, while MolReGPT obtains lower ROUGE scores.
  • Impact of Retrieval Strategies: Morgan FTS is best for Mol2Cap retrieval, outperforming BM25 by 37% in Text2Mol, whereas BM25 is best for Cap2Mol because it captures specific caption details.Morgan FTS nearly doubles ROUGE-L relative to Random or BM25; BM25 better matches structural terms that semantic embeddings may miss.
  • Impact of Example Number for In-Context Learning: Performance generally rises with more in-context examples, but gains plateau from 5 to 10 shots because of LLM maximum input-length limits.For example, BM25 improves Cap2Mol BLEU from 0.489 to 0.790 and Text2Mol from 0.479 to 0.571 as examples increase.

5 CONCLUSION

MolReGPT is a retrieval-based in-context learning paradigm for molecule-caption translation that uses molecular similarity to guide LLM predictions without fine-tuning. It achieves competitive Text2Mol performance with GPT-4-0314, including outperforming MolT5-large on Cap2Mol.

  • MolReGPT retrieves similar molecule-caption examples from a local database to guide LLM generation without fine-tuning.It uses BM25 for caption retrieval and Morgan Fingerprints with Dice similarity for molecule retrieval.
  • MolReGPT achieves 0.585 and 0.593 Text2Mol scores in Mol2Cap and Cap2Mol, respectively.
  • GPT-4-0314 reaches comparable MolT5-large performance in Mol2Cap and outperforms MolT5-large in Cap2Mol without fine-tuning.
  • The framework offers a versatile in-context learning paradigm for deploying LLMs in molecule discovery.

6 BROADER IMPLICATION & FUTURE DIRECTIONS

MolReGPT works across different LLMs without additional pre-training or fine-tuning and shows potential for broader scientific use. The authors identify stronger language models and improved retrieval as directions for further development.

  • MolReGPT enables GPT-4 to achieve comparable or superior performance to existing methods without additional pre-training or fine-tuning.The authors also describe better generalization potential for customized inputs than fine-tuned methods.
  • The framework's model-agnostic design supports application across various LLMs for molecule discovery.
  • Future work includes developing stronger retrieval algorithms, combining BM25 with chemical LLMs, and applying Graph Neural Networks for molecular similarity.
  • The authors anticipate that broader LLM access could benefit scientific researchers and drug discovery.
Loading 2306.06615v2…