InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery

• Drug discovery demands substantial time and expertise, while task-specific models have restricted adaptability and require laborious task-specific training.Source: §1 · p. 1
• Prior molecular LLMs face insufficient modality alignment, unexplored molecular-encoder choices, and training pipelines that neglect updating LLM knowledge.Source: §1 · pp. 1–2
• InstructMol aligns molecular graphs and chemical sequences with natural language through molecule-description alignment pretraining followed by instruction fine-tuning.Source: §1 · p. 2
• Across compound property prediction, molecule description generation, and reaction analysis, results improve LLM performance and reduce the disparity with specialized models.Source: §1 · p. 2
• InstructMol supports lightweight plugin checkpoints that can load or combine cross-modality functions while retaining a general LLM’s dialogue and reasoning capabilities.Source: §1 · p. 2

2.1 Multimodal Instruction Tuning

• Multimodal instruction-tuning work targets instruction datasets, modality bridges, and evaluation, while graph–language integration remains nascent.Source: §2.1 · p. 3

2.2 Molecule Foundation Models

• Molecule foundation models represent molecules as sequences, graphs, 3D conformations, or text, using encoder-decoder, joint-generation, and contrastive approaches for multimodal integration.Source: §2.2 · p. 3

2.3 Molecule-related LLMs

• Molecule-related LLM efforts target lead optimization, interaction prediction, and structure–activity understanding; InstructMol combines a graph encoder with sequence data in a generalist LLM.Source: §2.3 · pp. 3–4

3.1 Multimodal Instruction Tuning

• Multimodal instruction tuning conditions textual answers on an instruction and molecule-derived inputs, using an autoregressive token-prediction objective.Source: §3.1 · p. 4

3.2 Construction of Molecular Instruction

• Molecular inputs combine graph structure with SELFIES sequences, which are used to improve robustness against syntactic and semantic invalidity in SMILES.Source: §3.2 · p. 4

3.3 Architecture

• The method uses a contrastively pretrained MoleculeSTM graph encoder and a linear projector to map graph features into word-embedding space, avoiding complex aligners that require many pairs.Source: §3.3 · p. 5

3.2 Construction of Molecular Instruction

• The authors assemble 330K filtered PubChem molecule-text pairs and generate diverse task descriptions, addressing scarce expert-annotated biomolecular data.Source: §3.2 · p. 4, §3.4 · p. 5

3.4 Two-Stage of Instruction Tuning

• Task-specific tuning covers property prediction, reaction analysis, and molecule-description generation using datasets including QM, MoleculeNet, and ChEBI-20.Source: §3.4 · pp. 5–6
• The second stage initializes from alignment pretraining, keeps the graph encoder frozen, and updates the projector and LLM with LoRA to mitigate forgetting.Source: §3.4 · pp. 5–6

4 Experiments

• Experiments use a graph neural network initialized from the MoleculeSTM graph encoder and Vicuna-v-1.3-7B as the base LLM.InstructMol+GS uses graph and sequence tokens, InstructMol+G uses graph tokens only, and Instruct-S fine-tunes Vicuna-7B on downstream tasks.Source: §4 · p. 6

4.1 Property Prediction Task

• In property regression, InstructMol improves over previous single-modal instruction-tuned LLM-based methods; across three task datasets it improves accuracy over LLM-based generalists, yet performs worse than GIN on imbalanced HIV.Table 1 reports QM9 regression results in MAE (hartree); Table 2 reports classification ROC-AUC.Source: §4.1 · pp. 6–7, Table 1 · p. 6, Table 2 · p. 6

4.2 Molecule Description Generation Task

• Combining molecular structure and sequence inputs yields about 2% higher molecule-description quality than structure alone, while expert models remain more effective.InstructMol outperforms other generalist LLM-based models on the ChEBI-20 test split.Source: §4.2 · p. 7, Table 3 · p. 7

4.3 Chemical Reaction-related task

• InstructMol significantly outperforms baselines on chemical reaction tasks, while generalist LLMs struggle to generate valid molecular answers.The comparison supports its effectiveness as a chemical reaction assistant relative to single-modality models.Source: §4.3 · p. 8

4.4 Ablation Studies

• Ablations show that an MLP connector improves multimodal capability, scaling to 13B yields only minor gains, and a text-pre-aligned graph encoder improves modality alignment.Skipping stage-one alignment performs worse; freezing the LLM in stage two impedes convergence and normal inference.Source: §4.4 · p. 8

Conclusion

• InstructMol integrates molecular structural and sequential information into LLMs through dual-stage alignment pretraining and instruction tuning.The conclusion frames the model as a multi-modality foundation model connecting molecular modalities with human natural language.Source: §5 · p. 8
• Extensive experiments support the effectiveness of InstructMol’s architecture and training approach for drug-discovery tasks.The conclusion identifies practical drug-discovery applications as a potential, not an established deployment outcome.Source: §5 · p. 8

Future Work

• Future work must address dataset scale and quality, ill-defined task objectives, and more scientifically robust evaluation of hallucinations in generated outputs.Source: §5 · p. 8

6 Limitations

• Scarcity of high-quality annotated data may hinder generalization across diverse, intricate molecular landscapes in real-world applications.The authors identify dataset scale and quality as significant constraints.Source: §6 · pp. 8–9
• Further refinement is needed for robust alignment and use of molecular modalities, and for accurate interpretation and generation across the molecular domain.Source: §6 · p. 9
• The general-domain base LLM is not chemistry-specialized; the authors call for larger, more versatile domain-specific LLMs to enhance performance and expand applications.Source: §6 · p. 9

7 Potential Risks

• AI in drug discovery could be misused to develop hazardous or illicit substances, creating safety and ethical concerns.Source: §7 · p. 9
• Inaccurate AI-generated outputs could lead to hazardous chemical reactions, while dependence on them raises accident and unsafe-practice risks.The authors also note possible harm or equipment damage from hazardous reactions.Source: §7 · p. 9
• The authors call for stringent oversight and rigorous adherence to ethical guidelines to mitigate these risks.Source: §7 · p. 9

A Tasks Definition and Dataset Details

• Property prediction covers three binary MoleculeNet tasks—BBBP, BACE, and HIV—and three QM9 regression tasks focused on molecular quantum properties.Source: A Tasks Definition and Dataset Details · p. 17
• Molecule-description generation uses PubChem annotations and 33,010 ChEBI-20 pairs split 80%/10%/10%, excluding test-split compounds from PubChem annotations to prevent leakage.The datasets are described in relation to PubChem (Kim et al., 2022; Kim et al., 2019) and ChEBI-20.Source: A Tasks Definition and Dataset Details · p. 17
• Forward reaction prediction maps reactants and reagents to products using 138,768 USPTO samples from Fang et al. (2023).The dataset is attributed to Fang et al. (2023) and sourced from USPTO.Source: A Tasks Definition and Dataset Details · p. 17
• Reagent prediction maps reactants and products to the reagents involved, using data from Fang et al. (2023) sourced from USPTO_500MT.Source: A Tasks Definition and Dataset Details · p. 17
• Retrosynthesis prediction maps products to potential reactants using Fang et al. (2023) data sourced from USPTO_500MT.Source: A Tasks Definition and Dataset Details · p. 17
• The authors report scrutinizing data origins and permissions for compatibility with the research objectives and subsequent use, with sources cited throughout the paper.Source: A Tasks Definition and Dataset Details · p. 17

B Implementation Details

• The molecular graph encoder is a five-layer GIN with hidden dimension 300, initialized from the MoleculeSTM graph encoder pretrained by graph-text contrastive learning.Source: B Implementation Details · p. 17
• InstructMol uses instruction-tuned Vicuna-v-1.3-7B as its base LLM and has around 6.9B parameters.Source: B Implementation Details · p. 17
• Stage one trains on around 264K PubMed molecule-caption pairs for five epochs; stage two uses task-specific data for 20–50 epochs on description generation and 10 epochs on property and reaction tasks.Source: B Implementation Details · p. 17
• Table 5 details the data composition for the two-stage training process, while Table 6 covers data resources and licenses and Table 7 lists training hyperparameters.Source: Table 5 · p. 18, Table 6 · p. 18, Table 7 · p. 18
• Training uses batch size 128 and learning rate 8e-5, with LoRA rank 64 and scaling α=16; experiments run on four RTX A6000 GPUs.Source: B Implementation Details · p. 18

C Evaluate Metrics

• BLEU, ROUGE, and METEOR assess molecule-description proximity to ground truth on ChEBI-20, while reaction outputs are checked for validity and compared with ground truth using sequence and fingerprint metrics.The authors report convergence after 50 fine-tuning epochs on the training split, compared with prior approaches often using over 100 epochs.Source: C Evaluate Metrics · p. 18

D.1 Ablation study results

• Table 10 presents an ablation of architecture and training design on molecule description generation, with “fg” denoting the molecule graph encoder.Source: Table 10 · p. 20

D.2 More Results of Molecule Description Generation

• Figure 4 shows additional ChEBI-20 molecule-description examples with Mol-Instruction as the baseline; CID denotes a PubChem accession identifier.Source: Figure 4 · p. 20

D.3 More Results of Forward Reaction Prediction

• Figure 5 presents additional forward-reaction prediction examples, using Mol-Instruction and Multitask-Text-and-Chemistry-T5 as baselines.Source: Figure 5 · p. 21

D.4 More Results of Reagent Prediction

• Figure 6 presents additional reagent-prediction examples with Mol-Instruction as the baseline.Source: Figure 6 · p. 22

D.5 More Results of Retrosynthesis Prediction

• Figure 7 presents additional retrosynthesis examples, using Mol-Instruction and Multitask-Text-and-Chemistry-T5 as baselines.Source: Figure 7 · p. 23

D.6 Error Analysis

• The error analysis shows cases of misalignment with ground truth and reports RDKit fingerprint similarity; the authors attribute task difficulty to reaction-compound complexity and propose concatenating graph tokens from multiple molecules.They also consider separate tokenization and embedding for distinct modalities to improve semantic accuracy.Source: §D.6 · p. 24

D.7 More Results of Molecule Property Prediction

• The authors report that integrating 1D and 2D molecular modalities enhances understanding, while noting InstructMol’s initial alignment used about 300K molecule-text pairs and updated fewer than 1 million projector parameters.The comparison includes MoleculeNet multi-class tasks and domain-specific models pretrained on 11M–110M molecules; broader molecular coverage could improve performance.Source: §D.7 · p. 24, Table 11 · p. 24

D.8 From LoRA to Full-Finetuning

• Full fine-tuning results on molecule captioning are compared with state-of-the-art models, and the authors report that more trainable parameters and pretraining data further boost InstructMol’s performance.Source: §D.8 · p. 24, Table 12 · p. 25
• Despite fine-tuning-related forgetting, InstructMol retains most of the original LLM’s capabilities on zero-shot MMLU science subjects, alongside qualitative communication, common-sense inference, and reasoning skills.Source: §D.8 · p. 24, Table 13 · p. 25
• Figure 8 shows reaction-compound cases with some misalignment from ground truth and RDKit fingerprint similarities; the caption describes poor Mol-Instruction performance on these complex compounds.Source: Figure 8 · p. 25

E Comparison with Current Agents Framework

• The agent comparison covers property prediction, forward reaction prediction, and retrosynthesis; the authors report ChemCrow outputs are unstable and strongly dependent on prompt construction.They describe query refusals in retrosynthesis and reliance on internet searches for property predictions.Source: Table 14 · p. 26, E Comparison with Current Agents… · p. 26
• The authors propose augmenting domain-specific LLMs with external tools and plan to develop an instruction-following tool-use dataset for InstructMol.Source: E Comparison with Current Agents… · p. 26