From Generalist to Specialist: A Survey of Large Language Models for Chemistry

1 Introduction

• Earlier chemistry language models based on BERT and T5 require task-specific fine-tuning and lack emergent abilities such as chain-of-thought reasoning and tool use.The survey distinguishes its focus on generative models with Transformer decoder architectures.Source: §1 · pp. 1–2
• General LLMs underperform on chemistry tasks, and the survey identifies domain knowledge, multimodal data, and chemistry-tool integration as key challenges.These challenges concern both the chemistry information models receive and the specialized operations they can perform.Source: §1 · p. 2
• Chemistry-specific data are scarce in pretraining, supervised fine-tuning, and reinforcement learning from human feedback.The passage notes that chemistry-related data in web-sourced pretraining corpora are minimal and that tailored data are lacking across these development stages.Source: §1 · p. 2
• Chemistry spans 1D sequences, 2D graphs, 3D structures, and spectra whose information LLMs do not fully exploit.The passage names NMR, LC-MS, and IR spectra as examples.Source: §1 · p. 2
• The absence of chemistry-tool integration hinders LLM performance in chemistry, where tools support retrieval and specialized operations.Examples include structure-knowledge retrieval and systems for reaction prediction and retrosynthesis planning.Source: §1 · p. 2
• The survey reviews efforts addressing the three challenges, chemistry-LLM benchmarks, and future research directions, and describes itself as the first systematic survey of decoder-based transfer.Its stated scope is transferring general LLMs to chemistry-specific LLMs in decoder architectures.Source: §1 · p. 2

2.1 Pre-training

• Continued pre-training on chemistry corpora in SMILES and other chemical representations is proposed to add domain knowledge to foundation models.The survey describes language modeling as estimating joint sequence probabilities through next-token prediction.Source: §2.1 · p. 2
• Chemistry pre-training data are immense, difficult to obtain, and sometimes copyright-restricted; ChemDFM is identified as the sole chemistry-corpus-pretrained chemistry LLM.ChemDFM’s corpus includes 34 billion tokens from 3.9 million papers and 49 million tokens from 1.4 thousand books; Nach0 also uses chemistry-related corpora.Source: §2.1 · p. 3

2.2 SFT

• Multi-task SFT uses chemistry instruction/output pairs across SMILES understanding, reaction understanding, notation alignment, and chemistry QA.Reported dataset volumes range from 1.5M to 3M, and task distribution affects the model’s chemistry capabilities.Source: §2.2 · pp. 3–4
• Up to 16.6% average improvement over leading LLM baselines was reported across all classification tasks for hybrid instruction tuning.Liu et al. (2024d) applied it to more than 1,000 property tasks with LLaMA2-7b-chat.Source: §2.2 · p. 4
• Exact accuracy ranging from 69% to 95% was reported across chemical text-mining tasks using minimal annotated data.Zhang et al. (2024c) extended joint named-entity recognition and relation-extraction work to additional chemical text-mining tasks.Source: §2.2 · p. 4

2.3 RLHF

• RLHF and related preference methods address hallucination, with chemistry work using rank loss or docking-software feedback to guide molecular generation and design.The survey describes PPO, DPO, AI feedback, and environment feedback; docking rewards are also used with multiple GPT agents to pursue desirable, diverse molecules.Source: §2.3 · p. 4
• Chemistry-specific human feedback is often unavailable because it requires domain expertise and effective collection tools are lacking.A GUI for exploring and evaluating molecules is presented as a way to capture chemists’ implicit knowledge and preferences more efficiently.Source: §2.3 · p. 4

3 Multi-Modal Data

• The survey organizes multimodal chemistry methods around 1D sequences, 2D graphs, 3D structures, and other modalities because chemical data is fragmented and cannot be directly processed by vanilla LLMs.It reviews approaches that integrate chemical modalities with LLMs through alignment components.Source: §3 · pp. 4–5

3.1 1D Sequences

• SMILES is often tokenized as text, which misses its inherent chemical information; specialized encoders and projection can align it with other modalities.Other sequence representations include SELFIES, IUPAC names, molecular fingerprints, and InChI; specialized encoders are also used for SELFIES and fingerprints.Source: §3.1 · p. 5

3.2 2D Graphs

• 2D graph encoders, commonly GNNs, produce vectors that projectors align with LLMs; approaches include contrastive learning, frozen-LLM generation-loss training, and hierarchical graph tokenization.Q-Former is widely used, but cited work argues it can require substantial paired data and lose fine-grained attributes or spatial locality; InstructMol instead uses a lightweight MLP.Source: §3.2 · p. 5

3.3 3D Structures

• 3D structures encode atom arrangements and distances; cited models report downstream gains, especially for 3D-dependent properties, while 3D-MolT5 reports improvements over 3D-MoLM.3D-MoLM aligns 3D conformations with other modalities using Uni-Mol and Q-Former; 3D-MolT5 instead trains 1D, 3D, and text modalities in a unified architecture with a specialized 3D vocabulary.Source: §3.3 · p. 5

3.4 Other Modalities

• Chemical images offer an intuitive representation that can use vision encoders; GIT-Mol and ChemVLM use different encoder-and-alignment designs, and ChemVLM introduced three datasets not then open-source.GIT-Mol uses Swin Transformer with cross-attention, while ChemVLM uses InternViT-6B in a ViT-MLP-LLM architecture.Source: §3.4 · p. 6
• Spectra contain structural information, but MSNovelist's de novo structure generation from tandem mass spectrometry has accuracy below 50%.The survey identifies fuller exploration of spectral information as important for advancing research in this area.Source: §3.4 · p. 6

4 Chemistry Tools

• Training and multimodal enhancement are constrained by scalability and intrinsic memory capacity, motivating tools for complex chemistry and embodied research tasks.The survey groups chemistry tools into structured knowledge retrieval, ML models, and embodied robots.Source: §4 · p. 6

4.1 Structured Knowledge Retrieval

• Structured retrieval uses sources such as databases, scientific literature, and knowledge graphs; examples include API-based database agents, synthesis-condition text mining, and top-k graph-knowledge injection.The survey identifies retrieval-augmented generation as an approach proposed to alleviate hallucinations.Source: §4.1 · pp. 6–7

4.2 ML Models

• Chemistry tool systems use APIs and specialist models for tasks including reaction planning, molecular design, and MOF generation and property prediction.ChemCrow combines reaction tools for synthesis planning, while ChatChemTS uses AI-based molecule generators for molecular design.Source: §4.2 · p. 7
• GPT-4 achieved 95.7% accuracy for MOF property prediction and 87.5% for MOF generation in ChatMOF.ChatMOF uses MOFTransformer for property prediction and a genetic algorithm for generating MOFs.Source: §4.2 · p. 7
• ChemReasoner uses GNN scores as rewards to steer an LLM toward catalysts with specified properties.Its atomistic GNNs are trained on quantum chemistry simulations and score structures.Source: §4.2 · p. 7

4.3 Embodied Robots

• Robotic chemistry systems range from autonomous experiment design and execution to generating valid programs under environmental constraints and coordinating multiple robots.Coscientist performs complex experiments; ORGANA extends CLAIRify with visual perception and support for multi-robot experiments.Source: §4.3 · p. 7

5 Benchmarks

• The survey organizes existing chemistry LLM benchmarks into science benchmarks and molecule-specific benchmarks.The two categories differ in whether they are designed specifically around molecule-related sciences.Source: §5 · pp. 7–8
• Science benchmarks span disciplines including chemistry, but are not designed specifically for chemistry and fail to address many chemistry-specific problems.Examples include SciQ, SciCode, ScienceQA, AGIEval, SciEval, SciBench, and VisScience.Source: §5 · p. 8
• Molecule-specific benchmarks assess molecule-related sciences, with tests covering general chemistry tasks, diverse LLM abilities, and molecular structure elucidation from MS/MS data.SciKnowEval includes 50,000 problems assessing knowledge coverage, reflection and reasoning, and application; MassSpecGym evaluates structure elucidation from MS/MS data.Source: §5 · p. 8

6 Future Directions

• The survey frames future work across data, model, and application directions while noting that substantial room for improvement remains.Source: §6 · p. 8

6.1 Data

• More diverse datasets and explicit reasoning paths are needed because existing datasets cover limited chemistry tasks and chemistry LLMs often lack Chain-of-Thought reasoning.The survey also notes that chemistry-specific spectra remain underused despite the structural information they can provide, including for compound identification and reaction-mechanism elucidation.Source: §6.1 · p. 8

6.2 Model

• Future chemistry LLMs should align three or more complementary modalities, since current work often pairs only two and thereby limits representation ability.The survey also proposes using advanced LLMs to assist human feedback and professional chemistry software results as alignment rewards.Source: §6.2 · p. 8

6.3 Application

• Chemistry LLMs could assist literature review, data analysis, hypothesis generation, protocol suggestions, and research-direction proposals.The survey envisions models understanding complex scientific queries and drawing insights from chemical literature.Source: §6.3 · pp. 8–9
• Integrating chemistry LLMs with automated laboratories could enable real-time experiment design, execution, and analysis, with reliability, safety, and standards compliance as requirements.Source: §6.3 · p. 9

7 Conclusion

• The survey systematically reviews adaptation of general LLMs to chemistry, highlighting challenges in domain knowledge, multimodal data, and chemistry-tool integration.Source: §7 · p. 9
• The authors conclude that chemical general intelligence remains a distant goal and identify promising future research directions.Source: §7 · p. 9

Limitations

• The paper describes its review as comprehensive but acknowledges that certain limitations may be present without specifying them here.Source: Limitations · p. 9

A Related Work

• The paper describes the intersection of LLMs and chemistry as an urgent, rapidly growing field, noting that numerous works and reviews have addressed the topic.Source: A Related Work · p. 16

A.1 General Science

• General-science surveys cover chemistry among many fields, while their broad scope limits chemistry-specific depth; another survey includes biology and BERT-style models but omits chemistry agents.Source: §A.1 · p. 16

A.2 Chemistry-Specific Surveys

• Chemistry-specific surveys variously emphasize pretrained models, molecular discovery, molecule encoding, multimodality, or agent applications, leaving some chemistry-LLM aspects uncovered.The paper also notes that some surveys classify BERT-style models as LLMs despite their need for task-specific fine-tuning and lack of emergent abilities.Source: §A.2 · p. 16

B SFT Tasks Description

• Table 1 describes commonly used chemistry SFT tasks, and Figure 4 shows the distribution of commonly used SFT datasets according to that task division.Source: B SFT Tasks Description · p. 16

• Table 2 describes the existing molecular image dataset.Source: C Molecule Image Dataset · p. 16

• Table 3 introduces existing benchmarks across subject, task type, dynamics, source, and modality.Source: D Benchmarks · p. 16, Table 3 · p. 18
• Table 1 lists the most frequently used chemistry tasks for SFT.Source: Table 1 · p. 17
• Figure 4 shows representative SFT dataset composition, with task definitions, sources, sizes, and dataset percentages indicated.Source: Figure 4 · p. 17
• Table 2 groups molecular image datasets as synthetic or realistic and describes their scale and characteristics, including closed-source or evaluation-only datasets.Source: Table 2 · p. 18
• Table 3 defines MCQ as Multi-Choice Questions and DA as Direct-Answer tasks, and notes that spectra may be represented as images or text.Source: Table 3 · p. 18