Source-linked AI summary
Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models
Haonan He, Yuchen Ren, Yining Tang, Ziyang Xu, Junxian Li, Minghao Yang, Di Zhang, Dong Yuan, Tao Chen, Shufei Zhang, Yuqiang Li, Nanqing Dong, Wanli Ouyang, Dongzhan Zhou, Peng Ye
TL;DR
Biological sequence understanding remains a challenge for general-purpose LLMs, which often perform near randomly on these tasks without specialized training. The paper introduces Biology-Instructions, a 21-task multi-omics instruction dataset, and trains ChatMultiOmics with a three-stage pipeline; it reports significant performance improvements from that pipeline.
Problem
Most LLMs, including GPT-4o, perform at near-random levels on biological sequence-related tasks without prior specialized training.
Source: §1 · p. 2Method
The paper introduces Biology-Instructions for multi-omics tasks and trains ChatMultiOmics using biological sequence pretraining, instruction tuning, and reasoning data.
Source: §1 · pp. 2–3, §4 · p. 6Results
Takeaways & Limitations
Biology-Instructions covers 21 tasks across DNA, RNA, proteins, and multi-molecule interactions, including single-sequence analysis and interaction-based reasoning.
Source: §1 · p. 2, §6 · p. 9Benchmarking long-chain-of-thought reasoning models on biological sequence tasks is left for future research because it is labor-intensive.
Source: §7 · p. 10
Abstract
from arXiv · showhide
Large language models (LLMs) have shown remarkable capabilities in general domains, but their application to multi-omics biology remains underexplored. To address this gap, we introduce Biology-Instructions, the first large-scale instruction-tuning dataset for multi-omics biological sequences, including DNA, RNA, proteins, and multi-molecules. This dataset bridges LLMs and complex biological sequence-related tasks, enhancing their versatility and reasoning while maintaining conversational fluency. We also highlight significant limitations of current state-of-the-art LLMs on multi-omics tasks without specialized training. To overcome this, we propose ChatMultiOmics, a strong baseline with a novel three-stage training pipeline, demonstrating superior biological understanding through Biology-Instructions. Both resources are publicly available, paving the way for better integration of LLMs in multi-omics analysis. The Biology-Instructions is publicly available at: https://github.com/hhnqqq/Biology-Instructions.
1 Introduction
The introduction frames multi-omics sequence understanding as a gap for LLMs and presents Biology-Instructions and ChatMultiOmics as a dataset and training approach for addressing it. It reports that most LLMs, including GPT-4o, perform near randomly on biological sequence tasks without specialized training.
1 Introduction
- Earlier sequence models rely on task-specific prediction heads, limiting versatility and requiring repeated fine-tuning across tasks.The introduction contrasts this with LLMs’ use of natural language as an intermediary for diverse biological tasks.Source: §1 · p. 1
- Biology-Instructions is a large-scale benchmark spanning 21 tasks across DNA, RNA, proteins, and multi-molecular prediction.It includes both single-sequence analysis and interaction-based reasoning, with reasoning-oriented instructions.Source: §1 · pp. 2–3
- Most LLMs, including GPT-4o, perform at near-random levels on biological sequence tasks without prior specialized training.The authors present this evaluation as evidence of limited inherent biological sequence knowledge.Source: §1 · p. 2
- ChatMultiOmics applies a three-stage pipeline of biological sequence pretraining, instruction tuning, and reasoning-data training.The stages are intended to warm up sequence understanding, teach instruction following, and deepen reasoning while retaining conversational capabilities.Source: §1 · pp. 2–3
- The paper positions Biology-Instructions as the first dataset designed specifically for multi-omics instruction following, including reasoning and multi-sequence, multi-molecule data.The dataset is described as undergoing a multi-stage quality-control process.Source: §1 · p. 3
2 Related works
The related-work section surveys general-purpose and biology-focused LLMs. It notes that missing specialized biological pretraining limits performance, while existing biology-focused approaches often remain restricted by omics domain or task.
2.1 Large Language Models
- 2.1 Large Language Models: Models without specialized biological pretraining have significantly limited performance in biology.The section includes both broad scientific models and closed-source state-of-the-art models in this limitation.Source: §2.1 · p. 3
2.2 Biological Large Language Models
- 2.2 Biological Large Language Models: Existing approaches include protein-focused instruction tuning, multiprotein models, multimodal systems, and DNA-focused models.The section says the efficacy of several methods is often confined to a few protein-specific tasks, while integrating encoders from diverse omics domains with an LLM is difficult.Source: §2.2 · p. 3
3 Biology-Instructions
Biology-Instructions is a large-scale dataset of multi-omics sequence tasks, paired with varied instruction templates and reasoning-focused answer construction. It covers predictive tasks across DNA, RNA, proteins, and multi-molecule interactions, with task-specific evaluation metrics.
3.1 Overview of Biology-Instructions
- Joint training across omics can improve efficiency and performance in a specific omics domain, according to the paper's reported findings.A single model handles multiple omics tasks.Source: §3.1 · p. 3
- The dataset contains over 3 million training samples and 21 subtasks spanning multi-omics biology.It is described as a large-scale foundation for biological sequence-related instruction data.Source: §3.1 · pp. 3–4
- Reasoning data construction aims to improve answer fluency and validity by prompting sequence analysis followed by a reasoned, refined answer.The construction process produced 8,000 AI-polished training examples, excluding two multi-molecule tasks.Source: §3.1 · p. 4, §3.2.3 · p. 6
3.2.1 Tasks
- The 21 tasks comprise 6 DNA, 6 RNA, 5 protein, and 4 multi-molecule tasks, including 17 single-molecule and 4 interaction tasks.The dataset focuses on predictive sequence understanding; generative applications such as sequence design are left for future work.Source: §3.2.1 · pp. 4–5
- The construction pipeline collects and categorizes data, creates instruction prompts, uses language models for quality enhancement, and applies reasoning-data principles.The overview also includes data collection, template construction, and reasoning-data construction.Source: Figure 3 · p. 5, §3.2.3 · p. 6
3.2.2 Templates
- The dataset uses varied question-answer templates to teach instruction following without overfitting to specific language patterns.Templates are manually drafted and expanded with language models; each task has 10,000 to 100,000 question-answer template pairs.Source: §3.1 · p. 4, §3.2.2 · p. 5
3.3 Evaluation Pipeline and Metrics
- Evaluation groups outputs by task type and applies tailored metrics for regression and classification tasks.Metric values are scaled by 100 and rounded to two decimal places.Source: §3.3 · p. 6
4 Model
ChatMultiOmics is based on Llama3.1-8B-Instruct and uses a three-stage training pipeline. The stages progress from biological-sequence pretraining to instruction tuning and reasoning-data fine-tuning.
4 Model
- ChatMultiOmics is trained from Llama3.1-8B-Instruct using multi-omics pretraining data and Biology-Instructions.The stated aim is to improve interactive, biology-related chat performance.Source: §4 · p. 6
4.1 Stage 1: Biological Sequences Continued Pre-training
- Continued pretraining on DNA, RNA, and protein sequences serves as a warm-up before instruction tuning.The authors report that LoRA tuning on Biology-Instructions alone yielded near-random classification and regression performance; they used LoRA+ with a learning-rate multiplier of 4 for continued pretraining.Source: §4.1 · p. 6
4.2 Stage 2: Massive Instruction Tuning
- Stage 2 tunes on Biology-Instructions without its reasoning subset, with task labels added to 30% of training questions to help distinguish tasks and output objectives.The task labels use a classification-or-regression prefix and the task name.Source: §4.2 · p. 6
4.3 Stage 3: Reasoning Instruction Tuning
- Stage 3 fine-tunes on the reasoning subset while adding 3,000 non-reasoning validation samples to preserve classification and regression performance.The reasoning data use a more detailed system prompt and a step-by-step explanation format.Source: §4.3 · p. 7
5 Results
Across the reported experiments, continued pretraining improves subsequent instruction tuning, while general-purpose LLMs perform around average or worse on biological sequence tasks. Reasoning tuning improves most tasks and extends performance to untrained tasks, though some regression tasks decline.
5.1 Experimental Setups
- The experiments compare ChatMultiOmics with general-purpose open- and closed-source LLMs and biology-specialized models, using restricted output formats and task information for baselines.Results are visualized across DNA, RNA, protein, and multi-molecule datasets, with numerical comparisons in Appendix C Tables 4–7.Source: §5.1 · pp. 7–8
5.4 Finding 3: Continued pre-training boosts fine-tuning performance.
- After continued pretraining on multi-omics sequences, Biology-Instructions LoRA tuning significantly surpasses instruction tuning without continued pretraining.The comparison is shown as an ablation across one task from each omics type.Source: Figure 6 · p. 8, §5.4 · p. 9
5.2 Finding 1: General LLMs lack biological sequence understanding capability.
- Open- and closed-source general-purpose LLMs do not surpass average performance on biological sequence tasks, including GPT-4o and GPT-4o-mini.Models of different versions or sizes show comparable performance, which the authors say suggests language capability does not directly correlate with biological sequence understanding.Source: §5.2 · p. 8
5.3 Finding 2: Current Biology-specialized LLMs fail on multi-omics tasks.
- Biology-specialized LLMs remain constrained in broader multi-omics use, lacking transfer across omics tasks and sometimes failing to follow instructions correctly.The authors also report that these models fail to outperform general-purpose baselines even on some single-omics tasks.Source: §5.3 · p. 8
5.5 Finding 4: Fine-tuning on the reasoning dataset further enhances performance.
- Reasoning-data fine-tuning improves performance on most tasks, but slightly harms performance on some regression tasks.The evaluation uses the Stage 3 model with the system prompt used for results computation.Source: §5.5 · p. 9
- With the reasoning system prompt, ChatMultiOmics extends performance to untrained antibody-antigen neutralization and RNA-protein interaction prediction tasks.The paper describes the model's reasoning capabilities as excellent under this prompt.Source: §5.5 · p. 9
5.6 Finding 5: ChatMultiOmics generalizes to out-of-domain biological sequence tasks.
- After 1,000 prompt-answer alignment samples per task, ChatMultiOmics achieves highly competitive performance across four protein-related Mol-Instructions tasks.The tasks are Protein Function, Functional Description, Catalytic Activity, and Domain/Motif Prediction.Source: §5.6 · p. 9
6 Conclusion
Biology-Instructions covers biological sequences and multi-molecule interactions, while ChatMultiOmics applies a three-stage training pipeline. The authors report significant performance improvements and present the pipeline as an effective, scalable approach for training biological LLMs.
6 Conclusion
- Biology-Instructions is a large-scale, multi-omics instruction-tuning dataset for biological sequences, spanning DNA, RNA, proteins, and multi-molecule interactions.It includes 21 diverse tasks covering single-sequence analysis and interaction-based reasoning.Source: §6 · p. 9
- Reasoning-oriented instructions are intended to improve task versatility while preserving conversational fluency.Source: §6 · p. 9
- ChatMultiOmics uses continued pre-training, massive instruction tuning, and reasoning instruction tuning; the pipeline yields significant performance improvements.The authors describe it as an effective, scalable approach for training biological LLMs.Source: §6 · pp. 9–10
7 Limitations
The dataset focuses primarily on predictive tasks, while the authors identify areas for improvement in model outputs, data coverage, evaluation, and future benchmarking. They also describe long-chain-of-thought benchmarking as labor-intensive work left for future research.
7 Limitations
- Biology-Instructions covers primarily predictive tasks, leaving generative tasks such as novel protein-sequence design for future versions.Source: §7 · p. 10
- ChatMultiOmics outputs need further improvement in practicality and reliability; hybrid biological tokenizers or encoders could reduce sequence-tokenization information loss.The authors also suggest refining the model architecture.Source: §7 · p. 10
- Adding 3D molecular coordinates could improve capture of structural functional implications, while multi-hop data could support reasoning across interconnected omics datasets.Source: §7 · p. 10
- Future evaluation should extend beyond accuracy to include interpretability, robustness, and computational efficiency.Source: §7 · p. 10
- Benchmarking long-chain-of-thought reasoning models on biological sequence tasks is labor-intensive and is left for future research.Source: §7 · p. 10
- The authors propose exploring rejection sampling to build a long-chain-of-thought-enhanced dataset, then using supervised fine-tuning or reinforcement learning to improve biological sequence understanding.Source: §7 · p. 10
A Detailed information of Biology-Instructions and Evaluation Metrics
Biology-Instructions spans predictive tasks across DNA, RNA, proteins, and molecular interactions. The section describes task definitions and the metrics and scoring procedures used to evaluate them.
A.1 Impact
- The benchmark connects multi-omics tasks to gene regulation and therapeutic research, including ncRNA–protein and antibody–antigen interaction prediction.It also covers antibody–antigen interaction classification using MCC.Source: §A.1 · p. 15, §A.2.4 · pp. 18–19
A.2.1 DNA tasks
- DNA tasks include epigenetic-mark, enhancer-activity, promoter, transcription-binding-site, and enhancer–promoter interaction prediction.Promoter Detection 300 uses a 300-bp window; core detection targets a shorter sequence upstream of the transcription start site.Source: §A.2.1 · p. 15
A.2.2 RNA tasks
- RNA tasks predict APA isoform usage, CRISPR on-target efficacy, and siRNA silencing efficiency from RNA sequences.Outputs include isoform proportions, numeric efficacy, and the percentage of mRNA remaining after siRNA treatment.Source: §A.2.2 · pp. 15, 17
- Other RNA tasks classify non-coding RNA functions and modifications or predict ribosome loading and programmable RNA-switch states.The switch task predicts ON, OFF, and ON/OFF values.Source: §A.2.2 · pp. 16–17
A.2.3 Protein tasks
- Protein tasks cover enzyme commission numbers, stability, and fluorescence prediction for higher-order GFP mutants.The fluorescence setting trains on mutants with few mutations and tests on mutants with four or more.Source: §A.2.3 · pp. 17–18
- Protein tasks also assess solubility and thermostability, with the latter using the Human-cell split and identity-clustered sequences.The solubility setting excludes training sequences above 30% identity to any test sequence.Source: §A.2.3 · p. 18
A.3 Evaluation Metrics
- Evaluation covers numeric extraction for regression, keyword-based binary classification with a sentiment-model fallback, and multi-label outputs converted to multi-hot vectors.The listed metrics include MCC, accuracy, AUC, Fmax, Spearman correlation, R2, and PCC.Source: §A.3 · pp. 19–20
A.3.1 Mixed Score Calculation
- The siRNA Mixed Score combines global MAE, in-range Range-MAE, and F1 to balance regression error with classification performance in a designated range.For siRNA, the important “low remaining” range is [0, 30].Source: §A.3.1 · p. 20
B Model Training Details
ChatMultiOmics is trained in three stages, with stage-specific parameter updates and efficiency measures. The training setup reflects computational-resource constraints.
Stage 1
- Stage 1 warms up on 523,933 RNA, 1,561,639 DNA, and 2,000,000 protein sequences, with RNA sampled twice per epoch.Sequences are capped at 2,000 characters, and training uses packing to avoid padding tokens.Source: B Model Training Details · pp. 20–21
Stages 2–3
- Stage 2 fine-tunes on 3,330,232 instruction samples, fully updating attention query/key and RMS normalization layers while using LoRA+ on other linear layers.Stage 3 fully fine-tunes all but the embedding and output layers.Source: B Model Training Details · p. 21
Training efficiency
- Training uses stage-specific optimizers and mixed precision; Stage 1 runs on 24 A100 GPUs for 1.5 days, Stage 2 on 20 for about 16 hours, and Stage 3 on 12 for 2 hours.All stages use a warmup scheduler with cosine learning-rate decay, ZeRO stage 2, and FlashAttention-2.Source: B Model Training Details · p. 21
C Additional Results
The additional results report comprehensive task evaluations, examine the effect of Stage 2 dataset balancing, and compare example responses. The examples include a correct answer produced through an incorrect reasoning path.
Dataset balance
- Balancing Stage 2 data generally reduced performance, with particularly significant drops on APA and Enhancer Activity Prediction.The authors suggest balancing may distort real-world data distributions and reduce usable data to the size of the smallest task.Source: C Additional Results · pp. 21–22
Example comparisons
- In two examples, ChatMultiOmics answered correctly while baselines failed, but one correct antibody–antigen answer followed an incorrect reasoning path.The authors suspect missing biology-specific text pretraining may relate to the faulty reasoning.Source: C Additional Results · p. 22, Figure 8 · p. 23
Comprehensive task results
- Tables 4–7 report results across DNA, RNA, protein, and multi-molecule tasks, which the authors say demonstrate the effectiveness of the dataset and three-stage pipeline.The main text presents only the radar chart and key findings; the comprehensive results are in the tables.Source: C Additional Results · p. 21, Table 4 · p. 24, Table 5 · p. 24
D Data quality control for Stage 3 Reasoning Data
Stage 3 reasoning data undergoes a multistep validation process. The authors report human reviewers’ positive assessment and describe model-based checks and independent review.
D Data quality control for Stage 3 Reasoning Data
- The authors report that PhD students and professors evaluated the reasoning data and confirmed its high quality and reliability.An example of the quality-control process is provided in Table 14.Source: D Data quality control for Stage 3… · p. 27
D.1 Self-validation by the model
- The generating model checks sequence analysis, task knowledge, reasoning, and references, then regenerates outputs that fail the criteria.The model must report its self-check and recommend improvements for outputs that fall short.Source: §D.1 · p. 27
D.2 Secondary review by an independent model
- Gemini-1.5-pro independently reviews reasoning paths, and GPT4o-mini reconstructs cases that do not pass review using Gemini’s feedback.The authors describe this review as a quality-assurance step for the training data.Source: §D.2 · p. 27
D.3 Final review by MEME tool and human
MEME-identified DNA motifs were strongly consistent with the reasoning data, providing additional validation. The accompanying examples show prompts, training pairs, and quality-control materials.
D.3 Final review by MEME tool and human
- MEME-identified motifs in DNA sequences showed strong consistency with the reasoning data, providing additional validation.The passage also says biology experts were invited, but does not provide the outcome of that review.Source: §D.3 · p. 27
- Prompt examples cover each omics type and illustrate how GPT-4o-mini reasoning data were prompted, refined, and paired with questions.The reasoning-data prompt's system message outlines construction requirements.Source: Figure 9 · p. 28, Figure 10 · p. 29
- The appendices provide example training pairs for stage 2 and three stage 3 tasks, plus an example quality-control process.The stage 3 examples cover siRNA efficiency prediction, noncoding RNA family prediction, and core promoter detection.Source: Table 10 · p. 29, Table 11 · p. 30, Table 12 · p. 31