CellAgent: An LLM-driven Multi-Agent Framework for Automated Single-cell Data Analysis

1 Introduction

• scRNA-seq analysis involves multiple steps and tool configurations, requiring programming skills and biological expertise that increase its cost.Steps may include preprocessing, batch correction, clustering, marker gene visualization, cell type annotation, and trajectory inference.Source: §1 · p. 2
• The paper frames its research question as how to build a biologically proficient LLM-agent framework for automating single-cell data analysis.The question follows growing interest in LLM-driven agents for tasks such as software programming and office-suite assistance.Source: §1 · p. 2
• General LLMs may lack accurate knowledge of specialized biological tools and struggle to retain crucial information across lengthy multi-step tasks.The paper identifies these challenges as sources of potentially unreliable outcomes.Source: §1 · p. 2
• CellAgent combines Planner, Executor, and Evaluator roles with hierarchical planning and self-iterative optimization for automated scRNA-seq analysis.The Evaluator assesses solutions from a biological expert’s perspective, while Executors can optimize plans using evaluation results and account for code execution exceptions.Source: §1 · p. 3
• CellAgent completed 92% of tasks across the full dataset, more than doubling the completion rate of direct GPT-4 use.On 22 expert-annotated datasets, its performance matched or surpassed several leading tools.Source: §1 · p. 3

2.1 A multi-agent collaborative automated workflow for single-cell RNA-seq analysis

• CellAgent takes user data and natural-language task descriptions through automated task decomposition, sequential subtask execution, and optimization.The Planner, Executor, and Evaluator collaborate within a hierarchical decision-making framework and self-iterative optimization mechanism.Source: §2.1 · pp. 3–5
• CellAgent's Planner decomposes natural-language requests into analysis subtasks, while Executors sequentially run them and the Evaluator selects the best solution.The Evaluator's judgments can prompt tool selection or hyperparameter tuning; execution exceptions also trigger regenerated solutions.Source: §2.1 · p. 5

2.2 CellAgent enables efficient batch correction

• CellAgent averaged 0.684 overall versus scVI's 0.642 and ranked first on four batch-correction datasets.Its overall scores had a median of approximately 0.69, and heart-dataset UMAPs showed cells of the same type from different batches mixed together.Source: Figure 2 · p. 6, §2.2 · p. 7

2.3 CellAgent facilitates more accurate cell type annotation

• CellAgent achieved the highest average annotation accuracy among seven methods across datasets spanning five human tissues and mouse tissues.The benchmark included human PBMC, liver, lung, and pancreas data, plus mouse retina.Source: §2.3 · p. 7, Figure 3 · p. 8
• CellAgent effectively annotated 94% of human PBMC clusters, with full matches in 10 clusters, partial matches in five, and one mismatch.The figure also compares CellAgent's detailed PBMC annotations with expert-annotated labels.Source: §2.3 · p. 7, Figure 3 · p. 8
• CellAgent's PBMC annotations were close to expert-annotated cell-type labels.For specified genes, CellAgent also generates expression UMAP and violin plots.Source: §2.3 · p. 8

2.4 CellAgent enhances the performance of trajectory inference

• CellAgent scored 0.496 overall on gold-standard trajectory datasets, exceeding Slingshot's 0.473.It also showed advantages on NMSE rf, R2 rf, isomorphic, F1 milestones, and cor features.Source: Figure 4 · p. 9, §2.4 · p. 10
• On the “aging hsc kowalczyk” dataset, CellAgent traced differentiation from LT-HSC through ST-HSC to MPP cells.CD48 and MPO expression gradually increased during this trajectory.Source: Figure 4 · p. 9, §2.4 · p. 10

3 Discussion

• Planner, Executor, and Evaluator roles, hierarchical planning, and iterative optimization coordinate complex tasks, with retrieval, memory, and a code sandbox supporting output quality.Source: §3 · p. 10
• GPT-4V evaluates batch-correction and trajectory-inference results to guide optimization, while GPT-4 aggregates annotation results from different tools into a final annotation.Source: §3 · pp. 10–11
• CellAgent runs without human intervention, reducing analysis difficulty and cost; its open architecture lets users add knowledge and tools to better match expectations.Source: §3 · p. 11
• On expert-annotated datasets, CellAgent was in most cases best among all tools and significantly outperformed analysis using a single GPT-4 model.Source: §3 · p. 11
• Self-evaluation relies mainly on GPT-4V or GPT-4, limiting the range of optimization directions; users can specify evaluation processes for particular preferences.Source: §3 · p. 11

4.1 The composition of CellAgent

• The workflow automates task decomposition, code execution, and iterative optimization for scRNA-seq analysis tasks.Source: §4.1 · p. 11
• Three LLM-driven biological expert roles form CellAgent’s core, communicating and collaborating to think through tasks and provide solutions.Source: §4.1 · p. 11
• Memory, tool retrieval, and a code sandbox support task management and execution, allowing the expert roles to focus on solutions.Source: §4.1 · p. 11

4.1.1 Biological Expert Agents

• CellAgent constructs Planner, Executor, and Evaluator roles by assigning responsibilities and integrating natural-language expert experience and analysis tools.Users provide data, a task description, and optionally a data description or preferences.Source: §4.1.1 · pp. 11–12

Planner

• The Planner uses user requirements and a representation of the data to produce JSON task plans that support subtask extraction.Its prompt also includes role specifications and pre-collected expert experience.Source: Planner · p. 12

Executor

• For each subtask, the Tool Selector identifies available tools, while the Code Programmer generates executable code using tool documentation and prior-step code history.When execution raises exceptions, the Code Programmer generates corrected code.Source: Executor · pp. 12–13

Evaluator

• The Evaluator assesses step results and selects the best solution among Executor trials when it can; otherwise, the Code Programmer is prompted to optimize the solution.Its prompt includes evaluation methods and, during execution, the task, data representation, user preferences, and execution code.Source: Evaluator · p. 13

Memory Control

• Global memory retains the final code from each completed step, while local memory preserves the current subtask’s optimization history and resets when that subtask ends.The global memory can reduce redundant work and improve code-generation accuracy; local memory helps avoid repeated errors.Source: Memory Control · pp. 13–14

Tool Retrieval

• Tool Retrieval registers analysis tools and supplies the Tool Selector with potentially useful options and the Executor with standardized tool documentation.The documentation is intended to enhance code-generation accuracy.Source: Tool Retrieval · p. 14

Code Sandbox

• The Code Sandbox runs data loading and generated analysis steps in a Jupyter notebook via nbconvert, decoupling framework operation from code execution.The described approach is intended to improve execution security and support result management and reproducibility.Source: Code Sandbox · p. 14

4.2.1 Preprocessing

• For preprocessing, CellAgent removes low-quality cells and genes, identifies highly variable genes, and lets the Evaluator assess and optimize the code for the dataset.Unless the user specifies otherwise, preprocessing uses one self-optimization iteration.Source: §4.2.1 · p. 14

4.2.2 Batch Correction

• For batch correction, the Evaluator ranks methods using UMAP plots and selects the highest-scoring result from three iterations by default.Source: §4.2.2 · pp. 14–15

4.2.3 Cell Type Annotation

• For cell type annotation, CellAgent combines database-based and gene-expression-based methods, then uses the Evaluator to aggregate annotations and determine each cluster’s final cell type.The methods include CellMarker 2.0, ACT, CellTypist, SCSA, ScType, and GPT-4.Source: §4.2.3 · p. 15

4.2.4 Trajectory Inference

• For trajectory inference, the Evaluator scores visualizations from different algorithms and selects the highest-scoring one, with expert-written task knowledge included in its prompts.The visualizations typically use UMAP plots depicting trajectories alongside cells.Source: §4.2.4 · p. 15

4.3.1 Cell Type Annotation Metrics

• Cell type annotation accuracy is the average consistency score across cell types, assigning 1 for fully matched labels, 0.5 for partial matches, and 0 for mismatches.The scoring compares predicted and actual labels by annotation terms or cell ontology names.Source: §4.3.1 · p. 15

4.3.2 Batch Correction Metrics

• Batch correction is assessed with ten metrics: five for batch-effect removal and five for biological conservation, combined as a weighted-average overall score.The removal metrics include Graph Connectivity, PCR Comparison, iLISIGraph, kBET, and ASWbatch; conservation metrics include Isolated Labels, ARI, NMI, ASWcell, and cLISIGraph.Source: §4.3.2 · pp. 15–16

4.3.3 Trajectory Inference Metrics

• Trajectory inference uses topology, branch or milestone assignment, cell-position, and feature-agreement measures, with the overall score defined as a geometric mean.That mean uses edgeflip, F1 branches, cor dist, and cor features.Source: §4.3.3 · p. 16

• Batch correction used nine datasets from human organ atlases covering blood, bone marrow, heart, intestine, kidney, liver, lung, pancreas, and skeletal muscle.Source: §5 · p. 16
• Cell type annotation used PBMC, pancreas, multi-donor PBMC, multi-omics, immune-donor, and mouse brain-region datasets.The pancreas data included Smart-seq2, 10X Chromium, and Drop-seq.Source: §5 · p. 16
• Trajectory inference used datasets spanning human hematopoietic stem-cell aging, human embryonic development, NKT differentiation, pancreatic alpha-cell maturation, germline, cell cycle, and stimulated dendritic cells.The listed datasets are deposited on Zenodo.Source: §5 · p. 16