PaperlayerView exact revision on arXiv

Source-linked AI summary

scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery

Yiming Gao, Zhen Wang, Jefferson Chen, Mark Antkowiak, Mengzhou Hu, JungHo Kong, Dexter Pratt, Jieyuan Liu, Enze Ma, Zhiting Hu, Eric P. Xing

arXiv:2602.11609v1cs.AIq-bio.GN

TL;DR

Single-cell workflows still lack language-native, transparent reasoning over raw omics data, limiting how biological conclusions are explained and evaluated. scPilot operationalizes omics-native reasoning through iterative LLM-guided tool use and introduces scBench for expert-verified benchmarking. Across core tasks, iterative reasoning improves reported performance while producing auditable traces of biological inference.

  • Problem

    Single-cell analysis often relies on implicit human reasoning, while existing LLM approaches either invoke fixed tools or produce opaque representations that limit interpretability.

  • Method

    scPilot converts single-cell analyses into iterative reasoning workflows in which LLMs inspect raw data, invoke specialized tools, evaluate evidence, and articulate biological inferences.

  • Results

    Iterative reasoning improves performance across benchmarks, including 11% higher annotation accuracy, 30% lower trajectory graph-edit distance, and transparent traces explaining biological ambiguity and regulatory logic.

  • Takeaways & Limitations

    By grounding LLM reasoning in raw omics data, scPilot enables auditable, interpretable, and diagnostically informative single-cell analyses.

  • Takeaways & Limitations

    Current data compression may miss subtle signals from rare cell populations, and scaling, hallucination mitigation, and wet-lab validation remain unresolved challenges.

Abstract

from arXiv · show

We present scPilot, the first systematic framework to practice omics-native reasoning: a large language model (LLM) converses in natural language while directly inspecting single-cell RNA-seq data and on-demand bioinformatics tools. scPilot converts core single-cell analyses, i.e., cell-type annotation, developmental-trajectory reconstruction, and transcription-factor targeting, into step-by-step reasoning problems that the model must solve, justify, and, when needed, revise with new evidence. To measure progress, we release scBench, a suite of 9 expertly curated datasets and graders that faithfully evaluate the omics-native reasoning capability of scPilot w.r.t various LLMs. Experiments with o1 show that iterative omics-native reasoning lifts average accuracy by 11% for cell-type annotation and Gemini-2.5-Pro cuts trajectory graph-edit distance by 30% versus one-shot prompting, while generating transparent reasoning traces explain marker gene ambiguity and regulatory logic. By grounding LLMs in raw omics data, scPilot enables auditable, interpretable, and diagnostically informative single-cell analyses. Code, data, and package are available at https://github.com/maitrix-org/scPilot

1 Introduction

Single-cell analysis still relies on implicit human reasoning, while many LLM approaches either invoke fixed tools or produce opaque representations. scPilot addresses this gap with omics-native reasoning that couples natural-language hypotheses, direct operations on raw data, evidence evaluation, and iterative refinement.

  • Single-cell pipelines remain dependent on implicit human reasoning despite the field’s shift toward million-cell expression matrices.
  • Existing LLM uses commonly act as interfaces to bioinformatics tools or encode counts into opaque vector spaces, limiting interpretability.
  • Omics-native reasoning has the LLM summarize expression data, articulate hypotheses, invoke targeted operations, evaluate numerical evidence, and revise conclusions iteratively.
  • scPilot automates cell-type annotation, trajectory inference, and gene-regulatory network prediction while preserving scientific transparency.
  • SCBENCH provides task-specific metrics and expert-verified ground truth for objective evaluation of LLMs on biologically meaningful problems.
  • 11% average cell-type annotation accuracy improvement, 26% lower trajectory graph-edit distance, and 0.03 higher GRN prediction AUROC were observed over direct prompting and conventional baselines.
  • Transparent reasoning traces expose marker ambiguities, lineage inconsistencies, and tissue-specific regulatory logic for interpretable and diagnostically informative analyses.

2 Related Work

Prior single-cell models, automated pipelines, and biological benchmarks often optimize execution or numeric performance without exposing biologically meaningful reasoning. scPilot instead combines LLM reasoning with iterative tool use and evaluates that capability on raw omics problems.

  • Single-cell foundation models mainly learn gene- and cell-level embeddings for tasks such as imputation, perturbation prediction, and cross-dataset analysis.
  • Embedding models, chat wrappers, and tool agents generally sidestep the cognitive work of arguing, justifying, and iteratively refining biological conclusions.
  • SCPILOT integrates a problem-to-text converter, an LLM planner, and a bio-tool library for iterative reasoning across three single-cell workflows.
  • Tool agents and web-based platforms automate package selection or execution, but conventional workflows retain hyperparameters, opaque defaults, or rigid heuristics.
  • Existing biological benchmarks emphasize embedding quality, numeric metrics, or factual recall rather than operation on raw omics data and transparent interpretation.

3 SCPILOT: Automation of Single-Cell Analysis by LLMs

SCPILOT operationalizes omics-native reasoning by coupling LLM claims with verifiable bioinformatics operations over single-cell data. Its modular framework converts large expression matrices into biologically salient summaries, supports iterative evidence gathering, and evaluates analyses through SCBENCH.

  • Traditional pipelines rely on manually tuned tools and implicit biological assumptions, obscuring rationale and limiting reproducibility, transparency, and interpretability.
  • SCPILOT represents each reasoning step as a natural-language claim paired with one verifiable omics operation applied to the current data state.The resulting claim-operation sequence forms a verbal and computational proof whose final state answers the biological query.
  • SCPILOT combines a problem-to-text converter, curated bio-tool library, and LLM reasoner to produce predictions alongside transparent reasoning traces.The converter preserves salient information such as cluster sizes, marker genes, trajectory connections, and transcription-factor target scores while reducing data dimensionality.
  • Iterative reasoning incorporates biological context, accumulated computational evidence, and previous mistakes without task-specific fine-tuning of LLM parameters.The framework is designed as a modular blueprint that can be customized to biological queries.
  • SCBENCH evaluates cell-type annotation, trajectory inference, and gene-regulatory-network prediction using curated datasets, task-specific metrics, and expert-supported ground truth.Its termination conditions are pre-specified to support fairness, reproducibility, and rigorous evaluation.
  • Cell-type annotation results are organized as mean ± SD scores across datasets and models, with higher values indicating better performance and top-three results highlighted.

4 Experiments

Across cell-type annotation, trajectory inference, and GRN prediction, SCPILOT’s iterative reasoning generally improved performance and exposed biologically interpretable evidence, while local open-source deployment remained impractical.

  • Cell-type Annotation: SCPILOT improved annotation accuracy for 19 of 24 model–dataset combinations, with median gains of +0.180 on Retina, +0.042 on PBMC3k, and +0.024 on Liver.Iterative dotplot-based reasoning helped distinguish closely related retinal populations that one-step marker prompting often confused.
  • Cell-type Annotation: o1 and Gemini-2.0-Pro implementations ranked highest for annotation, reaching 0.792 on PBMC3k and 0.728/0.763 on Retina.
  • Trajectory Inference: SCPILOT reduced structural errors in 10 of 21 trajectory model–metric pairs, with median improvements of GED -2.0 and spectral distance -0.14.Gemini-2.5-Pro consistently delivered optimal trajectory results, closely followed by Gemini-2.0-Pro.
  • GRN TF-Gene Prediction: SCPILOT improved GRN prediction over direct prompting by +0.098 average AUROC and generally outperformed GNN and LLM-based baselines.The o1 pipeline achieved AUROC values of 0.873 for stomach, 0.760 for liver, and 0.797 for kidney.
  • Practical Constraints: Local Gemma-3 inference required 135.7 seconds per PBMC3k evaluation on four A100 GPUs versus 8.8 seconds for GPT-4o, exceeding 15-fold slower.The authors therefore use API-based models as SCPILOT’s backbone rather than pursuing fully on-premise deployment.
  • Ablation Studies: Removing contextual metadata reduced PBMC3k annotation accuracy by 0.104 for o1, 0.063 for GPT-4o, and 0.188 for 4o-mini.The ablation indicates that contextual information is especially important for precise interpretation by smaller or mid-sized models.
  • Ablation Studies: Randomizing transcription-factor–gene annotations reduced stomach AUROC from 0.873 to 0.813 for o1, demonstrating sensitivity to accurate GO information.
  • Interpretability: Transparent reasoning traces resolved marker ambiguity, trajectory inconsistencies, and tissue-specific regulatory logic through multi-gene, diagnostic, and context-sensitive reasoning.Examples include combining NKG7 with CD3D and GNLY, auditing lineage trees with py-Monocle, and filtering spurious GO overlaps.

5 Conclusions

SCPILOT operationalizes Omics-Native Reasoning as a transparent conversational workflow for single-cell analysis, with iterative reasoning improving benchmark performance. The framework also identifies unresolved challenges involving rare-cell signals, scalability, hallucinations, and experimental validation.

  • Contributions: SCPILOT operationalizes Omics-Native Reasoning by having LLMs inspect raw single-cell data, invoke analytic tools, and articulate biological inferences.The workflow covers cell-type annotation, trajectory inference, and gene-regulatory network prediction.
  • Results: 11% annotation accuracy improvement and 30% lower trajectory graph-edit distance were reported for iterative SCPILOT reasoning with o1 and Gemini-2.5-Pro, respectively.The comparisons were against traditional one-shot prompting approaches.
  • Interpretability: SCPILOT generates transparent reasoning traces that support interpretable biological inference and expose marker ambiguities and regulatory logic.The framework is presented as shifting analysis from implicit heuristics to explicit, data-informed reasoning.
  • Limitations and Future Work: Current data compression may miss subtle signals from rare cell populations, while scaling, hallucination control, and wet-lab validation remain open challenges.Future work includes improved representations, scalable reasoning chains, robust trustworthiness methods, and in-vitro validation.

A Data and Code Availability

SCPILOT uses publicly sourced single-cell datasets and releases processed data, labels, reference networks, source code, graders, metrics, and benchmark drivers. Its model zoo spans seven proprietary models and one open-source model.

  • Data Availability: All raw and processed single-cell RNA-seq datasets used in SCPILOT are publicly sourced collections.The released processed objects include H5AD files for scRNA datasets and CSV files for GRN data.
  • Data Availability: Released benchmark materials include cell-type labels, trajectory gold standards, and GRN reference networks.These materials accompany the processed datasets.
  • Code Availability: The source code for preprocessing, automatic graders, evaluation metrics, and benchmark drivers is released under the MIT license.The repository is available through the SCPILOT GitHub project.
  • Model Availability: The evaluated model zoo contains 7 proprietary models and 1 open-source model spanning large-scale multimodal and lightweight inference-optimized variants.The design supports evaluation across diverse performance tiers and trade-offs.

B.2 Dataset description

SCBENCH evaluates cell-type annotation, trajectory inference, and GRN TF–gene prediction using curated datasets and task-specific procedures. Its evaluation combines ontology-aware annotation scoring with graph- and classification-based trajectory and GRN metrics.

  • Dataset description: SCBENCH includes 3 cell-type annotation datasets, 3 trajectory datasets, and 3 GRN tissue cohorts.Trajectory datasets are characterized by developmental timepoints and trajectory nodes, while GRN cohorts are selected from GRNdb tissues.
  • GRN TF–gene prediction: GRN evaluation labels half of the TF–gene questions positive using TRRUST-verified or SCENIC-generated edges and half negative using unverified, non-generated pairs.The procedure compares SCENIC-generated interactions with experimentally verified TRRUST relationships.
  • Cell-type annotation: Cell-type annotations are cleaned, standardized, dynamically mapped to Cell Ontology identifiers, and retained for review when direct mapping fails.Cleaning includes singularization and removal of redundant whitespace and punctuation; a language model handles names absent from the predefined dictionary.
  • GRN TF–gene prediction: GRN TF–gene predictions are evaluated with binary-classification metrics including AUROC, sensitivity, specificity, and confusion-matrix counts.AUROC measures discrimination across decision thresholds using true- and false-positive rates.

C.1 Occasional suboptimal performance

SCPILOT usually outperforms direct prompting but can underperform when model capacity is limited or datasets contain nuanced, overlapping biological signals. The clearest failure occurs on the complex Liver dataset, where extended reasoning merges related cell subtypes.

  • Overall pattern: 87 of 108 comparisons favor SCPILOT, while occasional direct-prompting wins are systematic rather than random.13 of 21 losses come from less-capable mini models with limited capacity for sustained logic.
  • Dataset-specific behavior: PBMC3k has 8 clusters and 8 cell types with distinct markers, allowing SCPILOT to perform well while mini models may saturate the performance ceiling.Its clear one-to-one cluster-to-cell-type mapping makes the dataset relatively simple.
  • Dataset-specific behavior: SCPILOT consistently improves accuracy over simpler baselines on Retina, which contains 18 clusters, 9 cell types, and slight cell-type ambiguity.Retina represents an intermediate level of dataset complexity.
  • Dataset-specific behavior: On Liver, SCPILOT can overthink overlapping developmental lineages and noisy expression patterns, merging distinct subtypes.The dataset includes 28 clusters and 31 cell types.
  • Dataset-specific behavior: 0.518 versus 0.560: o1 with SCPILOT scored lower than Direct on Liver after confusing developmentally related hepatocytes and hepatoblasts.Shared markers made deeper reasoning amplify ambiguity, whereas Direct was incidentally correct by ignoring the nuance.
  • Future improvements: Adaptive reasoning depth and marker-clarity assessments are proposed as future improvements for complex biological datasets.The analysis is intended to delineate boundaries for applying LLMs to complex biological data.

C.2 Time Cost Analysis

The analysis compares inference latency across LLMs and shows a trade-off between reasoning depth, model scale, and runtime. OpenAI models are generally faster, while Gemini and o1 incur greater latency.

  • 3.8–4.0 seconds: OpenAI models consistently returned TF-gene prediction results within this range, except o1.Full-scale Gemini models took about 30 seconds per run, while Gemini flash-thinking took approximately 11 seconds.
  • 3× extra time: o1 required roughly three times the latency of its mini variant.The comparison indicates additional reasoning depth carries a substantial runtime overhead.
  • Reasoning depth, inference latency, and model scale present clear trade-offs in the tested configurations.Gemini’s longer processing and o1’s greater latency were associated with slower execution than OpenAI baseline models.

C.3 Accessibility and Financial Cost Analysis

SCPILOT’s omics-native reasoning requires powerful LLMs, while its API-based execution can remain inexpensive. The main accessibility boundary is model capability, especially for biological knowledge and instruction following.

  • Powerful LLMs are required because omics-native reasoning is more demanding than standard text analysis.Experiments with Gemma 3 27B confirmed this requirement in the authors’ assessment.
  • Weaker models may lack the domain knowledge needed to interpret raw markers and biological pathways correctly.The limitation concerns nuanced biological understanding required for omics interpretation.
  • Complex instructions and biologist-defined output formats can disrupt weaker models’ reasoning chains.JSON formatting is given as an example of an instruction-following requirement.
  • A complete Gemini-2.5-Pro run for the most complex tasks costs only a few cents, supporting low-cost access without expensive local GPUs.The analysis used rates of 1.25 / 1M input tokens and 10 / 1M output tokens without caching.

C.4.1 Sensitivity analysis on Cell type annotation marker gene selection

The experiments examine sensitivity to marker-gene selection and dependence on intermediate bioinformatics outputs. Accuracy peaks at K = 10 but remains strong across tested K values, while corrupted tool inputs degrade performance.

  • Sensitivity analysis on Cell type annotation marker gene selection: K = 10: annotation performance peaked at the chosen number of top marker genes.The tested values were K = 5, 10, and 20 on PBMC3k.
  • Sensitivity analysis on Cell type annotation marker gene selection: Strong results across K = 5, 10, and 20 indicate that the approach is not highly sensitive to this hyperparameter.The robustness claim is based on the PBMC3k sensitivity analysis.
  • Additional perturbation experiments: Corrupting GO term associations significantly degraded GRN prediction AUROC, including p = 0.044 for GPT-4o-mini.The perturbation supports dependence on accurate Gene Ontology information in the intermediate reasoning step.
  • Additional perturbation experiments: Randomizing cluster relationships and pseudotime order in the Monocle report caused performance drops across trajectory metrics.The result indicates that accurate outputs from tools such as Monocle are critical for trajectory inference.
  • Additional perturbation experiments: SCPILOT’s success fundamentally depends on the integrity of data supplied by its integrated bioinformatics tools.This conclusion summarizes the GO and Monocle perturbation experiments.

C.4.3 Additional Experiments in Error Propagation and Uncertainty Assessment

Additional experiments test error propagation, uncertainty, and comparison with Biomni. They show that contextual and tool-output integrity matter, o1 is more reliable on GRN prediction, and SCPILOT’s specialized reasoning-first design outperforms the general-purpose agent in the reported tasks.

  • Error Propagation: Removing contextual metadata from PBMC3k annotation inputs caused significant performance drops, confirming that rich metadata is critical for reliability.The removed context included information such as tissue type.
  • Uncertainty and Reliability Assessment: o1 was more accurate, stable, and reliable on stomach GRN prediction, with tighter confidence intervals and lower ECE and Brier scores.The assessment used 95% confidence intervals from 10 trials and 1,000-bootstrap resampling.
  • SCPILOT vs. Biomni: SCPILOT uses a specialized reasoning-first design, whereas Biomni uses a general-purpose tool-first design.The contrast is presented as the core design difference between the systems.
  • SCPILOT vs. Biomni: SCPILOT significantly outperformed Biomni across all three tasks.The reported comparisons cover cell annotation, trajectory inference, and GRN prediction.
  • Qualitative comparison: SCPILOT correctly resolved fine-grained retinal subtypes and identified the Epiblast root, while Biomni made subtype and lineage errors.Examples include Biomni confusing Müller glia with amacrine cells and misplacing cardiac muscle in the liver lineage.
  • Efficiency and Cost Comparison: Up to 30× cheaper: SCPILOT was also reported as faster and more cost-effective than Biomni, and it succeeded on GRN prediction where Biomni failed.The efficiency advantage was attributed to its curated toolchain and reasoning-first method avoiding costly broad tool searches.
  • Retina annotation: GPT-4o achieved 0.789 on retina annotation, correctly annotating 14 clusters, with 2 partially matched and 3 incorrect.The analysis describes marker-based recognition of rods, low-abundance cones, and horizontal cells.
  • Retina and PBMC reasoning comparison: GPT-4o-mini proposed too many T-cell subtypes and repeatedly mislabeled clusters because it failed to connect multiple expressed genes.The cited examples include NKG7/GZMB/FCGR3A+ and NKG7, CXCR4, and CD3E expression patterns.

D.2 Trajectory Inference

SCPILOT combines iterative biological reasoning with trajectory tools to reconstruct developmental relationships and inspect where predictions succeed or fail. In the liver example, it recovered the broad developmental outline but struggled to order closely related mesodermal states.

  • Prediction strengths: The liver trajectory preserved the global organogenesis skeleton, including early tissue splits, mesodermal branching, and hepatoblast maturation.
  • Prediction weaknesses: Three missed and five extra edges clustered around the mesodermal hub, where cell types were recognized but their emergence was mis-ordered.The associated analysis reports Jaccard = 1 for recognizing the cell types despite ordering errors.
  • Prediction weaknesses: Early specification and late hepatic maturation were accurate, whereas mesodermal diversification produced an over-connected mesh of descendants.
  • Interpretation: The analysis attributes the main limitation to insufficient temporal cues for distinguishing closely related mesoderm-derived cell types.It also reports that functional relevance, expression context, and known regulatory pathways supported biologically informed inference when sufficient cues were available.
  • Method: Its transparent agent loops log intermediate reasoning, and its plug-and-play design integrates tools such as Monocle without code changes.
  • Method: SCPILOT’s trajectory workflow validates annotations and lineage structure with biological context and analytical reports, while requiring consistency between trajectory and annotation dictionaries.
Loading 2602.11609v1…