PertMind: Eliciting Emergent Biological Reasoning in LLM via Reinforcement Learning on Cellular Perturbation Data

Zhenchao Tang; Xiaogang Xu; Tianxu Lv; Jiahui Guan; Jiale Zhou; Haohuai He; Zhi Song; Hanbo Huang; Jiehui Huang; Jiafei Wu; Zhe Liu

Source-linked AI summary

PertMind: Eliciting Emergent Biological Reasoning in LLM via Reinforcement Learning on Cellular Perturbation Data

Zhenchao Tang, Xiaogang Xu, Tianxu Lv, Jiahui Guan, Jiale Zhou, Haohuai He, Zhi Song, Hanbo Huang, Jiehui Huang, Jiafei Wu, Zhe Liu

arXiv:2608.16419v1Aug 17, 2026cs.LGcs.AIq-bio.QM

TL;DR

Biological reasoning models need scalable supervision beyond costly expert-authored trajectories. PertMind uses experimentally measured perturbation responses as reinforcement signals and transfers from forward prediction to diverse biological tasks and representations.

Problem
Expert-authored biological reasoning trajectories are costly and difficult to scale across the drug–gene–context combinations in perturbation atlases.
Method
PertMind trains a language-model policy with trusted-trajectory initialization and gene-, pathway-, and format-level rewards derived from experimentally measured perturbation responses.
Results
PertMind strengthened unseen-context response prediction and transferred without task-specific adaptation across reverse inference, double perturbations, screening, process interpretation, and multiscale representations.
Takeaways & Limitations
Experimental endpoints can select reusable biological strategies from pretrained models across biological operations requiring rankings, process names, mechanistic briefs, and continuous representations.
Takeaways & Limitations
The rewards do not causally verify free-text reasoning or directly measure protein activity, metabolic flux, spatial signaling, or causal pathway activation.

Abstract

from arXiv · show

Large language models can describe mechanisms, yet scalable post-training still depends on costly, manually curated biological reasoning traces. Here we show that cellular perturbation atlases can instead become reinforcement-learning environments, where measured gene responses provide computable rewards for biological reasoning. We introduce PertMind, which combines trusted-trajectory supervised initialization with gene-, pathway-, and format-level reinforcement signals. Trained only on forward perturbation-response prediction, PertMind improved response inference in unseen cellular contexts while retaining general language capabilities. It also transferred without task-specific post-training to reverse perturbation identification, double-perturbation reasoning, phenotypic-screen prioritization, and biological-process interpretation. PertMind further generated biological profiles that supported competitive gene, cell, and donor representations across multiscale downstream tasks. These results support the hypothesis that reinforcement on experimental endpoints can concentrate reusable biological strategies already accessible to pretrained models. More broadly, perturbation-derived reinforcement learning offers a scalable route for transforming expanding experimental atlases into training environments for general-purpose biological reasoning.

1 INTRODUCTION

PertMind reframes cellular perturbation atlases as scalable reinforcement-learning environments, using measured gene responses to train biological reasoning in LLMs. The resulting model transfers beyond forward response prediction to reverse inference, double-perturbation reasoning, and phenotypic-screen prioritization.

Motivation: PertMind addresses the supervision bottleneck by turning measured interventions from public cellular perturbation atlases into reinforcement-learning experience for LLMs.The framework uses experimental endpoints as computable rewards rather than relying only on manually curated biological reasoning traces.
Method: The method combines trusted model-generated trajectories, atlas-derived differential-expression labels, and reinforcement learning to establish structured, evidence-grounded biological reasoning.It starts from Qwen3-4B Base and builds gene-centered queries from the roughly 100-million-profile Tahoe-100M perturbation atlas.
Results: PertMind transfers beyond its forward target-gene objective, approaching a task-specialized model for reverse perturbation-condition inference and extending to double-perturbation queries without double-perturbation training.These cross-task results test whether reinforcement learning acquired reusable biological strategies rather than only optimizing the training objective.
Results: PertMind-generated biology briefs improve phenotypic-screen hit prioritization on AssayBench, while the overall cross-task evidence supports reusable biological reasoning strategies learned from perturbation endpoints.The reported scope also includes reasoning-based perturbation applications and reusable molecular, cellular, and donor representations.

2 METHODS

PertMind frames cellular perturbation-response prediction as auditable, knowledge-grounded reasoning over cell line–drug–gene queries, using experimentally derived gene-response labels and pathway supervision. Its training combines trusted-trajectory initialization with gene-, pathway-, and format-level reinforcement signals while preserving gene-label correctness as the dominant objective.

Task formulation: PertMind predicts an Up, Down, or No response for a target gene from a cell line–perturbation–gene triplet and produces an auditable mechanistic rationale grounded in retrieved evidence.The ternary label is a scalable training interface for the sign of a multi-step regulatory response, while the natural-language trajectory explains the final gene-level conclusion.
Data and labels: Labels come from Tahoe-100M pseudobulk differential-expression statistics, with ambiguous records excluded and cell lines partitioned disjointly so five held-out lines define evaluation.Up and Down require padj ≤ 0.05 with log2 fold-change thresholds of ±0.585, whereas No requires weak effect, high padj, and sufficient expression; No is not an exact-null claim.
Initialization: PertMind first performs one epoch of trusted-trajectory SFT, using retrieved biological context and balanced weighting to initialize structured, mechanism-oriented reasoning before GRPO.Trajectories must have the correct label, required format, no hidden-label claims, and supported entities; the resulting policy initializes GRPO and serves as its fixed KL reference.
Reinforcement rewards: The core reinforcement method combines a gene-level outcome reward with a pathway-level checkpoint reward derived from outcome-independent transcriptional proxies.Candidate Reactome pathways are selected without examining the specific expression outcome, and pathway evidence is aggregated conservatively across matched dose-and-plate conditions.
Reward constraints: Auxiliary pathway and format rewards provide partial credit without allowing any response with an incorrect gene label to outrank a correct one.Positive auxiliary weights satisfy λpw + λfmt < 1, while zero-variance groups contribute no gradient and weakly supported triplets receive uniformly scaled updates.

3 DATASETS AND EXPERIMENTAL SETUP

PertMind is trained and evaluated under cell-line-held-out protocols using Tahoe-100M forward perturbation supervision, with strict separation of training records from validation and held-out contexts. The experimental suite covers forward response prediction, reverse intervention identification, cross-task biological reasoning, and multiscale representation quality.

Training setup: PertMind is initialized from Qwen3-4B Base, then trained with validated trusted trajectories through BFT-weighted SFT followed by GRPO.Trusted-trajectory construction samples M = 8 candidates per query; SFT runs for one epoch and GRPO for three epochs, using LoRA rank 64 in both stages.
Ablations: The ablation protocol compares Base, BFT-weighted SFT, shuffled-pathway control, gene-only GRPO, prompt-only pathway context, and full PertMind under the same held-out-cell protocol.The shuffled control preserves pathway frequency and prompt structure while permuting pathway labels within the training partition.
Forward perturbation response: Forward evaluation uses VCWorld/GeneTAK on held-out Tahoe-100M triplets for differential-expression detection and regulation-direction prediction, reporting Accuracy and Macro-F1 against multiple baselines.Differential-expression detection predicts whether a target gene changes, while direction prediction distinguishes up- from downregulation among changed genes.
Generalization and representation evaluation: Additional evaluations test reverse perturbation ranking, phenotypic-screen gene prioritization, and gene-, cell-, and donor-level representations using task-specific accuracy, F1, ranking, AUC, decoding, and error metrics.Reverse prediction ranks 69 candidate interventions on the Schmidt et al. (2022) human T-cell dataset; AssayBench uses its official year-fold0 test split of 334 screens and AnDCG@100; representation tests span molecular, cellular, and donor scales.

4 RESULTS

PertMind matched or modestly exceeded a Gemini-based reference on unseen-cell-line perturbation-response benchmarks while largely retaining general language ability. Without task-specific post-training, it transferred to reverse and combinatorial perturbation reasoning, phenotype-screen prioritization, biological-process interpretation, and competitive molecular and cellular representations.

Forward perturbation-response prediction: PertMind matched or modestly exceeded the Gemini-based VCWorld reference across unseen-cell-line DE and DIR tasks and evaluation metrics, outperforming it on most held-out conditions.This evaluation used the open-source VCWorld perturbation-response benchmark and compared PertMind with the VCWorld harness configured with Gemini-2.5-Flash.
Training ablations: Pathway-supervised reinforcement learning produced the strongest variant gains, beyond SFT and gene-only reinforcement, whereas pathway context without its corresponding reward did not improve performance.The ablation compared six training variants across task–metric combinations and held-out cell lines.
General capability retention: PertMind remained close to Base and SFT models on zero-shot and few-shot MMLU and CMMLU, with small consistent accuracy reductions and no observed catastrophic forgetting.The reported retention was associated with design choices including BFT and constraints limiting policy drift.
Transfer to reverse and combinatorial reasoning: Despite forward-only post-training, PertMind approached specialized CellNavi across reverse-task Top-1, Top-5, and F1 performance, substantially exceeded Base and SFT, and was more robust across perturbation shifts.CellNavi had stronger average performance but its accuracy collapsed for some targets, whereas PertMind maintained a flatter profile across ordered perturbations and expression-shift directions.
Phenotypic-screen prioritization: PertMind briefs improved phenotype-screen ranking across all five phenotype classes and three ranking backbones, while matched briefs achieved the strongest AnDCG@100, most top-100 hits, and lowest hallucination rate.The all-screen evaluation used 334 test screens; the stratified analysis covered 36 screens.
Biological interpretation and representations: PertMind improved lexical and semantic biological-process agreement, supplied transferable context to frontier models without parameter updates, and produced competitive molecular and cellular representations.Its embeddings were competitive with strong references on Geneformer molecular tasks and approached scProtoTransformer for STATE-based perturbation prediction while outperforming standard STATE-SM overall.

5 DISCUSSION

PertMind shows that experimentally measured perturbation endpoints can train a reusable biological language-model policy that generalizes beyond forward-response prediction while largely retaining pretrained capabilities. The results support selection-and-concentration of biologically aligned strategies, while limitations constrain claims about mechanistic faithfulness.

Core finding: PertMind strengthened response prediction in unseen cellular contexts, retained general pretrained capabilities, and transferred across task direction, perturbation composition, functional interpretation, and representation scale.These findings support experimentally measured perturbation endpoints as training signals for a reusable biological language-model policy rather than only a forward-response classifier.
Learning interpretation: The gene-level reward selected trajectories reaching measured responses, while pathway rewards provided intermediate guidance favoring coordinated transcriptional consistency before final gene decisions.A shuffled-label control indicated that improvement depended on biological alignment rather than merely denser rewards or output structure.
Learning interpretation: Correct conclusions already appeared in the Base model’s sampling support, but reinforcement learning concentrated them into reliable decisions by reallocating probability toward useful accessible strategies.Pre-RL best@N provided the complementary evidence that the model already sampled correct conclusions, whereas greedy decoding and majority voting did not reliably concentrate them.
Cross-task transfer: PertMind transferred without reverse-task adaptation to reverse perturbation-condition prediction and unseen double-perturbation reasoning, approaching a specialized transition model on primary-cell perturbations.This transfer beyond the post-training objective is presented as evidence that selected strategies are reusable.
Representations: PertMind-generated gene profiles externalized distributed model knowledge as natural-language representations that supported molecular annotation and extended the GenePT premise.These profiles could be encoded once and adapted for distinct biological tasks.
Limitations: Terminal rewards may permit endpoint-prediction shortcuts, and free-text trajectories are auditable but not causally verified; trusted-trajectory filters cannot establish faithful reporting of underlying computations.These limitations constrain interpretation of generated explanations as mechanistic accounts.

A ADDITIONAL METHODS DETAILS

This appendix provides reproducibility details deferred from Section 2 while retaining the main text’s notation, symbols, and labels.

The appendix collects reproducibility details deferred from Section 2.
Notation in the appendix follows the main text.
The appendix also preserves the main text’s symbols and labels.

A.1 DATA CONSTRUCTION DETAILS

The dataset construction pipeline defines condition-level differential-expression records, filters unreliable comparisons, and collapses eligible measurements by drug for gene-centered labels. It separately preserves condition-level records for pathway rewards, applies confidence weighting, and partitions data by cell line while preventing leakage.

Full experimental condition and drug-level collapsing: Condition-level records retain DESeq2-style statistics, experimental identifiers, and treated/control cell counts after discarding missing-effect or missing-adjusted-significance records.The retained statistics include baseMean, log2FoldChange, lfcSE, stat, pvalue, and padj.
Full experimental condition and drug-level collapsing: For gene-centered labels, surviving conditions sharing a drug are collapsed by retaining the record with the largest absolute log2 fold change, while reward-side tables preserve individual conditions for pathway calculations.This separates primary label assignment and confidence weighting from condition-specific pathway-response proxy computation.
Confidence weighting: Confidence weights range from 0.1 to 1 and increase effective supervision for strong significant directional calls and high-expression near-zero-effect null calls without changing labels.The weights are computed from effect size, adjusted significance, and expression statistics.
Split, class stratification, and leakage: Samples are split by cell line into held-out test and development domains, with development groups further divided 90:10 into stratified training and validation partitions.Eligibility and label rules are applied within both domains before the development split.
Split, class stratification, and leakage: Retrieval excludes repeated measurements, validation outcomes, and held-out-cell outcomes, while batches and prompt streams are stratified by cell line, drug, and label without capping or downsampling.Each candidate example is an eligible cell-line–perturbation–gene record derived from aggregated pseudobulk statistics.

A.2 TRAJECTORY AND SFT DETAILS

The trajectory pipeline retrieves outcome-independent biological context, samples eight reasoning trajectories from the base Qwen3-4B policy, and trains on trusted responses with confidence-balanced weighting. Query expression measurements and labels remain hidden from retrieval prompts, while low-confidence spans receive stronger training emphasis without allowing weight manipulation.

Retrieval configuration: Retrieval aligns entities across biomedical resources, removes frequent low-information nodes and duplicate facts, and computes drug, gene, and cell-context similarity using outcome-independent information restricted to Ctrain.Cell-context similarity uses tissue, lineage, and curated genotype annotations; drug and gene similarity combines semantic annotation with knowledge-graph topology.
Retrieval configuration: The prompt excludes current-query expression measurements and labels, so inference uses only knowledge-graph evidence and outcome-stratified training cases with historical outcomes.Figure 7 summarizes the model-visible fields, required response order, and reward quantities withheld from the prompt.
Balanced Fine-Tuning objective: Balanced Fine-Tuning uses local confidence and sequence-level difficulty weights, down-weighting isolated low-confidence tokens while emphasizing trajectories containing difficult local spans.Stop-gradient prevents the model from manipulating confidence-derived weights instead of increasing response-token likelihood.

A.3 PATHWAY SUPERVISION DETAILS

Pathway supervision uses outcome-independent Reactome candidate selection and condition-level transcriptional response proxies that exclude the target gene. Conservative consensus labeling, uncertainty handling, and masking prevent leakage and avoid fusing incompatible experimental conditions.

Candidate pathway selection: Reactome candidates are restricted to 10–200 members, exclude the target gene from scoring, and are ranked using pre-outcome gene-membership and drug-relevance information.Drug relevance prioritizes direct targets, high-confidence one- or two-hop network links, and curated mechanism-of-action axes, with αgene = 0.60 and αdrug = 0.40; unavailable annotations remain unknown.
Condition-level pathway-response proxy: Pathway-response proxies summarize coordinated non-target transcriptional effects within each single dose-and-plate condition, rather than directly measuring protein activity, metabolic flux, or causal activation.Conditions with fewer than five reliable members or gray-zone effects are labeled uncertain; thresholds use γ = 0.60, τdir = 0.20, τno = 0.05, and ηno = 0.20.
Cross-condition consensus for drug-level pathway labels: Drug-level pathway labels are assigned only when all determinate conditions agree on Up, Down, or No; disagreements and the absence of determinate conditions produce uncertain.Uncertain condition labels are ignored, and a single determinate condition can supply the drug-level label without combining measurements across doses or plates.
Model-visible information and pathway masking: The prompt exposes the complete retained candidate set with fixed, outcome-independent representative members, while uncertain candidates are masked from pathway rewards without removing their triplets from main-task training.This preserves the training corpus and lets pathway augmentation increase supervision density without expression leakage.

A.4 FULL OPTIMIZATION DEFINITION

PertMind’s full optimization uses grouped, confidence-weighted GRPO with clipped policy updates and KL regularization toward the frozen SFT reference. Standardization preserves within-group rankings, while monitoring tracks correctness, formatting, class balance, length, and policy divergence to detect reward hacking.

Group standardization and confidence scaling: Zero-variance groups receive vanishing gradient contributions, while auxiliary rewards preserve preference signals when the primary reward is uniform.Confidence is applied after group standardization as a positive per-triplet multiplier on the complete sample objective, preserving within-group rankings.
Clipped, KL-regularized policy optimization: The objective distinguishes the refreshed behavior policy used for clipped importance ratios from the frozen SFT reference used for KL regularization.The confidence weight scales the complete per-response objective, including both the clipped surrogate and KL penalty, exactly once.
Clipped, KL-regularized policy optimization: Token-level KL regularization is nonnegative, vanishes when current and SFT-reference probabilities match, and uses only sampled tokens rather than enumerating the vocabulary.The clipping term limits reward-driven update sizes, while KL toward πθSFT preserves linguistic competence and reasoning structure established during SFT.
Training monitors: Training monitors gene accuracy, per-class recall, pathway accuracy, format pass rate, response length, and policy KL to identify reward hacking and guide early stopping or hyperparameter adjustment.The monitored failure modes include format-only optimization, majority-class overprediction, and indirect advantage from response length instead of correctness.

A.5 CROSS-SCALE EMBEDDING CONSTRUCTION

PertMind constructs hierarchical biological representations by generating gene profiles, adapting their text embeddings for downstream tasks, and aggregating them into expression-weighted cell and attention-based donor embeddings. Task-specific adapters are reinitialized within each training fold and optimized only on reference-training data.

Gene representations: PertMind generates a profile for each gene describing its function, interactions, pathways, and context-dependent roles, then uses text embeddings as numerical representations.The approach follows GenePT by treating generated text as an interface between biological knowledge and numerical representation.
Gene representations: Task-specific learned projections adapt the common gene representation to downstream decoders without assuming a shared fixed-dimensional latent space.Each adapter is reinitialized within every training fold and jointly optimized with the downstream classifier or decoder using only the reference-training partition.
Cell representations: Cell embeddings combine represented-gene profiles using nonnegative preprocessed expression weights and row-wise unit ℓ2 normalization.A positive ϵ prevents division by zero during the expression-weighted aggregation.
Donor representations: Donor embeddings aggregate their constituent cells as multi-instance bags with an attention module, producing representations for donor-level classification.The construction forms a gene-to-cell-to-donor hierarchy.

B ADDITIONAL RESULTS

Additional analyses show that the Base model samples correct forward-perturbation answers but does not reliably select them, with biological decision errors—not parsing—remaining the main bottleneck. PertMind also preserves biotype-enriched gene-embedding structure, while supplementary evaluations examine subject-level capabilities and a literature-audited case.

Pre-RL sampling analysis: Oracle best@N rises sharply before reinforcement learning, then gains diminish but remain nonzero, indicating latent support for correct answers without deployable inference-time selection.The oracle uses gold labels and is an evaluation-only diagnostic; it cannot be implemented by the model at inference time.
Pre-RL sampling analysis: Parse rates remain near saturation, so most forward-perturbation failures reflect biological decision errors rather than missing terminal labels.The mean parse rate stays close to 1 and the any-parse rate is essentially saturated across rollouts.
Pre-RL sampling analysis: Class-conditional sampling is uneven: Up is easiest, Down improves more gradually, and No remains hardest across rollout counts.Correctly identifying No is especially challenging for differential-expression detection, while the Up–Down gap indicates that direction prediction requires genuine discrimination.
Gene-embedding visualization: PertMind’s genome-wide UMAP retains distinct biotype-enriched gene regions, providing qualitative molecular organization complementary to reference-mapping results.Because UMAP geometry depends on preprocessing and visualization hyperparameters, it is not a quantitative performance metric.
Additional evaluations: Supplementary evaluations include per-subject MMLU and CMMLU accuracy for Base, SFT, and PertMind under 0-shot and 5-shot settings, plus a literature-audited Vemurafenib–C32–MKI67 case.The literature support for highlighted case claims does not establish that the generated reasoning process was faithful.