PaperlayerView exact revision on arXiv

Source-linked AI summary

MorphoCLIP: Text-Supervised Contrastive Learning for Perturbation Matching in Cell Painting Images

Sukhrobbek Ilyosbekov, Shubham Gajjar, Rongfei Jin

arXiv:2608.22690v1cs.CV

TL;DR

Cell Painting perturbation matching is difficult because biological effects are subtle and technical variation can obscure them. MorphoCLIP uses frozen vision and language backbones with a compact trainable cross-channel module, and retrieves matching images and descriptions above chance, while cross-modality biological matching remains unresolved.

  • Problem

    Cell Painting images must be connected to their chemical or genetic perturbations despite subtle biological effects and substantial variation across plates, imaging days, and perturbation types.

  • Method

    MorphoCLIP learns a shared image–text space for compounds, CRISPR knockouts, and ORF overexpressions using frozen backbones, cached features, projection layers, and a cross-channel transformer.

  • Results

    On held-out CPJUMP1 data, MorphoCLIP retrieves correct perturbation descriptions and cell images above chance, while replicate alignment increases pooled mAP from 0.298 to 0.343 and the combined model reaches 0.369.

  • Takeaways & Limitations

    Text supervision can organize chemical and genetic Cell Painting data, but improved replicate consistency does not establish reliable gene–compound matching.

  • Takeaways & Limitations

    Small variant differences remain uncertain, and mismatched primary-target training annotations and broader benchmark target lists may weaken cross-modality retrieval.

Abstract

from arXiv · show

Cell Painting microscopy captures how cells change after a chemical or genetic perturbation. Connecting these images to the perturbations that produced them could make large imaging screens easier to search and interpret, but the task remains difficult because biological effects are subtle and technical variation is substantial. We introduce MorphoCLIP, a contrastive model that links Cell Painting profiles with text descriptions of compounds, CRISPR knockouts, and ORF overexpressions. The model keeps its vision and language backbones frozen and trains only a compact cross-channel module and projection layers, so it can be trained on a single consumer GPU. On held-out CPJUMP1 data, MorphoCLIP searches in both directions: from a cell image to its perturbation description and from a description to matching cell images. In both cases, a correct match appears among the top ten results much more often than expected by chance. Adding a replicate-alignment loss makes profiles from repeated experiments more consistent, although this improvement does not yet translate into reliable gene-compound matching. Gene-aware labels and plate correction also show no consistent retrieval benefit. These findings suggest that text supervision can help organize chemical and genetic Cell Painting data. Matching compounds with genetic perturbations, however, remains an open problem.

I. INTRODUCTION

MorphoCLIP addresses perturbation matching in Cell Painting by aligning well-level image representations with text descriptions across chemical and genetic perturbations. It freezes the foundation backbones and evaluates bidirectional retrieval while testing replicate alignment, label softening, and plate correction.

  • Cell Painting enables broad morphological profiling of chemical and genetic perturbations, but matching images to their causes remains difficult because biological effects are subtle and technical variation is substantial.
  • MorphoCLIP places a Cell Painting well near text describing its perturbation in a shared normalized embedding space.The model supports text-to-image, image-to-text, and image-to-image retrieval.
  • Frozen vision and language backbones with cached outputs leave only the cross-channel transformer and projection layers to train, reducing the trainable model size.This design targets training on a single consumer GPU despite CPJUMP1’s multi-terabyte image scale.
  • The model jointly represents compounds, CRISPR knockouts, and ORF overexpressions while evaluating replicate alignment, gene-aware labels, and plate correction.The contribution includes a retrieval protocol with analytic chance baselines on CPJUMP1.
  • Image-to-image retrieval remains especially challenging because batch effects, imperfect biological labels, and cross-modal relationships complicate similarity judgments.The same perturbation may vary across plates and imaging days, while shared targets do not guarantee identical phenotypes or mechanisms.

III. RELATED WORK

Prior Cell Painting methods span handcrafted profiling, compound-focused representation learning, and text-supervised retrieval. MorphoCLIP builds on this progression while targeting genetic perturbations and multi-channel microscopy.

  • CellProfiler extracts more than a thousand handcrafted cell properties and remains a strong CPJUMP1 baseline, although compound–genetic matching is still difficult.
  • Self-supervised vision transformers motivate frozen foundation-model features because they transfer to biological imaging and have shown stronger transfer on drug-target and gene-family classification.
  • CLOOME applies CLIP-style contrastive learning to Cell Painting with molecular fingerprints but covers compounds only and treats fluorescence channels as RGB.
  • MolPhenix improves compound retrieval using a frozen Phenom-1 encoder, replicate averaging, and softer labels, but depends on a proprietary backbone and excludes genetic perturbations.
  • CellCLIP combines templated text prompts, per-channel encoding, cross-channel attention, cell pooling, and continuously weighted contrastive loss for chemical and genetic perturbations.Its larger DINOv2-g backbone and cross-class results leave technical variation unresolved.

C. Batch-Effect Correction

MorphoCLIP constructs well-level image representations from frozen per-channel DINOv3 features and trainable cross-channel and projection modules. The architecture can incorporate plate correction before contrastive training.

  • A. Image Encoder: Each five-channel Cell Painting site is encoded separately by a frozen DINOv3 ViT-L/16 into one 1024-dimensional CLS token per channel.The channel features are cached and reused, so DINOv3 does not run during model training.
  • C. Batch-Effect Correction: Optional plate correction is applied to the image embedding before the training losses, while the broader model aligns image and text representations and can encourage replicate agreement.
  • A. Image Encoder: The CrossChannelFormer combines normalized channel tokens with learned channel embeddings and an aggregation token to form one site representation.Its cross-channel attention is designed to capture changes appearing across multiple stains.
  • A. Image Encoder: Valid site vectors are averaged under a mask to produce a fixed-size well representation regardless of the number or order of imaged sites.
  • A. Image Encoder: A two-layer image projection head maps the well vector into the shared embedding space using LayerNorm, GELU, and dropout.

B. Text Encoder

MorphoCLIP converts structured perturbation descriptions into shared-space text embeddings using frozen BioClinical ModernBERT and a trainable projection head. Prompts are keyed by perturbation identity so replicates share one text vector.

  • Each perturbation is represented by a template built from CPJUMP1 metadata and external annotations.
  • Compound prompts include cell line, compound name, SMILES, target gene, protein function, and chemical modality.
  • CRISPR and ORF prompts describe gene identity, gene annotations, and whether the perturbation is a knockout or overexpression.
  • A frozen 150M-parameter BioClinical ModernBERT encodes each prompt, and a trainable projection maps its 768-dimensional CLS token into the shared space.
  • The 768-dimensional BERT outputs and projected 512-dimensional vectors are cached separately, so changing the projection head does not rerun BERT.

C. Training Objective

MorphoCLIP combines a soft-label image–text contrastive objective with optional replicate alignment and condition-relative plate correction. These additions target shared perturbation structure, replicate consistency, and technical drift.

  • Contrastive objective: Soft-label contrastive learning weights identical perturbations as positives, overlapping target genes by α, and unrelated perturbations as zero-weight pairs.The base model uses α = 0, while the soft-label ablation uses α = 0.6.
  • Replicate alignment: The replicate alignment loss adds an image–image contrastive term that brings wells sharing a perturbation closer together.Wells without an in-batch replicate are excluded from the replicate-loss mean.
  • Replicate alignment: Training combines the text loss and replicate loss as L = LCWCL + λrepLrep, with λrep = 0 for the base model and 0.3 for the ablation.Validation loss uses only the text term, so the added loss does not affect model selection.
  • Batch construction: Perturbation-aware batches group replicate wells into pairs before packing batches, while spanning multiple plates where possible.This supports replicate alignment but does not eliminate plate confounding from individual negative pairs.
  • Plate correction: Condition-relative plate correction estimates plate means and shifts wells by offsets relative to other plates sharing the same experimental condition.The offsets sum to zero within each condition, removing plate-to-plate drift without shifting the condition mean; corrected embeddings are stored for evaluation.

D. Optimization

The model is optimized with AdamW under a scheduled, mixed-precision training procedure. The base model uses a longer schedule than the ablation runs.

  • Optimization: AdamW training uses learning rate 10^-4, weight decay 0.2, and a batch of 256 wells.Weight decay excludes biases, normalization parameters, and the learned temperature.
  • Optimization: The learning rate warms up for 100 steps, follows a cosine schedule to zero, and training uses FP16 mixed precision with gradient clipping at norm 1.0.The base model runs for 100 epochs, whereas ablations use 30 epochs with early stopping.

V. EXPERIMENTAL SETUP

Experiments use the CPJUMP1 pilot and evaluate retrieval at well and perturbation levels in both image-to-text and text-to-image directions. Splitting is performed by perturbation to keep replicates together.

  • Feature processing: Cached DINOv3 features and precomputed ModernBERT prompt features are reused across training runs after plate-wise image processing.The raw local image copy is removed after feature extraction because the raw images occupy several terabytes.
  • Data split: An 80/10/10 perturbation-level hash split prevents replicate wells from appearing in different train, validation, and test sets.The validation set contains 2,220 wells from 98 perturbations, and the test set contains 1,860 wells from 86 perturbations.
  • Retrieval protocol: Retrieval uses P distinct text vectors and N well vectors because replicate wells share a perturbation description.Analytic chance baselines are computed separately for the retrieval settings.
  • Retrieval protocol: Well-level image-to-text retrieval ranks P perturbation descriptions for each well, with random Recall@k equal to k/P.Well-level text-to-image retrieval ranks N wells for each of the P descriptions and scores the first correct replicate.
  • Retrieval protocol: Perturbation-level retrieval averages and re-normalizes wells into one profile before ranking profiles against texts in both directions.The protocol reports Recall@{1, 5, 10} and the median rank of the first correct match.

E. Standard CPJUMP1 Benchmark

The standard CPJUMP1 benchmark evaluates exported well embeddings with a reference preprocessing pipeline and compares MorphoCLIP with established baselines. Small stochastic differences in fraction retrieved are treated descriptively.

  • Benchmark protocol: The CPJUMP1 benchmark encodes every well in fp32, removes the negative-control mean, and evaluates on 40 qualifying plates.It covers short and long timepoints for each perturbation type and measures replicability and target matching.
  • Benchmark interpretation: Fraction-retrieved differences from the benchmark’s random-permutation significance test are descriptive rather than conclusive, while replicability mAP is deterministic.This limits the interpretation of small benchmark changes across otherwise identical runs.
  • Baselines: CellProfiler provides a published handcrafted-feature baseline, but its CPJUMP1 target-matching range was not rerun in this study.The comparison therefore relies on the published range.
  • Baselines: CellCLIP is the closest text-supervised baseline, but comparison is approximate because it applies an additional KernelPCA correction and uses a stochastic significance test.The released checkpoint is also run through the short-timepoint benchmark pipeline.
  • Baselines: CWA-MSN is included as a related batch-aware method, but its published benchmarks do not support direct numerical comparison.
  • Ablation design: Ablations add gene-aware soft labels, replicate alignment, and condition-relative plate offsets individually and jointly against a shared base model.All runs use seed 42, batch size 256, and the same data split; recall changes of about one percentage point are considered too small for reliable interpretation.
  • Results table: Table II compares validation and test retrieval for image-to-text and text-to-image settings at well and perturbation levels against analytic random baselines.Text-to-image retrieval includes both individual wells and perturbation-level mean profiles.

H. Implementation

MorphoCLIP is evaluated on held-out CPJUMP1 retrieval tasks, where perturbation-level matching exceeds chance and replicate alignment yields the clearest improvement. Other tested changes do not produce consistent retrieval gains, and gene–compound matching remains unresolved.

  • Cross-modal retrieval: Roughly two out of five perturbation-level queries retrieve the correct match in the top ten in either direction, more than three times chance.Similar performance on unseen test perturbations suggests the model learns morphology–text relationships rather than memorizing training data.
  • Cross-modal retrieval: Well-level text-to-image retrieval is harder because each description searches individual wells, while replicate averaging produces more balanced bidirectional results.The first matching replicate usually appears near the top of the ranking.
  • Ablations: No single-factor addition improves retrieval consistently across validation and test data, and the combined model is competitive but not uniformly best.Observed differences are too small to establish a reliable retrieval gain.
  • Standard CPJUMP1 benchmark: Replicate alignment increases pooled mAP from 0.298 to 0.343, improving replicability across all twelve tracks.The combined model reaches 0.369, while plate offsets alone do not improve the pooled result.
  • Standard CPJUMP1 benchmark: Target-matching results do not show that replicate alignment or plate correction improves relationships beyond perturbation identity.No variant is consistently better on the larger compound tracks.
  • Standard CPJUMP1 benchmark: Cross-modality gene–compound retrieval is weak for every variant and does not solve matching compounds to genetic perturbations with the same target.Almost no gene–compound relationships pass the significance threshold, and pooled mAP shows no consistent ordering.

D. Comparison with Baselines

MorphoCLIP provides a compact text-supervised representation for chemical and genetic perturbations, with retrieval performance above chance and replicate alignment improving consistency. Comparisons with baselines remain contextual because evaluation procedures differ, while gene–compound matching remains unresolved.

  • Comparison with CellCLIP: MorphoCLIP and CellCLIP are close on average, while replicate-loss and combined variants are higher on most shared tracks.The comparison is approximate because CellCLIP uses its own encoder path and control-fitted KernelPCA preprocessing.
  • Published benchmark context: On CPJUMP1 compound target matching, MorphoCLIP spans a similar range to CellProfiler and has a four-track mean of 16.3%.Published results are comparable only within matching dataset–task groups.
  • Overall comparison: MorphoCLIP retrieves correct images or perturbation descriptions above chance on held-out perturbations.The model supports both image-to-text and text-to-image retrieval in a shared representation.
  • Ablation results: Replicate alignment consistently improves agreement between repeated wells across benchmark tracks.Its immediate benefit is stronger replicate structure, not necessarily better biological understanding.
  • Ablation results: Gene-aware labels and plate correction do not show a consistent retrieval benefit in the reported experiments.The tested changes do not resolve the broader cross-modality limitation.
  • Cross-modality matching: None of the tested variants reliably connects genetic perturbations with compounds acting on the same targets.The model succeeds for perturbation identity and replicate structure but not yet as a general model of shared biological mechanism.

A. Limitations and Future Work

The paper identifies uncertainty in variant differences and cross-model comparisons, and points to gene–compound matching and biological usefulness as key unresolved issues. Future work should align target definitions, test richer supervision and evaluate transfer across broader experimental settings.

  • Limitations: Small variant differences remain uncertain because fraction retrieved depends on a stochastic permutation-based significance test.Comparisons with CellProfiler and CellCLIP are contextual rather than strict rankings because preprocessing and evaluation procedures differ.
  • Representation diagnostics: The cached-token diagnostics separate variance into between-channel, between-site and residual components.The associated figures also examine channel similarity and principal-component structure in the frozen token space.
  • Limitations: A stronger case for biological usefulness requires improvement on tasks not directly optimized by the replicate loss.Replicate alignment improves replicability, but that metric is also the quantity the loss directly optimizes.
  • Future work: A mismatch between primary-target training annotations and broader benchmark target lists may contribute to weak cross-modality retrieval.The paper proposes aligning these definitions and testing richer target supervision.
  • Future work: Future evaluation should test transfer across sites, cell lines and experimental conditions using the larger and more diverse JUMP corpus.Prompt ablations are also needed to assess the contribution of chemical structures and gene-function descriptions.
Loading 2608.22690v1…