PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Molecular-Morphology Contrastive Pretraining using Deep-Learning-based Morphology Profiles

Jie Li, Kathryn E. Kirchoff, Dante A. Pertusi, Zhizhuo Zhang

arXiv:2609.30433v1cs.LGq-bio.BM

TL;DR

Molecular representations learned from chemical structure alone do not directly encode cellular responses, and conventional morphology features may miss subtle patterns. MoCoP v2 aligns molecular embeddings with deep-learning-derived morphology profiles, reporting more accurate morphology retrieval and gains across QSAR, toxicity, ADME, and activity evaluations. Its current contrastive setup uses separate encoders connected by cosine similarity.

  • Problem

    Chemical-data representations do not directly capture how molecules interact with living cells, and handcrafted CellProfiler features may miss subtle morphological patterns.

    Source: Introduction · pp. 1–2

  • Method

    MoCoP v2 aligns molecular embeddings with deep-learning-based morphology embeddings extracted from Cell Painting images, using a fine-tuned convolutional encoder and CORAL batch correction.

    Source: Deep-Learning-based Morphological… · p. 2, Batch Correction with CORAL · p. 4, Conclusion · p. 11

  • Results

    MoCoP v2 improves QSAR predictions, achieves state-of-the-art ToxCast toxicity performance and competitive ADME and activity results, and retrieval scales log-linearly with training data size.

    Source: Conclusion · pp. 11–12

  • Takeaways & Limitations

    The pretrained encoder can be adapted by linear probing or end-to-end fine-tuning for QSAR and ADMET prediction, providing a morphology-grounded signal complementary to structure-derived descriptors.

    Source: Conclusion · p. 12

    The current architecture keeps molecule and morphology encoders separate and connects them with a cosine-similarity objective.

    Source: Conclusion · p. 12

Abstract

from arXiv · show

Recent advancements in image-based profiling techniques have enabled the collection of high-volume cell morphology data, allowing new molecular embedding models to learn from the experimental phenotypic perturbations of a molecule in a cell. Previously, we developed Molecule-Morphology Contrastive Pretraining (MoCoP), a strategy for aligning small molecule embeddings to morphology fingerprints extracted through CellProfiler. The resulting molecular representation showed transferable performance for quantitative structure--activity relationship (QSAR) prediction tasks. Here, we extend the method by using a deep-learning-based cell image encoding pipeline to extract more feature-rich morphology profiles and align them to the molecular embeddings through contrastive learning. The new embeddings encode more accurate information on how molecules perturb cell morphology and enable improvements for QSAR predictions through either fixed-embedding linear probes or fully flexible fine-tuning. Morphology retrieval performance scales log-linearly with training data size, suggesting continued improvements as larger datasets become available. The improved MoCoP v2 also achieves superior performance on toxicity prediction and competitive results on ADME and activity benchmarks, when compared with existing molecular embedding models that use both cell morphology and transcriptomic data during training.

Introduction

Molecular representations learned from chemical data do not directly capture how molecules affect living cells, while conventional morphology features may miss subtle phenotypic patterns. MoCoP v2 addresses this gap by aligning molecular embeddings with deep-learning-based morphology profiles and reports improvements across several downstream tasks.

Introduction

  • Chemical-data representations, including graph and SMILES models and 3D methods, learn expressive features but do not directly capture how molecules interact with living cells.Traditional descriptors such as ECFPs capture substructural motifs derivable from chemical graphs alone.Source: Introduction · p. 1
  • Cell Painting measures cellular responses to chemical perturbations, and JUMP Cell Painting datasets span more than 116,000 compounds.Morphology profiles have been used to predict bioactivity, mechanism of action, and drug targets.Source: Introduction · p. 1
  • Prior cross-modal methods align molecular and phenotypic representations, with approaches including CLOOME, InfoAlign, CHMR, and other morphology- or transcriptomics-based strategies.InfoAlign integrates morphology and transcriptomic data; CHMR models hierarchical dependencies across molecular, cellular, and genomic levels.Source: Introduction · pp. 1–2
  • CellProfiler’s handcrafted features may miss subtle morphological patterns, motivating learned image embeddings as alignment targets for molecule–morphology pretraining.Prior microscopy studies found deep-learning representations outperformed CellProfiler on mechanism-of-action prediction and batch-effect correction.Source: Introduction · p. 2
  • MoCoP v2 replaces CellProfiler features with deep-learning morphology embeddings and reports improved QSAR predictions, log-linear retrieval scaling, and competitive or state-of-the-art downstream results.The reported comparisons include toxicity, ADME, and activity benchmarks.Source: Introduction · p. 2

Methods

MoCoP v2 extracts deep-learning morphology profiles from Cell Painting images, aligns them with molecular representations through contrastive pretraining, and transfers the pretrained molecular encoder to downstream tasks.

Methods

  • MoCoP v2 combines image-encoder training, molecule–morphology contrastive pretraining, and transfer of the pretrained molecular encoder to downstream applications.The image encoder is frozen during contrastive pretraining, with its embeddings used as static input.Source: Methods · p. 2, Deep-Learning-based Morphological… · p. 4

Deep-Learning-based Morphological Embeddings

  • The DL morphology pipeline replaces CellProfiler features with representations learned by fine-tuning a convolutional network to distinguish compound treatments by their morphological effects.Training uses triplet margin loss with online semihard mining; embeddings are extracted from all wells and mean-aggregated per well.Source: Deep-Learning-based Morphological… · pp. 2, 4

Batch Correction with CORAL

  • CORAL aligns batch embedding covariance statistics using DMSO control wells, with source-batch embeddings transformed toward the Source 3 reference.The authors state that DMSO controls exhibit minimal compound-induced variation, so alignment preserves biological signals while removing technical variability.Source: Batch Correction with CORAL · p. 4

Molecule-Morphology Contrastive Pretraining

  • A GGNN molecular encoder and an MLP morphology encoder are jointly trained with symmetric InfoNCE to align paired molecule–morphology representations in a shared space.After pretraining, the molecular encoder is transferred to downstream tasks, while the morphology encoder and projection heads are discarded.Source: Molecule-Morphology Contrastive… · p. 5

Results and Discussion

MoCoP v2 shows strong benchmark performance, improves QSAR results with frozen embeddings and sufficient fine-tuning data, and retrieves morphology more accurately as training data grows. Comparisons with MoCoP v1 and ECFP4 indicate that v2 better captures phenotypic relationships, particularly for chemically isolated compounds.

Morphology Retrieval Performance and Scaling Behavior

  • In 1:100 retrieval, Top-1 accuracy rose from 3.3% to 6.8% and Top-10 from 22.4% to 41.6% as training grew from 10K to 98K compounds.The 1:1000 setting showed the same trend: Top-1 rose from 0.6% to 1.6%, and Top-10 from 4.9% to 9.8%; retrieval followed an approximately log-linear scaling relationship.Source: Figure 2 · p. 6, Morphology Retrieval Performance and… · p. 6

MoCoP v2 Captures More Accurate Morphological Relationships

  • Against CellProfiler-profile retrieval, v2 overtook ECFP4 at k ≥10 and outperformed v1 at every tested k, with its strongest advantage at k = 1.ECFP4 reached 44.6% of the oracle ceiling at k = 1 but fell to 38.6% at k = 10; v2 remained between 38.8% and 42.0%.Source: MoCoP v2 Captures More Accurate… · p. 7, Figure 4 · p. 9
  • For chemically isolated compounds with maximum Tanimoto similarity below 0.3, v2 outperformed both v1 and ECFP4 in morphology retrieval.Examples also show v2 distinguishing compounds with different phenotypes and identifying shared phenotypic effects between structurally dissimilar compounds that v1 missed.Source: MoCoP v2 Captures More Accurate… · pp. 6–7, Figure 4 · p. 9

Improved QSAR Prediction on ChEMBL20

Comparing with Other State-of-the-Art Methods

  • MoCoP v2 outperformed InfoAlign on 3 of 4 benchmarks and achieved the best ToxCast result (70.3±0.2), while remaining competitive on ADME and activity tasks.It ranked second on Biogen 3K and matched InfoAlign on ChEMBL 2K; both InfoAlign and CHMR additionally use transcriptomic data during training.Source: Comparing with Other State-of-the-Art… · p. 10, Table 3 · p. 11

Conclusion

MoCoP v2 aligns molecular representations with deep-learning morphology embeddings, improving morphology retrieval and downstream prediction. The authors identify scaling and several possible extensions for future work.

Conclusion

  • A fine-tuned ResNet-18 encoder with CORAL batch correction produces richer Cell Painting morphology representations for molecular alignment.Replacing CellProfiler-derived features with these deep-learning embeddings leads to improved molecular embeddings after alignment.Source: Conclusion · p. 11
  • MoCoP v2 achieves state-of-the-art ToxCast toxicity prediction and competitive ADME and activity benchmark results against recent methods that also use transcriptomic data.The pretrained embeddings also improve QSAR prediction in linear probes across all data proportions.Source: Conclusion · p. 11
  • Morphology retrieval performance scales log-linearly with training data size, improving consistently from 10K to 98K training compounds.Source: Conclusion · p. 12
  • The pretrained encoder can be adapted through linear probing or end-to-end fine-tuning for QSAR and ADMET prediction, and its embeddings can support mechanism-of-action retrieval by cellular phenotype.Nearest-neighbor retrieval can identify compounds with similar phenotypes regardless of structural similarity, supporting scaffold hopping and target deconvolution.Source: Conclusion · p. 12
  • Future work could add concentration-aware embeddings because MoCoP v2 currently maps each compound to a single embedding regardless of dose.Other proposed directions include alternative image encoders, joint multimodal modeling, transcriptomic data, and cell-line- or tissue-specific fine-tuning.Source: Conclusion · p. 12

Appendix A : DL Embedding Training Hyperparameters

Appendix A points to a table of DL morphological image-encoder training hyperparameters and describes the image transformations used for training, validation, and testing.

Appendix A : DL Embedding Training Hyperparameters

Data Augmentation.

  • Training images received random horizontal and vertical flips and 90-degree rotations, each with probability 0.5.Per-channel multiplicative noise used factors sampled uniformly from [0.9, 1.1].Source: Data Augmentation · p. 16
  • Validation and test images received per-channel normalization only.Source: Data Augmentation · p. 16

Appendix B : Dataset Summary Statistics

Appendix B summarizes datasets used across the MoCoP v2 pipeline and describes the JUMP-CP sources and compound-based splitting procedure.

Appendix B : Dataset Summary Statistics

  • JUMP-CP data spans 10 independent sources, each contributing varying numbers of plates and compounds.Source: Appendix B · p. 17
  • Compound-based splitting keeps compounds out of both training and test sets, using canonical SMILES to avoid leakage from alternative SMILES representations.Source: Appendix B · p. 17
  • Table 6 summarizes datasets used for DL embedding extraction, contrastive pretraining, and evaluation.It defines wells as compound-treated wells excluding DMSO controls, and compounds as unique chemical entities identified by canonical SMILES.Source: Table 6 · p. 17

Appendix C : CORAL Batch Correction Visualization

The CORAL visualization compares DL embedding projections by data source before and after correction using DMSO negative controls. Before correction, embeddings cluster by source; afterward, source-specific clusters dissolve and samples intermingle.

Appendix C : CORAL Batch Correction Visualization

  • After CORAL correction using DMSO negative controls, source-specific clusters dissolve and embeddings intermingle across sources.Source: Appendix C · p. 18, Figure 5 · p. 18
  • The corrected visualization excludes 352 of 47,393 samples (0.7%) with abnormally large embedding norms linked to numerical instability in covariance inversion.The figure caption reports that outlier norms affected less than 1% of data.Source: Appendix C · p. 18, Figure 5 · p. 18
  • Before CORAL correction, DL embeddings cluster by data source, indicating that batch effects dominate the representation.Figure 5 shows UMAP projections using cosine distance and 30 neighbors, subsampled to about 1,500 wells per source.Source: Figure 5 · p. 18, Appendix C · p. 18

Appendix D : MoCoP Training Hyperparameters

Table 7 summarizes MoCoP v2 pretraining hyperparameters, and the appendix adds divergence examples comparing v1 and v2 embeddings. The examples use compounds imaged on the same microscope, and visual inspection supports the v2 assessments.

Appendix D : MoCoP Training Hyperparameters

  • Table 7 summarizes the key hyperparameters used for MoCoP v2 contrastive pretraining.The table caption identifies its scope as the pretraining hyperparameters.Source: Table 7 · p. 19, Appendix D · p. 19

Appendix E : Additional MoCoP Embedding Divergence Examples

  • The divergence examples use compounds from the same microscope to eliminate batch effects, and visual inspection supports the v2 assessments.The Cell Painting composite encodes DNA in blue, AGP in green, and mitochondria in red.Source: Appendix E · p. 20

Additional strong divergence examples.

Divergence examples among structurally dissimilar compounds.

For structurally dissimilar compounds, v2 similarity scores align with visually concordant phenotypes, while high v1 scores can occur for visibly different phenotypes. The examples compare pairs with Tanimoto similarity below 0.2.

Divergence examples among structurally dissimilar compounds.

Appendix F : Downstream Task Training Details

Downstream evaluation followed the InfoAlign protocol, using scaffold-based splits and task-specific metrics. Table 8 summarizes the benchmark datasets.

Appendix F : Downstream Task Training Details

  • Evaluation used scaffold-based train, validation, and test splits, with AUROC for classification and MAE for Biogen 3K regression.Results were reported as three-seed test-set mean ± standard deviation using separate models initialized from the pretrained checkpoint.Source: Appendix F · p. 23
  • Table 8 summarizes the downstream benchmark datasets.The supplied table caption identifies its scope but does not specify dataset contents.Source: Table 8 · p. 23, Appendix F · p. 23

Hyperparameter Sweep.

The sweep found that higher learning rates and moderate dropout improved validation performance under specified settings, while cosine annealing helped only slightly on ChEMBL 2K. AdamW weight decay did not outperform Adam.

Hyperparameter Sweep.

  • Learning rates from 3×10−4 to 7×10−4 consistently improved validation performance over the 5 × 10−5 baseline.Learning rate was swept from 5 × 10−5 to 10−3.Source: Hyperparameter Sweep · p. 23
  • Reducing early-stopping patience from 20 to 10 epochs reduced overfitting to the validation set.The sweep compared patience values of 10 and 20 epochs.Source: Hyperparameter Sweep · p. 23
  • Dropout 0.2 generalized better than 0.1 at high learning rates, while dropout 0.3 hurt validation performance.The sweep covered dropout values from 0.1 to 0.3.Source: Hyperparameter Sweep · p. 23
  • +0.3% AUC was the marginal improvement from cosine annealing, for ChEMBL 2K only.The sweep compared cosine annealing with a constant learning-rate scheduler.Source: Hyperparameter Sweep · p. 23
  • AdamW weight decay did not outperform Adam.The optimizer comparison was part of the hyperparameter sweep.Source: Hyperparameter Sweep · p. 23
Loading 2609.30433v1…